JP7208486B2 - Image decoding device, image decoding method and image decoding program - Google Patents

Description

The present invention relates to a technique of dividing an image into blocks and performing encoding and decoding on a block-by-block basis.

In image encoding and decoding, an image is divided into blocks, each of which is a set of a predetermined number of pixels, and each block is processed. At that time, by dividing into appropriate block units, the efficiency of intra prediction, inter prediction, orthogonal transform, entropy coding, etc. is improved, and as a result, the coding efficiency is improved.

Japanese Patent Publication No. 2015-526008

JVET, Versatile Video Coding(Draft 2), July 2018

If the blocks are not divided with proper size and shape, the coding efficiency will be degraded. In particular, at the edges of the screen, blocks containing pixels located beyond picture boundaries have inappropriate sizes and shapes, resulting in reduced coding efficiency.

The present invention has been made in view of such circumstances, and its object is to provide a technique for improving coding efficiency by performing block division suitable for image coding and decoding.

One aspect of the present invention for solving the above problems is an image encoding device. This apparatus is an image coding apparatus that divides an image into blocks and performs coding in units of divided blocks, and recursively divides the image into rectangles of a predetermined size to generate blocks to be coded. It comprises a block dividing unit (101) and an encoding unit (105) for encoding block dividing information to be encoded. The block dividing unit includes a 4-dividing unit that divides the target block in the recursive division into four halves in each of the horizontal direction and the vertical direction to generate four blocks, and and a 2-3 divider that divides into two or three to generate two or three blocks. When a pixel located beyond an arbitrary boundary is divided by division in either the horizontal direction or the vertical direction, the 4-dividing section restricts block division in that direction.

Yet another aspect of the present invention is an image coding method. This method divides an image into blocks and encodes each divided block unit, and recursively divides the image into rectangles of a predetermined size to generate encoding target blocks. It has a block division step and an encoding step of encoding block division information to be encoded. The block dividing step includes dividing the target block in the recursive division into four halves each in the horizontal direction and the vertical direction to generate four blocks, and dividing the target block in the recursive division in the horizontal or vertical direction. and a 2-3 division step of dividing into 2 or 3 to generate 2 or 3 blocks. The 4-division step restricts block division in either the horizontal direction or the vertical direction, if the pixel at the position beyond an arbitrary boundary is divided by the division in that direction.

Yet another aspect of the present invention is an image decoding device. This apparatus is an image decoding apparatus that divides an image into blocks and performs decoding in units of divided blocks, and includes a decoding unit (201) that decodes block division information to be decoded; a block division unit (202) that recursively divides the image into rectangles of a predetermined size based on division information to generate blocks to be decoded. The block dividing unit includes a 4-dividing unit that divides the target block in the recursive division into four halves in each of the horizontal direction and the vertical direction to generate four blocks, and and a 2-3 divider that divides into two or three to generate two or three blocks. When a pixel located beyond an arbitrary boundary is divided by division in either the horizontal direction or the vertical direction, the 4-dividing section restricts block division in that direction.

Yet another aspect of the present invention is an image decoding method. This method is an image decoding method in which an image is divided into blocks and decoding is performed in units of divided blocks. and a block dividing step of recursively dividing the image into rectangles of a predetermined size based on the block size to generate decoding target blocks. The block dividing step includes dividing the target block in the recursive division into four halves each in the horizontal direction and the vertical direction to generate four blocks, and dividing the target block in the recursive division in the horizontal or vertical direction. and a 2-3 division step of dividing into 2 or 3 to generate 2 or 3 blocks. The 4-division step restricts block division in either the horizontal direction or the vertical direction, if the pixel at the position beyond an arbitrary boundary is divided by the division in that direction.

Any combination of the above constituent elements, and any conversion of expressions of the present invention into methods, devices, systems, recording media, computer programs, etc. are also effective as embodiments of the present invention.

According to the present invention, block division suitable for image encoding and decoding becomes possible, and encoding efficiency can be improved.

1 is a block diagram of an image coding device according to a first embodiment; FIG. 1 is a block diagram of an image decoding device according to a first embodiment; FIG. Fig. 4 is a flow chart illustrating splitting into treeblocks and splitting inside treeblocks; FIG. 4 is a diagram showing how an input image is divided into treeblocks; FIG. 4 is a diagram for explaining z-scan; FIG. 4 is a diagram for explaining division of treeblocks; FIG. 10 is a flowchart for explaining processing of each divided block when a treeblock is divided into four; FIG. FIG. 10 is a flowchart for explaining processing of each divided block when a treeblock is divided into two or three. FIG. FIG. 3 is a diagram showing the relationship between treeblocks and picture boundaries; FIG. 4 is a diagram showing the relationship between picture boundaries and pixels; 4 is a flowchart for explaining block division in the first embodiment; It is a figure which shows the block division in 1st Embodiment. FIG. 2 is a diagram showing syntax for block division in the first embodiment; FIG. It is a figure explaining intra prediction. It is a figure explaining inter prediction. 9 is a flowchart for explaining block division in the second embodiment; It is a figure which shows the block division in 2nd Embodiment. FIG. 11 is a flowchart for explaining block division in the third embodiment; FIG. It is a figure which shows the block division in 3rd Embodiment.

Embodiments of the present invention provide an image coding technique that divides an image into rectangular blocks and encodes and decodes the divided blocks.

(First embodiment)
An image encoding device 100 and an image decoding device 200 according to the first embodiment of the present invention will be described. In the first embodiment, block division is restricted when a block is divided into four.

FIG. 1 is a block diagram of an image coding device 100 according to the first embodiment. FIG. 1 shows only the flow of data relating to image signals, and does not show the flow of data relating to additional information other than image signals, such as motion vectors and prediction modes. An image signal for at least one screen is input to the image coding apparatus 100 .

The block division unit 101 divides an image into blocks to be encoded, which are processing units for encoding, and supplies image signals in the blocks to be encoded to the residual signal generation unit 103 . In addition, the block dividing unit 101 supplies the image signal of the encoding target block to the predicted image generation unit 102 in order to evaluate the degree of matching of the predicted images.

The block division unit 101 recursively divides an image into rectangles of a predetermined size to generate encoding target blocks. A block division unit 101 divides a target block into four blocks in recursive division to generate four blocks, and divides a target block into two or three blocks in recursive division to generate two or three blocks. and a 2-3 split. A detailed operation of the block dividing unit 101 will be described later.

The prediction image generation unit 102 is supplied with the image signal of the encoding target block from the block division unit 101 and the decoded image signal from the decoded image memory 108 . The predicted image generation unit 102 uses the supplied signal to perform intra prediction (intra prediction) or inter prediction (inter prediction) based on the prediction mode to generate a predicted image signal. In intra prediction, an image signal of an already-encoded block adjacent to the block to be coded in the same picture (coded picture) as the block to be coded is supplied from the decoded image memory 108 to the prediction image generator 102 . Then, the predicted image generation unit 102 uses this image signal and the image signal of the coding target block supplied from the block division unit 101 to generate a predicted image signal. In inter-prediction, an image signal of a coded picture (reference image) that precedes or follows a coded picture in time series is supplied from the decoded image memory 108 to the predicted image generator 102 . Then, the predicted image generation unit 102 uses this image signal and the encoding target block supplied from the block division unit 101 to evaluate the degree of matching by block matching or the like, and obtains a motion vector indicating the amount of motion. The predicted image generation unit 102 performs motion compensation from the reference image based on this motion vector to generate a predicted image signal. The predicted image generator 102 supplies the predicted image signal thus generated to the residual signal generator 103 .

The residual signal generation unit 103 subtracts the image signal to be encoded from the prediction signal generated by the prediction image generation unit 102 to generate a residual signal, and supplies the residual signal to the orthogonal transformation/quantization unit 104 .

The orthogonal transformation/quantization unit 104 orthogonally transforms/quantizes the residual signal supplied from the residual signal generation unit 103 . The orthogonal transform/quantization unit 104 supplies the orthogonal transform/quantized residual signal to the encoding unit 105 and the inverse quantization/inverse orthogonal transform unit 106 .

The coding unit 105 generates a coded bitstream corresponding to the orthogonally transformed and quantized residual signal supplied from the orthogonal transforming/quantizing unit 104 . Also, the coding unit 105 generates a corresponding coded bitstream for additional information such as motion vectors, prediction modes, and block division information supplied from each component. The encoding unit 105 then outputs the encoded bitstream from the image encoding device 100 .

The inverse quantization/inverse orthogonal transformation unit 106 performs inverse quantization/inverse orthogonal transformation on the orthogonally transformed/quantized residual signal supplied from the orthogonal transformation/quantization unit 104 to obtain a residual signal. The inverse quantization/inverse orthogonal transform unit 106 supplies the residual signal to the decoded image signal superimposing unit 107 .

The decoded image signal superimposing unit 107 superimposes the predicted image signal generated by the predicted image generating unit 102 and the residual signal obtained by the inverse quantization/inverse orthogonal transform unit 106 to generate a decoded image. Store in memory 108 . Note that the decoded image signal superimposing unit 107 may perform filtering processing for reducing block distortion or the like due to encoding on the decoded image, and store the filtered image in the decoded image memory 108 .

FIG. 2 is a block diagram of the image decoding device 200 according to Embodiment 1. As shown in FIG. FIG. 2 shows only the flow of data relating to image signals, and does not show the flow of data relating to additional information other than image signals, such as motion vectors and prediction modes. An encoded bitstream is input to the image decoding apparatus 200 .

The decoding unit 201 decodes the supplied encoded bitstream and supplies the orthogonally transformed and quantized residual signal to the block dividing unit 202 . The decoding unit 201 also supplies additional information such as motion vectors, prediction modes, and block division information to each component, and uses the additional information for processing corresponding to the additional information.

Block division section 202 determines the shape of the decoding target block based on the decoded block division information, and performs inverse quantization and inverse orthogonalization of the determined orthogonal transform/quantized residual signal of the decoding target block. It is supplied to the conversion unit 203 .

The block division unit 202 recursively divides the image into rectangles of a predetermined size based on the decoded block division information to generate decoding target blocks. The block division unit 202 divides the target block into four blocks in the recursive division to generate four blocks, and divides the target block into two or three blocks in the recursive division to generate two or three blocks. and a 2-3 split. A detailed operation of the block dividing unit 202 will be described later.

The inverse quantization/inverse orthogonal transformation unit 203 performs inverse quantization/inverse orthogonal transformation on the supplied orthogonally transformed/quantized residual signal to obtain a residual signal. It is supplied to the decoded image signal superimposing unit 205 .

The predicted image generation unit 204 generates a predicted image signal from the decoded image signal supplied from the decoded image memory 206 and supplies it to the decoded image signal superimposing unit 205 .

The decoded image signal superimposing unit 205 superimposes the predicted image signal generated by the predicted image generating unit 204 and the residual signal obtained by the inverse quantization/inverse orthogonal transform unit 203 to generate a decoded image signal. Also, the decoded image signal superimposing unit 205 stores the decoded image signal in the decoded image memory 206 . The decoded image signal superimposing unit 205 may store the decoded image in the decoded image memory 206 after filtering the decoded image to reduce block distortion due to encoding. Then, the decoded image signal superimposing unit 205 outputs the decoded image from the image decoding device 200 .

Next, the operation of the block dividing section 101 in the image encoding device 100 will be explained using FIG. FIG. 3 shows the operation of the block dividing unit 101 dividing an image into tree blocks and dividing the inside into blocks.

First, an input image is divided into tree blocks of a predetermined size (S1000). Here, the treeblock is assumed to be 128×128 pixels. However, the treeblock is not limited to this size, and any size and aspect ratio may be used as long as it is a rectangle. Also, the size of the treeblock may be determined in advance between the encoding device and the decoding device. Further, the encoding device may determine the treeblock size and record it in the encoded bitstream, and the decoding device may use the treeblock size recorded in the encoded bitstream. FIG. 4 shows how an input image is divided into treeblocks. The treeblocks are encoded in raster scan order, left to right, top to bottom.

The interior of the treeblock is further divided into rectangular blocks. The treeblock interior is encoded in the z-scan order shown in FIG. The z-scan order refers to the order top left, top right, bottom left, bottom right. The division inside the treeblock can be 4 divisions, 2 divisions or 3 divisions.

The division into four blocks is performed by dividing the block into four halves in the horizontal direction and the vertical direction as shown in FIG. 6(a) to generate four blocks.

Blocks are divided into two or three by dividing horizontally or vertically. When dividing a block into two in the horizontal direction, it is divided into two halves as shown in FIG. 6B to generate two blocks. Also, when dividing a block into three in the horizontal direction, it is divided into 1:2:1 as shown in FIG. 6(c) to generate three blocks. On the other hand, when dividing a block into two in the vertical direction, the block is divided in half as shown in FIG. 6(d) to generate two blocks. Also, when dividing a block into three in the vertical direction, it is divided into 1:2:1 as shown in FIG. 6(e) to generate three blocks.

The operation of the block division unit 101 will be described with reference to FIG. 3 again. First, it is determined whether or not the inside of the treeblock is divided into four halves in the horizontal direction and the vertical direction (S1001).

There is an existing method called RD optimization (Rate-Distortion Optimization) for determining the optimum case from a plurality of conditions, including determining whether to divide a block into four or not. In RD optimization, the coding cost is calculated from the code amount and coding distortion. Then, the encoding cost is calculated for each of the cases of encoding under a plurality of conditions, and the case with the lowest encoding cost is selected. In other words, to determine whether to divide a block into 4 or not, the encoding cost when the block is divided into 4 and the encoding cost when the block is not divided into 4 are calculated, and the case with the lowest coding cost is selected. It is done by A method other than the RD optimization may be used to determine the optimum case from a plurality of conditions.
When it is determined that the inside of the treeblock is to be divided into four (S1001: Yes), the inside of the treeblock is divided into four (S1002). The re-division processing of the blocks divided into four will be described later (FIG. 7).

If it is determined not to divide the inside of the treeblock into four (S1001: No), it is determined whether to divide the inside of the treeblock into two or three (S1003).

If it is determined that the inside of the treeblock is to be divided into two or three (S1003: Yes), it is determined whether or not the direction of division is the vertical direction (S1004).

If it is determined that the dividing direction is the vertical direction (S1004: Yes), it is determined whether or not to divide the inside of the treeblock into two (S1005).

When it is determined that the inside of the treeblock is to be divided into two (S1005: Yes), the inside of the treeblock is vertically divided into two (S1006). On the other hand, when it is determined that the inside of the treeblock is to be divided into three (S1005: No), the inside of the treeblock is vertically divided into three (S1007). Re-division processing of a block divided into two or three in the vertical direction will be described later (FIG. 8).

If it is determined that the direction of division is the horizontal direction (S1004: No), it is determined whether or not to divide the inside of the treeblock into two (S1008).

If it is determined that the inside of the treeblock is to be divided into two (S1008: Yes), the inside of the treeblock is horizontally divided into two (S1009). On the other hand, when it is determined that the inside of the treeblock is to be divided into three (S1008: No), the inside of the treeblock is horizontally divided into three (S1010). A re-division process for a block horizontally divided into two or three will be described later (FIG. 8).

If it is determined not to divide the inside of the treeblock into two or three (S1003: No), the block division processing ends without dividing the inside of the treeblock into blocks (S1011).

Next, the processing of each divided block when the treeblock is divided into four halves in the horizontal direction and the vertical direction will be described with reference to the flowchart of FIG.

First, it is determined whether or not to divide the inside of the block into four halves in the horizontal direction and the vertical direction again (S1101).

If it is determined that the inside of the block should be divided into four again (S1101: Yes), the inside of the block is divided into four again (S1102).

If it is determined not to divide the inside of the block into four again (S1101: No), it is determined whether to divide the inside of the block into two or three (S1103).

If it is determined that the inside of the block is to be divided into two or three (S1103: Yes), it is determined whether or not the dividing direction is the vertical direction (S1104).

If it is determined that the dividing direction is the vertical direction (S1104: Yes), it is determined whether or not to divide the inside of the block into two (S1105).

If it is determined that the inside of the block is to be divided into two (S1105: Yes), the inside of the block is vertically divided into two (S1106). On the other hand, if it is determined that the inside of the block is to be divided into three (S1105: No), the inside of the block is vertically divided into three (S1107).

If it is determined that the dividing direction is the horizontal direction (S1104: No), it is determined whether or not to divide the inside of the block into two (S1108).

When it is determined that the inside of the block is to be divided into two (S1108: Yes), the inside of the block is horizontally divided into two (S1109). On the other hand, when it is determined that the inside of the block is to be divided into three (S1108: No), the inside of the block is horizontally divided into three (S1110).

If it is determined that the inside of the block should not be divided into two or three (S1103: No), the block division processing ends without re-dividing the inside of the block (S1111).

The processing shown in the flowchart of FIG. 7 is recursively executed for each of the four divided blocks. The inside of the 4-partitioned block is coded in z-scan order.

Next, the processing of each divided block when the treeblock is vertically divided into two or three will be described with reference to the flowchart of FIG.

When the treeblock is vertically divided into two or three, it is determined whether the inside of each divided block is again divided into two or three (S1201).

If it is determined that the inside of the block is divided into two or three (S1201: Yes), it is determined whether or not the direction of division is the vertical direction (S1202).

If it is determined that the dividing direction is the vertical direction (S1202: Yes), it is determined whether or not to divide the inside of the block into two (S1203).

If it is determined that the inside of the block is to be divided into two (S1203: Yes), the inside of the block is vertically divided into two (S1204). On the other hand, when it is determined that the inside of the block is to be divided into three (S1203: No), the inside of the block is vertically divided into three (S1205).

If it is determined that the dividing direction is the horizontal direction (S1202: No), it is determined whether or not to divide the inside of the block into two (S1206).

If it is determined to divide the inside of the block into two (S1206: Yes), the inside of the block is horizontally divided into two (S1207). On the other hand, when it is determined that the inside of the block is to be divided into three (S1206: No), the inside of the block is horizontally divided into three (S1208).

If it is determined not to divide the inside of the block into two or three again (S1201: No), the block division processing ends without re-dividing the inside of the block (S1209).

The processing shown in the flowchart of FIG. 8 is recursively executed for each divided block in the case of vertically dividing into two or three. The inside of a block divided into two or three is encoded in order from left to right.

Similarly, the processing shown in the flowchart of FIG. 8 is recursively executed for each divided block in the case of horizontally dividing into two or three. The inside of a block divided into two or three is encoded in order from top to bottom.

Although re-division of a divided block when a treeblock is divided has been described, the parent block need not be a treeblock. For example, when a tree block (128×128 pixels) is divided into 4 blocks and the 4 blocks (64×64 pixels) are further divided, the above process is also applied to division of the re-divided blocks.

For recursive block division, the number of divisions may be defined and the number of divisions may be limited. Also, the number of divisions may be determined in advance between the encoding device and the decoding device. Furthermore, the encoding device may determine the number of divisions and record it in the encoded bitstream, and the decoding device may use the number of divisions recorded in the encoded bitstream.

Next, block division at the edge of the screen will be described. FIG. 9 shows the relationship with picture boundaries when an image is divided into treeblocks. As shown in this figure, the size of the image is not necessarily an integral multiple of the size of the treeblock, so the treeblock at the edge of the screen may include the in-screen and out-of-screen parts across the picture boundary. . In this case, as shown in FIG. 10, the out-of-screen portion is treated as having the same pixel value as the outermost pixel value within the screen.

Then, when the block is divided into four, the block division is restricted. As a result, it is possible to divide the blocks at the screen edges into appropriate sizes and shapes, and to improve the coding efficiency.

Block division restrictions apply when a block is divided into four at screen edges. That is, the 4-division processing (S1002) in FIG. 3 and the 4-division processing (S1102) in FIG. 7 are replaced with the processing described below.

Block division restrictions will be described with reference to FIG. First, when dividing a block into four, it is determined whether or not a pixel at a position beyond a picture boundary is divided by either horizontal or vertical division (S1301).

If the pixel is not divided (S1301: No), the block is divided into four blocks without limiting the block division (S1302).

If the pixel is to be divided (S1301: Yes), it is determined whether or not the direction of dividing the pixel is vertical (S1303).

In the case of the vertical direction (S1303: Yes), vertical division is restricted, and the block is horizontally divided into two (S1304). On the other hand, in the case of the horizontal direction (S1303: No), horizontal division is restricted and the block is vertically divided into two (S1305).

In other words, when a pixel located beyond a picture boundary is divided by block division in either the horizontal direction or the vertical direction, block division in that direction is restricted.

A specific example will now be described. FIG. 12(a) shows how the treeblock at the bottom of the screen includes the in-screen and out-of-screen portions across the picture boundary. When dividing this treeblock into four, pixels at positions beyond picture boundaries are divided. However, the directions are both horizontal and vertical. Therefore, since the condition for limiting block division is not met (S1301: No), the block is divided into four (S1302).

Next, FIG. 12(b) shows how the upper left block of each block divided into four blocks is further divided into four blocks. At this time, pixels located beyond the picture boundaries are divided by vertical division. At this time, the horizontal division does not divide the pixels beyond the picture boundaries. Therefore, the condition for restricting block division is met (S1301: Yes). Then, since the direction of pixel division is vertical (S1303: Yes), vertical division is restricted and the block is horizontally divided into two (S1304). FIG. 12(c) shows how the blocks are divided at this time. These block division restrictions prevent the divided blocks from becoming too small.

This block division restriction applies to the right edge of the screen as well. As shown in FIG. 12(d), after dividing the treeblock into 4 parts, consider the case where the upper left block is further divided into 4 parts. At this time, if the block is divided in the horizontal direction, the pixels at positions beyond the picture boundary are divided. Therefore, the horizontal division is restricted and the block is vertically divided into two. These block division restrictions prevent the divided blocks from becoming too small.

Next, the operation of the block dividing section 202 of the image decoding device 200 will be described. The block division unit 202 divides blocks by the same processing procedure as the block division unit 101 in the image encoding device 100 described above. The block division unit 101 selects a block division pattern and outputs the selected block division information. On the other hand, the block division unit 202 divides blocks using block division information decoded from the encoded bitstream. Block division restrictions are the same as those of the image encoding device 100 described above.

FIG. 13 shows the syntax (code bitstream syntax rules) for block division in the first embodiment. In FIG. 13, QT( ) represents the syntax for dividing a block into 4 blocks, and MTT( ) represents the syntax for dividing the block into 2 or 3 blocks. The image encoding device 100 encodes according to this syntax, and the image decoding device 200 decodes according to this syntax.

First, QTflag indicates whether or not to divide a block into four. QTflag=1 when dividing into four, and QTflag=0 when not dividing into four. QTflag=1 when block division is restricted in 4-division. When dividing into 4 blocks (QTflag=1), if each block divided into 4 blocks can be further divided into 4 blocks (QTvalid=1), 4 dividing processing is performed recursively. When not dividing into 4 (QTflag=0), MTTflag indicates whether to divide into 2 or 3. When splitting into two or three (MTTflag=1), vertical_flag indicates whether to split in the vertical direction, and BTflag indicates whether to split in two. Set vertical_flag=1 for vertical splitting and vertical_flag=0 for horizontal splitting. Also, BTflag=1 when dividing into two, and BTflag=0 when dividing into three. If each block divided into 2 or 3 can be further divided into 2 or 3 (MTTvalid=1), the block is recursively divided into 2 or 3.

Here, the variable QTvalid indicating whether or not each block divided into four can be further divided into four will be described. QTvalid is defined for each block divided into four. QTvalid=0 when a pixel in the screen is not included in the four-divided block. Also, when a block is divided into four, QTvalid=0 if a pixel beyond the picture boundary is divided by either horizontal or vertical division. Otherwise, QTvalid=1.

Also, a variable MTTvalid indicating whether or not each block divided into two or three can be further divided into two or three will be explained. MTTvalid is defined for each block divided into two or three. MTTvalid=0 when the block divided into two or three does not contain pixels in the screen. Otherwise, MTTvalid=1.

These block division restrictions prevent the divided blocks from becoming too small. Therefore, it is possible to divide the blocks at the screen edge into appropriate sizes and shapes, and to improve the coding efficiency.

In the image encoding device 100 and the image decoding device 200, intra prediction and inter prediction are performed using the divided blocks. Both intra prediction and inter prediction involve copying pixels from memory.

FIG. 14 shows an example of intra prediction. FIGS. 14A and 14B show prediction directions and mode numbers of intra prediction. In intra prediction, as shown in FIGS. 14(c) and 14(d), pixels are copied from adjacent encoded/decoded pixels on the left or top of the encoding/decoding target block. , to generate a predicted image of the encoding/decoding target block. In this intra-prediction, a series of processes from generation of a predicted image to pixel generation for encoding/decoding are sequentially repeated for each block. Therefore, the smaller the block is divided, the more times the above process is repeated, and the required memory bandwidth increases. Therefore, by not dividing blocks into too small blocks, it is possible to prevent an increase in the memory bandwidth required for intra prediction.

In addition, the smaller the block is divided, the smaller the unit of intra prediction becomes, and the finer pixel changes in the screen can be coded. However, in a block including pixels outside the screen, the pixel values of the portion outside the screen are constant. Therefore, the change in the pixel values of the in-screen portion of the block is relatively small compared to blocks that do not include out-of-screen pixels. Therefore, there is little need to encode fine pixel changes, and rather, by not dividing blocks into too small blocks, the amount of code can be reduced and the encoding efficiency can be improved.

FIG. 15 shows an example of inter prediction. In inter-prediction, pixels are copied block by block from a reference image to generate a predicted image of a block to be encoded/decoded. In the process of copying pixels from the reference image, the pixels required for prediction are obtained from the memory in which the reference image is stored. At that time, the device is often configured so that the memory is accessed along its management unit. As the blocks are divided into smaller blocks, the number of copying processes increases, and the required memory bandwidth increases. Further, when performing decimal-precision motion compensation using an interpolation filter on a reference image, it is necessary to copy pixels in which several pixels are added to the pixels included in the block. Therefore, the smaller the block, the larger the relative proportion of pixels to be added, and the more memory bandwidth required. Therefore, by not dividing blocks into too small blocks, it is possible to prevent an increase in the memory bandwidth required for inter prediction.

In addition, the smaller the block is divided, the smaller the unit of inter prediction becomes, and fine pixel changes between screens can be coded. However, in a block containing off-screen pixels, there is no movement of the off-screen portion. Therefore, the change in the pixel values of the in-screen portion of the block is relatively small compared to blocks that do not include out-of-screen pixels. Therefore, there is little need to encode fine pixel changes, and rather, by not dividing blocks into too small blocks, the amount of code can be reduced and the encoding efficiency can be improved.

(Second embodiment)
An image encoding device and an image decoding device according to the second embodiment of the present invention will be described. In the second embodiment, when a block is divided into four blocks, part of block division is restricted. Since the configuration other than this is the same as that of the first embodiment, description thereof is omitted.

Block division restrictions apply when a block is divided into four at screen edges. That is, the 4-division processing (S1002) in FIG. 3 and the 4-division processing (S1102) in FIG. 7 are replaced with the processing described below.

Block division restrictions will be described with reference to FIG. First, when dividing a block into four, it is determined whether or not a pixel at a position beyond a picture boundary is divided by either horizontal or vertical division (S1401).

If the pixels are not divided (S1401: No), the blocks are divided into four without limiting the block division (S1402).

If the pixel is to be divided (S1401: Yes), it is determined whether or not the direction of dividing the pixel is vertical (S1403).

In the case of the vertical direction (S1403: Yes), vertical division is restricted and the block is horizontally divided into two (S1404). Further, among the blocks divided into two in the horizontal direction, those blocks that do not include pixels located beyond the picture boundary are divided into two in the vertical direction (S1405).

On the other hand, in the case of the horizontal direction (S1403: No), horizontal division is restricted and the block is vertically divided into two (S1406). Furthermore, among the blocks divided into two in the vertical direction, those blocks that do not include pixels located beyond the picture boundary are divided into two in the horizontal direction (S1407).

That is, when a pixel at a position beyond a picture boundary is divided by block division in either the horizontal direction or the vertical direction, block division is restricted at the position where the pixel is divided.

A specific example will now be described. FIG. 12(a) shows how the treeblock at the bottom of the screen includes the in-screen and out-of-screen portions across the picture boundary. When dividing this treeblock into four, pixels at positions beyond picture boundaries are divided. However, the directions are both horizontal and vertical. Therefore, since the condition for restricting block division is not met (S1401: No), the block is divided into four (S1402).

Next, FIG. 12B shows how the upper left block of each of the four divided blocks is divided into four blocks. At this time, pixels located beyond the picture boundaries are divided by vertical division. At this time, the horizontal division does not divide the pixels beyond the picture boundaries. Therefore, the condition for restricting block division is met (S1401: Yes). Since the direction of pixel division is vertical (S1403: Yes), vertical division is restricted and the block is horizontally divided into two (S1404). Further, among the blocks divided into two in the horizontal direction, those blocks that do not include pixels located beyond the picture boundary are divided into two in the vertical direction (S1405). FIG. 17A shows how blocks are divided at this time. These block limits prevent split blocks from becoming too small.

This block division restriction applies to the right edge of the screen as well. As shown in FIG. 17(b), after dividing the treeblock into four parts, consider the case where the upper left block is further divided into four parts. At this time, if the block is divided in the horizontal direction, the pixels at positions beyond the picture boundary are divided. Therefore, the horizontal division is restricted and the block is vertically divided into two. Further, among the blocks divided vertically into two, those blocks that do not include pixels located beyond the picture boundary are horizontally divided into two. These block division restrictions prevent the divided blocks from becoming too small.

In the first embodiment, the direction of block division is restricted. On the other hand, in the present embodiment, part of the direction of block division is restricted. As a result, blocks that do not contain pixels outside the screen are divided and blocks that contain pixels outside the screen are not divided, thereby preventing the divided blocks from becoming too small. Therefore, it is possible to divide the blocks at the screen edge into appropriate sizes and shapes, and to improve the coding efficiency.

(Third Embodiment)
An image encoding device and an image decoding device according to the third embodiment of the present invention will be described. In the third embodiment, block division is restricted when the depth of block division reaches the limited depth. Since the configuration other than this is the same as that of the first embodiment, description thereof is omitted.

Here, the depth of block division will be described. In the first embodiment, the process of dividing a block into four and then recursively dividing each block into four has been described. In this process, the first quadrant process is defined as depth 0. Also, the second quadrant processing for each block divided by the first quadrant processing is defined as depth 1, and the third quadrant processing for each block divided by the second quadrant processing is defined as depth 2. , and the depth is defined in the same way below. Also, a depth that limits block division is determined in advance and defined as a limited depth.

Block division restrictions apply when a block is divided into four at screen edges. That is, the 4-division processing (S1002) in FIG. 3 and the 4-division processing (S1102) in FIG. 7 are replaced with the processing described below.

Block division restrictions will be described with reference to FIG. First, when dividing a block into four, it is determined whether or not a pixel at a position beyond a picture boundary is divided by either horizontal or vertical division (S1501).

If the pixel is not divided (S1501: No), the block is divided into four without limiting the block division (S1502).

If the pixel is divided (S1501: Yes), it is determined whether or not the depth has reached the limit depth (S1503).

If the depth limit has not been reached (S1503: No), block division is not limited and the block is divided into four (S1502). If the depth limit has been reached (S1503: Yes), it is determined whether or not the direction in which the pixels are divided is vertical (S1504).

In the case of the vertical direction (S1504: Yes), vertical division is restricted and the block is horizontally divided into two (S1505). On the other hand, in the case of the horizontal direction (S1504: No), horizontal division is restricted and the block is vertically divided into two (S1506).

That is, when a pixel beyond a picture boundary is divided by block division in either the horizontal direction or the vertical direction, and the depth of block division reaches the limit depth, block division in that direction is restricted.

A specific example will now be described. Now, let the limit depth be 1. FIG. 19(a) shows how the treeblock at the bottom of the screen includes the in-screen and out-of-screen portions across the picture boundary. When dividing this treeblock into four, the vertical division divides the pixels located beyond the picture boundaries. At this time, the horizontal division does not divide the pixels beyond the picture boundaries. Therefore, one condition that limits block division is met (S1501: Yes). However, the depth is 0 and does not reach the limit depth of 1 (S1503: No). Therefore, the block division is not limited, and the treeblock is divided into four (S1502).

Next, FIG. 19B shows how the lower left block of each block divided into four blocks is divided into four blocks. At this time, pixels located beyond the picture boundaries are divided by vertical division. At this time, the horizontal division does not divide the pixels beyond the picture boundaries. Therefore, one condition that limits block division is met (S1501: Yes). The depth is 1, which is the limited depth of 1 (S1503: Yes). Also, since the direction of pixel division is vertical (S1504: Yes), vertical division is restricted and the block is horizontally divided into two (S1505). These block division restrictions prevent the divided blocks from becoming too small.

This block division restriction applies to the right edge of the screen as well. As shown in FIG. 19(c), when the treeblock is divided into four, the depth is 0 and the limit depth of 1 is not reached. Subsequently, when the upper right block is divided into four blocks, the depth is 1 and the limit depth is 1. Therefore, the division is restricted and the block is not divided in the horizontal direction. This block division restriction prevents the divided blocks from becoming too small.

In this embodiment, the depth of block division is defined for four divisions. This may be defined for bipartition or tripartition. Also, in the present embodiment, block division is restricted according to the depth of block division. This may be limited by the number or percentage of pixels located beyond the picture boundary that the block contains. That is, when these values are larger than predetermined values, the block division is restricted. In addition, these values may be different values for each block division depth. Blocks with a small number or ratio of out-of-screen pixels are divided, and blocks with a large number or ratio of out-of-screen pixels are not divided, thereby preventing the divided blocks from becoming too small. Values related to block division limits, such as the limit depth of block division and the number and ratio of pixels located beyond picture boundaries contained in blocks, are recorded in the coded bitstream by the encoding device, and are recorded by the decoding device. It may be configured to use values recorded in the encoded bitstream.

In the first embodiment, block division is restricted regardless of the depth of block division. On the other hand, in the present embodiment, block division is restricted according to the depth of block division. As a result, blocks with a small proportion of pixels outside the screen are divided, and blocks with a large proportion of pixels outside the screen are not divided, thereby preventing the divided blocks from becoming too small. Therefore, it is possible to divide the blocks at the screen edge into appropriate sizes and shapes, and to improve the coding efficiency.

In all the embodiments described above, the target for block division control is the position beyond the picture boundary. This may define an arbitrary boundary to control block division for locations beyond it. Alternatively, a pixel having a higher degree of importance than surrounding pixels may be determined as an arbitrary boundary, and a position beyond the boundary may be set as a target for block division control. Furthermore, the position beyond an arbitrary boundary is not limited to the bottom edge or right edge of the screen, but may be the top edge or left edge of the screen, or may not be the edge. In that case, blocks can be divided into appropriate sizes and shapes even if they are not at the screen edges, and the coding efficiency can be improved.

All the embodiments described above may be combined in plural.

In all the embodiments described above, the encoded bitstream output by the image encoding device has a specific data format so that it can be decoded according to the encoding method used in the embodiments. have. Also, an image decoding device corresponding to this image encoding device can decode the encoded bitstream of this specific data format.

When a wired or wireless network is used to exchange an encoded bitstream between an image encoding device and an image decoding device, the encoded bitstream is converted into a data format suitable for the transmission mode of the communication channel. may be transmitted. In this case, a transmission device that converts an encoded bitstream output from an image encoding device into encoded data in a data format suitable for the transmission mode of a communication channel and transmits the encoded data to a network, and a transmission device that receives encoded data from the network. A receiving device is provided for restoring an encoded bitstream to be supplied to an image decoding device.

The transmission device includes a memory that buffers the encoded bitstream output from the image encoding device, a packet processing unit that packetizes the encoded bitstream, and a transmission unit that transmits the packetized encoded data via the network. including. The receiving device includes a receiving unit that receives packetized encoded data via a network, a memory that buffers the received encoded data, and packetizes the encoded data to generate an encoded bitstream, and a packet processing unit provided to the image decoding device.

Further, the display device may be configured by adding a display section for displaying an image decoded by the image decoding device. In this case, the display unit reads the decoded image signal generated by the decoded image signal superimposing unit 205 and stored in the decoded image memory 206 and displays it on the screen.

Further, an imaging device may be configured by adding an imaging unit to the configuration and inputting a captured image to an image encoding device. In that case, the imaging unit inputs the image signal of the imaged image to the block dividing unit 101 .

The above-described encoding and decoding processes may of course be implemented as transmission, storage, and reception devices using hardware, and may also be stored in a ROM (read only memory), flash memory, or the like. It may be implemented by firmware or software such as a computer. The firmware program or software program may be recorded on a computer-readable recording medium and provided, or may be provided from a server through a wired or wireless network, or data broadcasting of terrestrial or satellite digital broadcasting. may be provided as

The present invention has been described above based on the embodiments. It should be understood by those skilled in the art that the embodiments are examples, and that various modifications can be made to combinations of each component and each treatment process, and such modifications are also within the scope of the present invention. .

100 Image coding device 101 Block division unit 102 Predicted image generation unit 103 Residual signal generation unit 104 Orthogonal transformation/quantization unit 105 Encoding unit 106 Inverse quantization/Inverse orthogonal transformation unit 107 Decoded image Signal superimposing unit 108 Decoded image memory 200 Image decoding device 201 Decoding unit 202 Block dividing unit 203 Inverse quantization/inverse orthogonal transform unit 204 Predicted image generating unit 205 Decoded image signal superimposing unit 206 Decoding image memory.

Claims (5)

An image decoding device that divides an image into blocks and performs decoding in units of divided blocks,
a decoding unit that decodes the block division information of the target block;
a block division unit that recursively divides the image into rectangles of a predetermined size based on the block division information to generate the target block;
The block division unit
a 4-dividing unit that divides the target block in the recursive division into four halves each in the horizontal direction and the vertical direction to generate four blocks;
a 2-3 division unit that horizontally or vertically divides the target block in the recursive division into two or three to generate two or three blocks;
The 4-dividing unit divides the target block into 4 parts if the target block is divided into 4 parts and a pixel located beyond a picture boundary is divided by both the horizontal and vertical dividing lines. allow you to
If the target block is divided into four parts, the 4-partitioning unit divides pixels at positions beyond picture boundaries by either horizontal or vertical dividing lines, and divides the block into four parts in the recursive division. when the depth, which is the number of times, reaches a limit depth, which is a predetermined depth that limits block division, limiting the division of the target block into four;
If the target block is divided into four, the 2-3 division unit divides pixels at positions beyond picture boundaries by either horizontal or vertical division lines , and the depth is the limited depth. is reached , permitting the target block to be divided into two along a dividing line in the direction in which pixels beyond the picture boundary are not divided.
An image decoding device characterized by: The 2-3 division unit divides the target block into two along a dividing line in a direction in which pixels at positions beyond picture boundaries are not divided, and generates a first block that does not include pixels at positions beyond picture boundaries. and allowing the first block to be further divided into two;
The image decoding device according to claim 1. The 4 -part dividing unit restricts the 4 -part dividing of the target block when the number or ratio of pixels in positions beyond the picture boundary included in the target block is larger than a predetermined value.
The image decoding device according to claim 1. An image decoding method for dividing an image into blocks and decoding in units of divided blocks,
a decoding step of decoding the block division information of the target block;
a block division step of recursively dividing the image into rectangles of a predetermined size based on the block division information to generate the target block;
The block dividing step includes:
a 4-division step of dividing the target block in the recursive division into four halves each in the horizontal direction and the vertical direction to generate four blocks;
a 2-3 division step of horizontally or vertically dividing the target block in the recursive division into two or three to generate two or three blocks;
In the 4-division step, if the target block is divided into 4 parts, and if a pixel located beyond a picture boundary is divided by both horizontal and vertical dividing lines, the target block is divided into 4 parts. allow you to
In the 4-partitioning step, if the target block is divided into 4 parts, pixels at positions beyond picture boundaries are divided by either horizontal or vertical parting lines, and block division in the recursive division is performed. when the depth, which is the number of times, reaches a limit depth, which is a predetermined depth that limits block division, limiting the division of the target block into four;
In the 2-3 division step, if the target block is divided into four, pixels at positions beyond picture boundaries are divided by either horizontal or vertical division lines , and the depth is the limited depth. is reached , permitting the target block to be divided into two along a dividing line in the direction in which pixels beyond the picture boundary are not divided.
An image decoding method characterized by: An image decoding program that divides an image into blocks and decodes each divided block,
a decoding step of decoding the block division information of the target block;
a block division step of recursively dividing the image into rectangles of a predetermined size based on the block division information to generate the target block;
The block dividing step includes:
a 4-division step of dividing the target block in the recursive division into four halves each in the horizontal direction and the vertical direction to generate four blocks;
a 2-3 division step of horizontally or vertically dividing the target block in the recursive division into two or three to generate two or three blocks;
In the 4-division step, if the target block is divided into 4 parts, and if a pixel located beyond a picture boundary is divided by both horizontal and vertical dividing lines, the target block is divided into 4 parts. allow you to
In the 4-partitioning step, if the target block is divided into 4 parts, pixels at positions beyond picture boundaries are divided by either horizontal or vertical parting lines, and block division in the recursive division is performed. when the depth, which is the number of times, reaches a limit depth, which is a predetermined depth that limits block division, limiting the division of the target block into four;
In the 2-3 division step, if the target block is divided into four, pixels at positions beyond picture boundaries are divided by either horizontal or vertical division lines , and the depth is the limited depth. is reached , permitting the target block to be divided into two along a dividing line in the direction in which pixels beyond the picture boundary are not divided.
An image decoding program characterized by:

Images