KR102751535B1 - Image encoding method/apparatus, image decoding method/apparatus and and recording medium for storing bitstream - Google Patents

Abstract

The present invention provides an image encoding method and an image decoding method. The image encoding method of the present invention includes a first division step of division of a current image into a plurality of blocks, and a second division step of division of a division target block including a boundary of the current image among the plurality of blocks into a plurality of sub-blocks, wherein the second division step is recursively performed by using a sub-block including a boundary of the current image as a division target block until no sub-block including a boundary of the current image exists among the sub-blocks.

Description

IMAGE ENCODING METHOD/APPARATUS, IMAGE DECODING METHOD/APPARATUS AND AND RECORDING MEDIUM FOR STORING BITSTREAM

The present invention relates to a method and device for image encoding/decoding. In particular, it relates to a method and device for image boundary processing in image encoding/decoding. More specifically, it relates to a method and device for efficiently processing image boundaries in image padding and tree structures.

Recently, the demand for multimedia data such as video has been rapidly increasing on the Internet. However, the speed at which the bandwidth of the channel is growing is not keeping up with the rapidly increasing amount of multimedia data. As part of this trend, the Video Coding Expert Group (VCEG) of the international standardization organization ITU-T and the Moving Picture Expert Group (MPEG) of ISO/IEC are conducting continuous joint research to study video compression standards.

Video compression is largely composed of intra-frame prediction, inter-frame prediction, transformation, quantization, entropy coding, and in-loop filter. Among these, intra-frame prediction refers to a technology that generates a prediction block for the current block using restored pixels existing around the current block. Since the encoding process is performed in block units, there is a problem that encoding is difficult to perform in block units if the image size is not a multiple of the block size.

The purpose of the present invention is to provide a padding method and device for adjusting the size of an image in block units in image encoding/decoding.

In addition, the present invention aims to provide a block division method and device that are efficient for image boundary processing in image encoding/decoding.

In addition, the present invention aims to provide a computer-readable recording medium storing a bitstream generated by an image encoding method/device according to the present invention.

A video encoding method according to the present invention comprises a first division step of division of a current image into a plurality of blocks, and a second division step of division of a division target block including a boundary of the current image among the plurality of blocks into a plurality of sub-blocks, wherein the second division step may be recursively performed by using a sub-block including a boundary of the current image as a division target block until no sub-block including a boundary of the current image exists among the sub-blocks.

In the image encoding method according to the present invention, the first division step may be a step of dividing the current image into a plurality of maximum blocks of the same size.

In the image encoding method according to the present invention, the second division step may be a step of performing quad tree division or binary tree division on the division target block.

In the video encoding method according to the present invention, if the target block for division is of a size that allows quad-tree division, the second division step may perform the quad-tree division on the target block for division, and if the target block for division is of a size that does not allow quad-tree division, the second division step may perform the binary tree division on the target block for division.

In the image encoding method according to the present invention, when the quad tree division or the binary tree division is performed on the division target block in the second division step, the first division information indicating whether the division target block is divided into a quad tree or the second division information indicating whether the division target block is divided into a binary tree may not be encoded.

In the image encoding method according to the present invention, when the binary tree division is performed on the division target block in the second division step, the division direction of the binary tree division can be determined based on the shape of the encoding target block which is included in the division target block and is also an area within the boundary of the current image.

In the image encoding method according to the present invention, when the binary tree division is performed on the division target block in the second division step, division direction information indicating the division direction of the binary tree division may not be encoded.

In the image encoding method according to the present invention, the second division step further includes a step of comparing the size of a remaining area, which is included in the division target block and is simultaneously an area outside the boundary of the current image, with a predetermined threshold value, and based on the comparison result, one of the quad tree division and the binary tree division can be performed on the division target block.

In the image encoding method according to the present invention, the size of the remaining region is a smaller one of the horizontal length and the vertical length of the remaining region, and in the second division step, if the size of the remaining region is greater than the predetermined threshold value, the quad tree division can be performed on the division target block, and if the size of the remaining region is less than the predetermined threshold value, the binary tree division can be performed on the division target block.

In the image encoding method according to the present invention, the second division step is a step of performing binary tree division on the division target block, and the division direction of the binary tree division can be determined based on the shape of a remaining area that is included in the division target block and is also an area outside the boundary of the current image.

In the image encoding method according to the present invention, the division direction of the binary tree division may be a vertical direction when the shape of the remaining region is a vertically long block, and may be a horizontal direction when the shape of the remaining region is a horizontally long block.

A method for decoding an image according to the present invention comprises a first division step of division of a current image into a plurality of blocks, and a second division step of division of a division target block including a boundary of the current image among the plurality of blocks into a plurality of sub-blocks, wherein the second division step may be recursively performed by using a sub-block including a boundary of the current image as a division target block until no sub-block including a boundary of the current image exists among the sub-blocks.

In the image decoding method according to the present invention, the second division step may be a step of performing quad tree division or binary tree division on the division target block.

In the image decoding method according to the present invention, if the target block for division is of a size that allows quad-tree division, the second division step may perform the quad-tree division on the target block for division, and if the target block for division is of a size that does not allow quad-tree division, the second division step may perform the binary tree division on the target block for division.

In the image decoding method according to the present invention, when the quad tree division or the binary tree division is performed on the division target block in the second division step, the first division information indicating whether the division target block is divided into a quad tree or the second division information indicating whether the division target block is divided into a binary tree may be derived as a predetermined value without being decoded from the bitstream.

In the image decoding method according to the present invention, when the binary tree division is performed on the division target block in the second division step, the division direction of the binary tree division can be determined based on the shape of the decoding target block which is included in the division target block and is also an area within the boundary of the current image.

In the image decoding method according to the present invention, when the binary tree division is performed on the division target block in the second division step, division direction information indicating the division direction of the binary tree division may be derived as a predetermined value without being decoded from the bitstream.

In the image decoding method according to the present invention, the second division step further includes a step of comparing the size of a remaining area, which is included in the division target block and is simultaneously an area outside the boundary of the current image, with a predetermined threshold value, and based on the comparison result, one of the quad tree division and the binary tree division can be performed on the division target block.

In the image decoding method according to the present invention, the size of the remaining region is a smaller one of the horizontal length and the vertical length of the remaining region, and in the second division step, if the size of the remaining region is greater than the predetermined threshold value, the quad tree division can be performed on the division target block, and if the size of the remaining region is less than the predetermined threshold value, the binary tree division can be performed on the division target block.

In the image decoding method according to the present invention, the second division step is a step of performing binary tree division on the division target block, and the division direction of the binary tree division can be determined based on the shape of a remaining area that is included in the division target block and is also an area outside the boundary of the current image.

In the image decoding method according to the present invention, the division direction of the binary tree division may be a vertical direction when the shape of the remaining region is a vertically long block, and may be a horizontal direction when the shape of the remaining region is a horizontally long block.

A computer-readable recording medium according to the present invention can store a bitstream generated by an image encoding method according to the present invention.

According to the present invention, a padding method and device for adjusting the size of an image in block units in image encoding/decoding can be provided.

In addition, according to the present invention, a block division method and device that are efficient for image boundary processing in image encoding/decoding can be provided.

In addition, according to the present invention, a computer-readable recording medium storing a bitstream generated by an image encoding method/device according to the present invention can be provided.

The compression efficiency of an image can be improved by using the padding method and device or the block division method and device according to the present invention.

FIG. 1 is a block diagram briefly illustrating an image encoding device according to one embodiment of the present invention.
Figure 2 illustrates the types of intra-screen prediction modes defined in a video encoding/decoding device.
Figure 3 illustrates an on-screen prediction method based on planar mode.
Figure 4 illustrates reference pixels for on-screen prediction.
Figure 5 illustrates an on-screen prediction method based on horizontal and vertical modes.
FIG. 6 is a block diagram briefly illustrating an image decoding device according to one embodiment of the present invention.
Figure 7 is a drawing for explaining a padding process according to one embodiment of the present invention.
Figures 8a and 8b are drawings for explaining a padding process according to the present invention.
Figure 9 is a drawing for explaining the operation of the image segmentation unit (101) of Figure 1.
Figure 10 is a drawing for explaining the operation of a decoder according to the present invention.
Figure 11 is an example diagram illustrating a case where the size of the current image is not a multiple of the size of the maximum block.
FIGS. 12a to 12e are exemplary diagrams for explaining an image segmentation method according to one embodiment of the present invention.
FIGS. 13a to 13e are exemplary diagrams for explaining an image segmentation method according to another embodiment of the present invention.

The present invention can be modified in various ways and has various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention. In describing each drawing, similar reference numerals are used for similar components.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are only used to distinguish one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and/or includes a combination of a plurality of related described items or any one of a plurality of related described items.

When it is said that a component is "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. On the other hand, when it is said that a component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprises" or "has" and the like are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted.

FIG. 1 is a block diagram showing an image encoding device according to an embodiment of the present invention.

Referring to FIG. 1, an image encoding device (100) may include an image segmentation unit (101), an intra-screen prediction unit (102), an inter-screen prediction unit (103), a subtraction unit (104), a transformation unit (105), a quantization unit (106), an entropy encoding unit (107), an inverse quantization unit (108), an inverse transformation unit (109), an increase unit (110), a filter unit (111), and a memory (112).

Each component shown in FIG. 1 is independently depicted to indicate different characteristic functions in the image encoding device, and does not mean that each component is composed of separate hardware or a single software configuration unit. That is, each component is listed and included as a separate component for convenience of explanation, and at least two components among each component may be combined to form a single component, or one component may be divided into multiple components to perform a function, and such integrated and separated embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

In addition, some components may not be essential components that perform essential functions in the present invention, but may be optional components that are merely used to improve performance. The present invention may be implemented by including only essential components for implementing the essence of the present invention, excluding components that are merely used to improve performance, and a structure that includes only essential components, excluding optional components that are merely used to improve performance, is also included in the scope of the present invention.

The image segmentation unit (100) can segment an input image into at least one block. At this time, the input image can have various shapes and sizes such as pictures, slices, tiles, and segments. The block can mean a coding unit (CU), a prediction unit (PU), or a transformation unit (TU). The segmentation can be performed based on at least one of a quadtree or a binary tree. A quadtree is a method of dividing an upper block into four lower blocks whose width and height are half of those of the upper block. A binary tree is a method of dividing an upper block into lower blocks whose width or height is half of that of the upper block. Through the aforementioned binary tree-based segmentation, a block can have a non-square shape as well as a square shape.

Hereinafter, in the embodiments of the present invention, the encoding unit may be used to mean a unit that performs encoding, or may be used to mean a unit that performs decoding.

The prediction unit (102, 103) may include an inter-prediction unit (103) that performs inter-prediction and an intra-prediction unit (102) that performs intra-prediction. It may be determined whether to use inter-prediction or intra-prediction for a prediction unit, and specific information (e.g., intra-prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. At this time, the processing unit where the prediction is performed and the processing unit where the prediction method and specific contents are determined may be different. For example, the prediction method and the prediction mode, etc. may be determined in the prediction unit, and the prediction may be performed in the transformation unit.

The residual value (residual block) between the generated prediction block and the original block can be input to the transformation unit (105). In addition, the prediction mode information, motion vector information, etc. used for prediction can be encoded by the entropy encoding unit (107) together with the residual value and transmitted to the decoder. When a specific encoding mode is used, it is also possible to encode the original block as it is and transmit it to the decoder without generating the prediction block through the prediction unit (102, 103).

The prediction unit (102) within the screen can generate a prediction block based on reference pixel information surrounding the current block, which is pixel information within the current picture. If the prediction mode of a block surrounding the current block on which intra prediction is to be performed is inter prediction, a reference pixel included in a block surrounding the current block to which inter prediction is applied can be replaced with a reference pixel within another block surrounding the current block to which intra prediction is applied. That is, if a reference pixel is not available, the unavailable reference pixel information can be replaced with at least one reference pixel among the available reference pixels and used.

In intra prediction, the prediction mode can have a directional prediction mode that uses reference pixel information according to the prediction direction and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luminance information and the mode for predicting chrominance information can be different, and the intra prediction mode information used for predicting luminance information or the predicted luminance signal information can be utilized to predict chrominance information.

The prediction unit (102) within the screen may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a filter that performs filtering on the reference pixels of the current block and can adaptively determine whether to apply the filter according to the prediction mode of the current prediction unit. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

The reference pixel interpolation unit of the prediction unit (102) within the screen can interpolate the reference pixel to generate a reference pixel at a fractional unit position when the intra prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value interpolated from the reference pixel. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

A residual block including residual value information, which is a difference value between a prediction unit generated in a prediction unit (102, 103) and an original block of the prediction unit, can be generated. The generated residual block can be input to a transformation unit (130) and transformed.

Fig. 2 is a diagram for explaining an example of an intra prediction mode. The intra prediction mode illustrated in Fig. 2 has a total of 35 modes. Mode 0 represents a planar mode, mode 1 represents a DC mode, and modes 2 to 34 represent angular modes.

Figure 3 is a drawing for explaining on-screen prediction in flat mode.

FIG. 3 is a diagram for explaining the planar mode. In order to generate a prediction value of the first pixel P1 in the current block, the restored pixel T located at the upper right of the current block and the restored pixel at the same position in the Y-axis are linearly interpolated to generate the prediction value, as illustrated. Similarly, in order to generate a prediction value of the second pixel P2, the restored pixel at the same position in the X-axis and the restored pixel L located at the lower left of the current block are linearly interpolated to generate the prediction value, as illustrated. The average value of the two prediction pixels P1 and P2 becomes the final prediction pixel. In the planar mode, the prediction pixels are derived in the same manner as above to generate the prediction block of the current block.

Figure 4 is a diagram for explaining the DC mode. The average of the restored pixels around the current block is calculated, and then the average value is used as the predicted value of all pixels in the current block.

Fig. 5 is a diagram for explaining an example of generating a prediction block using mode 10 (horizontal mode) and mode 26 (vertical mode) of Fig. 2. When mode 10 is used, each reference pixel bordering the left side of the current block is copied in the right direction to generate a prediction block of the current block. Similarly, mode 26 generates a prediction block of the current block by copying each reference pixel bordering the upper side of the current block in the downward direction.

Referring back to FIG. 1, the inter-screen prediction unit (103) may predict a prediction unit based on information of at least one picture among the previous picture or the subsequent picture of the current picture, and in some cases, may predict a prediction unit based on information of a portion of an encoded region within the current picture. The inter-screen prediction unit (103) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

The reference picture interpolation unit can receive reference picture information from the memory (112) and generate pixel information below an integer pixel from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter (DCT-based Interpolation Filter) with different filter coefficients can be used to generate pixel information below an integer pixel in units of 1/4 pixels. In the case of a chrominance signal, a DCT-based 4-tap interpolation filter (DCT-based Interpolation Filter) with different filter coefficients can be used to generate pixel information below an integer pixel in units of 1/8 pixels.

The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can be used to derive a motion vector. The motion vector can have a motion vector value of 1/2 or 1/4 pixel units based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by using a different motion prediction method. Various methods such as the Skip method, the Merge method, and the AMVP (Advanced Motion Vector Prediction) method can be used as the motion prediction method.

The subtraction unit (104) subtracts the block currently to be encoded from the prediction block generated from the intra-screen prediction unit (102) or inter-screen prediction unit (103) to generate a residual block of the current block.

In the transformation unit (105), the residual block including the residual data can be transformed using a transformation method such as DCT, DST, KLT (Karhunen Loeve Transform), etc. At this time, the transformation method can be determined based on the intra prediction mode of the prediction unit used to generate the residual block. For example, depending on the intra prediction mode, DCT may be used in the horizontal direction and DST may be used in the vertical direction.

The quantization unit (106) can quantize the values converted to the frequency domain in the transformation unit (105). The quantization coefficients can vary depending on the block or the importance of the image. The values produced by the quantization unit (106) can be provided to the dequantization unit (108) and the entropy encoding unit (107).

The above transformation unit (105) and/or quantization unit (106) may be optionally included in the image encoding device (100). That is, the image encoding device (100) may encode the residual block by performing at least one of transformation or quantization on the residual data of the residual block, or skipping both transformation and quantization. Even if neither transformation nor quantization is performed in the image encoding device (100), a block that enters the input of the entropy encoding unit (107) is typically referred to as a transformation block.

The entropy encoding unit (107) entropy encodes the input data. Entropy encoding can use various encoding methods such as, for example, Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding).

The entropy encoding unit (107) can encode various information such as residual value coefficient information of an encoding unit and block type information, prediction mode information, division unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information from the prediction units (102, 103). In the entropy encoding unit (107), the coefficients of a transform block can encode various types of flags indicating, for each partial block within the transform block, a non-zero coefficient, a coefficient whose absolute value is greater than 1 or 2, and the sign of the coefficient. A coefficient that is not encoded by the flag alone can be encoded by the absolute value of the difference between the coefficient encoded by the flag and the coefficient of the actual transform block. The inverse quantization unit (108) and the inverse transform unit (109) inverse quantize the values quantized by the quantization unit (106) and inverse transform the values transformed by the transform unit (105). The residual value generated from the inverse quantization unit (108) and the inverse transformation unit (109) can be combined with the predicted unit predicted through the motion estimation unit, motion compensation unit, and the in-screen prediction unit (102) included in the prediction unit (102, 103) to generate a reconstructed block. The multiplier (110) multiplies the predicted block generated from the prediction unit (102, 103) and the residual block generated through the inverse transformation unit (109) to generate a reconstructed block.

The filter unit (111) may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter).

A deblocking filter can remove block distortion caused by boundaries between blocks in a restored picture. In order to determine whether to perform deblocking, it is possible to determine whether to apply a deblocking filter to the current block based on pixels included in several columns or rows included in the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied depending on the required deblocking filtering strength. In addition, when applying a deblocking filter, horizontal filtering and vertical filtering can be processed in parallel when performing vertical filtering and horizontal filtering.

The offset correction unit can correct the offset from the original image on a pixel basis for the image on which deblocking has been performed. In order to perform offset correction for a specific picture, a method can be used in which the pixels included in the image are divided into a certain number of regions, the regions to be offset are determined, and the offset is applied to the regions, or a method can be used in which the offset is applied by considering the edge information of each pixel.

Adaptive Loop Filtering (ALF) can be performed based on the value compared between the filtered restored image and the original image. After dividing the pixels included in the image into a predetermined group, one filter to be applied to the group is determined, and filtering can be performed differentially for each group. Information related to whether to apply ALF can be transmitted by luminance signal for each coding unit (CU), and the shape and filter coefficient of the ALF filter to be applied can be different for each block. In addition, the same shape (fixed shape) of the ALF filter can be applied regardless of the characteristics of the target block.

The memory (112) can store a restored block or picture produced through the filter unit (111), and the stored restored block or picture can be provided to the prediction unit (102, 103) when performing inter prediction.

FIG. 6 is a block diagram showing an image decoding device (600) according to one embodiment of the present invention.

Referring to FIG. 6, the image decoding device (600) may include an entropy decoding unit (601), an inverse quantization unit (602), an inverse transformation unit (603), an amplification unit (604), a filter unit (605), a memory (606), and a prediction unit (607, 608).

When an image bitstream generated by an image encoding device (100) is input to an image decoding device (600), the input bitstream can be decoded according to a process opposite to the process performed in the image encoding device (100).

The entropy decoding unit (601) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit (107) of the video encoding device (100). For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied corresponding to the method performed in the video encoder. In the entropy decoding unit (601), the coefficients of the transform block can be decoded based on various types of flags indicating, for each partial block within the transform block, a coefficient that is not 0, a coefficient whose absolute value is greater than 1 or 2, and the sign of the coefficient. A coefficient that is not expressed by the flag alone can be decoded by the sum of the coefficient expressed by the flag and the signaled coefficient.

The entropy decoding unit (601) can decode information related to intra prediction and inter prediction performed in the encoder. The inverse quantization unit (602) performs inverse quantization on a quantized transform block to generate a transform block. It operates substantially the same as the inverse quantization unit (108) of Fig. 1.

The inverse transform unit (603) performs an inverse transform on the transform block to generate a residual block. At this time, the transform method can be determined based on information about the prediction method (inter or intra prediction), the size and/or shape of the block, the intra prediction mode, etc. It operates substantially the same as the inverse transform unit (109) of Fig. 1.

The multiplication unit (604) multiplies the prediction block generated from the intra-screen prediction unit (607) or inter-screen prediction unit (608) and the residual block generated through the inverse transformation unit (603) to generate a restored block. It operates substantially the same as the multiplication unit (110) of Fig. 1.

The filter unit (605) reduces various types of noise occurring in restored blocks.

The filter unit (605) may include a deblocking filter, an offset correction unit, and an ALF.

Information on whether a deblocking filter has been applied to a corresponding block or picture from a video encoding device (100) and, if a deblocking filter has been applied, information on whether a strong filter or a weak filter has been applied can be provided. The deblocking filter of the video decoding device (600) can receive information related to the deblocking filter provided from the video encoding device (100) and perform deblocking filtering on the corresponding block in the video decoding device (600).

The offset correction unit can perform offset correction on the restored image based on information such as the type of offset correction applied to the image during encoding and the offset value.

ALF can be applied to an encoding unit based on ALF application information, ALF coefficient information, etc. provided from a video encoding device (100). This ALF information can be provided while being included in a specific parameter set. The filter unit (605) operates substantially the same as the filter unit (111) of Fig. 1.

Memory (606) stores the recovery block generated by the increase unit (604). It operates substantially the same as memory (112) of Fig. 1.

The prediction unit (607, 608) can generate a prediction block based on prediction block generation related information provided from the entropy decoding unit (601) and previously decoded block or picture information provided from the memory (606).

The prediction unit (607, 608) may include an intra-screen prediction unit (607) and an inter-screen prediction unit (608). Although not illustrated separately, the prediction unit (607, 608) may further include a prediction unit determination unit. The prediction unit determination unit may receive various information such as prediction unit information input from the entropy decoding unit (601), prediction mode information of an intra-prediction method, and motion prediction-related information of an inter-prediction method, and may distinguish a prediction unit from a current encoding unit and determine whether the prediction unit performs inter-prediction or intra-prediction. The inter-screen prediction unit (608) may perform inter-prediction on a current prediction unit based on information included in at least one of a previous picture or a subsequent picture of a current picture including the current prediction unit, by using information necessary for inter-prediction of the current prediction unit provided from the video encoding device (100). Alternatively, inter prediction can be performed based on information from some previously-reconstructed region within the current picture containing the current prediction unit.

In order to perform inter-screen prediction, it is possible to determine whether the motion prediction method of the prediction unit included in the encoding unit is Skip Mode, Merge Mode, or AMVP Mode based on the encoding unit.

The prediction unit (607) within the screen generates a prediction block using pixels located around the block to be currently encoded and previously restored.

The prediction unit (607) within the screen may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a filter that performs filtering on the reference pixels of the current block and may adaptively determine whether to apply the filter according to the prediction mode of the current prediction unit. The AIS filtering may be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the image encoding device (100). If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

The reference pixel interpolation unit of the prediction unit (607) within the screen can interpolate the reference pixel to generate a reference pixel at a fractional unit position when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value interpolated from the reference pixel. The generated reference pixel at the fractional unit position can be used as a prediction pixel of a pixel in the current block. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

The on-screen prediction unit (607) operates substantially the same as the on-screen prediction unit (102) of Fig. 1.

The inter-screen prediction unit (608) generates an inter-prediction block using reference pictures and motion information stored in the memory (606). The inter-screen prediction unit (608) operates substantially the same as the inter-screen prediction unit (103) of Fig. 1.

The present invention relates particularly to padding and image boundary processing, and various embodiments of the present invention will be described in more detail below with reference to the drawings.

According to one embodiment of the present invention, a preprocessing process may be performed before the image to be encoded is input to the image segmentation unit (101) of Fig. 1. The image to be encoded may have various horizontal and vertical sizes in pixel units. However, since encoding and decoding of the image are performed in arbitrary block units rather than pixel units, a padding process may be required to match the size of the image to be encoded in block units.

Figure 7 is a drawing for explaining a padding process according to one embodiment of the present invention.

In the example illustrated in Fig. 7, the target image to be encoded includes a block region and a non-block region. By performing the above preprocessing process on the target image to be encoded, a padded region can be added. The target image to which the padded region has been added can be input to the image segmentation unit (101) of Fig. 1.

In the example illustrated in FIG. 7, the unit of the minimum block may be 4x4. The horizontal or vertical length of the block may be 2 ⁿ . Therefore, for example, if the horizontal and vertical lengths of the image to be encoded are each 9, the right 1st column and the bottom 1st row of the image to be encoded may be a non-block area that is not included in the block area. In this way, if the image to be encoded includes a non-block area, padding may be performed considering the size of the minimum block. In this way, by performing padding considering the block size, the padded image to be encoded does not include a non-block area. The padded image to be encoded is input to the image segmentation unit (101) of FIG. 1 and is divided into a plurality of blocks, so that block-by-block encoding may be performed.

Figures 8a and 8b are drawings for explaining a padding process according to the present invention.

First, padding can be performed in the horizontal direction using the pixels closest to the pixels to be padded. In the example shown in Fig. 8a, the top three padding target pixels can be padded using the pixels A in the non-block area adjacent to the left.

Thereafter, padding can be performed in the vertical direction using the pixels closest to the pixels to be padded. In the example shown in Fig. 8b, the three leftmost padding target pixels can be padded using the pixels I of the non-block area adjacent to the top.

In the above example, the padding process from the horizontal direction to the vertical direction has been described, but it is not limited thereto, and padding from the vertical direction to the horizontal direction may also be performed. The size of the minimum block may be a block unit as described with reference to Fig. 9, or may be a sub-block unit.

Figure 9 is a drawing for explaining the operation of the image segmentation unit (101) of Figure 1.

The input image can be divided into multiple maximum blocks. The size of the maximum block can be preset or signaled via the bitstream. For example, if the size of the input image is 128x128 and the size of the maximum block is 64x64, the input image can be divided into four maximum blocks.

Each maximum block can be input to the image segmentation unit (101) of Fig. 1 and divided into multiple blocks. The input image can be input directly to the image segmentation unit (101) without being divided into multiple maximum blocks. The minimum size of the divided block and/or the maximum size of the block that can be divided into sub-blocks can be preset or signaled in the header of the upper level of the block. The upper level can be at least one of video, sequence, picture, slice, tile, and largest coding unit (LCU).

The current block can be split into four sub-blocks or into two sub-blocks.

The current block can be divided into four sub-blocks, which can be called a quad-tree division. The current block can be divided into four sub-blocks with the same size by a quad-tree division. The first division information is information indicating whether the current block is divided into a quad-tree. The first division information can be, for example, a 1-bit flag.

The current block being split into two sub-blocks can be called a binary-tree split. The current block can be split into two sub-blocks having the same or different sizes by the binary-tree split. The second split information is information indicating whether the current block is split into a binary-tree split. The second split information can be, for example, a 1-bit flag. If the current block is split into a binary-tree split, split direction information can be further signaled. The split direction information can indicate whether the split of the current block is a horizontal split or a vertical split. The split direction information can be, for example, a 1-bit flag. If the current block is split into two sub-blocks having different sizes, split shape information can be further signaled. The split shape information can indicate a split ratio of the two sub-blocks.

The current block can be recursively split using quad-tree splitting and binary-tree splitting. For example, if the current block is quad-tree split into four sub-blocks, each sub-block can be recursively quad-tree split, binary-tree split, or not split. If the current block is binary-tree split into two sub-blocks, each sub-block can be recursively quad-tree split, binary-tree split, or not split. Alternatively, quad-tree splitting may not be performed on sub-blocks generated by binary-tree splitting.

The encoder can determine whether or not to quad-tree-divide the input current block. Based on the determination, first division information can be encoded (S901). If the current block is quad-tree-divided (Yes in S902), the current block can be quad-tree-divided into four blocks (S903). Each of the four blocks generated by the quad-tree division can be re-input to step S901 according to a predetermined order (S904, S916). The predetermined order can be a Z-scan order. In step S904, a first block among the four blocks according to the predetermined order can be specified and input to step S901 as the current block.

If the current block is not quad-tree split (No in S902), the encoder can determine whether to split the current block into a binary tree. Second split information can be encoded based on the determination (S905). If the current block is split into a binary tree (Yes in S906), the encoder can determine whether to split the current block into horizontal or vertical splits, and can encode split direction information based on the determination (S907). If the current block is split into horizontal splits (Yes in S908), horizontal splitting for the current block can be performed (S909). Otherwise (No in S908), vertical splitting for the current block can be performed (S910). If the binary tree splitting for the current block is an asymmetric split, a split ratio can be determined so that split shape information can be further encoded. In this case, steps S909 and S910 can be performed in consideration of the split shape information. Each of the two sub-blocks generated by the binary tree partitioning can be input back to step S905 in a predetermined order (S911, S914). The predetermined order may be from left to right, or from top to bottom. In step S911, a first sub-block among the two sub-blocks in the predetermined order can be specified and input as a current block to step S905. If quad-tree partitioning is allowed again for the sub-block generated by the binary tree partitioning, each of the two blocks generated by the binary tree partitioning can also be input to step S901 in a predetermined order.

If the current block is not binary tree split (No in S906), encoding for the current block or current sub-block may be performed (S912). The encoding in step S912 may include prediction, transformation, quantization, etc.

If the encoded sub-block in step S912 is not the last sub-block generated by binary tree partitioning (No in S913), the next sub-block generated by binary tree partitioning can be specified and input as the current block in step S905 (S914).

If the encoded sub-block in step S912 is the last sub-block generated by binary tree partitioning (Yes in S913), it can be determined whether the block to which the encoded sub-block belongs is the last block among the blocks generated by quad tree partitioning (S915). If it is the last block (Yes in S915), encoding for the maximum block input in step S901 or the current image can be terminated. If it is not the last block (No in S915), the next block generated by quad tree partitioning can be specified and input as the current block in step S901 (S916).

Figure 10 is a drawing for explaining the operation of a decoder according to the present invention.

The decoder can decode the first division information of the input current block (S1001). If the current block is quad-tree divided (Yes in S1002), the current block can be quad-tree divided into four blocks (S1003). Each of the four blocks generated by the quad-tree division can be input again to step S1001 according to a predetermined order (S1004, S1016). The predetermined order can be a Z-scan order. In step S1004, a first block among the four blocks according to the predetermined order can be specified and input as the current block to step S1001.

If the current block is not quad-tree split (No in S1002), the decoder can decode second split information of the current block (S1005). If the current block is binary-tree split (Yes in S1006), the decoder can decode split direction information (S1007). If the current block is horizontally split (Yes in S1008), horizontal splitting for the current block can be performed (S1009). Otherwise (No in S1008), vertical splitting for the current block can be performed (S1010). If the binary tree splitting for the current block is an asymmetric split, split shape information can be further decoded. In this case, steps S1009 and S1010 can be performed in consideration of the split shape information. Each of the two sub-blocks generated by the binary tree splitting can be re-input to step S1005 in a predetermined order (S1011, S1014). The above predetermined order may be from left to right, or from top to bottom. In step S1011, a first sub-block among the two sub-blocks according to the above predetermined order may be specified and input as a current block to step S1005. If quad-tree division is allowed again for the sub-block generated by binary tree division, each of the two blocks generated by binary tree division may be input to step S1001 according to the predetermined order.

If the current block is not binary tree split (No in S1006), decryption for the current block or the current sub-block may be performed (S1012). The decryption in step S1012 may include prediction, inverse quantization, inverse transformation, etc.

If the decrypted sub-block in step S1012 is not the last sub-block generated by binary tree partitioning (No in S1013), the next sub-block generated by binary tree partitioning can be specified and input as the current block in step S1005 (S1014).

If the decrypted sub-block in step S1012 is the last sub-block generated by binary tree partitioning (Yes in S1013), it can be determined whether the block to which the decrypted sub-block belongs is the last block among the blocks generated by quad tree partitioning (S1015). If it is the last block (Yes in S1015), decryption for the maximum block input in step S1001 or the current image can be terminated. If it is not the last block (No in S1015), the next block generated by quad tree partitioning can be specified and input as the current block in step S1001 (S1016).

Figure 11 is an example diagram illustrating a case where the size of the current image is not a multiple of the size of the maximum block.

As illustrated in Fig. 11, when dividing the current image into multiple maximum blocks, an area corresponding to a portion of the maximum block size remains on the right or bottom of the current image. That is, in the case of maximum block 2, maximum block 5, maximum block 6, maximum block 7, or maximum block 8, pixels of the current image exist only for some areas.

Below, an image segmentation method according to the present invention for efficiently segmenting a current image as illustrated in FIG. 11 is described.

FIGS. 12A to 12E are exemplary diagrams for explaining an image segmentation method according to one embodiment of the present invention. Hereinafter, a segmentation method of the maximum block 2 of FIG. 11 and/or encoding of segmentation information will be described with reference to FIGS. 12A to 12E. In the following description with reference to FIGS. 12A to 12E, it is assumed that the size of the current image is 146x146, the size of the maximum block is 64x64, the minimum size of a block that can be split into a quad tree is 16x16, and the minimum size of a sub-block is 2x2.

Fig. 12a is an enlargement of maximum block 2, and the part surrounded by a bold line is an area within the current image. Maximum block 2 may be a segmentation target block described with reference to Fig. 9. As illustrated in Fig. 12a, the rightmost position and the bottommost position of maximum block 2 are not completely included in the area of the current image. In addition, the size of maximum block 2 is 64x64, which is a size that allows quad-tree segmentation. In this case, maximum block 2 may be quad-tree segmented (S903). That is, step S901 for maximum block 2 may be omitted. Thereafter, segmentation and encoding may be performed on block A0, which is the first block among four blocks (A0, A1, A2, and A3) generated by quad-tree segmenting maximum block 2. Since blocks A1 and A3 are not completely included in the area of the current image, segmentation and encoding may be omitted. Block A2 may be segmented and encoded according to the present invention, similarly to block A0.

FIG. 12b is an enlargement of block A0 of FIG. 12a, and a portion surrounded by a bold line is an area within the current image. Block A0 may be a segmentation target block described with reference to FIG. 9. As illustrated in FIG. 12b, the rightmost position (B1) and the bottommost position (B3) of block A0 are not completely included in the area of the current image. In addition, the size of block A0 is 32x32, which is a size that allows quad-tree segmentation. In this case, block A0 may be quad-tree segmented (S903). That is, step S901 for block A0 may be omitted. Thereafter, segmentation and encoding may be performed on block B0, which is the first block among four blocks (B0, B1, B2, and B3) generated by quad-tree segmenting block A0. Block B0 may be segmented and encoded by the method described with reference to FIG. 9. That is, block B0 can be encoded without being split or split by quad tree splitting and/or binary tree splitting. Block B2 can be split and encoded according to the present invention similarly to block B0. Since blocks B1 and B3 are not completely included in the area of the current image, splitting and encoding can be omitted. Block B2 can be split and encoded in the same manner as block B0. That is, step S901 cannot be omitted for blocks B0 and B2. However, step S901 can be omitted for blocks B1 and B3, as described below.

FIG. 12C is an enlarged view of block B1 of FIG. 12B, and a portion surrounded by a bold line is an area within the current image. Block B1 may be a segmentation target block described with reference to FIG. 9. As illustrated in FIG. 12C, the rightmost position and the bottommost position of block B1 are not completely included in the area of the current image. In addition, the size of block B1 is 16x16, which is a size that cannot be quad-tree segmented. In this case, block B1 cannot be quad-tree segmented, but can be binary-tree segmented. In addition, since the encoding target block is a vertically long rectangle, the segmentation direction can be determined in the vertical direction. Therefore, steps S901 to S907 may be omitted for block B1, and binary-tree segmentation of step S909 or S910 may be performed depending on the shape of the encoding target block. After that, segmentation and encoding can be performed on block C1, which is the first block among the two blocks (C1 and C2) generated by binary tree segmentation of block B1. Since block C2 is not completely included in the area of the current image, segmentation and encoding can be omitted.

FIG. 12d is an enlargement of block C1 of FIG. 12c, and a portion surrounded by a bold line is an area within the current image. Block C1 may be a segmentation target block described with reference to FIG. 9. As illustrated in FIG. 12d, the rightmost position and the bottommost position of block C1 are not completely included in the area of the current image. In addition, since binary tree segmentation is applied to B1, which is an upper block of block C1, block C1 may also be binary tree segmented. In addition, since the encoding target block is a vertically long rectangle, the segmentation direction may be determined in the vertical direction. Therefore, steps S905 to S907 may be omitted for block C1, and binary tree segmentation of step S909 or S910 may be performed depending on the shape of the encoding target block. Thereafter, segmentation and encoding may be performed on block D1, which is the first block among two blocks (D1 and D2) generated by binary tree segmenting block C1. Since block D2 is not fully contained within the current image area, segmentation and encoding can be omitted.

FIG. 12e is an enlarged view of block D1 of FIG. 12d, and a portion surrounded by a bold line is an area within the current image. Block D1 may be a segmentation target block described with reference to FIG. 9. As illustrated in FIG. 12e, the rightmost position and the bottommost position of block D1 are not completely included in the area of the current image. In addition, since binary tree segmentation is applied to C1, which is an upper block of block D1, block D1 may also be binary tree segmented. In addition, since the encoding target block is a vertically long rectangle, the segmentation direction may be determined in the vertical direction. Therefore, steps S905 to S907 may be omitted for block D1, and binary tree segmentation of step S909 or S910 may be performed depending on the shape of the encoding target block. Thereafter, segmentation and encoding may be performed on block E1, which is the first block among two blocks (E1 and E2) generated by binary tree segmenting block D1. The horizontal size of block E1 is equal to 2, which is assumed to be the minimum size of a sub-block. Therefore, vertical segmentation cannot be performed on block E1. That is, block E1 can be encoded without being segmented, or can be encoded after being horizontally segmented. In this case, only information about whether block E1 is segmented can be signaled. Since block E2 is not completely included in the area of the current image, segmentation and encoding can be omitted.

The partition information omission methods of maximum block 2 and maximum block 5 illustrated in Fig. 11 may be the same. In addition, the partition information omission methods of maximum block 6 and maximum block 7 may be the same as the partition information omission methods of maximum block 2 and maximum block 5 except for the difference in the direction of horizontal or vertical. For maximum block 8, the partition information omission may be performed in the same manner as other maximum blocks by using a criterion set horizontally or vertically.

Hereinafter, with reference to FIGS. 12a to 12e, a method for dividing the maximum block 2 of FIG. 11 and/or decoding the division information will be described.

Fig. 12a is an enlargement of maximum block 2, and a part surrounded by a bold line is an area within the current image. Maximum block 2 may be a segmentation target block described with reference to Fig. 9. As illustrated in Fig. 12a, the rightmost position and the bottommost position of maximum block 2 are not completely included in the area of the current image. In addition, the size of maximum block 2 is 64x64, which is a size that allows quad-tree segmentation. In this case, maximum block 2 may be quad-tree segmented (S1003). That is, step S1001 for maximum block 2 may be omitted. Thereafter, segmentation and decoding may be performed on block A0, which is the first block among four blocks (A0, A1, A2, and A3) generated by quad-tree segmenting maximum block 2. Since blocks A1 and A3 are not completely included in the area of the current image, segmentation and decoding may be omitted. Block A2 may be segmented and decoded according to the present invention, similarly to block A0.

FIG. 12b is an enlargement of block A0 of FIG. 12a, and a portion surrounded by a bold line is an area within the current image. Block A0 may be a segmentation target block described with reference to FIG. 9. As illustrated in FIG. 12b, the rightmost position and the bottommost position of block A0 are not completely included in the area of the current image. In addition, the size of block A0 is 32x32, which is a size that allows quad-tree segmentation. In this case, block A0 may be quad-tree segmented (S1003). That is, step S1001 for block A0 may be omitted. Thereafter, segmentation and decoding may be performed on block B0, which is the first block among four blocks (B0, B1, B2, and B3) generated by quad-tree segmenting block A0. Block B0 may be segmented and decoded by the method described with reference to FIG. 9. That is, block B0 can be decoded with or without being split by quad tree splitting and/or binary tree splitting. Block B2 can be split and decoded according to the present invention similarly to block B0. Since blocks B1 and B3 are not completely included in the area of the current image, splitting and decoding can be omitted. Block B2 can be split and decoded in the same manner as block B0. That is, step S1001 cannot be omitted for blocks B0 and B2. However, step S1001 can be omitted for blocks B1 and B3, as described below.

FIG. 12C is an enlargement of block B1 of FIG. 12B, and a portion surrounded by a bold line is an area within the current image. Block B1 may be a segmentation target block described with reference to FIG. 9. As illustrated in FIG. 12C, the rightmost position and the bottommost position of block B1 are not completely included in the area of the current image. In addition, the size of block B1 is 16x16, which is a size that cannot be quad-tree segmented. In this case, block B1 cannot be quad-tree segmented, but can be binary-tree segmented. In addition, since the block to be decoded is a vertically long rectangle, the segmentation direction can be determined in the vertical direction. Therefore, steps S1001 to S1007 may be omitted for block B1, and binary-tree segmentation of step S1009 or S1010 may be performed depending on the shape of the block to be decoded. After that, segmentation and decoding can be performed on block C1, which is the first block among the two blocks (C1 and C2) generated by binary tree segmentation of block B1. Since block C2 is not completely included in the area of the current image, segmentation and decoding can be omitted.

FIG. 12d is an enlargement of block C1 of FIG. 12c, and a portion surrounded by a bold line is an area within the current image. Block C1 may be a segmentation target block described with reference to FIG. 9. As illustrated in FIG. 12d, the rightmost position and the bottommost position of block C1 are not completely included in the area of the current image. In addition, since binary tree segmentation is applied to B1, which is an upper block of block C1, block C1 may also be binary tree segmented. In addition, since the block to be decoded is a vertically long rectangle, the segmentation direction may be determined in the vertical direction. Therefore, steps S1005 to S1007 may be omitted for block C1, and binary tree segmentation of step S1009 or S1010 may be performed depending on the shape of the block to be decoded. Thereafter, segmentation and decoding may be performed on block D1, which is the first block among two blocks (D1 and D2) generated by binary tree segmenting block C1. Since block D2 is not fully contained within the current image area, segmentation and decoding can be omitted.

FIG. 12E is an enlargement of block D1 of FIG. 12D, and a portion surrounded by a bold line is an area within the current image. Block D1 may be a segmentation target block described with reference to FIG. 9. As illustrated in FIG. 12E, the rightmost position and the bottommost position of block D1 are not completely included in the area of the current image. In addition, since binary tree segmentation is applied to C1, which is an upper block of block D1, block D1 may also be binary tree segmented. In addition, since the block to be decoded is a vertically long rectangle, the segmentation direction may be determined in the vertical direction. Accordingly, steps S1005 to S1007 may be omitted for block D1, and binary tree segmentation of step S1009 or S1010 may be performed depending on the shape of the block to be decoded. Thereafter, segmentation and decoding may be performed on block E1, which is the first block among two blocks (E1 and E2) generated by binary tree segmenting block D1. The horizontal size of block E1 is equal to 2, which is assumed to be the minimum size of a sub-block. Therefore, vertical segmentation cannot be performed on block E1. That is, block E1 can be decoded without being segmented, or can be decoded after being horizontally segmented. In this case, only information about whether block E1 is segmented can be signaled. Since block E2 is not completely included in the area of the current image, segmentation and decoding can be omitted.

In the embodiment described with reference to FIG. 12, when quad-tree division is performed on a division target block and a predetermined condition is satisfied, binary tree division may be performed. In the above embodiment, whether the size of the division target block is a size that allows quad-tree division corresponds to the above predetermined condition. The above predetermined condition may be set using the minimum size and/or threshold value of a block that allows quad-tree division and a block that allows binary tree division.

When a threshold value is used, one of quad tree segmentation and binary tree segmentation can be performed based on the result of comparing the size of an area (hereinafter referred to as “remaining area”) that is not completely included in the current image among the segmentation target blocks with the threshold value. In the following description, it is assumed that the threshold value is 32.

For example, in Fig. 12a, the size of the target block to be encoded is 18x64, and the size of the remaining area is 46x64. Since the remaining area is a vertically long rectangle, the horizontal length of the remaining area can be compared with a threshold value. The horizontal length of the remaining area is 46, which is larger than the threshold value 32, so that quad tree partitioning can be performed on the 64x64 target block to be partitioned. In Fig. 12b, the horizontal length of the remaining area is 14, which is smaller than the threshold value 32, so that binary tree partitioning can be performed on the 32x32 target block to be partitioned. That is, even if the size of the target block to be partitioned is a size that allows quad tree partitioning, binary tree partitioning can be performed instead of quad tree partitioning according to the above conditions. At this time, the partitioning direction of the binary tree partitioning is based on the smaller one of the width and height of the 18x32 target block to be encoded. Therefore, block A0 of Fig. 12b can be partitioned into two vertically long 16x32 blocks. After this, vertical binary tree splitting can be performed repeatedly until the block size becomes such that binary tree splitting is no longer possible.

The above threshold value can be signaled through the upper level header of the above-mentioned block. The above threshold value should be a value between the maximum size and the minimum size of a block that can be split into quad tree divisions. In addition, the above threshold value can be set smaller than the maximum size of a block that can be split into binary tree divisions. If the above threshold value is set larger than the maximum size of a block that can be split into binary tree divisions, the above threshold value can be changed to the maximum size of a block that can be split into binary tree divisions.

FIGS. 13a to 13e are exemplary diagrams for explaining an image segmentation method according to another embodiment of the present invention.

Fig. 13a is an enlargement of maximum block 2, and the part surrounded by a bold line is an area within the current image. Maximum block 2 may be a segmentation target block described with reference to Fig. 9. As shown in Fig. 13a, the rightmost position and the bottommost position of maximum block 2 are not completely included in the area of the current image. In this case, a block segmentation method may be selected based on the shape of the remaining area. Since the remaining area in Fig. 13a is a vertically long 46x64 block, a vertical binary tree segmentation may be performed for maximum block 2. That is, steps S901 to S907 for maximum block 2 may be omitted, and step S909 or S910 may be performed depending on the shape of the remaining area.

FIG. 13b is an enlarged view of block A0 of FIG. 13a, and a portion surrounded by a bold line is an area within the current image. Block A0 may be a segmentation target block described with reference to FIG. 9. As shown in FIG. 13b, the rightmost position and the bottommost position of block A0 are not completely included in the area of the current image. In this case, a block segmentation method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13b is a vertically long 14x64 block, a vertical binary tree segmentation may be performed on block A0. Block B0 may be encoded using sub-block segmentation since it is a block within the current image area. Steps S905 to S908 cannot be omitted for block B0.

Fig. 13c is an enlarged view of block B1 of Fig. 13b, and a portion surrounded by a bold line is an area within the current image. Block B1 may be a segmentation target block described with reference to Fig. 9. As shown in Fig. 13c, the rightmost position and the bottommost position of block B1 are not completely included in the area of the current image. In this case, a block segmentation method may be selected based on the shape of the remaining area. Since the remaining area in Fig. 13c is a vertically long 14x64 block, a vertical binary tree segmentation may be performed on block B1. That is, steps S901 to S907 for block B1 may be omitted, and step S909 or S910 may be performed depending on the shape of the remaining area. Since block C2 is not completely included in the area of the current image, segmentation and encoding may be omitted.

FIG. 13d is an enlarged view of block C1 of FIG. 13c, and a portion surrounded by a bold line is an area within the current image. Block C1 may be a segmentation target block described with reference to FIG. 9. As shown in FIG. 13d, the rightmost position and the bottommost position of block C1 are not completely included in the area of the current image. In this case, a block segmentation method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13c is a vertically long 6x64 block, a vertical binary tree segmentation may be performed on block C1. That is, steps S901 to S907 for block C1 may be omitted, and step S909 or S910 may be performed depending on the shape of the remaining area. Since block D2 is not completely included in the area of the current image, segmentation and encoding may be omitted.

Fig. 13e is an enlargement of block D1 of Fig. 13d, and the part surrounded by a thick line is an area within the current image. Block D1 may be a segmentation target block described with reference to Fig. 9. As shown in Fig. 13e, the rightmost position and the bottommost position of block D1 are not completely included in the area of the current image. In this case, a block segmentation method may be selected based on the shape of the remaining area. In Fig. 13c, since the remaining area is a vertically long 2x64 block, a vertical binary tree segmentation may be performed on block D1. That is, steps S901 to S907 for block D1 may be omitted, and step S909 or S910 may be performed depending on the shape of the remaining area. The horizontal size of block E1 is equal to 2, which is assumed to be the minimum size of a sub-block. Therefore, vertical segmentation cannot be performed on block E1. That is, block E1 may be encoded without being segmented, or may be encoded after being horizontally segmented. In this case, only information about whether block E1 is segmented can be signaled. Since block E2 is not completely included in the area of the current image, segmentation and encoding can be omitted.

Hereinafter, with reference to FIGS. 13a to 13e, a method for dividing the maximum block 2 of FIG. 11 and/or decoding the division information will be described.

Fig. 13a is an enlargement of maximum block 2, and the part surrounded by a bold line is an area within the current image. Maximum block 2 may be a segmentation target block described with reference to Fig. 9. As shown in Fig. 13a, the rightmost position and the bottommost position of maximum block 2 are not completely included in the area of the current image. In this case, a block segmentation method may be selected based on the shape of the remaining area. Since the remaining area in Fig. 13a is a vertically long 46x64 block, a vertical binary tree segmentation may be performed for maximum block 2. That is, steps S1001 to S1007 for maximum block 2 may be omitted, and step S1009 or S1010 may be performed depending on the shape of the remaining area.

FIG. 13b is an enlarged view of block A0 of FIG. 13a, and a portion surrounded by a bold line is an area within the current image. Block A0 may be a segmentation target block described with reference to FIG. 9. As shown in FIG. 13b, the rightmost position and the bottommost position of block A0 are not completely included in the area of the current image. In this case, a block segmentation method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13b is a vertically long 14x64 block, a vertical binary tree segmentation may be performed on block A0. Block B0 may be decoded using sub-block segmentation since it is a block within the current image area. Steps S1005 to S1008 cannot be omitted for block B0.

Fig. 13c is an enlarged view of block B1 of Fig. 13b, and a portion surrounded by a bold line is an area within the current image. Block B1 may be a segmentation target block described with reference to Fig. 9. As shown in Fig. 13c, the rightmost position and the bottommost position of block B1 are not completely included in the area of the current image. In this case, a block segmentation method may be selected based on the shape of the remaining area. Since the remaining area in Fig. 13c is a vertically long 14x64 block, a vertical binary tree segmentation may be performed on block B1. That is, steps S1001 to S1007 for block B1 may be omitted, and step S1009 or S1010 may be performed depending on the shape of the remaining area. Since block C2 is not completely included in the area of the current image, segmentation and decoding may be omitted.

FIG. 13d is an enlarged view of block C1 of FIG. 13c, and a portion surrounded by a bold line is an area within the current image. Block C1 may be a segmentation target block described with reference to FIG. 9. As illustrated in FIG. 13d, the rightmost position and the bottommost position of block C1 are not completely included in the area of the current image. In this case, a block segmentation method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13c is a vertically long 6x64 block, a vertical binary tree segmentation may be performed on block C1. That is, steps S1001 to S1007 for block C1 may be omitted, and step S1009 or S1010 may be performed depending on the shape of the remaining area. Since block D2 is not completely included in the area of the current image, segmentation and decoding may be omitted.

Fig. 13e is an enlargement of block D1 of Fig. 13d, and the part surrounded by a thick line is an area within the current image. Block D1 may be a segmentation target block described with reference to Fig. 9. As shown in Fig. 13e, the rightmost position and the bottommost position of block D1 are not completely included in the area of the current image. In this case, a block segmentation method may be selected based on the shape of the remaining area. In Fig. 13c, since the remaining area is a vertically long 2x64 block, a vertical binary tree segmentation may be performed on block D1. That is, steps S1001 to S1007 for block D1 may be omitted, and step S1009 or S1010 may be performed depending on the shape of the remaining area. The horizontal size of block E1 is equal to 2, which is assumed to be the minimum size of a sub-block. Therefore, vertical segmentation cannot be performed on block E1. That is, block E1 may be decoded without being segmented, or may be decoded after being horizontally segmented. In this case, only information about whether block E1 is segmented can be signaled. Since block E2 is not completely included in the area of the current image, segmentation and decoding can be omitted.

In the embodiment described with reference to FIG. 13, only binary tree division is performed on the division target block, and the direction of the binary tree division can be determined according to the shape of the remaining area.

In the embodiment described with reference to FIGS. 9 and 10, segmentation processing for the boundary of the image can be performed in units of sub-blocks or blocks. Information regarding whether segmentation processing for the boundary of the image is to be performed in which unit can be signaled. For example, the information can be signaled from the encoder to the decoder through the header of the upper level of the block described above.

Although the exemplary methods of the present disclosure are presented as a series of operations for clarity of description, this is not intended to limit the order in which the steps are performed, and each step may be performed simultaneously or in a different order, if desired. In order to implement a method according to the present disclosure, additional steps may be included in addition to the steps illustrated, or some of the steps may be excluded and the remaining steps may be included, or some of the steps may be excluded and additional other steps may be included.

The various embodiments of the present disclosure are not intended to list all possible combinations but rather to illustrate representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combinations of two or more.

Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, the embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

The scope of the present disclosure includes software or machine-executable instructions (e.g., an operating system, an application, firmware, a program, etc.) that cause operations according to the methods of various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or instructions stored thereon and being executable on the device or the computer.

Claims (16)

A first segmentation step for dividing the current image into multiple blocks; and
A second segmentation step is included for segmenting a segmentation target block including a boundary of the current image among the plurality of blocks into a plurality of sub-blocks.
The second segmentation step is performed recursively using a sub-block including a boundary of the current image as a segmentation target block until there is no sub-block including a boundary of the current image among the sub-blocks.
The second division step selectively applies quad tree division or binary tree division based on the size and threshold of the division target block to divide the division target block, and if the quad tree division or binary tree division is performed on the division target block in the second division step, division information indicating whether the division target block is divided is not encoded.
The above quad tree division is performed based on the result of comparing the size of the division target block and the size of a block that can be divided into quad tree divisions.
The above binary tree division is performed based on the result of comparing the size of the division target block with the size of a block capable of binary tree division and the threshold value when the quad tree division is not performed.
A method for encoding an image, wherein the threshold value is different from the size of a block capable of binary tree division, and is a value between the maximum size and the minimum size of a block capable of quad tree division. delete In the first paragraph,
An image encoding method wherein the second segmentation step and encoding are omitted for sub-blocks among the above sub-blocks that do not include the boundary of the current image and are located outside the current image. In the first paragraph,
A video encoding method wherein, when the quad tree division or the binary tree division is performed on the division target block in the second division step, the first division information indicating whether the division target block is subjected to a quad tree division or the second division information indicating whether the division target block is subjected to a binary tree division is not encoded. In the first paragraph,
If the binary tree division is performed on the division target block in the second division step,
An image encoding method wherein the division direction of the binary tree division is determined based on whether the rightmost position of the division target block and the bottommost position of the division target block are not included in the area of the current image. In the first paragraph,
If the binary tree division is performed on the division target block in the second division step,
An image encoding method in which the split direction information indicating the split direction of the above binary tree split is not encoded. In paragraph 5,
The division direction of the above binary tree division is,
If the rightmost position of the above-mentioned segmentation target block is not included in the area of the above-mentioned current image, it is in the vertical direction,
A horizontal image encoding method, wherein the lowest position of the above-mentioned segmentation target block is not included in the area of the above-mentioned current image. A first segmentation step for dividing the current image into multiple blocks; and
A second segmentation step is included for segmenting a segmentation target block including a boundary of the current image among the plurality of blocks into a plurality of sub-blocks.
The second segmentation step is performed recursively using a sub-block including a boundary of the current image as a segmentation target block until there is no sub-block including a boundary of the current image among the sub-blocks.
The second division step selectively applies quad tree division or binary tree division based on the size and threshold of the division target block to divide the division target block, and when the quad tree division or binary tree division is performed on the division target block in the second division step, division information indicating whether the division target block is divided is derived as a predetermined value without being decoded from the bitstream.
The above quad tree division is performed based on the result of comparing the size of the division target block and the size of a block that can be divided into quad tree divisions.
The above binary tree division is performed based on the result of comparing the size of the division target block with the size of a block capable of binary tree division and the threshold value when the quad tree division is not performed.
A method for decoding an image, wherein the threshold value is different from the size of a block capable of binary tree division, and is a value between the maximum size and the minimum size of a block capable of quad tree division. delete In Article 8,
If either the rightmost position of the above division target block or the bottommost position of the above division target block is not included in the area of the current image,
An image decoding method for determining whether to apply binary tree splitting to a block to be split based on a maximum block size for which binary tree splitting is possible. In Article 8,
An image decoding method wherein the second segmentation step and decoding are omitted for sub-blocks among the above sub-blocks that do not include the boundary of the current image and are located outside the current image. In Article 8,
A method for decoding an image, wherein, in the second division step, when the quad tree division or the binary tree division is performed on the division target block, the first division information indicating whether the division target block is divided into a quad tree or the second division information indicating whether the division target block is divided into a binary tree is not decoded from the bitstream but is derived as a predetermined value. In Article 8,
If the binary tree division is performed on the division target block in the second division step,
An image decoding method, wherein the division direction of the binary tree division is determined based on whether the rightmost position of the division target block and the bottommost position of the division target block are not included in the area of the current image. In Article 8,
If the binary tree division is performed on the division target block in the second division step,
An image decoding method in which the splitting direction information indicating the splitting direction of the above binary tree split is not decoded from the bitstream but is derived as a predetermined value. In Article 8,
The division direction of the above binary tree division is,
If the rightmost position of the above-mentioned segmentation target block is not included in the area of the above-mentioned current image, it is in the vertical direction,
A horizontal image decoding method when the lowest position of the above-mentioned segmentation target block is not included in the area of the above-mentioned current image. A computer-readable recording medium storing a bitstream generated by a video encoding method,
The above image encoding method is,
A first segmentation step for dividing the current image into multiple blocks; and
A second segmentation step is included for segmenting a segmentation target block including a boundary of the current image among the plurality of blocks into a plurality of sub-blocks.
The second segmentation step is performed recursively using a sub-block including a boundary of the current image as a segmentation target block until there is no sub-block including a boundary of the current image among the sub-blocks.
The second division step selectively applies quad tree division or binary tree division based on the size and threshold of the division target block to divide the division target block, and if the quad tree division or binary tree division is performed on the division target block in the second division step, division information indicating whether the division target block is divided is not encoded.
The above quad tree division is performed based on the result of comparing the size of the division target block and the size of a block that can be divided into quad tree divisions.
The above binary tree division is performed based on the result of comparing the size of the division target block with the size of a block capable of binary tree division and the threshold value when the quad tree division is not performed.
A recording medium, characterized in that the threshold value is different from the size of a block capable of binary tree division, and is a value between the maximum size and the minimum size of a block capable of quad tree division.