JP2013223049A - Image decoding apparatus and image encoding apparatus - Google Patents

Abstract

Even when bi-prediction is not applicable, high encoding efficiency is achieved by creating various or more preferable merge candidates.
When bi-prediction is applicable, a merge candidate list is derived using a combined merge candidate derivation unit 1212D3, and when bi-prediction is not applicable, a corrected merge candidate derivation unit 1212D5 is provided. And a merge candidate derivation control unit 1212A for deriving a merge candidate list including one or more merge candidates.
[Selection] Figure 1

Description

  The present invention relates to an image decoding apparatus that decodes encoded data representing an image, and an image encoding apparatus that generates encoded data by encoding an image.

  In order to efficiently transmit or record a moving image, a moving image encoding device that generates encoded data by encoding the moving image, and a moving image that generates a decoded image by decoding the encoded data An image decoding device is used.

  As a specific moving picture encoding method, for example, H.264 is used. H.264 / MPEG-4. A method proposed by AVC and HEVC (High-Efficiency Video Coding) as a successor codec (Non-Patent Document 1) and the like can be mentioned.

  In such a moving image coding system, an image (picture) constituting a moving image is a slice obtained by dividing the image, a coding unit (Coding Unit) obtained by dividing the slice. And is managed by a hierarchical structure composed of blocks and partitions obtained by dividing an encoding unit, and is normally encoded / decoded block by block.

  In such a moving image coding method, a predicted image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and the predicted image is generated from the input image (original image). A prediction residual obtained by subtraction (sometimes referred to as “difference image” or “residual image”) is encoded. In addition, examples of the method for generating a predicted image include inter-screen prediction (inter prediction) and intra-screen prediction (intra prediction).

  In intra prediction, based on a locally decoded image in the same frame, predicted images in the frame are sequentially generated.

  For inter prediction, in a prediction unit in merge mode, a list of motion compensation parameter candidates (merge candidates) is generated, and motion compensation of a predicted image is performed using a motion compensation candidate selected by an index from the list. In prediction units other than the merge mode, a list of motion compensation parameter candidates (predicted motion vector candidates) is generated, and motion compensation parameters are derived from the motion compensation candidates selected by index from the list and the difference motion vector, and motion compensation is performed. I do.

`` High efficiency video coding (HEVC) text specification draft 6 (JCTVC-H1003_dK) '', Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 7th Meeting: Geneva, CH, 21-30 November, 2011 (released April 2, 2012)

  However, according to the merge candidate list generation process according to the prior art, various merge candidates can be created when bi-prediction is applicable, but bi-prediction is applicable when bi-prediction is not applicable. There is a problem that fewer types of merge candidates are generated than in some cases. That is, there is a problem that an appropriate merge candidate cannot be selected when bi-prediction is not applicable, and the improvement in coding efficiency due to merge candidate selection is smaller than when bi-prediction is applicable.

  The present invention has been made in view of the above problems, and its purpose is to achieve high coding efficiency by creating various or more preferable merge candidates even when bi-prediction is not applicable. An object of the present invention is to provide an image decoding apparatus and an image encoding apparatus that can be achieved.

  In order to solve the above-described problem, the image decoding apparatus according to the present invention uses either a single prediction that refers to one reference image or a bi-prediction that refers to two reference images. In an image decoding apparatus that restores using the inter-screen prediction prediction method, a bi-predictive merge candidate derivation unit that derives a bi-predictive merge candidate, a single-predictive merge candidate derivation unit that derives one or more uni-predictive merge candidates, A bi-prediction application determination unit that determines whether or not prediction is applicable, and a bi-prediction merge candidate derivation unit that derives a merge candidate list when bi-prediction is applicable. In some cases, the apparatus further comprises merge candidate list deriving means for deriving a merge candidate list including one or more merge candidates using the single prediction merge candidate deriving means.

  According to the above configuration, the bi-predictive merge candidate derivation means and the uni-predictive merge candidate derivation means are included. In addition, when bi-prediction is applicable, a merge candidate list is derived using the bi-prediction merge candidate derivation unit. On the other hand, when bi-prediction is not applicable, the single prediction merge candidate derivation unit is To derive a merge candidate list including one or more merge candidates.

  Thereby, even when bi-prediction is not applicable, various or more preferable merge candidates can be created. For this reason, it is possible to suppress the decrease in the number of merge candidates and improve the selection rate of skip CUs or merge PUs. As a result, the encoding efficiency can be improved.

  In the image decoding apparatus according to the present invention, the bi-predictive merge candidate derivation unit derives merge candidates based on motion vectors of two different merge candidates, and the uni-predictive merge candidate derivation unit includes a single merge It is preferable to derive merge candidates based on the candidates.

  According to the above configuration, by combining motion vectors of two different merge candidates or correcting a single merge candidate, various or more preferable merge candidates can be obtained even when bi-prediction is not applicable. Can be created.

  In the image decoding apparatus according to the present invention, the merge candidate list derivation unit uses the bi-prediction merge candidate derivation unit and bi-prediction merge candidate derivation unit when bi-prediction is applicable. If the bi-prediction is not applicable, the merge candidate list is derived without using the bi-predictive merge candidate deriving unit and without using the bi-predictive merge candidate deriving unit. It is preferable to do.

  According to the above configuration, when bi-prediction is applicable and there is a high possibility that the merge candidates are sufficient, the single prediction merge candidate derivation means is not used, so that the processing amount can be reduced. .

In the image decoding apparatus according to the present invention, the bi-prediction application determination unit is a slice type determination unit that determines a slice type including a target prediction unit,
Preferably, the slice type determination means determines that bi-prediction is applicable when the slice type is B slice, and determines that bi-prediction is not applicable when the slice type is P slice.

  According to the above configuration, when the slice type is a P slice having a smaller number of merge candidates compared to a B slice to which bi-prediction can be applied, the decrease in the number of merge candidates is suppressed, and the skip CU or merge PU selection rate is increased. Can be improved.

Moreover, in the image decoding apparatus according to the present invention, the bi-prediction application determination unit is a bi-prediction restriction determination unit that determines whether bi-prediction restriction is applied to the target prediction unit,
The bi-prediction restriction determination means determines that bi-prediction is applicable when the size of the target prediction unit is greater than or equal to a predetermined size, and applies bi-prediction when the size of the target prediction unit is smaller than the predetermined size. It is preferable to determine that it is impossible.

  According to the above configuration, even when the B slice can refer to two reference images, in the case where bi-prediction is limited, the decrease in the number of merge candidates is suppressed, and the skip CU or merge PU The selectivity can be improved.

  In the image decoding apparatus according to the present invention, it is preferable that the single prediction merge candidate derivation unit derives the single prediction merge candidate by determining a reference motion compensation parameter and adjusting the reference motion compensation parameter. .

  According to the above configuration, if a motion compensation parameter that is highly likely to be generated is used as the reference motion compensation parameter, a merge candidate with a high occurrence probability can be generated by performing adjustment with the reference motion compensation parameter as a reference. .

  In the image decoding device according to the present invention, the reference motion compensation parameter is a motion compensation parameter corresponding to a merge candidate to which a minimum index is assigned among merge candidates included in a merge candidate list. preferable.

  According to the above configuration, if there is a derived merge candidate, the merge candidate associated with the index indicating the merge candidate with the highest priority can be used as the reference motion compensation parameter. Further, if the derived merge candidate is reused as the reference motion compensation parameter, the processing amount can be reduced.

  In the image decoding apparatus according to the present invention, the adjustment of the reference motion compensation parameter is an adjustment by adding an offset to a motion vector associated with the reference image list L0 or the reference image list L1 in the reference motion compensation parameter. It is preferable.

  By appropriately adding an offset to the motion vector of the reference motion compensation parameter, various merge candidates can be generated by a relatively simple process.

  In the image decoding device according to the present invention, the adjustment of the reference motion compensation parameter is performed by using a reference image index associated with the reference image list L0 or the reference image list L1 in the reference motion compensation parameter as the reference image index. It is preferable that the adjustment be made to the minimum value among the different values.

  According to the above configuration, the coding efficiency can be improved when a reference image that is more likely to be used as a reference image is assigned a smaller index in the reference image list.

In the image decoding device according to the present invention, in the adjustment by adding the offset, the offset is selected from an offset list,
The offset list is preferably selected from a plurality of predetermined offset lists based on the direction or magnitude of the motion vector in the reference motion compensation parameter.

  In general, the larger the motion vector, the lower the possibility that a motion vector suitable for the target PU will be derived by adjusting the offset. This is based on the fact that when the motion vector is large, a larger number of offset candidates are required to make an appropriate adjustment, and conversely, a limited offset candidate is likely not to make an appropriate adjustment. .

  Therefore, when the number of offset candidates is limited and the absolute value of one component of the reference motion vector is larger than the other component, a large number of offset candidates to be adjusted in the direction of the component having the smaller absolute value It is preferable to include it in the offset list.

  In order to solve the above problems, an image encoding apparatus according to the present invention performs uni-prediction with reference to one reference image or bi-prediction with reference to two reference images for an image in a prediction unit. Bi-predictive merge candidate derivation means for deriving bi-predictive merge candidates and one or more uni-predictive merge candidates are derived in an image encoding apparatus that encodes information indicating which inter-frame prediction prediction method should be restored. A single prediction merge candidate derivation unit; a bi-prediction application determination unit that determines whether bi-prediction is applicable; and a bi-prediction merge candidate derivation unit that derives a merge candidate list when bi-prediction is applicable. Then, when bi-prediction is not applicable, a merge candidate list deriving unit for deriving a merge candidate list including one or more merge candidates using the single prediction merge candidate deriving unit is provided. And features.

  Thus, an image encoding device having a configuration corresponding to the image decoding device also falls within the scope of the present invention. In addition, the data structure of the encoded image data generated by the image encoding apparatus also falls within the scope of the present invention. According to the image encoding device and the data structure of the image encoded data configured as described above, it is possible to achieve the same effect as the image decoding device according to the present invention.

  The image decoding apparatus according to the present invention uses an inter-screen prediction prediction method for uni-prediction with reference to one reference image or bi-prediction with reference to two reference images. In the image decoding apparatus to be restored, bi-predictive merge candidate derivation means for deriving bi-predictive merge candidates, uni-predictive merge candidate derivation means for deriving one or more uni-predictive merge candidates, and bi-prediction for determining whether bi-prediction is applicable When the bi-prediction is applicable, the merge prediction list is derived using the bi-predictive merge candidate derivation unit when the bi-prediction is applicable, and when the bi-prediction is not applicable, the single-predictive merge candidate And a merge candidate list derivation unit that derives a merge candidate list including one or more merge candidates using the derivation unit.

  The image encoding apparatus according to the present invention uses any prediction method for inter-screen prediction, ie, uni-prediction with reference to one reference image or bi-prediction with reference to two reference images. In an image encoding apparatus that encodes information indicating whether to restore, a bi-predictive merge candidate derivation unit that derives a bi-predictive merge candidate, a single-predictive merge candidate derivation unit that derives one or more uni-predictive merge candidates, A bi-prediction application determination unit that determines whether or not prediction is applicable, and a bi-prediction merge candidate derivation unit that derives a merge candidate list when bi-prediction is applicable. In some cases, there is provided a merge candidate list deriving unit that derives a merge candidate list including one or more merge candidates using the single prediction merge candidate deriving unit.

  Therefore, even when bi-prediction cannot be applied, various or more preferable merge candidates can be created, and the encoding efficiency can be improved.

It is a functional block diagram which illustrates about the structure of the merge motion compensation parameter derivation | leading-out part contained in the moving image decoding apparatus which concerns on one Embodiment of this invention. It is a functional block diagram which shows the schematic structure of the said moving image decoding apparatus. It is a figure for demonstrating the structure of the coding data which concern on embodiment of this invention, Comprising: (a) has shown the sequence layer which prescribes | regulates sequence SEQ, (b) is the picture which prescribes | regulates picture PICT (C) shows a slice layer that defines a slice S, (d) shows a tree block layer that defines a tree block TBLK, and (e) shows 2 shows a CU layer that defines a coding unit (CU) included in the tree block TBLK. It is a functional block diagram shown about the structural example of PU information decoding part and decoding module contained in the said moving image decoding apparatus. It is a flowchart which shows operation | movement of the said merge motion compensation parameter derivation | leading-out part. It is a figure for demonstrating operation | movement of the spatial merge candidate derivation | leading-out part with which the said merge motion compensation parameter derivation | leading-out part is provided. It is a figure explaining operation | movement of the time merge candidate derivation | leading-out part with which the said merge motion compensation parameter derivation | leading-out part is provided. It is a figure which shows the example of a merge candidate combination list. It is a figure which shows operation | movement of the zero merge candidate derivation | leading-out part with which the said merge motion compensation parameter derivation | leading-out part is provided. It is a figure for demonstrating the position of the offset contained in an offset list. It is a figure which illustrates operation | movement of the correction | amendment merge candidate derivation | leading-out part with which the said merge motion compensation parameter derivation | leading-out part is provided. It is a flowchart shown about an example of the flow of CU decoding process (inter / skip CU) in a moving image decoding apparatus. It is a flowchart which shows another example of operation | movement of a merge motion compensation parameter derivation | leading-out part. It is a block diagram which shows the structure of the merge motion compensation parameter derivation | leading-out part which concerns on one modification. It is a flowchart which shows another example of operation | movement of a merge motion compensation parameter derivation | leading-out part. It is a functional block diagram shown about the structure of the moving image encoder which concerns on this embodiment. It is a block diagram which shows the structural example of the PU information generation part with which the said moving image encoder is provided. It is a flowchart shown about an example of the flow of CU encoding process (inter / skip CU) in the said moving image encoder. It is the figure shown about the structure of the transmitter which mounts the said moving image encoder, and the receiver which mounts the said moving image decoder. (A) shows a transmitting apparatus equipped with a moving picture coding apparatus, and (b) shows a receiving apparatus equipped with a moving picture decoding apparatus. It is the figure shown about the structure of the recording device which mounts the said moving image encoder, and the reproducing | regenerating apparatus which mounts the said moving image decoder. (A) shows a recording apparatus equipped with a moving picture coding apparatus, and (b) shows a reproduction apparatus equipped with a moving picture decoding apparatus.

  An embodiment of the present invention will be described with reference to FIGS. First, an overview of the moving picture decoding apparatus (image decoding apparatus) 1 and the moving picture encoding apparatus (image encoding apparatus) 2 will be described with reference to FIG. FIG. 2 is a functional block diagram showing a schematic configuration of the moving picture decoding apparatus 1.

  The moving picture decoding apparatus 1 and the moving picture encoding apparatus 2 shown in FIG. The technology adopted in the H.264 / MPEG-4 AVC standard and the technology proposed in HEVC (High-Efficiency Video Coding) which is a successor codec are implemented.

  The video encoding device 2 generates encoded data # 1 by entropy encoding a syntax value defined to be transmitted from the encoder to the decoder in these video encoding schemes. .

  As entropy coding methods, context adaptive variable length coding (CAVLC) and context adaptive binary arithmetic coding (CABAC) are known. ing.

  In encoding / decoding by CAVLC and CABAC, processing adapted to the context is performed. The context is an encoding / decoding situation (context), and is determined by past encoding / decoding results of related syntax. Examples of the related syntax include various syntaxes related to intra prediction and inter prediction, various syntaxes related to luminance (Luma) and chrominance (Chroma), and various syntaxes related to CU (Coding Unit coding unit) size. In CABAC, the binary position to be encoded / decoded in binary data (binary string) corresponding to the syntax may be used as the context.

  In CAVLC, various syntaxes are encoded by adaptively changing the VLC table used for encoding. On the other hand, in CABAC, binarization processing is performed on syntax that can take multiple values such as a prediction mode and a conversion coefficient, and binary data obtained by this binarization processing is adaptive according to the occurrence probability. Are arithmetically encoded. Specifically, multiple buffers that hold the occurrence probability of binary values (0 or 1) are prepared, one buffer is selected according to the context, and arithmetic coding is performed based on the occurrence probability recorded in the buffer I do. Further, by updating the occurrence probability of the buffer based on the binary value to be decoded / encoded, an appropriate occurrence probability can be maintained according to the context.

  The moving image decoding apparatus 1 receives encoded data # 1 obtained by encoding a moving image by the moving image encoding apparatus 2. The video decoding device 1 decodes the input encoded data # 1 and outputs the video # 2 to the outside. Prior to detailed description of the moving picture decoding apparatus 1, the configuration of the encoded data # 1 will be described below.

[Configuration of encoded data]
A configuration example of encoded data # 1 that is generated by the video encoding device 2 and decoded by the video decoding device 1 will be described with reference to FIG. The encoded data # 1 exemplarily includes a sequence and a plurality of pictures constituting the sequence.

  A hierarchical structure of data in the encoded data # 1 is shown in FIG. 3A to 3E respectively show a sequence layer that defines a sequence SEQ, a picture layer that defines a picture PICT, a slice layer that defines a slice S, and a tree block that defines a tree block TBLK. It is a figure which shows the CU layer which prescribes | regulates the coding unit (Coding Unit; CU) contained in a layer and tree block TBLK.

(Sequence layer)
In the sequence layer, a set of data referred to by the video decoding device 1 for decoding a sequence SEQ to be processed (hereinafter also referred to as a target sequence) is defined. As shown in FIG. 3A, the sequence SEQ includes a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), an adaptive parameter set APS (Adaptation Parameter Set), and pictures PICT1 to PICTNP ( The NP includes the total number of pictures included in the sequence SEQ) and supplemental enhancement information (SEI).

  In the sequence parameter set SPS, a set of encoding parameters referred to by the video decoding device 1 in order to decode the target sequence is defined. Details of the SPS will be described later.

  In the picture parameter set PPS, a set of encoding parameters referred to by the video decoding device 1 for decoding each picture in the target sequence is defined. For example, a quantization width reference value (pic_init_qp_minus26) used for picture decoding and a flag (weighted_pred_flag) indicating application of weighted prediction are included. A plurality of PPS may exist. In that case, one of a plurality of PPSs is selected from each picture in the target sequence.

  The adaptive parameter set APS defines a set of coding parameters that the video decoding device 1 refers to in order to decode each slice in the target sequence. Details of the APS will be described later. A plurality of APSs may exist. In that case, one of a plurality of APSs is selected from each slice in the target sequence.

(Picture layer)
In the picture layer, a set of data referred to by the video decoding device 1 for decoding a picture PICT to be processed (hereinafter also referred to as a target picture) is defined. As shown in FIG. 3B, the picture PICT includes a picture header PH and slices S1 to SNS (NS is the total number of slices included in the picture PICT).

  Hereinafter, when it is not necessary to distinguish each of the slices S1 to SNS, the subscripts may be omitted. The same applies to other data with subscripts included in encoded data # 1 described below.

  The picture header PH includes a coding parameter group that is referred to by the video decoding device 1 in order to determine a decoding method of the target picture. Note that the encoding parameter group is not necessarily included directly in the picture header PH, and may be included indirectly, for example, by including a reference (pic_parameter_set_id) to the picture parameter set PPS.

(Slice layer)
In the slice layer, a set of data referred to by the video decoding device 1 for decoding the slice S to be processed (also referred to as a target slice) is defined. As shown in FIG. 3C, the slice S includes a slice header SH and a sequence of tree blocks TBLK1 to TBLKNC (NC is the total number of tree blocks included in the slice S).

  The slice header SH includes an encoding parameter group that is referred to by the video decoding device 1 in order to determine a decoding method of the target slice. Slice type designation information (slice_type) for designating a slice type is an example of an encoding parameter included in the slice header SH.

  As slice types that can be specified by the slice type specification information, (1) I slice that uses only intra prediction at the time of encoding, (2) P slice that uses unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.

  Note that the slice header SH may include a reference to the picture parameter set PPS (pic_parameter_set_id) and a reference to the adaptive parameter set APS (aps_id) included in the sequence layer.

(Tree block layer)
In the tree block layer, a set of data referred to by the video decoding device 1 for decoding a processing target tree block TBLK (hereinafter also referred to as a target tree block) is defined. The tree block may be referred to as a coding tree block (CTB) or a maximum coding unit (LCU).

  The tree block TBLK includes a tree block header TBLKH and coding unit information CU1 to CUNL (NL is the total number of coding unit information included in the tree block TBLK). Here, first, a relationship between the tree block TBLK and the coding unit information CU will be described as follows.

  Tree block TBLK is divided into units for specifying a block size for each process of intra prediction or inter prediction and transformation.

  The unit of the tree block TBLK is divided by recursive quadtree division. The tree structure obtained by this recursive quadtree partitioning is hereinafter referred to as a coding tree.

  Hereinafter, a unit corresponding to a leaf which is a node at the end of the coding tree is referred to as a coding node. In addition, since the encoding node is a basic unit of the encoding process, hereinafter, the encoding node is also referred to as an encoding unit (CU).

  That is, the coding unit information CU1 to CUNL is information corresponding to each coding node (coding unit) obtained by recursively dividing the tree block TBLK into quadtrees.

  Also, the root of the coding tree is associated with the tree block TBLK. In other words, the tree block TBLK is associated with the highest node of the tree structure of the quadtree partition that recursively includes a plurality of encoding nodes.

  Note that the size of each coding node is half of the size of the coding node to which the coding node directly belongs (that is, the unit of the node one level higher than the coding node).

  The size of the tree block TBLK and the size that each coding node can take are the minimum coding node size designation information (log2_min_coding_block_size_minus3) and the maximum coding node included in the sequence parameter set SPS of the coded data # 1. And the difference (log2_diff_max_min_coding_block_size) of the layer depth of the minimum coding node. For example, when the size of the minimum coding node is 8 × 8 pixels and the difference in the layer depth between the maximum coding node and the minimum coding node is 3, the size of the tree block TBLK is 64 × 64 pixels. The size of the encoding node can take any of four sizes, namely, 64 × 64 pixels, 32 × 32 pixels, 16 × 16 pixels, and 8 × 8 pixels.

(Tree block header)
The tree block header TBLKH includes an encoding parameter referred to by the video decoding device 1 in order to determine a decoding method of the target tree block. Specifically, as shown in FIG. 3D, tree block division information SP_TBLK that specifies a division pattern of the target tree block into each CU, and a quantization parameter difference that specifies the size of the quantization step Δqp (qp_delta) is included.

  The tree block division information SP_TBLK is information representing a coding tree for dividing the tree block. Specifically, the shape and size of each CU included in the target tree block, and the position in the target tree block Is information to specify.

  Note that the tree block division information SP_TBLK may not explicitly include the shape or size of the CU. For example, the tree block division information SP_TBLK may be a set of flags (split_coding_unit_flag) indicating whether or not the entire target tree block or a partial area of the tree block is divided into four. In that case, the shape and size of each CU can be specified by using the shape and size of the tree block together.

  The quantization parameter difference Δqp is a difference qp−qp ′ between the quantization parameter qp in the target tree block and the quantization parameter qp ′ in the tree block encoded immediately before the target tree block.

(CU layer)
In the CU layer, a set of data referred to by the video decoding device 1 for decoding a CU to be processed (hereinafter also referred to as a target CU) is defined.

  Here, before describing specific contents of data included in the coding unit information CU, a tree structure of data included in the CU will be described. The encoding node is a node at the root of a prediction tree (PT) and a transform tree (TT). The prediction tree and the conversion tree are described as follows.

  In the prediction tree, the encoding node is divided into one or a plurality of prediction blocks, and the position and size of each prediction block are defined. In other words, the prediction block is one or a plurality of non-overlapping areas constituting the encoding node. The prediction tree includes one or a plurality of prediction blocks obtained by the above division.

  Prediction processing is performed for each prediction block. Hereinafter, a prediction block that is a unit of prediction is also referred to as a prediction unit (PU).

  Broadly speaking, there are two types of division in the prediction tree: intra prediction and inter prediction.

  In the case of intra prediction, there are 2N × 2N (the same size as the encoding node) and N × N division methods.

  In the case of inter prediction, there are 2N × 2N (the same size as the encoding node), 2N × N, N × 2N, N × N, and the like.

  In the transform tree, the encoding node is divided into one or a plurality of transform blocks, and the position and size of each transform block are defined. In other words, the transform block is one or a plurality of non-overlapping areas constituting the encoding node. The conversion tree includes one or a plurality of conversion blocks obtained by the above division.

  The conversion process is performed for each conversion block. Hereinafter, a transform block that is a unit of transform is also referred to as a transform unit (TU).

(Data structure of encoding unit information)
Next, specific contents of data included in the coding unit information CU will be described with reference to FIG. As shown in FIG. 3E, the coding unit information CU specifically includes a skip mode flag SKIP, CU prediction type information Pred_type, PT information PTI, and TT information TTI.

[Skip flag]
The skip flag SKIP is a flag indicating whether or not the skip mode is applied to the target CU. When the value of the skip flag SKIP is 1, that is, when the skip mode is applied to the target CU, the code The PT information PTI in the unit information CU is omitted. Note that the skip flag SKIP is omitted for the I slice.

[CU prediction type information]
The CU prediction type information Pred_type includes CU prediction method information PredMode and PU partition type information PartMode.

  The CU prediction method information PredMode specifies whether intra prediction (intra CU) or inter prediction (inter CU) is used as a predicted image generation method for each PU included in the target CU. Hereinafter, the types of skip, intra prediction, and inter prediction in the target CU are referred to as a CU prediction mode.

  The PU partition type information PartMode designates a PU partition type that is a pattern of partitioning the target coding unit (CU) into each PU. Hereinafter, dividing the target coding unit (CU) into each PU according to the PU division type in this way is referred to as PU division.

  For example, the PU partition type information PartMode may be an index indicating the type of PU partition pattern, and the shape, size, and position of each PU included in the target prediction tree may be It may be specified.

  The selectable PU partition types differ depending on the CU prediction method and the CU size. Furthermore, the PU partition types that can be selected are different in each case of inter prediction and intra prediction. Details of the PU partition type will be described later.

  If the slice is not an I slice, the value of the PU partition type information PartMode and the value of the PU partition type information PartMode are indices that specify combinations of tree block partition (partition), prediction method, and CU split (split) methods. It may be specified by (cu_split_pred_part_mode).

[PT information]
The PT information PTI is information related to the PT included in the target CU. In other words, the PT information PTI is a set of information on each of one or more PUs included in the PT. As described above, since the generation of the predicted image is performed in units of PUs, the PT information PTI is referred to when the moving image decoding apparatus 1 generates a predicted image. As shown in FIG. 3E, the PT information PTI includes PU information PUI1 to PUINP (NP is the total number of PUs included in the target PT) including prediction information in each PU.

  The prediction information PUI includes intra prediction information or inter prediction information depending on which prediction method is specified by the prediction type information Pred_mode. Hereinafter, a PU to which intra prediction is applied is also referred to as an intra PU, and a PU to which inter prediction is applied is also referred to as an inter PU.

  The inter prediction information includes a motion compensation parameter that is referred to when the moving image decoding apparatus 1 generates an inter predicted image by inter prediction.

  As motion compensation parameters, for example, a merge flag (merge_flag), a merge index (merge_idx), an estimated motion vector index (mvp_idx), a reference image index (ref_idx), an inter prediction flag (inter_pred_flag), and a motion vector residual (mvd) Is mentioned.

  The intra prediction information includes an encoding parameter that is referred to when the video decoding device 1 generates an intra predicted image by intra prediction.

  Examples of intra prediction parameters include an estimated prediction mode flag, an estimated prediction mode index, and a residual prediction mode index.

  In the intra prediction information, a PCM mode flag indicating whether to use the PCM mode may be encoded. When the PCM mode flag is encoded and the PCM mode flag indicates that the PCM mode is used, the prediction process (intra), the conversion process, and the entropy encoding process are omitted. .

[TT information]
The TT information TTI is information regarding the TT included in the CU. In other words, the TT information TTI is a set of information regarding each of one or a plurality of TUs included in the TT, and is referred to when the moving image decoding apparatus 1 decodes residual data. Hereinafter, a TU may be referred to as a block.

  As shown in (e) of FIG. 3, the TT information TTI includes TT division information SP_TU that designates a division pattern for each transform block of the target CU, and TU information TUI1 to TUINT (NT is included in the target CU). The total number of blocks).

  Specifically, the TT division information SP_TU is information for determining the shape and size of each TU included in the target CU and the position within the target CU. For example, the TT division information SP_TU can be realized from information (split_transform_flag) indicating whether or not the target node is to be divided and information (trafoDepth) indicating the depth of the division.

  For example, when the size of the CU is 64 × 64, each TU obtained by the division can take a size from 32 × 32 pixels to 4 × 4 pixels.

  The TU information TUI1 to TUINT is individual information regarding each of one or more TUs included in the TT. For example, the TU information TUI includes a quantized prediction residual.

  Each quantization prediction residual is encoded data generated by the moving image encoding device 2 performing the following processes 1 to 3 on a target block that is a processing target block.

Process 1: DCT transform (Discrete Cosine Transform) of the prediction residual obtained by subtracting the prediction image from the encoding target image;
Process 2: Quantize the transform coefficient obtained in Process 1;
Process 3: Variable length coding is performed on the transform coefficient quantized in Process 2;
The quantization parameter qp described above represents the magnitude of the quantization step QP used when the moving image coding apparatus 2 quantizes the transform coefficient (QP = 2 ^{qp / 6} ).

[Video decoding device]
Below, the structure of the moving image decoding apparatus 1 which concerns on this embodiment is demonstrated with reference to FIGS.

(Outline of video decoding device)
The video decoding device 1 generates a predicted image for each PU, generates a decoded image # 2 by adding the generated predicted image and a prediction residual decoded from the encoded data # 1, and generates The decoded image # 2 is output to the outside.

  Here, the generation of the predicted image is performed with reference to an encoding parameter obtained by decoding the encoded data # 1. An encoding parameter is a parameter referred in order to generate a prediction image. In addition to prediction parameters such as a motion vector referred to in inter-screen prediction and a prediction mode referred to in intra-screen prediction, the encoding parameters include PU size and shape, block size and shape, and original image and Residual data with the predicted image is included. Hereinafter, a set of all information excluding the residual data among the information included in the encoding parameter is referred to as side information.

  In the following, a picture (frame), a slice, a tree block, a block, and a PU to be decoded are referred to as a target picture, a target slice, a target tree block, a target block, and a target PU, respectively. .

  Note that the size of the tree block is, for example, 64 × 64 pixels, and the size of the PU is, for example, 64 × 64 pixels, 32 × 32 pixels, 16 × 16 pixels, 8 × 8 pixels, 4 × 4 pixels, or the like. . However, these sizes are merely examples, and the sizes of the tree block and PU may be other than the sizes shown above.

(Configuration of video decoding device)
Referring to FIG. 2 again, the schematic configuration of the moving picture decoding apparatus 1 will be described as follows. FIG. 2 is a functional block diagram showing a schematic configuration of the moving picture decoding apparatus 1.

  As shown in FIG. 2, the moving picture decoding apparatus 1 includes a decoding module 10, a CU information decoding unit 11, a PU information decoding unit 12, a TU information decoding unit 13, a predicted image generation unit 14, an inverse quantization / inverse conversion unit 15, A frame memory 16 and an adder 17 are provided.

[Decryption module]
The decoding module 10 performs a decoding process for decoding a syntax value from binary. More specifically, the decoding module 10 decodes a syntax value encoded by an entropy encoding method such as CABAC and CAVLC based on encoded data and a syntax type supplied from a supplier, Returns the decrypted syntax value to the supplier.

  In the example shown below, the sources of encoded data and syntax type are the CU information decoding unit 11, the PU information decoding unit 12, and the TU information decoding unit 13.

  As an example of the decoding process in the decoding module 10, a case where a binary (bit string) of encoded data and a syntax type “split_coding_unit_flag” are supplied from the CU information decoding unit 11 to the decoding module 10 will be described next. It is as follows. That is, in this case, the decoding module 10 derives the syntax value from the binary with reference to the association between the bit string and the syntax value regarding “split_coding_unit_flag”, and the derived syntax value is the CU information decoding unit. Return to 11.

[CU information decoding unit]
The CU information decoding unit 11 uses the decoding module 10 to perform decoding processing at the tree block and CU level on the encoded data # 1 for one frame input from the moving image encoding device 2. Specifically, the CU information decoding unit 11 decodes the encoded data # 1 according to the following procedure.

  First, the CU information decoding unit 11 refers to various headers included in the encoded data # 1, and sequentially separates the encoded data # 1 into slices and tree blocks.

  Here, the various headers include (1) information about the method of dividing the target picture into slices, and (2) information about the size, shape, and position of the tree block belonging to the target slice. .

  Then, the CU information decoding unit 11 refers to the tree block division information SP_TBLK included in the tree block header TBLKH, and divides the target tree block into CUs.

  Next, the CU information decoding unit 11 acquires coding unit information (hereinafter referred to as CU information) corresponding to the CU obtained by the division. The CU information decoding unit 11 performs the decoding process of the CU information corresponding to the target CU, with each CU included in the tree block as the target CU in order.

  That is, the CU information decoding unit 11 demultiplexes the TT information TTI related to the conversion tree obtained for the target CU and the PT information PTI related to the prediction tree obtained for the target CU.

  The TT information TTI includes the TU information TUI corresponding to the TU included in the conversion tree as described above. Further, as described above, the PT information PTI includes the PU information PUI corresponding to the PU included in the target prediction tree.

  The CU information decoding unit 11 supplies the PT information PTI obtained for the target CU to the PU information decoding unit 12. Further, the CU information decoding unit 11 supplies the TT information TTI obtained for the target CU to the TU information decoding unit 13.

[PU information decoding unit]
The PU information decoding unit 12 uses the decoding module 10 to perform decoding processing at the PU level for the PT information PTI supplied from the CU information decoding unit 11. Specifically, the PU information decoding unit 12 decodes the PT information PTI by the following procedure.

  The PU information decoding unit 12 refers to the PU partition type information Part_type to determine the PU partition type in the target prediction tree. Subsequently, the PU information decoding unit 12 performs a decoding process of PU information corresponding to the target PU, with each PU included in the target prediction tree as a target PU in order.

  That is, the PU information decoding unit 12 performs a decoding process on each parameter used for generating a predicted image from PU information corresponding to the target PU.

  The PU information decoding unit 12 supplies the PU information decoded for the target PU to the predicted image generation unit 14.

[TU information decoding unit]
The TU information decoding unit 13 uses the decoding module 10 to perform decoding processing at the TU level for the TT information TTI supplied from the CU information decoding unit 11. Specifically, the TU information decoding unit 13 decodes the TT information TTI by the following procedure.

  The TU information decoding unit 13 divides the target conversion tree into nodes or TUs with reference to the TT division information SP_TU. Note that the TU information decoding unit 13 recursively performs TU division processing if it is specified that further division is performed for the target node.

  When the division process ends, the TU information decoding unit 13 performs the decoding process of the TU information corresponding to the target TU, with each TU included in the target prediction tree as the target TU in order.

  That is, the TU information decoding unit 13 performs a decoding process on each parameter used for restoring the transform coefficient from the TU information corresponding to the target TU.

  The TU information decoding unit 13 supplies the TU information decoded for the target TU to the inverse quantization / inverse transform unit 15.

[Predicted image generator]
The predicted image generation unit 14 generates a predicted image based on the PT information PTI for each PU included in the target CU. Specifically, the prediction image generation unit 14 performs intra prediction or inter prediction for each target PU included in the target prediction tree according to the parameters included in the PU information PUI corresponding to the target PU, thereby generating a decoded image. A predicted image Pred is generated from a certain local decoded image P ′. The predicted image generation unit 14 supplies the generated predicted image Pred to the adder 17.

  A method in which the predicted image generation unit 14 generates a predicted image of a PU included in the target CU based on motion compensation prediction parameters (motion vector, reference image index, inter prediction flag) is as follows.

  When the inter prediction flag indicates single prediction, the predicted image generation unit 14 generates a predicted image corresponding to the decoded image located at the location indicated by the motion vector of the reference image indicated by the reference image index.

  On the other hand, when the inter prediction flag indicates bi-prediction, the predicted image generation unit 14 generates a predicted image by motion compensation for each of the two sets of reference image indexes and motion vectors, and calculates an average. Alternatively, the final predicted image is generated by weighting and adding each predicted image based on the display time interval between the target picture and each reference image.

[Inverse quantization / inverse transform unit]
The inverse quantization / inverse transform unit 15 performs an inverse quantization / inverse transform process on each TU included in the target CU based on the TT information TTI. Specifically, the inverse quantization / inverse transform unit 15 performs inverse quantization and inverse orthogonal transform on the quantization prediction residual included in the TU information TUI corresponding to the target TU for each target TU included in the target conversion tree. By doing so, the prediction residual D for each pixel is restored. Here, the orthogonal transform refers to an orthogonal transform from the pixel region to the frequency region. Therefore, the inverse orthogonal transform is a transform from the frequency domain to the pixel domain. Examples of inverse orthogonal transform include inverse DCT transform (Inverse Discrete Cosine Transform), inverse DST transform (Inverse Discrete Sine Transform), and the like. The inverse quantization / inverse transform unit 15 supplies the restored prediction residual D to the adder 17.

[Frame memory]
Decoded decoded images P are sequentially recorded in the frame memory 16 together with parameters used for decoding the decoded images P. In the frame memory 16, at the time of decoding the target tree block, decoded images corresponding to all tree blocks decoded before the target tree block (for example, all tree blocks preceding in the raster scan order) are stored. It is recorded. Examples of the decoding parameters recorded in the frame memory 16 include CU prediction method information PredMode.

[Adder]
The adder 17 adds the predicted image Pred supplied from the predicted image generation unit 14 and the prediction residual D supplied from the inverse quantization / inverse transform unit 15 to thereby obtain the decoded image P for the target CU. Generate.

  In the video decoding device 1, the encoded data for one frame input to the video decoding device 1 at the time when the decoded image generation processing for each tree block is completed for all tree blocks in the image. Decoded image # 2 corresponding to # 1 is output to the outside.

  Below, the structure of PU information decoding part 12 is demonstrated in detail with the structure of the decoding module 10 corresponding to each structure.

(1) Details of PU Information Decoding Unit Next, configuration examples of the PU information decoding unit 12 and the decoding module 10 will be described with reference to FIG. FIG. 4 is a functional block diagram illustrating the configuration for decoding motion compensation parameters, that is, the configurations of the PU information decoding unit 12 and the decoding module 10 in the video decoding device 1.

  Hereinafter, the configuration of each unit will be described in the order of the PU information decoding unit 12 and the decoding module 10.

(PU information decoding unit)
As illustrated in FIG. 4, the PU information decoding unit 12 includes a motion compensation parameter derivation unit 121. The motion compensation parameter derivation unit 121 includes a basic motion compensation parameter derivation unit 1211, a merge motion compensation parameter derivation unit (merge motion compensation parameter derivation unit) 1212, a motion compensation method selection unit 1213, and a switch 1214.

  The motion compensation parameter deriving unit 121 derives a motion compensation parameter based on the input PT setting information PTI and various syntax values supplied from the decoding module, and outputs the motion compensation parameter as prediction information PUI.

  In the motion compensation prediction, two reference image lists, that is, a reference image list L0 and a reference image list L1 are used. The motion compensation parameter includes a reference image list use flag (predFlag), a reference image index (refIdx), and a motion vector (mv) corresponding to each reference image list. Hereinafter, the reference image list use flag corresponding to the reference image list LX (LX is L0 or L1) is referred to as predFlagLX, the reference image index is referred to as refIdxLX, and the motion vector is referred to as mvLX.

  The procedure for deriving the motion compensation parameter corresponding to one PU (target PU) in the CU in the motion compensation parameter deriving unit 121 is as follows.

  First, the motion compensation method selection unit 1213 acquires the skip flag (skip_flag) of the target CU from the decoding module, and determines whether the target CU is a skip CU. If the target CU is a skip CU, it selects that the motion compensation method is merge motion compensation, and outputs it to the switch 1214. When the target CU is a non-skip CU, the motion compensation method selection unit 1213 acquires the merge flag (merge_flag) of the target PU from the decoding module. If the target PU is a merge PU, the motion compensation method is merge motion compensation. If it is selected, it is output to the switch 1214, and if it is not (the target PU is a non-merged PU), it is selected that the motion compensation method is basic motion compensation and output to the switch 1214.

  When the switch 1214 specifies from the motion compensation method selection unit 1213 that the motion compensation method is merge motion compensation, the merge motion compensation parameter derivation unit 1212 is selected. The merge motion compensation parameter derivation unit 1212 acquires the merge index (merge_idx) of the target PU from the decoding module, derives a motion compensation parameter corresponding to the merge index, and outputs it as prediction information PUI. Details of the configuration and operation of the merge motion compensation parameter deriving unit 1212 will be described later.

  When the switch 1214 specifies from the motion compensation method selection unit 1213 that the motion compensation method is basic motion compensation, the basic motion compensation parameter derivation unit 1211 is selected. The basic motion compensation parameter derivation unit 1211 obtains the motion vector residual (mvd), the estimated motion vector index (mvp_idx), the inter prediction flag (inter_pred_flag), and the reference image index (refIdx) of the target PU from the decoding module. The basic motion compensation parameter deriving unit 1211 determines a reference image list use flag for each of the reference image list L0 and the reference image list L1 based on the value of the inter prediction flag. Subsequently, when the corresponding reference image list use flag indicates that the reference image is used, an estimated motion vector is derived based on the value of the estimated motion vector index, and the motion vector residual and the estimated motion are A motion vector is derived based on the vector. The derived motion vector is output as prediction information PUI together with the reference image list use flag and the reference image index.

(Decryption module)
As shown in FIG. 4, the decoding module 10 includes a skip information decoding unit 1021 and a motion information decoding unit 1022.
The skip information decoding unit 1021 decodes the syntax value of the syntax element related to the skip information from the binary included in the encoded data according to the encoded data and syntax type supplied from the motion compensation parameter deriving unit 121. . The syntax element decoded by the skip information decoding unit 1021 is a skip flag (skip_flag).
The motion information decoding unit 1022 decodes the syntax value of the syntax element related to the motion information from the binary included in the encoded data according to the encoded data and the syntax type supplied from the motion compensation parameter deriving unit 121. . The syntax elements decoded by the motion information decoding unit 1022 are merge flag (merge_flag), merge index (merge_idx), estimated motion vector index (mvp_idx), reference image index (ref_idx), inter prediction flag (inter_pred_flag), and motion vector remaining. Difference (mvd).

(Merge motion compensation parameter derivation unit)
Next, the configuration and operation of the merge motion compensation parameter deriving unit 1212 will be described in detail with reference to FIG. FIG. 1 is a functional block diagram illustrating the configuration of the merge motion compensation parameter derivation unit 1212.

  As shown in FIG. 1, the merge motion compensation parameter derivation unit 1212 includes a merge candidate derivation control unit 1212A, a merge candidate storage unit 1212B, a merge candidate selection unit 1212C, a merge candidate derivation unit (merge candidate derivation unit) 1212D, and a slice. It is the structure provided with the type determination part 1212E. The merge candidate derivation unit 1212D includes a spatial merge candidate derivation unit 1212D1, a temporal merge candidate derivation unit 1212D2, a merge merge candidate derivation unit 1212D3, a zero merge candidate derivation unit 1212D4, and a correction merge candidate derivation unit 1212D5. Although not shown in FIG. 1, the adjacent merging candidate derivation unit 1212A and the temporal merging candidate derivation unit 1212D2 have decoded CU and PU encoding parameters stored in the frame memory 16, in particular, motion in units of PUs. Compensation parameters are provided.

  In the following, when referring to the spatial merge candidate derivation unit 1212D1, the temporal merge candidate derivation unit 1212D2, the combined merge candidate derivation unit 1212D3, the zero merge candidate derivation unit 1212D4, and the correction merge candidate derivation unit 1212D5, Collectively referred to as “candidate deriving unit”.

(Merge candidate derivation control unit)
The merge candidate derivation control unit 1212A controls each merge candidate derivation unit, derives a predetermined number (merge candidate derivation number) of merge candidates, and stores them in the merge candidate storage unit 1212B. As the merge candidate derivation number, for example, the value of merge_idx + 1 is normally used. Note that an arbitrary integer greater than or equal to the value of merge_idx + 1 may be used as the merge candidate derivation number. For example, a value obtained by adding 1 to the maximum value of merge_idx may be used as MRG_MAX_NUM_CANDS as the merge candidate derivation number.

(Merge candidate derivation unit)
The merge candidate derivation unit 1212D derives and outputs merge candidates by the designated derivation method. The detailed operation of each merge candidate derivation unit selected based on the designated derivation method will be described later. The derived merge candidate is illustratively composed of a reference image list use flag (predFlagLX), a reference image index (refIdxLX), and a motion vector (mvLX) for the reference image list LX. Here, LX is L0 or L1.

(Merge candidate storage)
The merge candidate storage unit 1212B stores a plurality of merge candidates. Merge candidates are recorded as an ordered list (merge candidate list).

(Merge candidate selection part)
The merge candidate selection unit 1212C selects a merge candidate specified by the merge index, reads the merge candidate selected from the merge candidate storage unit 1212B, and outputs the merge candidate as prediction information PUI.

(Slice type determination unit)
The slice type determination unit 1212E determines the slice type of the slice including the target PU in response to the request, and outputs the result.

(Details of merge motion compensation parameter deriving unit 1212)
(Flow diagram)
FIG. 5 is a flowchart showing the operation of the merge motion compensation parameter derivation unit 1212. First, the merge candidate S0 to merge candidate S2 are derived in the spatial merge candidate derivation unit 1212D1 (S101). Subsequently, the merge candidate T is derived in the temporal merge candidate deriving unit 1212D2 (S102). The slice type determination unit 1212E determines the slice type of the slice including the target PU (S103). When the slice type is a B slice (YES in S103), the merge candidate C deriving unit 1212D3 derives a merge candidate C (S104), and then proceeds to step S106. On the other hand, if the slice type is not B slice (NO in S103), the merge candidate R is derived in the corrected merge candidate deriving unit 1215D5 (S105), and then the process proceeds to step S106. Finally, the merge candidate Z is derived in the zero merge candidate deriving unit 1215D4 (S106).

(Spatial merge candidate derivation unit)
FIG. 6 is a diagram for explaining the operation of the spatial merge candidate derivation unit 1212D1. FIG. 6 shows the positional relationship between the target PU and adjacent blocks A0, A1, B0, B1, and B2. In general, the spatial merge candidate derivation unit 1212D1 outputs the motion compensation parameters in the adjacent blocks as merge candidates. Assume that the derivation order is A1, B1, B0, A0, and B2, for example. The derived merge candidates are stored in the merge candidate storage unit 1212B. More precisely, the merge candidates are added to the end of the merge candidate list stored in the merge candidate storage unit 1212B in the derived order. The position of each adjacent block can be expressed as follows, where the upper left coordinates of the PU are (xP, yP) and the PU sizes are nPSW and nPSH.
A0: (xP-1, yP + nPSH)
A1: (xP-1, yP + nPSH-1)
B0: (xP + nPSW, yP-1)
B1: (xP + nPSW-1, yP-1)
B2: (xP-1, yP-1)
Note that when any of the following conditions is satisfied, a merge candidate corresponding to the position N (N is any one of A0, A1, B0, B1, or B2) is not derived.
The block at position N is not available (not available).
-The block at position N is intra-coded.
When N is B2 and all merge candidates corresponding to positions A0, A1, B0, and B1 are derived.
When the PU partition type is 2N × N or N × 2N, and the PU index is 1, and the PU of block N and index 0 has the same motion compensation parameter.
When N is B0 and block N and block B1 have the same motion compensation parameter.
When N is A0 and block N and block A1 have the same motion compensation parameter.
If N is B2 and block N has the same motion compensation parameters as either block A1 or block B1.

  Here, a case where a certain block is not available is a case where the block is outside the screen, outside the slice, or undecoded. In addition, two blocks having the same motion compensation parameter means that the reference image list use flag, the reference image index, and the motion vector are all equal for both the reference image lists L0 and L1.

(Time merge candidate derivation unit)
FIG. 7 is a diagram for explaining the operation of the time merge candidate derivation unit 1212D2. Referring to (a) of FIG. 7, the temporal merging candidate is roughly a reference image index that occupies the same spatial position as the target PU in the current picture when the current picture is currPic. It is derived by copying the motion compensation parameter of the PU on the reference image specified by refIdxL0 or the PU on the reference image specified by the reference image index refIdxL1. A method for deriving the reference index number refIdxL0 and the reference index number refIdxL1 will be described with reference to FIG. The reference index number refIdxLX (where X is 0, 1 or C) is obtained as follows using the reference pictures refIdxLXA, refIdxLXB, and refIdxLXC of the adjacent PU, A, B, and C blocks of the target PU.
(1) If refIdxLXA = refIdxLXB = refIdxLXC,
When refIdxLXA = -1, refIdxLX = 0
Otherwise, refIdxLX = refIdxLXA
(2) When refIdxLXA = refIdxLXB
When refIdxLXA = -1, refIdxLX = refIdxLXC Otherwise, refIdxLX = refIdxLXA
(3) If refIdxLXB = refIdxLXC,
When refIdxLXB = -1, refIdxLX = refIdxLXA
Otherwise, refIdxLX = refIdxLXB
(4) When refIdxLXA = refIdxLXC
When refIdxLXA = -1, refIdxLX = refIdxLXB
Otherwise, refIdxLX = refIdxLXA
(5) When refIdxLXA = -1,
refIdxLX = min (refIdxLXB, refIdxLXC)
(6) When refIdxLXB = -1,
refIdxLX = min (refIdxLXA, refIdxLXC)
(7) When refIdxLXC = -1,
refIdxLX = min (refIdxLXA, refIdxLXB)
(8) In other cases,
refIdxLX = min (refIdxLXA, refIdxLXB, refIdxLXC)
Here, min is a function that takes a minimum value.
The coordinates of the blocks A and B are as follows.
A: (xP-1, yP + nPSH-1)
B: (xP + nPSW-1, yP-1)
The coordinates of the block C are any of the following C0, C1, and C2. If the PU corresponding to each position is available and other than intra, the refIdxLX of the PU at that position is set as refIdxLXC.
C0: (xP + nPSW-1, yP-1)
C1: (xP-1, yP + nPSH)
C2: (xP-1, yP-1)
When refIdxL0 and refIdxL1 are derived as described above, the motion compensation parameter of the position of the reference picture (xP + nPSW, yP + nPSH) indicated by refIdxL0 is used to determine the L0 motion vector, and the reference indicated by refIdxL1 A temporal merge candidate is derived by determining the L1 motion vector using the motion compensation parameter of the picture position (xP + nPSW, yP + nPSH). That is, the motion vectors mvLXCol [0] and mvLXCol [0] for each reference picture list LX (X = 0, X = 1 or X = C) are calculated from the reference picture indicated by the LX list and refIdxLX. Specifically, if the PU at the position of the reference picture (xP + nPSW, yP + nPSH) indicated by refIdxLX is unavailable or in the intra prediction mode, the motion vector mvLXCol of the temporal merge candidate LX [0], mvLXCol [1] is set to 0. Otherwise, that is, when PredFlagL0 of the PU is 0, the L1 motion vector MvL1 of the PU is used as the temporal merge candidate LX motion vectors mvLXCol [0] and mvLXCol [1]. In other cases, the L0 motion vector MvL0 of the PU is used as the temporal merge candidate LX motion vectors mvLXCol [0] and mvLXCol [1].

  Subsequently, the motion vector mvLXCol is scaled using the POC (Picture Order Count) of the current frame and the POC of the reference picture to obtain a final temporal merge candidate. The derived time merge candidates are stored in the merge candidate storage unit 1212B.

(Join merge candidate derivation unit)
In general, the merge merge candidate derivation unit 1212D4 derives merge merge candidates by combining motion vectors of two different derived merge candidates already derived and stored in the merge candidate storage unit 1212B.

  The merge merge candidate derivation unit 1212D4 increases the merge merge candidate count combCnt from 0, and merge merges until the number of elements in the merge candidate list matches the merge candidate derivation number, or until combCnt exceeds the maximum value (5). Candidates are derived and added to the end of the merge candidate list.

  The procedure for deriving the merge merge candidate corresponding to the specific merge merge candidate count combCnt is as follows.

First, the merge candidate combination list is referred to with combCnt as an index (combIdx), and two merge candidates used for the combination, that is, indexes l0CandIdx and L1CandIdx indicating the positions of the L0 merge candidate and the L1 merge candidate on the merge candidate list, respectively To derive. An example of the merge candidate combination list is shown in FIG. The merge merge candidate is generated by copying the motion compensation parameters of the L0 merge candidate to the reference image list L0 and the L1 merge candidate to the reference image list L1. Note that when any of the following conditions is satisfied, a merge merge candidate corresponding to combCnt is not derived.
-L0 reference image list use flag of L0 merge candidate is 0
-L1 reference image list use flag of L1 merge candidate is 0
The motion vectors or reference images of the L0 merge candidate and the L1 merge candidate match. The combined merge candidate is derived by the above procedure. In the motion compensation parameter corresponding to the merge merge candidate, both the L0 and L1 reference image list use flags are 1. That is, the merge merge candidate is a merge candidate that performs bi-prediction. Thus, in situations where bi-prediction is not applicable (eg, PUs in P slices), merge merge candidates are not included in the merge candidate list.

(Zero merge candidate derivation unit)
FIG. 9 is a diagram illustrating the operation of the zero merge candidate derivation unit 1212D4. If the number of merge candidates in the merge candidate storage unit 1212B has reached the number of merge candidate derivations, the zero merge candidate derivation unit 1212D4 does not perform processing (zero merge candidates are not derived). On the other hand, if the number of merge candidates has not reached the number of merge candidate derivations, the zero merge candidate derivation unit 1212D4 generates a merge candidate having a zero vector until the number of merge candidates reaches the number of merge candidate derivations to generate a merge candidate list. Add to That is, the index of the merge candidate to be referenced is mvL0zeroCand _m , and the L0 motion vectors (mvL0zeroCand _m [0], mvL0zeroCand _m [1]) and L1 motion vectors (mvL1zeroCand _m [0], mvL1zeroCand _m [1]) are both A candidate such as 0 is derived. Here, the index zeroCand _m uses a value obtained by adding 1 to the value of the last index of the merge candidate list already derived. m is an index starting from 0, and is incremented by 1 when a zero merge candidate is added to the merge candidate list.

(Correction merge candidate derivation unit)
The correction merge candidate derivation unit 1212D5 generates a merge candidate based on a motion parameter obtained by adjusting a predetermined motion compensation parameter (reference motion compensation parameter). In the present embodiment, the motion compensation parameter of the merge candidate recorded in the merge candidate storage unit 1212B is used as the reference motion compensation parameter. More specifically, in the merge candidate list recorded in the merge candidate storage unit 1212B, the merge candidate assigned to the smallest index is set as the reference motion compensation parameter. In other words, the merge candidate associated with the index indicating the merge candidate with the highest priority in the merge candidate list is set as the reference motion compensation parameter.

  The index of the merge candidate is included in the encoded data in the form of a syntax element merge_idx. In general, merge_idx is binarized so that a small value is associated with a shorter code string. For example, the value of merge_idx is binarized by a truncated unary code whose maximum value is the maximum number of merge candidates. Therefore, a merge candidate having a relatively high selection probability is associated with the minimum merge_idx, that is, merge_idx having a value of 0. Therefore, by using the merge candidate assigned to the smallest index as the reference motion compensation parameter, a corrected merge candidate having a higher selection probability is derived compared to the case where the merge candidate assigned to another index is used. Therefore, encoding efficiency is improved.

  By using the derived merge candidate included in the merge candidate list as the reference motion compensation parameter, the processing amount can be reduced as compared with the case of newly generating the reference motion compensation parameter. In particular, the merge candidate at the top of the merge candidate list is derived before other merge candidates. Therefore, when the merge candidate at the head of the merge candidate list is used as the reference motion compensation parameter, the correction merge candidate derivation process can be started with less delay than when other merge candidates are used as the reference motion compensation parameter.

  The adjustment of the reference motion compensation parameter is executed by adding a predetermined offset to the motion vector (reference motion vector) included in the reference motion compensation parameter. The offset does not necessarily need to be one type, and the motion compensation parameter may be adjusted by defining an offset list and adding the offset included in the list to the motion vector. If the horizontal and vertical components of the motion vector corresponding to the reference image list L0 in the standard motion compensation parameter are mvBL0X and mvBL0Y, the horizontal and vertical components of the offset are mvOffX and mvOffY, respectively, the horizontal and vertical components of the motion vector of the merge candidate are mvMrgX and mvMrgY are derived from the following equations, respectively.

mvMrgX = mvBL0X + mvOffX
mvMrgY = mvBL0Y + mvOffY
When the range of the motion vector is limited to a predetermined range, clipping may be performed so that the mvMrgX and mvMrgY are within the predetermined range.

Next, an offset list is illustrated with reference to FIG. FIG. 10 is a diagram for explaining the positions of offsets included in the offset list. In FIG. 10, the position described as “C” is a position corresponding to the offset (0, 0), that is, the position of the motion vector in the reference motion parameter. Each number in FIG. 10 represents that the position where the number is shown is included in the offset list and corresponds to the index indicated by the value of the number in the offset list. 10A to 10F show the positions of the offsets included in the following offset lists (a) to (f), respectively.
(A) {(1,0), (-1,0)}
(B) {(0,1), (0, -1)}
(C) {(1,0), (-1,0), (0,1), (0, -1)}
(D) {(1,1), (-1, -1), (1, -1), (-1,1)}
(E) {(4,0), (-4,0)}
(F) {(1,0), (-1,0), (2,0), (-2,0)}
In the above example, for example, (1,0) indicates that mvOffX is 1 and mvOffY is 0. The accuracy of mvOffX and mvOffY in the above example is a quarter pixel accuracy. Further, the accuracy of mvOffX and mvOffY is preferably the same as the accuracy of the motion vector in the reference motion compensation parameter.

  Hereinafter, in FIG. 10, for example, it is assumed that the accuracy of the offsets mvOffX and mvOffY is the same as the accuracy of the motion vector. That is, the minimum unit of the motion vector corresponds to the grid shown in FIG.

  According to (a) to (d) of the offset list, the motion vector is adjusted by the minimum unit of the motion vector in either the horizontal, vertical, or diagonal direction of 45 degrees. Such adjustment is particularly effective in a situation where a rounding error occurs in the reference motion vector and the motion compensation parameter is not suitable for the target PU.

  According to (e) of the offset list, the motion vector is adjusted in the horizontal direction with an accuracy larger than the minimum unit of the motion vector. Such adjustment is effective in a situation where there is a large change in spatial or temporal motion vector.

  Although only the adjustment in the horizontal direction is illustrated in (e) of the offset list, the direction as shown in the offset list (b) to (d) is larger than the minimum unit shown in the offset list (e). A combination of adjustments with accuracy may be used.

  For example, in the case of combining adjustments with an accuracy larger than the minimum unit shown in the offset list (e) in the direction of the offset list (b), they can be combined as follows. That is, the index “1” shown in FIG. 10B may be arranged in the third upper cell, and the index “0” may be arranged in the third lower cell. Note that such an arrangement is merely an example, and is not limited thereto. Arrangements appropriately changed can also be adopted.

  According to (f) of the offset list, the motion vector is adjusted in two different sizes in the horizontal direction. Such adjustment is effective when the degree of change of the spatial or temporal motion vector changes depending on the region. Although only the adjustment in the horizontal direction is illustrated, it is possible to use a combination of a direction as shown in the offset list (b) to (d) and an adjustment with two or more types of accuracy shown in the offset list (f). Good.

  For example, in the case of combining the adjustments with two or more kinds of accuracy shown in the offset list (e) in the direction of the offset list (b), they can be combined as follows. That is, “3” is further arranged in the cell immediately above the index “1” shown in FIG. 10B, and “2” is further arranged in the cell immediately below the index “0”. Also good. Note that such an arrangement is merely an example, and is not limited thereto. Arrangements appropriately changed can also be adopted.

  The correction merge candidate derivation unit 1212D5 can generate a plurality of merge candidates by adding the offset included in the offset list to the motion vector of the reference motion compensation parameter. Actually, merge candidates are derived and recorded in the merge candidate list storage unit 1212B in such a range that the number of merge candidates recorded in the merge candidate list storage unit 1212B does not exceed the merge candidate derivation number.

  Here, the operation of the merge candidate derivation process in the correction merge candidate derivation unit 1212D5 will be described with reference to FIG. FIG. 11 is a diagram illustrating an operation of the correction merge candidate derivation unit 1212D5. First, a reference motion compensation parameter is determined (S121). Subsequently, an offset index for specifying the position of the offset in the offset list is set (S122). The offset index is initially set to 0, and after the second time, a value obtained by adding 1 to the already set offset index is set. Next, a correction merge candidate is derived based on the reference motion compensation parameter and the offset specified by the offset index (S123). The derived correction merge candidate is added to the merge candidate list (S124). At this time, 1 is added to the number of merge candidates (number of merge candidates) included in the merge list. Subsequently, it is determined whether or not the number of merge candidates is equal to or greater than a predetermined number of merge candidate derivations (S125). If the number of merge candidates is equal to or greater than the number of merge candidate derivations (YES in S125), the correction merge candidate derivation process ends because the required number of merge candidates has been derived. When the number of merge candidates is less than the predetermined number of merge candidate derivations (NO in S125), it is subsequently determined whether or not the offset index is the maximum value (S126). Here, the maximum value of the offset index matches the number of offsets included in the offset list. If the offset index is not the maximum value (NO in S126), the process proceeds to S122, and the next correction merge candidate derivation process is continued. If the offset index is the maximum value (YES in S126), the correction merge candidate derivation process is terminated.

(Effect of correction merge candidate)
A correction merge candidate can be derived by the configuration and operation described above and added to the merge candidate list. The correction merge candidate is generated by adjusting with reference to a motion compensation parameter that is highly likely to occur. In general, a motion parameter that is highly likely to occur, specifically, a parameter derived as a spatial merge candidate or a temporal merge candidate may include an error with respect to a motion compensation parameter optimal for the target PU. For example, such an error may be caused by movement or deformation of an object, a continuous change in motion in a spatial domain, and a rounding process of a motion vector. By using the corrected merge candidate, even when such an error occurs, it is possible to increase the possibility that the merge candidate list includes a merge candidate list having a motion compensation parameter suitable for the PU. Therefore, the selection rate of skip CUs or merge PUs can be increased, and the amount of codes can be reduced.

(Combination effect of merge merge candidate and correction merge candidate)
As described above, the merge motion compensation parameter derivation unit 1212 includes a plurality of merge candidate derivation units, and generates various merge candidates and adds them to the merge candidate list so that a motion compensation parameter suitable for the target PU can be obtained. Increase the possibility of being included in the merge candidate list as a merge candidate. Thereby, the selection rate of the skip CU or merge PU can be increased, so that the amount of codes can be reduced. In general, if the number of merge candidates included in the merge candidate list is large, the selection rate of the merge candidate list can be increased, but on the other hand, the amount of processing necessary for deriving the merge candidate list increases.

  In this embodiment, the merge motion compensation parameter deriving unit 1212 includes at least a correction merge candidate deriving unit and a combined merge candidate deriving unit in the merge candidate deriving unit, and according to the slice type of the slice including the target PU. Then, the merge candidate is derived by selecting either the merge candidate deriving unit or the corrected merge candidate deriving unit. The combined merge candidate can be used as a merge candidate when bi-prediction is applicable, but cannot be used as a merge candidate when bi-prediction is not applicable, that is, when single prediction is applied. Therefore, when the slice type is a P slice that prescribes that bi-prediction is not applicable, the number of merge candidates is reduced as compared to a B slice to which bi-prediction is applicable, and the merge selection rate is reduced. In the present embodiment, by using corrected merge candidates instead of merge merge candidates in the P slice, the decrease in the number of merge candidates is suppressed, and the selection rate of skip CUs or merge PUs increases, thereby improving the coding efficiency. To do. In addition, in the B slice, since correction merge candidates are not added, there is an advantage that the processing amount does not increase.

(Flow of CU decoding process)
The CU decoding process in the moving image decoding apparatus 1 will be described with reference to FIG. In the following, it is assumed that the target CU is an inter CU or a skip CU. FIG. 12 is a flowchart illustrating an example of a flow of CU decoding processing (inter / skip CU) in the video decoding device 1.

  When the CU decoding process is started, the CU information decoding unit 11 decodes the CU prediction information for the target CU using the decoding module 10 (S11). This process is performed on a CU basis.

  Specifically, in the CU information decoding unit 11, the CU prediction mode determination unit 111 decodes the skip flag SKIP using the decoding module 10. If the skip flag does not indicate a skip CU, the CU prediction mode determination unit 111 further decodes the CU prediction type information Pred_type using the decoding module 10.

  Next, PU unit processing is performed. That is, the motion compensation parameter deriving unit 121 included in the PU information decoding unit 12 decodes the motion information (S12), and the predicted image generation unit 14 generates a predicted image by inter prediction based on the decoded motion information. (S13).

  Next, the TU information decoding unit 13 performs a TU division decoding process (S14). Specifically, in the TU information decoding unit 13, the TU partition setting unit 131 sets the TU partitioning method based on the parameters decoded from the encoded data, the CU size, and the PU partition type. This process is performed on a CU basis.

  Next, TU unit processing is performed. That is, the TU information decoding unit 13 decodes the transform coefficient (S15), and the inverse quantization / inverse transform unit 15 derives a prediction residual from the decoded transform coefficient (S16).

  Next, the adder 17 adds the predicted image and the prediction residual, thereby generating a decoded image (S17). This process is performed on a CU basis.

  As described above, the image decoding apparatus 1 performs inter-frame prediction of an image in a prediction unit, either uni-prediction that refers to one reference image or bi-prediction that refers to two reference images. In the image decoding apparatus 1 restored by the prediction method, a merge merge candidate deriving unit 1212D3 for deriving a merge candidate using motion vectors of two different merge candidates, and a plurality of merge candidates based on a single merge candidate A merge candidate list is derived using the combined merge candidate deriving unit 1212D3 when the correction merge candidate deriving unit 1212D5 to be derived, the slice type determining unit 1212E that determines whether bi-prediction is applicable, and bi-prediction are applicable. On the other hand, when bi-prediction is not applicable, a merge candidate list including one or more merge candidates using the corrected merge candidate derivation unit 1212D5 And merge candidate derivation controller 1212A to derive, a structure comprising a.

(Modification)
Below, some modifications of the moving image decoding apparatus 1 demonstrated above are shown.
[Variation 1: Correction merge candidate addition timing]
In the above embodiment, in the description of the merge candidate list derivation operation in the merge motion compensation parameter derivation unit 1212 using FIG. 5, an example of the order and conditions in which each merge candidate is derived is shown. May be used. For example, the merge candidate list may be derived by the operation shown in FIG.

  FIG. 13 is a flowchart showing another example of the operation of the merge motion compensation parameter deriving unit 1212. First, a merge candidate is derived in the spatial merge candidate deriving unit 1212D1 (S201). Subsequently, a merge candidate is derived in the time merge candidate deriving unit 1212D2 (S202). The slice type determination unit 1212E determines the slice type of the slice including the target PU (S203). When the slice type is a B slice (YES in S203), the merge candidate derivation unit 1212D3 derives merge candidates (S204), and then proceeds to step S205. On the other hand, if the slice type is not B slice (NO in S103), the process proceeds to step S205. Subsequently, a merge candidate is derived in the zero merge candidate deriving unit 1215D4 (S205). The slice type determination unit 1212E determines the slice type of the slice including the target PU (S206). If the slice type is B slice (YES in S206), the merge candidate list derivation process is terminated. On the other hand, if the slice type is not a B slice (NO in S206), the merge candidate is derived in the corrected merge candidate derivation unit 1212D5 (S207), and the merge candidate list derivation process is terminated.

  In the merge candidate list derivation process in FIG. 13, the corrected merge candidate derivation process is executed after the zero merge candidate derivation process. Thereby, in the merge candidate list, an index smaller than the correction merge candidate is assigned to the zero merge candidate. In other words, the zero merge candidate has priority over the correction merge candidate. In general, the zero merge candidate is effective when the motion compensation parameter is difficult to estimate spatially or temporally. In such a situation, the possibility that the zero merge candidate becomes a motion compensation parameter suitable for the target PU is higher than that of the corrected merge candidate. Therefore, by applying the merge candidate list derivation process described in FIG. Will improve. Note that the merge candidate list derivation process described with reference to FIG. 5 has an advantage that the number of slice type determinations is smaller and the processing amount is smaller than the process in FIG.

It should be noted that which one of the correction merge candidate derivation and the zero merge candidate derivation should be performed first may be determined based on the derived merge candidate or the reference motion compensation parameter used for the correction merge candidate derivation. For example, when the motion vector of the reference motion compensation parameter is a zero vector (values of both components are 0), the correction merge candidate derivation process is executed first, and conversely, the motion vector of the reference motion compensation parameter is the zero vector. Otherwise, the zero merge candidate derivation process may be executed first. When merge candidates having a zero vector already exist in the merge candidate list, the effectiveness of the zero merge candidate is relatively lowered, so that the zero merge candidate and the corrected merge candidate are derived based on the presence or absence of the merge candidate having the zero vector. Coding efficiency is improved by changing the order, that is, the priority order.
[Variation 2: Correction variation adjustment candidate correction]
In the above-described embodiment, the correction merge candidate derivation unit 1212D5 has explained the example in which the correction merge candidate is derived by adjusting the motion vector corresponding to the reference image list L0 in the standard motion compensation parameter by the offset. Correction merge candidates may be derived by adjusting other parameters included.

  For example, the motion vector corresponding to the reference image list L1 may be adjusted by offset. Specifically, in the standard motion compensation parameter, when the reference image list L0 use flag predFlagL0 is 1, the motion vector corresponding to the reference image list L0 is used as a reference for adjustment, and when predFlagL0 is 0, The motion vector corresponding to the reference image list L1 can be used as a reference for adjustment. The reference motion compensation parameter does not necessarily include a motion vector corresponding to the reference image list L0. In such a case, encoding efficiency can be improved by generating merge candidates using a motion vector corresponding to the reference image list L1.

  Note that when both reference image lists of L0 and L1 are available in the standard motion compensation parameter, the motion vector corresponding to each reference image list can be adjusted to be a correction merge candidate. In this case, since the number of merge candidates increases, the amount of processing necessary for derivation of merge candidates increases, but the possibility of merging merge candidates suitable for the target PU increases, so that the coding efficiency is improved.

  Further, for example, the reference image index may be adjusted. That is, adjustment may be performed based on the value of the reference image index refIdxL0 corresponding to the reference image list L0, and the value of the reference image index refIdxMrg in the correction merge candidate may be derived. Derivation of refIdxMrg can be realized as follows, for example.

refIdxMrg = 0 (when refIdxL0 ≠ 0)
refIdxMrg = 1 (when refIdxL0 = 0)
Normally, a reference image that is more likely to be used as a reference image is assigned a smaller value index in the reference image list. Therefore, when adjusting, it is preferable to adjust so that the reference image index is as small as possible.

  When the syntax element for transmitting the reference image index is encoded or binarized so that a smaller value is associated with a shorter code string, for example, by a truncated unary code, A reference image index having a small value is assigned to a reference image that is likely to be used as a reference image in order to reduce the code amount of the reference image index.

When adjusting the reference image index, the interval between the reference image indicated by refIdxL0 and the target frame (for example, the difference between the POC of the reference image and the POC of the target frame), and the reference image and the target frame indicated by refIdxMrg. It is preferable to refer to the interval and adjust the motion vector corresponding to the reference image list L0 in the standard motion compensation parameter based on the ratio of both intervals.
[Variation 3: Variation of standard correction merge candidates]
In the above embodiment, the correction merge candidate derivation unit 1212D5 has described that the motion compensation parameter corresponding to the merge candidate having the smallest index recorded in the merge candidate list is used as the reference motion compensation parameter. .

  For example, the motion compensation parameter corresponding to the merge candidate that is the merge candidate derived by the spatial merge candidate derivation unit 1212D1 and has the smallest index among the merge candidates recorded in the merge candidate list is used as the reference motion compensation parameter. It may be used. By limiting the merge candidates that can be used as the reference motion compensation parameter to the spatial merge candidates, the possibility that the temporal merge candidate is used as the reference motion compensation parameter is eliminated. In general, temporal merge candidate derivation requires a larger amount of processing than spatial merge candidate derivation. Therefore, by limiting the reference motion compensation parameter candidates to the spatial merge candidates, the reference motion compensation parameters can be determined without waiting for the derivation of the temporal merge candidates, so that there is an advantage that the delay is reduced.

As another example, a motion compensation parameter that is likely to be selected can be defined, and the defined motion compensation parameter can be used as a reference motion compensation parameter. Here, it is preferable that the value of the prescribed motion compensation parameter does not change in the slice. Specific specific motion compensation parameters include the following.
(A) predFlagL0 = 1, predFlagL1 = 0, mvL0 = (0,0), refIdxL0 = 0
(B) Motion compensation parameters transmitted with picture parameter set PPS and slice header SH.
(C) Motion compensation parameters derived based on motion compensation parameters used in the previous picture and previous slice.

When the default motion compensation parameter is used as the reference motion compensation parameter, the encoding efficiency is slightly reduced compared to the case where the spatial merge candidate using the local characteristics or the temporal merge candidate is used as the reference motion compensation parameter. Since the reference motion compensation parameter can be derived without waiting for candidate derivation, the delay can be reduced.
[Variation 4: Switching the offset list]
In the above embodiment, the correction merge candidate derivation unit 1212D5 describes an example in which the adjustment of the reference motion vector is executed based on a single offset list, but the present invention is not limited to this. A plurality of offset lists may be defined, one of the offset lists may be selected based on a predetermined condition, and the motion vector may be adjusted based on the offset list.

(Switching according to mv direction)
For example, the offset list may be switched according to the direction of the reference motion vector mvB. Specifically, when either the horizontal offset list offListH or the vertical offset list offListV can be selected as the offset list, the offset list offList is selected based on the horizontal component mvBX and the vertical component mvBY of mvB as follows. To do. The horizontal offset list is, for example, the offset list described with reference to FIG. 10A, and the vertical offset list is, for example, the offset list described with reference to FIG.

offList = offLixtV (| mvBX | ≧ | mvBY |)
offList = offListH (| mvBX | <| mvBY |)
In other words, the vertical offset list is selected when the absolute value of the horizontal component of the reference motion vector is greater than or equal to the absolute value of the vertical component. Conversely, when the absolute value of the horizontal component is less than the absolute value of the vertical component, the horizontal offset is selected. Select a list.

  The horizontal offset list is not limited to the above example, and may be an offset list in which the sum of absolute values of offset horizontal components included in the offset list is larger than the sum of absolute values of vertical components. For example, the offset list of (e) or (f) of FIG. 10 can also be used as the horizontal offset list. Similarly, the vertical offset list may be an offset list in which the sum of absolute values of offset vertical components included in the offset list is larger than the sum of absolute values of horizontal components.

  In general, the larger the motion vector, the lower the possibility that a motion vector suitable for the target PU will be derived by adjusting the offset. This is based on the fact that when the motion vector is large, a larger number of offset candidates are required to make an appropriate adjustment, and conversely, a limited offset candidate is likely not to make an appropriate adjustment. . Therefore, when the number of offset candidates is limited and the absolute value of one component of the reference motion vector is larger than the other component, a large number of offset candidates to be adjusted in the direction of the component having the smaller absolute value It is preferable to include it in the offset list.

  As a similar example, if the cross offset list offListC ((c) in FIG. 10) can be selected as the offset list in addition to the horizontal and vertical offset lists, the offset list is selected as follows. It is preferable.

offList = offLixtV (| mvBX |> 2 × | mvBY |)
offList = offListH (| mvBY |> 2 × | mvBX |)
offList = offLixtC (other than above)
That is, the vertical offset list is selected when the absolute value of the horizontal component of the reference motion vector is larger than the absolute value of the vertical component by a certain amount or more (in the case of the above formula, it is larger than twice). Conversely, when the absolute value of the vertical component is larger than the absolute value of the horizontal component by a certain amount, the horizontal offset list is selected. If none of the above, a cross offset list is selected. In the above, a configuration using the offset list shown in (d) of FIG. 10 instead of the cross offset list can be adopted.

(Switching according to the size of mv)
Further, for example, the offset list may be switched according to the magnitude of the reference motion vector mvB. Specifically, when either the small displacement offset list offListS or the large displacement offset list offListL can be selected as the offset list, the offset list is as follows according to the value of the length ‖mvB‖ of the vector mvB: Select offList. For example, (a) in FIG. 10 can be used as the small displacement offset list, and (e) in FIG. 10 can be used as the large displacement offset list.

offList = offLixtL (‖mvB‖> THL)
offList = offLixtS (‖mvB‖ ≤ THL)
That is, the large displacement offset list is selected when the length of the reference motion vector is larger than the predetermined threshold value THL. Conversely, when the length of the reference motion vector is less than the predetermined threshold THL, the small displacement offset list is selected.

  In general, the larger the reference motion vector, the higher the possibility that adjustment using a larger offset is required. Therefore, when the reference motion vector is large, an offset list that includes many offsets representing a large displacement is used, and conversely, when the reference motion vector is small, an offset list that includes many offsets representing a small displacement is used. Since there is a high possibility that an offset required for the adjustment is included in the offset list, the coding efficiency is improved.

  Note that the definitions of the small displacement offset list and the large displacement offset list are relative, but for example, the magnitude of the displacement of the offset list can be defined as follows, for example. The offset list displacement is the average of the norms of all offsets contained in the offset list. When two types of offset lists exist, the offset list with the larger displacement is the large displacement offset list, and the offset list with the smaller displacement is the small displacement offset list.

(Switching based on the sign of each component of mv)
Further, for example, the offset list may be switched based on the sign of each component of the reference motion vector mvB. Specifically, when the horizontal positive offset list offListHP and the horizontal negative offset list offListHN can be selected, the horizontal negative offset list is selected when the horizontal component of mvB is positive or 0, and the horizontal component of mvB is negative. In some cases, a horizontal positive offset list may be selected.

offList = offLixtHN (mvBX> = 0)
offList = offListHP (mvBX <0)
Here, the horizontal positive offset list is an offset list in which the horizontal components of all included offsets are positive, for example, an offset list including only offset (1,0), offset (1,0) and (4, The offset list including 0) is the horizontal positive offset list. Similarly, the horizontal negative offset list is an offset list in which the horizontal components of all the included offsets are negative.

  According to the selection of the offset list, a correction merge candidate is derived by adjusting the reference motion vector mvB in the direction opposite to the horizontal component of the reference motion vector. By such correction, the absolute value of the horizontal component of the reference motion vector is reduced on average. In a general motion vector probability distribution, a motion vector having a small horizontal component value has a higher probability of occurrence than a motion vector having a large horizontal component value. Therefore, when correction is performed, by correcting so that the value of the horizontal component becomes small, a correction merge candidate that can be selected more easily can be generated, so that the encoding efficiency is improved.

  In addition, although the case where the horizontal component is adjusted has been described above, the same can be said for the vertical component. Therefore, the offset list may be switched based on the sign of the vertical component of the reference motion vector. Furthermore, the offset list may be switched according to the combination of the horizontal component code and the vertical component code.

(Switching depending on whether the specific component of mv is 0)
Further, for example, the offset list may be switched depending on whether the value of the horizontal component or the vertical component of the reference motion vector mvB is 0. Specifically, when one of the horizontal offset list offListH and the vertical offset list offListV can be selected as the offset list, the offset list offList is based on the horizontal component mvBX and the vertical component mvBY of mvB as follows. Select.

offList = offLixtH (| mvBY | = 0)
offList = offListV (other than above)
That is, the horizontal offset list is selected when the absolute value of the horizontal component of the reference motion vector is 0, and the vertical offset list is selected otherwise.

  Generally, panning and tilting are often used when capturing a moving image with a video camera. In such a case, a large number of motion vectors in a complete vertical direction or a complete horizontal direction are used. In such a case, the correction efficiency is improved by correcting the correction merge candidate in a direction in which the value of the reference motion vector component is not zero.

(Switch according to image size)
Further, for example, the offset list may be switched according to the image size. Specifically, when either the small displacement offset list offListS or the large displacement offset list offListL can be selected as the offset list, the large displacement offset list is selected when the image size is larger than a predetermined size.

  In general, when the image size is large, the possibility of generating a large motion vector is higher than when the image size is small. For this reason, when the image size is large, the coding efficiency can be improved by selecting the large displacement offset list.

(Switch according to the level)
Further, for example, the offset list may be switched according to the level to which the encoded data is suitable. Specifically, the large displacement offset list is selected if the level to which the encoded data matches is equal to or higher than a predetermined level, and the small displacement offset list is selected otherwise.

  Here, the compatible level of the encoded data is defined by a syntax element level_idc included in the sequence parameter set SPS, for example. It should be noted that the level at which the encoded data is suitable is a decoding device or encoding that includes at least the maximum number of luminance pixels that the decoding device can decode in units of one second, the maximum number of luminance pixels in the picture, and the maximum bit rate of the encoded data. The performance that the data should comply with is related. For example, H.M. In H.264 and HEVC, level 4 indicates that the decoding device needs to be capable of decoding encoded data including a picture of at least 1920 × 1088 pixels. Level 5 indicates that the decoding apparatus needs to be capable of decoding encoded data including a picture of at least 4096 × 2160 pixels. In general, it means that the higher the level (the larger the value of level_idc), the more the decoding device is required to be capable of decoding a picture of a larger size.

From the above level definition, encoded data that conforms to a higher level is more likely to be encoded data obtained by encoding a larger image size. Therefore, for the reason described above, when the level to which the encoded data is compatible is high (for example, when the level is 4 or higher), the encoding efficiency is improved by selecting the large displacement offset list.
[Variation 5: Application example to bi-prediction constraint]
In the above embodiment, in the description of the merge candidate list derivation operation in the merge motion compensation parameter derivation unit 1212 using FIG. 5, the configuration in which the merge merge candidate derivation unit and the correction merge candidate derivation unit are switched according to the slice type is shown. . More generally, the selection may be made based on whether bi-prediction is applicable in the target PU. Hereinafter, an example in which the merge candidate derivation unit is selectively used when bi-prediction application restriction according to the PU size is performed will be described with reference to FIGS. 14 and 15.

  FIG. 14 is a block diagram showing the configuration of the merge motion compensation parameter derivation unit 1212 according to this modification. In the configuration of the merge motion compensation parameter derivation unit 1212 illustrated in FIG. 1, the merge motion compensation parameter derivation unit 1212 includes a merge candidate derivation determination unit 1212A, a merge candidate derivation determination unit 1212G, a slice type determination unit 1212E, and bi-prediction. The restriction determination unit 1212F is replaced. Components other than those described above are the same as the corresponding components of the merge motion compensation parameter derivation unit 1212 in FIG.

(Merge candidate derivation control unit)
The merge candidate derivation control unit 1212F controls the merge candidate derivation unit, derives a predetermined number (merge candidate derivation number) of merge candidates, and stores them in the merge candidate storage unit 1212B. In merging candidate derivation, the merge candidate derivation control unit 1212F accesses the bi-prediction restriction determination unit 1212G as necessary, acquires bi-prediction restriction application presence / absence information, and uses it to derive merge candidates. The control procedure of the merge candidate derivation means will be described later as a merge candidate list derivation process flow.

(Bi-prediction restriction determination unit)
The bi-prediction restriction determination unit 1212G determines whether or not the bi-prediction restriction is applied in the target PU, and outputs the result. Here, when bi-prediction restriction is applied in the target PU, only single prediction using either the reference image list L0 or the reference image list L1 is applicable in the target PU. On the other hand, when the bi-prediction restriction is not applied, bi-prediction using two reference image lists of L0 and L1 can be applied.

  In this modification, bi-prediction restriction is applied when the size of the target PU is smaller than a predetermined size. Specifically, the value of the flag restBiPredFlag indicating whether or not the bi-prediction restriction is applied is determined under the following conditions.

restBiPredFlag = 1 (the length of at least one side of the target PU is less than 8)
restBiPredFlag = 0 (the length of both sides of the target PU is 8 or more)
Here, the value of restBipredFlag indicates that “1” applies the bi-prediction restriction, and “0” indicates that the bi-prediction restriction is not applied. The bi-prediction restriction determination unit 1212G determines the application / non-application of the bi-prediction restriction by determining the value of restBiPredFlag.

  The bi-prediction restriction applicable to the present invention is not limited to the above, and may be a bi-prediction restriction based on another different standard. For example, bi-prediction restriction may be performed based on the size of the target CU including the target PU, the level at which encoded data is compatible, the combination of the target CU size and the target PU size, the combination of the tree block size and the target PU size, and the like.

  FIG. 15 is a flowchart showing another example of the operation of the merge motion compensation parameter derivation unit 1212. First, a merge candidate is derived in the spatial merge candidate deriving unit 1212D1 (S301). Subsequently, a merge candidate is derived in the temporal merge candidate deriving unit 1212D2 (S302). The bi-prediction restriction determination unit 1212F determines whether or not the bi-prediction restriction is applied to the target PU (S303). When the bi-prediction restriction is not applied (NO in S303), the merge candidate derivation unit 1212D3 derives a merge candidate (S304), and then proceeds to step S306. On the other hand, when the bi-prediction restriction is applied (YES in S303), the merge candidate is derived in the corrected merge candidate deriving unit 1212D5 (S305), and then the process proceeds to step S306. Finally, in the zero merge candidate derivation unit 1215D4, merge candidates are derived (S306), and the merge candidate list derivation process ends.

(Combination effect of merge merge candidate and correction merge candidate)
As described above, the merge motion compensation parameter derivation unit 1212 according to the present modification includes a plurality of merge candidate derivation units, generates various merge candidates, and adds them to the merge candidate list, thereby adding the merge candidate list to the target PU. The possibility that a suitable motion compensation parameter is included in the merge candidate list as a merge candidate is increased. Thereby, the selection rate of the skip CU or merge PU can be increased, so that the amount of codes can be reduced. In general, if the number of merge candidates included in the merge candidate list is large, the selection rate of the merge candidate list can be increased, but on the other hand, the amount of processing necessary for deriving the merge candidate list increases.

The merge motion compensation parameter deriving unit 1212 in the present modification includes at least a corrected merge candidate deriving unit and a combined merge candidate deriving unit in the merge candidate deriving unit, and the bi-prediction restriction is applied to the target PU. Depending on whether or not, a merge candidate is derived by selecting either the merge candidate deriving unit or the correction merge candidate deriving unit. The merge merge candidate can be used as a merge candidate when bi-prediction is applicable, that is, when bi-prediction restriction is not applied, but as a merge candidate when bi-prediction is not applicable, that is, when applying single prediction. Not available. Therefore, when the slice type is a P slice that prescribes that bi-prediction is not applicable, the number of merge candidates is reduced as compared to a B slice to which bi-prediction is applicable, and the merge selection rate is reduced. In the present embodiment, by using corrected merge candidates instead of merge merge candidates in the P slice, the decrease in the number of merge candidates is suppressed, and the selection rate of skip CUs or merge PUs increases, thereby improving the coding efficiency. To do. In addition, in the B slice, since correction merge candidates are not added, there is an advantage that the processing amount does not increase.
[Modification 6: Configuration example other than correction merge candidate derivation unit and combined merge candidate derivation unit]
In the above embodiment, the merge motion compensation parameter deriving unit 1212 derives a merge candidate list by switching between the combined merge candidate deriving unit 1212D3 and the corrected merge candidate deriving unit 1212D5 based on whether bi-prediction is applicable. As described above, the merge merge candidate derivation unit 1212D3 and the correction merge candidate derivation unit 1212D5 may be replaced with another merge candidate derivation unit.

  The merge merge candidate derivation unit 1212D3 can derive a merge candidate that performs bi-prediction and replace it with another merge candidate derivation unit that does not derive a merge candidate that performs single prediction. Specifically, a merge candidate derivation unit that derives bi-predictive merge candidates by the following procedure may be used instead of the merged merge candidate derivation unit 1212D3.

  First, a reference motion compensation parameter is derived in the same manner as described in the correction merge candidate derivation unit 1212D5. The motion vector mvL0 and the reference image index refIdxL0 of the merge candidate to be derived are set as corresponding values of the standard motion compensation parameter. The value of the reference image refIdxL1 is set to 1. The value of the motion vector mvL1 is generated by scaling the motion vector mvL0. Note that the scaling is performed based on the ratio between the difference between the reference image refIdxL1 and the POC of the target frame and the difference between the reference image refIdxL0 and the POC of the target frame. Finally, the values of the reference image list use flags predFlagL0 and predFlagL1 are set to 1, respectively, and set as bi-predictive merge candidates.

  Further, the corrected merge candidate derivation unit 1212D5 can derive a merge candidate for performing single prediction and replace it with another merge candidate derivation unit that does not derive a merge candidate for performing bi-prediction. Specifically, a merge candidate derivation unit that derives a single prediction merge candidate by the following procedure may be used instead of the correction merge candidate derivation unit 1212D5.

  First, of the merge candidates recorded in the merge candidate list, a merge candidate for performing bi-prediction (a merge candidate with predFlagL0 = 1 and predFlagL1 = 1) is selected. In the selected merge candidate, the merge candidate is generated by changing the value of predFlagL1 to 0.

  Even when the above replacement is performed, the same effect as in the present embodiment can be obtained. That is, in the P slice, by using the uni-predictive merge candidate instead of the bi-predictive merge candidate, the decrease in the number of merge candidates with respect to the number of merge candidates in the B slice is suppressed, and the skip CU or merge PU selection rate is increased. Encoding efficiency is improved. In addition, in the B slice, since a single prediction merge candidate is not added, there is an advantage that the processing amount does not increase.

[Moving picture encoding device]
Hereinafter, the moving picture encoding apparatus 2 according to the present embodiment will be described with reference to FIGS.

(Outline of video encoding device)
Generally speaking, the moving image encoding device 2 is a device that generates and outputs encoded data # 1 by encoding the input image # 10.

(Configuration of video encoding device)
First, a configuration example of the video encoding device 2 will be described with reference to FIG. FIG. 16 is a functional block diagram showing the configuration of the moving image encoding device 2. As illustrated in FIG. 16, the moving image encoding device 2 includes an encoding setting unit 21, an inverse quantization / inverse conversion unit 22, a predicted image generation unit 23, an adder 24, a frame memory 25, a subtractor 26, a conversion / A quantization unit 27 and an encoded data generation unit (encoding means) 29 are provided.

  The encoding setting unit 21 generates image data related to encoding and various setting information based on the input image # 10.

  Specifically, the encoding setting unit 21 generates the next image data and setting information.

  First, the encoding setting unit 21 generates the CU image # 100 for the target CU by sequentially dividing the input image # 10 into slice units, tree block units, and CU units.

  Also, the encoding setting unit 21 generates header information H ′ based on the result of the division process. The header information H ′ includes (1) information on the size and shape of the tree block belonging to the target slice and the position in the target slice, and (2) the size, shape and shape of the CU belonging to each tree block. CU information CU ′ for the position at

  Further, the encoding setting unit 21 refers to the CU image # 100 and the CU information CU 'to generate PT setting information PTI'. The PT setting information PTI 'includes at least the prediction type of the target CU, the division information of the target CU into PUs, and the motion compensation parameter in each PU when the target CU is an inter CU. Contains information about.

  The encoding setting unit 21 includes (1) a prediction type of the target CU, (2) a possible division pattern for each PU of the target CU, and (3) a prediction mode that can be allocated to each PU (if it is an intra CU). The cost is calculated for all combinations of the intra prediction mode and the motion compensation parameter in the case of an inter CU, and the prediction type, the division pattern, and the prediction mode with the lowest cost are determined and set as PT setting information PTI ′. .

  The encoding setting unit 21 supplies the CU image # 100 to the subtractor 26. In addition, the encoding setting unit 21 supplies the header information H ′ to the encoded data generation unit 29. Also, the encoding setting unit 21 supplies the PT setting information PTI ′ to the predicted image generation unit 23 and the PU information generation unit 30.

  The inverse quantization / inverse transform unit 22 performs inverse quantization and inverse orthogonal transform on the quantized prediction residual for each block supplied from the transform / quantization unit 27, thereby predicting the prediction residual for each block. To restore. The inverse orthogonal transform is as already described for the inverse quantization / inverse transform unit 15 shown in FIG.

  Further, the inverse quantization / inverse transform unit 22 integrates the prediction residual for each block according to the division pattern specified by the TT division information (described later), and generates a prediction residual D for the target CU. The inverse quantization / inverse transform unit 22 supplies the prediction residual D for the generated target CU to the adder 24.

  The predicted image generation unit 23 refers to the locally decoded image P ′ and the PT setting information PTI ′ recorded in the frame memory 25 to generate a predicted image Pred for the target CU. Note that the predicted image generation process performed by the predicted image generation unit 23 is the same as that performed by the predicted image generation unit 14 included in the video decoding device 1, and thus description thereof is omitted here.

  The adder 24 adds the predicted image Pred supplied from the predicted image generation unit 23 and the prediction residual D supplied from the inverse quantization / inverse transform unit 22 to thereby obtain the decoded image P for the target CU. Generate.

  Decoded decoded images P are sequentially recorded in the frame memory 25. In the frame memory 25, decoded images corresponding to all tree blocks decoded prior to the target tree block (for example, all tree blocks preceding in the raster scan order) at the time of decoding the target tree block. Are recorded together with the parameters used for decoding the decoded image P.

  The subtractor 26 generates a prediction residual D for the target CU by subtracting the prediction image Pred from the CU image # 100. The subtractor 26 supplies the generated prediction residual D to the transform / quantization unit 27.

  The transform / quantization unit 27 generates a quantized prediction residual by performing orthogonal transform and quantization on the prediction residual D. Here, the orthogonal transform refers to an orthogonal transform from the pixel region to the frequency region. Examples of inverse orthogonal transform include DCT transform (Discrete Cosine Transform), DST transform (Discrete Sine Transform), and the like.

  Specifically, the transform / quantization unit 27 refers to the CU image # 100 and the CU information CU ', and determines a division pattern of the target CU into one or a plurality of blocks. Further, according to the determined division pattern, the prediction residual D is divided into prediction residuals for each block.

  The transform / quantization unit 27 generates a prediction residual in the frequency domain by orthogonally transforming the prediction residual for each block, and then quantizes the prediction residual in the frequency domain to Generate quantized prediction residuals.

  In addition, the transform / quantization unit 27 generates the quantization prediction residual for each block, TT division information that specifies the division pattern of the target CU, information about all possible division patterns for each block of the target CU, and TT setting information TTI ′ including is generated. The transform / quantization unit 27 supplies the generated TT setting information TTI ′ to the inverse quantization / inverse transform unit 22 and the encoded data generation unit 29.

  The PU information generation unit 30 derives the PT setting information PTI based on the PT setting information PTI ′ and supplies it to the encoding setting unit 21. Details of the PU information generation unit 30 will be described later.

  The encoded data generation unit 29 encodes header information H ′, TT setting information TTI ′, and PT setting information PTI ′, and multiplexes the encoded header information H, TT setting information TTI, and PT setting information PTI. Coded data # 1 is generated and output.

(Correspondence relationship with video decoding device)
The video encoding device 2 includes a configuration corresponding to each configuration of the video decoding device 1. Here, “correspondence” means that the same processing or the reverse processing is performed.

  For example, as described above, the prediction image generation process of the prediction image generation unit 14 included in the video decoding device 1 and the prediction image generation process of the prediction image generation unit 23 included in the video encoding device 2 are the same. .

  For example, the process of decoding a syntax value from a bit string in the video decoding device 1 corresponds to a process opposite to the process of encoding a bit string from a syntax value in the video encoding device 2. Yes.

  In the following, it will be described how each configuration in the video encoding device 2 corresponds to the CU information decoding unit 11, the PU information decoding unit 12, and the TU information decoding unit 13 of the video decoding device 1. . Thereby, the operation and function of each component in the moving image encoding device 2 will be clarified in more detail.

  The encoded data generation unit 29 corresponds to the decoding module 10. More specifically, the decoding module 10 derives a syntax value based on the encoded data and the syntax type, whereas the encoded data generation unit 29 encodes the code based on the syntax value and the syntax type. Generate data.

  The encoding setting unit 21 corresponds to the CU information decoding unit 11 of the video decoding device 1. A comparison between the encoding setting unit 21 and the CU information decoding unit 11 is as follows.

  The CU information decoding unit 11 supplies the encoded data related to the CU prediction type information and the syntax type to the decoding module 10 and determines the PU partition type based on the CU prediction type information decoded by the decoding module 10.

  In response to this, the encoding setting unit 21 determines the PU partition type, generates CU prediction type information, and supplies the syntax value and syntax type related to the CU prediction type information to the encoded data generation unit 29. To do.

  The PU information generation unit 30 and the predicted image generation unit 23 correspond to the PU information decoding unit 12 and the predicted image generation unit 14 of the video decoding device 1. These are compared as follows.

  As described above, the PU information decoding unit 12 supplies the encoded data related to the motion information and the syntax type to the decoding module 10 and derives a motion compensation parameter based on the motion information decoded by the decoding module 10. Further, the predicted image generation unit 14 generates a predicted image based on the derived motion compensation parameter.

  In response to this, the PU information generation unit 30 determines a motion compensation parameter, and supplies the syntax value and syntax type related to the motion compensation parameter to the encoded data generation unit 29.

(Details of PU information generation unit 30)
FIG. 17 is a block diagram illustrating a configuration of the PU information generation unit 30. The PU information generation unit 30 includes a motion compensation parameter generation unit 301 and a CU prediction type determination unit 302 for generating a motion compensation parameter. The motion compensation parameter generation unit 301 includes a motion compensation method determination unit 3011, a basic motion compensation parameter generation unit 3012, a merge motion compensation parameter generation unit 3013, and a switch 3014.

  When the prediction type indicated by the input PT setting information PTI 'is an intra CU, the CU prediction type determination unit 302 outputs the PT setting information PTI as PT setting information PTI' to the outside. On the other hand, when the prediction type is inter CU, PT setting information PTI ′ is supplied to the motion compensation parameter generation unit 301.

  The basic motion compensation parameter generation unit 3012 derives the corresponding syntax element values inter_pred_flag, mvd, mvp_idx, and refIdx from the motion compensation parameters in each PU included in the input PT setting information PTI ′, and performs PT. Output as setting information PTI.

  The merge motion compensation parameter generation unit 3013 generates a merge candidate having a motion compensation parameter similar to the motion compensation parameter in each PU included in the input PT setting information PTI ′, and a syntax element value merge_idx indicating the merge candidate Is output. The merge candidate is supplied to the motion compensation method determination unit 3011.

  The merge motion compensation parameter generation unit 3013 includes the same configuration as the merge motion compensation parameter derivation unit 1212 described with reference to FIG. Therefore, merge candidates derived corresponding to the same merge_idx are the same between the merge motion compensation parameter generation unit 3013 and the merge motion compensation parameter derivation unit 1212.

  The motion compensation method determination unit 3011 evaluates the degree of similarity between the input merge candidate and the motion compensation parameter, determines whether to apply merge, and supplies the result to the switch 3014. When the input indicates that merging is applied, the switch 3014 selects the output of the merge motion compensation parameter generation unit 3013 and outputs it as the PT setting information PTI. On the other hand, when the input indicates that merging is not applied, the output of the basic motion compensation parameter setting unit 3013 is selected and output to the outside as PT setting information PTI.

(Process flow)
The CU encoding process in the moving image encoding device 2 will be described with reference to FIG. In the following, it is assumed that the target CU is an inter CU or a skip CU. FIG. 18 is a flowchart illustrating an example of the flow of CU encoding processing (inter / skip CU) in the moving image encoding device 2.

  When the CU encoding process is started, the encoding setting unit 21 determines CU prediction information for the target CU, and the encoded data generation unit 29 encodes the CU prediction information determined by the encoding setting unit 21. (S21). This process is performed on a CU basis.

  Specifically, the encoding setting unit 21 determines whether or not the target CU is a skip CU. When the target CU is a skip CU, the encoding setting unit 21 causes the encoded data generation unit 20 to encode the skip flag SKIP. When the target CU is not a skip CU, the encoding setting unit 21 causes the encoded data generation unit 20 to encode the CU prediction type information Pred_type.

  Next, PU unit processing is performed. That is, the encoding setting unit 21 derives motion information (motion compensation parameter). Based on the motion information, the PU information generation unit 30 derives a syntax element value corresponding to the motion information, and the encoded data generation unit 29 encodes the syntax element value (S22). Further, the predicted image generation unit 14 generates a predicted image by inter prediction based on the derived motion information (S23).

  Next, the transform / quantization unit 27 performs TU division coding processing (S24). Specifically, the transform / quantization unit 27 sets a TU partitioning scheme based on the CU size and PU partition type of the target CU. This process is performed on a CU basis.

  Next, TU unit processing is performed. That is, the transform / quantization unit 27 transforms / quantizes the prediction residual into transform coefficients (S25), and the encoded data generation unit 29 encodes the transformed / quantized transform coefficients (S26).

  Next, the inverse quantization / inverse transform unit 22 performs inverse quantization / inverse transform on the transformed / quantized transform coefficient to restore the prediction residual, and the adder 24 restores the predicted image and the prediction residual. Is added to generate a decoded image (S27). This process is performed on a CU basis.

[Application example]
The above-described moving image encoding device 2 and moving image decoding device 1 can be used by being mounted on various devices that perform transmission, reception, recording, and reproduction of moving images. The moving image may be a natural moving image captured by a camera or the like, or may be an artificial moving image (including CG and GUI) generated by a computer or the like.

  First, it will be described with reference to FIG. 19 that the above-described moving image encoding device 2 and moving image decoding device 1 can be used for transmission and reception of moving images.

  FIG. 19A is a block diagram illustrating a configuration of a transmission device PROD_A in which the moving image encoding device 2 is mounted. As illustrated in (a) of FIG. 19, the transmission device PROD_A modulates a carrier wave with an encoding unit PROD_A1 that obtains encoded data by encoding a moving image and the encoded data obtained by the encoding unit PROD_A1. Thus, a modulation unit PROD_A2 that obtains a modulation signal and a transmission unit PROD_A3 that transmits the modulation signal obtained by the modulation unit PROD_A2 are provided. The moving image encoding apparatus 2 described above is used as the encoding unit PROD_A1.

  The transmission device PROD_A is a camera PROD_A4 that captures a moving image, a recording medium PROD_A5 that records the moving image, an input terminal PROD_A6 that inputs the moving image from the outside, as a supply source of the moving image input to the encoding unit PROD_A1. An image processing unit A7 that generates or processes an image may be further provided. FIG. 19A illustrates a configuration in which the transmission apparatus PROD_A includes all of these, but a part of the configuration may be omitted.

  The recording medium PROD_A5 may be a recording of a non-encoded moving image, or a recording of a moving image encoded by a recording encoding scheme different from the transmission encoding scheme. It may be a thing. In the latter case, a decoding unit (not shown) for decoding the encoded data read from the recording medium PROD_A5 according to the recording encoding method may be interposed between the recording medium PROD_A5 and the encoding unit PROD_A1.

  FIG. 19B is a block diagram illustrating a configuration of the receiving device PROD_B in which the video decoding device 1 is mounted. As illustrated in FIG. 19B, the receiving device PROD_B includes a receiving unit PROD_B1 that receives a modulated signal, a demodulating unit PROD_B2 that obtains encoded data by demodulating the modulated signal received by the receiving unit PROD_B1, and a demodulator. A decoding unit PROD_B3 that obtains a moving image by decoding the encoded data obtained by the unit PROD_B2. The moving picture decoding apparatus 1 described above is used as the decoding unit PROD_B3.

  The receiving device PROD_B has a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal for outputting the moving image to the outside as a supply destination of the moving image output by the decoding unit PROD_B3. PROD_B6 may be further provided. FIG. 19B illustrates a configuration in which the reception apparatus PROD_B includes all of these, but a part of the configuration may be omitted.

  The recording medium PROD_B5 may be used for recording a non-encoded moving image, or may be encoded using a recording encoding method different from the transmission encoding method. May be. In the latter case, an encoding unit (not shown) for encoding the moving image acquired from the decoding unit PROD_B3 according to the recording encoding method may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.

  Note that the transmission medium for transmitting the modulation signal may be wireless or wired. Further, the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the transmission destination is not specified in advance) or communication (here, transmission in which the transmission destination is specified in advance). Refers to the embodiment). That is, the transmission of the modulation signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

  For example, a terrestrial digital broadcast broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by wireless broadcasting. Further, a broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by cable broadcasting.

  Also, a server (workstation or the like) / client (television receiver, personal computer, smartphone, etc.) such as a VOD (Video On Demand) service or a video sharing service using the Internet transmits and receives a modulated signal by communication. This is an example of PROD_A / reception device PROD_B (usually, either a wireless or wired transmission medium is used in a LAN, and a wired transmission medium is used in a WAN). Here, the personal computer includes a desktop PC, a laptop PC, and a tablet PC. The smartphone also includes a multi-function mobile phone terminal.

  Note that the client of the video sharing service has a function of encoding a moving image captured by a camera and uploading it to the server in addition to a function of decoding the encoded data downloaded from the server and displaying it on the display. That is, the client of the video sharing service functions as both the transmission device PROD_A and the reception device PROD_B.

  Next, it will be described with reference to FIG. 20 that the above-described moving image encoding device 2 and moving image decoding device 1 can be used for recording and reproduction of moving images.

  FIG. 20A is a block diagram showing a configuration of a recording apparatus PROD_C in which the above-described moving picture encoding apparatus 2 is mounted. As shown in FIG. 20 (a), the recording device PROD_C has an encoding unit PROD_C1 that obtains encoded data by encoding a moving image, and the encoded data obtained by the encoding unit PROD_C1 on the recording medium PROD_M. A writing unit PROD_C2 for writing. The moving image encoding apparatus 2 described above is used as the encoding unit PROD_C1.

  The recording medium PROD_M may be of a type built in the recording device PROD_C, such as (1) HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) SD memory. It may be of a type connected to the recording device PROD_C, such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registration) Or a drive device (not shown) built in the recording device PROD_C.

  In addition, the recording device PROD_C serves as a moving image supply source to be input to the encoding unit PROD_C1. The unit PROD_C5 and an image processing unit C6 that generates or processes an image may be further provided. In FIG. 20A, a configuration in which the recording apparatus PROD_C includes all of these is illustrated, but a part may be omitted.

  The receiving unit PROD_C5 may receive a non-encoded moving image, or may receive encoded data encoded by a transmission encoding scheme different from the recording encoding scheme. You may do. In the latter case, a transmission decoding unit (not shown) that decodes encoded data encoded by the transmission encoding method may be interposed between the reception unit PROD_C5 and the encoding unit PROD_C1.

  Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, and an HDD (Hard Disk Drive) recorder (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is a main supply source of moving images). . In addition, a camcorder (in this case, the camera PROD_C3 is a main source of moving images), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is a main source of moving images), a smartphone (in this case In this case, the camera PROD_C3 or the receiving unit PROD_C5 is a main supply source of moving images) is also an example of such a recording device PROD_C.

  (B) of FIG. 20 is a block showing a configuration of a playback device PROD_D in which the above-described video decoding device 1 is mounted. As shown in FIG. 20 (b), the playback device PROD_D reads a moving image by decoding a read unit PROD_D1 that reads encoded data written on the recording medium PROD_M and a coded data read by the read unit PROD_D1. And a decoding unit PROD_D2 to be obtained. The moving picture decoding apparatus 1 described above is used as the decoding unit PROD_D2.

  Note that the recording medium PROD_M may be of the type built into the playback device PROD_D, such as (1) HDD or SSD, or (2) such as an SD memory card or USB flash memory, It may be of a type connected to the playback device PROD_D, or (3) may be loaded into a drive device (not shown) built in the playback device PROD_D, such as DVD or BD. Good.

  In addition, the playback device PROD_D has a display PROD_D3 that displays a moving image, an output terminal PROD_D4 that outputs the moving image to the outside, and a transmission unit that transmits the moving image as a supply destination of the moving image output by the decoding unit PROD_D2. PROD_D5 may be further provided. FIG. 20B illustrates a configuration in which the playback apparatus PROD_D includes all of these, but a part of the configuration may be omitted.

    The transmission unit PROD_D5 may transmit an unencoded moving image, or transmits encoded data encoded by a transmission encoding method different from the recording encoding method. You may do. In the latter case, it is preferable to interpose an encoding unit (not shown) that encodes a moving image with an encoding method for transmission between the decoding unit PROD_D2 and the transmission unit PROD_D5.

  Examples of such a playback device PROD_D include a DVD player, a BD player, and an HDD player (in this case, an output terminal PROD_D4 to which a television receiver or the like is connected is a main supply destination of moving images). . In addition, a television receiver (in this case, the display PROD_D3 is a main supply destination of moving images), a digital signage (also referred to as an electronic signboard or an electronic bulletin board), and the display PROD_D3 or the transmission unit PROD_D5 is the main supply of moving images. Desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main video image supply destination), laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a moving image) A smartphone (which is a main image supply destination), a smartphone (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a main moving image supply destination), and the like are also examples of such a playback device PROD_D.

(Hardware implementation and software implementation)
Each block of the moving picture decoding apparatus 1 and the moving picture encoding apparatus 2 described above may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or may be a CPU (Central Processing). Unit) may be implemented in software.

  In the latter case, each device includes a CPU that executes instructions of a program that realizes each function, a ROM (Read Only Memory) that stores the program, a RAM (Random Access Memory) that expands the program, the program, and various types A storage device (recording medium) such as a memory for storing data is provided. An object of the present invention is to provide a recording medium in which a program code (execution format program, intermediate code program, source program) of a control program for each of the above devices, which is software that realizes the above-described functions, is recorded so as to be readable by a computer. This can also be achieved by supplying each of the above devices and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include tapes such as magnetic tape and cassette tape, magnetic disks such as floppy (registered trademark) disks / hard disks, CD-ROM (Compact Disc Read-Only Memory) / MO disks (Magneto-Optical discs), and the like. ) / MD (Mini Disc) / DVD (Digital Versatile Disc) / CD-R (CD Recordable) / Blu-ray Disc (Blu-ray Disc: registered trademark) and other optical discs, IC cards (including memory cards) / Cards such as optical cards, mask ROM / EPROM (Erasable Programmable Read-Only Memory) / EEPROM (Electrically Erasable and Programmable Read-Only Memory) / Semiconductor memories such as flash ROM, or PLD (Programmable Logic Device) or FPGA Logic circuits such as (Field Programmable Gate Array) can be used.

  Further, each of the above devices may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited as long as it can transmit the program code. For example, Internet, Intranet, Extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Antenna television / Cable Television) communication network, Virtual Private Network (Virtual Private Network) Network), telephone line network, mobile communication network, satellite communication network, and the like. The transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type. For example, infra-red such as IrDA (Infrared Data Association) or remote control, such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, etc. , Bluetooth (registered trademark), IEEE 802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance), mobile phone network, satellite line, terrestrial digital network, etc. Is possible. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  Note that the image decoding apparatus and the image encoding apparatus according to the present invention can be expressed as follows. That is, the image decoding apparatus according to the present invention predicts inter-frame prediction, which is either uni-prediction that refers to one reference image or bi-prediction that refers to two reference images. In the image decoding apparatus to be restored by the method, the image processing apparatus further comprises merge motion compensation parameter derivation means for deriving a merge candidate list including one or more merge candidates, the merge motion compensation parameter derivation means comprising: Candidate derivation means and bi-prediction application determination means, and the merge motion compensation parameter derivation means, when the bi-prediction application determination means indicates that bi-prediction is applicable, the bi-prediction merge candidate derivation means Is used to derive a merge candidate list and the bi-prediction application determination means indicates that bi-prediction is not applicable. It is configured to derive a merge candidate list using means.

  In addition, the image encoding apparatus according to the present invention performs inter-screen prediction of an image in a prediction unit, either uni-prediction referring to one reference image or bi-prediction referring to two reference images. The image coding apparatus that is restored by the prediction method includes a merge motion compensation parameter deriving unit that derives a merge candidate list including one or more merge candidates, and the merge motion compensation parameter deriving unit is different from the bi-predictive merge candidate deriving unit. A prediction merge candidate derivation unit; and a bi-prediction application determination unit, wherein the merge motion compensation parameter derivation unit includes the bi-prediction merge candidate when the bi-prediction application determination unit indicates that bi-prediction is applicable. When the merge candidate list is derived using the means, and the bi-prediction application determination means indicates that bi-prediction is not applicable, the single prediction merge candidate means is It is configured to derive the merge candidate list have.

  The present invention can be suitably applied to an image decoding apparatus that decodes encoded data obtained by encoding image data and an image encoding apparatus that generates encoded data obtained by encoding image data. Further, the present invention can be suitably applied to the data structure of encoded data generated by an image encoding device and referenced by the image decoding device.

1 video decoding device (image decoding device)
DESCRIPTION OF SYMBOLS 10 Decoding module 12 PU information decoding part 16 Frame memory 121 Motion compensation parameter derivation part 1021 Skip information decoding part 1022 Motion information decoding part 1212 Merge motion compensation parameter derivation part 1212A Merge candidate derivation control part (merge candidate list derivation means)
1212B Merge candidate storage unit 1212C Merge candidate selection unit 1212D Merge candidate deriving unit 1212D1 Spatial merge candidate deriving unit 1212D2 Temporal merge candidate deriving unit 1212D3 Join merge candidate deriving unit (bi-predictive merge candidate deriving unit)
1212D4 Zero merge candidate derivation unit 1212D5 Correction merge candidate derivation unit (single prediction merge candidate derivation means)
1212E Slice type determination unit (bi-prediction application determination means, slice type determination means)
1212F Merge candidate derivation control unit 1212G Bi-prediction restriction determination unit (bi-prediction application determination means, bi-prediction restriction determination means)
2 Video encoding device (image encoding device)
25 frame memory 29 encoded data generation unit 30 PU information generation unit 301 motion compensation parameter generation unit 3013 merge motion compensation parameter generation unit (bi-prediction merge candidate derivation unit, uni-prediction merge candidate derivation unit, bi-prediction application determination unit, merge candidate List derivation means)

Claims (11)

In an image decoding apparatus that restores an image in a prediction unit by a prediction method of inter prediction of either single prediction referring to one reference image or bi-prediction referring to two reference images,
Bi-predictive merge candidate derivation means for deriving bi-predictive merge candidates;
Uni-predictive merge candidate derivation means for deriving one or more uni-predictive merge candidates;
Bi-prediction application determination means for determining whether bi-prediction is applicable;
When bi-prediction is applicable, a merge candidate list is derived using the bi-prediction merge candidate derivation unit, whereas when bi-prediction is not applicable, the single prediction merge candidate derivation unit is used. An image decoding apparatus comprising: merge candidate list deriving means for deriving a merge candidate list including one or more merge candidates. The bi-predictive merge candidate derivation means derives merge candidates based on motion vectors of two different merge candidates,
The image decoding apparatus according to claim 1, wherein the single prediction merge candidate derivation unit derives merge candidates based on a single merge candidate.   The merge candidate list derivation means uses the bi-prediction merge candidate derivation means when bi-prediction is applicable, and derives the merge candidate list without using the single prediction merge candidate derivation means, 3. The merge candidate list is derived using the single prediction merge candidate derivation means and not using the bi prediction merge candidate derivation means when bi-prediction is not applicable. The image decoding apparatus described in 1. The bi-prediction application determination unit is a slice type determination unit that determines a slice type including a target prediction unit,
The slice type determination means determines that bi-prediction is applicable when the slice type is B slice, and determines that bi-prediction is not applicable when the slice type is P slice. The image decoding device according to any one of 1 to 3. The bi-prediction application determination unit is a bi-prediction limitation determination unit that determines whether bi-prediction limitation is applied to a target prediction unit.
The bi-prediction restriction determination means determines that bi-prediction is applicable when the size of the target prediction unit is greater than or equal to a predetermined size, and applies bi-prediction when the size of the target prediction unit is smaller than the predetermined size. The image decoding device according to claim 1, wherein the image decoding device is determined to be impossible.   6. The uni-predictive merge candidate derivation unit determines a reference motion compensation parameter and derives the uni-predictive merge candidate by adjusting the reference motion compensation parameter. The image decoding device according to item.   The image decoding according to claim 6, wherein the reference motion compensation parameter is a motion compensation parameter corresponding to a merge candidate to which a minimum index is assigned among merge candidates included in a merge candidate list. apparatus.   The adjustment of the base motion compensation parameter is an adjustment by adding an offset to a motion vector associated with the reference image list L0 or the reference image list L1 in the base motion compensation parameter. The image decoding device described.   The adjustment of the standard motion compensation parameter is an adjustment in which the reference image index associated with the reference image list L0 or the reference image list L1 in the standard motion compensation parameter is the smallest value different from the reference image index. The image decoding apparatus according to claim 6, wherein: In the adjustment by adding the offset, the offset is selected from an offset list,
9. The image decoding apparatus according to claim 8, wherein the offset list is selected from a plurality of predetermined offset lists based on a direction or magnitude of a motion vector in the reference motion compensation parameter. . Encodes information indicating whether the image in the prediction unit should be restored by the prediction method of the single prediction referring to one reference image or the bi-prediction referring to two reference images. In an image encoding device,
Bi-predictive merge candidate derivation means for deriving bi-predictive merge candidates;
Uni-predictive merge candidate derivation means for deriving one or more uni-predictive merge candidates;
Bi-prediction application determination means for determining whether bi-prediction is applicable;
When bi-prediction is applicable, a merge candidate list is derived using the bi-prediction merge candidate derivation unit, whereas when bi-prediction is not applicable, the single prediction merge candidate derivation unit is used. An image coding apparatus comprising: merge candidate list deriving means for deriving a merge candidate list including one or more merge candidates.

Images