JPH07264597A - Motion compensation method - Google Patents

Abstract

(57) [Abstract] [Purpose] For a motion compensation method, improve inaccurate region segmentation and increase coding efficiency. A region dividing unit 3 divides an image 7 at the current time into a region by an in-frame region dividing unit 8 by threshold processing based on a luminance signal and a color difference signal in the frame, and each of the divided regions in a motion estimating unit 10. Motion estimation is performed by referring to the image 11 at the previous time using the region as a block, the region integration unit 13 refers to the motion vector 12 to update the region division, and the region correction unit 15 updates the region division image 14 by the edge extraction operator. The final area-divided image 1
6 is output. The predicted image creation unit 20 creates a motion-compensated predicted image 29 and data 33 indicating the code amount based on the final area-divided image 16.

Description

Detailed Description of the Invention

[0001]

FIELD OF THE INVENTION The present invention relates to a motion compensation method.

[0002]

2. Description of the Related Art Conventionally, as a region dividing method for performing motion compensation of an image, a region dividing method using a luminance signal or a color difference signal of an image, a region dividing method using motion information between two frames, and a region using an edge extraction filter. There was a division method.
For example, a region segmentation method using a K-means algorithm using a luminance signal and a color difference signal is often used as the region segmentation method.

[0003]

Among the conventional methods, in the area division method using only motion information, the object (object)
When a part of the image is stationary, it is not extracted as a region, and may become different objects in successive image frames. In the area division method using the luminance signal, the contour of the foreground object cannot be accurately obtained when the luminance of the foreground object and the background luminance are the same in a certain frame. Since the affine transformation is a parameter that describes the deformation of the area when the shape (area) of the object is obtained accurately, if the shape of the object collapses in this way, the parameters of the affine transformation become inaccurate. Interframe prediction efficiency is reduced.

Further, in the area dividing method using the K-averaging algorithm using the luminance signal and the color difference signal, since the motion information is not reflected, a large number of objects are detected even when there is a pattern on the stationary background. It When the parameters of each motion are coded and transmitted, the coding efficiency decreases.

Further, in the area division method using the edge detection filter or the contour extraction filter, fine edge detection is possible, but it is difficult to detect where the boundary of the object is. In general, the edge detection filter does not determine the closed area, and therefore a large number of fine lines with broken ends appear.

An object of the present invention is to provide a motion compensation method which improves the conventional inaccurate area division and has high coding efficiency.

[0007]

According to the motion compensation method of the present invention, a plurality of types of area division methods are given a rank to which the area division method is applied, and the area division methods are sequentially applied according to the rank. The divided areas are updated, and a predicted image is created based on the final area divided image.

[0008]

According to the invention of claim 1, an image signal in a frame, 2
A case where the motion information between frames and each area division method of the edge extraction filter are used will be described.

First, the order of application is given to the image signal in a frame, the motion information between frames, and the respective area division methods by the edge extraction filter. The present invention is based on the area division obtained by the highest priority area division method and
This is a method of sequentially applying the area division method and updating the divided areas.

Now, the application order will be described assuming that the intra-frame image signal, the motion information, and the edge extraction filter are in this order. Note that here, a luminance signal and a color difference signal are applied as the in-frame image signal. First, the area is divided based on threshold processing based on the luminance signal. Next, threshold value processing is performed on each of the obtained areas based on the color difference signal, and areas in which the color difference value changes beyond the threshold value are further divided. However, here, it is also possible to adopt a method of simultaneously handling a luminance signal and a color difference signal, such as area division using a K-averaging algorithm. Next, the motion vector of each area is obtained between the frame that divides the current area and the temporally continuous frame, and adjacent areas in which the difference between the motion vectors falls within a certain threshold are integrated. Finally, an edge extraction operator is applied to the current frame to perform processing such as deleting / correcting an area boundary that does not match the extracted edge to obtain a final area division.

The method described above is merely an example. By changing the application order of the three types of area division methods, various area division methods can be obtained. For example, when prioritizing the area division method based on the motion information, the area division of the initial screen is sequentially updated according to the movement information, and based on the area division of the current frame, the luminance signal, the color difference signal, and the edge information are used. It is also possible to take a method such as correcting the area boundary. Even if the application order of the three types of area division methods is the same, it is possible to obtain an alternative method by changing each area division algorithm. Further, two kinds of the above three kinds of area division methods may be combined, and further area division methods other than the above three kinds of area division methods may be combined.

Next, the invention of claim 2 will be described.
As described above, there are many area division methods according to claim 1. Therefore, several kinds of these are prepared in advance, identification information is attached to each, and the identification information of the used area division method is transmitted to the decoding side. With this information, the same area division can be performed on the encoding side and the decoding side even when a method is adopted in which an image reproduced by a local decoder instead of the original image is divided and the area information is not sent.

Next, the invention of claim 3 will be described.
The amount of code that combines the region information, motion information, and prediction error information is calculated based on the region division obtained by several types of region division methods prepared in advance, and the region division with the smallest amount of encoded information is calculated from these data. By determining the method and transmitting the predicted image and the identification information to the decoding side, it is possible to select the most efficient area division method.

Therefore, according to the present invention, it is possible to accurately segment an object region, improve prediction efficiency by motion compensation, reduce redundant region segmentation of a non-object region, and reduce region information. To be done.

[0015]

Embodiments of the present invention will now be described with reference to the drawings.

FIG. 1 is a block diagram showing a motion compensation method according to an embodiment of the present invention, and FIGS. 2 and 3 are claims 1, 2 and 3, respectively.
It is a flowchart which shows the process corresponding to 3.

In this embodiment, an area division method using an in-frame image signal (hereinafter referred to as area division method A), an area division method based on motion information between two frames (hereinafter referred to as area division method B), and edge extraction. The area division is performed by using three area division methods, which are an area division method using a filter (hereinafter, referred to as an area division method C).

(Embodiment of claim 1) The area dividing unit 3
The area division method is applied in the order of A, B, and C (step 4
1, 42), the predicted image creating unit 20 receives the final area-divided image 16 output from the area dividing unit 3, creates the predicted image 29 and the code amount data 33, and outputs (Step 43).

The processing of the area dividing section 3 will be described in detail. First, the image 7 at the current time is divided into regions by the intra-frame region dividing unit 8 by threshold processing based on the luminance signal and the color difference signals in the frame. Algorithms such as the separation / integration method and K-means clustering are used for the threshold processing. The information 9 of the area division obtained here is the motion estimation unit 10.
The divided regions are sent as blocks and the motion estimation is performed with reference to the image 11 at the previous time. Next, the region integration unit 13 refers to the obtained motion vector 12, integrates regions having similar vectors included within a certain threshold, and updates the region division. The updated area-divided image 14 is further modified in the area modification unit 15 with reference to the edge obtained by the edge extraction operator,
The final area division image 16 is output to the prediction image creation unit 20. Next, the processing of the predicted image creating unit 20 will be described.
Based on the area division image 16 obtained from the area division unit 3, the motion estimation / compensation unit 24 estimates a motion vector and creates a motion compensation prediction image. The predicted image 25 created here is stored in the frame memory 26. Further, the code amount calculation unit 28 calculates the code amount indicating the divided area of the image 16, the code amount of the motion vector 27, and the code amount of the prediction error signal, and outputs the combined data 33.

(Embodiment 2) In addition to the area dividing section 3, area dividing sections 4, 5, 6 are provided, and the area dividing sections 4, 5, 5 are provided.
The predicted image creating units 21, 22, and 23 are provided corresponding to 6 (step 51).

The area dividing unit 4 applies the area dividing method in the order of B, C, A, the area dividing unit 5 applies the area dividing method in the order of C, A, B, and the area dividing unit 6 , A, C, B are applied in the order of the region division method, and the prediction image creation units 21, 22,
23 inputs the final area-divided images 17, 18, and 19 output from the area-division units 4, 5, and 6, respectively, and predicts images 30, 31, 32 and data 34, 35, 3 of the encoding amount.
Create 6.

The area dividing units 4, 5 and 6 also perform the same processing as that of the area dividing unit 3, and the predicted image creating units 21, 22 and 23 perform the same processing as the predicted image creating unit 20.

When the area division method designation information 1 (identification information) is input to the area division control unit 2, the area division control unit 2 activates the area division unit designated by the area division method designation information 1 to perform the area division. Then, the predicted image creating unit connected to the area dividing unit creates the predicted image and the data of the code amount (step 52). The generated predicted image and code amount data are sent to the decoding side (not shown) together with the area division method designating information 1 (step 53). On the decoding side, the same area division as on the encoding side is performed in the same order.

(Claim 3) A plurality of area dividing units 3
6 and the predicted image creating units 20 to 23 are prepared in advance as in the above-described embodiment (step 61). next,
An activation signal is sequentially output from the area division control unit 2 to the area division units 3, 4, 5, and 6, and the area division and prediction image creation units 20, 21, 22, and 23 by the area division units 3, 4, 5, and 6 are performed. The predicted images 29, 30, 31, 32 and the code amount data 33, 34, 35, 36 are generated (step 62). Next, the area division method determination unit 37 inputs these encoding amount data 33, 34, 35, 36, and compares them with each other to obtain information indicating the area division unit having the smallest amount of encoded information and the area division. Motion compensated prediction image 38
Is output to the decoding side (step 63).

In the above embodiment, the area division method A,
An example of combining all B and C has been explained, but these 3
Any two of the two area division methods A, B and C may be combined, or area division methods other than the area division methods A, B and C may be combined.

[0026]

【The invention's effect】

(1) According to the first aspect of the present invention, since a plurality of different types of area division methods are applied in a specified order to update the area division, an area division that accurately cuts out an object area can be obtained. Since the motion estimation of the object is accurate, the prediction efficiency is improved, and the redundant area division of the non-object area can be reduced, the area information is reduced. As a result, efficient encoding can be realized. (2) In the invention of claim 2, an arbitrary number of combinations of a plurality of types of area division methods are prepared, and any one of these combinations is selected to select an arbitrary combination of area division methods. can do. (3) The invention of claim 3 further prepares a plurality of combinations of a plurality of types of area division methods, and selects a method with the smallest amount of coded information from these area division results, so that a more efficient moving image is obtained. Encoding can be realized.

[Brief description of drawings]

FIG. 1 is a block diagram showing a motion compensation method according to an embodiment of the present invention.

FIG. 2 is a flowchart showing claim 1.

FIG. 3 is a flowchart showing claim 2.

FIG. 4 is a flowchart showing claim 3.

[Explanation of symbols]

 1 area division method identification information 2 area division control section 3 to 6 area division section 7 input image at current time 8 intra-frame area division section 9 initial area division image 10 motion estimation section 11 input image at previous time 12 motion vector information 13 Region integration unit 14 Region division information after region integration 15 Region correction unit 16 to 19 Final region division image 20 to 23 Prediction image creation unit 24 Motion estimation / compensation unit 25 Prediction image 26 Frame memory 27 Motion vector information 28 Code amount Calculation unit 29 to 32 Predicted image 33 to 36 Code amount data 37 Region division method determination unit 38 Identification information of adopted region division method and predicted image 41 to 43, 51 to 53, 61 to 63 steps

Claims (4)

[Claims] 1. A motion compensation method that divides an image into several regions and applies a parameter representing motion to each region to create a predicted image, which is applicable to a plurality of types of region segmentation methods. A motion compensation method in which the order of applying the area division method is set, the area division methods are sequentially applied according to the order to update the divided areas, and a prediction image is created based on the final area division image. 2. A region segmentation method for applying a plurality of types of region segmentation methods in a predetermined order, even if different types of region segmentation methods are combined or the same region segmentation methods are combined. Is prepared, a plurality of area division methods with different orders are prepared, and one of these area division methods is designated to perform area division and subsequent prediction image creation, and the created prediction image and the specified The motion compensation method according to claim 1, wherein identification information indicating the generated area division method is transmitted to the decoding side. 3. A region segmentation method for applying a plurality of types of region segmentation methods in a predetermined order, even if different types of region segmentation methods are combined or the same region segmentation methods are combined. A plurality of area division methods with different order of application are prepared, area division by these area division methods and subsequent prediction image creation are executed, and the area division with the minimum amount of coding information calculated in the prediction image creation is performed. The motion compensation method according to claim 1, wherein the prediction image created by the method and the identification information indicating the area division method are transmitted to the decoding side. 4. As the plurality of types of area division methods,
4. The method according to claim 1, which includes at least one area division method using an image signal in a frame, an area division method using motion information between two frames, and an area division method using an edge extraction filter. The described motion compensation method.