JP2005277968A - Method for image coding and decoding - Google Patents

Abstract

PROBLEM TO BE SOLVED: In a conventional encoding method, when the boundary line of a moving object is in an oblique direction, the boundary line and the dividing line of the block do not coincide with each other, and the efficiency of motion compensation is lowered.
A division control unit has a plurality of methods for dividing a macroblock, including a method for dividing a macroblock by an arbitrary line segment. The division control unit 102 instructs the division unit 103 in order of a plurality of division methods. The dividing unit 103 divides the macroblock into divided regions according to the instructed method, and the motion detecting unit 104 performs motion detection for each divided region to obtain a motion vector and an evaluation value. The division control unit 102 divides and encodes the macroblock using a division method that optimizes the evaluation value obtained by the motion detection unit 103. The division method is described in the code string by the division method description unit 114.
[Selection] Figure 1

Description

  The present invention relates to an image encoding method and an image decoding method when performing high-efficiency compression encoding of a moving image signal using motion compensation.

In a conventional image encoding method typified by the MPEG video method, a screen is divided into predetermined units, and encoding is performed in the divided units. For example, in the MPEG-4 AVC (Advanced Video Coding) system (for example, see Non-Patent Document 1), a screen (picture) is processed in units of 16 horizontal pixels and 16 vertical pixels called macroblocks. When motion compensation is performed, the macroblock can be divided into rectangular blocks (minimum 4 horizontal pixels and 4 vertical pixels), and motion compensation can be performed using different motion vectors for each block.
ISO / IEC 14496-10 Advanced Video Coding Standard

  However, since the macro block is divided into rectangles in the above-described conventional method, for example, when the boundary line of the moving object is in an oblique direction, the boundary line does not match the block division line. In such a case, two regions having different motions exist in the block. For this reason, when motion compensation is performed using the motion of one region, motion compensation is performed with a different motion for the other region. For this reason, there has been a problem that the efficiency of motion compensation is lowered, and as a result, the coding efficiency is lowered.

  The present invention solves the above-described conventional problem, and is a moving image code capable of performing motion compensation in consideration of the boundary of a moving object even when the screen is divided into macro blocks and encoded. It is an object to provide an encoding method and a moving image decoding method.

  In order to solve this problem, the first invention is an image encoding method in which an input image is divided into blocks having a predetermined size, and an encoding process is performed in units of the blocks. Further, a division control step for instructing a method of dividing into a plurality of divisions, a division step for dividing the block by a division method instructed by the division control step to generate a divided region, and each divided region generated by the dividing step A motion detection step for performing motion estimation for the motion vector and detecting a motion vector, and a motion compensation step for generating a motion compensation image for each of the divided regions based on the motion vector detected by the motion detection step. As a division method instructed by the division control step, at least the division method of dividing the block by an arbitrary line segment An image coding method, which comprises a.

  A second invention is an image decoding method for decoding an input image into blocks having a predetermined size and decoding a code string generated by performing an encoding process in units of the blocks, A division method decoding step for obtaining a division method in the block described in a code string; a division step for dividing the block by a division method obtained by the division method decoding step and generating a divided region; A motion compensation step for generating a motion compensated image for each of the divided regions using a motion vector obtained from the code string, and dividing the block by an arbitrary line segment in the division method An image decoding method including the method.

  As described above, in the image encoding method and the image decoding method of the present invention, it is possible to perform diagonal division that could not be handled only by conventional rectangular division, and more suitable division for the shape of the moving object boundary. As a result, the code amount of the residual signal after motion compensation can be greatly reduced, and its practical value is high.

Hereinafter, embodiments of the present invention will be described with reference to FIGS.
(Embodiment 1)
FIG. 1 is a block diagram of an image encoding apparatus using the image encoding method of the present invention. As illustrated in FIG. 1, the image encoding device 100 includes a frame memory 101, a division control unit 102, a division unit 103, a motion detection unit 104, a reference picture memory 105, a difference unit 106, a conversion unit 107, and a quantization unit 108. , Variable length coding section 109, inverse quantization section 110, inverse transform section 111, addition section 112, switch 113, division method description section 114, and division method holding section 115.

  The input image is first held in the frame memory 101. Also, it is assumed that the decoded picture of the encoded picture is already stored in the reference picture memory 105, and this is used as a reference picture when the input picture is encoded.

The input image held in the frame memory 101 is divided into macro blocks each having a square area of 16 horizontal pixels and 16 vertical pixels, and is processed for each macro block.
At the start of processing of each macroblock, the switch 113 is connected to “a”. Therefore, the image of each macroblock is input to the dividing unit 103 and divided into a plurality of areas. How the division unit 103 divides the macroblock is determined by the division control unit 102. An example of the division method determined by the division control unit 102 is shown in FIG. As shown in FIG. 2, the division method determined by the division control unit 102 is roughly divided into rectangular division and arbitrary division. As shown in FIGS. 2A to 2D, there are four division methods for rectangular division.

  An arbitrary division method will be described with reference to FIG. When arbitrary division is performed, as shown in FIG. 3A, the macroblock is divided into two regions by a line segment passing through any two points among the candidate division points of point a to point p. In FIG. 3A, a region surrounded by a wavy line indicates one pixel. Here, for convenience, an area of 8 pixels in the vertical direction and 8 pixels in the horizontal direction is shown, but the size of the macroblock may be other sizes. In addition, what is necessary is just to distribute to the area | region about the pixel located on a line segment by the method defined beforehand. For example, classification into a region to which a larger area within a pixel belongs, classification into a region that is above (or below) a two-dimensional plane from a line segment, and classification into two regions alternately. In FIG. 3 (a), a case is shown in which the restriction that the line segment passes through the dividing point candidates every 2 pixels is provided, but this may be every 1 pixel, every 4 pixels, etc. It may be non-equally spaced. Further, the position of the dividing point does not have to be between adjacent pixels, and may be on the pixel. In the division point candidate of FIG. 3A, when the line segment passes through the points c and o, the macroblock is divided into regions 301 and 302 as shown in FIG. 3B. When the line segment passes through the points h and p, the macroblock is divided into regions 303 and 304 as shown in FIG.

FIG. 4 is a flowchart showing a flow of operations when determining a macroblock division method and a motion vector.
The division control unit 102 instructs the division unit 103 in order of the division method described above. Then, the division unit 103 divides the macroblock by the division method instructed by the division control unit 102, and generates a divided region image (step S101). Then, each divided region image is output to the motion detection unit 104.

  The motion detection unit 104 performs motion detection on the reference picture held in the reference picture memory 105 for the divided region image input from the division unit 103 (step S102). Motion detection detects the position where the evaluation value is optimal (minimum or maximum) within a predetermined range in the reference picture (for example, a rectangular area of ± 32 pixels in the horizontal direction and ± 24 pixels in the vertical direction). To do. For example, if the evaluation value is the sum of difference values between the divided region image and the corresponding image (reference image) in the reference picture, the position in the reference picture where the evaluation value is minimum is detected. Further, as the evaluation value, a weighted sum of a difference value sum and a code amount of a motion vector (difference between the position of the divided region image in the frame and the position of the reference image in the reference picture) can be used. The motion detection unit 104 performs motion detection on each divided region image in the macroblock, obtains an optimal motion vector and evaluation value in the divided region, and adds the evaluation value for each divided region (step S103). The total evaluation value in the macro block is obtained. Then, the total evaluation value and the motion vector are output to the division control unit 102.

  In the division control unit 102, the total evaluation value input from the motion detection unit 104 is compared with the current optimum value (step S104), and when the value is better than the optimum value (for example, when the minimum is used as the optimum criterion) If the value is smaller than the optimum value), the total evaluation value is replaced with the optimum value, and the division method and motion vector in this case are held (step S105). Here, the optimum value for the total evaluation value is reset (initialized) for each macroblock.

  The division control unit 102, the division unit 103, and the motion detection unit 104 sequentially perform the above-described operations for the division method instructed by the division control unit 102. Then, after processing for all the division methods, a division method and a motion vector held as a method for giving an optimum total evaluation value are determined as a division method and a motion vector for the macroblock (step S106).

  When the final division method of the macroblock is determined by the division control unit 102, the switch 113 is connected to “b”, and the input macroblock image is input to the difference unit 106. Then, the determined division method and motion vector are transmitted to the reference picture memory 105, and a reference image for each divided region is output to the difference unit 106. For example, when the macroblock 500 in FIG. 5 is arbitrarily divided into the region 501 and the region 502, the reference image 503 and the reference image 504 are acquired from the reference picture, and the motion compensated image MC for the input macroblock is generated. become. The determined division method DI is also output to the division method description unit 114, and the motion vector MV is also output to the variable length encoding unit 109.

The difference unit 106 calculates a difference between corresponding pixels of the input macroblock image and a reference image (an image obtained by synthesizing a reference image for each divided region) MC, and generates a residual image RS.
The residual image RS is subjected to frequency conversion in the conversion unit 107, and then subjected to quantization processing in the quantization unit 108 to become a quantized transform coefficient QC, which is a variable length encoding unit 109 and an inverse quantization unit. 110.

  The inverse quantization unit 110 performs inverse quantization on the quantized transform coefficient QC, and the inverse transform unit 111 performs inverse frequency conversion, and as a result, a decoded residual signal DR is generated. This is added to the reference picture MC output from the reference picture memory 105 in the adding unit 112 to become a decoded picture DC, which is held in the reference picture memory 105 for use as a reference picture at the time of subsequent picture coding. Is done.

  On the other hand, the division method DI is input from the division control unit 102 to the division method description unit 109. The division method description unit 109 generates a code string DIC indicating the division method DI. 6A and 6B show examples of code string formats.

  FIG. 6A shows the first format of the code string. First, the division method 601 is described. This is described using the code table A in FIG. When the division method is arbitrary division (that is, when division method 601 is code number 4), division line segment information 602 is further described. A description method of the dividing line segment information 602 will be described later.

  FIG. 6B shows a second format of the code string. First, the division method 603 is described. For this description, the code table B of FIG. 6C is used. When the division method is rectangular division (that is, when division method 603 is code number 0), division method 604 is further described. For this description, the code table C in FIG. 6C is used. When the division method is arbitrary division (that is, when division method 603 is code number 1), division line segment information 602 is further described.

  A description method of the dividing line segment information 602 will be described with reference to FIG. The symbol of the position through which the line segment passes is as shown in FIG. If the line segment corresponds to the division shown in FIG. 7A, it is described that the line segment passes through the point d and the point l. Here, it is assumed that numbers 0 to 15 are sequentially assigned to the points a to p.

As a first description method, the number (3) of the first division point (point d) and the number (11) of the second division point (point l) are described as shown in FIG. 7B.
As the second description method, as shown in FIG. 7C, the number (3) of the first dividing point (point d) and the first dividing point (point d) to the second dividing point (point l). The number difference (8) is described.

  In the third to sixth description methods, division information of neighboring macroblocks is used. This is because the moving object boundary often spans macroblocks, and it is considered that there is continuity (correlation) between the dividing method of the neighboring macroblocks and the dividing method of the current macroblock. Here, when the third to sixth description methods are used, the division method description unit 114 holds the division method of each macroblock in the division method holding unit 115, and the division method holding unit 115 divides the neighboring macroblocks. Let's get a method.

  The third and fourth description methods will be described with reference to FIG. 7D, and the fifth and sixth description methods will be described with reference to FIG. In FIG. 7D and FIG. 7G, the division method of the left and upper macroblocks (assuming that these have already been encoded) is used. In this case, if there is a division point on the line contacting the current macroblock of each of the left and upper macroblocks, that point is set as the initial point.

  For example, in FIG. 7D, since the point c is the point, the point c is set as the first initial point. In the third description method, as shown in FIG. 7E, the number difference (1) from the first initial point (point c) to the first division point (point d) and the first division point (point The number difference (8) from d) to the second division point (point l) is described. In terms of probability, the line segment is considered to have the highest probability of dividing the macroblock into two equal parts. Therefore, it is considered that the code amount of the second division point can be further reduced by setting the point symmetric position from the first division point to the center of the macroblock as the second initial point. If the first division point is point d, the second initial point is point l. Therefore, in the fourth description method, as shown in FIG. 7F, the number difference (1) from the first initial point (point c) to the first division point (point d) and the second initial point ( The number difference (0) from the point l) to the second division point (point l) is described.

  Next, as shown in FIG. 7G, when there are division points on both the left and upper macroblocks and the line tangent to the current macroblock, in the fifth and sixth description methods, respectively, ), First, initial point selection information is described as shown in FIG. That is, it is described which point c or point o is selected as the initial point. For example, if it is 0, the intersection (point o) of the dividing line with the left macroblock may be used, and if it is 1, the intersection (point c) of the dividing line with the upper macroblock may be used. The subsequent description methods may be described in the same manner as the third and fourth description methods, respectively.

  Here, when neither the left macroblock nor the upper macroblock has a dividing point on the line in contact with the current macroblock, a predetermined point may be set as the first initial point. For example, the point a can be used as the first initial point as a predetermined point.

  In the above description method, the description method of FIG. 7B has an advantage that it can be described by an easy method, and FIG. 7C, FIG. 7E, FIG. 7F, and FIG. As the description method of FIG. 7 (i) is used, the description method becomes more complicated and the amount of processing increases. However, the code amount can be reduced accordingly.

The code string DIC representing the division method generated by the division method description unit 114 is output to the variable length coding unit 109.
The variable length coding unit 114 performs variable length coding on the quantized transform coefficient QC output from the quantization unit 108 and the motion vector MV output from the division control unit 102, and the division method description unit 114. Is combined with a code string DIC that represents the division method generated by the above, and a code string is output.

  As described above, the image encoding method of the present invention divides an input image into macroblocks that are square regions when encoding the input image. Then, the macroblock is divided into regions by a predetermined division method, motion detection is performed on each divided region, and a division method with the best evaluation value is selected. Here, the dividing method includes a method of dividing a macroblock by an arbitrary line segment. The code string describes the division method. When division by an arbitrary line segment is selected, the line segment is expressed by describing the position information of the point through which the line segment passes in the code string. To do.

  By such an operation, it becomes possible to perform an oblique division and the like that could not be handled only by the rectangular division in the conventional macroblock, and it is possible to perform a division more suitable for the shape of the boundary of the moving object. Thereby, the code amount of the residual signal after motion compensation can be greatly reduced. Furthermore, the description of the division method is performed by describing the passing points of the line segments, and the description can be performed with a small code amount. Conventionally, a method of dividing so as to conform to the shape of the moving object boundary has been proposed. However, in the present invention, processing suitable for the object boundary can be performed in units of macroblocks. Since the same methods can be used for the methods (transform coding, quantization, rate control, mode selection, etc.) used in the method, there is also an advantage that the affinity with the conventional method is high.

  In the embodiment of the present invention, the case where a macroblock is divided into two by one line segment at the time of arbitrary division has been described, but this is divided into three or more by two or more line segments. Also good.

  In the embodiment of the present invention, the case where the macroblock is divided into line segments has been described. However, for example, the macroblock may be divided into smaller units. For example, the sub-macroblock (FIG. 2 (d)) obtained by dividing the macroblock into four may be divided by different line segments. Thereby, division according to the shape of the moving object boundary can be performed with higher accuracy.

In the embodiment of the present invention, the case where the number of reference images is 1 has been described, but two or more pictures may be used as reference images.
In the embodiment of the present invention, the case where motion detection is performed for all division methods to obtain evaluation values has been described. However, this may not be tried for all division methods. For example, it may be performed only for a typical division method, and motion detection may be performed only for a division method close to a division method with a low evaluation value. In this case, for example, as shown in FIG. 8A, first, motion detection is performed on the division using the line segment ai, the line segment ck, the line segment em, and the line segment go, and an evaluation value is obtained. Select the division method. If the line segment ai is a dividing method that gives an optimum evaluation value, then, as shown in FIG. 8B, motion detection is performed for the division using the line segment bj and the line segment hp, and the evaluation value is obtained. The optimum dividing method including the line segment ai is selected as the final dividing method. By doing in this way, the processing amount for determining a division | segmentation method can be reduced. Moreover, in the example of FIG. 8, although it is limited so that it may select from the method of dividing a macroblock into two equal parts, it is not limited to it.

  Further, in the embodiment of the present invention, when a macroblock is arbitrarily divided, a case has been described in which two points through which a line segment passes are described in a code string. An idea (y = ax + b), its inclination a and intercept b may be described.

(Embodiment 2)
FIG. 9 is a block diagram of an image decoding apparatus 900 using the image decoding method of the present invention. An image decoding apparatus 900 includes a variable length decoding unit 901, an inverse quantization unit 902, an inverse transform unit 903, an addition unit 904, a division method decoding unit 905, a motion compensation unit 906, a reference picture memory 907, and a division method The holding unit 908 is configured.

  It is assumed that the input code string BS is generated by an image encoding device using the image encoding method of the present invention. In addition, it is assumed that a decoded picture that has been decoded has already been stored in the reference picture memory 907 and is used as a reference picture when decoding a code string.

  The input code string BS is input to the variable length decoding unit 901. The variable length decoding unit 901 performs variable length decoding on the quantized transform coefficient QC, the coded macroblock division method DIC, the motion vector MV, and the like. The quantized transform coefficient QC is output to the inverse quantization unit 902, the coded macroblock division method DIC, the division method decoding unit 905, and the motion vector MV to the motion compensation unit 906.

The quantized transform coefficient QC is inversely quantized by the inverse quantizing unit 902, inversely transformed by the inverse transforming unit 903, becomes a decoded residual image DR, and is output to the adding unit 904. .
The division method decoding unit 905 performs decoding on the encoded macroblock division method DIC. Here, the coded macroblock division method DIC is coded (described) in the code string format of FIG. 6A or 6B. Here, which code sequence format is used may be determined in advance by the encoding device and the decoding device, and which code sequence format is used in the header or the like. Also good.

  When the code string format shown in FIG. 6A is used, the division method decoding unit 905 first processes the division method 601. Since the contents of the division method 601 are described by, for example, the code table A in FIG. 6C, when the division method 601 is code numbers 0 to 3, the macroblock is divided by rectangular division. to decide. In this case, no further processing of the code string is performed. On the other hand, if the division method 601 is code number 4, it is determined that the macroblock is arbitrarily divided, and the divided line segment information 602 is further processed. A method of processing the dividing line segment information 602 will be described later.

  When the code string format shown in FIG. 6B is used, the division method 603 is processed. This content is described by, for example, the code table B in FIG. 6C. Therefore, when the division method 603 is code number 0, the division method 603 determines that the division is rectangular and further performs division. Process 604 is processed. For example, the code table C in FIG. 6C is used for the description of the division method 604, and the division method (number of divisions) is determined from the code number. If the division method 603 is code number 1, it is determined that the division method is arbitrary division, and the division line segment information 602 is further processed.

  A method of processing the dividing line segment information 602 will be described with reference to FIG. The dividing line segment information 602 is described in any one of the encoding formats of FIGS. 7B, 7C, 7E, 7F, 7H, and 7I. In any of the methods, the description is made according to which two points (the first division point and the second division point) the line segment dividing the macroblock passes. Here, which code sequence format is used may be determined in advance by the encoding device and the decoding device, and which code sequence format is used in the header or the like. Also good. Further, FIG. 7 (e) (or (f)) and FIG. 7 (h) (or (i)) are uniquely determined as to which one to use according to the division method of the peripheral macroblock as will be described later. So it can be used at the same time.

The symbol of the position through which the line segment passes is as shown in FIG. In addition, numbers 0 to 15 are assigned to the points a to p in order.
In the first method, as shown in FIG. 7B, the number of the first division point and the number of the second division point are described. A division point is obtained from these division point numbers.

  In the second method, as shown in FIG. 7C, the number of the first division point and the number difference from the first division point to the second division point are described. Therefore, after obtaining the number of the first division point, the number of the second division point is obtained by adding the number difference from the first division point to the second division point to the number of the first division point, A division point is obtained from these division point numbers.

  In the third to sixth description methods, division points are described using division information of neighboring macroblocks. Here, when the third to sixth description methods are used, the division method decoding stage 905 holds the division method of each macroblock in the division method holding unit 908, and the division method holding unit 908 stores the macroblocks of the neighboring macroblocks. A division method shall be obtained.

  In FIG. 7D and FIG. 7G, the division method of the left and upper macroblocks (assuming that these have already been decoded) is used. In this case, if there is a division point on the line contacting the current macroblock of each of the left and upper macroblocks, that point is set as the initial point. For example, in FIG. 7D, since the point c is the point, the point c is set as the first initial point.

In the third method, as shown in FIG. 7E, the number difference from the first initial point to the first division point and the number difference from the first division point to the second division point are described. . Therefore, after obtaining the first initial point from the division information of the neighboring macroblocks, the number of the first initial point is obtained by adding the number difference from the first initial point to the first division point to the number of the first initial point. The number of the second division point can be obtained by adding the number difference from the first division point to the second division point to the number of the first division point.

  In the fourth method, as shown in FIG. 7F, the number difference from the first initial point to the first division point and the number difference from the second initial point to the second division point are described. Yes. Here, the second initial point is a point-symmetric position from the first division point to the center of the macroblock. Therefore, after obtaining the first initial point from the division information of the neighboring macroblocks, the number of the first initial point is obtained by adding the number difference from the first initial point to the first division point to the number of the first initial point. After obtaining the second initial point from the first dividing point, the number of the second dividing point is obtained by adding the number difference from the second initial point to the second dividing point to the number of the second initial point. Can be obtained.

  In the fifth and sixth methods, initial point selection information is first described as shown in FIGS. 7 (h) and 7 (i). This is information indicating which one of the left and upper macroblock division information determines the first initial point. For example, if it is 0, the intersection of the dividing line with the left macroblock may be the first initial point, and if it is 1, the intersection of the dividing line with the upper macroblock may be the second initial point. Since the subsequent description methods in the fifth and sixth methods are the same as the third and fourth description methods, respectively, the first and second description methods are processed in the same manner as the processing methods of the third and fourth description methods. A division point and a second division point can be obtained.

  The division information obtained as described above (division method of rectangular division or arbitrary division, division number and division shape in the case of rectangular division, division point information in the case of arbitrary division) DI is a motion compensation unit. 906 is output.

  The motion compensation unit 906 acquires a motion compensated image from the reference picture memory 907 based on the division information DI and the motion vector MV. Here, when the macroblock is divided, since a motion vector is described for each divided region, a motion compensated image is acquired for each divided region according to the motion vector. For example, as shown in FIG. 5, when the macroblock 500 is divided into an area 501 and an area 502 by arbitrary division, the reference image 503 and the reference image 504 are acquired from the reference picture to generate the motion compensation image MC. , Output to the adder 904.

  The adder 904 adds the decoded residual image DR and the motion compensated image MC to generate a decoded image DC. The decoded image DC is an output image and is held in the reference picture memory 907 when used as a reference picture in decoding of subsequent pictures.

  As described above, in the image decoding method of the present invention, the division information of each macroblock is acquired when decoding the code string generated by the image encoding method of the present invention. Then, motion compensation is performed according to the division information and the motion vector described in the code string to generate a decoded image.

By such an operation, by using the image decoding method of the present invention, it is possible to correctly decode the code string generated by the image encoding method of the present invention.
In the embodiment of the present invention, the case where the macroblock is divided into two by one line segment at the time of arbitrary division has been described, but this is divided into three or more by two or more line segments. May be. Even in this case, decoding can be performed as in the present embodiment.

  In the embodiment of the present invention, the case where the macroblock is divided into line segments has been described. However, this may be performed even if the macroblock is divided into smaller units. For example, the sub-macroblock (FIG. 2D) obtained by dividing the macroblock into four may be divided by different line segments. Even in this case, decoding can be performed as in the present embodiment.

  In the embodiment of the present invention, when a macroblock is arbitrarily divided, the case where two points through which the line segment passes is described in the code string has been described. The minute may be considered as a straight line (y = ax + b), and the slope a and the intercept b may be described. In this case, it is possible to determine how the macroblock is divided from the linear equation.

(Embodiment 3)
Furthermore, the program for realizing the image encoding method and the image decoding method shown in each of the above embodiments is recorded on a recording medium such as a flexible disk, so that the program shown in each of the above embodiments is shown. Processing can be easily performed in an independent computer system.

  FIG. 10 is an explanatory diagram when the image encoding method and the image decoding method of each of the above embodiments are implemented by a computer system using a program recorded on a recording medium such as a flexible disk.

  FIG. 10B shows the appearance, cross-sectional structure, and flexible disk as seen from the front of the flexible disk, and FIG. 10A shows an example of the physical format of the flexible disk that is the recording medium body. The flexible disk FD is built in the case F, and on the surface of the disk, a plurality of tracks Tr are formed concentrically from the outer periphery toward the inner periphery, and each track is divided into 16 sectors Se in the angular direction. ing. Therefore, in the flexible disk storing the program, the program is recorded in an area allocated on the flexible disk FD.

  FIG. 10C shows a configuration for recording and reproducing the program on the flexible disk FD. When the program for realizing the image encoding method and the image decoding method is recorded on the flexible disk FD, the program is written from the computer system Cs via the flexible disk drive. When the image encoding method and the image decoding method for realizing the image encoding method and the image decoding method by the program in the flexible disk are built in the computer system, the program is read from the flexible disk by the flexible disk drive. Transfer to computer system.

  In the above description, a flexible disk is used as the recording medium, but the same can be done using an optical disk. Further, the recording medium is not limited to this, and any recording medium such as an IC card or a ROM cassette capable of recording a program can be similarly implemented.

(Embodiment 4)
Furthermore, application examples of the image coding method and the image decoding method shown in the above embodiment and a system using the same will be described.

  FIG. 11 is a block diagram showing an overall configuration of a content supply system ex100 that implements a content distribution service. The communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed radio stations, are installed in each cell.

  The content supply system ex100 includes, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, a camera via the Internet ex101, the Internet service provider ex102, the telephone network ex104, and the base stations ex107 to ex110. Each device such as the attached mobile phone ex115 is connected.

  However, the content supply system ex100 is not limited to the combination shown in FIG. 11, and may be connected in combination. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.

  The camera ex113 is a device capable of shooting a moving image such as a digital video camera. The mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global System for Mobile Communications) system mobile phone, Alternatively, PHS (Personal Handyphone System) or the like may be used.

  In addition, the streaming server ex103 is connected from the camera ex113 through the base station ex109 and the telephone network ex104, and live distribution or the like based on the encoded data transmitted by the user using the camera ex113 becomes possible. The encoded processing of the captured data may be performed by the camera ex113 or may be performed by a server or the like that performs data transmission processing. Further, the moving image data shot by the camera 116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device that can shoot still images and moving images, such as a digital camera. In this case, the encoding of the moving image data may be performed by the camera ex116 or the computer ex111. The encoding process is performed in the LSI ex117 included in the computer ex111 and the camera ex116. Note that image encoding / decoding software may be incorporated into any storage medium (CD-ROM, flexible disk, hard disk, etc.) that is a recording medium readable by the computer ex111 or the like. Furthermore, you may transmit moving image data with the mobile telephone ex115 with a camera. The moving image data at this time is data encoded by the LSI included in the mobile phone ex115.

  In this content supply system ex100, the content (for example, video shot of music live) captured by the user with the camera ex113, camera ex116, etc. is encoded and transmitted to the streaming server ex103 as in the above embodiment. On the other hand, the streaming server ex103 distributes the content data to the requested client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, and the like that can decode the encoded data. In this way, the content supply system ex100 can receive and reproduce the encoded data at the client, and also realize personal broadcasting by receiving, decoding, and reproducing in real time at the client. It is a system that becomes possible.

The image encoding device or the image decoding device described in the above embodiments may be used for encoding and decoding of each device constituting this system.
A mobile phone will be described as an example.

  FIG. 12 is a diagram illustrating the mobile phone ex115 using the image coding method and the image decoding method described in the above embodiment. The cellular phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a camera such as a CCD camera, a camera unit ex203 capable of taking a still image, a video shot by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display that displays data obtained by decoding received video and the like, a main body unit composed of a group of operation keys ex204, an audio output unit ex208 such as a speaker for audio output, and audio input To store encoded data or decoded data such as a voice input unit ex205 such as a microphone, captured video or still image data, received mail data, video data or still image data, etc. Recording medium ex207, and slot portion ex20 for enabling recording medium ex207 to be attached to mobile phone ex115 The it has. The recording medium ex207 stores a flash memory element which is a kind of EEPROM (Electrically Erasable and Programmable Read Only Memory) which is a nonvolatile memory that can be electrically rewritten and erased in a plastic case such as an SD card.

  Further, the cellular phone ex115 will be described with reference to FIG. The cellular phone ex115 controls the power supply circuit ex310, the operation input control unit ex304, and the image coding for the main control unit ex311 which is configured to control the respective units of the main body unit including the display unit ex202 and the operation key ex204. Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / reproducing unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via a synchronization bus ex313 Are connected to each other.

  When the end call and power key are turned on by a user operation, the power supply circuit ex310 activates the camera-equipped digital mobile phone ex115 by supplying power from the battery pack to each unit. .

  The mobile phone ex115 converts the voice signal collected by the voice input unit ex205 in the voice call mode into digital voice data by the voice processing unit ex305 based on the control of the main control unit ex311 including a CPU, a ROM, a RAM, and the like. The modulation / demodulation circuit unit ex306 performs spread spectrum processing, and the transmission / reception circuit unit ex301 performs digital analog conversion processing and frequency conversion processing, and then transmits the result via the antenna ex201. In addition, the cellular phone ex115 amplifies the received signal received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-digital conversion processing, performs spectrum despreading processing by the modulation / demodulation circuit unit ex306, and analog audio by the voice processing unit ex305. After conversion into a signal, this is output via the audio output unit ex208.

  Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmits the text data to the base station ex110 via the antenna ex201.

  When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

  The image encoding unit ex312 has a configuration including the image encoding device described in the present invention, and an encoding method using the image data supplied from the camera unit ex203 in the image encoding device described in the above embodiment. The encoded image data is converted into encoded image data by compression encoding, and sent to the demultiplexing unit ex308. At the same time, the cellular phone ex115 sends the sound collected by the audio input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 as digital audio data via the audio processing unit ex305.

  The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and the multiplexed data obtained as a result is a modulation / demodulation circuit unit A spectrum spread process is performed in ex306, a digital analog conversion process and a frequency conversion process are performed in the transmission / reception circuit unit ex301, and then the signal is transmitted through the antenna ex201.

  When receiving data of a moving image file linked to a home page or the like in the data communication mode, the received signal received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexing is obtained. Is sent to the demultiplexing unit ex308.

  In addition, in order to decode the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data to generate an encoded bitstream of image data and an encoded bitstream of audio data. The encoded image data is supplied to the image decoding unit ex309 via the synchronization bus ex313, and the audio data is supplied to the audio processing unit ex305.

  Next, the image decoding unit ex309 is configured to include the image decoding device described in the present invention, and a decoding method corresponding to the encoding method described in the above embodiment for an encoded bit stream of image data. To generate playback moving image data, which is supplied to the display unit ex202 via the LCD control unit ex302, thereby displaying, for example, moving image data included in the moving image file linked to the homepage . At the same time, the audio processing unit ex305 converts the audio data into an analog audio signal, and then supplies the analog audio signal to the audio output unit ex208. Thus, for example, the audio data included in the moving image file linked to the home page is reproduced. The

  Note that the present invention is not limited to the above-described system, and recently, digital broadcasting using satellites and terrestrial waves has become a hot topic. As shown in FIG. 14, the digital broadcasting system also includes at least the image encoding device or the image according to the above embodiment. Any of the decoding devices can be incorporated. Specifically, in the broadcasting station ex409, the encoded bit stream of the video information is transmitted to the communication or broadcasting satellite ex410 via radio waves. Receiving this, the broadcasting satellite ex410 transmits a radio wave for broadcasting, and receives the radio wave with a home antenna ex406 having a satellite broadcasting receiving facility, such as a television (receiver) ex401 or a set top box (STB) ex407. The device decodes the encoded bit stream and reproduces it. In addition, the image decoding apparatus described in the above embodiment can also be implemented in a playback apparatus ex403 that reads and decodes an encoded bitstream recorded on a storage medium ex402 such as a CD or DVD that is a recording medium. is there. In this case, the reproduced video signal is displayed on the monitor ex404. Further, a configuration in which an image decoding device is mounted in a set-top box ex407 connected to a cable ex405 for cable television or an antenna ex406 for satellite / terrestrial broadcasting, and this is reproduced on the monitor ex408 of the television is also conceivable. At this time, the image decoding apparatus may be incorporated in the television instead of the set top box. It is also possible to receive a signal from the satellite ex410 or the base station ex107 by the car ex412 having the antenna ex411 and reproduce a moving image on a display device such as the car navigation ex413 that the car ex412 has.

  Further, the image signal can be encoded by the image encoding device shown in the above embodiment and recorded on a recording medium. As a specific example, there is a recorder ex420 such as a DVD recorder that records an image signal on a DVD disk ex421 or a disk recorder that records on a hard disk. Further, it can be recorded on the SD card ex422. If the recorder ex420 includes the image decoding device described in the above embodiment, the image signal recorded on the DVD disc ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.

  For example, the configuration of the car navigation ex413 may be a configuration excluding the camera unit ex203, the camera interface unit ex303, and the image encoding unit ex312 in the configuration illustrated in FIG. 13, and the same applies to the computer ex111 and the television (receiver). ) Ex401 can also be considered.

  In addition to the transmission / reception type terminal having both the encoder and the decoder, the terminal such as the mobile phone ex114 has three mounting formats: a transmitting terminal having only an encoder and a receiving terminal having only a decoder. Can be considered.

  As described above, the image encoding method and the image decoding method described in the above embodiment can be used in any of the above-described devices / systems, and by doing so, the effects described in the above embodiment can be obtained. Can be obtained.

  It should be noted that the present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the scope of the present invention.

  The image encoding method and the image decoding method according to the present invention can perform diagonal division that cannot be handled by conventional rectangular division alone, thereby greatly reducing the code amount of the residual signal after motion compensation. And is useful as an image encoding method and an image decoding method in storage, transmission, communication, and the like.

1 is a block diagram (Embodiment 1) of an image encoding apparatus using an image encoding method of the present invention. It is a schematic diagram (Embodiment 1) for demonstrating the division | segmentation method of the macroblock in the image coding method of this invention. It is a schematic diagram (Embodiment 1) for demonstrating the arbitrary division | segmentation method of the macroblock in the image coding method of this invention. 5 is a flowchart (Embodiment 1) showing a method for determining a macroblock division method in the image coding method of the present invention. It is a schematic diagram (Embodiment 1, Embodiment 2) for demonstrating the motion compensation of the macroblock in the image coding method of this invention. It is a schematic diagram (Embodiment 1, Embodiment 2) for demonstrating the description method in the code sequence of the macroblock division | segmentation method in the image coding method of this invention, and an image decoding method. It is a schematic diagram (Embodiment 1, Embodiment 2) for demonstrating the description method in the code sequence of the macroblock division | segmentation method in the image coding method of this invention, and an image decoding method. It is a schematic diagram (Embodiment 1) for demonstrating the arbitrary division | segmentation method of the macroblock in the image coding method of this invention. It is a block diagram (Embodiment 2) of the image decoding apparatus using the image decoding method of this invention. It is explanatory drawing (Embodiment 3) about the recording medium for storing the program for implement | achieving the image coding method and image decoding method of said each embodiment with a computer system. FIG. 10 is a block diagram showing the overall configuration of a content supply system (Embodiment 4). 10 is an example (Embodiment 4) of a mobile phone using an image encoding method and an image decoding method. FIG. 10 is a block diagram of a mobile phone (Embodiment 4). It is an example (Embodiment 4) of the system for digital broadcasting.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image coding apparatus 101 Frame memory 102 Division | segmentation control part 103 Division | segmentation part 104 Motion detection part 105 Reference picture memory 106 Difference part 107 Conversion part 108 Quantization part 109 Variable length encoding part 110 Dequantization part 111 Inverse conversion part 112 Adder 113 Switch 114 Division method description unit 115 Division method holding unit 900 Image decoding device 901 Variable length decoding unit 902 Inverse quantization unit 903 Inverse transformation unit 904 Adder 905 Division method decoding unit 906 Motion compensation unit 907 Reference picture Memory 908 Division method holding unit DI Division information DIC Encoded division information MV Motion vector RS Residual image QC Quantized transform coefficient DR Decoded residual image DC Decoded image Cs Computer system FD Flexible disk FDD Flexible De Disk drive

Claims (11)

An image encoding method for dividing an input image into blocks having a predetermined size and performing an encoding process in units of the blocks,
A division control step for instructing a method for further dividing the block into a plurality of blocks;
A division step of dividing the block by a division method instructed by the division control step to generate a divided region;
A motion detection step of performing motion estimation for each divided region generated by the division step and detecting a motion vector;
A motion compensation step for generating a motion compensated image for each of the divided regions based on the motion vector detected by the motion detection step,
The image coding method according to any one of claims 1 to 4, wherein the image dividing method instructed by the image dividing control step includes at least an image dividing method for dividing the block by an arbitrary line segment. In the motion detection step, when the motion vector is detected, an evaluation value is obtained together with the motion vector,
In the division control step, a plurality of division methods are provided, the plurality of division methods are sequentially instructed to the division step, and a division method in which the evaluation value obtained in the motion detection step is optimal is determined for the block. The image coding method according to claim 1, wherein the image coding method is determined as a final division method. The image coding method according to claim 1, further comprising a division method description step, wherein the block division method is described in a code string. The image according to claim 3, wherein, in the division method description step, when the division method of the block is division by the arbitrary line segment, two points through which the line segment passes are described in a code string. Encoding method. In the division method description step, when the block division method is division by the arbitrary line segment, one point through which the line segment passes and the inclination of the line segment are described in a code string. The image encoding method according to claim 3. An image decoding method for decoding an input image into blocks having a predetermined size and decoding a code string generated by performing an encoding process in units of the blocks,
A division method decoding step for obtaining a division method in the block described in the code string;
A division step of dividing the block by the division method obtained by the division method decoding step and generating a divided region;
A motion compensation step of generating a motion compensated image for each of the divided regions using a motion vector obtained from the code string,
The image decoding method, wherein the dividing method includes a dividing method for dividing the block by an arbitrary line segment. The image decoding according to claim 6, wherein, in the division method decoding step, when the block division method is division by the arbitrary line segment, the line segment information is notified to the division step. Method. In the division method decoding step, when the division method of the block is division by the arbitrary line segment, and the line segment is described by two points through which the line segment passes, two points through which the line segment passes through The image decoding method according to claim 6, wherein line segment information is identified from the information and the line segment information is notified to the dividing step. In the division method decoding step, when the division method of the block is division at the arbitrary line segment, and the line segment is described by one point through which the line segment passes and the inclination of the line segment, The image decoding method according to claim 6, wherein line segment information is specified from one point through which the line segment passes and a slope of the line segment, and the line segment information is notified to the division step. A program for image coding by a computer,
The above program is stored on the computer.
An image encoding method for dividing an input image into blocks having a predetermined size and performing an encoding process in units of the blocks,
A division control step for instructing a method for further dividing the block into a plurality of blocks;
A division step of dividing the block by a division method instructed by the division control step to generate a divided region;
A motion detection step of performing motion estimation for each divided region generated by the division step and detecting a motion vector;
A motion compensation step for generating a motion compensated image for each of the divided regions based on the motion vector detected by the motion detection step,
A program characterized by causing an image coding method characterized in that the division method instructed by the division control step includes at least a division method for dividing the block by an arbitrary line segment. A program for performing image decoding by a computer,
The above program is stored on the computer.
An image decoding method for decoding an input image into blocks having a predetermined size and decoding a code string generated by performing an encoding process in units of the blocks,
A division method decoding step for obtaining a division method in the block described in the code string;
A division step of dividing the block by the division method obtained by the division method decoding step and generating a divided region;
A motion compensation step of generating a motion compensated image for each of the divided regions using a motion vector obtained from the code string,
A program for performing an image decoding method characterized by including a dividing method for dividing the block by an arbitrary line segment in the dividing method.

Images