JP2000059789A - Dynamic image code string converter and method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a highly functional code string converter which produces image information a digital signal with fewer codings. SOLUTION: This device has a separation means 2 for obtaining a first motion vector from a first dynamic image code string, decoding means 3 to 5, 7 and 8 for performing inter-image prediction and decoding by using the first motion vector, corresponding to a first motion compensation block and obtaining the first decoding image signal, a pixel number conversion means 19 for converting the number pixels of the first decoding image signal and obtaining a second decoding image signal with different number of pixels, a motion vector reconstruction means 6 for obtaining a second motion vector for re-encoding the second decoding picture signal by a second motion compensation block on the basis of the first motion vector, re-encoding means 9 to 11 and 14 to 18 for inter-image prediction encoding the second decoding image signal by the second motion vector compensation block by using the second motion vector, and a multiplex means 12 for obtaining a second motion image code string by information on the second motion vector and a code string selected by the re-encoding means.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention
The present invention relates to conversion of a code string in high-efficiency coding for converting image information into a digital signal with a smaller code amount for display. In particular, the coding method is inter-picture prediction coding in which motion compensation is performed on a block basis, and relates to a case of converting into an image having a different number of pixels.

[0002]

2. Description of the Related Art <Conversion of moving image code string> A code string encoded by moving image high efficiency coding represented by MPEG or the like is converted into
There is a need to convert to a different data rate, or to convert a variable transfer rate to a fixed transfer rate. In this case, it is a principle that the image is completely decoded and re-encoded at a different rate. However, if the basic encoding process is the same, a part of the information can be used as it is. Specifically, the motion vector (MV) information is used as it is in the re-encoding, and the motion vector detection that requires many operations can be omitted.
In addition, since the motion-compensated inter-picture prediction processing does not change, deterioration due to re-encoding is only a difference in quantization and can be minimized. Such processing is described in 1993 Image Coding Symposium Proceedings 1-6 “Study of Coding Control in Image Recoding” and H. Sun, W. Kwok and J. Zdepski, “Architectures f
or MPEG Compressed Bitstream Scaling "(IEEE Transac
tion on Circuit and System for Video Technology, V
ol. 6, No. 2, April 1996).

<Conventional Moving Image Code String Conversion Apparatus> FIG. 5 shows an example of the configuration of a conventional moving image code string conversion apparatus. A code string of motion-compensated inter-picture prediction input from a code string input terminal 1 is converted into a code string of a prediction residual and a code string of an MV by a variable-length decoder 2 to a fixed-length code.
The DCT (Discrete Cosine Transform) coefficient obtained as a fixed-length code becomes a coefficient value in the inverse quantizer 3 and is given to the inverse DCT 4. The inverse DCT 4 converts the 8 × 8 coefficients into a reproduction prediction residual signal and supplies the signal to the adder 5. The adder 5 adds a prediction signal, which will be described later, to the reproduction prediction residual signal, thereby forming a reproduction image.
On the other hand, the MV information output from the variable length decoder 2 is provided to the motion compensation predictors 7 and 14.

[0004] The reproduced image signal thus obtained is supplied to an image memory 8 and a prediction subtractor 9. The motion compensation predictor 7 performs motion compensation on the image stored in the image memory 8 based on the MV to form a prediction signal. The obtained prediction signal is provided to the adder 5. Next, the re-encoding system will be described. The reproduced image signal obtained from the adder 5 is subtracted from the prediction signal supplied from the motion compensation predictor 14 in the subtractor 9 and is supplied to the DCT 10 as a prediction residual.

[0005] The DCT 10 performs a DCT conversion process and supplies the obtained coefficient to a quantizer 11. The quantizer 11 quantizes the coefficient with a predetermined step width, and provides the fixed-length code to the variable-length encoder 12 and the inverse quantizer 18. The quantization step width differs from the quantization step width of the inverse quantizer 3 in accordance with the transfer rate change.
The variable-length encoder 12 compresses the fixed-length prediction residual with a variable-length code, further performs variable-length coding on the MV, and outputs the resulting code string from a code string output terminal 13.

On the other hand, in the inverse quantizer 18 and the inverse DCT 17, inverse processing of the DCT 10 and the quantizer 11 is performed, and an inter-picture prediction residual is reproduced. The obtained inter-picture prediction residual is added to the inter-picture prediction signal by the adder 16 to become a reproduced picture, which is provided to the image memory 15. The reproduced image stored in the image memory 15 is provided to the motion compensation predictor 14. The motion compensation predictor 14 generates an inter-picture prediction signal according to the MV given from the variable length decoder 2,
To the adder 16. Here, since the motion-compensated inter-picture prediction processing is the same for the decoding unit and the coding unit, it seems that the adder 5 and the subtractor 9 are canceled out and only the intra-picture processing needs to be performed. Furthermore, since DCT4 is a reversible transform process for the inverse DCT11, it seems that only the requantization needs to be performed.

However, since the reproduced image stored in the image memory 8 of the decoding system and the reproduced image stored in the image memory 15 of the re-encoding system have different quantization processes, images having different quantization errors are used. And the prediction signal will be slightly different. Therefore, if the inter-picture prediction processing is omitted, a large error does not occur in one prediction processing, but a large error occurs due to accumulation of errors in the cyclic prediction processing. That is, omitting the prediction processing or the like will cause a large deterioration in image quality.

[0008]

The conventional moving picture code string conversion apparatus is compatible only with an image having the same number of pixels before and after the conversion. In the case of the same number of pixels, the motion vector can be used as it is, but if the number of pixels is different before and after the conversion, the boundary of the motion compensation block changes, so the motion vector cannot be used as it is. In the conversion involving the conversion of the number of pixels, it is necessary to perform the motion vector detection again, and the processing amount and the image quality deteriorate. The present invention has been made by paying attention to the above points, and decodes an incoming code string and reconstructs a motion vector corresponding to an image having a different number of pixels using a motion vector extracted from the code string. It is another object of the present invention to provide a moving image code sequence conversion apparatus that reduces a processing amount and image quality deterioration by re-encoding a decoded image whose number of pixels has been converted using a new motion vector.

[0009]

According to the present invention, a first motion vector code sequence is received and a first motion vector is separated from the code sequence in the conversion of a motion image code sequence predicted between motion compensated images. Performing inter-picture predictive decoding on the first video code sequence using the first motion vector corresponding to a first motion compensation block, obtaining a first decoded video signal, and After conversion, a second decoded image signal having a different number of pixels is obtained. On the other hand, a second motion vector for re-encoding the second decoded image using a second motion compensation block is reconstructed based on the first motion vector, and the second motion vector is reconstructed using the second motion vector. Moving image code string conversion apparatus for obtaining a code string by inter-picture predictive coding of the decoded image signal of the second motion compensation block with the second motion compensation block, and multiplexing with the information of the second motion vector to obtain a second moving image code string And its method.

In the above-described motion vector reconstruction, a first motion compensation block whose part is included in a second motion compensation block is specified on a real image, and a first motion compensation block for the first motion compensation block is specified. As a candidate, a motion vector obtained by scaling the motion vector of No. 1 in accordance with the conversion ratio of the number of pixels by the pixel number conversion means is used to search for a motion vector using a second decoded image signal. An obtained moving picture code string conversion apparatus and method are provided. In the above-described motion vector reconstruction, a first motion compensation block existing at a center position of a second motion compensation block on an actual image is specified, and a first motion vector for the first motion compensation block is specified. A moving vector obtained by scaling a motion vector according to a conversion ratio of the number of pixels in the pixel number converting means as the second motion vector.

(Operation) In the present invention, all or a part of an incoming code string is decoded, and a motion vector (MV) extracted from the code string is used to create a motion compensation block of an image having a different number of pixels. The corresponding MV is reconstructed, and the decoded image whose pixel number has been converted is re-encoded using the MV. The image to be re-encoded can optimize the balance between resolution and quantization error with respect to the re-encoding transfer rate by converting the number of pixels. Since the MV obtained by the reconstruction corresponds to the motion compensation block in the image in which the number of pixels has been converted, the motion compensation error in the re-encoding is minimized. Reconstruction of the MV is a small process compared to the case where motion vector detection is performed. Since the motion vector is also similar to the incoming one, the change in the inter-picture prediction residual is minimal and the image quality degradation is also reduced.

[0012]

BEST MODE FOR CARRYING OUT THE INVENTION <Embodiment of Moving Image Code Sequence Converter> An embodiment of a moving image code sequence converter of the present invention will be described below. FIG. 1 shows the configuration, and the same components as those in the conventional example of FIG. 5 are denoted by the same reference numerals. FIG.
MV reconstructor 6 and pixel number converter 1 as compared with FIG.
9 has been added. The internal configuration of the MV reconstructor 6 is shown in FIGS. What differs from the conventional example in the embodiment is the number of pixels of the re-encoded image and the accompanying motion vector (MV). The processing operation of each processing part such as DCT and quantization is basically the same as that of the conventional example.

The decoding system will be described first. As for the code input from the code input terminal 1, the code sequence of the prediction residual and the first MV code sequence corresponding to the motion compensation block of the code sequence are returned to the fixed-length code by the variable-length decoder 2. The DCT coefficient obtained as the fixed-length code becomes a coefficient value in the inverse quantizer 3 and is provided to the inverse DCT 4. The inverse DCT 4 converts the 8 × 8 coefficients into a reproduction prediction residual signal and supplies the signal to the adder 5. The adder 5 adds the prediction signal obtained from the motion compensation predictor 7 described below to the reproduction prediction residual signal, thereby forming a reproduction image. on the other hand,
The first MV extracted from the variable length decoder 2 is provided to a motion compensation predictor 7 and an MV reconstructor 6.

The reproduced image signal thus obtained is supplied to the image memory 8 and the pixel number converter 19. The pixel number converter 19 resamples the reproduced image in the horizontal direction to obtain an image whose number of pixels has been converted. Details of the resampling process will be described later. The resampled reproduced image signal is provided to the prediction subtractor 9 and the MV reconstructor 6. The MV reconstructor 6 uses the first MV obtained by the variable length decoder 2 to
A second MV corresponding to an image having a different number of pixels required for re-encoding is reconstructed. The method of reconstructing the MV will be described later.

Next, the re-encoding system will be described. The reproduced image signal having undergone the pixel number conversion by the pixel number converter 19 is subjected to subtraction of the prediction signal supplied from the motion compensation predictor 14 in the subtractor 9 and supplied to the DCT 10 as a prediction residual. The DCT 10 performs a DCT transform process and supplies the obtained coefficient to the quantizer 11. The quantizer 11 quantizes the coefficient with a predetermined step width, and provides the fixed-length code to the variable-length encoder 12 and the inverse quantizer 18. The variable-length encoder 12 converts the prediction residual into a code compressed by a variable-length code, also performs variable-length encoding on the second MV, and outputs a code string obtained by multiplexing the two from a code output terminal 13.

On the other hand, in the inverse quantizer 18 and the inverse DCT 17, the inverse processing of the DCT 10 and the quantizer 11 is performed, and the inter-picture prediction residual is reproduced. The obtained inter-picture prediction residual is added to the inter-picture prediction signal by the adder 16 to become a reproduced picture, which is provided to the image memory 15. The reproduced image stored in the image memory 15 is provided to the motion compensation predictor 14. The motion-compensated inter-picture predictor 14 generates an inter-picture prediction signal according to the second MV provided from the MV reconstructor 6,
By providing the signals to the subtractor 9 and the adder 16, inter-picture prediction encoding is performed. The processing contents of each part of the re-encoding are as follows.
This is basically the same as the conventional example of FIG. 5, but specific parameters and the like are different according to the difference in the number of pixels.

<Pixel Number Converter> The pixel number converter 19 normally resamples a reproduced image in the horizontal direction. Specifically, a reproduced image that is normally 720 pixels is converted to 540 pixels by multiplying it by 3/4, to 480 pixels by multiplying it by 2/3, and to 360 pixels by multiplying by 1/2. FIG. 6 shows the state of the pixels at that time (only one line). The resampling process is performed by generating pixels at different sample positions while switching tap coefficients by a digital filter, and extracting only necessary pixels. The number of coded pixels is 1 for the motion compensation block.
Since it is 6 × 16 pixels, it must be a multiple of that, and 54
For 0 pixels, add 4 dummy pixels and 544 pixels, 3
For 60 pixels, 8 pixels are deleted to be 352 pixels.

Such conversion of the number of pixels is usually performed in accordance with a significant change in the transfer rate. In other words, a code string of about 8 Mbps at 720 pixels is converted to 5 Mbps at 720 pixels.
If it is attempted to convert to ps or less, the motion compensation prediction information such as MV is kept as it is, and only the information of the prediction residual is reduced, resulting in extremely degraded image quality. When the number of pixels to be coded is reduced, the resolution is reduced, but the motion compensation prediction information is also reduced, so that the information balance can be appropriately adjusted. The degree to which the number of pixels is reduced depends on the transfer rate of re-encoding, and if a lower rate is desired, the number of pixels is also reduced. The conversion of the number of pixels in the vertical direction has a problem with an interlaced signal, but is possible with a progressive image. Also,
If the transfer rate of re-encoding is increased from the original transfer rate, the image quality is not improved, so it is hard to imagine increasing the number of pixels.

Another example is a case in which a total of 16 pixels of the right end and the left end of 720 pixels are deleted to be 704 pixels.
The Rec. 601 format used as a digital standard image has 720 horizontal effective pixels, but the image actually has only about 711 pixels in relation to the analog image signal standard, and blank samples exist on the left and right. I do. Therefore, blank samples are excluded and only 704 pixels are encoded. In this case, no resampling is performed and only pixels are deleted. Although the resolution does not decrease, the effect of reducing the number of pixels is small.

<MV Reconstructor> The MV needs to be adapted to the motion compensation block using it. In the present embodiment, the number of pixels differs between the image of the incoming code string and the image that is re-encoded and output, and the size and the block boundary position of the motion compensation block on the real image change. FIG. 4 shows this state, and the dotted line shows a block of 720 pixels (first block) before conversion. When the number of pixels is reduced by the pixel number conversion, the same 16 × 16 pixels are blocked.
(The second block) becomes larger on the real image. Accordingly, the block boundary position also changes. When 720 is converted to 704, the size of the block does not change, but the boundary position changes. Also, since the value of MV is indicated by the horizontal and vertical pixel movement amounts, even if the first MV and the second MV have the same movement on the real image, the pixel movement amount changes with the conversion of the number of pixels. Need to be converted.

The configuration of the MV reconstructor 6 is shown in FIG.
become that way. In FIG. 2, the incoming first MV is stored in the MV buffer 21, and the first block that is at least partially included in the target second block is specified based on the positional relationship between the first block and the second block, and the first MV is stored in the MV. The searcher 22 is provided. The MV searcher 22 multiplies the MV by the vertical component value by the conversion ratio of the number of pixels in the vertical direction, and multiplies the horizontal component value by the conversion ratio of the number of pixels in the horizontal direction. Specifically, horizontal 720 is 4
If converted to 80, the horizontal component value is multiplied by 2/3 while the vertical component value remains unchanged. The first M thus converted
V re-searches for the second MV candidate using the reproduced image whose pixel number has been converted, and outputs the MV with the least error. At this time, a peripheral value of the first MV obtained by multiplying the MV candidate by the conversion ratio may be added.

The MV reconstructor may be one capable of simple processing as shown in FIG. In Figure 3 the first incoming
The MV is stored in the MV buffer 21, and the first MV is
It is provided to a V selector 23. The MV selector 23 selects the first MV of the first block at the center of the target second block from the positional relationship between the first block and the second block. This is multiplied by the conversion ratio of the number of pixels in the same manner as described above to obtain the second MV. In this case, no search is performed and no reproduced image is needed.

[0023]

According to the present invention, all or a part of an incoming code sequence is decoded, and a motion vector corresponding to a motion compensation block of an image having a different number of pixels is obtained by using a motion vector extracted from the code sequence. It reconstructs and re-encodes the decoded image whose pixel number has been converted using the motion vector. By re-encoding the image in which the number of pixels has been converted, the balance between the resolution and the quantization error can be optimized with respect to the transfer rate of the re-encoding, and the quality of the reproduced image of the converted code sequence is improved. .

MV reconstruction is a small process compared to MV detection.
The size of the device can be greatly reduced. Also, if the MV detection is performed anew by re-encoding, the image used for the MV detection is deteriorated in the reproduced image, so that it is easy to detect a motion different from the original motion. Since the re-search is based on the MV, erroneous detection hardly occurs. Since the MV is also similar to the incoming one, the change in the inter-picture prediction residual due to re-encoding is minimal and image quality degradation is minimal.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of an embodiment of a moving image code string conversion apparatus of the present invention.

FIG. 2 is a diagram showing a first embodiment of an MV reconstructor of the moving picture code sequence conversion apparatus according to the present invention.

FIG. 3 is a diagram showing a second embodiment of the MV reconstructor of the moving picture code sequence conversion apparatus according to the present invention.

FIG. 4 is a diagram showing a state of a motion compensation block according to the present invention.

FIG. 5 is a diagram illustrating an example of a configuration of a conventional moving image code string conversion device.

FIG. 6 is a diagram illustrating a state of pixel number conversion according to the present invention.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 code string input terminal 2 variable length decoder (separating means) 3, 18 quantizer 4, 17 DCT 5, 16 adder 6 MV reorganizer 7, 14, dynamic compensation predictor 8, 15, image memory 9 subtractor DESCRIPTION OF SYMBOLS 10 DCT 11 Quantizer 12 Variable length encoder (multiplexing means) 13 Code string output terminal 21 MV buffer 22 MV searcher 23 MV converter

Claims (4)

[Claims] An apparatus for converting a code sequence of a moving image predicted between motion-compensated images, comprising the steps of: receiving a first moving image code sequence and obtaining a first motion vector from the code sequence; Means for decoding the first video code sequence by performing inter-picture prediction using the first motion vector corresponding to a first motion compensation block to obtain a first decoded video signal; A pixel number conversion means for converting the number of pixels of the first decoded image signal to obtain a second decoded image signal having a different number of pixels; and re-converting the second decoded image signal by a second motion compensation block. A second motion vector to encode is
Using the motion vector reconstructing means obtained based on the first motion vector, and the second motion vector, performing inter-picture predictive encoding on the second decoded image signal with the second motion compensation block. Re-encoding means for obtaining a code sequence; and multiplexing means for multiplexing the information of the second motion vector and the code sequence selected by the re-encoding device to obtain a second video code sequence. Moving image code string conversion device. 2. The video code stream conversion device according to claim 1, wherein said motion vector reconstructing means includes a first motion compensation block, a part of which is included in a second motion compensation block on a real image. And a second motion vector obtained by scaling the first motion vector for the first motion compensation block according to the conversion ratio of the number of pixels by the number-of-pixels conversion means as a candidate. A moving image code string conversion device, wherein a second motion vector is obtained by performing a search for a motion vector using the same. 3. The moving image code sequence conversion apparatus according to claim 1, wherein said motion vector reconstructing means converts the first motion compensation block existing at the center position of the second motion compensation block on the real image. A motion vector obtained by specifying and scaling the first motion vector for the first motion compensation block according to the conversion ratio of the number of pixels in the number-of-pixels conversion means,
A moving image code string conversion device characterized by using a second motion vector. 4. A moving image code sequence conversion method for converting a code sequence of a moving image predicted between motion compensated images, comprising the steps of: receiving a first moving image code sequence, separating a first motion vector from the code sequence; Performing inter-picture predictive decoding on the first video code sequence using the first motion vector corresponding to a first motion compensation block to obtain a first decoded video signal; To obtain a second decoded image signal having a different number of pixels, and re-encode the second decoded image signal with a second motion compensation block. A motion vector is reconstructed based on the first motion vector, and the second decoded image signal is subjected to inter-picture predictive coding by the second motion compensation block using the second motion vector. Obtaining a code sequence, and obtaining information of the second motion vector and the re-encoding Multiplexes the code string selected in the
A moving image code string conversion method characterized by obtaining a moving image code string.