JPH09182085A - Image encoding device, image decoding device, image encoding method, image decoding method, image transmitting method and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress the influence on a decoding image by the edge-shaped degradation appearing on a block boundary. SOLUTION: In this method, a motion vector is detected by a motion vector detector 11, a wavelet coefficient is generated by a wavelet converter 3, a predictive wavelet coefficient is generated from the detected motion vector and an already restored image by a motion compensator 12, the difference of the wavelet coefficient and the predictive wavelet coefficient is calculated by a substracter 4 and the difference is quantized by a quantizer 5, for an inputted image signal. By performing reverse quantization for the quantized difference of the wavelet coefficients by an reverse quantizer 13 and adding the difference to the predictive wavelet coefficient by an adder 14, the wavelet coefficient is restored, reverse wavelet conversion is performed for the wavelet coefficient by the reverse wavelet converter 15 and an image is restored.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image coding device,
The present invention relates to an image decoding device, an image encoding method, an image decoding method, an image transmission method, and a recording medium, and for example, an encoding device and an encoding method for encoding an image signal for the purpose of compressing a moving image, and the encoding thereof. A recording medium such as a video tape, a video disk, and a semiconductor memory in which the image data is recorded, a decoding device and a decoding method for decoding the encoded image signal recorded in the recording medium, the encoding thereof,
The present invention relates to an AV device including a decoding device, a transmission method in communication devices such as a television broadcast and a video conference system, and a device and method for encoding and decoding image signals for the communication device.

[0002]

2. Description of the Related Art Subband coding is one of the coding / decoding methods for the purpose of compressing digital signals. The sub-band coding is performed by band-dividing a digital signal by a filter for performing a wavelet transform (hereinafter referred to as a wavelet transform filter), for example.
The digital signal is compressed. That is, the sub-band coding is performed on the input signal by performing a filtering process with a plurality of filters having different pass bands, and then down-sampling at an interval according to each frequency band, and an output signal of each filter. The compression is performed by utilizing the energy bias of.

Regarding signal processing by band division by a subband filter and a wavelet filter, for example, reference is made to "Wavelet Transform and Subband Coding" by Martin Vetari, The Institute of Electronics, Information and Communication Engineers, Vol.74 N.
o.12 pp1275-1278 Described in December 1991.

Regarding signal processing by band division using subband coding and wavelet transform, for example, reference "Wavelet transform and subband coding" by Martin Vetari, Journal of the Institute of Electronics, Information and Communication Engineers, Vo1.74 No.12.
pp1275-1278, described in December 1991.

A conventional example of an image encoding device and an image decoding device using motion compensation and DCT will be described with reference to FIG. 28A shows the configuration of the image encoding device 100, and FIG. 28B shows the configuration of the image decoding device 200.

In the image coding apparatus 100, the input image is supplied to the frame memory 1022 via the input terminal 101. The motion vector detector 111 detects the motion vector v from the input image stored in the frame memory 102. The motion vector detection method is usually 16
Block matching is performed in block units of pixels × 16 pixels. Further, in order to realize higher accuracy, matching is performed on a half-pixel basis. Further, other known motion vector detection methods may be used instead of block matching.

The motion compensator 112 has a frame memory (not shown), and at present, the pixel value at each position of the image to be encoded has already been encoded and decoded and is stored in this frame memory. Predict from images. The pixel value I [i, i] at the position (i, j) on the image input at time t
The predicted value I ′ [i, j, t] of j, t] is the motion vector v = (vx (I, j, t), vy corresponding to this position.
(I, j, t)) is used to determine as in the following equation (1).

[0008]

[Equation 1]

Here, T is the difference between the time when the image I currently being predicted is input and the time when the image on the frame memory is input, and I on the right side of the equation (1).
[I ', j', t-T], I [i '+ 1, j', t-
T], I [i ', j' + 1, t-T], I [i '+ 1,
j ′ + 1, t−T] represents a pixel value on a frame memory (not shown). Also, int (x) represents the maximum integer value that does not exceed x.

The frame memory 102 also supplies the input image to the subtractor 103. The subtractor 103 calculates the difference between the value of the pixel currently to be encoded and the predicted value calculated by the motion compensator 112.

The difference value from the subtractor 90 is the DCT (Di
It is supplied to the screen cosine transform device 104. The DCT unit 104 is a two-dimensional DCT for an 8 pixel × 8 pixel block composed of difference values.
Give.

The DCT coefficient c from the DCT unit 104 is supplied to the quantizer 105. The quantizer 105 uses the DCT
The DCT coefficient c obtained by the device 104 is quantized by the following equation (2) using an appropriate step size Q.

[0013]

[Equation 2]

The quantized DCT coefficient is supplied to the variable length encoder 106 and the inverse quantizer 113. The inverse quantizer 113 performs the inverse quantization process as in the following Expression (3) with the same step size as that used in the quantizer 105.

[0015]

(Equation 3)

The inversely quantized data is subjected to inverse DCT by the IDCT unit 114 to restore the difference value of pixel values. This difference value is added to the prediction value output from the motion compensator 112 by the adder 115 to form pixel value data, which is sent to the motion compensator 112 and stored in a frame memory (not shown).

In the variable length encoder 106, the quantizer 10
Variable length coding is performed on the quantized DCT coefficient obtained in step 5 and the motion vector v obtained by the motion vector detector 111, and a bit stream is output.

On the other hand, in the image decoding apparatus 200, the inverse variable length encoder 202 first performs the inverse processing of the variable length encoder 106 in the image encoding apparatus 100 on the bit stream supplied via the input terminal 201. Quantized DCT from the bitstream
Restore the coefficients and motion vector v. The obtained DCT coefficient is supplied to the inverse quantizer 203, and the motion vector v is supplied to the motion compensator 206.

Inverse quantizer 203 and IDCT device 204
Are the same as those of the image encoding device 100, and in each of them, the inverse quantization processing and the IDCT of the expression (3) are performed.
Is performed to restore the difference value of the pixel values. This difference value is
The adder 205 adds the prediction value already generated by the motion compensator 206 to form pixel value data, and reconstructs an image corresponding to the input image to the image encoding device 100. Each pixel value of the decoded image is stored in a frame memory (not shown) included in the motion compensator 206 to generate a predicted image.

The motion compensator 206 is used by the image coding apparatus 10.
It is the same as that of 0, and is currently decoded using the motion vector v obtained by the inverse variable-length encoder 202 and the image already stored in the frame memory (not shown) included in the motion compensator 206. Predict each pixel value of the power image.

The above description is based on, for example, MPEG.
This is an operation performed for so-called interframes such as P-pictures such as 1 and 2, and B-pictures. Therefore, the subtractor 103 does not perform the difference calculation on the intra frame, and the pixel data from the frame memory is directly supplied to the DCT unit 104 as it is, and the subsequent processing is executed.

Further, a transmission method of a bit stream in a conventional encoding method such as MPEG1 will be briefly described. In MPEG1, the minimum unit header having a sync code is a slice header. Therefore, when an error or the like occurs and a part of the data cannot be decoded, it is decoded from the next slice header.

[0023]

As in the conventional method described above, in the coding using motion detection and motion compensation in block units, prediction error of the prediction error may occur in adjacent blocks existing in a discontinuous motion region. There is a problem that the properties are different and edge-like deterioration occurs along the block boundary. In the conventional method, since motion compensation is performed on the image, such deterioration of the block boundary remains as it is.
It appeared on the decoded image and was a cause of the image quality deterioration.

In the conventional bit stream transmission method described above, if an error occurs in part of the data,
All data will be lost in slice units.
That is, on the decoded image, all the data of the undecodable slice portion is lost, which causes fatal deterioration.

Therefore, the present invention has been made in consideration of the above points, and an object of the present invention is to alleviate the edge-like deterioration appearing on a block boundary and suppress the influence on a decoded image. An object is to provide an image encoding device, an image decoding device, an image encoding method, an image decoding method, an image transmission method, and a recording medium on which encoded image data is recorded.

Another object of the present invention is to provide an image transmission method capable of avoiding fatal deterioration on a decoded image even when data of a certain band is lost.

[0027]

According to the present invention, in an image coding apparatus for an image signal using a subband filter, a motion vector detecting means for detecting a motion vector from an input image signal, and an input image On the other hand, frequency band division processing by subband conversion is performed to generate a subband coefficient, subband conversion means, coding means for coding the difference between the subband coefficients and generating a coded signal, and the above coding Decoding means for decoding the signal and generating a difference between the decoded subband coefficients, the motion vector,
Motion compensation means for generating a predicted subband coefficient for predicting the subband coefficient from a restored image that has already been restored, and the predicted subband coefficient are added to the difference between the decoded subband coefficients and restored. Means for generating sub-band coefficients, inverse sub-band transforming means for subjecting the restored sub-band coefficient to inverse sub-band transformation, and restoring the image, the sub-band coefficient and the predicted sub-band coefficient And a coefficient difference calculating means for calculating the difference of

In the image coding apparatus according to the present invention, for example, wavelet coefficients are generated by the subband conversion means.

In the image coding apparatus according to the present invention,
In the motion compensation means, for example, the restored image is subjected to filtering processing so as to correspond to the frequency band to which the wavelet coefficient to be predicted belongs, and the predicted value is obtained from the output.

According to the present invention, in an image signal image decoding apparatus using a sub-band filter, receiving means for receiving an image signal including at least a difference value between a motion vector and an encoded sub-band coefficient; Separation means for separating the difference value of the coded subband coefficients, decoding means for decoding the difference value of the coded subband coefficients, and generating a difference value of the decoded subband coefficients, and Motion compensation means for generating a prediction subband coefficient for predicting a subband coefficient from a motion vector and a restored image, a difference value between the decoded subband coefficients, and a prediction wavelet obtained by the motion compensation means Coefficient adding means for adding the coefficient and inverse subband conversion to the subband coefficient from the coefficient adding means to restore the image. And having an inverse subband transform means.

The image decoding apparatus according to the present invention uses, for example, wavelet coefficients as the subband coefficients.

Further, in the image decoding apparatus according to the present invention, in the motion compensating means, for example, the restored image is subjected to filtering processing so as to correspond to the frequency band to which the subband coefficient to be predicted belongs, The predicted value is obtained from the output.

According to the present invention, in the image coding method for an image signal using a subband filter, the step of detecting a motion vector from the input image signal, and the frequency band by subband conversion for the input image. Applying division processing to generate sub-band coefficients, encoding the difference between the sub-band coefficients, generating an encoded signal, decoding the encoded signal, and calculating the difference between the decoded sub-band coefficients A step of generating, a step of generating a predicted subband coefficient for predicting the subband coefficient from the motion vector and a restored image already restored, and a step of generating the predicted subband coefficient in a difference between the decoded subband coefficients. Adding the coefficients to generate the reconstructed subband coefficients, and the inverse subband for the reconstructed subband coefficients. Subjected to command conversion, characterized and restoring an image, that a step of calculating a difference between the subband coefficients and the predictive subband coefficients.

In the image coding method according to the present invention, for example, wavelet coefficients are used as the subband coefficients.

In the image coding method according to the present invention,
In the step of generating the predicted subband coefficient,
For example, filtering processing is performed on the restored image so as to correspond to the frequency band to which the subband coefficient to be predicted belongs, and the predicted value is obtained from the output.

According to the present invention, in an image decoding method for an image signal using a subband filter, a step of receiving an image signal including at least a difference value between a motion vector and an encoded subband coefficient, the motion vector and the above A step of separating the difference value of the coded subband coefficients, a step of decoding the difference value of the coded subband coefficients, and a step of generating a difference value of the decoded subband coefficients, the motion vector and Generating a predicted subband coefficient for predicting a subband coefficient from the restored image; adding a difference value of the decoded subband coefficient and the generated predicted subband coefficient; Inverse sub-band transform is performed on the added sub-band coefficients, and an image is restored.

In the image decoding method according to the present invention, a wavelet coefficient is used as the subband coefficient.

In the image decoding method according to the present invention, in the step of generating the predicted subband coefficient, for example,
The restored image is filtered so as to correspond to the frequency band to which the subband coefficient to be predicted belongs, and the predicted value is obtained from the output.

According to the present invention, in a method of transmitting an image signal using a subband filter, a motion vector is detected from the input image signal, and the input image is subjected to frequency band division processing by subband conversion. To generate a subband coefficient, encode the difference between the subband coefficients to generate a coded signal, decode the coded signal to generate a difference between the decoded subband coefficients, and generate the motion vector and the motion vector already. Generate a predicted subband coefficient for predicting the subband coefficient from the restored image being restored, and add the predicted subband coefficient to the difference between the decoded subband coefficients to obtain the restored subband coefficient. The generated subband coefficient is subjected to inverse subband transform to restore the image, and the subband coefficient and the predicted subband coefficient are Calculating a difference, characterized in that so as to transmit a bit stream consisting of the motion vector and the differential subband coefficients.

In the image transmission method according to the present invention, the bit stream has, for example, a header, and the header is added to each band of subband coefficients.

According to the present invention, in a recording medium on which a recording signal that can be decoded by a decoding device is recorded, the recording signal is
A motion vector detected from the input image signal, and a subband coefficient obtained by performing frequency band division processing by subband conversion on the input image,
The difference between the predicted subband coefficient and the predicted subband coefficient is included, and the predicted subband coefficient is generated from the motion vector and the restored image that has already been restored, and the restored image indicates the difference between the subband coefficients. By decoding the coded signal obtained by coding the coded signal, the difference between the decoded subband coefficients is generated, and the predicted subband coefficient is added to the difference between the decoded subband coefficients. It is characterized in that the restored subband coefficient obtained by addition is subjected to inverse subband transform to be restored.

[0042]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIGS. 1A and 1B show the first embodiment of the present invention.
3 shows configurations of an image encoding device and an image decoding device using the wavelet transform according to the embodiment of FIG. However,
It is assumed that each pixel value on the image I is represented by I [i, j], and these are scanned and input in the order shown in FIG.
Here, i represents the horizontal coordinate on the image, and j represents the vertical coordinate.

In the image coding apparatus 10, the input image is supplied to the frame memory 2 via the input terminal 1. The input image supplied to the frame memory 2 is supplied to the wavelet converter 3. The wavelet converter 3 performs frequency band division processing on the input image. This corresponds to the DCT device in the conventional image coding apparatus. However, in the conventional image coding apparatus, the DCT device is present after the subtractor in order to perform motion compensation on the image, but in the present invention, the motion compensation is performed on the wavelet coefficient after the wavelet transform. In order to perform the above, the wavelet converter 3 is installed before the subtractor 4. Further, the wavelet transformer 3 executes wavelet transformation in units of one screen image.

That is, in this image coding apparatus 10,
The input image is supplied to the frame memory 2, and the motion vector v is detected by the motion vector detector 11 from the input image stored in the frame memory 2. As a method of detecting a motion vector, usually, block matching is performed for each block of 16 pixels × 16 pixels. Further, in order to realize higher accuracy, matching is performed on a half-pixel basis. Further, other known motion vector detection methods may be used instead of block matching.

Further, in this image encoding device 10,
The input image is supplied to the wavelet converter 3 via the frame memory 2. Then, the wavelet transformer 3 supplies the wavelet coefficient obtained by executing the wavelet transform to the input image to the subtractor 4. The subtractor 4 subtracts the predictive wavelet coefficient from the motion compensator 12 described below from the wavelet coefficient from the wavelet transformer 3 and outputs a differential wavelet coefficient.

The differential wavelet coefficient from the subtractor 4 is supplied to the quantizer 5. The quantizer 5 is a subtractor 4
For example, the quantization process such as the above-described equation (2) is performed on the differential wavelet coefficient from (3) by using an appropriate step size Q.

The differential wavelet coefficient quantized by the quantizer 5 is supplied to the variable length encoder 6 and the inverse quantizer 13. In the inverse quantizer 13, for example, the inverse quantization processing as in the above-described expression (3) is performed with the same step size as that used in the quantizer 5.

The data subjected to the inverse quantization processing by the inverse quantizer 13, that is, the differential wavelet coefficient, is supplied to the adder 14, and the predictive wavelet from the motion compensator 12 which will be described later is supplied to the adder 14. It is added with the coefficient. Then, this added value is sent to the inverse wavelet transformer 15.

Further, the inverse wavelet transformer 15 corresponds to the IDCT device in the conventional image coding apparatus, but is installed after the adder 14 for the same reason as above. The restored image subjected to the inverse wavelet transform is supplied to the motion compensator 12 and stored in a frame memory (not shown) included in the motion compensator 12.

The motion compensator 12 is the motion vector detector 1
The wavelet coefficient to be encoded at present is predicted from the motion vector v sent from 1 and the restored image already stored in the frame memory (not shown). The wavelet coefficient predicted by the motion compensator 12 is sent to the subtractor 4 as a predictive wavelet coefficient, and the difference from the wavelet coefficient to be encoded at present is calculated. Further, the predicted wavelet coefficient is sent to the adder 14 and added to the dequantized differential wavelet coefficient to restore the wavelet coefficient.

The variable length encoder 6 performs variable length encoding on the quantized differential wavelet coefficient obtained by the quantizer 5 and the motion vector v obtained by the motion vector detector 11, The variable length coded signal is output to the header adder 7. The header adder 7 adds each header described below to the supplied variable length coded signal and outputs it as a bit stream. This bit storm is supplied to a recording device (not shown), and a video disc (for example,
It is recorded on a recording medium 18 such as an optical, magnetic or magneto-optical disk), a video tape or a semiconductor memory. Or
This bit storm is supplied to a transmission device (not shown),
It is transmitted via the transmission line 19.

In the image decoding apparatus 20, first, the recording medium 1 such as a video disk (for example, an optical, magnetic or magneto-optical disk), a video tape or a semiconductor memory.
The bit stream reproduced by the reproducing apparatus (not shown) from 8 is supplied to the input terminal 21. Alternatively, the bit stream transmitted via the transmission path 19 is received by a receiving device (not shown), and the received bit stream is supplied to the input terminal 21. Then, the bit stream supplied to the input terminal 21 is supplied to the inverse variable length encoder 22. The inverse variable length encoder 22 separates the header added by the image encoding device 10, and
The inverse processing of the variable length encoder 6 in the image encoding device 10 is performed, and the quantized differential wavelet coefficient and the motion vector v are restored from the bit stream. The differential wavelet coefficient obtained by the inverse variable length encoder 22 is supplied to the adder 24 via the inverse quantizer 23, and the motion vector v is supplied to the motion compensator 12.

The inverse quantizer 23 and the adder 24 have the same functions as the inverse quantizer 13 and the adder 14 in the image encoding apparatus 10, and the difference obtained by the inverse variable length encoder 22. The wavelet coefficient is subjected to the above-described inverse quantization processing of the equation (3) by the inverse quantizer 23,
The wavelet coefficient is restored by adding the predicted wavelet coefficient from the motion compensator 26 described later in the adder 91.

Then, the wavelet coefficient thus restored is supplied to the inverse wavelet transformer 25. The inverse wavelet transformer 25 has the same function as that of the image coding device 10, and restores and outputs an image corresponding to the input image to the image coding device 10. Further, each pixel value of the restored image is stored in a frame memory (not shown) provided in the motion compensator 26 to generate a predicted image.

The motion compensator 26 has the same function as that of the image coding apparatus 10, and the motion vector v obtained by the inverse variable length coder 22 and the motion compensator 12 are not shown. The image already stored in the frame memory is used to generate a prediction wavelet coefficient by the method of calculating a prediction value described later, and the prediction wavelet coefficient is supplied to the adder 24. Then, the adder 24 adds the difference wavelet coefficient and the predicted wavelet coefficient.

FIG. 3 shows the configuration of the wavelet converter 3. This wavelet converter 3 shows a two-stage band division process for dividing the band into seven bands. First, the input image data I [i, j] is supplied to the analysis horizontal low-pass filters 30A and 30B in order to perform frequency band division processing in the horizontal direction on the image. As these analysis filters, for example, linear filters having coefficients as shown in Table 1 below can be used.

[0058]

[Table 1]

The analysis filter and the synthesis filter used in the wavelet division / synthesis are given by the following equation (4).
And the relation of the equation (5) are satisfied completely or approximately.

[0060]

(Equation 4)

[0061]

(Equation 5)

H ₀ (z), H ₁ (z), F ₀ (z) and F ₁
(Z) are transfer functions of the analysis low-pass filter, the analysis high-pass filter, the synthesis low-pass filter, and the synthesis high-pass filter, respectively, and L is an arbitrary integer. This constraint condition guarantees that the synthesized signal completely or approximately matches the input signal before band division. The filter coefficients shown in Table 2 below are used as the filter coefficients for synthesis corresponding to the filters shown in Table 1.

[0063]

[Table 2]

The horizontal low frequency band signal L [i, j] obtained by the analysis horizontal low pass filter 30A and the horizontal high frequency band signal H [i, j] obtained by the analysis horizontal high pass filter 30B correspond to each other. Are supplied to the horizontal sub-sampling devices 31A and 31B, and the horizontal sub-sampling devices 31A and 31B perform thinning processing for each sample as in the following Expression (6).

[0065]

(Equation 6)

The horizontal low frequency band signal L subjected to the thinning processing by the horizontal sub-sampling devices 31A and 31B.
[I ', j] and horizontal high frequency band signal H [i', j]
The supplied memory 32A, 32B is a storage device including a plurality of line memories for securing data required in the analysis vertical low-pass filters 33A, 33C and the analysis vertical high-pass filters 34B, 34D. The number of line memories is equal to the number of taps of the filter used for the filtering process. For example, when the filter shown in Table 1 is used, nine line memories are installed in accordance with the low-pass filter having a large number of taps.

Vertical low-pass filters for analysis 33A, 33
The C and analysis vertical high-pass filters 34B and 34D perform vertical low-pass filtering processing and high-pass filtering on the image with respect to the data stored in the memories 32A and 32B in order to perform frequency band division in the vertical direction. Processing is performed. As the filter used here, the same filter as that used in the horizontal filtering process may be used.

The output signal LL of the analysis vertical low pass filter 33A for the horizontal low frequency band signal L [i ', j].
[I ′, j], the output signal LH [i ′, j] of the analysis vertical high pass filter 33B, and the horizontal high frequency band signal H.
Analysis vertical low-pass filter 33 for [i ', j]
The output signal HL [i ′, j] of C and the output signal HH [i ′, j] of the vertical high-pass filter for analysis 33D correspond to the corresponding vertical sub-samplers 34A, 34B, 34, respectively.
The sub-sampling processing in the vertical direction on the image, that is, the thinning processing for each line, is supplied to C and 34D.
It is performed as in the following Expression (7).

[0069]

(Equation 7)

Each frequency band signal LH [i ', j'], HL [i ',
j ′] and HH [i ′, j ′] are output from the wavelet converter 3 as they are. On the other hand, LL subjected to low-pass filtering processing in both the horizontal and vertical directions.
[I ′, j ′] is an input signal to the second-stage analysis horizontal low-pass filter 35A and analysis horizontal high-pass filter 35B, and the horizontal sub-samplers 36A, 36
B, memories 37A and 37B, analysis vertical low-pass filters 38A and 38C, analysis vertical high-pass filter 38
B, 38D and vertical sub-samplers 39A, 39B, 3
49 and 39D perform exactly the same processing as the band division processing of the first stage, and further, band-divided signals LLLL [i ", j"], LHLL [i ", j"], H
LLL [i ", j"] and HHLL [i ", j"] are generated.

Output signal L from the wavelet converter 3
LLL [i ", j"], LHLL [i ", j"], HL
LL [i ", j"], HHLL [i ", j"], LH
[I ', j'], HL [i ', j'], HH [i ',
j ′] is the wavelet coefficient corresponding to each frequency band.

Although not shown, in the band division processing of the third and subsequent stages, the same processing as the band division processing of the second stage is executed for the output signal LLLL.

FIG. 4 shows the configuration of the inverse wavelet converter 15. Here, each frequency band signal obtained by the wavelet converter 3 shown in FIG. 3, that is, the wavelet coefficients LLLL [i ″, j ″], LHLL.
[I ", j"], HLLL [i ", j"], HHLL
[I ", j"], LH [i ', j'], HL [i ',
j ′] and HH [i ′, j ′] are sequentially combined to restore the image.

First, each of the eight frequency band signals is received, and among the input frequency band signals, the wavelet coefficient LLLL subjected to the two-stage band division processing.
[I ", j"], LHLL [i ", j"], HLLL
[I ", j"] and HHLL [i ", j"] are combined. First, these signals are stored in the memories 40A, 40B, 40C and 40, each of which is composed of a plurality of line memories.
Vertical low-pass filter for synthesis 4 stored in D
2A, 42C and a vertical high-pass filter for synthesis 42B,
Data for the number of lines required for 42D is secured.
Here, the required number of lines is (the number of taps + 1) / 2 when the number of taps of the vertical filter is odd, and the number of taps / 2 when the number of taps is even. For example, when the low pass filter shown in Table 2 is used, two line memories are installed in front of the combining vertical low pass filters 42A and 42C. Here, the memory 40
The number of line memories required in A, 40B, 40C, and 40D is different from that on the image encoding device 10 side, as will be described later, in combination vertical low-pass filters 42A and 42A.
This is because the up-sampling processing in the vertical direction is performed before the interpolation processing by the C and combining vertical high-pass filters 42B and 42D.

That is, the memories 40A, 40B, 40
The signals LLLL [i ", read from C and 40D,
j ″], LHLL [i ″, j ″], HLLL [i ″,
j ″] and HHLL [i ″, j ″] are supplied to the vertical upsamplers 41A, 41B, 41C and 41D, respectively.
In 41C and 41D, as shown in the following equation (8), a process of inserting one line of data, which is all zero, is performed between the input lines.

[0076]

(Equation 8)

Vertical up-samplers 41A, 41
The frequency band signals subjected to the vertical upsampling processing by B, 41C, and 41D are subjected to vertical interpolation processing by the combining vertical low-pass filters 42A and 42C and the combining vertical high-pass filters 42B and 42D, respectively. To the adders 43A and 43B.
Here, as described above, these synthesizing filters need to satisfy the relationship between the corresponding analysis filter and the expressions (4) and (5), and when the analysis filter shown in Table 1 is used, The filters shown in Table 2 will be used as synthesis filters.

Vertical low-pass filters for synthesis 42A, 42
C and the frequency band signal LLLL [i ", which is interpolated by the vertical high-pass filters for synthesis 42B and 42D,
j '], LHLL [i ", j'], HLLL [i",
j ′] and HHLL [i ″, j ′] are respectively added by the adder 4
After being added by 3A and 43B, they are supplied to the horizontal upsamplers 43A and 43B, and the upsampling processing in the horizontal direction as in the following equation (9) is performed.

[0079]

[Equation 9]

Then, the horizontal low-pass filter for synthesis 45
The A and synthesizing horizontal high-pass filter 45B performs a filtering process in the horizontal direction on each signal subjected to the horizontal upsampling process, and performs an interpolation process. The adder 46 receives the two signals LLL [i ′,
j ′] and HLL [i ′, j ′] are added to restore the low frequency band signal LL [i ′, j ′].
Then, the low frequency band signal LL [i ′, j ′] restored by the adder 46 is stored in the memory 47A.

Next, this restored LL [i, 'j']
And the input frequency band signals LH [i ', j'], H
L [i ', j'] and HH [i ', j'] are read from the corresponding memories 47A, 47B, 47C, 47D, respectively, and the bands as described above are applied to these read signals. The same processing as the combining processing is applied to the image signal I
[I, j] is restored.

That is, the memories 47A, 47B, 47
The signals LL [i ', j'] read from C and 47D,
LH [i ', j'], HL [i ', j'], HH
[I ', j'] is subjected to vertical upsampling processing by the vertical upsampling units 48A, 48B, 48C, 48D, respectively, and further by the synthesis vertical low-pass filters 49A, 49C and the synthesis vertical high-pass filters 49B, 49D. Vertical interpolation processing is performed. Frequency band signals LL [i ', j], LH interpolated by the synthesizing vertical low-pass filters 49A, 49C and the synthesizing vertical high-pass filters 49B, 49D.
[I ', j], HL [i', j], HH [i ', j]
Are added by adders 50A and 50B, respectively, and then supplied to horizontal up-sampling devices 51A and 51B to be subjected to horizontal up-sampling processing.

Then, the horizontal low-pass filter for synthesis 52
The A and synthesizing horizontal high-pass filter 52B performs a filtering process in the horizontal direction on each signal subjected to the horizontal upsampling process, and performs an interpolation process. The adder 53 receives the two signals L [i, j], which have been subjected to the interpolation processing,
By adding H [i, j], the image signal I
Restore [i, j].

Although not shown, when the image signal is divided into 10 bands, the input signal LLLL
A similar compositing process is performed to form

FIG. 5 shows a method of calculating the predicted value (predicted wavelet coefficient) in the motion compensators 12 and 26.
I'is the image already restored by the inverse wavelet transformers 15 and 25 and stored in the frame memory (not shown) of the motion compensators 12 and 26, and the wavelet for which W [p, q] should be predicted. The coefficient. Now, assuming that this wavelet coefficient is obtained in the N-th band division, N times of subsampling processing is performed in each of the horizontal direction and the vertical direction until the coefficient is obtained. It has been given. This indicates that this W [p, q] corresponds to the position (k, l) = (p × 2N, q × 2N) as in the following equation on the original image. Therefore, the predicted value (predicted wavelet coefficient) is four positions (k ′, l ′), (in the image I ′ on the unillustrated frame memory, as shown by the equation (1) in the conventional example. k '+ 1, l'),
It can be calculated from (k ', l' + 1) and (k '+ 1, l' + 1). Here, k ′, l ′ is the motion vector v = (vx (k, l), vy (k, l) obtained by the motion vector detector 11 and corresponding to the position (k, l).
l)) is an integer coordinate value represented by the following equations (10) and (11).

[0086]

(Equation 10)

[0087]

[Equation 11]

Here, as in the equation (1), T is the time when the image corresponding to the currently predicted wavelet coefficient is input and the time when the image I'on the frame memory is input. , And int (x) represents the maximum integer value that does not exceed x. However, the data on the frame memory is the pixel value of the image I ′, and in order to predict the wavelet coefficient W [p, q], the pixel value I ′ [k ′, l ′] at the above four positions is calculated. , I '[k' +
1, l], I '[k', l '+ 1], I' [k '+ 1,
It is necessary to convert l ′ + 1] into wavelet coefficients in the frequency band to which W [p, q] belongs. Below, image I '
The wavelet coefficient LHLL [i] obtained by the second division in the wavelet converter 14 shown in FIG. 3 is a method for converting the pixel value at an arbitrary position above into a wavelet coefficient in a desired frequency band. ", J"] will be described as an example.

As can be seen from FIG. 3, the wavelet coefficients LHLL [i ", j"] are two horizontal low-pass filters, one vertical low-pass filter, and one horizontal low-pass filter for the input image signal. It is calculated by applying a vertical high-pass filter. However, in the first stage, while the filtering process is performed on the input image,
The second-stage filtering process is performed after the sub-sampling process is performed once in both the horizontal and vertical directions. Therefore, the second-stage filtering process is equivalent to applying a filter obtained by subjecting the image before sub-sampling to the original filter coefficient to the up-sampling process as shown in equation (9). For example, when the analysis filter shown in Table 1 is used, the filtering process in the second stage corresponds to applying the filter shown in Table 3 below to the input image. More generally, the filtering process performed in the M-th division is
This is equivalent to applying a filter obtained by performing (M-1) times of upsampling to the original filter on the input image.

[0090]

[Table 3]

Therefore, the pixel value on the image I'stored in the frame memory (not shown) is set to LHLL [i ", j"].
In order to convert to the wavelet coefficient of the frequency band to which the belongs, the original analysis low-pass filter filter and the low-pass filter obtained by up-sampling the same in the horizontal direction, and the analysis low-pass filter in the vertical direction,
A high-pass filter obtained by up-sampling the analysis high-pass filter may be applied.

By the way, as shown in the following equation (12),
Two consecutive filtering processes can be realized by a new filter obtained by convoluting filters. Here, f and g are arbitrary linear filters, and * represents a filtering process, that is, a convolution operation.

[0093]

(Equation 12)

Therefore, the upsampling process for the analysis filter used for the wavelet division and the convolution operation of the filters obtained by the upsampling process are performed in advance, and the horizontal filter corresponding to each frequency band and the vertical filter corresponding to each frequency band are obtained. A filter can be generated. By using the filter generated in this way (hereinafter referred to as a joint analysis filter), the wavelet coefficient W '(k',
l ′) can be calculated by the processing shown in FIG.

That is, the current position of interest (k ', l')
And a horizontal combination analysis filter is placed so that outputs can be obtained at positions having the same horizontal coordinate k'on the upper and lower lines, and in each line, the inner product operation of the filter coefficient and the corresponding pixel value is performed, Obtain the output value of the filter. At this time, the number of lines used should be equal to the number of taps of the vertical coupling analysis filter, and
It is chosen so that when the vertical filter is applied to these lines, the output value is obtained at the vertical coordinate l '. After the horizontal operation is performed on a plurality of lines, the inner product operation of the obtained vertical data string and the coefficient of the vertical joint analysis filter is performed to obtain the position (k ', l').
Thus, the wavelet coefficient W '(k', l ') at is obtained. Similarly, the wavelet coefficients W ′ (k ′, l ′), W ′ at the four positions shown in FIG.
(K '+ 1, l'), W '(k', l '+ 1), W'
The predicted value W ″ (p, q) of the coefficient W (p, q) to be currently predicted is obtained by obtaining (k ′ + 1, l ′ + 1) and substituting it in the following equation (13).

[0096]

(Equation 13)

Next, this motion compensator 12 as described above is used.
The operation of will be described using a flowchart. FIG. 7 shows a flowchart of the operation for obtaining the predicted value in the motion compensator 12.

First, in step S1, the position (k, l) on the original image corresponding to the wavelet coefficient W [p, q] to be predicted is expressed by the equation (k, l) = (p × 2N,
q * 2N).

Next, in step S2, the motion vector v = (vx (k, l), from the motion vector detector 11 is calculated.
vy (k, l)) on the original image to four positions (k, l), (k + 1, l), (k, l) in the image I.
Motion compensator 12 corresponding to (+1), (k + 1, l + 1)
Image I ′ stored in the frame memory (not shown) of
At four positions (k ', l'), (k '+ 1,
l '), (k', l '+ 1), (k' + 1, l '+ 1)
Are sequentially calculated by the above equations (10) and (11).

In step S3, it is determined whether or not all four positions in the image I'stored in the frame memory (not shown) of the motion compensator 12 have been calculated. If not, the process returns to step S2 to continue the calculation. Also, if 4
If all the calculations for one position have been completed, the process proceeds to step S4.

In step S4, the four positions (k ', l') and (k '+ 1, calculated in step S2 are calculated.
l '), (k', l '+ 1), (k' + 1, l '+ 1)
Pixel values I '[k', l '], I' [k '+ 1,
l], I '[k', l '+ 1], I' [k '+ 1, l'
+1] and neighboring pixel values necessary for calculating the wavelet coefficients corresponding to these are read out from the frame memory, and the wavelet coefficients W [p, q] to be predicted are calculated.
Of the wavelet coefficient W [p, q] to be predicted using the joint analysis filter coefficient corresponding to the frequency band to which
Of the frequency band to which W belongs (W '(k',
l '), W' (k '+ 1, l'), W '(k', l '+
1), W '(k' + 1, l '+ 1).

In step S5, four wavelet coefficients W '(k', l '), W' (k '+ 1, l'),
W '(k', l '+ 1), W' (k '+ 1, l' + 1)
Has been calculated, and if all four pixel values have not been calculated, the process returns to step S4 to continue the calculation. If all four wavelet coefficients have been calculated, step S6
Proceed to.

In step S6, the four wavelet coefficients W '(k',
l '), W' (k '+ 1, l'), W '(k', l '
+1), W ′ (k ′ + 1, l ′ + 1), and the equation (1
3), the wavelet coefficient W (p,
The predicted value W ″ (p, q) of q) is calculated. Such processing is executed for each wavelet coefficient W (p, q) to be predicted.

Next, the format of the bit stream output from the image coding apparatus 10 of the present invention will be described with reference to FIGS. 8 to 13. 8 to 13 show
ISO-IEC / JTC1 / SC29 / WG11 IS
The format used in 13818-2 is the format of the bit stream of this embodiment. The part shown in bold is the flag that is actually transmitted. The coded image signal in the present invention includes a sequence, a GOP (group of pictures), a picture,
It has a hierarchical structure composed of layers of macroblocks and blocks. The sequence layer consists of a series of screen groups with the same attributes. The GOP layer is the minimum unit of the screen group that is the unit of random access. The picture layer is composed of one screen having common attributes. The macroblock layer is a collection of coefficients (pixel blocks) existing in the same position in each frequency band. The block layer is composed of transform coefficients, that is, wavelet coefficients themselves. Each layer will be described later.

In FIG. 14, 10 is obtained by the wavelet transform.
The band is shown when the band is divided into bands. Further, FIG. 15 shows the structure of the header adder 7 for forming the bit stream. The sequence header circuit 61 adds (transmits) a sequence header to the beginning of the bit stream. This initializes the decoder. After that, GO in the GOP header circuit 62
A P header is added (transmitted). Furthermore, after that, the picture header in the picture header circuit 63
The macroblock header circuit 64 adds (transmits) a macroblock header. Then, the wavelet coefficient in the macroblock is transmitted as a bitstream. Further, after the transmission of the macroblocks existing in the same picture is completed, the processing for the next picture is started and the same operation is repeated. Also, the same GO
After the processing of the picture existing in P is completed, the next GOP
Then, the same operation is executed.

Here, the macro block will be described. A macroblock is composed of coefficients existing at the same position in each frequency band. That is, as shown in FIG. 14, for example, each of the LLLLLL band, the LLLLHL band, the LLLLHH band, and the LLLLLLH band, each of which has one wavelet coefficient at the same position,
Corresponding to the positions where the coefficients of the above four frequency bands exist, respectively corresponding to four wavelet coefficients existing at the same positions of the LLHL band, the LLHH band and the LLLH band, and the positions where the coefficients of the above seven frequency bands exist. The HL band, the HH band, and the LH band each have 16 wavelet coefficients existing at the same position, and are composed of a total of 64 wavelet coefficients. That is, these coefficients are obtained from images existing in the same position on the original image, though the frequency bands are different. The format of each header will be described below.

FIG. 8 shows the format of the sequence header. sequence_start_cod
e is a 32-bit code and indicates the start of the sequence header. This sequence_start_co
Other than de, various start codes are unique bit strings and are codes that do not occur in other parts of this bit stream. Therefore, if the image cannot be decoded due to a part of the bitstream being lost for some reason, it becomes possible to decode again by finding this start code. horizont
al_size_value is the number of horizontal pixels of the image is 1
It is indicated by 2 bits, and vertical_size_
value indicates the number of vertical pixels of the image in 12 bits. aspect_ratio_informati
on indicates the image aspect ratio in 4 bits,
The frame_rate_code indicates the display cycle of the image with 4 bits. bit_rate_value
Shows the number of bits per second as 18 bits, and mar
ker_bit is a 1-bit value of “1”,
The vbv_buffer_size_value is a 10-bit parameter that determines the size of the code generation amount control virtual buffer.

[Wa] in this sequence header
"velet_layer_number" indicates how many layers (the number of four-band synthesis band synthesis filter banks) the inverse wavelet transform circuit 15 is composed of.
It is a bit flag. Also, wavelet_coe
fincients () defines the filter coefficient used in the filter of each layer of the inverse wavelet transform circuit. Here, there are two methods for transmitting each filter coefficient of the inverse wavelet transform.

FIG. 13A shows the format for transmitting the flag corresponding to the filter coefficient.
That is, the number of filter coefficients possessed by each filter of each layer is determined in advance, and a flag indicating which one is used is transmitted. The combination of the high-pass filter and the low-pass filter is determined by determining either one. Therefore, it is sufficient to transmit one index to each layer. In FIG. 13A, this index wavelet_coeff_i
The index is an 8-bit flag, and it is possible to specify 256 combinations of filter coefficients. In the present embodiment, it is 8 bits, but any number of bits may be used.

Further, another transmission format of the inverse wavelet transform coefficient is shown in FIG. 13 (B). This is a method of directly transmitting the filter coefficient possessed by each layer. In this case, the number of taps of the low-pass filter and the high-pass filter and the filter coefficient according to the number of taps are transmitted. That is, the number of taps of the low-pass filter is 8 according to lowpass_tap_number.
A filter coefficient indicated by a bit flag and corresponding to the number of taps is transmitted by wavelet_coeff []. In addition, the number of taps of the high-pass filter is high
It is indicated by an ass_tap_number with an 8-bit flag, and the filter coefficient corresponding to the number of taps is w
It is transmitted as aavelet_coeff [].

Also, load_i in the sequence header
ntra_weighting_matrix and l
oad_non_intra_weighting_m
Atrix is a flag indicating whether or not to download the weighting factors used in the intra macroblock and the other macroblocks. When downloading, each flag will be followed by a weighting factor. This is 8 bits x (wavelet_layer_nu
The data size is m ber × 3 + 1). (The number of bands is determined by wavelet_layer_number, which is wavelet_layer_num
ber × 3 + 1. ) The weight coefficient used is different between the intra macroblock and other macroblocks.

Next, the GOP header is added (transmitted). This is a header indicating the beginning of GOP and is mainly used for random access. FIG. 9 shows the GOP header. GOP header is group_star
t_code is a code indicating the start of GOP and is 3
It is represented by 2 bits. The time_code indicates the time from the beginning of the sequence with a 25-bit flag. Closed_gop is a 1-bit flag and indicates that an image in a GOP can be independently reproduced from another GOP. broken_link indicates that the preceding GOp data cannot be used for editing. Next to the 1-bit GOP header, a picture header indicating the beginning of the data of each picture is added (transmitted).

FIG. 10 shows the format of the picture header. number_of_macroblo
ck is a 16-bit flag indicating the number of macroblocks existing in the frame. As another flag, p
image_start_code is a 32-bit flag indicating the start of a picture, and is a temporal
_Reference is the consistent No. of the picture. And is a 10-bit flag. picture_co
ding_type is a 3-bit flag indicating the picture type, and vbv_delay is a 16-bit flag indicating the initial state of the buffer during random access. Also, full_pel_forward_
vector is a 1-bit flag indicating whether the precision of the motion vector is an integer or a half-pixel unit, and forward_
f_code indicates the range of description of the forward motion vector. full_pel_backward_ve
ctor is a 1-bit flag indicating whether the precision of the motion vector is an integer or a half-pixel unit, and backword_f
_Code indicates the range of description of the forward motion vector.

Further, FIGS. 12A and 12B show the format of the motion vector header. mot
ion_vectors (s) is a subroutine mot
has ion_vector (s), and
The n_horizontal_code is a variable-length encoded version of the difference between the horizontal component of the motion vector of the macroblock and the previous motion vector, and is a flag of 1 to 11 bits. motion_
horizontal_r is motion_hori
This flag is used together with the zone_code and is indicated by any one of 1 to 8 bits. dmv_h
The horizontal is a variable length coding that represents a horizontal difference vector in the case of dual prime prediction, and is a flag of either 1 or 2 bits. motion
_Vertical_code represents a difference between the vertical component of the motion vector of the macroblock and the previous motion vector, which is variable-length coded, and is a flag of 1 to 11 bits. motion_ver
temporal_r is motion_vertical_
This flag is used together with the code and is indicated by any one of 1 to 8 bits. dmv_vertical
Is a variable length coding that represents a horizontal difference vector in the case of dual prime prediction, and is a flag of either 1 or 2 bits.

Next, the first transmission method of picture data will be described. Consider a case where motion compensation and quantization are performed in macroblock units. In the first transmission method, each motion vector, quantization scale, and wavelet coefficient in a picture are collectively transmitted as one unit. That is, as shown in FIG. 15, in this case, the macroblock header is added (transmitted) by the macroblock header circuit 64 after the picture header, and then the wavelet coefficient corresponding to the macroblock is wavelet coefficient circuit 65. Transmitted by. After the same transmission for all macroblocks in a picture, the next picture header or GOP header is transmitted. FIG. 11 shows the macroblock header in this case. More specifically, macroblock_escape indicates an escape code represented by 11 bits.
roblock_address_increment
Is a variable length coding code representing the number of macroblocks to be skipped +1, that is, a flag representing the number of macroblocks +1 from the right end of the image, and this flag is one of 1 to 11 bits. Is. ma
“Croblock_type” indicates a variable-length coding code that represents a coding mode of a macroblock, and is a flag of 1 to 9 bits. macrobl
ock_size indicates the size of the macroblock, which is 2
It is a bit flag. quantizer_scal
e_code is a 5-bit flag indicating a quantization step value. mc_code is a 1-bit flag indicating the prediction direction of the motion vector. intra_dc_
size_luminance indicates the size of the luminance signal of the LLLLLLL band of the intra block,
It is a flag from 2 to 9 bits and is intra_dc_d
differential is a 1- to 11-bit flag indicating the value. intra_dc_size_c
romance is the intrablock LLLLLL
The size of the color difference signal of the L band is shown, and is a flag of 2 to 9 bits, and is an intra_dc_differ.
"ental" is a 1- to 11-bit flag indicating the value.

Next, the second transmission method of picture data will be described. The second transmission method is a modification of the first transmission method. The wavelet coefficient is transmitted for each band. The structure of the header adder 7 for forming the bit stream in this case is shown in FIG. First,
First, the lowest frequency component is transmitted. In the case of FIG. 14, LL
LLLL is transmitted. In this case, the picture header circuit 63 transmits the picture header, and then the band header circuit 66 adds (transmits) the band header as a part of the bit stream. The band header is a header indicating which band of data in FIG. 14 starts, and a sync code is added. After this band header, the macroblock header is transmitted by the macroblock header circuit 64, and further, the corresponding coefficient of the LLLLLLL band is transmitted from the wavelet coefficient circuit 65. After that, the band header to be transmitted next is transmitted, and the same operation is repeated. After all bands have been processed, the next picture or GOP header transmission processing starts.

FIG. 17 shows the order of transmitted bands. In the next band, after the band header is transmitted, L
Wavelet coefficients are transmitted in the same order as LLLLL. At this time, the macroblock header is not transmitted.
That is, as described above, since the macroblock is composed of wavelet coefficients of each band, LLL
This is because the macroblock header has already been added (transmitted) when transmitting the LLL band coefficient. After the same transmission for all bands, the next picture header or GOP header is transmitted. The format of this band header is shown in FIG. 18, and the header of the macroblock header in this case is shown in FIG. As shown in FIG. 18, band_sta
It has flags of rt_code and band_id.
band_start_code is a 32-bit flag indicating the start of a band, and band_id is a 6-bit flag indicating the band.

Next, a third transmission method of picture data will be described. The third transmission method is a modification of the first transmission method. The wavelet coefficient can be configured in a tree shape as shown in FIG. That is, LLLL
The HL band, the LLHL band, and the HL band are configured as one tree (tree 1), and the LLLLHH band,
LLHH band and HH band are configured as one tree (tree 2), and further LLLLLLH band, LL
One tree for LH band and LH band (Tree 3)
Configure as. The third transmission method transmits data in units of this tree. The structure of the header adder 7 for forming the bit stream in this case is shown in FIG.
First, the tree 0 (LLLLLL band) is transmitted first. That is, after the picture header circuit 63 transmits the picture header, the tree header circuit 67 transmits the tree header. This tree header is a header indicating which tree data starts, and a sync code is added. After this tree header, the macroblock header is transmitted by the macroblock header circuit 64, and the corresponding wavelet coefficient of the tree 0 is further transmitted by the wavelet coefficient circuit 75.

The order of transmitted trees is tree 0, tree 1, tree 2 and tree 3 in FIG. After the tree header is transmitted in the next tree, the wavelet coefficients are transmitted in the same order as in tree 0. At this time,
As with the second transmission method, the macroblock header is not transmitted. That is, as described above, since the macroblock is composed of the wavelet coefficients of each band, when transmitting the coefficients of the LLLLLLL band,
This is because the macroblock header has already been added (transmitted). After the same transmission for all trees, the next picture header or GOP header is transmitted. FIG.
2 and Table 4 below show the tree header. That is, tree_start_code is a 32-bit flag indicating the start of the tree, and tree_id is
It is a 2-bit flag indicating the tree.

[0120]

[Table 4]

In the above second and third transmission methods,
A sync code, that is, a band header and a tree header is added to each band and tree.
Therefore, due to an error in a certain band or tree,
Even if decoding becomes impossible, data of another band or another tree can be decoded and fatal deterioration of image quality can be avoided. Therefore, the second and third transmission methods are error-resistant transmission methods. The above transmission method can be applied to a coding method using the wavelet transform.

Next, the configuration of the second embodiment according to the present invention will be described. The configuration of the second embodiment is the same as that of the first embodiment shown in FIG. 1, but the configuration of the motion compensator used is different. FIG. 23 shows the configuration of the motion compensator according to the second embodiment.

The frame memory 71 corresponds to the frame memory (not shown) in the first embodiment,
Save the image I ′ restored by the inverse wavelet transform. The combination analysis filters 72A to 72G are combination analysis filters corresponding to the respective frequency bands, and with respect to the entire restored image I ′ on the frame memory 71, horizontal combinations corresponding to the respective frequency bands described in the first embodiment. , And the vertical coupling analysis filter process are performed, and the obtained wavelet coefficients are stored in the corresponding frame memory in the frame memory group 73.

The predictive value calculator 74 calculates the wavelet coefficients corresponding to all the frequency bands as the frame memory group 73.
Then, the prediction of all wavelet coefficients for the image currently to be encoded is started. This prediction process is performed by the coefficient W to be predicted in the frame memory group 73.
This can be realized by referring to the data on the frame memory corresponding to the frequency band to which (p, q) belongs and substituting it in the equation (13). The operation of the predicted value calculator 36 will be described using a flowchart.

FIG. 24 is a flow chart for explaining the operation of the prediction value calculator 74. First, step S1
1, the image I ′ stored in the frame memory 71
On the other hand, it is determined whether the wavelet coefficients corresponding to all the frequency bands are prepared in the frame memory group 73. If the wavelet coefficients corresponding to all the frequency bands are not prepared in the frame memory group 73, the process waits until they are prepared. If it is determined that the wavelet coefficients corresponding to all the frequency bands are prepared in the frame memory group 73, step S1
Proceed to 2. In step S12, the position (k, l) on the original image corresponding to the wavelet coefficient W [p, q] to be predicted is expressed by the equation (k, l) = (p × 2N, q × 2).
N).

Next, in step S13, the motion vector v = (vx (k,
l), vy (k, l)), the four positions (k, l), (k + 1, l), in image I on the original image are
Four positions (k ', l'), (k '+ in image I'stored on the frame memory (not shown) of the motion compensator 12 corresponding to (k, l + 1), (k + 1, l + 1)
1, l '), (k', l '+ 1), (k' + 1, l '+
1) is sequentially calculated by the above equations (10) and (11).

In step S14, it is determined whether all four positions in the image I'stored in the frame memory (not shown) of the motion compensator 12 have been calculated. If all four positions have been calculated, If not, the process returns to step S13 to continue the calculation. If all four positions have been calculated, the process proceeds to step S15. In step S15, the data on the frame memory corresponding to the frequency band to which the wavelet coefficient W (p, q) to be predicted belongs is referred to, and step S1
Read out the wavelet coefficients corresponding to the four positions calculated in 3. Then, in step S16, the read four values are substituted into the equation (13) to obtain the wavelet coefficient W to be predicted.
The predicted value W ″ (p, q) of (p, q) is calculated.

Therefore, the second embodiment of the present invention requires as many frame memories as the number of frequency bands obtained by wavelet division, but the process of converting a restored image into wavelet coefficients is performed at once. Therefore, the duplication of the filtering operation as in the first embodiment does not occur.

Next, the configuration of the third embodiment according to the present invention will be described. FIG. 25 shows the configuration of the third embodiment of the present invention. The configuration of the third embodiment is almost the same as that of the first embodiment shown in FIG. 1, except that it is behind the wavelet transformer 3 on the image encoding device 10 side and before the inverse wavelet transformer 15. And a second frame memory 8 and a third frame memory 16, respectively. The second frame memory 8 stores the wavelet coefficients subjected to the wavelet transform for one image and sequentially outputs the divided frequency bands. In addition, the third frame memory 16 stores the dequantized wavelet coefficients sequentially supplied for each frequency band for one screen, and stores each coefficient in the same order as input to the second frame memory 8. Rearrange it so that it is output. Correspondingly, the frame memory 28 is installed in front of the inverse wavelet converter 225 also on the image decoding device 20 side.

Further, the motion compensators 17 and 29 perform prediction processing for each frequency band. FIG. 26 shows the configuration of the motion compensators 17 and 29 of the third embodiment.

The frame memory 81 corresponds to the frame memory (not shown) in the first embodiment,
Save the image I ′ restored by the inverse wavelet transform. Further, the combination analysis filter 82 uses the combination analysis filter coefficient corresponding to the control signal cntrl sent from the prediction value calculator 84 to calculate the frame memory 8
The whole image I ′ on 1 is filtered, and the obtained wavelet coefficient is stored in the frame memory 83. The prediction value calculator 84 performs the prediction process for each frequency band. Before starting the prediction process for each frequency band, the cntrl signal is sent to the coupling analysis filter 82 to notify the frequency band for which the prediction process is to be performed, and The filtering process using the corresponding combination analysis filter coefficient is performed. When the wavelet coefficient of a desired frequency band for one image is prepared in the frame memory 83, the prediction process is started. The operation of the predicted value calculator 84 will be described using a flowchart.

FIG. 27 is a flow chart for explaining the operation of the predicted value calculator 84. First, step S2
1, the wavelet coefficient W (p,
control signal c corresponding to the frequency band to which q) belongs
ntrl is supplied to the combination analysis filter 82. As a result, the combination analysis filter 82 executes the filtering process using the combination analysis filter coefficient corresponding to the control signal on the image I ′ supplied from the frame memory 81. Next, in step S22, it is determined whether the wavelet coefficient processed by the combination analysis filter 82 is prepared in the frame memory 83. if,
If the wavelet coefficient corresponding to the requested frequency band is not prepared in the frame memory 83, the process waits until it is prepared. If it is determined that the wavelet coefficient corresponding to the requested frequency band is prepared in the frame memory 83, the process proceeds to step S23.

In step S23, the position (k, l) on the original image corresponding to the wavelet coefficient W [p, q] to be predicted is expressed by the equation (k, l) = (p × 2N, q ×). Two
N). Next, in step S24, the motion vector v = from the motion vector detector 11
Using (vx (k, l), vy (k, l)), four positions (k, l) and (k +
1, l), (k, l + 1), (k + 1, l + 1) corresponding to four positions (k ′, 4) in the image I ′ stored in the frame memory (not shown) of the motion compensator 12.
l '), (k' + 1, l '), (k', l '+ 1),
(K ′ + 1, l ′ + 1) are, in order, the above equations (10),
Calculated by (11).

In step S25, it is determined whether all four positions in the image I'stored on the frame memory (not shown) of the motion compensator 12 have been calculated. If all four positions have been calculated. If not, the process returns to step S24 to continue the calculation. If all four positions have been calculated, the process proceeds to step S25.

In step S26, the frame memory 45
With reference to the above data, the wavelet coefficients corresponding to the four positions calculated in step S23 are read. Then, in step S27, the read out four values are substituted into the equation (13) to obtain the predicted value W ″ of the wavelet coefficient W (p, q) to be predicted.
(P, q) is calculated.

Therefore, the third embodiment of the present invention requires the frame memories 81 and 83 for rearranging the wavelet coefficients, but the filtering operation does not overlap as in the first embodiment. Further, the frame memory for storing the wavelet coefficient in the motion compensator may be one image.

In the above-described embodiment, in the wavelet conversion, the LL signal obtained in the first stage is further divided by another stage. However, in the corresponding division / synthesis, As long as a filter pair satisfying equations (4) and (5) is used, the band for division / combination and the number of divisions are arbitrary, and the present invention can be applied to any method.

Further, in the above embodiment, the same filter pair is used in the division / synthesis of the first and second stages.
It is also possible to use different filter pairs in each stage.
Furthermore, it is also possible to perform horizontal and vertical divisions with different filter pairs.

Further, the present invention relates to motion compensation, and any motion detecting method can be used.

Further, in the conventional motion compensation method, there is one in which the forward prediction and the backward prediction are performed using the images of the past and the future for the image to be currently predicted. , Can be predicted in both directions.

In the above embodiment, the wavelet filter is used, but the same processing can be performed by using the subband filter.

Further, although the present invention has been described with reference to the block diagram in the present embodiment, the present invention is not limited to this, and
The present invention may be expressed as software and processed by a CPU or the like.

[0143]

As described above, according to the present invention,
The edge-like deterioration appearing on the block boundary is mitigated by the wavelet inverse transform or the subband inverse transform,
To provide an image encoding device, an image decoding device, an image encoding method, an image decoding method, an image transmission method, and a recording medium in which encoded image data is recorded, which can suppress the influence on a decoded image. You can draw.

Further, according to the present invention, by adding the sync code to each band of the subband (wavelet), even if the data of a certain band is lost, fatal deterioration on the decoded image is avoided. be able to.

Various modifications and application examples are conceivable without departing from the gist of the present invention. Therefore, the gist of the present invention is not limited to the embodiments.

[Brief description of the drawings]

FIG. 1 is a block diagram showing configurations of an image encoding device and an image decoding device according to a first embodiment of the present invention.

FIG. 2 is a schematic diagram for explaining a scanning direction of an image.

FIG. 3 is a block diagram showing a configuration of a wavelet converter.

FIG. 4 is a block diagram showing a configuration of an inverse wavelet converter.

FIG. 5 is a schematic diagram for explaining a method of calculating a prediction value in a motion compensator.

FIG. 6 is a schematic diagram for explaining a method of inner product calculation for calculating a predicted value.

FIG. 7 is a flowchart showing the operation of the motion compensator of the image encoding device according to the first embodiment.

FIG. 8 is a diagram showing a format of a sequence header of a bitstream.

FIG. 9 is a diagram showing a format of a GOP header of a bitstream.

FIG. 10 is a diagram showing a format of a picture header of a bitstream.

FIG. 11 is a diagram showing a format of a macroblock header of a bitstream.

FIG. 12 is a diagram showing a format of a motion vector header of a bitstream.

FIG. 13 is a diagram showing each coefficient of the inverse wavelet transform.

FIG. 14 is a schematic diagram showing bands divided into 10 bands by wavelet transform.

FIG. 15 is a block diagram showing a configuration of a header adder that constitutes a bitstream according to the first transmission method of the present invention.

FIG. 16 is a block diagram showing a configuration of a header adder that constitutes a bitstream according to a second transmission method of the present invention.

FIG. 17 is a schematic diagram showing the order of bands to be transmitted.

FIG. 18 is a diagram showing a format of a band header according to the second transmission method of the present invention.

FIG. 19 is a diagram showing a format of a macro header according to the second transmission method of the present invention.

FIG. 20 is a schematic diagram showing wavelet coefficients configured in a tree shape.

FIG. 21 is a block diagram showing a configuration of a header adder that constitutes a bitstream according to a third transmission method of the present invention.

FIG. 22 is a diagram showing a format of a tree header according to the third transmission method of the present invention.

FIG. 23 is a block diagram showing a configuration of a motion compensator according to a second embodiment of the present invention.

FIG. 24 is a flowchart showing the operation of the motion compensator of the image encoding device according to the second embodiment.

FIG. 25 is a block diagram showing configurations of an image encoding device and an image decoding device according to a third embodiment of the present invention.

FIG. 26 is a block diagram showing a configuration of a motion compensator according to a third embodiment of the present invention.

FIG. 27 is a flowchart showing the operation of the motion compensator of the image encoding device according to the third embodiment.

[Fig. 28] Fig. 28 is a block diagram illustrating configurations of a conventional image encoding device and image decoding device.

[Explanation of symbols]

2,8,16 frame memory, 3 wavelet converter, 4 subtractor, 5 quantizer, 6 variable length encoder,
7 header adder, 10 image encoding device, 11 motion vector detector, 12, 17 motion compensator, 13 inverse quantizer, 14 adder, 15 inverse wavelet transformer,
20 image decoding device, 22 variable length encoder, 23 inverse quantizer, 24 adder, 25 inverse wavelet transformer, 26, 29 motion compensator, 28 frame memory

Claims (15)

[Claims] 1. An image coding apparatus for an image signal using a subband filter, comprising: a motion vector detecting means for detecting a motion vector from an input image signal; and a frequency band by subband conversion for the input image. Sub-band transforming means for performing division processing to generate sub-band coefficients, coding means for coding the difference between the sub-band coefficients and generating a coded signal, and decoding the coded signal Decoding means for generating a difference in band coefficient, motion compensation means for generating a predicted subband coefficient for predicting the subband coefficient from the motion vector and the restored image already restored, and the decoded subband Adding means for adding the predicted subband coefficient to the coefficient difference to generate a restored subband coefficient; and the restored subband coefficient. And a coefficient difference calculation unit that calculates a difference between the subband coefficient and the predicted subband coefficient. Image encoding device. 2. The image coding apparatus according to claim 1, wherein the subband transforming means generates wavelet coefficients. 3. The motion compensating means performs a filtering process on the restored image so as to correspond to a frequency band to which a wavelet coefficient to be predicted belongs, and obtains a predicted value from the output. The image coding apparatus according to claim 1, wherein 4. An image decoding apparatus for an image signal using a subband filter, receiving means for receiving an image signal including at least a difference value between a motion vector and an encoded subband coefficient, the motion vector and the code. Separation means for separating the difference value of the encoded subband coefficient, decoding means for decoding the difference value of the encoded subband coefficient, and generating a difference value of the decoded subband coefficient, the motion vector And a motion compensating means for generating a predictive subband coefficient for predicting a subband coefficient from the restored image, a difference value of the decoded subband coefficient, and a predictive wavelet coefficient obtained by the motion compensating means. And an inverse substituting means for performing an inverse subband transform on the subband coefficients from the coefficient adding means to restore an image. An image decoding apparatus comprising: a band conversion unit. 5. The image decoding apparatus according to claim 4, wherein the sub-band coefficient is a wavelet coefficient. 6. The motion compensating means performs a filtering process on the reconstructed image so as to correspond to a frequency band to which a subband coefficient to be predicted belongs, and obtains a predicted value from its output. The image decoding device according to claim 4, wherein 7. An image coding method for an image signal using a subband filter, which comprises a step of detecting a motion vector from an input image signal, and a frequency band division process by subband conversion for the input image. And generating a subband coefficient, encoding the difference between the subband coefficients to generate an encoded signal, and decoding the encoded signal to generate a difference between the decoded subband coefficients And a step of generating a prediction subband coefficient for predicting the subband coefficient from the motion vector and a restored image already restored, and adding the prediction subband coefficient to a difference between the decoded subband coefficients. And generating reconstructed subband coefficients, and an inverse subband for the reconstructed subband coefficients. Subjected to conversion, and restoring an image, the image coding method characterized by a step of calculating a difference between the subband coefficients and the predictive subband coefficients. 8. The image coding method according to claim 7, wherein the subband coefficient is a wavelet coefficient. 9. The step of generating the predicted sub-band coefficient performs a filtering process on the restored image so as to correspond to a frequency band to which the sub-band coefficient to be predicted belongs, and outputs a predicted value from the output. The image encoding method according to claim 7, wherein the image encoding method is performed. 10. An image decoding method for an image signal using a subband filter, the step of receiving an image signal including at least a difference value between a motion vector and an encoded subband coefficient, the motion vector and the encoding. A step of separating the difference value of the decoded subband coefficients, a step of decoding the difference value of the coded subband coefficients, and a step of generating a difference value of the decoded subband coefficients; Generating a prediction subband coefficient for predicting a subband coefficient from the image, adding a difference value of the decoded subband coefficient and the generated prediction subband coefficient, and adding the Image substituting is performed on the subband coefficients, and the image is restored. No. method. 11. The image decoding method according to claim 10, wherein the sub-band coefficient is a wavelet coefficient. 12. The step of generating the predicted sub-band coefficient performs a filtering process on the restored image so as to correspond to a frequency band to which the sub-band coefficient to be predicted belongs, and outputs a predicted value from the output. The image decoding method according to claim 10, wherein the image decoding method is performed. 13. An image transmission method for an image signal using a subband filter, wherein a motion vector is detected from an input image signal, and the input image is subjected to frequency band division processing by subband conversion to perform subband conversion. A band coefficient is generated, the difference between the sub-band coefficients is coded to generate a coded signal, the coded signal is decoded to generate a difference between the decoded sub-band coefficients, and the motion vector is already restored. The predicted subband coefficient for predicting the subband coefficient is generated from the restored image, and the predicted subband coefficient is added to the difference between the decoded subband coefficients to generate the restored subband coefficient. , An image is restored by performing an inverse subband transformation on the restored subband coefficient, and the subband coefficient and the predicted subband coefficient Min is calculated and the image transmission method being characterized in that so as to transmit a bit stream consisting of the motion vector and the differential subband coefficients. 14. The image transmission method according to claim 13, wherein the bit stream has a header, and the header is added to each band of the sub-band coefficient. 15. A recording medium on which a recording signal that can be decoded by a decoding device is recorded, wherein the recording signal is a motion vector detected from an input image signal and a subband conversion is performed on the input image. The sub-band coefficient obtained by performing the frequency band division processing and the difference sub-band coefficient between the predicted sub-band coefficient are included, and the predicted sub-band coefficient is the motion vector and the restored image already restored. The decoded image is generated by decoding the coded signal obtained by coding the difference between the subband coefficients to generate a difference between the decoded subband coefficients, Inverse sub-substituting with respect to the restored sub-band coefficient obtained by adding the above-mentioned predicted sub-band coefficient to the above-mentioned decoded sub-band coefficient difference. A recording medium characterized by being restored by performing band conversion.