JPH11177992A - Image decoding device - Google Patents

Abstract

(57) Abstract: In a device for decoding digitally represented image data such as MPEG, the timing margin of a decoding operation is improved. In an image decoding apparatus for decoding a plurality of portions of a screen by a plurality of decoders, a means is provided for preferentially accessing a memory as decoding is delayed among a plurality of partial decoders. In the case of two decoders, the first partial decoder A performs memory access (one of reading a stream, reading predicted data, and writing reproduced data).
When the second partial decoder B makes a request and makes a memory access (one of reading of a stream, reading of predicted data, and writing of reproduced data),
The decoding speed of the partial decoder A is compared with the decoding speed of the second partial decoder (decoder B), and the access of the decoder which is not ahead of the decoding is prioritized.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image decoding apparatus, and more particularly to an image decoding apparatus which decodes an image signal which has been coded so as to reduce redundancy by utilizing temporal or spatial correlation of image information (image data). Related to the device.

[0002]

2. Description of the Related Art When digitally represented image data is transmitted or stored, the degree of redundancy is exploited by utilizing the temporal or spatial correlation of image information (image data) in order to reduce the amount of transmitted or stored data. Is known.

As a method utilizing the temporal correlation, there are methods of encoding a difference between two screens (frames) and detecting motion of an image to perform motion compensation. As a method using spatial correlation, an image is divided into blocks of a predetermined size (for example, 8 pixels each in the vertical and horizontal directions).
There is a method in which data in a block is orthogonally transformed, a transform coefficient is scan-transformed (for example, rearranged from a low-frequency component to a high-frequency component), and variable-length coding is performed. MPEG (Moving Picture Exper) for broadcasting and storage
ts Group) has established a standard for image coding MPEG.
The above two methods are used in combination. MPEG
Recommendation 2 is described in ISO / IEC 13818-2.

An image signal is encoded in the order of a predetermined frame with reference to the preceding and succeeding frames. The case of predicting the current frame from a past frame is called forward prediction, the case of predicting the current frame from a future frame is called backward prediction, and the case of predicting the current frame from both past and future frames is called bidirectional prediction. Image coding system MPE
In G, encoding is performed for each macroblock. A macroblock is a unit that divides a frame into 16 vertical pixels and 16 horizontal pixels. A frame in which all macroblocks are encoded without using inter-frame prediction is an intra frame (I frame), a frame permitted to include a macroblock encoded by forward prediction is a forward prediction frame (P frame), A frame permitted to contain a macroblock encoded by bidirectional prediction is called a bidirectional predicted frame (B frame). As shown in FIG. 11, an image decoding apparatus that decodes data encoded by the above method includes a video decoder 10 and a memory 120.
It consists of.

[0005] The video decoder 10 includes a buffer control unit 1
1, a variable length decoding unit 12, a scan converter 13, an inverse quantizer 14, an inverse DCT unit 15, a motion compensation image reproducing unit 16, a memory control unit 18, and a display control unit 18. Memory 120
Is a buffer memory 121, an I frame memory 122,
It has a P frame memory and a B frame memory 124.

In the above configuration, the input bit stream (input data) is stored in the buffer memory 121 under the control of the buffer memory control unit 11. The data read from the buffer memory 121 is stored in the variable-length decoder 1
2, the variable-length decoding is performed. Although not all data is variable-length coded, it is assumed that fixed-length codes are also decoded by the variable-length decoder 12. The decoded signal from the variable length decoder 12 is rearranged in order by the scan converter 13 and then inversely quantized by the inverse quantizer 14. Further, the inverse DCT unit 15 performs an inverse discrete cosine transform. When the inter-frame difference is received, the motion-compensated image reproducing unit 16 reads the reference data that has been decoded in advance from the predicted frame memory 122 or 123, adds the data to the inverse discrete cosine transformed data, and generates a predicted frame as a reproduced image. Write to the memory 122, 123 or 124. When the data encoded in the frame is received, the received data is directly used as the predicted frame 122 or 12.
Write to 3. The reproduced image is reproduced as described above. The display control unit 17 reads the display data from the memory 120 and outputs a decoded image. The memory control unit 18 controls data transfer between the memory 120 and the decoder 10, that is, writing and reading access to the memory 120.

In MPEG, the number of pixels in one frame varies from the current TV resolution of 720 horizontal pixels and 480 vertical lines to HDTV (High Definition TV) of 1920 horizontal pixels and 1080 vertical lines shown in FIG. Can handle large images. In HDTV, there are 120 horizontal macroblocks and 68 vertical macroblocks. Here, a row of macroblocks in one horizontal row is called a macroblock line (MB line). Since the HDTV has a large number of pixels, the amount of calculation for decoding also increases. In order to process this at high speed, a method of performing parallel operation using a plurality of partial decoders is known. For example, as shown in FIG. 13, the screen is vertically divided into two parts, and the first to 34th MB lines are decoded by the first partial decoder (decoder A), and the 35th line is decoded.
From MB line to 68th MB line (shaded area)
Is decoded by the second partial decoder (decoder B). Such an image decoding apparatus is disclosed in Japanese Unexamined Patent Application Publication No.
0457.

In an image decoding apparatus for image data compressed by the above-mentioned backward prediction and bidirectional prediction, the order of the input stream and the order of the display stream are not the same. For example, the order of the input stream is I frame I1, B frame B0, P
In the case of the frame P3, the B frame B2, and the P frame P5, the display stream order is the display frame order: B0,
It becomes like I1, B2, P3. In this case, FIG. 14 shows the timing when decoding is performed using two partial decoders. In FIG. 14, the horizontal axis represents time, the vertical axis represents the position of the scanning line, the thick solid line represents the state of decoding,
The broken line shows the interlaced display. After decoding, the B frame is stored in the B frame memory 124, read out again, and displayed. Therefore, the B2 frame must be decoded after the display of the I1 frame and before the display of the B2 frame (the parallelogram area between the I1 display and the B2 display in FIG. 14).

[0009]

In an image decoding apparatus using a plurality of partial decoders, the processing speed required for each individual decoder is considerably reduced, but the processing of each partial decoder depends on the nature of the image. The speed does not become uniform, and the decoding of some partial decoders may not be completed before displaying the B frame. For example, in FIG. 14, the B2 frame may not be decoded after the display of the I1 frame and before the display of the B2 frame (a parallelogram area between the I1 display and the B2 display in FIG. 14). In that case, the display deteriorates the image quality of the reproduced image by mixing the image of the past frame in the middle of the field.

It is a main object of the present invention to provide an image decoding apparatus using a partial decoder with an allowance for a processing time of each decoder and to prevent deterioration of the image quality of a reproduced image.

[0011]

In order to achieve the above object, in an image decoding apparatus for decoding information compressed image data using a plurality of partial decoders by forward prediction, backward prediction, and bidirectional prediction. , The processing speed of one frame of the plurality of partial decoders is controlled to be substantially equal. According to a preferred embodiment of the present invention, in the case of two decoders, the first partial decoder (decoder A) performs a memory access (one of reading a stream, reading predicted data, and writing reproduced data). When the second partial decoder (decoder B) requests a memory access (one of reading a stream, reading predicted data, and writing reproduced data), the first partial decoder (decoder B) Priority control for comparing the progress of decoding of the (decoder A) with the progress of decoding of the second partial decoder (decoder B), and giving priority to the access of the partial decoder to which the progress of decoding is being sent. Means are provided. To check the progress of decoding, M
In the case of PEG image data, a means is provided for comparing the address of a slice start code or a macroblock during or after decoding. In the image decoding apparatus of the present invention, priority control can be performed in a fixed processing unit, for example, in the case of an MPEG image, in units of slices or macroblocks during decoding processing of one field. Since the processing speed is made uniform, it is possible to complete decoding of an image necessary for display before display, regardless of the nature of the image information of the image frame.

[0012]

<First Embodiment> FIG. 1 is a block diagram showing a first embodiment of an image decoding apparatus according to the present invention. This image decoding apparatus processes MPEG2 input image data (input data) and obtains an image output from a display or the like. The image decoding device includes a video decoder 1
0 and a memory 120. The video decoder 10 includes a buffer control unit 11, a variable-length decoding unit 12A, a scan converter 13A, an inverse quantizer 14A, an inverse DCT unit 15A, and a motion-compensated image reproducing unit 16A. A), variable-length decoding unit 12B, scan converter 13
B, a second partial decoder (decoder B) including an inverse quantizer 14B, an inverse DCT unit 15B, and a motion compensation image reproducing unit 16B, and a display control unit that reads display data from the memory 120 and outputs a decoded image. 17, the memory 120, the buffer controller 11, the decoders A and B, and the display controller 17
And a memory control unit 18 for controlling the transfer of data to and from the memory. The memory control unit 18 has access priority control means 19 for checking the progress of the processing of the partial decoders A and B, and for giving priority to the memory access of the partial decoders whose decoding processing is delayed.

The memory 120 includes a buffer memory 121, an I frame memory 122, a P frame memory 123,
It has a frame memory 124. An I frame is not always written in an I frame memory, and a P frame is sometimes written. Conversely, a P frame is not always written in a P frame memory.
Although a frame may be written, 122 is called an I-frame memory and 123 is called a P-frame memory for the sake of simplicity.

Next, the operation will be described. The input data is stored in the buffer memory 121 under the control of the buffer memory control unit 11. An identifier (MPEG slice header) in the input data is detected so that the two partial decoders can operate in parallel, and the input data is divided and stored in the memory 121.

In the decoder A, the buffer memory 121
Is read by the variable-length decoder 12A to be variable-length decoded. Although not all data is variable-length coded, fixed-length codes are also decoded by the variable-length decoder 12A. The output of the variable length decoder 12A is rearranged in order by the scan converter 13A and then dequantized by the dequantizer 14A.
The inversely quantized signal is subjected to inverse discrete cosine transform by the inverse DCT unit 15A. In the motion-compensated image reproducing unit 16A,
When the difference between the frames is received, that is, for the P and B frames, the reference data that has been decoded in advance is read from the predicted frame memory 122 or 123, and is added to the received data.
Write to 3 or 124. When the data (I frame) encoded in the frame is received, the inverse DCT unit 15
The data of A is written to the predicted frame 122 or 123 as it is. The operation of the decoder B is the same as the operation of the decoder A. In the above configuration operation, except for the memory control unit 18, the configuration is substantially the same as that of a conventionally known image multifunction peripheral, and a detailed description of each unit will be omitted.

The memory control unit 18 controls various accesses to the memory 120 as described below.

(1) Writing of input data (2A) Reading of input data by decoder A (3A) Reading of predicted data by decoder A (4A) Writing of reproduced data by decoder A (2B) By decoder B Reading of input data (3B) Reading of predicted data by decoder B (4B) Writing of reproduced data by decoder B (5) Reading of display data Data transfer in these various accesses is performed by a common bus. Therefore, access must be controlled within a predetermined time so that data collision does not occur. In the present embodiment, in particular, when the accesses of the partial decoders A and B conflict with each other, the access priority control unit 19 performs the access control so as to give priority to the access of the decoder to which the decoding process is being sent.

FIG. 2 is a flowchart showing the priority order control for each access (1), (2A)... (5), which is the processing of the access priority control unit 19. In the figure,
(1), (2A)... (5) indicate the respective accesses. The number given to the line with the arrow drawn from each access indicates the priority. For example, (1) when an access (1) for writing an input stream occurs, as the next access, the access (2A) to (4A) and (2B) to (2B) to (2A) to (4A) of the decoder A There may be a request for access (2B) to (4B) of decoder B in 4B) and a request for access (5) for reading display data. In response to access requests from these, first, access (5) is prioritized, and then access (2A) to (4A) of decoder A or (2B) to (4B) of decoder B is prioritized. If these are not requested and there is a request for access (1) again, the request is responded to the request for access (1). The same applies to the priority order of the access to be performed after the display data read access (5).

The feature of this embodiment is that after the access (1) or (5), the access (2A) to the partial decoder A or B is performed.
When executing (4A), (2B) to (4B), a means for determining which of the decoders A or B is prioritized is provided.
A precedence determination for checking the progress of the processing of the B decoder is performed. In the preceding determination of the decoders A / B, when the progress of the decoding of the decoder A is ahead, the accesses (2B) to (4B) are prioritized, and the progress of the decoding of the decoder B is advanced. When there is access, priority is given to access (2A) to (4A). These accesses are performed until access (1) or (5) is requested. The transition to access (1) or (5) is determined on a first-come, first-served basis.

FIG. 3 shows the structure of the judging means provided in the access priority control section 19. Macro block counter 2
0A and 20B are tunable decoders 12A and 1B, respectively.
2B is reset at the start of each frame, that is, when a picture start code in the input stream is detected, and is incremented by 1 at the start of each macroblock, that is, at the time of decoding the macroblock address increment in the input stream. The count values of the counters 20A and 20B are added to the comparator 21 and compared. The decoder with the larger count value indicates that decoding is ahead. The comparator 21 includes counters 20A and 20B
Are represented by A and B, respectively. If A ≧ B, the selection signal S = H for selecting the addresses of the data 2B, 3B, 4B necessary for the B decoder is applied to the address selector 22. If <B, data 2A required for A decoder,
A selection signal S = L for selecting the addresses of 3A and 4A is applied to the address selector 22.

FIG. 4 is a diagram showing timings of decoding and display when an image frame is bisected and two partial decoders share the upper and lower divided decoding. In FIG. 4, the horizontal axis represents time, the vertical axis represents the position of the scanning line,
A thick solid line indicates a state of decoding, and a broken line indicates a state of interlace display (the same applies to FIGS. 8 and 14 described below). In the figure, the solid line indicating the decoding is indicated by a straight line, but strictly speaking, the solid line is in a stepwise form using the processing time of a macroblock as a basic unit. Since the preceding state of the decoder is determined by the processing time of the macroblock, it is represented by a substantially linear state. For the above reasons, the decoding of the upper half screen and the decoding of the lower half screen proceed at substantially the same speed, so that the operating margin for decoding is greatly improved.

<Embodiment 2> FIG. 5 is a flowchart showing the control of the access priority control unit in the second embodiment of the image decoding apparatus according to the present invention. Note that the configuration of the entire apparatus is substantially the same as that shown in FIG.
After the access (2A) to (4A) or (2B) to (4B), if there is no access request of (1) or (5), A
It differs from the access priority control flow of FIG. 3 in that it returns to the preceding determination of the / B decoder.

<Embodiment 3> FIG. 6 is a flowchart showing the control of the access priority control unit in the third embodiment of the image decoding apparatus according to the present invention. Note that the configuration of the entire apparatus is substantially the same as that shown in FIG.
In the present embodiment, the access (2A) to (4A) or (2A)
After B) to (4B), when there is no request for access (1) or (5), the process transits to the continuous access determination. The first operation example of the continuous access determination includes access (2A) to (4)
If the identical access of (A) or (2B) to (4B) does not continue, the access (2A) to (4A) or (2A)
B) to (4B) are permitted. For example, access (2A)
(3A), (4A), (2B) ~
(4B) In the request, access (3A) or (4A)
Priority. Furthermore, access (3A) or (4A)
If the request for the access (2A) is made again after the completion of the above, the request is met. As a result, the delay of the A / B decoder is decoded continuously, and the delay is recovered.

The second operation example of the continuous access determination is to lower the priority order of identical accesses (2A) to (4A) or (2B) to (4B). For example,
After completion of access (2A), access (3A), (4
A), (3A) or (4A) is prioritized among the requests (2B) to (4B). Furthermore, after the access (3A) or (4A) is completed, even if the access request (2A) is made again, the access (2B) to (4B) is given priority.

<Embodiment 4> The configuration of an image decoding apparatus according to a fourth embodiment of the present invention is the same as that of FIG. In the present embodiment, the two partial decoders A and B respectively include an odd macroblock (MB) line and an even macroblock (M) of an image frame as shown in FIG.
B) Sharing decoding of the line.

As shown in FIG. 8, the decoding and display timings are the same as those described above for the decoder A and the decoder B.
Can be decoded at almost the same speed, and the margin of the decoding operation is improved. The decoding speed of the decoder A and the decoder B can be determined based on the number of macroblocks (MB) processed in the frame being decoded. That is, the number of MB lines processed in the vertical direction can be determined from the slice start code, and the number of MB lines processed in the vertical direction can be compared.
If the number of lines is the same, the number of MBs processed in the horizontal direction can be found from the MB address, and this is compared.

<Embodiment 5> FIG. 9 is a block diagram showing the configuration of a fifth embodiment of the image decoding apparatus according to the present invention. In the figure, substantially the same components as those in FIG.
The same reference numerals are used to omit detailed description. The difference from FIG. 1 is that only the variable length decoders 12A and 12B have a parallel configuration,
Scan converter 83, inverse quantizer 14, inverse DCT unit 1
5. The point that the motion compensated image reproducing unit 16 is shared. The variable-length decoders 12A and 12B decode the stream of the divided screen as shown in FIG. 7 or 13 in parallel, receive the result, and receive the scan converter 83 and the inverse quantizer 1
4. The inverse DCT unit 15 and the motion-compensated image reproducing unit 16 operate in a time-division manner. The scan converter 83 includes the variable length decoder 1
2A, so that both outputs of the variable length decoder 12 can be received. The preceding determination of the A / B decoder is based on the access (2A)
And (2B). The present embodiment is effective when high speed is required for the variable length decoding process.

<Sixth Embodiment> FIG. 10 is a block diagram showing a configuration of a sixth embodiment of the image decoding apparatus according to the present invention. In the figure, components substantially the same as those in FIG. 1 are denoted by the same reference numerals as those in FIG. 1, and detailed description thereof will be omitted. The difference from FIG. 1 is that the scan converters 13A and 13B, the inverse quantizer 1
Although 4A and 14B, inverse DCT sections 15A and 15B, and motion compensation image reproducing sections 16A and 16B have a parallel configuration,
The variable length decoder 92 has a single configuration. The result decoded by the variable length decoder 92 is in a process waiting state of the decoder A (13A, 14A, 15A, 16A) or the decoder B (13B, 14B, 15B, 16B) for each macroblock. Forwarded to The preceding determination of the A / B decoder includes access (3A) to (4A) and (3B)
To (4B). This embodiment is effective when high-speed processing is required for processing from scan conversion to motion-compensated image reproduction for variable-length decoding.

[0029]

As described above, the conventional image parallel decoder has a drawback that the timing margin of the decoding operation is insufficient because the decoding speed of each decoder is not controlled. According to the present invention, by giving priority to the memory access of each decoder, the timing margin of the decoding operation can be greatly improved.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a first embodiment of an image decoding apparatus according to the present invention;

FIG. 2 is a processing flowchart of an access priority control unit in the first embodiment of the image decoding apparatus according to the present invention;

FIG. 3 is a configuration block diagram of a main part of an access priority control unit of the image decoding device according to the present invention.

FIG. 4 is a view for explaining decoding and display timings in the first embodiment of the image decoding apparatus according to the present invention;

FIG. 5 is a processing flowchart of an access priority control unit in a second embodiment of the image decoding apparatus according to the present invention;

FIG. 6 is a processing flowchart of an access priority control unit in a third embodiment of the image decoding device according to the present invention;

FIG. 7 is a view for explaining screen division in a fourth embodiment of the image decoding apparatus according to the present invention.

FIG. 8 is a view for explaining decoding and display timing in an image decoding apparatus according to a fourth embodiment of the present invention.

FIG. 9 is a block diagram showing a fifth embodiment of the image decoding apparatus according to the present invention;

FIG. 10 is a block diagram showing a sixth embodiment of the image decoding apparatus according to the present invention;

FIG. 11 is a block diagram showing a conventional configuration.

FIG. 12 is a diagram illustrating a macroblock structure; FIG. 12 is a diagram illustrating screen division according to a conventional example and the first embodiment of the present invention;

FIG. 13 is a view for explaining screen division according to the conventional example and the first embodiment of the present invention.

FIG. 14 is a diagram for explaining decoding and display timing of a conventional image decoding device.

[Explanation of symbols]

10: image (video) decoder, 11: buffer controller, 12, 12A, 12B, 92: variable length decoder, 13, 13A, 13B, 83: scan converter, 14, 14A, 14B: inverse quantizer , 15, 15A, 1
5B: inverse DCT unit, 16, 16A, 16B: motion compensation, image reproducing unit, 17:
Display control unit, 18: memory control unit, 19: access priority control unit, 20A, 20B: macroblock counter, 21: comparator, 22: selector 120: memory, 121: buffer memory, 122, 123, 124: frame memory.

Claims (6)

[Claims] An image decoding apparatus for decoding image data information-compressed by forward prediction, backward prediction, and bidirectional prediction using a plurality of partial decoders, wherein the processing progress of the plurality of partial decoders is determined. An image decoding apparatus, comprising: access priority control means for giving priority to accessing a memory for a partial decoder whose checking and decoding processing is delayed. 2. The image decoding apparatus according to claim 1, wherein each of said plurality of partial decoders has a variable length decoder. 3. An output of the variable-length decoder is configured to be applied to a motion-compensated image reproducing unit via a sequential scan converter, an inverse quantizer, and an inverse DCT unit. The image decoding device according to claim 2. 4. The plurality of partial decoders divides a common variable length decoder and an output of the variable length decoder into a plurality of outputs, and outputs the separated outputs sequentially as a scan converter, an inverse quantizer, 2. The image decoding apparatus according to claim 1, wherein the image decoding apparatus is configured to be added to a motion compensation image reproduction unit via an inverse DCT unit. 5. The image decoding apparatus according to claim 1, wherein said plurality of partial decoders are two partial decoders, and said access priority control means is configured such that one of said two partial decoders reads and predicts a stream. A comparing unit that compares the decoding speed of the first partial decoder and the decoding speed of the second partial decoder when a memory access request for reading data or writing reproduction data is made; 2. The image decoding apparatus according to claim 1, further comprising a processing unit configured to perform a switching process for giving priority to an access of a decoder whose decoding is delayed based on the processing. 6. The image decoding apparatus according to claim 5, wherein said image data is MPEG image data, and said comparing section performs a decoding operation of a partial decoder based on a slice start code of decoding or decoding completion or an address of a macroblock. 2. The image decoding device according to claim 1, wherein the image decoding device is configured to compare a degree of progress of the processing.