JP2000165857A - Hierarchical moving picture evaluation device and moving picture communication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a hierarchical moving picture evaluation device where number of times of coding, number of times of communication and a scale of decoding processing in the case of evaluating a hierarchical moving picture code can be reduced and that evaluates image quality when a hierarchical moving picture repeater transmits a hierarchical stream with a reduced hierarchical number. SOLUTION: When a moving picture code transmission section 40 transmits a hierarchical moving picture code hierarchically processed depending on resolution to a moving picture code reception section 60 via a transmission line 20, the hierarchical moving picture code is received by a hierarchical moving picture evaluation device 110. A hierarchical separate section 111 reconfigures a received hierarchical moving picture code for each layer and gives each moving picture stream of layers A-C to decoding sections 112a-112c. The decoding sections decode the received moving picture stream to obtain each picture of the layers A-C and give the result to evaluation sections 113a-113c. The evaluation sections calculate the image quality evaluation value of the received picture and feed back the result to the moving picture code transmission section 40. The moving picture code transmission section 40 properly selects a coding mode and a coding parameter of the hierarchical moving picture code to be sent on the basis of the received image quality evaluation value.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture evaluation apparatus for evaluating the degree of deterioration of the quality of an encoded digital moving picture from an original picture, and a moving picture communication apparatus to which the moving picture evaluation apparatus is applied. In particular, the present invention relates to a hierarchical moving image evaluation device and a moving image communication device that perform image quality evaluation on moving image codes hierarchized according to resolution.

[0002]

2. Description of the Related Art Digital moving pictures are composed of a series of pictures at a moment called a "frame". Each frame is composed of "pixels" or "dots" obtained by decomposing a display screen into a number of grid-like points, and each pixel has a digital value representing its color and brightness (also called gradation). Data is allocated. In general, when the amount of information per unit time of a digital moving image is expressed in [bit / sec] (bits per second), even a typical value reaches several hundred megabits [bit / sec]. It is huge. For this reason, when communicating or recording a digital moving image, it is usual that information compression called “moving image encoding” is performed. It should be noted that data generated by such moving image decoding is called a “moving image code”. Since each part of the moving picture code is expected to be reproduced and displayed according to the time originally set as a video, these parts must be arranged in a line in the order of screen scanning and frames (that is, the time lapse). become. In the present specification, such a bit string is referred to as a “moving image stream”.

[0003] International standards for video codes include ISO (International Organization for Statis- tics) which is an international standardization organization.
andardization) and IEC (International Electro
Technical Commission) MPEG (Moving Pictu)
re Experts Group) -1 (ISO / IEC-11172), MPEG-
2 (ISO / IEC-13818), MPEG-4 (ISO / IEC-14496)
And ITU (International Telecommunication Unio
n) Recommendation H.264. 261, H .; 263 and the like. In such a moving image encoding method, the size of the moving image code obtained from the dot data of the same moving image can be varied. For example, in each of the above-described moving image coding methods, encoding is performed using a frequency component value of a spatial distribution of gradation values called “DCT (Discrete Cosine Transform) coefficient”. The width or roughness of the coefficient quantization (digital coding)
The coding is performed by selecting a "quantization width" from tens to hundreds of stages. Here, the code amount per unit time is called an “encoding rate”. The coding rate decreases as the quantization width decreases, and the coding rate increases as the quantization width decreases. Therefore, the compression rate of a moving image can be changed by changing the quantization width.

As described above, since the DCT coefficient is a spatial frequency component value, the difference between the decoded image and the original image increases as the quantization width increases, while as the quantization width increases, the difference increases. The decoded image becomes closer to the original image and becomes more faithful. In addition, the compression ratio mentioned above is DCT
Prediction method of DC (direct current) component which is the lowest order component of coefficient,
Inter-frame prediction methods such as bidirectional prediction and forward prediction,
It also depends on which of several standardized schemes to choose, such as a motion vector prediction scheme. As described above, in the moving picture coding method, a desired moving picture code can be obtained by appropriately selecting a numerical value such as a quantization width and a prediction method. In the present specification, the former as an option at the time of encoding is referred to as “encoding parameter”, and the latter is referred to as “encoding mode”.

[0005] In addition, when performing moving image communication, delays and fluctuations of moving images occur and moving images are reproduced and displayed mainly due to an insufficient bandwidth of a transmission path compared with an encoding rate. Time is incorrect, video buffering increases, some video code is discarded,
This has various adverse effects on playback display, such as skipped frames being played back and a playback display image being disturbed. In addition, the coding rate changes depending on the selection of the coding parameter and the coding mode, and accordingly, the degree of influence on the reproduced image also changes. In view of the above, in moving image communication, it is important to evaluate how much the image quality of the reproduced display image has deteriorated from the original image, and therefore, a device called a “moving image evaluation device” is required. come.

[0006] As this kind of moving picture evaluation device, for example,
A "moving image communication management device" disclosed in Japanese Patent Application No. 9-216814 filed by the present applicant has been mentioned.
Therefore, the conventional moving image evaluation device disclosed in the prior application will be briefly described below. FIG. 4 shows a main part of the configuration of a video communication device to which the video evaluation device is applied. In the figure, a moving image code transmission unit 10
Is a moving image server computer, a moving image encoder, and the like. The moving image code receiving unit 30 connected to the moving image code transmitting unit 10 via the transmission path 20 is a personal computer, a set-top box, an image display device, or the like.
Note that a set-top box is a device for using a two-way service such as a video-on-demand service using a home television, and a function for transmitting a request from a user to a server computer and digital information. It has functions to convert video and audio.

When the moving picture code transmitting section 10 transmits a moving picture code (that is, a moving picture stream) to the moving picture code receiving section 30 through the transmission line 20, the sent moving picture code is input to the moving picture evaluation section 100. Here, the method for inputting the moving image code to the moving image evaluation unit 100 includes the moving image code receiving unit 3.
0 receives the moving image code from the moving image code transmitting unit 10 and then outputs the moving image code to the moving image evaluation unit 100. Alternatively, the moving image code is extracted from the transmission path 20 using a distributor or a collector (not shown). There is a method of outputting to the evaluation unit 100 and the like. Next, the moving image code input to the moving image evaluation unit 100 is supplied to the decoding unit 101, and the decoding unit 101 decodes the moving image code into a moving image and outputs the moving image to the evaluation unit 102.
02 evaluates the image quality of the decoded moving image and feeds back the image quality evaluation value to the moving image code transmission unit 10.

[0008] The "image quality evaluation value" referred to here is, for example, the original image acquired by the evaluation unit 102 in advance and the decoding unit 101
The difference between the images (hereinafter, referred to as “decoded images”) decoded in step (1) is quantified. For example, the image quality evaluation value is dB
The SNR (Signal to Noise Ratio) value in (decibel) display, and the calculation procedure is as follows. That is, first, the difference between the gradation values of the pixels corresponding to each other on the original image and the decoded image is obtained for each pixel, and the square value is obtained for each of them. Next, the average value of all pixels for the square values of these differences is regarded as noise power. Next, the squared value of the total amplitude of the gradation is obtained, the ratio between the squared value and the noise power is obtained, and the value obtained by multiplying the common logarithm of the calculated ratio by 10 is determined as the SNR value in dB display. I do. Here, the “total amplitude of the gray scale” is as follows. That is, for example, when the gradation of each pixel is represented by 8 bits, for example, the case where the gradation value is the minimum value “0” is “black”,
When the maximum value is "255" (decimal number), it becomes "white",
The gradation value of the maximum value “255” corresponds to the full amplitude of the gradation.
Incidentally, the calculation method of the image quality evaluation value is not limited to the method just described, and another calculation method can be used.

[0009] Then, the moving picture code transmitting section 10 transmits the moving picture stream coded for a plurality of combinations of the coding mode and the coding parameter to the transmission line 20, and uses the moving picture evaluation section 100 to calculate the SNR value for each moving picture stream. The image quality evaluation values are obtained, and the image quality evaluation values are compared with each other to determine the combination of the encoding mode and the encoding parameter that gives the highest image quality. As described above, by feeding back the image quality evaluation result of the video stream from the video evaluation unit 100 to the video code transmission unit 10, it is possible to support the operation of the video code encoding performed by the video code transmission unit 10. Become.

Incidentally, there is a method called "hierarchical moving image code" as one of the methods of forming the moving image code. Examples of literature on this method include “Hiroyuki Kasai, Akira Kodama, Kyoji Maeda, Hideyoshi Tominaga, Waseda University, Department of Electronic Communication,
"A Study on Visual Information Model Description Method Using Hierarchical Video Coding", Proc. Of the 1996 IEICE General Conference, Vol. 7, D-301, p89, IEICE. Therefore, the layered moving image code will be briefly described below with reference to FIGS.

[0011] As is well known, the situation when transmitting a moving image is quite diverse. For example, the transmission path is a telephone line or a local area network (hereinafter referred to as "LA").
N "), using wireless communication, and connecting these various kinds of communication means one after another to transmit and receive at both ends. Further, the characteristics of the transmission path for transmitting the moving image stream may fluctuate over time. For example, in wireless communication, the bandwidth that can be transmitted changes due to a factor called fading multipath, and in LAN, the throughput allocated to one communication channel may change due to traffic fluctuation by a plurality of users. Also, when a moving image is transmitted to a plurality of receiving terminals, the characteristics of a transmission path to each of the receiving terminals may be various. For example, wireless and LA as described above
N, heterogeneous connection networks, and even in the same telephone line, one terminal is connected at a maximum bandwidth of 64 kbps (bits per second), and another terminal is connected at a maximum bandwidth of 28.8 kbps. It often happens. In addition, the processing performance of each transmitting / receiving terminal is also diverse, and various performances are required depending on the sharing and configuration of a general-purpose CPU (central processing unit) and a dedicated integrated circuit, the processing speed of the CPU, the cache memory size, the video decoding performance, and the like. Become.

[0012] Because of the variety of situations in the above-described moving picture transmission, what kind of moving picture stream should be transmitted in order to reproduce the highest quality moving picture achievable by each receiving terminal becomes a problem. There is also a specification for determining an optimum coding rate, coding parameter, coding mode, and the like according to the specification. For example, the maximum bandwidth allocated to the transmission path determines the video encoding rate,
The specifications of the optimal parameters such as the screen resolution, the number of frames (frames) per second, and the roughness of quantization are determined according to the maximum bandwidth.

However, in order to distribute the same content to various receiving terminals at once, it is necessary to simultaneously transmit a moving image stream to these receiving terminals in accordance with an encoding parameter optimal for each receiving terminal. Therefore, for example, to simultaneously transmit a video code having a coding rate of 64 kbps per terminal to eight terminals, 64 × 8 = 512.
A transmission capacity of kilo bps is required. That is,
In such a method, as the number of receiving terminals increases, the bandwidth required for the transmission path immediately after the moving image transmitting terminal becomes enormous. For this reason, by transmitting the “layered video stream” according to a code transmission method called a hierarchical video code, the above-described problem is avoided, and the video transmission terminal can transmit the common video stream without duplication. Thus, it becomes possible to support various transmission and receiving terminals.

[0014] The layered stream (hereinafter referred to as "hierarchical stream") as used herein includes a single moving picture stream including several streams having different image qualities. Can be included or extracted from the hierarchical stream. For example,
5 (a) to 5 (c) are images showing the same object having a triangular shape, and are distinguished according to the level of the resolution. That is, FIG. 10A is a high-resolution hierarchical A image, FIG. 10B is a hierarchical B image having a medium resolution, and FIG. 10C is a low-resolution hierarchical C image. FIGS. 6A to 6C show configuration examples of the layered streams, each of which is a layered stream corresponding to the layers A to C described above. As shown in the figure, in the hierarchical stream, after moving picture codes necessary for each layer are once separated, they are connected in a line in frame units, for example.

Of these, the frame image in the low-resolution layer C is immediately obtained by decoding the “layer C code 41” of the layer C stream shown in FIG.
On the other hand, the middle-resolution layer B stream is formed by connecting the “layer B additional code 42” and the “layer C code 41”.
The layer B additional code 42 is obtained by encoding a difference image between the layer B image and the layer C image. By using such a difference image, there is an advantage that the difference between the two images becomes zero in many parts of the image and the code amount is small. Although the calculation method of the difference image is well known, in a nutshell, the gradation value of two pixels corresponding to (ie, located at the same position on) both images of interest. Is obtained for all pixels to obtain a difference image.

The frame image of the layer B is obtained by decoding the layer C code 41 and the layer B additional code 42. That is, for the image obtained by decoding the layer C, the layer B
A layer B image is obtained by additionally adding and combining the difference image obtained by decoding the additional code 42. On the other hand, the high-resolution layer A stream is formed by concatenation of “layer A additional code 43”, “layer B additional code 42”, and “layer C code 41”. The layer A additional code 43 encodes the difference image between the layer A image and the layer B image,
As with the additional code 42, there is an advantage that the code amount is small. The frame image of the layer A is obtained by decoding the layer C code 41, the layer B additional code 42, and the layer A additional code 43. That is, a difference image obtained by decoding the layer A additional code 43 is additionally added to the frame image of the layer B obtained from the layer C code 41 and the layer B additional code 42 to synthesize the layer A Is obtained. Incidentally, in the present specification, the image having the lower resolution (for example, the hierarchical C image at the time of creating the hierarchical B additional code 42) among the two images that are the basis of the above-described difference images is referred to as a “standard I will call it an "image."

Here, as a method of forming a hierarchy, in addition to the method of separating at the spatial resolution described above, as shown in FIG. 7, images are divided by a frame rate (that is, the number of frames per unit time), There is also a method of forming a hierarchy by resolution. That is, the high-resolution hierarchical A images (# 1, # 2, # 3,...) Shown in FIG. Tier B
This is a method of forming images (# 1, # 3, # 5,...).
From a visual point of view, the layer A image looks smooth in the movement of the reflected object, whereas the layer B image looks like the movement of the object is awkward, just as in frame advance.

In addition, a method of setting even-numbered frame codes based on the hierarchical B image shown in FIG. 7B as a hierarchical A additional code, or a method of coding even-numbered frames and odd-numbered frames immediately before them. There is also a method of setting the code of the difference image as a layer A additional code. The latter of these methods is called “inter-frame predictive coding” in the international standard for the moving picture coding method described above. Then, a frame coded by this method is referred to as a “P frame”, a frame coded without referring to other frames is referred to as an “I frame”, and a prediction residual is obtained not only from the immediately preceding frame but also from the preceding and succeeding frames. The resulting frame is called a “B frame”, and a hierarchy can be formed by using these P, I, and B frames. For example, if the frame order is "IBBPB
, BPBB, IBBPBBPBB,..., ", The layer C is composed only of I frames, and the layer B is composed of I frames and P frames.
All frames P and B constitute the hierarchy A.

In addition to the above, the following hierarchical configuration method can be considered. That is, the DCT coefficients used in the above-mentioned international standard are 64 from the lowest order to the highest order.
Since it is composed of a number of coefficients, for example, only the coefficients from the lowest order to a certain intermediate order are used, and the coefficients of the higher orders are all set to zero to decode the image by inverse DCT transform. Then, an image having a relatively low resolution can be obtained. Utilizing this, the layer C is composed of images obtained from only the coefficients of the lowest order, and the layer B is composed of coefficients from the lowest order to a certain intermediate order.
It is possible to configure the hierarchy A with all coefficients up to the highest order.

In order to realize the hierarchical stream as described in detail above, a relay function for relaying the hierarchical stream may be mounted on an exchange or a line concentrator installed on the network. When the traffic on the network increases, this relay function extracts a hierarchical stream having a transferable coding rate corresponding to the degree of congestion from the transmitted hierarchical streams and reduces the number of hierarchical streams. Is reconfigured before forwarding to the destination. In addition, when there are a plurality of routes to be transmitted, the number of hierarchical streams in the hierarchical stream is reduced according to each route and the terminal connected to each route, and then the stream is transferred to each route. In the present specification, an exchange or a line concentrator having such a function is referred to as a “hierarchical video relay device”.
I will call it. As described above, various hierarchical video coding schemes are used corresponding to various reception paths and reception terminals.

Next, a procedure for evaluating the image quality of a hierarchical stream using the conventional moving picture evaluation apparatus shown in FIG. 4 will be described with reference to a flowchart shown in FIG. Here, as shown in FIG. 8, layer A, layer B, layer C
The following describes an example in which the image quality of a moving image stream having three layers is evaluated in order from the layer having the highest resolution to the layer having the lowest resolution. First, the moving image code transmitting unit 10 transmits the hierarchical A moving image code to the moving image code receiving unit 30 (Step S1). This layer A moving image code is a moving image evaluation unit 100
Is supplied to the decoding unit 101, and the decoding unit 101 decodes the moving image of the layer A, and the evaluation unit 102 calculates the image quality evaluation value of the decoded moving image (step S2). Similarly, the moving image code transmitting unit 10 sends the moving image code
When the moving image code is transmitted (step S3), the decoding unit 101
Decodes the moving image of the layer B from the layer B moving image code, and the evaluation unit 102 calculates the image quality evaluation value of the moving image (step S4). Further, when the moving image code transmitting unit 10 transmits the hierarchical C moving image code to the moving image code receiving unit 30 (Step S5),
The decoding unit 101 decodes the moving image of the layer C from now on, and the evaluation unit 102 calculates the image quality evaluation value of this moving image (step S
6). Thereafter, the moving image evaluation unit 100 comprehensively evaluates the image quality evaluation values in each of the layers A to C and feeds back the result to the moving image code transmission unit 10 (step S).
7).

[0022]

As described above, in the conventional moving image evaluation apparatus, in order to evaluate moving image codes of all layers, a moving image stream is transmitted for each layer, and after decoding this, the image quality is reduced. It is necessary to repeat the process of calculating the evaluation value. In other words, as the number of hierarchies to be evaluated increases, a series of processing procedures such as encoding processing of a moving image code of each layer, communication processing using the transmission path 20, and decoding processing of a transmitted moving image code are performed many times. Must be implemented. For this reason, a load is imposed on the network, and in addition, the moving image code transmitting unit 1
0, there is a burden that the encoding process and the decoding process must be performed many times in the moving image evaluation unit 100, respectively.

Further, when the above-described MPEG-4 is applied to a moving picture evaluation apparatus, there is a problem that the processing is further complicated. That is, MPEG-4 is a system in which each screen is divided into objects, backgrounds, and the like existing therein, and encoding and decoding processes are performed on each of them. Note that these objects, backgrounds, and the like are each called a “video object”. For example, it is assumed that a moving image is a video in which three video objects of a person, a square object, and a background are combined and reflected, and a person and a square object are arranged in front of the background. Here, a person is, for example, a composite digital image obtained by removing a background portion from a digital moving image captured with a plain background, a square object is a composite digital image that generates a non-existent box image using computer graphics, and a background is, for example, a composite digital image. For example, a distant view image of a landscape that has been captured in another place using a video camera can be considered. Now, assuming that a rectangular object is a video object # 0, a person is a video object # 1, and a background is a video object # 2, these video objects are respectively set with different coding parameters and coding modes, and are individually hierarchically coded. The data is transmitted from the transmitting side to the receiving side. Then, the receiving side performs a decoding process on the received moving image code for each video object, thereby obtaining a composite image in which a person and a rectangular object are arranged in front of the background.

However, when various moving image codes subjected to encoding and hierarchical coding are to be evaluated for each video object using the conventional moving image evaluation apparatus, the number of layers for each video object is equal to the number of layers of each video object. It becomes necessary to transmit the moving image stream and calculate the image quality evaluation value. For this reason, the same problem as described above occurs, and the load on the network and the load required for the encoding / decoding processing are further increased. In addition, when the above-described hierarchical moving image relay device is used, when the hierarchical moving image relay device transmits the hierarchical stream to a specific transmission path or a receiving terminal by reducing the number of layers of the hierarchical stream, the transmission path or the receiving terminal It may be useful to know how the image quality of the sent moving image changes or is maintained.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a hierarchical moving image evaluation apparatus capable of reducing a load required for evaluating a moving image composed of a plurality of layers and a plurality of video objects. To provide. Another object of the present invention is to provide a method for changing or maintaining the image quality of a transmitted moving image when a hierarchical stream whose number of layers has been reduced by the hierarchical moving image relay apparatus is transmitted to a transmission path or a receiving terminal. An object of the present invention is to realize a hierarchical moving image evaluation device for evaluating whether or not a moving image has been evaluated.

[0026]

In order to solve the above-mentioned problems, the invention according to claim 1 is a layer for evaluating a layer stream obtained by encoding a plurality of layers of moving images divided according to the resolution of an image. In the moving image evaluation device, a decomposing unit that decomposes the hierarchical stream into moving image codes for each layer, a decoding unit that decodes the moving image code for each layer to obtain an image for each layer, and an image for each layer are also provided. And an evaluation means for calculating an image quality evaluation value of each image. The invention according to claim 2 is a hierarchical moving image evaluation device that evaluates a hierarchical stream obtained by encoding moving images of a plurality of layers divided according to the resolution of an image. The moving image code of the lowest resolution layer is a moving image code obtained by encoding the image of the lowest layer. The moving image code of each layer other than the lowest resolution layer includes the image of each layer and the lower layer layer. An additional code obtained by encoding a difference image between the reference image and the reference image, and a decoding unit for decomposing the hierarchical stream into a moving image code for each of the layers, and decoding the moving image code for each of the layers. Decoding means for calculating an image for each layer, and evaluation means for calculating an image quality evaluation value of each image based on the image for each layer.

According to a third aspect of the present invention, in the second aspect, the decoding means decodes the moving picture code of the lowest resolution hierarchy to obtain an image of the hierarchy, An additional code decoding unit that decodes the additional code to obtain a difference image corresponding to each layer other than the lowest resolution layer, and a difference image decoded by the additional code decoding unit and a source of the difference image. Combining means for obtaining an image of each layer other than the lowest resolution layer based on the reference image. Claim 4
In the invention described in any one of claims 1 to 3, a set of the decomposing means, the decoding means, and the evaluation means is provided for each video object in a moving image included in the hierarchical stream. The hierarchical stream of each video object is provided to the decomposing means provided for each video object, and evaluation is performed for each video object.

Further, the invention described in claim 5 is the same as that in claim 1
4. The hierarchical moving image evaluation device according to any one of items 4, a moving image code transmitting unit that transmits the hierarchical stream, and a moving image code receiving unit that receives the transmitted hierarchical stream via a transmission path, The moving image code transmitting means encodes and transmits the hierarchical stream based on an image quality evaluation value calculated by the hierarchical moving image evaluation device. According to a sixth aspect of the present invention, in the invention of the fifth aspect, at least one of the moving picture code transmitting means and the moving picture code receiving means reconfigures the hierarchical stream into a hierarchical stream that can be transferred to a predetermined destination. And has a hierarchical video relay function of transferring to the destination, and outputs the hierarchical streams before and after the relay to the hierarchical video evaluation device, respectively, and the evaluation means includes the video code transmitting means having the hierarchical video relay function. Alternatively, an image quality evaluation value of a difference image of an image obtained from each of the input and output hierarchical streams before and after the relay is calculated according to an instruction of the moving image code receiving unit.

[0029]

Embodiments of the present invention will be described below with reference to the drawings. [First Embodiment] FIG. 1 is a block diagram showing the configuration of a video communication device to which a hierarchical video evaluation device according to the present embodiment is applied. In the drawing, the same components as those shown in FIG. 4 are the same. Are attached. FIG. 2 is a flowchart showing an outline of the evaluation procedure according to the present embodiment. In FIG. 1, a moving image code transmitting unit 40 transmits a moving image code to a moving image code receiving unit 60 via a transmission path 20 in the form of a hierarchical stream. The moving image code receiving unit 60 receives the moving image code transmitted from the moving image code transmitting unit 40. Note that, in the present embodiment, the hierarchical stream to be transmitted is shown in FIG.
Is assumed, but a hierarchical stream having another configuration such as a hierarchical B stream or a hierarchical C stream may be used. In the present embodiment, one or both of the moving image code transmitting unit 40 and the moving image code receiving unit 60 may have the function of the hierarchical moving image relay device described above.
In addition, the moving image code transmitting unit 40 or the moving image code receiving unit 60
A stream obtained by performing various conversions on the input hierarchical stream may be output, or such a conversion may not be performed.

Next, the hierarchical moving image evaluation device 110 receives the hierarchical stream as input, and evaluates, for each layer, how much the image quality of the reproduced display image of the hierarchical stream has deteriorated from the original image. As means for inputting the hierarchical stream to the hierarchical moving image evaluation device 110, the same means as described in the related art can be considered. That is, a method in which the video code receiving unit 60 transmits the hierarchical stream received from the transmission path 20 to the hierarchical video evaluation device 110,
A method of installing a distributor or a collector (not shown) on the transmission path 20 and taking it into the hierarchical moving image evaluation device 110;
0 is directly transmitted to the hierarchical moving image evaluation device 110. Hierarchical moving image evaluation device 1 through moving image code receiver 60
If the hierarchical stream is transmitted to 10, the image quality evaluation is performed on the path including all of the moving picture code transmitting unit 40, the transmission path 20, and the moving picture code receiving unit 60. On the other hand, if a distributor or a collector is installed and a hierarchical stream is taken in, image quality evaluation including a transmission path to the installation point can be performed. On the other hand, the moving image code transmitting section 40
If the layer stream is transmitted directly to the layer moving image evaluation device 110 from the
The image quality is evaluated for the route excluding. That is, the portion beyond the transmission path 20 can be excluded from the evaluation.

Next, in the hierarchical moving image evaluation device 110,
The layer decomposition unit 111 reconstructs the given layer stream for each layer, and then decodes the decoding unit 112 provided for each layer.
The stream for each layer is transmitted to a to 112c. Each of the decoding units 112a, 112b, and 112c receives the layer A stream, the layer B stream, and the layer C stream (see FIG. 6) as input, decodes these layer streams, and evaluates the image of each layer for each layer. Parts 113a to 113
c. Note that the decoding units 112a to 112
Since the procedure for decoding an image based on the hierarchical stream to which c has been input has already been described in the description of the related art shown in FIG. 6, it will not be repeated here. Then, the evaluation units 113a to 113c calculate the image quality evaluation value of the image input from the corresponding decoding unit, and feed back to the moving image code transmission unit 40. These evaluation units 1
In calculating the image quality evaluation value, 13a to 113c acquire an original image for each layer in advance, and calculate the above-described SNR value from a difference between the original image and a decoded image for each layer input. However, the image quality evaluation value is SNR
Of course, it may be something other than the value.

Next, the hierarchical moving image evaluation apparatus 1 having the above configuration
The operation of No. 10 will be described. When the layer A stream [FIG. 6 (a)] is input to the layer moving image evaluation device 110 using the three methods exemplified above, the layer decomposition section 111
Reconstructs a given layer A stream into layers A to C, outputs a layer A stream to the decoding unit 112a, and outputs a layer B stream (FIG. 6B) to the decoding unit 112b. , And outputs the layer C stream [FIG. 6 (c)] to the decoding unit 112c. Then, the decryption unit 11
2a to 112c respectively decode the layer A stream to the layer C stream to obtain images of the layer A to the layer C, and output these images to the corresponding evaluation units 113a to 113c, respectively (the above, step S11 in FIG. 2). . Evaluation unit 11
3a to 113c calculate the image quality evaluation values of layers A to C in accordance with the above-described evaluation procedure, and
0 (step S12). Video code transmitter 4
0 is based on the three transmitted image quality evaluation values, and appropriately selects an encoding mode and an encoding parameter of the hierarchical stream to be transmitted to the hierarchical moving image evaluation device 110, and encodes the hierarchical stream to be transmitted. The image quality is optimized (step S13).

As described above, in the present embodiment, the layer A stream including the moving image codes of layers A to C is transmitted to the layer moving image evaluation device 110, and the layer moving image evaluation device 110 After decoding the images of the layers A, B, and C to evaluate the image quality of each, the obtained image quality evaluation value is fed back to the moving image code transmission unit 40. Through these steps,
Compared with the conventional procedure shown in FIG. 8, the number of times of encoding, the number of times of transmission, the scale of decoding processing, and the like can be reduced.

When the moving picture code transmitting section 40 or the moving picture code receiving section 60 has the same function as the hierarchical moving picture relay apparatus, the hierarchical moving picture evaluation function The effect on image quality can be evaluated. For example, assuming that the moving image code transmitting unit 40 has a hierarchical moving image relay function, the moving image code transmitting unit 40 first inputs the hierarchical stream input thereto to the hierarchical moving image evaluation device 110 as it is. Thereby, each layer A to A included in this layer stream
As described above, the images of the hierarchy C are respectively evaluated by the evaluation units 113a.
To 113c. At that time, the moving image code transmitting unit 40
These evaluation units 113a to 113a
13c is notified that the supplied image is included in the hierarchical stream before relay. Then, the evaluation units 113a to 113c store therein the images supplied from the decoding units 112a to 112c, respectively.

Next, the moving picture code transmitting section 40 inputs the reduced number of layers of the input hierarchical stream according to the conditions of the transmission path and the like as described above to the hierarchical moving picture evaluation device 110. As a result, the images of layers A to C included in the layer stream are evaluated by the evaluation units 113a to 113c, respectively.
c. At this time, the moving picture code transmitting section 40 uses these paths (not shown) to evaluate these evaluation sections 113a to 113a.
13c is notified that the supplied image is included in the relayed hierarchical stream. Therefore, the evaluation units 113a to 113c determine whether the original image and the decoding units 112a to 112a
Instead of calculating a difference image between the image decoded in step 12c and the image before relaying, a difference image between the image before relaying and the image after relaying is calculated, and image quality evaluation values are calculated for the difference images. The result is fed back to the moving image code transmission unit 40. In this way, the moving picture code transmitting section 40 can evaluate the effect of the hierarchical moving picture relay function of the moving picture code transmitting section 40 on the image quality.

When the moving picture code receiving section 60 has a hierarchical moving picture relay function, the operation is the same as that of the moving picture code transmitting section 40. That is, the moving picture code receiving unit 60 first inputs the hierarchical stream before relaying to the hierarchical moving picture evaluation device 110, and inputs and stores the images of each layer to the evaluation units 113a to 113c. Next, the moving image code receiving unit 60 inputs the layer stream after the relay to the hierarchical moving image evaluation device 110 and inputs the image of each layer to the evaluation units 113a to 113c.
Therefore, the evaluation units 113 a to 113 c calculate the image quality evaluation value for each layer from the difference image between the images before and after the relay and feed back to the moving image code transmission unit 40.

[Second Embodiment] This embodiment is intended to further reduce the load on the network and the load associated with the encoding and decoding processes. FIG. 3 is a block diagram showing a configuration of a moving image communication device to which the hierarchical moving image evaluation device according to the present embodiment is applied. In the drawing, the same components as those shown in FIG. is there. Also,
The flowchart showing the outline of the operation of this embodiment is the same as that of the first embodiment, and is as shown in FIG. Now, in FIG. 3, the hierarchical moving image evaluation device 110 includes a number of hierarchical moving image evaluating units corresponding to the number of video objects included in the hierarchical stream transmitted by the moving image code transmitting unit 40. However, FIG. 3 shows only the hierarchical moving image evaluation units 110-1 and 110-2 to which the hierarchical streams of the video objects # 1 and # 2 are input. Further, since these hierarchical moving image evaluation units all have the same configuration and function including those not shown, only the detailed structure of the hierarchical moving image evaluating unit 110-1 is shown in FIG.
Note that a method for inputting a hierarchical stream to the hierarchical moving image evaluation device 110 can be exactly the same as that in the first embodiment.

In the hierarchical moving picture evaluation section 110-1, the hierarchical additional code decomposition section 114 converts the input hierarchical A stream into the hierarchical A additional code 43, the hierarchical B additional code 42, the hierarchical C
Each of the codes (see FIGS. 6A to 6C) is extracted, and the extracted codes are respectively added to the additional code decoder 115a and the additional code decoder 115.
b, output to the decoding unit 112c. On the other hand, the additional code decoding unit 115a obtains a difference image between the hierarchical A image and the hierarchical B image by decoding the hierarchical A additional code 43 extracted from the hierarchical A stream. In addition, the additional code decoding unit 115b
The difference image between the layer B image and the layer C image is obtained by decoding the layer B additional code 42 extracted from the stream. On the other hand, the decoding unit 116a adds and combines the difference image between the layer A image and the layer B image decoded by the additional code decoding unit 115a and the layer B image output from the decoding unit 116b (described later), An A image is obtained, and this hierarchical A image is output to the evaluation unit 113a. Similarly, the decoding unit 116b
A layer B image is obtained by adding and combining the difference image between the layer B image and the layer C image decoded by the additional code decoding unit 115b and the layer C image output from the decoding unit 112c,
This layer B image is output to the evaluation unit 113b and the decoding unit 116a.

Next, the hierarchical moving image evaluation apparatus 1 having the above configuration
The operation of the evaluation procedure using No. 10 will be described. In the present embodiment, a layer A stream (FIG. 6A) is assumed as the layer stream of each video object as in the first embodiment, but a layer stream having another configuration may be used. Now, the moving image code transmitting unit 40
Transmits a moving image code composed of a plurality of video objects to the transmission path 20 in the form of a hierarchical stream, the hierarchical stream is input to the hierarchical moving image evaluation device 110 using the method exemplified in the first embodiment.

In the following, a case where a hierarchical stream of the video object # 1 is input to the hierarchical moving image evaluation unit 110-1 as a representative of a hierarchical stream of a plurality of video objects will be described. First, the layer additional code decomposition unit 114 decomposes the input layer A stream into the moving image codes of layers A to C, and outputs the layer A additional code 43 to the additional code decoding unit 115a. The layer B additional code 42 is output to 115b, and the layer C code 41 is output to the decoding unit 112c. As a result, the additional code decoding unit 115a outputs the difference image between the layer A image and the layer B image obtained by decoding the layer A additional code 43 to the decoding unit 116a. Also, additional code decoding section 115
“b” outputs the difference image between the layer B image and the layer C image obtained by decoding the layer B additional code 42 to the decoding unit 116b. Further, the decoding unit 112c outputs the layer C image obtained by decoding the layer C code 41 to the decoding unit 116b and the evaluation unit 113c.

When the hierarchical C image is supplied in this way, the decoding unit 116b adds the difference image between the hierarchical B image and the hierarchical C image output from the additional code decoding unit 115b and the hierarchical C image supplied from the decoding unit 112c. The images are combined to obtain a layer B image, and this is output to the decoding unit 116a and the evaluation unit 113b. When the layer B image is supplied, the decoding unit 116
a obtains a layer A image by adding and combining the difference image between the layer A image and the layer B image output from the additional code decoding unit 115a and the layer B image supplied from the decoding unit 116b, and sends this to the evaluation unit 113a. Output. As described above, the hierarchy A image to the hierarchy C image are input to the evaluation units 113a to 113c, respectively (the above, step S11 in FIG. 2).
Then, the evaluation units 113a to 113c receive the hierarchy A image to the hierarchy C image, respectively, calculate the image quality evaluation value in exactly the same way as in the first embodiment, and transmit each to the moving image code transmission unit 40 (step S12). The above-described operation is exactly the same when, for example, the video object # 2 is input to the hierarchical moving image evaluation unit 110-2. Further, the hierarchical moving image evaluation unit to which the hierarchical stream of the video object (not shown) is input. The same applies to. In this way, the image quality evaluation value of each layer for all video objects is transmitted to the moving image code transmission unit 40.
The moving picture code transmission unit 40 comprehensively evaluates these image quality evaluation values, and selects a coding mode and a coding parameter of the moving picture stream to optimize the picture quality of the moving picture (step S13).

As described above, in the present embodiment, the image quality is evaluated according to the procedure shown in FIG. 2 as in the first embodiment. It is possible to reduce the number of transmissions, the decoding processing scale, and the like. When either or both of the moving image code transmitting unit 40 and the moving image code receiving unit 60 have a hierarchical moving image relay function, a desired hierarchical moving image evaluation unit is performed according to the same procedure as in the first embodiment. 110-1,110-2, ...,
By inputting a hierarchical stream, the effect of the hierarchical moving image relay function on the image quality can be evaluated for each video object.

In each of the above embodiments, the layers A to
Although the configuration and operation of the hierarchical moving image evaluation apparatus have been described only for the three layers of the layer C, it is clear from the above description that the present invention is applicable to an arbitrary number of layers. In the first embodiment, a hierarchical moving image evaluator is provided for each video object as in the second embodiment.
1, # 2,... May be input to evaluate the image quality of each of these video objects.

[0044]

As described above, according to the first aspect of the present invention, a hierarchical stream is decomposed into moving image codes for each layer,
Each of the moving image codes is decoded to obtain an image for each layer, and an image quality evaluation value of the image for each layer is calculated. This makes it possible to perform image quality evaluation on all of the images of a plurality of layers included in the hierarchical stream by encoding the hierarchical stream and transmitting it to the hierarchical moving image evaluation device only once. Therefore,
Conventionally, the load required for the encoding process and the transmission process required for the number of times corresponding to the number of layers included in the layer stream is reduced to one time, and the load on the network and the like can be greatly reduced. .

According to the second or third aspect of the present invention, only the moving picture code of the lowest resolution layer is directly encoded from an image to be a moving picture code. The additional code obtained by encoding the difference image obtained from the reference image of the low resolution hierarchy is a moving image code. Then, the hierarchical stream is decomposed into moving image codes for each layer, and each of them is decoded to obtain an image for each layer,
The image quality evaluation value of the image for each layer is calculated. As a result, for each layer other than the lowest resolution, it is sufficient to decode an additional code having a small code amount. It is possible to reduce the load of the decoding process as compared with the case where it has been performed.

According to the fourth aspect of the present invention, each means constituting the hierarchical moving image evaluation device is provided for each video object, and the image quality of each video object is independently evaluated. This makes it possible to reduce the number of times of encoding, the number of times of communication, and the scale of decoding processing, respectively, when compared with the conventional procedure in which the processing related to image quality evaluation is performed by the number of video objects multiplied by the number of layers. Becomes According to the fifth aspect of the present invention, the moving picture code transmitting means can encode and transmit the hierarchical stream based on the image quality evaluation value calculated by the hierarchical moving picture evaluation device. By appropriately selecting the encoding mode, the encoding parameter, and the like, the image quality of the hierarchical stream transmitted from the moving image code transmission unit can be optimized. According to the sixth aspect of the present invention, one or both of the moving image code transmitting means and the moving image code receiving means have a hierarchical moving image relay function of a hierarchical stream, and output the hierarchical streams before and after the relay to the hierarchical moving image evaluation device. I am trying to do it. Then, the evaluation means in the hierarchical moving image evaluation device calculates the image quality evaluation value of the difference image of the hierarchical stream before and after the relay according to the instruction of the moving image code transmitting means or the moving image code receiving means having the hierarchical moving image relay function. This makes it possible to provide a mechanism for evaluating the effect of the hierarchical moving image relay function of the moving image code transmitting unit and the moving image code receiving unit on the image quality.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a moving image communication device to which a hierarchical moving image evaluation device according to a first embodiment of the present invention is applied.

FIG. 2 is a flowchart illustrating an outline of an operation of a moving image communication device to which the hierarchical moving image evaluation device according to each embodiment of the present invention is applied.

FIG. 3 is a block diagram illustrating a configuration of a moving image communication device to which a hierarchical moving image evaluation device according to a second embodiment of the present invention is applied.

FIG. 4 is a block diagram showing a configuration of a moving image communication device to which a moving image evaluation device according to a conventional technique is applied.

5A and 5B are explanatory diagrams showing an example of a method of forming a hierarchy according to the spatial resolution of an image, wherein FIG. 5A is an explanatory diagram of a high-resolution hierarchical A image, and FIG. FIG. 3C is an explanatory diagram of a layer B image having a lower layer, and FIG.

FIG. 6 is an explanatory diagram showing a configuration of a hierarchical stream, wherein (a) to (c) are hierarchical streams corresponding to the hierarchical A image to the hierarchical C image shown in FIGS. 5 (a) to (c), respectively. is there.

FIGS. 7A and 7B are explanatory diagrams showing an example of a method of forming a hierarchy according to the temporal resolution of an image. FIG. 7A is an explanatory diagram of a high-resolution hierarchical A image, and FIG. FIG. 7 is an explanatory diagram of a hierarchy B image having a lower resolution.

FIG. 8 is a flowchart schematically showing an operation of a moving image communication device to which a moving image evaluation device according to a conventional technique is applied.

[Explanation of symbols]

 Reference Signs List 20 transmission path 40 moving image code transmitting unit 60 moving image code receiving unit 110 hierarchical moving image evaluation device 110-1, 110-2 hierarchical moving image evaluating unit 111 layer decomposing unit 112a to 112c, 116a, 116b decoding unit 113a to 113c evaluating unit 114 additional layer Code decomposition section 115a, 115b additional code decoding section

Claims (6)

[Claims] 1. A hierarchical moving image evaluation device that evaluates a hierarchical stream obtained by encoding moving images of a plurality of layers divided according to the resolution of an image, comprising: a decomposing unit that decomposes the hierarchical stream into moving image codes for each layer; A decoding unit that decodes the moving image code for each layer to obtain an image for each layer; and an evaluation unit that calculates an image quality evaluation value of each image based on the image for each layer. Hierarchical video evaluation device. 2. A hierarchical moving image evaluation device that evaluates a hierarchical stream obtained by encoding moving images of a plurality of layers divided according to the resolution of an image, comprising: The moving image code is a moving image code obtained by encoding the image of the layer, and the moving image code of each layer other than the lowest resolution layer is an image of each layer and an image of a layer having a lower resolution than each layer. An additional code obtained by encoding a difference image between the image and the image, and a decomposing unit that decomposes the hierarchical stream into a moving image code for each layer; A hierarchical moving image evaluation device, comprising: decoding means for obtaining an image; and evaluation means for calculating an image quality evaluation value of each image based on the image for each layer. 3. The decoding means comprises: an image decoding means for decoding the moving image code of the lowest resolution hierarchy to obtain an image of the hierarchy; and each hierarchy other than the lowest resolution hierarchy by decoding the additional code. Additional code decoding means for obtaining a difference image corresponding to the difference image decoded by the additional code decoding means and the reference image based on which the difference image is based. 3. The hierarchical moving image evaluation device according to claim 2, further comprising a synthesizing unit for obtaining an image. 4. A set of the decomposing means, the decoding means, and the evaluating means is provided for each video object in a moving image included in the hierarchical stream, and a hierarchical stream of each video object is provided for each video object. The hierarchical moving image evaluation device according to any one of claims 1 to 3, wherein the evaluation is performed for each of the video objects by giving the evaluation result to the disassembly means provided in (1). 5. The hierarchical moving image evaluation device according to claim 1, a moving image code transmitting unit that transmits the hierarchical stream, and a moving image that receives the transmitted hierarchical stream via a transmission path. A moving image communication device, comprising: a code receiving unit; and the moving image code transmitting unit encodes and transmits the hierarchical stream based on an image quality evaluation value calculated by the hierarchical moving image evaluation device. 6. At least one of the moving picture code transmitting means and the moving picture code receiving means has a hierarchical moving picture relay function of reconstructing the hierarchical stream into a hierarchical stream that can be transferred to a predetermined destination and transferring the hierarchical stream to the destination. The hierarchical stream before and after the relay is output to the hierarchical video evaluation device, respectively, and the evaluation unit is input according to an instruction of the video code transmission unit or the video code reception unit having the hierarchical video relay function. The moving image communication apparatus according to claim 5, wherein an image quality evaluation value of a difference image of an image obtained from each of the hierarchical streams before and after the relay is calculated.