CN103945213B - A kind of video flow quality monitoring method and device - Google Patents

Abstract

The invention discloses a video stream quality monitoring method and device. Wherein, the method includes: obtaining video stream information on the network side; calculating the quality score of the reference frame and the coding quality score of the current video frame according to the video stream information; according to the quality score of the reference frame, the coding quality score of the current video frame, And the coding type of the current video frame, calculate the source coding quality score; calculate the network damage score according to the video stream information; where, the network damage is the video stream quality damage caused by packet loss and delay jitter; according to the source coding quality score and network Impairment score, which determines the quality monitoring score of the video stream. Through the present invention, the problem of low monitoring accuracy and complex implementation of the network side video stream quality monitoring method in the related art is solved, and the dynamic change of video quality is accurately reflected, and the accuracy of video stream quality monitoring is improved, and the complexity is not high , and can be easily applied to the real-time monitoring environment of video quality on the network side.

Description

Method and device for monitoring video stream quality

technical field

The invention relates to the field of mobile communication, in particular to a video stream quality monitoring method and device.

Background technique

In recent years, with the increase of network bandwidth, real-time video services such as streaming media services, video telephony services, and video conference services have developed very rapidly. Accurate evaluation of the quality of these video services can not only let users understand the service level of video services, but also provide service providers with a yardstick for service level improvement, which plays an important role in the good operation of services. Especially in the actual operation of the business, the online monitoring of the business can dynamically grasp the quality status of the business operation, and timely discover the quality problems in the process of business operation, which has strong practical value.

The online monitoring of video streaming service quality can be realized on the service side, network side and terminal side. The service quality monitoring on the service side mainly monitors the quality provided by the service side, and cannot reflect the user experience quality on the terminal side. Terminal-side service quality monitoring needs to be deployed on user terminals, which generally requires the consent of users to deploy. Deployment is difficult, and it can only reflect the service experience of a single user, and it is difficult to reflect the experience of network group users. Some terminal-side service quality monitoring methods may also affect the user's use of the service, thereby affecting the user's service experience. The service quality monitoring on the network side can be deployed on each network node without the cooperation of the user and will not affect the user. It can monitor multiple service sessions passing through the network node at the same time, thus reflecting the quality status of the network that the network node is responsible for. If it is deployed on a network node very close to the user terminal, although it cannot directly reflect the user experience like the terminal-side method, it can also reflect the business operation status well.

However, there are some challenges in online monitoring of video streaming service quality on the network side. First of all, the service data throughput on the network side is relatively large, especially for video services, so the requirements for data processing speed are very high, and the corresponding quality monitoring methods cannot be too complicated; secondly, there is no original data of video services on the network side, only Quality evaluation can be performed by analyzing video service streams, which is called no-reference method in media quality evaluation. No-reference method has always been a difficulty in the field of quality evaluation, either with low accuracy or too complicated; thirdly, network-side service quality The monitoring basis for monitoring needs to be the data packets that have arrived at the current moment, and the quality status needs to be dynamically updated according to the currently received data packets, which requires the monitoring method to be able to process real-time business flow data in real time.

Existing no-reference methods that can be applied to the network side generally consider the impact of source video coding and network performance on video quality, but differ in terms of specific factors covered. For example, ITU-T G.1070 proposes a relatively complete set of no-reference framework for quality evaluation of video telephony services. However, when evaluating video quality, source video coding only considers the encoder type, bit rate, and frame rate. The network performance only considers the packet loss factor; the source video coding of the Moving Picture Quality Metric (MPQM) model only gives a fixed score according to the different encoder types, and the network performance only considers Packet loss leads to low monitoring accuracy of video stream quality. In other models, many factors are considered, but their specific implementation is very vague. For example, the T-V-model considers the damage caused by video coding, resolution change, transmission process, and display process, but the relationship between these damages and the final score is too simple, and the specific representation method and influencing factors of the damage are not clarified . Some other models have similar problems.

Aiming at the problem of low monitoring accuracy and complex implementation of the video stream quality monitoring method on the network side in the related art, no effective solution has been proposed yet.

Contents of the invention

Aiming at the problem of low monitoring accuracy and complex implementation of the video stream quality monitoring method on the network side in the related art, the present invention provides a video stream quality monitoring method and device to solve the above technical problems.

According to one aspect of the present invention, the present invention provides a video stream quality monitoring method, wherein the method includes: obtaining video stream information on the network side; calculating the quality score of the reference frame and the current video frame according to the above video stream information The coding quality score of the above-mentioned reference frame, the coding quality score of the above-mentioned current video frame, and the coding type of the above-mentioned current video frame, calculate the source coding quality score; according to the above-mentioned video stream information, calculate the network damage score; wherein, The above network damage is the video stream quality damage caused by packet loss and delay jitter; according to the above source coding quality score and the above network damage score, determine the quality monitoring score of the video stream.

Further, obtaining the video stream information on the network side includes: decapsulating and demultiplexing the video service network data on the network side to obtain the video stream; parsing the above video stream, and locating the current video frame and the previous video frame in the above video stream The starting point and the ending point of the current video frame; the starting point and the ending point of the current video frame, and the starting point and the ending point of the above-mentioned previous video frame are respectively reversely mapped to the above-mentioned video service network data packet, and the above-mentioned current video frame and the above-mentioned current video frame are respectively obtained. The arrival time of all network packets of the last video frame; respectively determine the arrival time of the current video frame and the arrival time of the last video frame according to the arrival time; determine according to the end point of the last video frame and the end point of the above-mentioned current video frame The number of bits of the above-mentioned current video frame; determine the real-time packet loss rate, real-time bit rate and real-time frame rate of the above-mentioned current video frame; analyze the encoded bit stream of the above-mentioned current video frame, and determine the encoding type, size information and quantization of the above-mentioned current video frame parameter.

Further, according to the above video stream information, calculating the quality score of the reference frame includes: judging whether the above current video frame is a reference frame according to the coded bit stream of the above current video frame; if it is a reference frame, performing blockiness evaluation on the above reference frame operation and blur evaluation operation; performing a weighted average on the result of the block effect evaluation operation and the result of the blur evaluation operation to obtain the quality score of the reference frame.

Further, calculating the encoding quality score of the current video frame according to the above video stream information includes: obtaining the number of bits per pixel of the above current video frame according to the number of bits and size information of the above current video frame; according to the above number of bits per pixel and the above For the quantization parameter, the calculation operation corresponding to the coding type of the above-mentioned current video frame is adopted to obtain the time and space complexity; according to the above-mentioned time and space complexity and the above-mentioned quantization parameter, the coding quality score of the above-mentioned current video frame is obtained.

Further, according to the quality score of the above-mentioned reference frame, the coding quality score of the above-mentioned current video frame, and the coding type of the above-mentioned current video frame, calculating the source coding quality score includes: if the coding type of the above-mentioned current video frame is intra-frame prediction I frame , and the above-mentioned current video frame is a reference frame, then the above-mentioned source encoding quality score is equal to the quality score of the above-mentioned reference frame; The quality score of the reference frame, then according to the quality score of the above-mentioned reference frame, the coding quality score of the above-mentioned current video frame, and the coding quality difference between the above-mentioned current video frame and the reference frame, calculate the above-mentioned source coding quality score; if the above-mentioned current video The encoding type of the frame is an intra-frame prediction I frame, and the above-mentioned current video frame is not a reference frame, and there is no quality score of the above-mentioned reference frame, then the above-mentioned source coding quality score is equal to the coding quality score of the above-mentioned current video frame; if the above-mentioned current video frame If the encoding type is forward predictive P frame, the quality monitoring score of the previous video frame is obtained; according to the quality monitoring score of the previous video frame, the encoding quality score of the current video frame, and the relationship between the current video frame and the previous The difference in encoding quality between video frames, computing the source encoding quality score above.

Further, according to the above video stream information, calculating the network damage score includes: setting the length of the first time window; wherein, the end point of the first time window is the above current video frame, and the start point of the first time window is the same as the above The time interval of the current video frame is not greater than the video frame of the above-mentioned first time window length; according to the real-time packet loss rate of each video frame in the above-mentioned first time window, the real-time average packet loss rate is calculated; according to the real-time average packet loss rate in the above-mentioned first time window The real-time average bit rate of each video frame is calculated to obtain the real-time average bit rate; according to the real-time frame rate of each video frame in the first time window, the real-time average frame rate is calculated; according to the above-mentioned real-time average bit rate and the above-mentioned real-time average frame rate, calculate the packet loss robustness factor; according to the above packet loss robustness factor and the above real-time average packet loss rate, calculate the packet loss damage score; according to the above real-time average frame rate at the arrival time of the current video frame, and the above The real-time average frame rate at the arrival time of the last video frame is calculated to obtain the delay jitter score; wherein, the above-mentioned network damage score includes the above-mentioned packet loss damage score and the above-mentioned delay jitter score.

Further, according to the above-mentioned source coding quality score and the above-mentioned network damage score, determining the quality monitoring score of the above-mentioned video stream includes: according to the above-mentioned source coding quality score, the above-mentioned packet loss damage score, and the above-mentioned delay jitter score, calculating the Dynamic scoring; setting the length of the second time window; wherein, the end point of the second time window is the above-mentioned current video frame, and the start point of the second time window is that the time interval with the above-mentioned current video frame is not greater than the above-mentioned second time The video frames of the window length; the dynamic score of each video frame in the above-mentioned second time window is averaged to obtain the quality monitoring score of the above-mentioned video stream.

According to another aspect of the present invention, the present invention also provides a video stream quality monitoring device, wherein the device includes: an information acquisition module, used to obtain video stream information on the network side; a source coding quality scoring module, used to Video stream information, calculate the quality score of the reference frame, and the encoding quality score of the current video frame; calculate the source encoding quality score according to the quality score of the reference frame, the encoding quality score of the current video frame, and the encoding type of the current video frame; network The damage scoring module is used to calculate the network damage score according to the video stream information; wherein, the network damage is the video stream quality damage caused by packet loss and delay jitter; the quality monitoring and scoring module is used to score and network damage according to the source coding quality Score, which determines the quality monitoring score for the video stream.

Further, the information acquisition module includes: a video stream acquisition subunit, used to decapsulate and demultiplex the video service network data on the network side, to obtain a video stream; a time determination subunit, used to analyze the above video stream, and locate the current The start point and end point of the video frame and the previous video frame in the above video stream; the start point and end point of the current video frame, and the start point and end point of the above last video frame are reversely mapped to the above video The service network data packet obtains the arrival time of all network packets of the above-mentioned current video frame and the last video frame respectively; respectively determine the arrival time of the current video frame and the arrival time of the last video frame according to the arrival time; the number of bits determines the subunit, with Determine the number of bits of the above-mentioned current video frame according to the end point of the above-mentioned last video frame and the end point of the above-mentioned current video frame; the encoding information determination subunit is used to determine the real-time packet loss rate and real-time bit rate of the above-mentioned current video frame and a real-time frame rate; analyzing the coded bit stream of the current video frame to determine the coding type, size information and quantization parameters of the current video frame.

Further, the above-mentioned source coding quality scoring module includes a quality scoring unit of the reference frame, which is used to calculate the quality score of the reference frame according to the above-mentioned video stream information; wherein, the quality scoring unit of the above-mentioned reference frame includes: a reference frame judging subunit, used Judging whether the above-mentioned current video frame is a reference frame according to the coded bit stream of the above-mentioned current video frame; the evaluation subunit is used to perform a block effect evaluation operation and a blur evaluation operation on the above-mentioned reference frame if it is a reference frame; the quality of the reference frame The scoring subunit is configured to perform a weighted average on the result of the block effect evaluation operation and the result of the blur evaluation operation to obtain the quality score of the reference frame.

Further, the above-mentioned source coding quality scoring module includes a coding quality scoring unit, which is used to calculate the coding quality scoring of the current video frame according to the above-mentioned video stream information; wherein, the above-mentioned coding quality scoring unit includes: a subunit for determining the number of bits per pixel, using According to the number of bits and the size information of the current video frame above, the number of bits per pixel of the above-mentioned current video frame is obtained; the time and space complexity determination subunit is used to adopt the above-mentioned number of bits per pixel and the above-mentioned quantization parameters. Computing operations corresponding to the encoding type of the video frame to obtain the time and space complexity; the encoding quality scoring subunit is used to obtain the encoding quality score of the current video frame according to the above-mentioned time and space complexity and the above-mentioned quantization parameters.

Further, the above-mentioned source coding quality scoring module includes a source coding quality scoring unit, which is used to calculate the source coding quality score according to the quality score of the above-mentioned reference frame, the coding quality score of the above-mentioned current video frame, and the coding type of the above-mentioned current video frame; Wherein, the above-mentioned source coding quality scoring unit includes: a first scoring subunit, used to score the above-mentioned source coding quality when the coding type of the above-mentioned current video frame is intra-frame prediction I frame, and the above-mentioned current video frame is a reference frame Equal to the quality score of the above-mentioned reference frame; the second scoring subunit is used to predict the I frame in the encoding type of the above-mentioned current video frame, the above-mentioned current video frame is not a reference frame, and there is a quality score of the above-mentioned reference frame , according to the quality score of the above-mentioned reference frame, the coding quality score of the above-mentioned current video frame, and the coding quality difference between the above-mentioned current video frame and the reference frame, calculate the above-mentioned source coding quality score; the third scoring subunit is used in the above-mentioned The coding type of the current video frame is an intra-frame prediction I frame, and the above-mentioned current video frame is not a reference frame, and there is no quality score of the above-mentioned reference frame, the above-mentioned source coding quality score is equal to the coding quality score of the above-mentioned current video frame; Four scoring subunits, used to obtain the quality monitoring score of the previous video frame when the coding type of the above-mentioned current video frame is a forward prediction P frame; according to the quality monitoring score of the above-mentioned last video frame, the above-mentioned current video frame and the coding quality difference between the current video frame and the previous video frame to calculate the source coding quality score.

Further, the above-mentioned network damage scoring module includes: a first setting unit, configured to set the length of the first time window; wherein, the end point of the first time window is the above-mentioned current video frame, and the start point of the first time window is It is a video frame whose time interval with the above-mentioned current video frame is not greater than the length of the above-mentioned first time window; the average unit is used to calculate the real-time average packet loss rate according to the real-time packet loss rate of each video frame in the above-mentioned first time window ; According to the real-time bit rate of each video frame in the above-mentioned first time window, calculate the real-time average bit rate; According to the real-time frame rate of each video frame in the above-mentioned first time window, calculate the real-time average frame rate; packet loss damage The scoring unit is used to calculate the packet loss robustness factor according to the above-mentioned real-time average bit rate and the above-mentioned real-time average frame rate; according to the above-mentioned packet loss robustness factor and the above-mentioned real-time average packet loss rate, calculate and obtain the packet loss damage score; The delay jitter scoring unit is used to calculate the delay jitter score according to the real-time average frame rate at the arrival time of the current video frame and the real-time average frame rate at the arrival time of the previous video frame; wherein the network damage score includes The aforementioned packet loss impairment score and the aforementioned latency jitter score.

Further, the above-mentioned quality monitoring and scoring module includes: a dynamic scoring unit, which is used to calculate and obtain the dynamic scoring of the current video frame according to the above-mentioned source coding quality score, the above-mentioned packet loss damage score, and the above-mentioned delay and jitter score; the second setting unit, It is used to set the length of the second time window; wherein, the end point of the second time window is the above-mentioned current video frame, and the start point of the second time window is that the time interval with the above-mentioned current video frame is not greater than the above-mentioned second time window length of the video frame; a quality monitoring and scoring unit, configured to average the dynamic scores of each video frame in the second time window to obtain the quality monitoring score of the video stream.

Through the present invention, after the video stream information on the network side is obtained, the source coding quality score and the network damage score are obtained according to the video stream information, wherein the reference frame is used in the calculation process of the source coding quality score, and the network damage score In the calculation process, the video stream quality damage caused by packet loss and delay jitter is considered. Finally, according to the above-mentioned source coding quality score and the above-mentioned network damage score, the quality monitoring score of the video stream is determined, which solves the problem of video stream quality on the network side in related technologies. The monitoring method has low monitoring accuracy and complicated implementation. It can more accurately reflect the dynamic changes in video quality and improve the accuracy of video stream quality monitoring. The complexity is not high, and it can be easily applied to network-side video quality. Real-time monitoring environment.

Description of drawings

Fig. 1 is the flow chart of the video flow quality monitoring method according to the embodiment of the present invention;

FIG. 2 is a flow chart of obtaining video stream information on the network side according to an embodiment of the present invention;

Fig. 3 is a flow chart of quality scoring of a reference frame according to an embodiment of the present invention;

Fig. 4 is a flow chart of encoding quality scoring according to an embodiment of the present invention;

Fig. 5 is a flow chart of scoring source coding quality according to an embodiment of the present invention;

FIG. 6 is a flow chart of network damage scoring according to an embodiment of the present invention;

Fig. 7 is a structural block diagram of a video stream quality monitoring device according to an embodiment of the present invention;

Fig. 8 is a schematic structural diagram of an apparatus for monitoring video stream quality according to an embodiment of the present invention.

detailed description

In order to solve the problem of low monitoring accuracy and complex implementation of the video stream quality monitoring method on the network side in the related art, the present invention provides a video stream quality monitoring method and device. The present invention is further described below in conjunction with the accompanying drawings and embodiments Detailed description. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

This embodiment provides a method for monitoring video stream quality, which can be implemented on the network side. FIG. 1 is a flow chart of a method for monitoring video stream quality according to an embodiment of the present invention. As shown in FIG. 1 , the method includes the following steps (step S102-step S108):

Step S102, acquiring video stream information on the network side; the video stream information provides a calculation basis for subsequent calculation of source coding quality score and network damage score. specifically:

The above video stream information includes: the coded bit stream of the current video frame, the number of bits of the current video frame, size information, encoding type, quantization parameters, real-time packet loss rate, real-time bit rate, real-time frame rate and the arrival time of the current video frame, the last The arrival time of a video frame;

Step S104: Calculate the quality score of the reference frame and the coding quality score of the current video frame according to the video stream information; according to the quality score of the reference frame, the coding quality score of the current video frame, and the coding type of the current video frame , calculate the source coding quality score;

Step S106, calculating the network damage score according to the above video stream information; wherein, the above network damage is the video stream quality damage caused by packet loss and delay jitter;

Step S108, according to the above-mentioned source coding quality score and the above-mentioned network damage score, determine the quality monitoring score of the video stream.

Through the above method, after the video stream information on the network side is obtained, the source coding quality score and the network damage score are obtained according to the video stream information, wherein, the reference frame is used in the calculation process of the source coding quality score, and the In the calculation process, the video stream quality damage caused by packet loss and delay jitter is considered. Finally, according to the above-mentioned source coding quality score and the above-mentioned network damage score, the quality monitoring score of the video stream is determined, which solves the problem of video stream quality on the network side in related technologies. The monitoring method has low monitoring accuracy and complicated implementation. It can more accurately reflect the dynamic changes in video quality and improve the accuracy of video stream quality monitoring. The complexity is not high, and it can be easily applied to network-side video quality. Real-time monitoring environment.

For the acquisition operation of the video stream information in the above step S102, this embodiment provides a preferred implementation mode, that is, acquiring the video stream information on the network side includes: decapsulating and demultiplexing the video service network data on the network side to obtain Video stream; analyze the above video stream, locate the start point and end point of the current video frame and the previous video frame in the above video stream; set the start point and end point of the current video frame, and the start point and end point of the above previous video frame The end points are respectively reversely mapped to the above-mentioned video service network data packets, and the arrival times of all network packets of the above-mentioned current video frame and the previous video frame are obtained respectively; according to the arrival times, the arrival times of the current video frame and the previous video frame are respectively determined Arrival time; according to the end point of the previous video frame and the end point of the current video frame, determine the number of bits of the current video frame; determine the real-time packet loss rate, real-time bit rate and real-time frame rate of the above-mentioned current video frame; analyze the above-mentioned current video The coded bit stream of the frame determines the coding type, size information and quantization parameters of the above-mentioned current video frame. Through this preferred embodiment, a calculation basis is provided for the subsequent calculation of the source coding quality score and the network damage score, and the monitoring precision and accuracy of the video stream quality are mentioned.

Fig. 2 is a flow chart of obtaining video stream information on the network side according to an embodiment of the present invention. As shown in Fig. 2, the process includes the following steps (step S202-step S210):

Step S202, decapsulating and demultiplexing the video service network data to obtain a video stream. For example, under the protocol framework of RTP (Real-time Transport Protocol)/UDP (User Datagram Protocol)/IP (Internet Protocol, a protocol for interconnection between networks), it can decapsulate Ethernet frames and IP packets , UDP packet decapsulation, RTP packet decapsulation to obtain the video service stream, and then obtain the video stream through demultiplexing (such as H.223 demultiplexing).

Step S204, analyzing the video stream to find the start code, so as to locate the start point and end point of the coded bit stream of the current video frame in the video stream.

Step S206, reversely map the start point and end point of the current video frame to the video service network data packet, and obtain the sequence numbers and arrival times of all network data packets used to encapsulate the current video frame.

Step S208, from the arrival time of all network packets of the video frame, the arrival time, earliest arrival time and latest arrival time of the video frame can be obtained. The earliest arrival time of the video frame is the arrival time of the first network data packet of the video frame, and both the arrival time and the latest arrival time are the arrival time of the last network data packet of the video frame.

Step S210, the interval between the end points between two frames is the number of bits contained in the current video frame, continue to analyze the encoded bit stream of the current video frame, and extract other encoding information: the encoding type of the frame (I frame or P frame); Dimensions (width and height); quantization parameters of the frame. In order to obtain the above information, only a small amount of information needs to be parsed, and the amount of calculation is very small. For example, the frame header of each frame can be parsed in H.261 and H.263 video streams, and SPS (Sequence Parameter Set, sequence parameter set) and PPS (Picture Parameter Set, image parameter set) can be parsed in H.264 video stream parameter set).

Wherein, if the frame encoding type of the current video frame is an I frame, and the interval with the previous reference frame is greater than G GoPs, then continue to judge whether it is an I frame without packet loss, and if so, use it as a new reference frame, and Save the coded bit stream of the frame, if not, the frame cannot be used as a new reference frame. The method for judging whether there is packet loss in a frame is: if the sequence number of the last network packet corresponding to the frame minus the sequence number of the first network packet is equal to the number of network packets corresponding to the frame, then the frame is considered to have no packet loss of. In practical applications, the value of G can be determined according to the nature of the business, the requirements for evaluation accuracy, and the requirements for real-time evaluation.

Of course, the implementation of the method for acquiring video stream information in this embodiment is not limited to the above process, as long as the video stream information can be accurately acquired. For example, the current video frame can be located by cooperating with the encapsulation protocol in the system. For example, if the RTP protocol is used to directly encapsulate the video frame, and the Fragmentation Unit (FU) method is adopted during the RTP encapsulation, the current frame can be located through the flag bit in the FU.

For the calculation of the quality score of the reference frame based on the video stream information in the above step S104, this embodiment provides a preferred implementation mode, that is, to judge whether the above-mentioned current video frame is a reference frame according to the coded bit stream of the above-mentioned current video frame; if so For the reference frame, the block effect evaluation operation and the blur evaluation operation are performed on the reference frame; the results of the above block effect evaluation operation and the results of the above blur evaluation operation are weighted and averaged to obtain the quality score of the reference frame. In this preferred implementation manner, a reference frame is introduced for quality scoring, which improves the monitoring accuracy of video stream quality.

The main spatial domain impairments of real-time video services are blocking effects and blurring. FIG. 3 is a flow chart of the quality scoring of reference frames according to an embodiment of the present invention. As shown in FIG. 3 , the process includes the following steps (step S302-step S310):

Step S302, determine whether the current video frame is a reference frame according to the coded bit stream in the video stream information; specifically, determine whether the coded bit stream of the current video frame exists, if it exists, then the current video frame is the reference frame; if not, Then the current video frame is a non-reference frame.

Step S304, performing video decoding on the current video frame.

Step S306, evaluating blockiness on the reconstructed image of the current video frame.

Specifically, the following methods can be used to implement: use a simple block effect level statistical method for calculation. Divide the luminance component of the reconstructed image into non-overlapping 8×8 blocks. In an 8×8 luminance block (that is, there are 8 rows and 8 columns), if there are values greater than or equal to 7 rows that are the same, it is considered that the block is “seriously blocked”; except for the edge, each 8× Each of the 8 luminance blocks has 4 adjacent 8×8 luminance blocks. If there are 3 or 4 of the 4 adjacent edges, the difference between more than 5 of the 8 gray values that are adjacent is greater than the threshold value, then It is considered that the block has "serious block effect"; if there are 2 or 1 of the 4 adjacent edges, the difference between the 8 gray values that are adjacent to more than 5 is greater than the threshold, and the block is considered to be a "block". general effect". All 8×8 blocks are classified according to the above method, and the proportions of blocks with various degrees of blockiness in the current video frame are counted respectively, and finally the influence of blockiness on the image is calculated.

Step S308, perform blur evaluation on the reconstructed image of the current frame.

Specifically, a simple average edge width method can be used for calculation: use the Sobel vertical direction operator to obtain the edge map, and for an edge point in the edge map, find the extreme point near the corresponding point of the image (such as the same row) , the distance between the extreme points is the width of the edge, the edge width is calculated for all the edge points in the edge map, and the ratio of the sum of all edge widths to the number of edge points is the blur evaluation of the image.

In step S310, the weighted average of the results of blockiness evaluation and blur evaluation is obtained as the quality score Q ₀ of the reference frame.

Of course, the methods of blockiness evaluation and blur evaluation are not limited to the methods listed in the above embodiments, as long as the quality score of the reference frame can be accurately obtained. For example, the specific implementation method of the quality scoring of the reference frame can be selected according to the nature of the business, the requirements for evaluation accuracy, and the requirements for real-time evaluation. For example, block effects can be evaluated using methods such as block boundary difference evaluation based on singular value decomposition, or block effect evaluation based on local block differences and HVS, and blur can be evaluated using methods such as edge contour gradient statistics and local kurtosis statistics.

In this embodiment, in order to improve the monitoring accuracy of the video stream, the first lossless I frame in one or more Group of Pictures (GoP) can be used as the reference frame, and the reference frame can be extracted from the video stream The coded bit stream of the reference frame is fully decoded for reference-free monitoring in the spatial domain. Use this monitoring result as a scoring basis for other subsequent frames in the one or more GoPs. In practical applications, the interval at which the reference frames appear can be selected according to the nature of the business, the requirements for monitoring accuracy, and the requirements for real-time monitoring.

For the calculation of the encoding quality score of the current video frame according to the video stream information in the above step S104, this embodiment provides a preferred implementation mode, that is, according to the bit number and size information of the current video frame, the per pixel of the current video frame is obtained The number of bits; according to the above-mentioned number of bits per pixel and the above-mentioned quantization parameters, the calculation operation corresponding to the encoding type of the current video frame is adopted to obtain the time and space complexity; according to the time and space complexity and the above-mentioned quantization parameters, the time and space complexity of the current video frame is obtained. Encoding quality rating. Through this preferred implementation manner, the monitoring accuracy of video stream quality is improved.

In practical applications, the larger the quantization parameter, the larger the quantization step size and the lower the video quality, which shows that there is an inverse relationship between the video quality and the quantization parameter, which is represented by a linear model here. At the same time, the encoding quality is also related to time and space complexity, and different time and space complexities will affect the slope of the linear model. The time and space complexity can be characterized by the properties of the residual frame, which is represented by the quantization parameter of the frame and the number of bits per pixel after encoding. Therefore, the encoding quality of the current video frame can be calculated according to the frame encoding type, quantization parameter, bits per pixel, and time and space complexity.

Based on this, in the above embodiments, besides using traditional quantization parameters (Quantization Parameter, QP for short) to measure the encoding quality of the current video frame, time and space complexity of the video frame are further taken into consideration. The time and space complexity of a video frame has the following characteristics: the time and space complexity of a video frame is simply and effectively characterized by the quantization parameter QP of the video frame and the number of coded bits per pixel of the frame, and varies according to the different coding types adopted by the frame Use a different calculation method. Because real-time video services have high requirements on time delay, generally only intra-frame prediction I frames and forward prediction P frames are used instead of bi-directional prediction B frames that introduce time delays. Therefore, only two encoding types are considered here, I frame or P frame.

Fig. 4 is a flowchart of encoding quality scoring according to an embodiment of the present invention. As shown in Fig. 4, the process includes the following steps (step S402-step S406):

Step S402, according to the bit number B and size (width W and height H) of the current video frame in the video stream information, calculate the number of bits per pixel R of the current video frame.

The calculation process of the number of bits per pixel R can be specifically implemented using the following formula:

Step S404, according to the encoding type and quantization parameter Q _P of the current video frame in the video stream information, calculate the time and space complexity δ of the current frame.

If the current frame is an I frame, the time and space complexity is calculated as follows: δ=(a ₁ ×R+b ₁ )×Qp.

If the current frame is a P frame, the time and space complexities are calculated as follows: δ=(a ₂ ×R+b ₂ )×Qp.

Wherein, a ₁ , b ₁ , a ₂ and b ₂ in the above formula are model parameters with constant values.

Step S406, according to the quantization parameter Q _P , and the time and space complexity, the coding quality Q _C of the current video frame is calculated. Specifically, the following formula can be used to achieve: Wherein, a ₃ , b ₃ , a ₄ and b ₄ in the above formula are model parameters with constant values.

After calculating the quality score of the reference frame and the encoding quality score of the current video frame, the source encoding quality score can be calculated. The error propagation impairment evaluation is used to measure the impairment impact of error propagation on the current video frame, and calculate the source coding quality score Q _E . If the current frame is an I frame, the quality of the frame is completely determined by itself and is not affected by error propagation. If the current frame is a P frame, the quality of this frame is not completely determined by itself, but also affected by the quality of its reference frame. If the quality of its reference frame deteriorates, the degradation will propagate to this frame. Real-time video services have strict requirements on delay, so the multi-reference frame technology is generally not used. Therefore, only the error propagation effect of the previous frame on the current frame is considered here.

For the calculation operation of the source coding quality score in the above step S104, this embodiment provides a preferred implementation mode, that is, according to the quality score of the reference frame, the coding quality score of the current video frame, and the coding type of the current video frame, calculate Source encoding quality scores include:

If the encoding type of the current video frame is an intraframe prediction I frame, and the current video frame is a reference frame, then the source coding quality score is equal to the quality score of the reference frame;

If the encoding type of the current video frame is an intra-frame prediction I frame, the current video frame is not a reference frame, and there is a quality score of the reference frame, then according to the quality score of the reference frame, the encoding quality score of the current video frame, and the current video frame and The encoding quality difference between reference frames, and calculate the source encoding quality score;

If the encoding type of current video frame is intraframe prediction I frame, current video frame is not reference frame, and there is no quality score of reference frame, then source encoding quality score is equal to the encoding quality score of current video frame;

If the encoding type of the current video frame is a forward prediction P frame, then obtain the quality monitoring score of the previous video frame; according to the quality monitoring score of the previous video frame, the encoding quality score of the current video frame, and The encoding quality difference between video frames, computing the source encoding quality score.

Fig. 5 is a flow chart of scoring source coding quality according to an embodiment of the present invention. As shown in Fig. 5, the process includes the following steps (step S502-step S506):

Step S502, if the current video frame is an I frame and is a reference frame (judging whether the encoded bit stream of the current video frame exists, if it exists, the current video frame is a reference frame), then the source coding quality score Q _E is the quality of the reference frame Score Q ₀ , ie Q _E |=Q ₀ |.

Step S504, if the current video frame is an I frame and not a reference frame, and the quality score _Q0 of the reference frame already exists, then the source coding quality score _QE is based on the quality score _Q0 of the reference frame, and the current video frame is considered The coding quality difference between the frame and the reference frame is as follows: Wherein, the calculation of Q _C,0 is the same as the calculation method of Qc in step S406 in the above embodiment.

If the current video frame is an I frame and not a reference frame, but the quality score of the reference frame is Q ₀ (at the beginning of the evaluation, a lossless I frame has not yet appeared), then the source coding quality score Q _E is the code of the current video frame Quality score Q _C , ie Q _E |=Q _C |.

Step S506, if the current video frame is a P frame, consider the error propagation of the previous video frame to the current video frame, that is, the source coding quality score Q _E takes the final quality dynamic score Q _pre of the previous video frame as a reference, and consider the current video frame The difference in encoding quality from the previous video frame is as follows: Wherein, the calculation of Q _C,pre is the same as the calculation method of Qc in step S406 in the above embodiment.

For the calculation of the network damage score based on the video stream information in step S106 of the above embodiment, the network damage also takes into account the impact of delay and jitter factors on video quality in addition to packet loss factors. Here, the impact of delay on video quality is not considered, because too much delay will cause packet loss, only packet loss can be considered. The overall delay of the video service flow is shown as initial buffering, which can be alleviated by some additional means such as advertisements.

When considering the impact of packet loss on video quality, the real-time average packet loss rate can be used instead of the commonly used real-time packet loss rate to make the dynamic scoring results more stable. The real-time packet loss rate is the packet loss rate for a period of time before the arrival of the current video frame. The real-time average packet loss rate is the average packet loss rate in a period of time before the current video frame arrives.

In addition, the robustness factor of packet loss can also be calculated to improve the accuracy of video stream quality monitoring. The robustness factor of packet loss is characterized by the real-time average bit rate and real-time average frame rate, that is, the tolerance for packet loss is different when the real-time average bit rate and real-time average frame rate of video streams are different. The larger the real-time average bit rate or real-time frame rate, the lower the tolerance to packet loss. The real-time bit rate is the video streaming rate through the network nodes during a period of time before the arrival of the current video frame. The real-time average bit rate is the video stream rate passing through network nodes during a period of time before the arrival of the current video frame. Since this embodiment is applied to the network side, the frame rate refers to the number of video frames passing through the network nodes within a certain period of time. The real-time frame rate is the number of video frames passing through the network node within a period of time before the arrival of the current video frame. The real-time average frame rate is the average number of video frames passing through the network node within a period of time before the arrival of the current video frame.

When considering the impact of delay jitter on video quality, the change of real-time average frame rate can be used to characterize the delay jitter. The change of the real-time average frame rate is characterized by the difference of the real-time average frame rate at the arrival moment of the video frame before and after. Since the delay jitter of a certain video frame relative to the previous video frame can often be alleviated by the terminal buffer on the terminal side, but for the terminal, the change of the real-time average frame rate will cause the video playback to be fast or slow. Affects the subjective experience of end users.

Based on the above multiple considerations, this embodiment provides a preferred implementation manner, that is, calculating the network damage score according to the video stream information includes: setting the length of the first time window; where the end point of the first time window is the current video frame, The starting point of the first time window is the video frame with the time interval of the current video frame being no greater than the first time window length; according to the real-time packet loss rate of each video frame in the first time window, the real-time average packet loss rate is calculated; Calculate the real-time average bit rate according to the real-time bit rate of each video frame in the first time window; calculate the real-time average frame rate according to the real-time frame rate of each video frame in the first time window; calculate the real-time average frame rate according to the real-time average bit rate and Calculate the packet loss robustness factor based on the real-time average frame rate; calculate the packet loss damage score based on the packet loss robustness factor and the real-time average packet loss rate; The real-time average frame rate at the arrival moment of the video frame is calculated to obtain the delay jitter score; the network damage score includes the packet loss damage score and the delay jitter score. Through this preferred implementation manner, the accuracy of video stream quality monitoring is improved.

FIG. 6 is a flow chart of network damage scoring according to an embodiment of the present invention. As shown in FIG. 6, the process includes the following steps (step S602-step S620):

Step S602, setting the real-time packet loss rate, the real-time average packet loss rate, the real-time bit rate, the real-time average bit rate, the real-time frame rate and the time window used for calculating the real-time average frame rate. Set the time window length T, the end point of the time window is the current video frame, and the start point of the time window is the first video frame whose arrival time interval with the current video frame is not greater than T.

Step S604, calculating the real-time packet loss rate PLR _r at the arrival moment of the current video frame. The calculation method is: the sequence number of the last network data packet corresponding to the current video frame minus the difference of the sequence number of the first network data packet corresponding to the first video frame in the time window, and then subtracting the difference of the sequence number corresponding to all video frames in the time window The number of all network data packets, the obtained difference is divided by the difference between the sequence number of the last network data packet corresponding to the current video frame and the sequence number of the first network data packet corresponding to the first video frame in the time window.

Step S606, calculating the real-time average packet loss rate PLR _a at the arrival moment of the current video frame. The calculation method is: calculate the average of the real-time packet loss rate at the arrival moment of all video frames in the time window. Using the real-time average packet loss rate instead of the real-time packet loss rate can make the evaluation results smoother.

Step S608, calculating the real-time bit rate BR _r at the arrival moment of the current video frame. The calculation method is: the sum of bits of all video frames in the time window, divided by the difference between the latest arrival time of the current video frame and the earliest arrival time of the first video frame in the time window.

Step S610, calculating the real-time average bit rate BR _a at the arrival moment of the current video frame. The calculation method is: calculating the average of the real-time bit rates at the arrival moments of all video frames within the time window. Using real-time average bitrate instead of real-time bitrate can smooth out the evaluation results.

Step S612, calculating the real-time frame rate FR _r at the arrival moment of the current video frame. The calculation method is: the number of all video frames in the time window, divided by the difference between the latest arrival time of the current video frame and the earliest arrival time of the first video frame in the time window.

Step S614, calculating the real-time average frame rate FR _a at the arrival moment of the current video frame. The calculation method is: calculate the average of the real-time frame rates of the arrival times of all video frames in the time window. Using the real-time average frame rate instead of the real-time frame rate can make the evaluation results smoother.

Step S616, calculating the packet loss robustness factor D _plv at the moment of arrival of the current video frame. According to the real-time average bit rate BRa and the real-time average frame rate FRa, use the following formula to calculate: Wherein, a ₅ and b ₅ in the formula are model parameters with constant values.

Step S618, calculating the packet loss damage score PL at the moment of arrival of the current video frame, using the following formula to achieve: $\mathrm{PL}=e^{-\frac{{\mathrm{PLR}}_a}{{\mathrm{D.}}_{\mathrm{plr}}}}.$

Step S620, calculating the delay jitter score DF at the arrival moment of the current video frame. The calculation method is: the average frame rate jitter is the real-time average frame rate at the arrival time of the current video frame minus the real-time average frame rate at the arrival time of the previous frame. The delay jitter score DF is a linear model of the average frame rate jitter, as follows: DF=k×|FR _a -FR _a,pre |.

After calculating the source coding quality score and the network damage score, the results of the source coding quality score, the network damage score and the delay jitter score are integrated, and the dynamic score of the current video frame is calculated by using the following formula: Q= Q _E × PL-DF.

In order to make the evaluation results more stable, time window smoothing technology can be used. Set the time window length T _q , the end point of the time window is the current video frame, and the start point of the time window is the first video frame whose time interval with the current video frame is not greater than T _q . The average value of the dynamic scores of all video frames in the time window is calculated as the final quality monitoring score of the video stream by the following formula:

Based on the implementation process of the quality monitoring score of the video stream described above, this embodiment provides a preferred implementation mode, that is, according to the source coding quality score and the network damage score, determining the quality monitoring score of the video stream includes: according to the source coding quality score, loss Packet damage score, delay jitter score, calculate the dynamic score of the current video frame; set the length of the second time window; wherein, the end point of the second time window is the current video frame, and the start point of the second time window is the same as the current The time interval of the video frame is not greater than the video frame of the second time window length; the dynamic score of each video frame in the second time window is averaged to obtain the quality monitoring score of the video stream. Through this preferred implementation manner, the accuracy and stability of video stream quality monitoring results are improved.

Corresponding to the video stream quality monitoring method introduced in the above embodiments, this embodiment provides a video stream quality monitoring device, which can be set on the network side to implement the above embodiments. Fig. 7 is a structural block diagram of a video stream quality monitoring device according to an embodiment of the present invention. As shown in Fig. 7, the device includes: an information acquisition module 10, a source coding quality scoring module 20, a network damage scoring module 30 and a quality monitoring and scoring module 40. The structure is described in detail below.

An information acquisition module 10, configured to acquire video stream information on the network side;

The source encoding quality scoring module 20 is used to calculate the quality score of the reference frame and the encoding quality score of the current video frame according to the video stream information; according to the quality score of the reference frame, the encoding quality score of the current video frame, and the Encoding type, calculate the source encoding quality score;

The network damage scoring module 30 is used to calculate the network damage score according to the video stream information; wherein, the network damage is the video stream quality damage caused by packet loss and delay jitter;

The quality monitoring and scoring module 40 is configured to determine the quality monitoring score of the video stream according to the source coding quality score and the network damage score.

Through the above method, after the information acquisition module 10 acquires the video stream information on the network side, the source coding quality scoring module 20 and the network damage scoring module 30 calculate the source coding quality score and the network damage scoring module according to the video stream information, wherein, at the source The reference frame is used in the calculation process of the coding quality score, and the video stream quality damage caused by packet loss and delay jitter is considered in the calculation process of the network damage score. Finally, the quality monitoring and scoring module 40 is based on the above-mentioned source coding quality score and the above-mentioned network damage. Score, determine the quality monitoring score of the video stream, solve the problem of low monitoring accuracy and complicated implementation of the video stream quality monitoring method on the network side in related technologies, reflect the dynamic changes of video quality more accurately, and improve the quality of video stream The accuracy of monitoring is not high, and the complexity is not high, and it can be easily applied to the real-time monitoring environment of video quality on the network side.

Preferably, the above device may also include a network data acquisition module for intercepting video streams from the network. For the acquisition operation of the video stream information of the above-mentioned information acquisition module 10, this embodiment provides a preferred implementation mode, that is, the information acquisition module 10 includes: a video stream acquisition subunit, which is used to decode the video service network data on the network side Encapsulation and demultiplexing to obtain the video stream; the time determination subunit is used to analyze the video stream, locate the start point and end point of the current video frame and the previous video frame in the video stream; set the start point and end point of the current video frame , and the start point and end point of the previous video frame are respectively reversely mapped to the video service network data packet, and the arrival time of all network packets of the current video frame and the previous video frame are respectively obtained; the current video frame is determined according to the arrival time The frame arrival time and the last video frame arrival time; the number of bits determines the subunit, which is used to determine the number of bits of the current video frame according to the starting point of the previous video frame and the starting point of the current video frame; the encoding information determines the subunit, using To determine the real-time packet loss rate, real-time bit rate and real-time frame rate of the current video frame; analyze the encoded bit stream of the current video frame, and determine the encoding type, size information and quantization parameters of the current video frame. Through this preferred embodiment, a calculation basis is provided for the subsequent calculation of the source coding quality score and the network damage score, and the monitoring precision and accuracy of the video stream quality are mentioned. The acquisition process of the video stream information has been introduced in detail above, and will not be repeated here.

Preferably, the above-mentioned information acquisition module 10 can also have the following characteristics: the extracted video coding information includes the coded bit stream of the reference frame; the coding type of the frame (I frame or P frame); the size of the frame (width and height); The quantization parameter Q _P of the frame; the number of bits contained in each frame; the moment of arrival of the frame. The extracted network packet information includes sequence numbers and arrival times of all network packets used to encapsulate a video frame.

For the above-mentioned source coding quality scoring module 20 to calculate the quality score of the reference frame according to the video stream information, this embodiment provides a preferred implementation mode, that is, the source coding quality scoring module 20 includes a quality scoring unit of the reference frame for information, to calculate the quality score of the reference frame; wherein, the quality score unit of the reference frame includes: a reference frame judging subunit, which is used to judge whether the current video frame is a reference frame according to the encoded bit stream of the current video frame; an evaluation subunit for If it is a reference frame, the block effect evaluation operation and the blur evaluation operation are performed on the reference frame; the quality scoring subunit of the reference frame is used to carry out weighted average to the results of the block effect evaluation operation and the blur evaluation operation to obtain the reference frame Quality rating. In this preferred implementation manner, a reference frame is introduced for quality scoring, which improves the monitoring accuracy of video stream quality. The quality scoring process of the reference frame has been introduced in detail above, and will not be repeated here.

Preferably, the quality scoring unit of the above-mentioned reference frame can also have the following characteristics: for the two most obvious spatial impairments in real-time video services: block effect and blur, respectively adopt a relatively low-complexity spatial domain quality evaluation method for evaluation, comprehensive Finally, the benchmark frame quality evaluation results are obtained. While ensuring the effect, the amount of calculation does not increase much.

For the above-mentioned source coding quality scoring module 20 to calculate the coding quality scoring of the current video frame according to the video stream information, this embodiment provides a preferred implementation mode, that is, the source coding quality scoring module 20 includes a coding quality scoring unit for The flow information calculates the encoding quality score of the current video frame; wherein, the encoding quality scoring unit includes: the number of bits per pixel determines a subunit, which is used to obtain the number of bits per pixel of the current video frame according to the number of bits and size information of the current video frame ; The time and space complexity determination subunit is used to take calculation operations corresponding to the coding type of the current video frame according to the number of bits per pixel and the quantization parameter to obtain the time and space complexity; the coding quality scoring subunit is used to calculate according to The time and space complexity and quantization parameters are used to obtain the encoding quality score of the current video frame. Through this preferred implementation manner, the monitoring accuracy of video stream quality is improved. The encoding quality scoring process has been described in detail above, and will not be repeated here.

For the calculation operation of the source coding quality score of the above-mentioned source coding quality scoring module 20, this embodiment provides a preferred implementation mode, that is, the source coding quality scoring module 20 includes a source coding quality scoring unit for , the encoding quality score of the current video frame, and the encoding type of the current video frame, calculate the source encoding quality score; wherein, the source encoding quality scoring unit includes:

The first scoring subunit is used to predict the I frame within the frame when the coding type of the current video frame is a frame, and the current video frame is a reference frame, and the source coding quality score is equal to the quality score of the reference frame;

The second scoring subunit is used to predict the I frame in the coding type of the current video frame, the current video frame is not a reference frame, and there is a quality score of the reference frame, according to the quality score of the reference frame, the current video frame The encoding quality score of , and the encoding quality difference between the current video frame and the reference frame, calculate the source encoding quality score;

The 3rd scoring subunit, for the encoding type of current video frame is intra-frame prediction I frame, current video frame is not reference frame, and under the situation that does not have the quality score of reference frame, source coding quality score is equal to current video frame Coding quality rating;

The fourth scoring subunit is used to obtain the quality monitoring score of the previous video frame when the encoding type of the current video frame is a forward prediction P frame; according to the quality monitoring score of the previous video frame, the encoding of the current video frame The quality score, and the encoding quality difference between the current video frame and the previous video frame, calculate the source encoding quality score. The source coding quality scoring process has been introduced in detail above, and will not be repeated here.

Preferably, the above-mentioned source coding quality scoring unit may also have the following characteristics: judging whether to introduce an error propagation mechanism according to the frame coding type, and only P frames will be affected by error propagation. When the quality of a certain frame is damaged, other frames after this frame and in the same GoP will be affected. Although the I frame does not consider the error propagation, in order to improve the evaluation accuracy, the source coding quality evaluation is based on the reference frame.

For the network damage scoring module 30 in the above embodiment to obtain the network damage score according to the video stream information, the network damage also takes into account the impact of the delay and jitter factors on the video quality in addition to the packet loss factor. This embodiment provides a preferred implementation manner, that is, the network damage scoring module 30 includes: a first setting unit, configured to set the length of the first time window; wherein, the end point of the first time window is the current video frame, and the second The starting point of a time window is a video frame whose time interval with the current video frame is not greater than the length of the first time window; the average unit is used to calculate the real-time average according to the real-time packet loss rate of each video frame in the first time window Packet loss rate; calculate the real-time average bit rate according to the real-time bit rate of each video frame in the first time window; calculate the real-time average frame rate according to the real-time frame rate of each video frame in the first time window; packet loss The damage scoring unit is used to calculate the packet loss robustness factor according to the real-time average bit rate and the real-time average frame rate; calculate the packet loss damage score according to the packet loss robustness factor and the real-time average packet loss rate; delay jitter The scoring unit is used to calculate the delay jitter score according to the real-time average frame rate at the arrival time of the current video frame and the real-time average frame rate at the arrival time of the previous video frame; wherein, the network damage score includes packet loss damage score and delay Jitter score. Through this preferred implementation manner, the accuracy of video stream quality monitoring is improved. The network damage scoring process has been introduced in detail above, and will not be repeated here.

Preferably, the network damage scoring module 30 can also have the following characteristics: the real-time average packet loss rate, real-time average bit rate and real-time average frame rate used in the calculation are the real-time packet loss rate within a period of time before the arrival of the current video frame, The average of real-time bitrate and real-time frame rate. This can make the evaluation results more stable. The real-time packet loss rate, real-time bit rate and real-time frame rate are also calculated within a period of time before the current video frame arrives.

Preferably, the network damage scoring module 30 can also have the following characteristics: in order to calculate the real-time packet loss rate, real-time bit rate and real-time frame rate within a period of time before the arrival of the current video frame, the concept of time window can be used, and a storage time A "first in first out" queue of all video frame information in the window. The time window length is the time length for calculating the real-time packet loss rate, real-time bit rate and real-time frame rate. After the current video frame arrives, the information of the current video frame is added to the queue, and all the information of the video frame at the head of the queue that differs from the arrival time of the current video frame by more than the length of the time window is all deleted. Queues can also be established for the real-time packet loss rate and real-time frame rate, which are used to calculate the real-time average packet loss rate and real-time average frame rate.

Based on the implementation process of the video stream quality monitoring and scoring of the quality monitoring and scoring module 40, this embodiment provides a preferred implementation mode, that is, the quality monitoring and scoring module 40 includes: a dynamic scoring unit for scoring according to the source coding quality, packet loss damage Score, delay jitter score, calculate the dynamic score of the current video frame;

The second setting unit is used to set the length of the second time window; wherein, the end point of the second time window is the current video frame, and the start point of the second time window is that the time interval with the current video frame is not greater than the second time The video frame of the window length; the quality monitoring and scoring unit, which is used to average the dynamic scores of each video frame in the second time window to obtain the quality monitoring score of the video stream. Through this preferred implementation manner, the accuracy and stability of video stream quality monitoring results are improved.

Preferably, the quality monitoring and scoring module 40 is used to synthesize the results of source coding quality evaluation and network damage evaluation, calculate the dynamic scoring result of the current video frame, and use the time window to smooth the dynamic scoring of the current video frame, and output a more stable Dynamic scoring results.

Figure 8 is a schematic structural diagram of a video stream quality monitoring device according to an embodiment of the present invention, as shown in Figure 8, the network data acquisition module is responsible for acquiring network data on the network side, and separates the video service network to be evaluated according to the protocol and port number data. Then, the information acquisition module extracts evaluation-related information from the video service network data, including video coding information and network packet information; the source coding quality scoring module and the network damage scoring module perform sub-item evaluation based on the above information. The source coding quality scoring module is further subdivided into three units, the quality scoring unit of the reference frame, the coding quality scoring unit, and the source coding quality scoring unit; the network damage scoring module performs packet loss damage evaluation and delay jitter damage evaluation; quality monitoring and scoring The module synthesizes the results of source coding quality evaluation and network damage evaluation to give a final dynamic score.

From the above description, it can be seen that the embodiments of the present invention obtain real-time video streams of specified services and sessions on the network side, analyze the real-time video streams, and score the quality of source codes. And evaluate the video quality damage caused by error propagation factors, and evaluate the video quality damage caused by network performance, and finally get the dynamic score of the real-time video stream quality according to the evaluation results. In the embodiment of the present invention, for the video stream acquired by the network side, each time the coded bit stream of a video frame is obtained by parsing, the service quality dynamic score at the arrival time of the frame can be given. The service quality dynamic score at the arrival time of a certain video frame not only considers the situation of the current frame, but also considers the situation of other frames in a period of time before the current frame, so that the score is both dynamic and stable.

The embodiment of the present invention considers the influence of source video encoding and network performance on video quality at the same time, innovatively uses reference frames in source video encoding evaluation, and considers quantization parameters, time and space complexity, and frame rate when measuring encoding quality. Encoding type, error propagation and other factors. In addition to packet loss, network performance also considers the quality damage caused by delay and jitter, which improves the evaluation accuracy. In addition, the calculation of the embodiment of the present invention is very simple, and the dynamic quality of the real-time service can also be monitored, which can be well applied to the online monitoring of the real-time video service quality on the network side.

Although preferred embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, and therefore, the scope of the present invention should not be limited to the above-described embodiments.

Claims (12)

1. a video flow quality monitoring method, it is characterised in that including:

Obtain the Video stream information of network side；

According to described Video stream information, calculate the quality score of reference frame, and the coding matter of current video frame Amount scoring；Quality score according to described reference frame, described current video frame coding quality scoring and The type of coding of described current video frame, calculates source code quality score；

According to described Video stream information, calculate and obtain network harm scoring；Wherein, described network harm be by The video flow quality damage that packet loss and delay variation cause；

Mark according to described source code quality score and described network harm, determine the quality prison of described video flowing Test and appraisal point；

Wherein, according to described Video stream information, calculate network harm scoring and include:

Set very first time length of window；Wherein, the end point of described very first time window is described to work as forward sight Frequently frame, the starting point of described very first time window is described in the time interval with described current video frame is not more than The frame of video of very first time length of window；

According to the real-time packet loss of each frame of video in described very first time window, it is calculated average Packet loss；According to the real time of each frame of video in described very first time window, it is calculated in real time Mean bit rate；According to the real time frame rate of each frame of video in described very first time window, it is calculated reality Shi Pingjun frame per second；

According to described real-time mean bit rate and average described frame per second, it is calculated the packet loss robustness factor； According to the described packet loss robustness factor and described real-time average packet loss ratio, it is calculated packet loss Injury score；

According to the described average frame per second of described current video frame due in, and a upper frame of video arrives The average frame per second in moment, is calculated delay variation scoring；Wherein, described network harm scoring includes Described packet loss Injury score and the scoring of described delay variation.

2. the method for claim 1, it is characterised in that obtain the Video stream information bag of network side Include:

The video traffic network data of network side are decapsulated and demultiplex, obtains video flowing；

Resolve described video flowing, location current video frame and upper frame of video starting point in described video flowing And end point；By starting point and the end point of current video frame, and the starting point of a described upper frame of video and End point, back mapping is to described video traffic network packet respectively, respectively obtains described current video frame Due in the all-network bag of a upper frame of video；Current video frame is determined respectively according to this due in Due in and a upper frame of video due in；

End point according to a described upper frame of video and the end point of described current video frame, determine described working as The bit number of front frame of video；

Determine real-time packet loss, real time and the real time frame rate of described current video frame；Resolve described The coded bit stream of current video frame, determines the type of coding of described current video frame, dimension information and amount Change parameter.

3. method as claimed in claim 2, it is characterised in that according to described Video stream information, calculate The quality score of reference frame includes:

Coded bit stream according to described current video frame judges frame on the basis of described current video frame is whether；

If reference frame, then described reference frame is performed blocking effect evaluation operation and fuzzy evaluation operation；

It is weighted putting down to the result of described blocking effect evaluation operation and the result of described fuzzy evaluation operation All, the quality score of described reference frame is obtained.

4. method as claimed in claim 2, it is characterised in that according to described Video stream information, calculate The coding quality scoring of current video frame includes:

Bit number according to described current video frame and dimension information, obtain every pixel of described current video frame Bit number；

According to described every pixel bit number, and described quantization parameter, take and the volume of described current video frame Code type calculates operation accordingly, obtains Time & Space Complexity；

According to described Time & Space Complexity, and described quantization parameter, obtain described current video frame Coding quality is marked.

5. method as claimed in claim 2, it is characterised in that according to the quality score of described reference frame, The coding quality scoring of described current video frame and the type of coding of described current video frame, the source that calculates is compiled Code quality score includes:

If the type of coding of described current video frame is infra-frame prediction I frame, and described current video frame is base Quasi-frame, the most described source code quality score is equal to the quality score of described reference frame；

If the type of coding of described current video frame is infra-frame prediction I frame, described current video frame is not base Quasi-frame, and there is the quality score of described reference frame, then according to the quality score of described reference frame, described work as The coding quality scoring of front frame of video, and the coding quality difference between described current video frame and reference frame, Calculate described source code quality score；

If the type of coding of described current video frame is infra-frame prediction I frame, described current video frame is not base Quasi-frame, and there is not the quality score of described reference frame, the most described source code quality score equal to described currently The coding quality scoring of frame of video；

If the type of coding of described current video frame is forward prediction P frame, then obtain the matter of a upper frame of video Amount monitoring scoring；Quality-monitoring scoring according to a described upper frame of video, the coding matter of described current video frame Coding quality difference between amount scoring and described current video frame and a described upper frame of video, calculates institute State source code quality score.

6. the method for claim 1, it is characterised in that according to described source code quality score and Described network harm is marked, and determines that the quality-monitoring scoring of described video flowing includes:

According to described source code quality score, described packet loss Injury score, the scoring of described delay variation, calculate Obtain the dynamic grading of current video frame；

Set the second time window length；Wherein, the end point of described second time window is described to work as forward sight Frequently frame, the starting point of described second time window is described in the time interval with described current video frame is not more than The frame of video of the second time window length；

The dynamic grading of each frame of video in described second time window is averaged, obtains described video The quality-monitoring scoring of stream.

7. a video flow quality monitoring device, it is characterised in that described device includes:

Data obtaining module, for obtaining the Video stream information of network side；

Source code quality score module, for according to described Video stream information, calculating the quality score of reference frame, And the coding quality scoring of current video frame；Quality score according to described reference frame, described current video The coding quality scoring of frame and the type of coding of described current video frame, calculate source code quality score；

Network harm grading module, for according to described Video stream information, calculates network harm scoring；Wherein, Described network harm is the video flow quality damage caused by packet loss and delay variation；

Quality-monitoring grading module, for marking according to described source code quality score and described network harm, Determine the quality-monitoring scoring of described video flowing；

Wherein, described network harm grading module includes:

First setup unit, is used for setting very first time length of window；Wherein, described very first time window End point is described current video frame, and the starting point of described very first time window is and described current video frame Time interval is not more than the frame of video of described very first time length of window；

Averaging unit, for the real-time packet loss according to each frame of video in described very first time window, meter Calculation obtains real-time average packet loss ratio；According to the real time of each frame of video in described very first time window, It is calculated real-time mean bit rate；According to the real time frame rate of each frame of video in described very first time window, It is calculated average frame per second；

Packet loss Injury score unit, is used for according to described real-time mean bit rate and average described frame per second, It is calculated the packet loss robustness factor；According to the described packet loss robustness factor and described real-time average packet loss ratio, It is calculated packet loss Injury score；

Delay variation scoring unit, for the described real-time average frame according to described current video frame due in Rate, and the average frame per second of a upper frame of video due in, be calculated delay variation scoring；Wherein, Described network harm scoring includes described packet loss Injury score and the scoring of described delay variation.

8. device as claimed in claim 7, it is characterised in that described data obtaining module includes:

Video flowing obtains subelement, for the video traffic network data of network side are decapsulated and demultiplexed With, obtain video flowing；

Moment determines subelement, is used for resolving described video flowing, and location current video frame and a upper frame of video exist Starting point in described video flowing and end point；By starting point and the end point of current video frame and described The starting point of a upper frame of video and end point, back mapping is to described video traffic network packet respectively, point Do not obtain the due in of the all-network bag of described current video frame and a upper frame of video；During according to this arrival Carve and determine current video frame due in and a upper frame of video due in respectively；

Bit number determines subelement, for the end point according to a described upper frame of video and described current video The end point of frame, determines the bit number of described current video frame；

Coding information determines subelement, for determining the real-time packet loss of described current video frame, real-time bit Rate and real time frame rate；Resolve the coded bit stream of described current video frame, determine described current video frame Type of coding, dimension information and quantization parameter.

9. device as claimed in claim 8, it is characterised in that described source code quality score module bag Include the quality score unit of reference frame, for according to described Video stream information, calculating the quality score of reference frame； Wherein, the quality score unit of described reference frame includes:

Reference frame judgment sub-unit, for according to the coded bit stream of described current video frame judge described currently Frame of video whether on the basis of frame；

Evaluate subelement, for if reference frame, then described reference frame is performed blocking effect evaluation operation with And fuzzy evaluation operation；

The quality score subelement of reference frame, for the result operating described blocking effect evaluation and described mould The result sticking with paste evaluation operation is weighted averagely, obtaining the quality score of described reference frame.

10. device as claimed in claim 8, it is characterised in that described source code quality score module bag Include coding quality scoring unit, for according to described Video stream information, calculating the coding quality of current video frame Scoring；Wherein, described coding quality scoring unit includes:

Every pixel bit number determines subelement, for the bit number according to described current video frame and dimension information, Obtain every pixel bit number of described current video frame；

Time & Space Complexity determines subelement, is used for according to described every pixel bit number, and described amount Change parameter, take corresponding with the type of coding of described current video frame to calculate operation, obtain time and space Complexity；

Coding quality scoring subelement, is used for according to described Time & Space Complexity, and described quantization is joined Number, obtains the coding quality scoring of described current video frame.

11. devices as claimed in claim 8, it is characterised in that described source code quality score module bag Include source code quality score unit, for according to the quality score of described reference frame, described current video frame Coding quality scoring and the type of coding of described current video frame, calculate source code quality score；Wherein, Described source code quality score unit includes:

First scoring subelement, is infra-frame prediction I frame for the type of coding at described current video frame, and In the case of described current video frame is reference frame, described source code quality score is equal to the matter of described reference frame Amount scoring；

Second scoring subelement, is infra-frame prediction I frame for the type of coding at described current video frame, institute Stating current video frame is not reference frame, and in the case of there is the quality score of described reference frame, according to described The quality score of reference frame, described current video frame coding quality scoring, and described current video frame with Coding quality difference between reference frame, calculates described source code quality score；

3rd scoring subelement, is infra-frame prediction I frame for the type of coding at described current video frame, institute Stating current video frame is not reference frame, and in the case of there is not the quality score of described reference frame, described source Coding quality scoring is equal to the coding quality scoring of described current video frame；

4th scoring subelement, for the feelings that type of coding is forward prediction P frame at described current video frame Under condition, obtain the quality-monitoring scoring of a upper frame of video；According to a described upper frame of video quality-monitoring scoring, Between coding quality scoring and described current video frame and a described upper frame of video of described current video frame Coding quality difference, calculate described source code quality score.

12. devices as claimed in claim 7, it is characterised in that described quality-monitoring grading module includes:

Dynamic grading unit, for according to described source code quality score, described packet loss Injury score, described Delay variation is marked, and is calculated the dynamic grading of current video frame；

Second setup unit, for setting the second time window length；Wherein, described second time window End point is described current video frame, and the starting point of described second time window is and described current video frame Time interval is not more than the frame of video of described second time window length；

Quality-monitoring scoring unit, for the dynamic grading to each frame of video in described second time window Average, obtain the quality-monitoring scoring of described video flowing.