CN112291566B - H.264 video coding method, device and storage equipment - Google Patents

Abstract

The invention discloses an H.264 video coding method, a device, a chip, a storage device and an electronic device, wherein the method comprises the following steps: for the P frame, if the position of the current frame is before the set frame position, setting the first mark of the current frame as a reference frame; if the position of the current frame is the set frame position, setting a first mark of the current frame as a non-reference frame; if the position of the current frame is at the set frame position, judging whether the frame interval between the current frame and the nearest non-reference frame is the set frame interval, if so, setting the first identification of the current frame as the non-reference frame, and if not, setting the first identification of the current frame as the reference frame; encoding the current frame to output a compressed code stream; if the code rate is greater than the first code rate threshold, the frame interval is increased to reduce the number of non-reference frames. The invention can give consideration to the anti-interference capability and the coding quality of the video frame sequence.

Description

H.264 video encoding method, device and storage device

Technical Field

The present invention relates to the field of image coding, and in particular to an H.264 video coding method, device, chip, storage device and electronic device.

Background technique

With the rapid development of wireless networks, mobile communications have an increasingly greater impact on people's lives. The rapid development of wireless transmission has led to higher requirements for services. Some special services in special occasions require high stability, low latency, and uninterrupted transmission, such as mobile video conferencing and video surveillance. Compared with traditional wired networks, wireless networks have the characteristics of instability and high mobility. During wireless network transmission, interference, signal attenuation, switching and other situations that damage transmission are prone to occur. These will lead to low transmission efficiency, network congestion, packet loss and packet damage. In this way, video decoding at the receiving end will have problems.

Since the H.264 video compression standard was officially announced in March 2003, it has been widely used in consumer electronics such as real-time video surveillance, low-latency video conferencing, and online video on demand. The reason why H.264 has excellent compression performance is due to the introduction of a variety of newly introduced technologies, such as intra-frame prediction, inter-frame prediction, multiple reference frames, inter-frame variable-size block motion estimation, deblocking filtering, and context-adaptive entropy coding technology. Among them, the P frame used as inter-frame prediction has a greater impact on the reliability of wireless network transmission, especially when the IDR (immediate refresh frame) interval is large. Once a P frame is lost in wireless transmission, the decoding of all P frames after the lost P frame in a GOP (group of pictures, specifically the interval between two IDR frames) will be affected.

At present, most H.264 coding schemes use only one forward reference frame for inter-frame prediction. One of the coding techniques is that each P frame uses the previous frame as a reference frame. Specifically, when encoding the group of pictures GOP (the group of pictures defined by H.264, specifically the interval between two IDR frames), each P frame after the first frame (I frame) uses the previous frame as a reference frame. Therefore, the P frame reference frame used as the inter-frame prediction has a greater impact on the reliability of wireless network transmission, especially when the IDR (immediate refresh frame) interval is large. Once a P frame is lost during transmission (for example, when the wireless network transmission bandwidth is tight), the decoding of all P frames in the GOP after the lost P frame cannot be decoded normally, and the decoding end screen will be stuck or abnormal.

H.264 can use up to 16 reference frames for inter-frame prediction. The use of multiple reference frames can reduce decoding problems caused by frame loss during network transmission. However, the use of multiple reference frames in the development of consumer electronics-grade video chips will increase encoding complexity, memory and bandwidth consumption, and chip costs.

Another technology, such as the prior art CN104754345A, introduces multiple non-reference frames P frames into an image group. This technology can objectively alleviate the problems in the aforementioned coding technology. However, research has found that this technology is prone to reduce coding quality. In addition, in this prior art, the encoded frame data is directly sent to the external cache. When the encoded frame data needs to be used as a reference frame, it needs to be read from the off-chip cache through the reference frame reading module, which not only reduces the cache utilization, but also has low read and write efficiency.

Summary of the invention

Research on the prior art CN104754345A found that when the number of consecutive non-reference frames is too large, the accuracy of inter-frame prediction will inevitably be reduced, that is, the greater the difference between the frame to be encoded and the reference frame, the more bits are needed to represent this difference, which will lead to an increase in the real-time bit rate. When the encoder needs to control the bit rate (bit rate) of the output bit stream below a certain bit rate, the quantization parameter qp will increase significantly, causing the encoding quality to drop sharply.

Based on the above situation, the main purpose of the present invention is to provide an H.264 video encoding method, apparatus, chip, storage device and electronic device, which detects the bit rate of the encoded output stream in real time, and reduces the frame interval between non-reference frames when the bit rate is greater than the bit rate threshold, so as to reduce the number of non-reference frames. Under the premise of the existence of non-reference frames and taking into account the quality of video encoding, the anti-interference ability of the video in a wireless network transmission environment and the visual imaging quality of decoding at the receiving end are improved.

To achieve the above purpose, the technical solution adopted by the present invention is as follows:

Referring to FIG. 1 , an H.264 video encoding method includes the following steps: S110, determining the type of the current frame, if the type of the current frame is an I frame, executing steps S140-S150, if the type of the current frame is a P frame, executing steps S120-S150; S120, determining the position of the current frame, if the position of the current frame is before a set frame position, setting the first identifier of the current frame as a reference frame, and executing steps S140-S150; if the position of the current frame is the set frame position, setting the first identifier of the current frame as a non-reference frame, and executing steps S140-S150; if the position of the current frame is after the set frame position, executing step S13 0-S150; wherein, the first identifier is used to indicate whether the current frame is a reference frame; S130, determining whether the frame spacing between the current frame and the nearest non-reference frame is a set frame spacing, if so, setting the first identifier of the current frame to a non-reference frame, if otherwise, setting the first identifier of the current frame to a reference frame; S140, encoding the current frame according to its first identifier, and outputting a compressed code stream; S150, detecting the bit rate of the compressed code stream in real time, determining whether the bit rate is greater than a first bit rate threshold, if otherwise, re-executing steps S110-S150, if so, increasing the frame spacing to reduce the number of non-reference frames, and re-executing steps S110-S150 until encoding is completed.

Preferably, in step S150, the frame interval is reduced by a value of 1 each time.

Preferably, referring to FIG6, before step S110, the video encoding method further includes the following steps: S100, providing two blocks of encoding cache addresses for storing reference frame data generated by the encoding process in the buffer for encoding; the step S110 further includes the following steps: if the type of the current frame is an I frame, setting the first identifier of the current frame as a reference frame and writing it into the frame header of the current frame, initializing the write cache address of the current frame to any one of the two encoding cache addresses for storing the reference frame data obtained by encoding the current frame; the step S130 further includes the following steps: S131, writing the first identifier of the current frame into the frame header of the current frame; S132, judging whether the previous frame of the current frame is a reference frame, and if so, executing Perform step S133, if not, perform step S134; S133, assign the read cache address of the current frame to the write cache address of the previous frame of the current frame, so as to read the reference frame data encoded by the previous frame of the current frame during encoding, and assign the write cache address to the other of the two encoding cache addresses except the write cache address of the previous frame of the current frame, so as to store the reference frame data encoded by the current frame when the current frame is a reference frame; S134, keep the read cache address of the current frame consistent with the read cache address of the previous frame of the current frame, so as to read the reference frame data encoded by the previous frame of the current frame during encoding, and keep the write cache address consistent with the write cache address of the previous frame of the current frame, so as to store the reference frame data encoded by the current frame when the current frame is a reference frame.

Preferably, in steps S110 and S130, the first identifier of the current frame is written into a padding network abstraction layer unit in a frame header of the current frame.

Referring to FIG. 7 , the present invention further provides an H.264 video encoding method, comprising the following steps: S210, determining the type of the current frame, if the type of the current frame is an I frame, executing steps S240-S250, if the type of the current frame is a P frame, executing steps S220-S250; S220, determining the position of the current frame, if the position of the current frame is before the set frame position, setting the first identifier of the current frame as a reference frame, and executing steps S240-S250; if the position of the current frame is the set frame position, setting the first identifier of the current frame as a non-reference frame, and executing steps S240-S250; if the position of the current frame is after the set frame position, executing steps S230-S250; wherein the first identifier is used to indicate whether the current frame is a reference frame; S230, determining whether the current frame is the longest Whether the frame spacing of the nearest non-reference frame is the set frame spacing, if so, the first identifier of the current frame is set to a non-reference frame, if not, the first identifier of the current frame is set to a reference frame; S240, encode the current frame according to its first identifier, and output a compressed code stream; S250, detect the code rate of the compressed code stream in real time, and judge the size of the code rate, if: the second code rate threshold ≤ the code rate ≤ the third code rate threshold, then do not adjust the frame spacing size, and re-execute steps S210-S250 until the encoding is completed; if: the code rate > the third code rate threshold, then increase the frame spacing to reduce the number of non-reference frames, and re-execute steps S210-S250 until the encoding is completed; if: the code rate < the second code rate threshold, then reduce the frame spacing to increase the number of non-reference frames, and re-execute steps S210-S250 until the encoding is completed.

Preferably, in step S250, the frame interval is reduced by a value of 1 each time and/or the frame interval is increased by a value of 1 each time.

Preferably, referring to FIG8, before step S210, the video encoding method further includes the following steps: S200, providing two blocks of encoding cache addresses for storing reference frame data generated by the encoding process in the buffer for encoding; the step S210 further includes the following steps: if the type of the current frame is an I frame, setting the first identifier of the current frame as a reference frame and writing it into the frame header of the current frame, initializing the write cache address of the current frame to any one of the two encoding cache addresses for storing the reference frame data obtained by encoding the current frame; the step S230 further includes the following steps: S231, writing the first identifier of the current frame into the frame header of the current frame; S232, judging whether the previous frame of the current frame is a reference frame, and if so, executing Perform step S233, if not, perform step S234; S233, assign the read cache address of the current frame to the write cache address of the previous frame of the current frame, so as to read the reference frame data encoded by the previous frame of the current frame during encoding, and assign the write cache address to the other of the two blocks of encoding cache addresses except the write cache address of the previous frame of the current frame, so as to store the reference frame data encoded by the current frame when the current frame is a reference frame; S234, keep the read cache address of the current frame consistent with the read cache address of the previous frame of the current frame, so as to read the reference frame data encoded by the previous frame of the current frame during encoding, and keep the write cache address consistent with the write cache address of the previous frame of the current frame, so as to store the reference frame data encoded by the current frame when the current frame is a reference frame.

Preferably, in steps S210 and S230, the first identifier of the current frame is written into a padding network abstraction layer unit in a frame header of the current frame.

The present invention also provides an H.264 video encoding device, comprising: a first processing unit, used to determine the type of the current frame, if the type of the current frame is an I frame, the first four processing units and the first five processing units are triggered to work, if the type of the current frame is a P frame, the first two processing units to the first five processing units are triggered to work; the first two processing units, used to determine the position of the current frame, if the position of the current frame is before the set frame position, the first identifier of the current frame is set as a reference frame, and the first four processing units and the first five processing units are triggered to work; if the position of the current frame is the set frame position, the first identifier of the current frame is set as a non-reference frame, and the first four processing units and the first five processing units are triggered to work; if the position of the current frame is after the set frame position, the first three processing units are triggered. The first to fifth processing units operate; wherein the first identifier is used to indicate whether the current frame is a reference frame; the first to third processing units are used to determine whether the frame spacing between the current frame and the nearest non-reference frame is a set frame spacing, and if so, the first identifier of the current frame is set to a non-reference frame, and if otherwise, the first identifier of the current frame is set to a reference frame; the first to fourth processing units are used to encode the current frame according to its first identifier and output a compressed code stream; the first to fifth processing units are used to detect the bit rate of the compressed code stream in real time, determine whether the bit rate is greater than a first bit rate threshold, and if otherwise, re-trigger the first to fifth processing units to operate, and if so, increase the frame spacing to reduce the number of non-reference frames, and re-trigger the first to fifth processing units to operate until the encoding is completed.

Preferably, the first five processing units are used to reduce the frame interval by a value of 1 each time.

Preferably, the H.264 video encoding device further includes a first six processing unit and a first seven processing unit, wherein the first six processing unit is used to provide two blocks of encoding cache addresses for storing reference frame data generated by the encoding process in the buffer for encoding; the first one processing unit is also used to: if the type of the current frame is an I frame, set the first identifier of the current frame as a reference frame and write it into the frame header of the current frame, and initialize the write cache address of the current frame to any one of the two encoding cache addresses for storing the reference frame data obtained by encoding the current frame; the first seven processing unit is used to: write the first identifier of the current frame into the frame header of the current frame; and determine whether the previous frame of the current frame is a reference frame. frame, if so: assigning the read cache address of the current frame to the write cache address of the frame before the current frame, so as to read the reference frame data encoded by the frame before the current frame during encoding, and assigning the write cache address to the other of the two blocks of encoding cache addresses except the write cache address of the frame before the current frame, so as to store the reference frame data encoded by the current frame when the current frame is a reference frame; if not, then: keeping the read cache address of the current frame consistent with the read cache address of the frame before the current frame, so as to read the reference frame data encoded by the reference frame before the current frame during encoding, and keeping the write cache address consistent with the write cache address of the frame before the current frame, so as to store the reference frame data encoded by the current frame when the current frame is a reference frame.

Preferably, the first-first processing unit and the first-third processing unit are further used to write the first identifier of the current frame into a padding network abstraction layer unit in a frame header of the current frame.

The present invention also provides an H.264 video encoding device, comprising: a second-first processing unit, used to determine the type of the current frame, if the type of the current frame is an I frame, the second-fourth processing unit and the second-fifth processing unit are triggered to work, if the type of the current frame is a P frame, the steps of triggering the second-second processing unit to the second-fifth processing unit to work; the second-second processing unit, used to determine the position of the current frame, if the position of the current frame is before the set frame position, the first identifier of the current frame is set as a reference frame, and the second-fourth processing unit and the second-fifth processing unit are triggered to work; if the position of the current frame is the set frame position, the first identifier of the current frame is set as a non-reference frame, and the second-fourth processing unit and the second-fifth processing unit are triggered to work; if the position of the current frame is after the set frame position, the second-third processing unit to the second-fifth processing unit are triggered to work; wherein, the first identifier is used to indicate whether the current frame is a reference frame; the second-third processing unit is used to determine whether the frame spacing between the current frame and the nearest non-reference frame is the set frame spacing, if so, set the first identifier of the current frame to the non-reference frame, if not, set the first identifier of the current frame to the reference frame; the second four processing units are used to encode the current frame according to its first identifier and output a compressed code stream; the second five processing units are used to detect the code rate of the compressed code stream in real time and determine the size of the code rate. If: the second code rate threshold ≤ the code rate ≤ the third code rate threshold, the frame spacing size is not adjusted, and the second one to the second five processing units are re-triggered to work; if: the code rate is greater than the third code rate threshold, the frame spacing is increased to reduce the number of non-reference frames, and the second one to the second five processing units are re-triggered to work until the encoding is completed; if: the code rate is less than the second code rate threshold, the frame spacing is reduced to increase the number of non-reference frames, and the second one to the second five processing units are re-triggered to work until the encoding is completed.

Preferably, the second fifth processing unit is used to reduce the frame interval by a value of 1 each time and/or increase the frame interval by a value of 1 each time.

Preferably, the H.264 video encoding device also includes: a second eighth processing unit and a second ninth processing unit, which are used to provide two blocks of encoding cache addresses for storing reference frame data generated by the encoding process in the buffer for encoding; the second first processing unit is also used to set the first identifier of the current frame as a reference frame and write it into the frame header of the current frame if the type of the current frame is an I frame, and initialize the write cache address of the current frame to any one of the two encoding cache addresses for storing the reference frame data obtained by encoding the current frame; the second ninth processing unit is used to write the first identifier of the current frame into the frame header of the current frame; write the first identifier of the current frame into the frame header of the current frame; and judge whether the current frame Whether the previous frame is a reference frame, if so: assigning the read cache address of the current frame to the write cache address of the previous frame of the current frame, so as to read the reference frame data encoded by the previous frame of the current frame during encoding, and assigning the write cache address to the other of the two encoding cache addresses except the write cache address of the previous frame of the current frame, so as to store the reference frame data encoded by the current frame when the current frame is a reference frame; if not, then: keeping the read cache address of the current frame consistent with the read cache address of the previous frame of the current frame, so as to read the reference frame data encoded by the previous frame of the current frame during encoding, and keeping the write cache address consistent with the write cache address of the previous frame of the current frame, so as to store the reference frame data encoded by the current frame when the current frame is a reference frame.

Preferably, the second-first processing unit and the second-third processing unit are further used to write the first identifier of the current frame into a padding network abstraction layer unit in a frame header of the current frame.

The present invention also provides a coding chip, characterized in that it is used to execute any of the above-mentioned coding methods.

The present invention also provides a storage medium storing a program, wherein the program is executed by a processor to perform any of the above-mentioned encoding methods.

The present invention also provides a wireless communication device, comprising the encoding chip.

The present invention also provides an electronic device, comprising the encoding chip.

【Beneficial Effects】

The present invention detects the bit rate of the encoded output code stream in real time, and reduces the frame interval between non-reference frames when the bit rate is greater than the bit rate threshold to reduce the number of non-reference frames, thereby ensuring a certain encoding quality. At the same time, it has the ability to generate non-reference frames in a video frame sequence, thereby taking into account the anti-interference ability and encoding quality of the video sequence in wireless transmission.

In addition, in a more optimal solution, through the solution of the present invention, in the process of encoding the group of pictures GOP, a certain algorithm is used to control whether each encoded frame generates a reference frame, thereby distinguishing between reference frames and non-reference frames. Since non-reference frames do not generate reconstructed frames, without increasing the memory too much, a certain method is used to reasonably configure the two-frame cache for reading and writing reference frames, so that the forward reference frame of each encoded P frame is not necessarily the previous frame immediately adjacent to itself, thereby achieving the effect of multi-reference frame encoding, while improving the utilization rate of the cache address and the read-write efficiency.

Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions.

BRIEF DESCRIPTION OF THE DRAWINGS

The preferred embodiments according to the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is a flow chart of an H.264 video encoding method according to a preferred embodiment of the present invention.

FIG. 2 is a schematic diagram of a video frame sequence to be encoded according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of a coded video frame sequence according to an embodiment of the present invention.

FIG. 4 is a schematic diagram of a buffer according to an embodiment of the present invention.

FIG. 5 is a schematic diagram of a coded video frame sequence according to another embodiment of the present invention.

FIG. 6 is a partial flow chart of an H.264 video encoding method according to a preferred embodiment of the present invention.

FIG. 7 is a flow chart of an H.264 video encoding method according to another preferred embodiment of the present invention.

FIG. 8 is a partial flow chart of an H.264 video encoding method according to a preferred embodiment of the present invention.

Detailed ways

FIG. 1 is an H.264 video encoding method according to an embodiment of the present invention. The encoding method is implemented by an encoding end, such as an electronic device, for example, a mobile electronic device, a camera device, etc. The encoding method includes the following steps.

S100, providing two encoding buffer addresses for storing reference frame data generated in the encoding process in the encoding end buffer for encoding.

Before encoding the group of pictures GOP, it is necessary to apply for two encoding cache addresses (such as buffer1 and buffer2, as shown in FIG4 ) in the cache to cache the reference frame data generated during the encoding process. It should be pointed out that the reference frame data generated during the encoding process referred to in the present invention refers to the data reconstructed after operations such as transformation, quantization, inverse quantization, inverse transformation and filtering, while the non-reference frame is no longer subjected to inverse quantization, inverse transformation and filtering reconstruction operations after transformation, quantization and other operations during the encoding process, and is directly entropy encoded to form a compressed code stream, so that no reconstructed reference frame data is formed. In the prior art, due to the use of poor reference frame strategies and unoptimized cache utilization strategies, it is often necessary to apply for multiple encoding cache addresses in the cache, thereby occupying large hardware resources and having low read and write efficiency. In the present invention, only two encoding cache addresses are needed to alternately cache the reference frame data generated by encoding, and the read and write efficiency of the reference frame data will be improved in conjunction with the read and write strategies described in detail below.

S110, initialization settings before encoding.

According to the H.264 standard, the frame data after encoding each frame includes two parts: a frame header and a frame body. For an I frame, the frame header includes four parts in sequence: a sequence parameter set SPS, a picture parameter set PPS, a padding network abstraction layer unit (NALU, whose identifier nal_unit_type = 12) and a slice header, wherein the padding NALU is reserved and can be used according to actual coding needs; for a P frame, the frame header includes two parts in sequence: a padding network abstraction layer unit NALU and a slice header, wherein the padding NALU is reserved and can be used according to actual coding needs. The type information of the frame is included in the slice header, such as whether it is an I frame or a P frame. However, in the present invention, a padding NALU is required in both I and P frames to carry the first identifier to be involved below, and the first identifier is used to indicate whether the current frame is a reference frame.

In order to deal with the situation where some frames are useless or missing, it is possible to allow some frames to be missing during the encoding process. Therefore, the sequence parameter set SPS in the frame header can be set so that the frame number (that is, the syntactic element frame_num, indicating the encoding order of the frame) can be discontinuous. The corresponding syntax element configuration is as follows: gaps_in_frame_num_value_allowed_flag=1.

The definition of frame interval (specifically, the number of frames between adjacent non-reference frames) refers to the number of frames between two adjacent non-reference frames in each image group GOP. By adjusting the size of the frame interval, the number of P frames that serve as non-reference frames in each image group can be adjusted, thereby adjusting the real-time bit rate and encoding quality.

S120, determine the type of the current frame of the image group. If the type of the current frame is an I frame, execute step S130. If the type of the current frame is a P frame, execute steps S140-S190. In the present invention, each image group includes an I frame (i.e., the first frame) and multiple P frames after the I frame. Prior to this step, the type of each frame in the image group has been determined by the algorithm specified in the H.264 standard, for example, whether a frame is an I frame or a P frame. Since this can adopt the existing frame type decision algorithm, it will not be repeated here.

S130, set the first identifier of the current frame (I frame in this case) as the reference frame and write it into the frame header of the current frame, initialize the write cache address of the current frame to any one of the two encoding cache addresses, encode the current frame to obtain reference frame data and write the reference frame data into any one of the encoding cache addresses.

According to the H.264 standard, an I frame is a reference frame. In addition to setting the frame type in the slice header of the frame header to indicate that it is an I frame, the present invention also needs to set the first identifier no_ref_flag of the I frame as a reference frame, and write the first identifier no_ref_flag of the I frame into the padding NALU of the frame header of the I frame, so that the decoding end receives the frame header and obtains the first identifier no_ref_flag and decodes it; in this embodiment, the first identifier no_ref_flag is used to indicate whether the current frame is a non-reference frame, a value of 1 indicates that the current frame is a non-reference frame, and a value of 0 indicates that the current frame is a reference frame.

The write buffer address of the I frame is initialized to any one of the two blocks of the code buffer address, for example, the write buffer address of the I frame is initialized to buffer1.

Then the hardware encoding circuit starts encoding the I frame (the I frame can be divided into one or more slices for separate encoding). Since the I frame in the image group does not need to refer to other frames for encoding, but adopts intra-frame encoding, therefore, in the process of encoding the I frame, the hardware encoding circuit needs to read the I frame data to be encoded from the address of the buffer dedicated to caching the frame data to be encoded outside the above two encoding buffer addresses, and then encode it. After each encoding is completed, the I frame reference frame data is written into the write buffer address buffer1. The reference frame data may be written into buffer1 multiple times, but the present invention does not limit it. After the I frame is encoded, the I frame No. 0 shown in Figure 3 is obtained.

S140, determine the position of the current frame (P frame in this case). If the position of the current frame is before the set frame position, set the first identifier of the current frame as a reference frame and execute steps S160-S190; if the position of the current frame is the set frame position, set the first identifier of the current frame as a non-reference frame and execute steps S160-S190; if the position of the current frame is after the set frame position, execute steps S150-S190.

The above-mentioned set frame position can be a frame position determined by taking the I frame as a reference position, such as the 1st or 2nd frame after the I frame, etc., or a frame position determined by taking the frame where the frame interval is updated (see step S190 below for detailed updating steps) as a reference position, such as the 1st or 2nd frame after the Px frame (the frame where the encoding is when the frame interval is updated). As shown in FIG2 , the above-mentioned set frame position is the 2nd frame num_a (P frame No. 2) after the I frame, and the initial value of the frame interval is 3. If the current frame is P frame No. 1, that is, before the reference frame position num_a is set, the first identifier of P frame No. 1 is set as the reference frame, and steps S160-S190 are executed.

If the current frame is P frame No. 2, that is, the current frame is the set frame position num_a, and therefore the first identifier of the current frame is set to a non-reference frame (at this point, the setting of the first non-reference frame of the video frame sequence shown in FIG. 2 is completed, and the setting of the next non-reference frame according to the frame interval and the first non-reference frame will be described in detail hereinafter), and steps S160-S190 are executed. For example, in the video frame sequence shown in FIG. 5 , frame No. 0 is an I frame, and frames 1-13 are all P frames. At the beginning of encoding, the frame interval is 1, and the frame position is set to the first frame after the I frame. Therefore, P frame No. 1 generates a non-reference frame, and P frame No. 3, which is 1 frame apart from P frame No. 1, also generates a non-reference frame.

The following is an example of the above-mentioned setting of the frame position, which is a frame position determined as a reference position by the frame where the current frame is located when the frame interval is updated (the detailed updating steps are shown in step S190 below). For example, if during the encoding process of the No. 4 P frame (i.e., the current frame), it is detected that the bit rate is greater than the first bit rate threshold, the frame interval is increased to 2, and a non-reference frame is generated at the next frame of the No. 4 P frame (this frame position is the set frame position), i.e., the No. 5 P frame, and then a non-reference frame is generated again at a position 2 frames away from the No. 5 P frame (i.e., the No. 8 P frame).

If the current frame is P frame No. 3, that is, after the frame position is set, steps S150 - S190 are executed.

S150, determine whether the frame spacing between the current frame and the nearest non-reference frame is the set frame spacing, if so, set the first identifier of the current frame to the non-reference frame, if not, set the first identifier of the current frame to the reference frame, and then execute steps S160-S190.

As shown in Figure 3, if the current frame is P frame No. 3, the interval between it and the nearest non-reference frame (i.e., P frame No. 2) is not equal to 3, so the first identifier of P frame No. 3 is set as the reference frame; if the current frame is P frame No. 6, that is, position num_b, the interval between it and the nearest non-reference frame (i.e., P frame No. 2) is equal to 3, so the first identifier of P frame No. 6 is set as the non-reference frame.

S160, write the first identifier no_ref_flag of the current frame (this is the P frame) into the frame header of the current frame. Specifically, write the first identifier no_ref_flag into the padding NALU of the frame header of the current frame.

S170, determine whether the previous frame of the current frame (P frame in this case) is a reference frame. If it is a reference frame, assign the read cache address of the current frame to the write cache address of the previous frame of the current frame so that the reference frame data obtained during the encoding process of the previous frame of the current frame can be read during encoding, and assign the write cache address of the current frame to the other of the two encoding cache addresses except the write cache address of the previous frame of the current frame so as to store the reference frame data obtained during the encoding process of the current frame. If it is a non-reference frame, keep the read cache address of the current frame consistent with the read cache address of the previous frame of the current frame so that the reference frame data obtained by encoding the reference frame of the previous frame of the current frame can be read during encoding, and keep the write cache address consistent with the write cache address of the previous frame so as to store the reference frame data obtained during the encoding process of the current frame when the current frame is a reference frame.

The present invention determines whether the previous frame of the current frame is a reference frame by setting a variable interval ref_id and determining the size of the variable interval ref_id. Specifically, the interval ref_id is assigned a value of Num1-Num_r-1, where Num1 is the value of the syntax element frame_num of the current frame, and Num_r is the frame_num value of the reference frame closest to the current frame in front of the current frame. As shown in Figure 3, for P frame No. 2, the reference frame that is in front of frame No. 2 and closest to frame No. 2 is P frame No. 1. For P frame No. 1, the reference frame that is in front of frame No. 1 and closest to frame No. 1 is I frame No. 0. Therefore, when the current frame is P frame No. 1, the interval ref_id is assigned a value of Num1-Num_r-1=1-0-1=0; when the current frame is P frame No. 2, the interval ref_id is assigned a value of Num1-Num_r-1=2-1-1=0; when the current frame is P frame No. 3, the interval ref_id is assigned a value of Num1-Num_r-1=3-1-1>0.

Determine the size of the interval ref_id. If the interval ref_id is 0, it means that the previous frame of the current frame is a reference frame. Therefore, the read cache address of the current frame is assigned to the write cache address of the previous frame, and the write cache address is assigned to the other of the two encoding cache addresses except the write cache address of the previous frame. If the interval is greater than 0, it means that the previous frame of the current frame is not a reference frame, that is, a non-reference frame. Therefore, the read cache address and write cache address of the current frame are not changed, and the read cache address and write cache address of the current frame are respectively kept consistent with the read cache address and write cache address of the previous frame of the current frame. For example, for P frame No. 1, its interval ref_id = 1-0-1 = 0, therefore, the read cache address of P frame No. 1 is assigned to the write cache address buffer1 of the previous frame No. 0 I frame, and the write cache address of P frame No. 1 is assigned to another cache address buffer2.

In some embodiments, the interval ref_id is configured to the hardware encoding circuit through a register so that the relevant information of the reference frame is encoded during hardware entropy encoding.

S180, encode the current frame and output a compressed code stream.

The hardware encoding circuit reads frame data from the read cache address of the current frame as a reference frame, and encodes the current frame. If the current frame is a non-reference frame, according to the H.264 encoding standard, after the current frame encoding is transformed, quantized, and other operations, the hardware encoding circuit no longer performs reconstruction operations such as inverse quantization, inverse transformation, and filtering. Therefore, even if the write cache address of the current frame belonging to the non-reference frame is assigned in the previous step S170, the hardware circuit cannot obtain the reconstructed reference frame data to write to the write cache address; and if the current frame is a reference frame, according to the H.264 encoding standard, after the current frame encoding is transformed, quantized, and entropy encoded to form a compressed code stream, the hardware encoding circuit also needs to generate reference frame data through reconstruction operations such as inverse quantization, inverse transformation, and filtering, and then the hardware encoding circuit writes the reference frame data obtained by the current frame encoding into the write cache address of the current frame. Finally, the compressed code stream formed as above is output.

S190, real-time detection of the bit rate of the compressed code stream, determining whether the bit rate is greater than a first bit rate threshold, if not, re-execute steps S120-S190 until the encoding is completed; if yes, increase the frame interval to reduce the number of non-reference frames, and re-execute steps S120-S190 until the encoding is completed. The encoded frame can be sent to another mobile device as a decoding end via wireless communication.

Set the first bitrate threshold THR1. When there are too many non-reference frames in the image group, the real-time bitrate will increase. If the current bitrate is greater than the first bitrate threshold THR1, it is necessary to reduce the frame interval to reduce the number of non-reference frames, thereby reducing the bitrate and ensuring a certain encoding quality. For example, the value of the frame interval is reduced by 1 each time until the bitstream is less than the first bitrate threshold THR1. For example, the target bitrate target_bit and the allowable deviation percentage X (this deviation percentage can be set according to the real-time encoding situation) can be set, that is, THR1 = target_bit + X*target_bit.

Through the solution of the present invention, both the coding quality and the guarantee of a certain number of non-reference frames are taken into account (ie, the anti-interference capability of the video sequence in wireless transmission is guaranteed).

Example 2

This embodiment provides another embodiment of the H.264 video encoding method, which is substantially the same as the first embodiment, with the main difference being that a bit rate interval determined by a plurality of bit rate thresholds is set, and the frame interval is processed differently according to the bit rate interval in which the current bit rate is located. Specifically, after step S190 of the first embodiment is replaced by the following steps, the scheme of the second embodiment is formed, as described in detail as follows.

A second bit rate threshold THR2 and a third bit rate threshold THR3 are set to detect the bit rate of the compressed bit stream in real time and determine the size of the bit rate.

If: the second code rate threshold ≤ the code rate ≤ the third code rate threshold, the size of the frame interval is not adjusted, and steps S120-S190 are re-executed until the encoding is completed. For example, when the target code rate target_bit and the allowable deviation percentage X (e.g., 2%) are set, that is, target_bit-X*target_bit≤code rate≤target_bit+X*target_bit, the frame interval is kept unchanged to prevent the frame interval from being constantly corrected.

If: the bit rate > the third bit rate threshold, the frame interval is increased to reduce the number of non-reference frames, and steps S120-S190 are performed again. For example, when the target bit rate target_bit and the allowable deviation percentage X are set, that is, when the bit rate > target_bit + X*target_bit, the frame interval is increased to reduce the number of non-reference frames, thereby reducing the bit rate while ensuring a certain encoding quality.

If: the bit rate is less than the second bit rate threshold, the frame interval is reduced to increase the number of non-reference frames, and steps S120-S190 are re-executed until the encoding is completed. For example, when the target bit rate target_bit and the allowable deviation percentage X are set, that is, the bit rate>target_bit-X*target_bit, the frame interval is reduced to increase the number of non-reference frames, and then when these non-reference frames are lost, the frames following the lost non-reference frames during decoding can still be decoded and recovered, that is, the anti-interference ability of the video sequence in wireless transmission is guaranteed.

FIG3 is a schematic diagram of a sequence of encoded video frames according to an embodiment of the present invention. The calculation results of the parameters related to frames 0-6 during the encoding process according to the aforementioned steps of this embodiment are shown in Table 1 below.

Table 1

It should be noted that the use of step numbers (letters or numbers) to refer to certain specific method steps in the present invention is only for the purpose of convenience and brevity of description, and is by no means intended to limit the order of these method steps. Those skilled in the art will appreciate that the order of the relevant method steps should be determined by the technology itself and should not be inappropriately limited by the existence of step numbers.

Those skilled in the art will appreciate that, without conflict, the above-mentioned preferred solutions can be freely combined and superimposed.

It should be understood that the above-mentioned embodiments are merely illustrative and not restrictive. Without departing from the basic principles of the present invention, various obvious or equivalent modifications or substitutions that can be made by those skilled in the art to the above-mentioned details will be included in the scope of the claims of the present invention.

Claims (17)

1. A H.264 video encoding method, comprising the following steps: S110, determining the type of the current frame. If the type of the current frame is an I frame, setting the first identifier of the current frame as a reference frame and executing steps S140-S150. If the type of the current frame is a P frame, executing steps S120-S150. S120, determining the position of the current frame. If the position of the current frame is before the set frame position, setting the first identifier of the current frame as a reference frame, and executing steps S140-S150; if the position of the current frame is the set frame position, setting the first identifier of the current frame as a non-reference frame, and executing steps S140-S150; if the position of the current frame is after the set frame position, executing steps S130-S150; wherein the first identifier is used to indicate whether the current frame is a reference frame; S130, determining whether the frame interval between the current frame and the nearest non-reference frame is a set frame interval, if yes, setting the first identifier of the current frame to a non-reference frame, if no, setting the first identifier of the current frame to a reference frame; S140, encoding the current frame according to its first identifier, and outputting a compressed code stream; S150, detecting the bit rate of the compressed code stream in real time, determining whether the bit rate is greater than a first bit rate threshold, if not, re-executing steps S110-S150, if yes, increasing the frame interval to reduce the number of non-reference frames, and re-executing steps S110-S150 until the encoding is completed. 2. The H.264 video encoding method according to claim 1, characterized in that: In step S150, the frame interval is increased by a value of 1 each time. 3. The H.264 video encoding method according to any one of claims 1-2, characterized in that: Before step S110, the video encoding method further includes the following steps: S100, providing two encoding buffer addresses for storing reference frame data generated during the encoding process in a buffer for encoding; The step S110 further includes the following steps: If the type of the current frame is an I frame, write the first identifier of the current frame into the frame header of the current frame, and initialize the write cache address of the current frame to any one of the two encoding cache addresses to store reference frame data obtained by encoding the current frame; The step S130 further includes the following steps: S131, writing the first identifier of the current frame into the frame header of the current frame; S132, determining whether the previous frame of the current frame is a reference frame, if so, executing step S133, if not, executing step S134; S133, assigning the read cache address of the current frame to the write cache address of the frame before the current frame, so as to read the reference frame data obtained by encoding the frame before the current frame during encoding, and assigning the write cache address to the other of the two encoding cache addresses except the write cache address of the frame before the current frame, so as to store the reference frame data obtained by encoding the current frame when the current frame is a reference frame; S134, keeping the read cache address of the current frame consistent with the read cache address of the frame before the current frame, so as to read the reference frame data obtained by encoding the reference frame of the frame before the current frame during encoding, and keeping the write cache address consistent with the write cache address of the frame before the current frame, so as to store the reference frame data obtained by encoding the current frame when the current frame is a reference frame. 4. The H.264 video encoding method according to any one of claims 1-2, characterized in that: In steps S110 and S130, the first identifier of the current frame is written into a padding network abstraction layer unit in a frame header of the current frame. 5. A H.264 video encoding method, characterized in that it comprises the following steps: S210, determining the type of the current frame. If the type of the current frame is an I frame, the first identifier of the current frame is set as a reference frame, and steps S240-S250 are executed. If the type of the current frame is a P frame, steps S220-S250 are executed. S220, determining the position of the current frame. If the position of the current frame is before the set frame position, setting the first identifier of the current frame as a reference frame, and executing steps S240-S250; if the position of the current frame is the set frame position, setting the first identifier of the current frame as a non-reference frame, and executing steps S240-S250; if the position of the current frame is after the set frame position, executing steps S230-S250; wherein the first identifier is used to indicate whether the current frame is a reference frame; S230, determining whether the frame interval between the current frame and the nearest non-reference frame is a set frame interval, if yes, setting the first identifier of the current frame to a non-reference frame, if no, setting the first identifier of the current frame to a reference frame; S240, encoding the current frame according to its first identifier, and outputting a compressed code stream; S250, detecting the bit rate of the compressed bit stream in real time and determining the size of the bit rate, If: the second code rate threshold ≤ the code rate ≤ the third code rate threshold, the frame interval size is not adjusted, and steps S210-S250 are re-executed until the encoding is completed; If: the bit rate> the third bit rate threshold, then increase the frame interval to reduce the number of non-reference frames, and re-execute steps S210-S250 until the encoding is completed; If: the code rate is less than the second code rate threshold, the frame interval is reduced to increase the number of non-reference frames, and steps S210 to S250 are re-executed until the encoding is completed. 6. The H.264 video encoding method according to claim 5, characterized in that: In step S250, the frame interval is decreased by a value of 1 each time and/or the frame interval is increased by a value of 1 each time. 7. The H.264 video encoding method according to claim 5, characterized in that: Before step S210, the video encoding method further includes the following steps: S200, providing two encoding buffer addresses for storing reference frame data generated during the encoding process in a buffer for encoding; The step S210 further includes the following steps: If the type of the current frame is an I frame, writing the first identification setting of the current frame into the frame header of the current frame, and initializing the write cache address of the current frame to any one of the two encoding cache addresses for storing reference frame data obtained by encoding the current frame; The step S230 further includes the following steps: S231, writing the first identifier of the current frame into the frame header of the current frame; S232, determining whether the previous frame of the current frame is a reference frame, if so, executing step S233, if not, executing step S234; S233, assigning the read cache address of the current frame to the write cache address of the frame before the current frame, so as to read the reference frame data obtained by encoding the frame before the current frame during encoding, and assigning the write cache address to the other of the two encoding cache addresses except the write cache address of the frame before the current frame, so as to store the reference frame data obtained by encoding the current frame when the current frame is a reference frame; S234, keeping the read cache address of the current frame consistent with the read cache address of the frame before the current frame, so as to read the reference frame data obtained by encoding the reference frame of the frame before the current frame during encoding, and keeping the write cache address consistent with the write cache address of the frame before the current frame, so as to store the reference frame data obtained by encoding the current frame when the current frame is a reference frame. 8. The H.264 video encoding method according to claim 7, characterized in that: In steps S210 and S230, the first identifier of the current frame is written into a padding network abstraction layer unit in a frame header of the current frame. 9. An H.264 video encoding device, comprising: The first processing unit is used to determine the type of the current frame. If the type of the current frame is an I frame, the first identifier of the current frame is set as a reference frame, and the first four processing units and the first five processing units are triggered to work. If the type of the current frame is a P frame, the first two processing units to the first five processing units are triggered to work. The first two processing units are used to determine the position of the current frame. If the position of the current frame is before the set frame position, the first identifier of the current frame is set as a reference frame, and the first four processing units and the first five processing units are triggered to work; if the position of the current frame is the set frame position, the first identifier of the current frame is set as a non-reference frame, and the first four processing units and the first five processing units are triggered to work; if the position of the current frame is after the set frame position, the first three processing units to the first five processing units are triggered to work; wherein the first identifier is used to indicate whether the current frame is a reference frame; The first three processing units are used to determine whether the frame interval between the current frame and the nearest non-reference frame is a set frame interval, and if so, set the first identifier of the current frame as a non-reference frame; if not, set the first identifier of the current frame as a reference frame; The first four processing units are used to encode the current frame according to its first identifier and output a compressed code stream; The first five processing units are used to detect the bit rate of the compressed code stream in real time, determine whether the bit rate is greater than the first bit rate threshold, and if not, re-trigger the first one to the first five processing units to work; if so, increase the frame interval to reduce the number of non-reference frames, and re-trigger the first one to the first five processing units to work until the encoding is completed. 10. The H.264 video encoding device according to claim 9, characterized in that: The first five processing units are used to increase the frame interval by a value of 1 each time. 11. The H.264 video encoding device according to claim 9, further comprising a first sixth processing unit and a first seventh processing unit, The first six processing units are used to provide two blocks of encoding buffer addresses storing reference frame data generated by the encoding process in the buffer for encoding; The first processing unit is further configured to: If the type of the current frame is an I frame, write the first identifier of the current frame into the frame header of the current frame, initialize the write cache address of the current frame to any one of the two encoding cache addresses, so as to store reference frame data obtained by encoding the current frame; the first seven processing units are used for: writing the first identifier of the current frame into the frame header of the current frame; determining whether the previous frame of the current frame is a reference frame, If yes, then: assigning the read cache address of the current frame to the write cache address of the frame before the current frame, so as to read the reference frame data obtained by encoding the frame before the current frame during encoding, and assigning the write cache address to the other of the two encoding cache addresses except the write cache address of the frame before the current frame, so as to store the reference frame data obtained by encoding the current frame when the current frame is a reference frame; If not, then: the read cache address of the current frame is kept consistent with the read cache address of the frame before the current frame, so as to read the reference frame data obtained by encoding the reference frame of the frame before the current frame during encoding, and the write cache address is kept consistent with the write cache address of the frame before the current frame, so as to store the reference frame data obtained by encoding the current frame when the current frame is a reference frame. 12. The H.264 video encoding device according to claim 9, characterized in that: The first-first processing unit and the first-third processing unit are further configured to write the first identifier of the current frame into a padding network abstraction layer unit in a frame header of the current frame. 13. An H.264 video encoding device, comprising: The second-first processing unit is used to determine the type of the current frame. If the type of the current frame is an I frame, the first identifier of the current frame is set as a reference frame, and the second-fourth processing unit and the second-fifth processing unit are triggered to work. If the type of the current frame is a P frame, the steps are executed to trigger the second-second processing unit to the second-fifth processing unit to work. The second processing unit is used to determine the position of the current frame. If the position of the current frame is before the set frame position, the first identifier of the current frame is set as a reference frame, and the second fourth processing unit and the second fifth processing unit are triggered to work; if the position of the current frame is the set frame position, the first identifier of the current frame is set as a non-reference frame, and the second fourth processing unit and the second fifth processing unit are triggered to work; if the position of the current frame is after the set frame position, the second third processing unit to the second fifth processing unit are triggered to work; wherein the first identifier is used to indicate whether the current frame is a reference frame; The second third processing unit is used to determine whether the frame interval between the current frame and the nearest non-reference frame is a set frame interval, and if so, set the first identifier of the current frame to a non-reference frame; if not, set the first identifier of the current frame to a reference frame; The second four-processing unit is used to encode the current frame according to its first identifier and output a compressed code stream; The second fifth processing unit is used to detect the code rate of the compressed code stream in real time and determine the size of the code rate. If: the second code rate threshold ≤ the code rate ≤ the third code rate threshold, the frame interval size is not adjusted, and the second-first processing unit to the second-fifth processing unit are re-triggered to work; If: the bit rate> the third bit rate threshold, then increasing the frame interval to reduce the number of non-reference frames, and re-triggering the second-first processing unit to the second-fifth processing unit to work until the encoding is completed; If: the code rate is less than the second code rate threshold, the frame interval is reduced to increase the number of non-reference frames, and the second-first processing unit to the second-fifth processing unit are re-triggered to work until the encoding is completed. 14. The H.264 video encoding device according to claim 13, characterized in that: The second fifth processing unit is used for reducing the frame interval by a value of 1 each time and/or increasing the frame interval by a value of 1 each time. 15. The H.264 video encoding device according to claim 13, further comprising: a second eighth processing unit and a second ninth processing unit, Two encoding buffer addresses for storing reference frame data generated by the encoding process in the buffer for encoding; The second-first processing unit is further configured to, if the type of the current frame is an I frame, write the first identifier of the current frame into the frame header of the current frame, and initialize the write cache address of the current frame to any one of the two encoding cache addresses, so as to store the reference frame data obtained by encoding the current frame; The second ninth processing unit is used to write the first identifier of the current frame into the frame header of the current frame; write the first identifier of the current frame into the frame header of the current frame; determine whether the previous frame of the current frame is a reference frame, If yes, then: assigning the read cache address of the current frame to the write cache address of the frame before the current frame, so as to read the reference frame data obtained by encoding the frame before the current frame during encoding, and assigning the write cache address to the other of the two encoding cache addresses except the write cache address of the frame before the current frame, so as to store the reference frame data obtained by encoding the current frame when the current frame is a reference frame; If not, then: the read cache address of the current frame is kept consistent with the read cache address of the frame before the current frame, so as to read the reference frame data obtained by encoding the reference frame of the frame before the current frame during encoding, and the write cache address is kept consistent with the write cache address of the frame before the current frame, so as to store the reference frame data obtained by encoding the current frame when the current frame is a reference frame. 16. The H.264 video encoding device according to claim 15, characterized in that: The second-first processing unit and the second-third processing unit are further configured to write the first identifier of the current frame into a padding network abstraction layer unit in a frame header of the current frame. 17. A storage medium storing a program, characterized in that the program is executed by a processor as the encoding method according to any one of claims 1 to 8.