WO2018161790A1

WO2018161790A1 - Video transmission method and device

Info

Publication number: WO2018161790A1
Application number: PCT/CN2018/076526
Authority: WO
Inventors: 周川
Original assignee: 华为技术有限公司
Priority date: 2017-03-08
Filing date: 2018-02-12
Publication date: 2018-09-13
Also published as: CN108574882B; CN108574882A

Abstract

Embodiments of the present application disclose a video transmission method and device, relating to the technical field of transmission, and are used to conserve transmission and cache resources. The video comprises a primary encoding stream and a secondary encoding stream. The secondary encoding stream is used to increase a bit rate of the primary encoding stream. The primary encoding stream comprises at least two primary stream fragments. The secondary encoding stream comprises at least two secondary stream fragments. Each primary stream fragment corresponds to one secondary stream fragment. The method comprises: sending a first primary stream fragment request to a server, wherein the first primary stream fragment request is used to request a first primary stream fragment, and the first primary stream fragment is a primary stream fragment in at least two primary stream fragments; receiving the first primary stream fragment sent by the server; sending a first secondary stream fragment request to the server if a current real-time network status satisfies a preset condition, wherein the first secondary stream fragment request is used to request a first secondary stream fragment corresponding to the first primary stream fragment; and receiving the first secondary stream fragment sent by the server.

Description

Video transmission method and device

The present application claims priority to Chinese Patent Application No. JP-A No. No. No. No. No. No. No. No. No. No. No. No. No. No. No. No. .

Technical field

The present application relates to the field of transmission technologies, and in particular, to a video transmission method and apparatus.

Background technique

HLS (HTTP live streaming) is a streaming media transmission protocol based on hypertext transfer protocol (HTTP), which enables live streaming and on-demand streaming. In an HLS on-demand scenario, the server encodes the video content to obtain a plurality of video files of different code rates, and then divides each of the plurality of video files into a plurality of slices according to time. When the terminal needs to play the video, it requests the required fragment according to the playing time and the current network condition, and buffers, decodes, and plays the requested fragment.

In the HLS technology, when the fragment requested by the terminal is switched from the low code rate to the high code rate, the cached but unplayed low bit rate fragments will be disabled, resulting in waste of transmission resources and cache resources.

Summary of the invention

The embodiment of the present application provides a video transmission method and apparatus, which are used to save transmission resources and cache resources.

In a first aspect, a video transmission method is provided, where the video includes a primary encoded stream and a secondary encoded stream, where the secondary encoded stream is used to increase a code rate of the primary encoded stream, the primary encoded stream includes at least two mainstream fragments, and the secondary encoded stream includes at least Two auxiliary stream segments, each of which corresponds to one auxiliary stream segment. The method includes: the terminal sends a first mainstream fragment request to the server, where the first mainstream fragment request is used to request the first mainstream fragment, and the first mainstream fragment is one mainstream fragment of the at least two mainstream fragments. Receiving a first mainstream fragment sent by the server; if the current real-time network condition meets the preset condition, sending a first auxiliary stream fragment request to the server, where the first auxiliary stream fragment request is used to request the first mainstream fragment Corresponding first auxiliary stream fragment; receiving the first auxiliary stream fragment sent by the server. In the technical solution, the terminal requests the mainstream fragment, and requests the auxiliary stream fragmentation as needed; wherein the auxiliary stream fragment is used to improve the code rate of the mainstream fragment. In the embodiment of the present application, the primary encoded stream can be played separately, or the primary encoded stream and the secondary encoded stream are superimposed and reconstructed, so the buffered primary encoded stream does not lose its role due to buffering the secondary encoded stream. Compared with the prior art, it is possible to save transmission resources and cache resources.

Optionally, the terminal and the server communicate through the HTTP/2 protocol. The server can be any server that supports the HTTP/2 protocol, and can be, for example, a WEB server.

In a possible implementation manner, the method may further include: the terminal superimposing and reconstructing the first mainstream fragment and the first auxiliary stream fragment, and playing the reconstructed code stream. In this way, the quality of video playback can be improved.

In a possible implementation, the receiving, by the terminal, the first mainstream fragment sent by the server may include: receiving, by a server, a first mainstream fragment sent by the server, where the link may be between the terminal and the server. Any of the links. In this case, the first auxiliary stream fragment sent by the receiving server includes: receiving, by the link, the first auxiliary stream fragment sent by the server. In this way, both the terminal and the server need only maintain one link, so the implementation is simple.

In a possible implementation, the method may further include: the terminal sending a second mainstream fragment request to the server, where the second mainstream fragment request is used to request the second mainstream fragment, and the second mainstream fragment is at least a mainstream fragment other than the first mainstream fragment in the two mainstream fragments; a second mainstream fragment sent by the receiving server; if the current real-time network condition meets the preset condition, sending the second auxiliary stream to the server a slice request, where the second auxiliary stream fragment request is used to request a second auxiliary stream fragment corresponding to the second mainstream fragment; if the second auxiliary stream fragmentation request is sent to the server, and the second auxiliary stream is received Before detecting the current real-time network condition, the interrupt request is sent to the server, where the interrupt request is used to request to interrupt the transmission process of the second auxiliary stream fragment. In this way, the terminal can obtain the next mainstream fragment of the second mainstream fragment to be reduced to a certain extent, thereby effectively reducing the occurrence of a stuck phenomenon in the process of playing the video by the terminal. It can be understood that the possible implementation manner can be performed separately without relying on any of the technical solutions provided above, such that the second mainstream fragment is arbitrary.

A second aspect provides a video transmission method, where the video includes a primary encoded stream and a secondary encoded stream, the primary encoded stream includes at least two mainstream fragments, and the secondary encoded stream includes at least two auxiliary stream fragments, each mainstream fragment. Corresponding to a secondary stream segment; the method may include: the server receiving the first mainstream fragment request sent by the terminal, where the first mainstream fragment request is used to request the first mainstream fragment, and the first mainstream fragment is at least two The first mainstream fragment is sent to the terminal; the first auxiliary stream fragment request sent by the terminal is received, where the first auxiliary stream fragment request is used to request the first mainstream fragment corresponding to the first mainstream fragment. The first auxiliary stream fragmentation request is sent by the terminal if the current network condition meets the preset condition; and the first auxiliary stream fragment is sent to the terminal.

In a possible implementation, sending the first mainstream fragment to the terminal may include: sending the first mainstream fragment to the terminal on a link. In this case, sending the first auxiliary stream fragment to the terminal may include: sending the first auxiliary stream fragment to the terminal on the link.

In a possible implementation manner, the priority of the mainstream fragment is higher than the priority of the auxiliary stream fragment corresponding to the mainstream fragment. In this case, sending the first auxiliary stream fragment to the terminal may include: if it is determined that the first mainstream fragment is sent to the terminal, sending the first auxiliary stream fragment to the terminal. In this way, the mainstream fragments can be preferentially transmitted, thereby ensuring the continuity of playback of the mainstream fragments.

In a possible implementation, the method may further include: the server receiving the second mainstream fragment request sent by the terminal, where the second mainstream fragment request is used to request the second mainstream fragment, and the second mainstream fragment is a mainstream fragment other than the first mainstream fragment of the at least two mainstream fragments; sending a second mainstream fragment to the terminal; receiving a second auxiliary stream fragmentation request sent by the terminal, where the second auxiliary stream is divided The slice request is used to request the second auxiliary stream fragment corresponding to the second mainstream fragment; if the interrupt request sent by the terminal is received before the second auxiliary stream fragmentation request is sent to the terminal, the second auxiliary stream fragment is interrupted. Transmission process. It can be understood that the possible implementation manner may be performed independently of any of the technical solutions provided above, such that the second mainstream fragment is arbitrary.

The beneficial effects of the technical solutions provided by the second aspect or any of the possible implementation manners of the second aspect may refer to the foregoing first aspect or the possible implementation manner of the first aspect, and details are not described herein again.

A third aspect provides a terminal, where the video includes a primary encoded stream and a secondary encoded stream, where the secondary encoded stream is used to increase a code rate of the primary encoded stream, the primary encoded stream includes at least two mainstream fragments, and the secondary encoded stream includes at least two secondary streams. Flow segmentation, where each mainstream segment corresponds to one auxiliary stream segment. The terminal may include: a transmitting unit and a receiving unit. The sending unit is configured to send a first mainstream fragment request to the server, where the first mainstream fragment request is used to request the first mainstream fragment, and the first mainstream fragment is one of the at least two mainstream fragments. Fragmentation. The receiving unit is configured to receive the first mainstream fragment sent by the server. The sending unit is further configured to: if the current real-time network condition meets the preset condition, send a first auxiliary stream fragment request to the server, where the first auxiliary stream fragment request is used to request the first auxiliary corresponding to the first mainstream fragment Streaming slices. The receiving unit is further configured to receive the first auxiliary stream fragment sent by the server.

In a possible implementation, the terminal may further include: a reconstruction unit and a playback unit. The reconstruction unit is configured to superimpose and reconstruct the first mainstream segment and the first auxiliary stream segment. The playing unit is configured to play the code stream generated after the reconstruction.

In a possible implementation manner, the receiving unit is configured to: receive, by a server, a first mainstream fragment sent by the server on a link; and receive, by the link, the first auxiliary stream fragment sent by the server.

In a possible implementation, the sending unit may be further configured to send a second mainstream fragment request to the server, where the second mainstream fragment request is used to request the second mainstream fragment, and the second mainstream fragment is at least One of the two mainstream slices except the first mainstream slice. The receiving unit is further configured to receive the second mainstream fragment sent by the server. The sending unit may be further configured to: if the current real-time network condition meets the preset condition, send a second auxiliary stream fragmentation request to the server, where the second auxiliary stream fragmentation request is used to request the second corresponding to the second mainstream fragment Auxiliary flow segmentation. The sending unit may be further configured to: if the current real-time network condition does not satisfy the preset condition after detecting the second auxiliary stream fragmentation request to the server and before receiving the second auxiliary stream fragmentation, send an interrupt request to the server, The interrupt request is used to request to interrupt the transmission process of the second auxiliary stream fragment.

A fourth aspect provides a server, where the video includes a primary encoded stream and a secondary encoded stream, the primary encoded stream includes at least two mainstream fragments, and the secondary encoded stream includes at least two auxiliary stream fragments, each mainstream fragment and one auxiliary stream. The stream is fragmented. The server may include: a receiving unit and a transmitting unit. The receiving unit is configured to receive a first mainstream fragment request sent by the terminal, where the first mainstream fragment request is used to request the first mainstream fragment, and the first mainstream fragment is one of at least two mainstream fragments. Mainstream segmentation. And a sending unit, configured to send the first mainstream fragment to the terminal. The receiving unit is further configured to: receive the first auxiliary stream fragmentation request sent by the terminal, where the first auxiliary stream fragmentation request is used to request the first auxiliary stream fragment corresponding to the first mainstream fragment; the first auxiliary stream fragmentation The request is sent by the terminal if the current network condition satisfies the preset condition. The sending unit is further configured to send the first auxiliary stream fragment to the terminal.

In a possible implementation manner, the sending unit is specifically configured to: send a first mainstream fragment to the terminal on a link; and send the first auxiliary stream fragment to the terminal on the link.

In a possible implementation manner, the priority of the mainstream fragment is higher than the priority of the auxiliary stream fragment corresponding to the mainstream fragment. In this case, the sending unit is specifically configured to: if it is determined that the first mainstream has been sent to the terminal Fragmentation, the first auxiliary stream fragment is sent to the terminal.

In a possible implementation, the receiving unit is further configured to: receive a second mainstream fragment request sent by the terminal, where the second mainstream fragment request is used to request the second mainstream fragment, and the second mainstream fragment is at least One of the two mainstream slices except the first mainstream slice. The sending unit is further configured to send the second mainstream fragment to the terminal. The receiving unit is further configured to: receive a second auxiliary stream fragmentation request sent by the terminal, where the second auxiliary stream fragmentation request is used to request the second auxiliary stream fragment corresponding to the second mainstream fragment. The receiving unit is further configured to receive an interrupt request sent by the terminal. The server may further include: an interrupting unit, configured to interrupt the transmission process of the second auxiliary stream fragment if the receiving unit receives the interrupt request sent by the terminal before the sending unit sends the second auxiliary stream fragmentation request to the terminal.

Optionally, the interrupt request in the above first to fourth aspects may include: an RST_STREM frame in HTTP/2. In addition, the interrupt request may be a newly designed frame, or other frames provided in the prior art may be multiplexed, which is not limited in this embodiment of the present application.

In a fifth aspect, a terminal is provided, and the terminal has a function of implementing terminal behavior in the foregoing method embodiment. This function can be implemented in hardware or in hardware by executing the corresponding software. The hardware or software includes one or more modules corresponding to the functions described above.

In a possible implementation manner, the terminal may include: a processor, a memory, a communication bus, and a communication interface; wherein the memory is configured to store a computer execution instruction, the processor and the memory are connected through a communication bus, and when the terminal is running, processing The computer executing the memory storage executes the instructions to cause the terminal to perform the video transmission method provided by the above first aspect or any one of the first aspects.

In a sixth aspect, a computer readable storage medium for storing computer software instructions for use by the terminal, when executed on a computer, causes the computer to perform the video transmission method of any of the above aspects.

In a seventh aspect, a computer program product comprising instructions for causing a computer to perform the video transmission method of any of the above first aspects when executed on a computer is provided.

In addition, the technical effects brought by the design manners of any one of the third aspect, the fifth aspect, and the sixth aspect can be referred to the technical effects brought by different design modes in the first aspect, and details are not described herein again.

In an eighth aspect, a server is provided, the server having a function of implementing server behavior in the above method embodiment. This function can be implemented in hardware or in hardware by executing the corresponding software. The hardware or software includes one or more modules corresponding to the functions described above.

In a possible implementation, the server may include: a processor, a memory, a communication bus, and a communication interface. The memory is used to store computer execution instructions, the processor and the memory are connected by a communication bus, and when the server is running, the processor executes the memory storage computer execution instructions, so that the server performs any of the above second aspect or the second aspect. The video transmission method provided by the implementation.

In a ninth aspect, a computer readable storage medium for storing computer software instructions for use by the server, when executed on a computer, causes the computer to perform the video transmission method of any of the above second aspects.

According to a tenth aspect, there is provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform the video transmission method of any of the above second aspects.

In addition, the technical effects brought by the design manners of any one of the fourth aspect, the ninth aspect, and the tenth aspect can be referred to the technical effects brought by different design modes in the first aspect, and details are not described herein again.

DRAWINGS

FIG. 1 is a schematic diagram of a system architecture applicable to a technical solution provided by an embodiment of the present disclosure;

2 is a schematic structural diagram of an encoding server according to an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a transmission server according to an embodiment of the present disclosure;

FIG. 4 is a schematic structural diagram of a terminal according to an embodiment of the present application;

FIG. 5 is a schematic diagram of interaction of a video transmission method according to an embodiment of the present application;

FIG. 6 is a schematic diagram of interaction of another video transmission method according to an embodiment of the present disclosure;

FIG. 7 is a schematic flowchart of a video transmission method according to an embodiment of the present application;

FIG. 8 is a schematic flowchart of playing a video according to an embodiment of the present application;

FIG. 9 is a schematic diagram of interaction of another video transmission method according to an embodiment of the present disclosure;

FIG. 10 is a schematic structural diagram of another terminal according to an embodiment of the present disclosure;

FIG. 11 is a schematic structural diagram of another server according to an embodiment of the present application.

detailed description

In the HLS technology, when the fragment requested by the terminal is switched from the low code rate to the high code rate, the cached but unplayed low bit rate fragments will be disabled, resulting in waste of transmission resources and cache resources. For example, suppose the terminal has requested and cached the low code rate fragments 1 to 10, and is currently playing the fragment 5; if the terminal detects that the current network condition is good when the fragment 5 is played, the high code can be requested from the server. The fragment 6 of the rate, such that the low-rate fragment 6 that has been cached but not played will be deactivated, thereby causing waste of transmission resources and cache resources.

Based on this, the embodiment of the present application provides a video transmission method and device, the basic principle is: the terminal requests the mainstream fragmentation, and requests the auxiliary stream fragmentation as needed; wherein the auxiliary stream fragment is used to improve the mainstream fragmentation. Code rate. In the prior art, fragments of the same content of different code rates exist independently, are independently transmitted, and are selectively played. In the present application, the role of the secondary encoded stream is to improve the code rate of the primary encoded stream, that is, the primary encoded stream can be played separately, or the primary encoded stream and the secondary encoded stream are superimposed and reconstructed, and thus the cached primary encoding is performed. The stream does not lose its effect by caching the secondary encoded stream. Compared with the prior art, it is possible to save transmission resources and cache resources.

The primary encoded stream and the secondary encoded stream involved in the embodiment of the present application may be a code stream obtained by the server after being encoded according to a reconstructive video coding (RVC) technology or the like. Among them, in the RVC technology, the encoder can generate two streams at the same time, one of which is the primary encoded stream and the other is the secondary encoded stream. The main encoded stream is a video stream with a lower resolution, and the player can directly decode and play. The role of the secondary encoded stream is to assist the player in improving the resolution of the primary encoded stream. For example, the code rate of the main code stream is 720p. By performing super-resolution operation and superimposing the auxiliary code stream and reconstructing, the code rate can be increased to 1080p, thereby improving the video playback quality. Of course, the embodiment of the present application is not limited to the coding technology of the RVC technology, and may be other coding technologies.

The technical solution provided by the embodiment of the present application can be applied to the system architecture shown in FIG. 1 , the system package shown in FIG. 1 : the encoding server 11 , the transmission server 12 , and the terminal 13 . The encoding server 11 is for encoding the video content according to a certain encoding technique and transmitting the encoded result to the transmission server 12. The transmission server 12 is configured to save the encoded result, and when receiving the fragmentation request sent by the terminal 13, transmits the encoded fragment to the terminal 13. The terminal 13 is configured to send a fragmentation request to the transmission server 12 according to a user indication or a self-trigger, and decode and play the encoded fragment received by the transmission server 12.

It should be noted that, FIG. 1 is only an example diagram, and the number of the encoding server 11 , the number of the transmission server 12 , and the number of at least one terminal 13 are not limited in the embodiment of the present application. In practical applications, Network deployment can be performed in different numbers as shown in Figure 1 as needed.

Optionally, one or more links may be established between the transport server 12 and the terminal 13, and the fragmentation request and the fragment are transmitted on the links. Here, the link here may be an HTTP/2 link, in which case the transport server 12 may be a server supporting HTTP/2. Of course, the specific implementation is not limited to this.

As shown in FIG. 2, the encoding server 11 in FIG. 1 may include: a downsampling module 1101, a mainstream encoder 1102, an auxiliary stream generating module 1103, a fragmentation control module 1104, and a mainstream. The package fragmentation module 1105 and the auxiliary stream package fragmentation module 1106.

The downsampling module 1101 is configured to receive the original video input stream, perform a down conversion rate on the input original video input stream, and output the video stream obtained by the downsampling rate to the mainstream encoder 1102. For example, receiving a 4K YUV format video input stream, and a 4K YUV format video input stream downrate, generating a 2K YUV format video stream, and then outputting the 2K YUV format video stream to the mainstream encoder 1102. Among them, YUV is an encoding format of color data.

The mainstream encoder 1102 is configured to receive the video stream input by the downsampling module 1101, and encode the video stream, for example, H.264 encoding, etc., generate a main encoded stream, and then output the main encoded stream to the mainstream package fragmentation module 1105.

The auxiliary stream information generating module 1103 is configured to compare the video stream output by the downsampling module 1101 with the main encoded stream output by the mainstream encoder 1102 to generate a secondary encoded stream. The secondary encoded stream includes content reconstructed for improving video quality, such as information for correcting partial pixel points in the primary encoded stream.

The mainstream encapsulation fragmentation module 1105 is configured to receive the main encoded stream output by the mainstream encoder 1102, and under the control of the fragmentation control module 1104, segment and encapsulate the main encoded stream according to a time point to obtain at least two mainstream fragments. For example, TS (transport stream) fragment or mp4 fragment, etc., and then output the at least two mainstream fragments.

The auxiliary stream encapsulation slice module 1106 is configured to receive the auxiliary code stream output by the auxiliary stream generation module 1103, and perform fragmentation and encapsulation on the auxiliary coded stream according to time under the control of the fragmentation control module 1104, to obtain at least two auxiliary stream segments. The slice then outputs the at least two auxiliary stream fragments. The format of the auxiliary stream fragment may be a custom format, and each auxiliary stream fragment corresponds to one mainstream fragment, and the time point of each auxiliary stream fragment is the same as the time point of the corresponding mainstream fragment, and each auxiliary The stream slice is used to increase the bit rate of the corresponding mainstream slice.

The fragmentation control module 1104 is configured to control the execution process of the mainstream package fragmentation module 1105 and the execution process of the auxiliary stream package fragmentation module 1106, and generate a warehouse receipt that describes the fragmentation (including the mainstream fragment and the auxiliary stream fragment). File, then output the warehouse order file. The warehouse order file may include a time point of each slice, a correspondence between each mainstream slice and each mainstream slice, and the like.

In addition, the encoding server 11 may further include a communication interface for communicating with an external device such as the transmission server 2. For example, the main stream fragment for receiving the output of the mainstream package fragmentation module 1105, the auxiliary stream fragment output by the auxiliary stream package fragmentation module 1106, and the warehouse order file output by the fragmentation control module 1104; and transmitting the information to The WEB server, wherein the WEB server may be a WEB server supporting the HTTP/2 protocol, and the server may be the transport server 12 above.

In hardware implementation, the communication interface 1107 can be a transceiver, a downsampling module 1101, a mainstream encoder 1102, a secondary stream information generating module 1103, a fragmentation control module 1104, a mainstream package fragmentation module 1105, and a secondary stream package fragmentation module 1106. Both may be embedded in the hardware of the encoding server 11 in hardware, or may be stored in the memory of the encoding server 11 in the form of software, so that the processor invokes the operations corresponding to the above modules. In addition, the encoding server 11 may include, in addition to the transceiver, the processor, and the memory, a communication bus for interconnecting the transceiver, the processor, and the memory.

In order to implement the technical solution provided by the embodiment of the present application, as shown in FIG. 3, the transmission server 12 in FIG. 1 may include: a communication interface 1201, a processor 1202, a memory 1203, and a communication bus 1204. The communication interface 1201, the processor 1202, and the memory 1203 are connected to each other through a communication bus 1204.

The communication interface 1201 is a unit for performing service data flow interaction between the transmission server 12 and external network elements such as the encoding server 11 and the terminal 13. Specifically, the communication interface 1201 can be configured to receive each mainstream fragment, each auxiliary stream fragment, and a warehouse order file sent by the encoding server 11. Alternatively, the main stream fragment request, the auxiliary stream encoding request or the interrupt request sent by the terminal 13 is received, or the requested main stream fragment, auxiliary stream fragment or warehouse order file is returned to the terminal 13. The memory 1203 can be used to store data and/or code, and the processor 1302 can implement various functions of the transport server 12 by running or executing program code stored in the memory 1203, as well as invoking data stored in the memory 1203.

It can be understood that the transmission server 12 shown in FIG. 3 does not constitute a limitation on the transmission server 12. In actual implementation, the transmission server 12 may further include more components than the illustration, or combine some components, or different. Assembly of parts. For example, the transmission server 12 may also include a power source, a fan, a clock (CLK), and the like.

In order to implement the technical solution provided by the embodiment of the present application, as shown in FIG. 4, the terminal 13 in FIG. 1 may include: a communication interface 1301, a processor 1302, a memory 1303, a communication bus 1304, a player 1305, and a display screen 1306. The communication interface 1301, the processor 1302, the memory 1303, the player 1305, and the display 1306 are connected to each other through a communication bus 1304.

The communication interface 1303 is a unit for performing service data flow interaction between the terminal 13 and an external network element (such as the transmission server 12). Specifically, the communication interface 1303 may be configured to send a mainstream fragment request, a secondary stream encoding request, or an interrupt request to the transmission server 12, or receive a mainstream fragment, a secondary stream fragment, or a warehouse order file sent by the transmission server. The memory 1303 can be used to store data and/or code, and the processor 1302 can implement various functions of the terminal 13 by running or executing program code stored in the memory 1303, and calling data stored in the memory 1303.

It can be understood that the terminal 13 shown in FIG. 4 does not constitute a limitation on the terminal 13. In actual implementation, the terminal 13 may further include more components than the illustrated ones, or combine some components, or different component arrangements. For example, the terminal 13 may also include a power source, a fan, a clock, and the like.

In addition, the communication interface 1201 and the communication interface 1301 may each include a receiving unit and a sending unit, and may specifically be a transceiver or a transmission interface.

The processor 1202 and the processor 1302 may include a plurality of central processing units (CPUs) or include a plurality of network processing units (CPUs), which may be application specific integrated circuits (ASICs). Or one or more integrated circuits configured to implement embodiments of the present application, such as one or more digital signal processors (DSPs), or one or more field programmable gate arrays (field programmable gates) Array, FPGA), can also be a multi-core system-on-chip (SoC).

The memory 1203 and the memory 1303 may each be a volatile memory, such as a random-access memory (RAM), or a non-volatile memory, such as a read-only memory. A read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), or a combination of the above types of memories. The communication bus 1204 and the communication bus 1304 can be divided into an address bus, a data bus, a control bus, etc., and can be an Ethernet bus, an industry standard architecture (ISA) bus, or a peripheral component (PCI). Bus or extended industry standard architecture (EISA) bus. For ease of representation, each of the communication buses is represented by only one thick line in FIGS. 3 and 4, but does not mean that there is only one bus or one type of bus.

It should be noted that, in the foregoing, the encoding server 11 and the transmission server 12 are described as independent devices. In specific implementation, the two devices may also be integrated, and in addition, any of the two devices A device may be integrated with other network-side devices, which is not limited in this embodiment of the present application.

For convenience of description, the following embodiments show and describe in detail the video transmission method provided by the present invention, wherein the steps shown may also be in the transmission server 12 and the terminal shown in FIG. Executed in any computer other than 13. In addition, although the logical sequence of the video transmission method provided by the embodiment of the present application is shown in the method flowchart, in some cases, the steps shown or described may be performed in an order different from that herein.

It should be noted that the "server" described in the embodiments provided below may specifically be the transmission server 12 described above. "and/or" in this article is only an association relationship describing the associated objects, indicating that there may be three kinds of relationships, for example, A and / or B, which may indicate that A exists separately, and A and B exist simultaneously, respectively. B these three situations. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship. "Multiple" as used herein refers to two or more.

FIG. 5 is a schematic diagram of interaction of a video transmission method according to an embodiment of the present application. The method shown in FIG. 5 may include the following steps S101 to S104:

S101: The terminal acquires a warehouse order file of the video to be played from the server.

The video to be played may be any video requested by the terminal to the server. The video may include a primary encoded stream and a secondary encoded stream, the secondary encoded stream is used to increase the code rate of the primary encoded stream, the primary encoded stream includes at least two mainstream fragments, and the secondary encoded stream includes at least two auxiliary stream fragments, each mainstream The slice corresponds to one auxiliary stream slice. The generation process of the primary coded stream and the secondary coded stream and the fragmentation process thereof may be referred to the above, and are not described herein again.

The warehouse order file of the to-be-played video may include, but is not limited to, information of each of the at least two mainstream segments, and information of each of the at least two auxiliary stream segments. For example, the time point of each fragment (including the mainstream fragment and the auxiliary stream fragment), the correspondence between the mainstream fragment and the auxiliary stream fragment, and the like. The time point of each mainstream slice is the same as the time point of the corresponding auxiliary stream slice.

When the user needs to play a video, the terminal may send the indication information to the terminal, for example, by using a touch screen operation, text input, voice input, etc.; after detecting the indication information, the terminal determines that the terminal has been established with the server. If the connection is made, the warehouse file of the video to be played is requested from the server; if it is determined that the terminal does not establish a connection with the server, a connection is established with the server, and then the warehouse order file of the time-frequency to be played is requested from the server. . For the process of establishing a connection between the terminal and the server, and the specific implementation process of the terminal requesting the warehouse order file from the server, refer to the prior art, and details are not described herein again.

S102: The terminal parses the warehouse order file, obtains a time point of each fragment of the video to be played, a correspondence relationship between each mainstream fragment and the auxiliary stream fragment, and the like.

S103: The terminal obtains the mainstream fragment of the to-be-played video from the server, and obtains the auxiliary stream fragment corresponding to the mainstream fragment when the current real-time network status meets the preset condition; and caches each fragment obtained.

The terminal may determine, according to the detected indication information sent by the user, which mainstream fragment in the time frequency to be played by the server, and then the terminal may request the mainstream segment from the server after receiving the mainstream fragment replied by the server. The next mainstream slice of the slice, and so on, until the terminal requests the server for the last slice of the video to be played. For example, if the video to be played includes 10 mainstream fragments, the mainstream fragment corresponding to the playback start time indicated by the indication information sent by the user may be the first mainstream fragment, or may be the fifth mainstream segment. Film and so on. If the mainstream fragment corresponding to the indicated playback start time is the fifth mainstream fragment, the terminal may request the fifth mainstream fragment from the server, and after receiving the fifth mainstream fragment of the server reply, The sixth mainstream fragment is requested from the server, and so on, until the terminal requests the 10th mainstream fragment.

The current network status is not static, and it can change in real time as factors such as environment and load change. The terminal can periodically detect the current network status. When the detection period is small, it can be considered as detecting the current network status in real time. The detecting the current network status may include, but is not limited to, detecting at least one of the following parameters: network delay, network packet loss rate, and network fluctuation system. For the specific detection process, reference may be made to the prior art, and details are not described herein again.

It should be noted that, in the technical solution provided by the embodiment of the present application, the current real-time network status refers to the detected network status when the terminal performs the action of detecting the network status. That is to say, the meaning represented by the “current real-time network status” involved in the different steps of the embodiment of the present application needs to be determined according to actual implementation or actual context, and the same word in the different steps cannot be considered as “current real-time network”. "Status" indicates the network status measured at the same time. Similarly, "current play time" refers to the moment the terminal is playing.

The current real-time network condition satisfies the preset condition, which can be understood as the quality of the current network condition is good, that is, the communication quality is good. In this case, the terminal may request the auxiliary stream fragment from the server to superimpose and reconstruct the auxiliary stream fragment and the mainstream fragment, thereby improving the code rate of the mainstream fragment corresponding to the auxiliary stream fragment, thereby improving user viewing. Video experience. The current network network condition meets the preset condition, and may include at least one of the following: the current network delay is less than or equal to the first threshold, the current network packet loss rate is less than or equal to the second threshold, and the current network fluctuation coefficient is less than or equal to the third threshold. The specific values of the first threshold, the second threshold, and the third threshold, and how to obtain values, are not limited in this embodiment.

In the embodiment of the present application, the terminal may separately cache the mainstream fragment and the auxiliary stream fragment, for example, opening a specific two-part storage space in the terminal, where a part of the storage space is used for storing the mainstream fragment, and another part of the storage space is used for storing the mainstream fragment. Store auxiliary stream fragments. Alternatively, the terminal may integrate the cache mainstream fragment and the auxiliary stream fragment, for example, open a specific storage space in the terminal, and store the mainstream fragment and the auxiliary stream fragment in the storage space.

The process of acquiring a mainstream fragment and its corresponding auxiliary stream fragment is described below. As shown in FIG. 6, the method includes the following steps S11 to S16:

S11: The terminal sends a first mainstream fragment request to the server, where the first mainstream fragment request is used to request the first mainstream fragment.

The first mainstream fragment request may include an identifier of the first mainstream fragment, such as a time point of the first mainstream fragment, or an index of the first mainstream fragment. The first mainstream fragment may be any one of the at least two mainstream fragments, which may be, from the mainstream fragment corresponding to the playback start time indicated by the indication information to the last included in the video to be played. Any of the mainstream shards in a mainstream shard.

S12: The server receives the first mainstream fragment request sent by the terminal, and sends the first mainstream fragment to the terminal according to the first mainstream fragment request.

S13: The terminal receives the first mainstream fragment sent by the server, and caches the first mainstream fragment.

S14: If the current real-time network condition meets the preset condition, the terminal sends a first auxiliary stream fragmentation request to the server, where the first auxiliary stream fragmentation request is used to request the first auxiliary stream fragment corresponding to the first mainstream fragment. .

The first auxiliary stream fragmentation request may include an identifier of the first auxiliary stream fragment, such as a time point of the first auxiliary stream fragment, or an index of the first auxiliary stream fragment.

S15: The server receives the first auxiliary stream fragmentation request sent by the terminal, and sends the first auxiliary stream fragment to the terminal.

S16: The terminal receives the first auxiliary stream fragment sent by the server, and buffers the first auxiliary stream fragment.

S11 to S13 are processes for acquiring the first mainstream fragment, and S14 to S16 are processes for acquiring the first auxiliary stream fragment. The execution order of the S11 to S13 and the S14 to S16 is not limited in the embodiment of the present application, for example, S11 to S16 are executed again, and S14 to S16 are executed. S14 to S16 may be executed first, and S11 to S13 may be executed. S14 to S16 may be executed during the execution of S11 to S13. That is to say, the process of obtaining a mainstream fragment by the terminal and the process of acquiring the auxiliary stream fragment corresponding to the mainstream fragment may be in no particular order.

The following describes the process in which the terminal acquires multiple mainstream fragments and auxiliary stream fragments corresponding to each mainstream fragment.

For the terminal, it is generally used to distinguish the order of requesting the main stream fragments according to the time point; and, the order of the auxiliary stream fragments may be requested according to the time point, or the auxiliary stream points may not be requested according to the time point. The order of the films. The request sequence of the mainstream fragment and the auxiliary stream fragment (that is, the mainstream fragment and the auxiliary stream fragment corresponding to the mainstream fragment) at the same time point may not be limited.

For the server, it generally sends each mainstream fragment to the terminal according to the time point, and sends each auxiliary stream fragment to the terminal according to time. In the embodiment of the present application, the priority sequence of the mainstream fragment and the auxiliary stream fragment (that is, the mainstream fragment and the auxiliary stream fragment corresponding to the mainstream fragment) may be determined by setting a priority. . Optionally, the server and the terminal may be pre-agreed, or the priority between the mainstream fragment and the auxiliary stream fragment corresponding to the mainstream fragment is agreed by the server and the terminal. For example, use the HTTP/2 PRIORITY frame to set the priority. For example, when the terminal sends a mainstream fragment request and/or a secondary stream fragmentation request to the server, the terminal carries the priority of the requested mainstream fragment and/or the auxiliary stream fragment. Optionally, the mainstream shards can be played separately, and the auxiliary stream shards need to be played together with the mainstream shards. The priority of the mainstream shards is higher than the priority of the auxiliary shards corresponding to the mainstream shards. Level, so as to ensure that the mainstream shards can be transmitted preferentially, so as to ensure the continuity of the mainstream shards; that is, after the server sends a mainstream shard to the terminal, it will send the auxiliary stream corresponding to the mainstream shards. sheet.

Based on this, the process of acquiring the multiple mainstream fragments and the auxiliary stream fragments corresponding to each of the mainstream fragments, as shown in FIG. 7, may specifically include the following steps S21 to S24:

S21: The terminal acquires the i-th main stream fragment from the server, and caches the i-th main stream fragment.

The i-th main stream fragment refers to the i-th main stream fragment of the video to be played, i is an integer greater than or equal to 1, the initial value of i is 1, and the maximum value of i is the mainstream fragment included in the video to be played. quantity.

S22: The terminal determines whether to request the auxiliary stream fragment corresponding to the i-th mainstream fragment.

If yes, go to step S23, if no, go to step S24.

Specifically, the terminal determines that the i-th auxiliary stream fragment needs to be requested when the current real-time network status meets the preset condition and the i-th auxiliary stream fragment is not buffered; In the case of a preset condition, or the i-th auxiliary stream fragment has been cached, it is determined that the i-th auxiliary stream fragment is not required to be requested.

S23: The terminal acquires the auxiliary stream fragment corresponding to the i-th main stream fragment, and caches the auxiliary stream fragment.

After step S23 is performed, step S24 is performed.

S24: The terminal determines whether the i-th mainstream fragment is the last mainstream fragment to be played.

If yes, it ends. If not, i=i+1 is executed, and the process returns to step S21.

It can be understood that, after the terminal obtains the last mainstream fragment, before obtaining the auxiliary stream fragment corresponding to the mainstream fragment, the current real-time network status may not be determined whether the preset condition is met. In addition, after performing the foregoing S21 to S24, the terminal has acquired each mainstream fragment to be played, and auxiliary stream fragments corresponding to some or all of the mainstream fragments. Optionally, the method may further include: the terminal acquiring the uncached auxiliary stream fragment corresponding to the unplayed mainstream fragment. For example, if the mainstream shards to be played are sorted according to the time point, the main shards are 1 to 5, and the main shards 1 to 5 are associated with the auxiliary stream shards 1 to 5, then the mainstream shards obtained by the terminal and The sequence of the auxiliary stream fragmentation may be: mainstream fragment 1, auxiliary stream fragment 1, mainstream fragment 2, mainstream fragment 3, mainstream fragment 4, auxiliary stream fragment 4, mainstream fragment 5, auxiliary stream fragmentation. 5. Further, if the video content corresponding to the mainstream fragment 1 is being played at the current playing time, the terminal may further acquire the following auxiliary stream fragments: the auxiliary stream fragment 2 and the auxiliary stream fragment. 3.

It should be noted that, in the specific implementation, after the terminal obtains each mainstream fragment, if the current real-time network condition meets the preset condition, the terminal acquires the uncached first auxiliary stream fragment corresponding to the unplayed mainstream fragment. Until the last mainstream fragment is obtained, and then the uncached auxiliary stream fragment corresponding to the unplayed mainstream fragment is obtained. Alternatively, the terminal may obtain one or more uncached auxiliary stream fragments corresponding to the unplayed mainstream fragments after acquiring the multiple mainstream fragments, which is not limited in this embodiment of the present application.

S104: The terminal plays the video according to the cached slice.

After requesting each fragment (including the mainstream fragment and the auxiliary stream fragment), the terminal first caches the fragment, and after buffering to a certain number of mainstream fragments, starts to follow the time points of the mainstream fragment according to the time. The cached mainstream shard plays the video to be played to ensure the continuity of playback. It should be noted that, since the mainstream fragment and the auxiliary stream fragment obtained by the terminal are obtained by the server, the terminal needs to decode the mainstream fragment and the auxiliary stream fragment before playing. In addition, the embodiment of the present application does not limit the specific value and how to determine the value.

It can be understood that, for a video to be played that includes multiple mainstream segments, the terminal can play the cached video while acquiring the video, but for the same mainstream segment, the terminal first acquires the mainstream segment and then plays the video. The mainstream fragment that has been cached.

Based on this, the process of playing the video by the terminal, as shown in FIG. 8 , may specifically include the following steps S31 to S35:

S31: The terminal determines the jth mainstream fragment according to the time point of the mainstream fragmentation. The jth mainstream fragment indicates the mainstream fragment that the terminal is about to play recently. For example, if the terminal is currently playing the mainstream fragment 1, the mainstream fragment to be played by the terminal is the mainstream fragment 2.

The jth mainstream fragment refers to the jth mainstream fragment of the video to be played, j is an integer greater than or equal to 1, the initial value of j is 1, and the maximum value of j is the mainstream fragment included in the video to be played. quantity.

S32: The terminal determines whether the auxiliary stream fragment corresponding to the jth mainstream fragment is buffered.

If yes, go to step S33; if no, go to step S34.

S33: The terminal plays the jth mainstream fragment. That is, the video content corresponding to the jth mainstream tile is played.

After step S33 is performed, step S35 is performed.

S34: The terminal superimposes and reconstructs the jth mainstream fragment and the auxiliary stream fragment to obtain a composite fragment, and plays the composite fragment.

After step S34 is performed, step S35 is performed.

It can be understood that the video content corresponding to the video content played in step S33 is the same as the video content corresponding to the combined component played in step S34, except that the code rates of the two are different, and thus the playback quality is different.

Mainstream fragmentation and superposition auxiliary stream segmentation and reconstruction are the key steps in reconstructing video coding. Since the encoding server encodes the video content and generates the mainstream fragment, the code rate processing is performed, for example, the 4K video stream is subjected to a down-coding rate to generate a 2K video stream. Therefore, the resolution of the image frame obtained by the terminal after decoding the mainstream slice is low. Generally, after decoding the mainstream slice, the terminal performs super-resolution processing on the decoded image to re-rate the code rate to 4K; however, during the promotion process, the time-frequency quality of the original 4K video stream is compared. It will fall, so it can be corrected with the auxiliary code stream, such as replenishing the missing details, correcting the errors introduced during the encoding process, and so on.

S35: The terminal determines whether the jth mainstream fragment is the last mainstream fragment to be played.

If yes, it ends. If not, i=i+1 is executed, and the process returns to step S31.

For example, suppose that the video to be played includes 10 mainstream fragments, and each mainstream fragment corresponds to one auxiliary stream fragment. The mainstream fragment indicated by the current playback time is the fifth mainstream fragment, and the sixth to ten mainstream streams are cached. Fragmentation, and 6th to 7th auxiliary stream segmentation, a possible case is: playing the synthesized slice 6, the synthesized slice 7, and the 8th to 10th mainstream segments in sequence. The composite slice 6 is a composite slice obtained by superimposing and reconstructing the sixth mainstream slice and the sixth auxiliary stream slice, and the composite slice 7 is the seventh mainstream slice and the seventh auxiliary stream segment. The composite fragments obtained after the slices are superimposed and reconstructed.

In the video transmission method provided by the embodiment of the present application, the terminal requests the mainstream fragment, and requests the auxiliary stream fragmentation as needed; wherein the auxiliary stream fragment is used to improve the code rate of the mainstream fragment. In the embodiment of the present application, the primary encoded stream can be played separately, or the primary encoded stream and the secondary encoded stream are superimposed and reconstructed, so the buffered primary encoded stream does not lose its role due to buffering the secondary encoded stream. Compared with the prior art, it is possible to save transmission resources and cache resources.

It can be understood that one or more links can be established between the terminal and the server. In this embodiment, the mainstream fragment and the auxiliary stream fragment can be transmitted on two independent links or in the same chain. Transfer on the road. If the transmission is on the same link, both the terminal and the server need only maintain this link, so the implementation is simple. By way of example, the link here can be an HTTP/2 link. That is, when the terminal sends the primary fragment request and the secondary stream fragment request, the same HTTP/2 link is used, that is, the mainstream fragment and the auxiliary stream fragment use different HTTP/2 frames, and the two streams are multiplexed. Go to the same HTTP/2 link.

The foregoing S11 to S16 provide a process of acquiring the first mainstream fragment and the first auxiliary stream fragment. In the process, an interrupt request can also be set to interrupt the transmission process of the auxiliary stream fragment. As shown in FIG. 9, specifically, the following steps S41 to S47 may be included. In order to distinguish from the first mainstream fragment, the second mainstream fragment is used as an example for description. It can be understood that the second mainstream fragment is arbitrarily, that is, the terminal can interrupt the transmission process of any one of the auxiliary stream fragments in the time-frequency to be played.

S41: The terminal sends a second mainstream fragment request to the server, where the second mainstream fragment request is used to request the second mainstream fragment, and the second mainstream fragment is the first mainstream fragment in the at least two mainstream fragments. A mainstream shard outside.

S42: The server receives the second mainstream fragmentation request sent by the terminal, and sends a second mainstream fragment to the server according to the second mainstream fragmentation request.

S43: The terminal receives the second mainstream fragment sent by the server, and caches the second mainstream fragment.

S44: If the current real-time network condition meets the preset condition, the terminal sends a second auxiliary stream fragmentation request to the server, where the second auxiliary stream fragmentation request is used to request the second auxiliary stream fragment corresponding to the second mainstream fragment. .

For specific implementation manners of S41 to S44, refer to S11 to S14 in the foregoing.

S45: The server receives the second auxiliary stream fragmentation request sent by the terminal.

S46: If the terminal detects that the current real-time network condition does not meet the preset condition after sending the second auxiliary stream fragmentation request to the server, and before receiving the second auxiliary stream fragmentation, the terminal sends an interrupt request to the server, where the interrupt request is sent. Used to request the transmission process of interrupting the second auxiliary stream segmentation.

The terminal can detect the current network status in real time. Therefore, if the terminal sends the second auxiliary stream fragmentation request to the server and receives the second auxiliary stream fragment, the terminal detects that the current real-time network status does not meet the preset condition. In this case, the network is in a bad condition. If the server sends the second auxiliary stream fragment to the terminal, it may take a long time, which may cause the terminal to play a video in the process of playing the video. The technical solution for interrupting the request. The terminal request may be an RST_STREM frame in the HTTP/2, or may be a new frame, or another frame, which is not limited in this embodiment of the present application.

It can be understood that if the terminal detects that the current real-time network condition still meets the preset condition after sending the second auxiliary stream fragmentation request to the server and before receiving the second auxiliary stream fragmentation, the terminal does not send an interrupt request to the server, that is, The server will not receive an interrupt request. In this case, the server sends a second secondary stream fragmentation request to the terminal, and the terminal receives the second auxiliary stream fragment and caches the second auxiliary stream fragment.

S47: The server receives the interrupt request sent by the terminal, and if it is determined that the second auxiliary stream fragment has not been sent to the terminal, interrupts the transmission process of the second auxiliary stream fragment according to the interrupt request.

Specifically, after receiving the interrupt request, the server may not respond to the second auxiliary stream fragmentation request message, that is, the second auxiliary stream fragment is not sent to the terminal. If the server has sent the second auxiliary stream fragment to the terminal when receiving the interrupt request sent by the terminal, the server may discard the interrupt request.

It can be understood that after performing S41-S47, the terminal may not receive the second auxiliary stream fragment sent by the server, but the terminal has acquired the second mainstream fragment, and therefore, the terminal may continue to request the second mainstream fragment from the server. The next mainstream shard.

In the technical solution provided by the embodiment, after the terminal sends the auxiliary stream fragmentation request to the server and does not receive the requested auxiliary stream fragment, if the current real-time network status does not meet the preset condition, the terminal may trigger the interruption. The transmission process of the requested auxiliary stream fragment. In this way, the terminal can obtain the next mainstream fragment of the mainstream fragment corresponding to the auxiliary stream fragment to a certain extent, thereby effectively reducing the occurrence of a stuck phenomenon in the process of playing the video by the terminal.

The solution provided by the embodiment of the present application is mainly introduced from the perspective of interaction between the network elements. It can be understood that each network element, such as a terminal or a server. In order to implement the above functions, it includes hardware structures and/or software modules corresponding to the execution of the respective functions. Those skilled in the art will readily appreciate that the present application can be implemented in a combination of hardware or hardware and computer software in combination with the elements and algorithm steps of the various examples described in the embodiments disclosed herein. Whether a function is implemented in hardware or computer software to drive hardware depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present application.

The embodiment of the present application may divide a function module into a terminal or a server according to the foregoing method. For example, each function module may be divided according to each function, or two or more functions may be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of the module in the embodiment of the present application is schematic, and is only a logical function division, and the actual implementation may have another division manner.

For example, in the case of dividing each functional module by using corresponding functions, FIG. 10 shows a possible structural diagram of the terminal 100 involved in the above embodiment. The terminal 100 includes a transmitting unit 1401 and a receiving unit 1402. Optionally, the method further includes: a reconstruction unit 1403 and a playback unit 1404. The transmitting unit 1401 may be configured to perform actions performed by the terminal in S11 and S14 in FIG. 6, actions performed by the terminal in S41 and S46 in FIG. 9, and/or other processes for supporting the techniques described herein. The receiving unit may be used to perform actions performed by the terminal in S12 and S15 in FIG. 6, actions performed by the terminal in S42 in FIG. 9, and/or other processes for supporting the techniques described herein. Reconstruction unit 1403 can be used to perform the actions reconstructed in S34 of Figure 8, and/or other processes for supporting the techniques described herein. Play unit 1404 can be used to perform the actions played in S34 of Figure 8, and/or other processes for supporting the techniques described herein.

All the related content of the steps involved in the foregoing method embodiments may be referred to the functional descriptions of the corresponding functional modules, and details are not described herein again.

In the embodiment of the present application, the terminal is presented in a form corresponding to each function to divide each functional module, or the policy entity is presented in a form of dividing each functional module in an integrated manner. A "module" herein may refer to an Application-Specific Integrated Circuit (ASIC), a processor and memory that executes one or more software or firmware programs, integrated logic circuits, and/or other devices that provide the above functionality. . In a simple embodiment, those skilled in the art will appreciate that the terminal can take the form shown in FIG. For example, the sending unit 1401 provided above may be a transmitter, the receiving unit 1402 may be a receiver, and the transmitter may form a transceiver together with the receiver, which may specifically be the communication interface 1301 in FIG. In addition, the reconstruction unit 1403 may be embedded in the hardware or in a processor independent of the terminal (such as the processor 1302 in FIG. 4), or may be stored in the memory of the terminal in the form of software (such as the memory 1303 in FIG. 4). )in. Play unit 1404 can be player 1305 and/or display 1306 in FIG.

The terminal provided by the embodiment of the present application can be used to perform the above-mentioned video transmission method. Therefore, the technical effects of the present invention can be referred to the foregoing method embodiments.

For example, in the case of dividing each functional module by corresponding functions, FIG. 11 shows a possible structural diagram of the server 110 involved in the above embodiment. The server 110 may include a receiving unit 1501 and a transmitting unit 1502. Optionally, the method further includes: an interrupting unit 1503. The receiving unit 1501 may be configured to perform the actions performed by the server in S11 and S14 in FIG. 6, the actions performed by the server in S41 and S44 in FIG. 9, and/or other processes for supporting the techniques described herein. The transmitting unit 1502 may be configured to perform the actions performed by the server in S12 and S15 in FIG. 6, the actions performed by the server in S45 and S46 in FIG. 9, and/or other processes for supporting the techniques described herein. Interrupt unit 1503 can be used to perform the actions of S47 in Figure 9, and/or other processes for supporting the techniques described herein.

In the embodiment of the present application, the server is presented in a form corresponding to each functional module, or the server is presented in a form of dividing each functional module in an integrated manner. A "module" herein may refer to a particular ASIC circuit, a processor and memory that executes one or more software or firmware programs, integrated logic circuitry, and/or other devices that can provide the functionality described above. In a simple embodiment, those skilled in the art will appreciate that the terminal can take the form shown in FIG. For example, the sending unit 1502 provided above may be a transmitter, the receiving unit 1501 may be a receiver, and the transmitter may form a transceiver together with the receiver, which may specifically be the communication interface 1201 in FIG. In addition, the interrupting unit 1503 may be embedded in hardware or in a processor independent of the terminal (such as the processor 1202 in FIG. 3), or may be stored in software in the memory of the terminal (such as the memory 1203 in FIG. 3). in.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present application are generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be from a website site, computer, server or data center Transmission to another website site, computer, server or data center via wired (eg coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg infrared, wireless, microwave, etc.). The computer readable storage medium can be any available media that can be accessed by a computer or a data storage device that includes one or more servers, data centers, etc. that can be integrated with the media. The usable medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (such as a solid state disk (SSD)) or the like.

Although the present application has been described herein in connection with the various embodiments, those skilled in the art can Other variations of the disclosed embodiments are achieved. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill several of the functions recited in the claims. Certain measures are recited in mutually different dependent claims, but this does not mean that the measures are not combined to produce a good effect.

While the present invention has been described in connection with the specific embodiments and embodiments thereof, various modifications and combinations can be made without departing from the spirit and scope of the application. Accordingly, the description and drawings are to be regarded as It will be apparent to those skilled in the art that various modifications and changes can be made in the present application without departing from the spirit and scope of the application. Thus, it is intended that the present invention cover the modifications and variations of the present invention.

Claims

A video transmission method, wherein the video includes a primary encoded stream and a secondary encoded stream, where the secondary encoded stream is used to increase a code rate of the primary encoded stream, and the primary encoded stream includes at least two mainstream fragments. The secondary encoded stream includes at least two auxiliary stream fragments, and each mainstream fragment corresponds to one auxiliary stream fragment; the method includes:

Sending a first mainstream fragment request to the server, where the first mainstream fragment request is used to request a first mainstream fragment, and the first mainstream fragment is a mainstream part of the at least two mainstream fragments. sheet;

Receiving the first mainstream fragment sent by the server;

Sending a first auxiliary stream fragmentation request to the server, where the current real-time network status meets a preset condition, where the first auxiliary stream fragmentation request is used to request the first auxiliary corresponding to the first mainstream fragment Flow segmentation

Receiving the first auxiliary stream fragment sent by the server.
The method of claim 1 wherein

Receiving the first mainstream fragment sent by the server, including:

Receiving, by using a link, the first mainstream fragment sent by the server;

Receiving the first auxiliary stream fragment sent by the server, including:

Receiving, on the link, the first auxiliary stream fragment sent by the server.
The method according to claim 1 or 2, wherein the method further comprises:

And sending, by the server, a second mainstream fragment request, where the second mainstream fragment request is used to request a second mainstream fragment, where the second mainstream fragment is divided by the at least two mainstream fragments. a mainstream slice other than the first mainstream slice;

Receiving the second mainstream fragment sent by the server;

Sending a second auxiliary stream fragmentation request to the server, where the current real-time network status meets the preset condition, where the second auxiliary stream fragmentation request is used to request the corresponding part of the second mainstream fragment Second auxiliary stream segmentation;

Sending to the server after detecting that the current real-time network condition does not satisfy the preset condition after sending the second auxiliary stream fragmentation request to the server and before receiving the second auxiliary stream fragmentation An interrupt request, wherein the interrupt request is used to request to interrupt a transmission process of the second auxiliary stream fragment.
The method of claim 3, wherein the interrupt request comprises: an RST_STREM frame in HTTP/2.
The method according to any one of claims 1 to 3, wherein the method further comprises:

And superimposing and reconstructing the first mainstream fragment and the first auxiliary stream fragment, and playing the reconstructed code stream.
A video transmission method, wherein the video includes a primary encoded stream and a secondary encoded stream, the primary encoded stream includes at least two mainstream fragments, and the secondary encoded stream includes at least two auxiliary stream fragments, each The primary fragment corresponds to one auxiliary stream fragment; the method includes:

Receiving, by the terminal, a first mainstream fragment request, where the first mainstream fragment request is used to request a first mainstream fragment, and the first mainstream fragment is one of the at least two mainstream fragments. Fragmentation;

Sending the first mainstream fragment to the terminal;

Receiving a first auxiliary stream fragmentation request sent by the terminal, where the first auxiliary stream fragmentation request is used to request a first auxiliary stream fragment corresponding to the first mainstream fragment; the first auxiliary stream The fragmentation request is sent by the terminal if the current network condition satisfies a preset condition;

Sending the first auxiliary stream fragment to the terminal.
The method of claim 6 wherein:

The sending the first mainstream fragment to the terminal includes:

Transmitting the first mainstream fragment to the terminal on a link;

The sending the first auxiliary stream fragment to the terminal includes:

Transmitting the first auxiliary stream fragment to the terminal on the link.
The method according to claim 6 or 7, wherein the priority of the mainstream fragment is higher than the priority of the auxiliary stream fragment corresponding to the mainstream fragment; and the sending the first auxiliary stream to the terminal Fragmentation, including:

And if it is determined that the first mainstream fragment is sent to the terminal, sending the first auxiliary stream fragment to the terminal.
The method according to any one of claims 6 to 8, wherein the method further comprises:

Receiving a second mainstream fragment request sent by the terminal, where the second mainstream fragment request is used to request a second mainstream fragment, and the second mainstream fragment is in the at least two mainstream fragments. a mainstream slice other than the first mainstream slice;

Sending the second mainstream fragment to the terminal;

Receiving a second auxiliary stream fragmentation request sent by the terminal, where the second auxiliary stream fragmentation request is used to request a second auxiliary stream fragment corresponding to the second mainstream fragment;

If the interrupt request sent by the terminal is received before the second auxiliary stream fragmentation request is sent to the terminal, the transmission process of the second auxiliary stream fragment is interrupted.
The method of claim 9, wherein the interrupt request comprises: an RST_STREM frame in HTTP/2.
A terminal, wherein the video includes a primary encoded stream and a secondary encoded stream, where the secondary encoded stream is used to increase a code rate of the primary encoded stream, and the primary encoded stream includes at least two mainstream fragments, the secondary The encoded stream includes at least two auxiliary stream fragments, each mainstream fragment corresponding to one auxiliary stream fragment; the terminal includes: a sending unit and a receiving unit;

The sending unit is configured to send a first mainstream fragment request to the server, where the first mainstream fragment request is used to request a first mainstream fragment, and the first mainstream fragment is the at least two mainstream a mainstream slice in the slice;

The receiving unit is configured to receive the first mainstream fragment sent by the server;

The sending unit is further configured to send a first auxiliary stream fragmentation request to the server, if the current real-time network status meets a preset condition, where the first auxiliary stream fragment request is used to request the first The first auxiliary stream segment corresponding to the mainstream segment;

The receiving unit is further configured to receive the first auxiliary stream fragment sent by the server.
The terminal of claim 11 wherein:

The receiving unit is configured to receive the first mainstream fragment sent by the server on a link, and receive the first auxiliary stream fragment sent by the server on the link.
A terminal according to claim 11 or 12, characterized in that

The sending unit is further configured to send a second mainstream fragment request to the server, where the second mainstream fragment request is used to request a second mainstream fragment, and the second mainstream fragment is the at least a mainstream slice other than the first mainstream slice in the two mainstream fragments;

The receiving unit is further configured to receive the second mainstream fragment sent by the server;

The sending unit is further configured to: if the current real-time network condition meets the preset condition, send a second auxiliary stream fragmentation request to the server, where the second auxiliary stream fragmentation request is used to request the a second auxiliary stream segment corresponding to the second mainstream slice;

The sending unit is further configured to: if the current real-time network condition does not satisfy the preset after sending the second auxiliary stream fragmentation request to the server and before receiving the second auxiliary stream fragmentation And sending an interrupt request to the server, where the interrupt request is used to request to interrupt the transmission process of the second auxiliary stream segment.
The terminal according to claim 13, wherein the interrupt request comprises: an RST_STREM frame in HTTP/2.
The terminal according to any one of claims 11 to 14, wherein the terminal further comprises:

a reconstruction unit, configured to superimpose and reconstruct the first mainstream segment and the first auxiliary stream segment;

The playing unit is configured to play the code stream generated after the reconstruction.
A server, wherein the video includes a primary encoded stream and a secondary encoded stream, the primary encoded stream includes at least two mainstream fragments, and the secondary encoded stream includes at least two auxiliary stream fragments, each mainstream fragment Corresponding to one auxiliary stream segment; the server includes: a receiving unit and a sending unit;

The receiving unit is configured to receive a first mainstream fragment request sent by the terminal, where the first mainstream fragment request is used to request a first mainstream fragment, and the first mainstream fragment is the at least two a mainstream segment in the mainstream segment;

The sending unit is configured to send the first mainstream fragment to the terminal;

The receiving unit is further configured to receive a first auxiliary stream fragmentation request sent by the terminal, where the first auxiliary stream fragmentation request is used to request a first auxiliary stream corresponding to the first mainstream fragment The first auxiliary stream fragmentation request is sent by the terminal if the current network condition meets a preset condition;

The sending unit is further configured to send the first auxiliary stream fragment to the terminal.
A server according to claim 16 wherein:

The sending unit is specifically configured to: send the first mainstream fragment to the terminal on a link; and send the first auxiliary stream fragment to the terminal on the link.
The server according to claim 16 or 17, wherein the priority of the mainstream fragment is higher than the priority of the auxiliary stream fragment corresponding to the mainstream fragment;

The sending unit is specifically configured to: if it is determined that the first mainstream fragment is sent to the terminal, send the first auxiliary stream fragment to the terminal.
A server according to any one of claims 16 to 18, characterized in that

The receiving unit is further configured to receive a second mainstream fragment request sent by the terminal, where the second mainstream fragment request is used to request a second mainstream fragment, and the second mainstream fragment is the a mainstream slice other than the first mainstream slice in at least two mainstream fragments;

The sending unit is further configured to send the second mainstream fragment to the terminal;

The receiving unit is further configured to receive a second auxiliary stream fragmentation request sent by the terminal, where the second auxiliary stream fragmentation request is used to request a second auxiliary stream corresponding to the second mainstream fragment sheet;

The receiving unit is further configured to receive an interrupt request sent by the terminal;

The server further includes:

An interruption unit, configured to: if the receiving unit receives the interrupt request sent by the terminal before the sending unit sends the second auxiliary stream fragmentation request to the terminal, interrupting the second auxiliary The flow of stream fragmentation.
The server according to claim 19, wherein said interrupt request comprises: an RST_STREM frame in HTTP/2.
A terminal, comprising: a processor, a memory, a communication bus, and a communication interface;

The memory is configured to store a computer to execute an instruction, the processor is connected to the memory through the communication bus, and when the terminal is running, the processor executes the computer-executed instruction stored in the memory, so that The terminal performs the video transmission method according to any one of claims 1-5.
A server, comprising: a processor, a memory, a communication bus, and a communication interface;

The memory is configured to store a computer to execute instructions, the processor is coupled to the memory via the communication bus, and when the server is running, the processor executes the computer-executed instructions stored by the memory to enable The server performs the video transmission method according to any one of claims 6-10.