CN108833963B - Method, computer device, readable storage medium and system for displaying interface screen - Google Patents

Abstract

The embodiments of the present invention disclose a method, computer device, readable storage medium and system for displaying an interface picture, which belong to the technical field of the Internet. The method includes: the server can receive a target video stream sent by the first terminal, and the target video stream is composed of an interface screen currently displayed by the first terminal; determining a second terminal corresponding to the first terminal, and the second terminal is for the first terminal. A terminal performing remote control; sending a target video stream to the second terminal, where the target video stream is an unencapsulated video stream, which is used for the second terminal to display an interface picture. During the process that the second terminal remotely controls the first terminal, the second terminal obtains the webpage decoding script and the webpage drawing script; obtains the target video stream sent by the server; through the webpage decoding script and the webpage drawing script, based on the target video stream, display interface screen. By adopting the present invention, the delay of the display interface picture can be reduced.

Description

Method, computer device, readable storage medium and system for displaying interface screen

technical field

The present invention relates to the field of Internet technologies, and in particular, to a method, computer device, readable storage medium and system for displaying an interface picture.

Background technique

Currently, a terminal such as a mobile phone can be deployed in the cloud, so that the user can control the terminal (which may be referred to as the first terminal) in the cloud through the network. In the process of using the first terminal, the user can trigger a terminal such as a local computer (which may be referred to as the second terminal) to acquire the interface picture currently displayed by the first terminal through an operation, and display the acquired interface picture through the browser , and further, the first terminal can be operated on the second terminal through buttons on the screen, that is, the first terminal can be remotely controlled.

In order for the second terminal to smoothly display the interface picture of the first terminal, the interface picture is often obtained in the form of a video stream. At present, the second terminal generally uses a flash plug-in to display the obtained interface screen. In order to enable the second terminal to display the interface screen obtained in the form of a video stream through the flash plug-in, the video stream sent to the second terminal is often sent to the second terminal. is the encapsulated video stream. Specifically, the first terminal may generate a video stream (which may be referred to as a first video stream) according to the currently displayed interface screen, and further, may send a video stream to a server (wherein, the server may be deployed on the network side for the first terminal and the second terminal to communicate with each other). The server that the terminal communicates with) sends the generated first video stream. After receiving the first video stream, the server may perform RTMP (Real Time Messaging Protocol, Real Time Messaging Protocol) encapsulation on the first video stream to obtain a second video stream, and then send the second video stream to the second terminal. After receiving the second video stream, the second terminal can decode and play the second video stream through the installed flash plug-in, that is, an interface picture can be displayed.

In the process of realizing the present invention, the inventor found that the related art has at least the following problems:

Based on the above-mentioned processing method of the display interface, when the server encapsulates the first video stream, it often needs a certain amount of buffering, and after the flash decodes the video frames in the video stream, it also needs to buffer several video frames before displaying. , thus, the display delay of the interface picture is relatively large.

SUMMARY OF THE INVENTION

The embodiments of the present invention provide a method, a computer device, a readable storage medium and a system for displaying an interface picture, which can solve the problem of relatively large display delay of the interface picture in the related art. The technical solution is as follows:

In one aspect, a system for displaying an interface picture is provided, the system includes a server and a second terminal, wherein:

The server is configured to receive a target video stream sent by a first terminal, where the target video stream is composed of an interface screen currently displayed by the first terminal; determine the second terminal corresponding to the first terminal, the first terminal The second terminal is a terminal that remotely controls the first terminal; sends the target video stream to the second terminal, where the target video stream is an unencapsulated video stream for display by the second terminal the interface screen;

The second terminal is used to obtain a webpage decoding script and a webpage drawing script; in the process of remote control of the first terminal by the second terminal, the target video stream sent by the server is obtained; and the webpage drawing script to display the interface screen based on the target video stream.

In one aspect, a method for displaying an interface picture is provided, which is applied to a server, and the method includes:

receiving a target video stream sent by a first terminal, where the target video stream consists of an interface screen currently displayed by the first terminal;

determining a second terminal corresponding to the first terminal, where the second terminal is a terminal that remotely controls the first terminal;

Sending the target video stream to the second terminal, where the target video stream is an unencapsulated video stream, for the second terminal to display the interface picture.

In one aspect, a method for displaying an interface picture is provided, which is applied to a second terminal, and the method includes:

Obtain webpage decoding scripts and webpage drawing scripts;

During the process of remote control of the first terminal by the second terminal, the target video stream sent by the server is obtained, where the target video stream is composed of the interface screen currently displayed by the first terminal and is not encapsulated video stream;

The interface picture is displayed based on the target video stream by using the webpage decoding script and the webpage drawing script.

In one aspect, a device for displaying an interface picture is provided, the device comprising:

a first receiving module, configured to receive a target video stream sent by a first terminal, where the target video stream is composed of an interface screen currently displayed by the first terminal;

a determining module, configured to determine a second terminal corresponding to the first terminal, where the second terminal is a terminal that remotely controls the first terminal;

The first sending module is configured to send the target video stream to the second terminal, where the target video stream is an unencapsulated video stream for the second terminal to display the interface picture.

In one aspect, a device for displaying an interface picture is provided, the device comprising:

The first obtaining module is used to obtain the webpage decoding script and the webpage drawing script;

A second acquiring module, configured to acquire a target video stream sent by the server during the process of remote control of the first terminal by the second terminal, where the target video stream is composed of the interface screen currently displayed by the first terminal , and unencapsulated video streams;

The first display module is configured to display the interface picture based on the target video stream by using the webpage decoding script and the webpage drawing script.

In one aspect, a computer device is provided, the computer device includes a processor and a memory, the memory stores at least one instruction, at least one segment of a program, a code set or an instruction set, the at least one instruction, the at least one segment The program, the code set or the instruction set is loaded and executed by the processor to implement the method for displaying an interface screen as described above.

In one aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, at least one piece of program, code set or instruction set, the at least one instruction, the at least one piece of program, the code set Or the instruction set is loaded and executed by the processor to implement the method for displaying an interface picture as described above.

The beneficial effects brought by the technical solutions provided by the embodiments of the present invention include at least:

In the embodiment of the present invention, after the server receives the target video stream sent by the first terminal, it does not encapsulate the target video stream, and directly forwards it to the second terminal, so as to avoid the situation that the server buffers the video frame first. Therefore, the delay in acquiring the target video stream by the second terminal can be reduced, and further, the delay in displaying the interface picture can be reduced. In addition, in the embodiment of the present invention, after receiving the target video stream through the second terminal, the web page decoding script and the web page drawing script are used to display the interface picture based on the target video stream, so that after the second terminal decodes the interface picture, the interface picture can be decoded in time. The purpose of its display is to reduce the delay of displaying the interface picture.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

1 is a schematic diagram of a system framework provided by an embodiment of the present invention;

2 is a flowchart of a method for displaying an interface screen provided by an embodiment of the present invention;

3 is a schematic diagram of an interface provided by an embodiment of the present invention;

4 is a schematic diagram of an interface provided by an embodiment of the present invention;

5 is a schematic structural diagram of a device for displaying an interface screen provided by an embodiment of the present invention;

6 is a schematic structural diagram of a device for displaying an interface screen provided by an embodiment of the present invention;

7 is a schematic structural diagram of a device for displaying an interface screen provided by an embodiment of the present invention;

8 is a schematic structural diagram of a device for displaying an interface screen provided by an embodiment of the present invention;

9 is a schematic structural diagram of a device for displaying an interface screen provided by an embodiment of the present invention;

FIG. 10 is a schematic diagram of an apparatus structure of a terminal according to an embodiment of the present invention;

FIG. 11 is a schematic diagram of an apparatus structure of a server according to an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

An embodiment of the present invention provides a method for displaying an interface picture, and the method may be jointly implemented by the first terminal 101 , the server 102 and the second terminal 103 . The first terminal 101 may be an electronic device in the cloud, for example, a mobile device such as a mobile phone in the cloud, which may be used by a user to remotely control the first terminal through a local terminal, and may also be called a cloud real device. The server 102 may be a server used for communication between the first terminal 101 and the second terminal 103, may be a single server, or may be a server group composed of multiple servers, for example, may include a video server and a signaling server , wherein the video server can be used to transmit the target video stream of the first terminal, and the signaling server can be used to transmit the operation command of the second terminal to the first terminal. The second terminal 103 may be a local terminal, a terminal that remotely controls the first terminal, or a terminal installed with a browser, for example, a terminal such as a PC (personal computer) installed with a browser.

Server 102 may include components such as processors, memory, transceivers, and the like. The processor may be a CPU (Central Processing Unit, central processing unit), etc., and may be used to determine the related processing of the second terminal corresponding to the first terminal. The memory can be RAM (Random Access Memory), Flash (flash memory), etc., which can be used to store received data, data required for processing, data generated during processing, etc., such as terminal identification and account Correspondence of identification, etc. The transceiver can be used for data transmission with the terminal or other servers, for example, receiving the target video stream sent by the first terminal and sending the target video stream to the second terminal. The transceiver can include an antenna, a matching circuit, a modem, and the like.

The second terminal 103 may include components such as a processor, a memory, a transceiver, a screen, and the like. The processor may be a CPU or the like, and may be used for remote control of the first terminal and related processing of displaying an interface picture based on a target video stream. The memory can be RAM, Flash, etc., and can be used to store received data, data required for processing, and data generated during processing, such as target video streams, webpage decoding scripts, and webpage rendering scripts. The screen can be a display, which can be used to display a target web page, a selection interface, etc., and the screen can also be a touch screen, in which case, it can also be used to detect touch signals and the like. The transceiver can be used for data transmission with other devices, for example, receiving the target video stream sent by the server, and can include antennas, matching circuits, modems, and the like.

In order to enable users to use various types of terminals without purchasing corresponding terminals, various types of terminals can be deployed in the cloud, so that users can control the terminals in the cloud through the network. The terminal may be referred to as the first terminal. For example, a terminal in the cloud can be used by business developers. During the development process, developers can develop services applicable to multiple operating systems (for example, developing services applicable to Android operating systems, and ios operating systems). After the corresponding service is developed, the developed service can be tested through the corresponding terminal in the cloud to find errors in the development process, and the user experience when the service is running in the real terminal can be predicted, so as to optimize the developed service. In this way, developers do not need to purchase terminals, and can complete development work through terminals in the cloud, thereby saving development costs.

In the process of using the first terminal, the user can trigger a terminal such as a local computer (which may be referred to as the second terminal) to acquire the interface picture currently displayed by the first terminal through an operation, and display the acquired interface picture through the browser , and further, the first terminal can be operated on the second terminal through buttons on the screen, that is, the first terminal can be remotely controlled. Currently, the interface picture currently displayed by the first terminal may be acquired in different ways, and the acquired interface picture is displayed. For example, the interface picture currently displayed by the first terminal may be acquired in the form of an image (this method may be referred to as mode 1), and the interface picture may be displayed, or the interface picture currently displayed by the first terminal may be acquired in the form of a video stream The interface picture (this method can be called the second method), and the interface picture is displayed.

The specific processing process of Manner 1 may be as follows: the second terminal may obtain the interface picture currently displayed by the first terminal from the server based on a preset period, or the server may automatically push the interface picture currently displayed by the first terminal to the second terminal. After acquiring the interface picture currently displayed by the first terminal, the second terminal can create a local temporary object URL through URL.createObjectURL(), set the src source of an interface picture as the URL of the temporary picture object, and further, can use the interface picture It is drawn into the Canvas canvas to realize the display of the interface picture currently displayed by the first terminal. Mode 1 is used to display the interface picture currently displayed by the first terminal. The data volume of the interface picture sent by the server is relatively large, generally 100KB (kilobytes). If the acquisition rate is 25 interface pictures per second, it takes 2.5 The network bandwidth of MB/s (megabytes/second) can display the interface screen smoothly. It can be seen that this method has higher requirements on the network and occupies a large bandwidth. If the network cannot meet the above conditions, the first The second terminal cannot display the interface picture smoothly, thereby causing a large delay in displaying the interface picture.

Since the image is compressed and encoded before the video stream is generated, so that the second terminal can smoothly display the interface picture of the first terminal, the interface picture can be obtained in the form of a video stream, that is, the second method does not need to be too large. the network bandwidth and occupy less bandwidth. At present, after obtaining the video stream, the browser of the second terminal can decode and play the video stream through the installed flash plug-in, and the flash plug-in can play the video stream in a specific form, such as RTMP video stream (that is, for the first video stream). The video stream generated by a terminal is RTMP encapsulated video stream) and H264 video stream (that is, the video stream compressed by the intra-frame-inter-frame compression algorithm, or the H265 video stream) for playback. All video streams require the server to perform encapsulation processing on the video stream generated by the first terminal. The form of the video stream is different, and the specific processing process of Mode 2 is different. For the case where the video stream is an RTMP video stream, the specific processing process of Mode 2 may be as follows: The first terminal may generate a video stream (which may be referred to as the currently displayed interface picture) the first video stream), and further, the generated first video stream may be sent to a server (wherein, the server may be a server deployed on the network side for communication between the first terminal and the second terminal). After receiving the first video stream, the server may perform RTMP (Real Time Messaging Protocol, Real Time Messaging Protocol) encapsulation on the first video stream to obtain a second video stream, and then send the second video stream to the second terminal. After receiving the second video stream, the second terminal can decode and play the second video stream through the installed flash plug-in, that is, an interface picture can be displayed. In this way, based on the above-mentioned processing method of displaying the interface picture, the server (in this case, the server may also be called a streaming media server) often needs a certain amount of buffering when encapsulating the first video stream, and the flash decodes the video stream into the video stream. After the video frames in the , it is also necessary to buffer several video frames before displaying them, which results in a relatively large delay in the display of the interface screen.

For the case where the video stream is an H264 video stream, the specific processing process of the second method may be as follows: the first terminal may generate a video stream (which may be referred to as a first video stream) according to the currently displayed interface picture, and further, may send a message to the server (wherein, The server may be a server deployed on the network side for communication between the first terminal and the second terminal) to send the generated first video stream. After receiving the video stream, the server can perform FLV (wherein FLV is a video format) file header encapsulation to obtain a second video stream, and then send the second video stream to the second terminal. After receiving the second video stream, the second terminal can decode and play the second video stream through the installed flash plug-in, that is, an interface picture can be displayed. In this way, based on the above-mentioned processing method of the display interface picture, the server needs to encapsulate the first video stream, which consumes a certain market, and after the flash decodes the video frames in the video stream, it also needs to cache several video frames before displaying. , thus, the display delay of the interface picture is relatively large. It can be seen that, using the second method to obtain the interface picture currently displayed by the first terminal, when the user wants to remotely control the first terminal through the second terminal, the flash plug-in needs to be installed first. In addition, the delay in displaying the interface picture is relatively large.

However, in this solution, after receiving the target video stream sent by the first terminal, the server can determine the second terminal that currently controls the first terminal remotely, and further, can directly send the target video sent by the first terminal to the second terminal stream, where the target video stream is an unencapsulated video stream. The second terminal may acquire the webpage decoding script and the webpage drawing script in the process of remotely controlling the first terminal. After acquiring the target video stream sent by the server, the interface picture currently displayed by the first terminal may be displayed based on the acquired target video stream by using the acquired webpage decoding script and webpage drawing script. In this way, after receiving the target video stream, the server directly forwards it to the second terminal without encapsulation processing, which can ensure that the second terminal can obtain the target video stream in time. In addition, the second terminal parses the webpage decoding script and the webpage drawing script through the browser, and after decoding the interface picture in the target video stream, the interface picture can be directly displayed without buffering the interface picture, thereby reducing the cost of displaying the interface picture. Delay.

The processing flow shown in FIG. 2 will be described in detail below in conjunction with the specific embodiments, and the content may be as follows:

Step 201, the second terminal displays a selection interface, and the selection interface includes options of multiple terminals.

In implementation, a browser may be installed in the second terminal. When the user wants to use the terminal in the cloud (that is, he wants to remotely control the terminal in the cloud), he can trigger the second terminal to open the browser through an operation, and then, The user can enter the website address of the selection interface in the address bar of the browser to trigger the second terminal to receive the acquisition instruction. After receiving the obtaining instruction, the second terminal may obtain the selection interface from the server (wherein, the second terminal may establish a Websocket connection with the server, and Websocket is a transmission protocol), and further, may display the selection interface, wherein the selection interface It includes options for multiple terminals deployed in the cloud, and each option includes the introduction information of the corresponding terminal, as shown in Figure 3.

Step 202, the second terminal obtains the webpage decoding script and the webpage drawing script.

In implementation, after receiving the obtaining instruction, the second terminal may obtain the webpage decoding script and the webpage drawing script in addition to obtaining the selection interface from the server, wherein the webpage decoding script and the webpage drawing script may be scripts used for browser parsing, Can be a JS file.

Optionally, the webpage decoding script and the webpage drawing script can be obtained from the server. Correspondingly, the processing procedure of step 202 can be as follows: when the display instruction for the target webpage is obtained, the webpage decoding script and the webpage drawing script are downloaded from the server.

In implementation, after the second terminal receives the acquisition instruction, it will display multiple web pages, for example, a web page including a selection interface, or a video stream display web page, etc. may be displayed. Before displaying the web page, the second terminal will receive a display instruction for the web page (may be called a target web page), and at this time, the second terminal may download the web page decoding script and the web page drawing script from the server. For example, when the target web page is a web page including the above selection interface, after receiving the obtaining instruction for the selection interface, the second terminal can download the web page decoding script and the web page drawing script from the server.

It should be noted that, based on the difference in triggering mechanisms for acquiring the webpage decoding script and the webpage drawing script, the processing sequence of step 202 has no sequential relationship with the processing sequence of step 201 and step 203 .

Step 203: When receiving the selection instruction for the first terminal among the multiple terminals, the second terminal acquires the target account identifier of the current login account and the target terminal identifier of the first terminal.

In the implementation, after the selection interface is displayed, the user can select a corresponding terminal according to his own needs. At this time, the second terminal can receive a selection instruction for a terminal (ie, the first terminal) among the multiple terminals, and further, the first terminal The second terminal can acquire the currently logged-in account identifier (that is, the target account identifier, which can be represented by UserID) and the terminal identifier of the first terminal (that is, the target terminal identifier, which can be represented by DeviceID).

Step 204, the second terminal encrypts the target account identifier and the target terminal identifier to obtain the encrypted target account identifier and the target terminal identifier.

In implementation, after acquiring the target account identifier and the target terminal identifier, the second terminal can encrypt them to obtain the encrypted target account identifier and the target terminal identifier, wherein the target account identifier and the target terminal identifier can be encrypted by a symmetric encryption algorithm. The ID is encrypted. Specifically, after obtaining the target account identifier and the target terminal identifier, the second terminal can encrypt the target account identifier and the target terminal identifier through Base64Encode (Base64Encode is an encryption method), and obtain the encrypted target terminal identifier and target account identifier. , which can be called a key, and further, the obtained key can be set into a cookie, wherein the key in the cookie can be as follows: Base64Encode(DeviceID+UserID).

Step 205, the second terminal sends the encrypted target account identifier and target terminal identifier to the server.

In implementation, after obtaining the encrypted target account identifier and target terminal identifier, the second terminal may send the encrypted target account identifier and target terminal identifier to the server. In the case where the second terminal sets the encrypted target account identifier and the target terminal identifier as data of the field key in the cookie, the second terminal may send the cookie to the server.

Correspondingly, after the second terminal sends the encrypted target account identifier and target terminal identifier to the server, the server may receive the encrypted target account identifier and target terminal identifier sent by the second terminal.

Step 206, the server decrypts the encrypted target account identifier and target terminal identifier to obtain the target account identifier and the target terminal identifier.

In implementation, after receiving the encrypted target account identifier and target terminal identifier sent by the second terminal, the server may decrypt them to obtain the target account identifier and the target terminal identifier. In the case where the second terminal encrypts the target account identifier and the target terminal identifier through Base64Encode, the server can decrypt the encrypted target account identifier and the target terminal identifier through Base64Decode to obtain the target account identifier and the target terminal identifier. In addition, in the case where the second terminal sets the encrypted target account identifier and target terminal identifier as the data of the field key in the cookie, and sends the cookie to the server, the server can read the data of the field key in the cookie, and read the data of the field key in the cookie. The received data is decrypted to obtain the target account ID and target terminal ID.

Step 207, the server performs verification processing on the target account identifier and the target terminal identifier, and if the verification is successful, sends a video stream acquisition request to the first terminal corresponding to the target terminal identifier, and the video stream acquisition request is used to request the target terminal from the first terminal. video stream.

In implementation, after obtaining the target account identifier and the target terminal identifier, the server may perform verification processing on them. If the verification is successful, the server may send a video acquisition request to the first terminal corresponding to the target terminal identifier. The terminal can be remotely controlled by a second terminal, that is, if the first terminal is not remotely controlled, a video stream acquisition request is sent to the first terminal corresponding to the target terminal identifier, and the second terminal enters the first terminal. The process of remote control. In addition, when the second terminal receives the selection instruction for the first terminal, in addition to sending the target account identifier and the target terminal identifier to the server, it can also send the target account identifier and the target terminal identifier to other servers. In this case, the server After obtaining the target account identifier and the target terminal identifier, the terminal identifier corresponding to the target account identifier can be obtained from other servers. If the terminal identifier obtained from the other server is consistent with the target terminal identifier obtained by the server, the verification is successful.

Correspondingly, after the server sends the video stream acquisition request to the first terminal, the first terminal may receive the video stream acquisition request sent by the server.

Optionally, after the verification is successful, the server may also store the target account identifier and the target terminal identifier correspondingly, and correspondingly, may also perform the following processing: store the target account identifier and the target terminal identifier in correspondence with the account identifier and the terminal identifier. in relationship.

In implementation, the server may store a correspondence between the account identifier and the terminal identifier, where the terminal identifier may be the identifier of the terminal deployed in the cloud. After successfully verifying the target account identifier and the target terminal identifier, the server may store the target account identifier and the target terminal identifier in the corresponding relationship between the account identifier and the terminal identifier, so that after receiving the target video stream sent by the first terminal, determine A second terminal that remotely controls the first terminal.

Step 208: The first terminal generates a target video stream according to the currently displayed interface picture.

In implementation, after receiving the video stream acquisition request sent by the server, the first terminal can light up the screen, and generate a video stream (which may be called a target video stream) based on the currently displayed interface screen. When a terminal displays a screen desktop, the first frame of the video stream generated by the first terminal is the screen desktop, as shown in FIG. 4 . Wherein, the first terminal may acquire the currently displayed interface picture by recording a screen, and generate a video stream based on the acquired interface picture.

After generating the target video stream, the first terminal may send the target video stream to the server.

Step 209: The server receives the target video stream sent by the first terminal, where the target video stream is composed of the interface picture currently displayed by the first terminal.

In implementation, after the first terminal sends the target video stream to the server, the server may receive the target video stream sent by the first terminal.

In step 210, the server determines a second terminal corresponding to the first terminal, and the second terminal is a terminal that remotely controls the first terminal.

In the implementation, after receiving the target video stream sent by the first terminal, the server may determine the second terminal that is currently remotely controlling the first terminal.

Optionally, for a situation where the server stores a corresponding relationship between the account identifier and the terminal identifier, correspondingly, the processing procedure of step 210 may be as follows: according to the pre-stored corresponding relationship, determine the second terminal corresponding to the first terminal.

In implementation, after receiving the target video stream sent by the first terminal, the server can determine the target account identifier corresponding to the target terminal identifier of the first terminal in the correspondence between the account identifier and the terminal identifier, and further, the target account identifier can be The terminal corresponding to the identification is determined to be the second terminal.

Step 211: The server sends a target video stream to the second terminal, where the target video stream is an unencapsulated video stream, which is used for the second terminal to display an interface picture.

In implementation, after determining the second terminal, the server may send the target video stream sent by the first terminal to the second terminal, where the target video stream is an unencapsulated video stream, and the server may send the target video stream to the second terminal through Websocket target video stream. That is to say, after receiving the target video stream sent by the first terminal, the server may directly forward it to the second terminal without processing it. In this way, the server may not buffer the video frames, and further, it may be ensured that the second terminal can obtain the target video stream in time.

Optionally, the server may send the target video stream to the second terminal in a preset format. Correspondingly, the processing process of step 211 may be as follows: send data in a preset format to the second terminal, and the data includes the target data type and the target video stream. , the target data type is the data type corresponding to the video stream.

In implementation, whenever the server sends data corresponding to the first terminal to the second terminal, it may send data in a preset format to the second terminal, where the format may be: Len+Type+Data, where Len is 4 bytes Integer, network byte order, used to indicate the length of the entire data packet, Type is an integer of 1 byte, indicating the type of the current data packet, Data is specific data, such as a target video stream. In this case, after receiving the target video stream, the server can send data in a preset format to the second terminal, where Type is the target data type (for example, it can be 0), Data is the target video stream, and the target data type is The data type corresponding to the video stream.

Step 212, the second terminal acquires the target video stream sent by the server in the process that the second terminal remotely controls the first terminal, and the target video stream is composed of the interface picture currently displayed by the first terminal and has not been encapsulated. video stream.

In implementation, after receiving the video acquisition request, the first terminal may continue to generate a target video stream based on the currently displayed interface screen, and continue to send the target video stream to the server. Correspondingly, the second terminal can continuously receive and acquire the target video stream sent by the server, where the target video stream is an unencapsulated video stream composed of the interface screen currently displayed by the first terminal.

Optionally, the second terminal can use the data type carried in the data sent by the server to determine whether the currently received data is the target video stream, and correspondingly, receive the data in the preset format sent by the server; if the data includes the data type is the target data type, the target video stream included in the data is obtained, and the target data type is the data type corresponding to the video stream.

In implementation, for a situation where the server sends data in a preset format to the second terminal, after receiving the data in the preset format sent by the server, the second terminal can determine the type of data included in the data, if the type of data included in the data is If the target data type is selected, the target video stream included in the data can be obtained, and the target data type is the data type corresponding to the video stream.

Step 213, the second terminal displays an interface picture based on the target video stream through the webpage decoding script and the webpage drawing script.

In implementation, after acquiring the target video stream, the second terminal may process the target video stream by using the acquired webpage decoding script and webpage drawing script, and display the interface picture currently displayed by the first terminal. In addition, when the second terminal receives the selection instruction for the first terminal, in addition to the above processing, it can also display a web page for displaying the interface screen. In this case, the second terminal can display the acquired interface screen in the web page is displayed.

Optionally, the second terminal can decode the target video stream through a webpage decoding script, and the webpage drawing script is used to display the interface picture obtained by decoding. Correspondingly, the processing process of step 206 can be as follows: through the webpage decoding script, the target video stream is decoded by the webpage decoding script. The stream is decoded to obtain the picture data of the interface picture; the script is drawn through the webpage, and the interface picture is displayed based on the picture data of the interface picture.

In implementation, after acquiring the target video stream, the second terminal can decode each video frame in the target video stream (wherein each video frame is the interface picture of the first terminal) through the webpage decoding script to obtain the interface picture The picture data, wherein the picture data may include the image data of the interface picture, the width and height of the image. After obtaining the picture data, the second terminal can display the interface picture on the canvas through the webpage drawing script, and can also adjust the size of the canvas based on the width and height of the image.

Optionally, for the case where the target video stream is a video stream compressed by the video frame through the intra-frame compression algorithm, the second terminal may store the determined network transmission units in the process of receiving the target video stream. In the cache, correspondingly, the processing process can be as follows: in the process of acquiring the target video stream, whenever a data packet containing the start identification of the network transmission unit is acquired, the data packet is sent to the next data packet containing the start identification of the network transmission unit The data packets between them are determined as complete network transmission units; the acquired network transmission units are stored in the cache; each network transmission unit in the cache is decoded through the webpage decoding script to obtain the picture data of the interface screen.

Compared with the solution that the second terminal uses the flash plug-in to play the video stream, using this solution, the delay of the display interface screen is relatively low, there is no buffering of video frames, and the network requirements are relatively low. If the network conditions are good, the delay will be Can be controlled within 50 milliseconds.

In implementation, in the process of acquiring the target video stream, it can be determined whether a complete network transmission unit (NAL) has been acquired based on the received data packets. If it is determined that a complete NAL has been acquired, the second terminal can NALs are stored into the cache. The first data packet corresponding to each NAL may contain a NAL start identifier. In this case, the second terminal may acquire the data containing the network transmission unit start identifier during the process of acquiring the target video stream. packet, the second terminal may determine the data packet between the data packet and the next data packet containing the start identifier of the network transmission unit as a complete NAL, and further, may store the complete NAL in the cache, where one or multiple NALs corresponding to a video frame. Subsequently, the second terminal may decode each network transmission unit in the cache through the webpage decoding script to obtain picture data of the interface picture.

In the embodiment of the present invention, after the server receives the target video stream sent by the first terminal, it does not encapsulate the target video stream, and directly forwards it to the second terminal, so as to avoid the situation that the server buffers the video frame first. Therefore, the delay in acquiring the target video stream by the second terminal can be reduced, and further, the delay in displaying the interface picture can be reduced. In addition, in the embodiment of the present invention, after receiving the target video stream through the second terminal, the web page decoding script and the web page drawing script are used to display the interface picture based on the target video stream, so that after the second terminal decodes the interface picture, the interface picture can be decoded in time. The purpose of its display is to reduce the delay of displaying the interface picture.

Based on the same technical concept, an embodiment of the present invention also provides a device for displaying an interface picture, and the device may be the above-mentioned server. As shown in FIG. 5 , the device includes:

a first receiving module 510, configured to receive a target video stream sent by a first terminal, where the target video stream is composed of an interface screen currently displayed by the first terminal;

a determining module 520, configured to determine a second terminal corresponding to the first terminal, where the second terminal is a terminal that performs remote control on the first terminal;

The first sending module 530 is configured to send the target video stream to the second terminal, where the target video stream is an unencapsulated video stream, for the second terminal to display the interface picture.

Optionally, the first sending module 530 is configured to:

Sending data in a preset format to the second terminal, the data includes a target data type and the target video stream, and the target data type is a data type corresponding to the video stream.

Optionally, as shown in Figure 6, the device further includes:

The second receiving module 540 is configured to receive the encrypted target account identifier and target device identifier sent by the second terminal;

Decryption module 550, for decrypting the encrypted target account identification and target device identification to obtain the target account identification and the target device identification;

The second sending module 560 is configured to perform verification processing on the target account identifier and the target device identifier, and if the verification is successful, send a video stream acquisition request to the first terminal corresponding to the target device identifier, and the The video stream acquisition request is used to request the target video stream from the first terminal.

Optionally, as shown in Figure 7, the device further includes:

A storage module 570, configured to store the target account identifier and the target device identifier in the corresponding relationship between the account identifier and the device identifier;

The determining module 520 is used for:

A second terminal corresponding to the first terminal is determined according to the pre-stored correspondence.

Based on the same technical concept, an embodiment of the present invention further provides a device for displaying an interface picture, which may be the above-mentioned second terminal. As shown in FIG. 8 , the device includes:

The first obtaining module 810 is used to obtain the webpage decoding script and the webpage drawing script;

The second acquiring module 820 is configured to acquire a target video stream sent by the server during the process of remote control of the first terminal by the second terminal, where the target video stream is an interface screen currently displayed by the first terminal composed, unencapsulated video streams;

The first display module 830 is configured to display the interface picture based on the target video stream by using the webpage decoding script and the webpage drawing script.

Optionally, the first obtaining module 810 is used for:

When the display instruction for the target web page is acquired, the target web page decoding script and the target web page drawing script are downloaded to the server.

Optionally, the second obtaining module 820 is used for:

During the process that the second terminal remotely controls the first terminal, receiving data in a preset format sent by a server;

If the data type included in the data is the target data type, the target video stream included in the data is acquired, and the target data type is the data type corresponding to the video stream.

Optionally, the first display module 830 is used for:

Decode the target video stream through the webpage decoding script to obtain picture data of the interface picture;

The interface picture is displayed based on the picture data of the interface picture through the webpage drawing script.

Optionally, the target video stream is a video stream compressed by a video frame through an intra-frame compression algorithm;

The first display module 830 is used for:

In the process of acquiring the target video stream, whenever a data packet containing the start identifier of the network transmission unit is acquired, the data packet between the data packet and the next data packet containing the start identifier of the network transmission unit is determined. It is a complete network transmission unit;

Store the acquired network transmission unit in the cache;

Through the webpage decoding script, each network transmission unit in the cache is decoded to obtain the picture data of the interface picture.

Optionally, as shown in Figure 9, the device further includes:

The second display module 840 is configured to display a selection interface, where the selection interface displays options of multiple terminals;

The third obtaining module 850 is configured to obtain the target account identifier of the current login account and the target device identifier of the first terminal when receiving the selection instruction for the first terminal in the plurality of terminals;

An encryption module 860, configured to encrypt the target account identification and the target device identification to obtain the encrypted target account identification and the target device identification;

A sending module 870, configured to send the encrypted target account identifier and target device identifier to the server.

In the embodiment of the present invention, after the server receives the target video stream sent by the first terminal, it does not encapsulate the target video stream, and directly forwards it to the second terminal, so as to avoid the situation that the server buffers the video frame first. Therefore, the delay in acquiring the target video stream by the second terminal can be reduced, and further, the delay in displaying the interface picture can be reduced. In addition, in the embodiment of the present invention, after receiving the target video stream through the second terminal, the web page decoding script and the web page drawing script are used to display the interface picture based on the target video stream, so that after the second terminal decodes the interface picture, the interface picture can be decoded in time. The purpose of its display is to reduce the delay of displaying the interface picture.

It should be noted that: when the device for displaying an interface picture provided by the above-mentioned embodiment displays an interface picture, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions may be allocated to different functional modules as required. To complete, that is, to divide the internal structures of the server and the second terminal into different functional modules, so as to complete all or part of the functions described above. In addition, the apparatus for displaying an interface picture provided by the above-mentioned embodiments and the method for displaying an interface picture belong to the same concept, and the specific implementation process thereof is detailed in the method embodiment, which will not be repeated here.

Based on the same technical concept, an embodiment of the present invention further provides a system for displaying an interface picture, the system includes a server and a second terminal, wherein:

The server is configured to receive a target video stream sent by a first terminal, where the target video stream is composed of an interface screen currently displayed by the first terminal; determine the second terminal corresponding to the first terminal, the first terminal The second terminal is a terminal that remotely controls the first terminal; sends the target video stream to the second terminal, where the target video stream is an unencapsulated video stream for display by the second terminal the interface screen;

The second terminal is used to obtain a webpage decoding script and a webpage drawing script; in the process of remote control of the first terminal by the second terminal, the target video stream sent by the server is obtained; and the webpage drawing script to display the interface screen based on the target video stream.

Optionally, the server is used for:

Sending data in a preset format to the second terminal, the data includes a target data type and the target video stream, and the target data type is a data type corresponding to the video stream;

The second terminal is used for:

During the process that the second terminal remotely controls the first terminal, receiving data in a preset format sent by a server;

If the data type included in the data is the target data type, the target video stream included in the data is acquired, and the target data type is the data type corresponding to the video stream.

Optionally, the second terminal is also used for:

displaying a selection interface, where the selection interface includes options of multiple terminals;

When receiving the selection instruction for the first terminal in the plurality of terminals, acquiring the target account identifier of the current login account and the target terminal identifier of the first terminal;

Encrypting the target account identification and the target terminal identification to obtain the encrypted target account identification and the target terminal identification;

sending the encrypted target account identifier and target terminal identifier to the server;

The server is also used to:

receiving the encrypted target account identifier and target terminal identifier sent by the second terminal;

Decrypting the encrypted target account identification and target terminal identification to obtain the target account identification and the target terminal identification;

Verification processing is performed on the target account identifier and the target terminal identifier, and if the verification is successful, a video stream acquisition request is sent to the first terminal corresponding to the target terminal identifier, and the video stream acquisition request is used to The first terminal requests the target video stream.

Optionally, the server is also used for:

correspondingly storing the target account identification and the target terminal identification in the corresponding relationship between the account identification and the terminal identification;

the server for:

A second terminal corresponding to the first terminal is determined according to the pre-stored correspondence.

Optionally, the second terminal is used for:

When the display instruction for the target web page is acquired, the web page decoding script and the web page drawing script are downloaded from the server.

Optionally, the second terminal is used for:

Decode the target video stream through the webpage decoding script to obtain the picture data of the interface picture;

The interface picture is displayed based on the picture data of the interface picture through the webpage drawing script.

Optionally, the target video stream is a video stream compressed by a video frame through an intra-frame compression algorithm;

The second terminal is used for:

In the process of acquiring the target video stream, whenever a data packet containing the start identifier of the network transmission unit is acquired, the data packet between the data packet and the next data packet containing the start identifier of the network transmission unit is determined. It is a complete network transmission unit;

Store the acquired network transmission unit in the cache;

Through the webpage decoding script, each network transmission unit in the cache is decoded to obtain the picture data of the interface picture.

In the embodiment of the present invention, after the server receives the target video stream sent by the first terminal, it does not encapsulate the target video stream, and directly forwards it to the second terminal, so as to avoid the situation that the server buffers the video frame first. Therefore, the delay in acquiring the target video stream by the second terminal can be reduced, and further, the delay in displaying the interface picture can be reduced. In addition, in the embodiment of the present invention, after receiving the target video stream through the second terminal, the web page decoding script and the web page drawing script are used to display the interface picture based on the target video stream, so that after the second terminal decodes the interface picture, the interface picture can be decoded in time. The purpose of its display is to reduce the delay of displaying the interface picture.

FIG. 10 shows a structural block diagram of a terminal 1000 provided by an exemplary embodiment of the present invention. The terminal 1000 can be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, the standard audio layer 3 of Moving Picture Experts compression), MP4 (Moving Picture Experts Group AudioLayer IV, the standard audio layer 4 of Moving Picture Experts compression) ) player, laptop or desktop computer. The terminal 1000 may also be called user equipment, portable terminal, laptop terminal, desktop terminal and other names, and may be the above-mentioned second terminal.

Generally, the terminal 1000 includes: a processor 1001 and a memory 1002 .

The processor 1001 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 1001 may use at least one hardware form among DSP (Digital Signal Processing, digital signal processing), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array, programmable logic array) accomplish. The processor 1001 may also include a main processor and a coprocessor. The main processor is a processor used to process data in a wake-up state, also called a CPU (Central Processing Unit, central processing unit); A low-power processor for processing data in a standby state. In some embodiments, the processor 1001 may be integrated with a GPU (Graphics Processing Unit, image processor), and the GPU is used for rendering and drawing the content that needs to be displayed on the display screen. In some embodiments, the processor 1001 may further include an AI (Artificial Intelligence, artificial intelligence) processor, where the AI processor is used to process computing operations related to machine learning.

Memory 1002 may include one or more computer-readable storage media, which may be non-transitory. Memory 1002 may also include high-speed random access memory, as well as non-volatile memory, such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1002 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 1001 to implement the display interface provided by the method embodiments in this application screen method.

In some embodiments, the terminal 1000 may optionally further include: a peripheral device interface 1003 and at least one peripheral device. The processor 1001, the memory 1002 and the peripheral device interface 1003 may be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 1003 through a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 1004 , a touch display screen 1005 , a camera 1006 , an audio circuit 1007 , a positioning component 1008 and a power supply 1009 .

The peripheral device interface 1003 may be used to connect at least one peripheral device related to I/O (Input/Output) to the processor 1001 and the memory 1002 . In some embodiments, processor 1001, memory 1002, and peripherals interface 1003 are integrated on the same chip or circuit board; in some other embodiments, any one of processor 1001, memory 1002, and peripherals interface 1003 or The two can be implemented on a separate chip or circuit board, which is not limited in this embodiment.

The radio frequency circuit 1004 is used for receiving and transmitting RF (Radio Frequency, radio frequency) signals, also called electromagnetic signals. The radio frequency circuit 1004 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1004 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 1004 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like. The radio frequency circuit 1004 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: metropolitan area network, mobile communication networks of various generations (2G, 3G, 4G and 5G), wireless local area network and/or WiFi (Wireless Fidelity, wireless fidelity) network. In some embodiments, the radio frequency circuit 1004 may further include a circuit related to NFC (Near Field Communication, short-range wireless communication), which is not limited in this application.

The display screen 1005 is used for displaying UI (User Interface, user interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen 1005 is a touch display screen, the display screen 1005 also has the ability to acquire touch signals on or above the surface of the display screen 1005 . The touch signal can be input to the processor 1001 as a control signal for processing. At this time, the display screen 1005 may also be used to provide virtual buttons and/or virtual keyboards, also referred to as soft buttons and/or soft keyboards. In some embodiments, there may be one display screen 1005, which is provided on the front panel of the terminal 1000; in other embodiments, there may be at least two display screens 1005, which are respectively arranged on different surfaces of the terminal 1000 or in a folded design; In still other embodiments, the display screen 1005 may be a flexible display screen, which is disposed on a curved surface or a folding surface of the terminal 1000 . Even, the display screen 1005 can also be set as a non-rectangular irregular figure, that is, a special-shaped screen. The display screen 1005 can be made of materials such as LCD (Liquid Crystal Display, liquid crystal display), OLED (Organic Light-Emitting Diode, organic light emitting diode).

The camera assembly 1006 is used to capture images or video. Optionally, the camera assembly 1006 includes a front camera and a rear camera. Usually, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth-of-field camera to realize the background blur function, the main camera It is integrated with the wide-angle camera to achieve panoramic shooting and VR (Virtual Reality, virtual reality) shooting functions or other integrated shooting functions. In some embodiments, the camera assembly 1006 may also include a flash. The flash can be a single color temperature flash or a dual color temperature flash. Dual color temperature flash refers to the combination of warm light flash and cold light flash, which can be used for light compensation under different color temperatures.

Audio circuitry 1007 may include a microphone and speakers. The microphone is used to collect the sound waves of the user and the environment, convert the sound waves into electrical signals, and input them to the processor 1001 for processing, or to the radio frequency circuit 1004 to realize voice communication. For the purpose of stereo collection or noise reduction, there may be multiple microphones, which are respectively disposed in different parts of the terminal 1000 . The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signal from the processor 1001 or the radio frequency circuit 1004 into sound waves. The loudspeaker can be a traditional thin-film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for distance measurement and other purposes. In some embodiments, the audio circuit 1007 may also include a headphone jack.

The positioning component 1008 is used to locate the current geographic location of the terminal 1000 to implement navigation or LBS (Location Based Service, location-based service). The positioning component 1008 may be a positioning component based on the GPS (Global Positioning System, global positioning system) of the United States, the Beidou system of China, the Grenas system of Russia, or the Galileo system of the European Union.

The power supply 1009 is used to power various components in the terminal 1000 . The power source 1009 may be alternating current, direct current, disposable batteries or rechargeable batteries. When the power source 1009 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

In some embodiments, the terminal 1000 further includes one or more sensors 1010 . The one or more sensors 1010 include, but are not limited to, an acceleration sensor 1011 , a gyro sensor 1012 , a pressure sensor 1013 , a fingerprint sensor 1014 , an optical sensor 1015 and a proximity sensor 1016 .

The acceleration sensor 1011 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 1000 . For example, the acceleration sensor 1011 can be used to detect the components of the gravitational acceleration on the three coordinate axes. The processor 1001 can control the touch display screen 1005 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 1011 . The acceleration sensor 1011 can also be used for game or user movement data collection.

The gyroscope sensor 1012 can detect the body direction and rotation angle of the terminal 1000 , and the gyroscope sensor 1012 can cooperate with the acceleration sensor 1011 to collect 3D actions of the user on the terminal 1000 . The processor 1001 can implement the following functions according to the data collected by the gyro sensor 1012: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

The pressure sensor 1013 may be disposed on the side frame of the terminal 1000 and/or the lower layer of the touch display screen 1005 . When the pressure sensor 1013 is disposed on the side frame of the terminal 1000, the user's holding signal of the terminal 1000 can be detected, and the processor 1001 performs left and right hand identification or shortcut operations according to the holding signal collected by the pressure sensor 1013. When the pressure sensor 1013 is disposed on the lower layer of the touch display screen 1005 , the processor 1001 controls the operability controls on the UI interface according to the user's pressure operation on the touch display screen 1005 . The operability controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

The fingerprint sensor 1014 is used to collect the user's fingerprint, and the processor 1001 identifies the user's identity according to the fingerprint collected by the fingerprint sensor 1014, or the fingerprint sensor 1014 identifies the user's identity according to the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 1001 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 1014 may be provided on the front, back or side of the terminal 1000 . When the terminal 1000 is provided with physical buttons or a manufacturer's logo, the fingerprint sensor 1014 may be integrated with the physical buttons or the manufacturer's logo.

The optical sensor 1015 is used to collect ambient light intensity. In one embodiment, the processor 1001 may control the display brightness of the touch display screen 1005 according to the ambient light intensity collected by the optical sensor 1015 . Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 1005 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 1005 is decreased. In another embodiment, the processor 1001 may also dynamically adjust the shooting parameters of the camera assembly 1006 according to the ambient light intensity collected by the optical sensor 1015 .

A proximity sensor 1016 , also called a distance sensor, is usually disposed on the front panel of the terminal 1000 . The proximity sensor 1016 is used to collect the distance between the user and the front of the terminal 1000 . In one embodiment, when the proximity sensor 1016 detects that the distance between the user and the front of the terminal 1000 gradually decreases, the processor 1001 controls the touch display screen 1005 to switch from the bright screen state to the off screen state; when the proximity sensor 1016 detects When the distance between the user and the front of the terminal 1000 gradually increases, the processor 1001 controls the touch display screen 1005 to switch from the off-screen state to the bright-screen state.

Those skilled in the art can understand that the structure shown in FIG. 10 does not constitute a limitation on the terminal 1000, and may include more or less components than the one shown, or combine some components, or adopt different component arrangements.

In the embodiment of the present invention, after the server receives the target video stream sent by the first terminal, it does not encapsulate the target video stream, and directly forwards it to the second terminal, so as to avoid the situation that the server buffers the video frame first. Therefore, the delay in acquiring the target video stream by the second terminal can be reduced, and further, the delay in displaying the interface picture can be reduced. In addition, in the embodiment of the present invention, after receiving the target video stream through the second terminal, the web page decoding script and the web page drawing script are used to display the interface picture based on the target video stream, so that after the second terminal decodes the interface picture, the interface picture can be decoded in time. The purpose of its display is to reduce the delay of displaying the interface picture.

FIG. 11 is a schematic structural diagram of a server according to an embodiment of the present invention. The server 1100 may vary greatly due to different configurations or performance, and may include one or more processors (central processing units, CPU) 1101 and one or more More than one memory 1102, wherein the memory 1102 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 1101 to implement the above method steps for displaying an interface screen.

An embodiment of the present invention further provides a computer-readable storage medium, where at least one instruction, at least one piece of program, code set or instruction set is stored in the storage medium, the at least one instruction, the at least one piece of program, the at least one piece of program, the The code set or the instruction set is loaded and executed by the processor to implement the above-mentioned method for displaying an interface picture.

In the embodiment of the present invention, after the server receives the target video stream sent by the first terminal, it does not encapsulate the target video stream, and directly forwards it to the second terminal, so as to avoid the situation that the server buffers the video frame first. Therefore, the delay in acquiring the target video stream by the second terminal can be reduced, and further, the delay in displaying the interface picture can be reduced. In addition, in the embodiment of the present invention, after receiving the target video stream through the second terminal, the web page decoding script and the web page drawing script are used to display the interface picture based on the target video stream, so that after the second terminal decodes the interface picture, the interface picture can be decoded in time. The purpose of its display is to reduce the delay of displaying the interface picture.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium mentioned may be a read-only memory, a magnetic disk or an optical disk, etc.

The above is only an embodiment of the present invention, and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention. within.

Claims (13)

1. A system for displaying an interface picture, wherein the system comprises a server and a second terminal, wherein: The server is configured to receive a target video stream sent by a first terminal, where the target video stream is composed of an interface screen currently displayed by the first terminal, and the first terminal is a cloud real machine; determine the first terminal a second terminal corresponding to a terminal, where the second terminal is a terminal that remotely controls the first terminal; sending the target video stream to the second terminal, where the target video stream is an unencapsulated video a stream, used for the second terminal to display the interface picture; The second terminal is used to obtain a webpage decoding script and a webpage drawing script; during the process of remote control of the first terminal by the second terminal, the target video stream sent by the server is obtained; The decoding script decodes the target video stream to obtain the picture data of the interface picture; the webpage drawing script is used to display the interface picture based on the picture data of the interface picture. 2. The system according to claim 1, wherein the second terminal is further used for: displaying a selection interface, where the selection interface includes options of multiple terminals; When receiving the selection instruction for the first terminal in the plurality of terminals, acquiring the target account identifier of the current login account and the target terminal identifier of the first terminal; Encrypting the target account identification and the target terminal identification to obtain the encrypted target account identification and the target terminal identification; sending the encrypted target account identifier and target terminal identifier to the server; The server is also used to: receiving the encrypted target account identifier and target terminal identifier sent by the second terminal; Decrypting the encrypted target account identification and target terminal identification to obtain the target account identification and the target terminal identification; Verification processing is performed on the target account identifier and the target terminal identifier, and if the verification is successful, a video stream acquisition request is sent to the first terminal corresponding to the target terminal identifier, and the video stream acquisition request is used to The first terminal requests the target video stream. 3. The system according to claim 2, wherein the server is further configured to: correspondingly storing the target account identification and the target terminal identification in the corresponding relationship between the account identification and the terminal identification; the server for: A second terminal corresponding to the first terminal is determined according to the pre-stored correspondence. 4. The system according to claim 1, wherein the second terminal is used for: When the display instruction for the target web page is acquired, the web page decoding script and the web page drawing script are downloaded from the server. 5. The system according to claim 1, wherein the target video stream is a video stream compressed by a video frame by an intra-inter-frame compression algorithm; The second terminal is used for: In the process of acquiring the target video stream, whenever a data packet containing the start identifier of the network transmission unit is acquired, the data packet between the data packet and the next data packet containing the start identifier of the network transmission unit is determined. It is a complete network transmission unit; Store the acquired network transmission unit in the cache; Through the webpage decoding script, each network transmission unit in the cache is decoded to obtain the picture data of the interface picture. 6. The system according to claim 1, wherein the server is configured to: Sending data in a preset format to the second terminal, the data includes a target data type and the target video stream, and the target data type is a data type corresponding to the video stream; The second terminal is used for: During the process that the second terminal remotely controls the first terminal, receiving data in a preset format sent by a server; If the data type included in the data is the target data type, acquire the target video stream included in the data, where the target data type is the data type corresponding to the video stream. 7. A method for displaying an interface picture, characterized in that, applied to a server, the method comprising: receiving a target video stream sent by a first terminal, where the target video stream is composed of an interface screen currently displayed by the first terminal, and the first terminal is a cloud real machine; determining a second terminal corresponding to the first terminal, where the second terminal is a terminal that remotely controls the first terminal; Send the target video stream to the second terminal, where the target video stream is an unencapsulated video stream, the second terminal is used to obtain the webpage decoding script and the webpage drawing script, and the second terminal compares the In the process of remote control by the first terminal, the target video stream sent by the server is obtained; the target video stream is decoded by the webpage decoding script, and the picture data of the interface picture is obtained; The webpage drawing script displays the interface picture based on the picture data of the interface picture. 8. The method according to claim 7, wherein the method further comprises: receiving the encrypted target account identifier and target terminal identifier sent by the second terminal; Decrypting the encrypted target account identification and target terminal identification to obtain the target account identification and the target terminal identification; Verification processing is performed on the target account identifier and the target terminal identifier, and if the verification is successful, a video stream acquisition request is sent to the first terminal corresponding to the target terminal identifier, and the video stream acquisition request is used to The first terminal requests the target video stream. 9. A method for displaying an interface picture, wherein the method is applied to a second terminal, the method comprising: Obtain webpage decoding scripts and webpage drawing scripts; During the process of remote control of the first terminal by the second terminal, the target video stream sent by the server is obtained, where the target video stream is composed of the interface screen currently displayed by the first terminal and is not encapsulated video stream, the first terminal is a cloud real machine; Decode the target video stream through the webpage decoding script to obtain picture data of the interface picture; The interface picture is displayed based on the picture data of the interface picture through the webpage drawing script. 10. The method according to claim 9, wherein the obtaining the webpage decoding script and the webpage drawing script comprises: When the display instruction for the target web page is acquired, the web page decoding script and the web page drawing script are downloaded from the server. 11. The method of claim 9, wherein the method further comprises: displaying a selection interface, where the selection interface includes options of multiple terminals; When receiving the selection instruction for the first terminal in the plurality of terminals, acquiring the target account identifier of the current login account and the target terminal identifier of the first terminal; Encrypting the target account identification and the target terminal identification to obtain the encrypted target account identification and the target terminal identification; Send the encrypted target account identifier and target terminal identifier to the server. 12. A computer device, characterized in that the computer device comprises a processor and a memory, and the memory stores at least one instruction, at least a piece of program, a code set or an instruction set, the at least one instruction, the at least one A piece of program, the code set or the instruction set is loaded and executed by the processor to implement the method for displaying an interface picture according to any one of claims 7 to 11. 13. A computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one piece of program, code set or instruction set, the at least one instruction, the at least one piece of program, the code The set or instruction set is loaded and executed by the processor to implement the method for displaying an interface picture as claimed in any one of claims 7 to 11 .

Images