CN108476339B - A remote control method and terminal - Google Patents

Abstract

Embodiments of the present invention provide a remote control method and terminal. The method includes: the first terminal displays a first control interface, the first control interface is consistent with the first display interface currently output by the second terminal, the second terminal is a terminal remotely controlled by the first terminal; the first display interface displays at least two icons; the first terminal receives a first touch operation input on the first control interface, where the first touch operation includes a selection operation on the first icon in the first control interface; the first terminal generates a first touch operation according to the first touch operation a remote control command, and send the first remote control command to the second terminal, where the first remote control command is used to instruct the second terminal to select the icon corresponding to the first icon in the first display interface; after receiving the first touch operation, the A terminal displays a second control interface, and the second control interface is consistent with the second display interface output after the second terminal executes the first remote control command. This embodiment can improve the efficiency and convenience of remote control operation.

Description

A remote control method and terminal

This application claims the priority of the Chinese patent application filed on December 30, 2016 with the application number 201611265792.0 and the title of the invention is "a method and device for remote control of a TV by an intelligent terminal", the entire contents of which are incorporated by reference in in this application.

technical field

Embodiments of the present invention relate to communication technologies, and in particular, to a remote control method and terminal.

Background technique

With the development of network technology, users can watch rich video content through smart TV or Internet Protocol Television (IPTV for short) or set-top box, and can on-demand, review or manage the video content.

When on-demand, playback or management of video content, it is usually necessary to use a remote control to operate a TV or a set-top box. In the prior art, a remote control generally includes some function buttons and direction buttons. When the user wants to switch from one TV interface to another TV interface, or when the user wants to select a program from multiple program options, it is usually necessary to perform multiple operations on the keys of the remote control. As a result, the operation of the remote controller is complicated, and the operation is inconvenient for the user.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a remote control method and terminal, which can improve the efficiency and convenience of remote control operations.

In a first aspect, an embodiment of the present invention provides a remote control method, and the method is applied to a first terminal having a touch screen. The first terminal remotely controls the second terminal through the method. The method includes: the first terminal displays a first control interface, the first control interface is consistent with the first display interface currently output by the second terminal; the first display interface displays at least two icons, the The number and arrangement of icons displayed on the first control interface are respectively the same as the number and arrangement of icons displayed on the first display interface; and the icons displayed on the first control interface are the same as those displayed on the first display interface. One-to-one correspondence between the icons; the first terminal receives a first touch operation input on the first control interface, and the first touch operation at least includes a selection operation on the first icon in the first control interface ; the first terminal generates a first remote control command according to the first touch operation; the first terminal sends the first remote control command to the second terminal, and the first remote control command is used to instruct the The second terminal selects the icon corresponding to the first icon in the first display interface; after receiving the first touch operation, the first terminal displays a second control interface, the second control interface and the The second display interface output by the second terminal is consistent, and the second display interface is the display interface output after the second terminal executes the first remote control command. The control interface displayed by the first terminal can change with the change of the display interface output by the second terminal, and the layout of the control interface is consistent with the current display interface, so that the user can touch the control interface with a small amount of touch. The display interface of the second terminal can be controlled by the operation, thereby making the use of the remote controller more convenient.

Optionally, the generating, by the first terminal, the first remote control command according to the first touch operation specifically includes: the first terminal acquiring the initial position of the cursor before the first icon is selected, and then obtaining the initial position of the cursor according to the initial position of the cursor. and the position where the first icon is located to generate the first remote control instruction.

Optionally, the first touch operation further includes an operation of selecting a second icon corresponding to the initial position of the cursor before the first icon is selected, and the second icon is included in the first control interface in; the initial position of the cursor is the initial position of the cursor on the first display interface. There are many ways to obtain the initial position of the cursor.

Optionally, the second display interface is an editable input interface, and the second control interface is an input interface including a soft keyboard. The interface layout of the soft keyboard in the second control interface is the same as or different from the interface layout of the soft keyboard in the second display interface. The second control interface may include the same soft keyboard as the second display interface, or may include different soft keyboards. For example, the second control interface may include a soft keyboard that is more convenient to operate.

Optionally, the method further includes: receiving, by the first terminal, a second touch operation input on a soft keyboard of the second control interface, where the second touch operation includes a soft touch operation on the second control interface. selection operation of the first character on the keyboard; the first terminal generates a second remote control command according to the second touch operation; the first terminal sends the second remote control command to the second terminal, the The second remote control instruction is used to instruct the second terminal to input the first character in the second display interface. Through the touch operation on the soft keyboard of the first terminal, characters can be conveniently input in the second terminal.

Optionally, after the first touch operation is received, the first terminal displays a second control interface, which specifically includes: the first terminal acquires the first control interface from a memory of the first terminal. The second control interface corresponding to the touch operation; the first terminal displays the second control interface. The second control interface may be pre-stored in the first terminal.

Optionally, after the first terminal sends the first remote control instruction to the second terminal, the method further includes: the first terminal sends an interface information request to the second terminal; the The first terminal receives the interface information of the second display interface sent by the second terminal; the first terminal displays the second control interface according to the interface information. When the first terminal does not store the second control interface in advance, the second control interface may be requested from the second terminal.

Optionally, after the first terminal sends the first remote control command to the second terminal, the method further includes: the first terminal receiving an operation for starting a photograph; the first terminal A picture with the second display interface is acquired by taking a photo; the first terminal identifies the content in the picture of the second display interface, and displays the identified content as the second control interface. When the first terminal does not store the second control interface in advance, the first terminal may acquire the second control interface by taking a picture.

Optionally, the second control interface that is consistent with the second display interface output by the second terminal specifically includes any one of the following. The first type: when the second display interface includes at least two icons, the number of icons displayed on the second control interface is the same as the number of icons displayed on the second display interface, and the second control interface displays The arrangement of the icons is the same as the arrangement of the icons displayed on the second display interface, and the icons displayed on the second control interface are in one-to-one correspondence with the icons displayed on the second display interface. The second type: when the second display interface is an editable input interface, the second control interface is an input interface including a soft keyboard. The third type: when the second display interface is an audio and video playback interface, the second control interface includes any one or more of the following: volume control items, playback progress control items, and play/stop control items.

Optionally, the method further includes: before the first terminal displays the first control interface, the first terminal displays thumbnail images or icons of at least two control interfaces; the first terminal receives a third touch operation; the first terminal selects the first control interface from the at least two control interfaces to display according to the third touch operation. The first terminal may be triggered to display the first control interface through the third touch operation.

Optionally, when the second terminal is a television or a similar display device, the first display interface and the second display interface are displayed by the second terminal. When the second terminal is a set-top box or similar device, the first display interface and the second display interface are output by the second terminal to an external device for display, and the external device may be a TV, a computer or similar display equipment.

Optionally, the first terminal sends the remote control instruction to the second terminal through any one of the following communication modes: infrared, Bluetooth, Wi-Fi and NFC.

Optionally, the first terminal is a mobile phone or a tablet computer, and the second terminal is a TV or a set-top box.

Optionally, the method further includes: the first terminal recognizes the user's gesture, the first terminal converts the gesture into a third remote control command, and sends the third remote control command to the second terminal. That is, during the process of remotely controlling the second terminal, the first terminal may perform a corresponding remote control operation by recognizing the user's gesture.

Optionally, the method further includes: the first terminal recognizes the user's voice, the first terminal converts the voice into a fourth remote control command, and sends the fourth remote control command to the second terminal. The first terminal may also control the second terminal through the user's voice.

In a second aspect, an embodiment of the present invention provides a first terminal, where the first terminal is used to remotely control a second terminal, and the first terminal includes: a touch screen; one or more processors; One or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions that, when the instructions are executed by the first terminal, cause the first terminal to execute the above-mentioned first terminal. Any of the methods described in the aspects.

In a third aspect, an embodiment of the present invention provides a graphical user interface (GUI), where the GUI is stored in the first terminal. The first terminal includes a touch screen, one or more memories, and one or more processors for executing one or more computer programs stored in the one or more memories. The GUI includes: a first GUI displayed on the touch screen, the first GUI is consistent with a first display interface currently output by a second terminal, and the second terminal is a terminal remotely controlled by the first terminal; The first display interface displays at least two icons, the number of icons displayed on the first GUI is the same as the number of icons displayed on the first display interface, and the arrangement of the icons displayed on the first GUI is the same as that of the icons displayed on the first GUI. The icons displayed on the first display interface are arranged in the same manner, and the icons displayed on the first GUI correspond one-to-one with the icons displayed on the first display interface; in response to the user's first gesture on the touch screen, according to The first gesture generates a first remote control command, and sends the first remote control command to the second terminal; the first gesture includes a selection operation on a first icon in the first GUI, the The first remote control instruction is used to instruct the second terminal to select an icon corresponding to the first icon in the first display interface; in response to the user's first gesture or third gesture on the touch screen, A second GUI is displayed on the touch screen, the second GUI is consistent with the second display interface output by the second terminal, and the second display interface is output after the second terminal executes the first remote control command display interface.

Optionally, the second display interface is an editable input interface, and the second GUI is an input interface including a soft keyboard. The interface layout of the soft keyboard in the second GUI is the same as or different from the interface layout of the soft keyboard in the second display interface.

Optionally, the GUI further includes: in response to a second gesture input by the user on the soft keyboard of the second GUI, generating a second remote control instruction according to the second gesture, and sending the second remote control instruction to the user. sending an instruction to the second terminal; the second gesture includes a selection operation on a first character on the soft keyboard of the second GUI, and the second remote control instruction is used to instruct the second terminal to The first character is input in the second display interface.

Optionally, the second GUI is consistent with the second display interface output by the second terminal. For details, refer to the foregoing description that the second control interface is consistent with the second display interface output by the second terminal.

In a fourth aspect, an embodiment of the present invention provides a first terminal, wherein the first terminal is used to remotely control a second terminal, and the first terminal includes: a display unit, an input unit, a control unit, and a sending unit. the display unit is configured to display a first control interface, the first control interface is consistent with the first display interface currently output by the second terminal, and the second terminal is a terminal remotely controlled by the first terminal; The first display interface displays at least two icons, the number of icons displayed on the first control interface is the same as the number of icons displayed on the first display interface, and the icons displayed on the first control interface are arranged in the same manner as The icons displayed on the first display interface are arranged in the same manner, and the icons displayed on the first control interface are in one-to-one correspondence with the icons displayed on the first display interface. The input unit is configured to receive a first touch operation input on the first control interface, where the first touch operation includes a selection operation of a first icon in the first control interface. The control unit is configured to generate a first remote control instruction according to the first touch operation received by the input unit. The sending unit is configured to send the first remote control command to the second terminal, where the first remote control command is used to instruct the second terminal to select the first icon in the first display interface and the first icon corresponding icon. The display unit is further configured to, after receiving the first touch operation, display a second control interface, where the second control interface is consistent with the second display interface output by the second terminal, and the second control interface is consistent with the second display interface output by the second terminal. The display interface is a display interface output after the second terminal executes the first remote control command. Each unit included in the first terminal is used to implement each method in the first aspect, and each unit may be implemented by hardware or software.

In a fifth aspect, an embodiment of the present invention provides a remote control device included in a first terminal, where the device has a function of implementing the behavior of the first terminal in each method described in the first aspect. The functions can be implemented by hardware, or can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions.

In a sixth aspect, an embodiment of the present invention provides a computer program product including instructions, which, when the computer program product runs on an electronic device, causes the electronic device to execute the method described in the first aspect.

In a seventh aspect, an embodiment of the present invention provides a computer-readable storage medium, including instructions, which, when the instructions are executed on an electronic device, cause the electronic device to execute the method described in the first aspect.

In an eighth aspect, an embodiment of the present invention further provides a data processing system, including modules for executing the methods provided in the first aspect.

Description of drawings

1 is a partial structural block diagram of a terminal provided by an embodiment of the present invention;

2A is a schematic diagram of a scene of a remote control method provided by an embodiment of the present invention;

2B is a flowchart of a remote control method provided by an embodiment of the present invention;

3 is a schematic interface diagram of a first terminal provided by an embodiment of the present invention;

4A-4E are schematic diagrams of a remote control interface provided by an embodiment of the present invention;

5 is a schematic interface diagram of a remote control method provided by an embodiment of the present invention;

6 is a schematic diagram of another remote control interface provided by an embodiment of the present invention;

7 is a schematic diagram of another remote control interface provided by an embodiment of the present invention;

8A and 8B are schematic diagrams of another remote control interface provided by an embodiment of the present invention;

9 is a schematic diagram of another remote control interface provided by an embodiment of the present invention;

10 is a schematic diagram of another remote control interface provided by an embodiment of the present invention;

11 is a flowchart of another remote control method provided by an embodiment of the present invention;

FIG. 12 is a flowchart of a method for remote control through gestures provided by an embodiment of the present invention.

Detailed ways

When using a TV, a set-top box, or other similar terminals to watch audio and video programs, conventional remote controls are generally used to control the TV, the set-top box, or other similar terminals. With the remote control methods provided by the embodiments of the present invention, a terminal with a touch screen can be used to control a TV, a set-top box or a similar terminal instead of a common remote control. The terminal may be, for example, a mobile phone, a Tablet Personal Computer, a Laptop Computer, a digital camera, a personal digital assistant (PDA), a navigation device, a Mobile Internet Device , MID) or wearable device (Wearable Device), etc.

FIG. 1 is a partial structural block diagram of a terminal provided by an embodiment of the present invention. The terminal is described by taking the mobile phone 100 as an example. Referring to FIG. 1 , the mobile phone 100 includes: a radio frequency (RF) circuit 110 , a power supply 120 , a processor 130 , a memory 140 , an input unit 150 , a display unit 160 , a camera 165 , a sensor 170 , an audio circuit 180 , a wireless fidelity ( wireless fidelity, Wi-Fi) module 190, infrared transmitter 192, and Bluetooth module 194 and other components. Those skilled in the art can understand that the structure of the mobile phone shown in FIG. 1 does not constitute a limitation on the mobile phone, and may include more or less components than the one shown, or combine some components, or arrange different components.

In the following, each component of the mobile phone 100 will be introduced in detail with reference to FIG. 1 :

The RF circuit 110 may be used to transmit and receive information or to receive and transmit signals during a call. For example, the RF circuit 110 may send the downlink data received from the base station to the processor 130 for processing, and send the uplink data to the base station. Generally, the RF circuit includes, but is not limited to, an RF chip, an antenna, at least one amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, a radio frequency switch, and the like. In addition, RF circuitry 110 may also communicate wirelessly with networks and other devices. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (Code Division) Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (Short Messaging Service, SMS) and the like.

The memory 140 may be used to store software programs and modules, and the processor 130 executes various functional applications and data processing of the mobile phone 100 by running the software programs and modules stored in the memory 140 . The memory 140 may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the storage data area may store Data (such as audio data, phonebook, etc.) created according to the use of the mobile phone 100, and the like. Additionally, memory 140 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The input unit 150 may be used to receive input numerical or character information, and generate key signal input related to user settings and function control of the mobile phone 100 . Specifically, the input unit 150 may include a touch panel 151 and other input devices 152 . The touch panel 151, also referred to as a touch screen, can collect the user's touch operations on or near it (such as the user's finger, stylus, etc., any suitable objects or accessories on or near the touch panel 151). operation), and drive the corresponding connection device according to the preset program. Optionally, the touch panel 151 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the touch controller. To the processor 130, and can receive the command sent by the processor 130 and execute it. In addition, the touch panel 151 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. Besides the touch panel 151 , the input unit 150 may further include other input devices 152 . Specifically, other input devices 152 may include, but are not limited to, one or more of physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, joysticks, and the like.

The display unit 160 may be used to display information input by the user or information provided to the user and various menus and interfaces of the mobile phone 100 . The display unit 160 may include a display panel 161. Optionally, the display panel 161 may be configured in the form of a liquid crystal display (LCD for short), an organic light-emitting diode (OLED for short), and the like. Further, the touch panel 151 may cover the display panel 161. When the touch panel 151 detects a touch operation on or near it, it transmits it to the processor 130 to determine the type of the touch event, and then the processor 130 determines the type of the touch event according to the touch event. Type provides corresponding visual output on display panel 161 . Although in FIG. 1, the touch panel 151 and the display panel 161 are used as two independent components to realize the input and input functions of the mobile phone 100, in some embodiments, the touch panel 151 and the display panel 161 may be integrated And realize the input and output functions of the mobile phone 100 .

Cell phone 100 may also include one or more cameras 165 . Camera 165 may be used to capture and capture images. The mobile phone 100 may include one or more front cameras and one or more rear cameras.

The cell phone 100 may also include at least one sensor 170, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 161 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 161 and the display panel 161 when the mobile phone 100 is moved to the ear. / or backlight. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (usually three axes), and can detect the magnitude and direction of gravity when it is stationary. games, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping), etc. The mobile phone 100 may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which will not be repeated here.

The audio circuit 180 , the speaker 181 , and the microphone 182 can provide an audio interface between the user and the mobile phone 100 . The audio circuit 180 can transmit the received audio data converted electrical signal to the speaker 181 , and the speaker 181 converts it into a sound signal for output; on the other hand, the microphone 182 converts the collected sound signal into an electrical signal, which is converted by the audio circuit 180 After receiving, it is converted into audio data, and then the audio data is output to the RF circuit 110 for transmission to, for example, another mobile phone, or the audio data is output to the memory 140 for further processing.

Wi-Fi, infrared and Bluetooth are all short-range wireless transmission technologies. Through the Wi-Fi module 190, the mobile phone 100 can help the user to send and receive emails, browse web pages, access streaming media, etc., which provides the user with wireless broadband Internet access. The mobile phone 100 can emit infrared light within a certain range through the infrared transmitter 192 . The mobile phone can communicate with external devices through the Bluetooth module 194 .

The processor 130 is the control center of the mobile phone 100, using various interfaces and lines to connect various parts of the entire mobile phone, by running or executing the software programs and/or modules stored in the memory 140, and calling the data stored in the memory 140, Execute various functions of the mobile phone 100 and process data, thereby realizing various services based on the mobile phone. Optionally, the processor 130 may include one or more processing units; preferably, the processor 130 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc. , the modem processor mainly deals with wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 130.

In this embodiment of the present invention, the processor 130 may execute program instructions stored in the memory 140, so that the mobile phone acts as the first terminal to execute the method shown in any of the following embodiments.

The mobile phone 100 also includes a power supply 120 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 130 through a power management system, so as to manage charging, discharging, and power consumption functions through the power management system.

The remote control method provided by the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The above-mentioned terminal that can replace the remote control is called the first terminal, and the above-mentioned remote-controlled TV, set-top box or similar terminal is called the second terminal. FIG. 2A is a schematic diagram of a scenario of a remote control method provided by an embodiment of the present invention. FIG. 2B is a flowchart of a remote control method provided by an embodiment of the present invention.

As shown in FIG. 2A , the first terminal 100 may control the second terminal 200 by using the remote control method provided in the embodiment of the present invention. It should be noted that the remote control method provided by the embodiment of the present invention does not conflict with the existing remote control method, that is, the second terminal 200 can also be controlled by a common remote control 300 at the same time.

FIG. 3 is a schematic interface diagram of a first terminal according to an embodiment of the present invention. 4A-4E are schematic diagrams of a remote control interface provided by an embodiment of the present invention. As shown in FIG. 3 , a plurality of application programs may be installed on the first terminal, which may include a remote control application program 400 . After the remote control application 400 is opened, the first terminal can use the remote control application 400 to control the second terminal. As shown in FIG. 2B , when the first terminal is used to remotely control the second terminal, the remote control method includes the following steps.

Step 201: The first terminal displays a first control interface. As shown in FIG. 4D , the second terminal currently outputs the first display interface 401 , and the first display interface displays at least two icons 402 . Six icons 402 are exemplarily displayed in FIG. 4D. Wherein, the first control interface 410 is consistent with the first display interface 401 . Specifically, the number of icons 412 displayed on the first control interface 410 may be the same as the number of icons 402 displayed on the first display interface 401 . The number of icons displayed on the first control interface 410 is the same as that of the icons displayed on the first display interface 401 ; and the icons displayed on the first control interface are the same as the icons displayed on the first control interface 410 The icons displayed on the display interface correspond one by one. Wherein, since the size and shape of the first control interface and the first display interface may be very different, the content displayed in the first control interface can be scaled relative to the content displayed in the first display interface or size adaptation. And the identifiers of the icons in the first control interface 410 and the icons in the first display interface 401 may be the same (as shown in FIG. 4D ) or different (as shown in FIG. 4E ). The first display interface may have various forms, and FIG. 4D is only an exemplary illustration. For example, as shown in FIG. 9 , the first display interface 901 may be in another form, and the corresponding first control interface 910 is in a form corresponding to the first display interface 901 . Through the interface shown in FIG. 9 , the programs broadcast during the seven days from January 1st to January 7th can be reviewed.

It should be noted that, based on the first display interface, the first control interface is displayed on the first terminal. That is to say, the embodiment of the present invention does not project the interface displayed on the first terminal to the second terminal. Moreover, when using the first terminal to control the second terminal, the second terminal can also be controlled by an ordinary remote controller at the same time, as shown in FIG. 2A .

Step 202: The first terminal receives a first touch operation input on the first control interface. Wherein, the first touch operation may include a selection operation of a first icon in the first control interface. The first touch operation may be one or more touch operations. That is, the user can perform one or more touch operations on the first control interface.

Step 203: The first terminal generates a first remote control instruction according to the first touch operation, and sends the first remote control instruction to the second terminal. The first remote control instruction is used to instruct the second terminal to select an icon corresponding to the first icon in the first display interface.

Specifically, the process for the first terminal to generate the first remote control instruction may include: the first terminal obtains, by the first terminal, the initial position of the cursor before the first icon is selected; and then the first terminal according to the initial cursor position The position and the position where the first icon is located to generate the first remote control instruction. As shown in the first display interface 401 shown in FIG. 5( a ), the initial position of the cursor is at the icon 4021 . The second terminal may send the relative position of the initial position of the cursor in the first display interface 401 to the first terminal, so that the first terminal obtains that the initial position of the cursor is in the first terminal. The control interface corresponds to the second icon 4121; alternatively, the first terminal may receive a user's touch operation on the second icon 4121 to identify that the initial position of the cursor corresponds to the second icon 4121; or, after entering the In the first control interface 410 , the initial position of the cursor is at the second icon 4121 by default, and the first terminal can obtain the default position when entering the first control interface 410 . After acquiring the second icon 4121 corresponding to the initial position of the cursor, the second terminal may display the initial position of the cursor on the first control interface 410, as shown in FIG. 5(b). When the user wishes to select the icon 4023 in the first display interface, the user can select the first icon 4123 corresponding to the icon 4023 on the first control interface 410, as shown in FIG. 5(c). After acquiring that the first icon 4123 is selected, the first terminal generates the first remote control command according to the initial position and the position where the first icon 4123 is located, and sends the first remote control command to the second terminal. The second terminal selects the icon 4023 according to the first remote control instruction, as shown in FIG. 5(d). The second icon is included in the first control interface. The initial position of the cursor refers to the position of the cursor when entering the first display interface. Or, the initial position of the cursor refers to the position where the cursor is located before the second terminal receives the first remote control command. Since the icons of the first control interface and the first display interface are distributed in the same manner, the first control interface includes icons corresponding to the initial position of the cursor.

Wherein, the first touch operation may be in different forms.

When the first terminal recognizes the initial position of the cursor according to the user's touch operation, the first touch operation may have the following two operations in sequence: firstly, on the second icon corresponding to the initial position of the cursor touch operation, and then a selection operation on the first icon; for example: the touch operation on the second icon can be a click operation; the selection operation on the first icon can be a click, double-click, long-press or re-press , or after clicking, double-clicking, long-pressing, or re-pressing, the touch operation of the OK button is also included. That is to say, the selection of the first icon can be realized only by clicking, double-clicking, long-pressing or re-pressing the first icon. After double-clicking, long-pressing or re-pressing the first icon, click or touch the OK button again to realize the selection of the first icon.

When the second terminal sends the initial cursor position to the first terminal, or when the first terminal can obtain the default initial cursor position when entering the first control interface 410, the The first touch operation may only include a selection operation on the first icon.

The first remote control command may have various forms.

One is: the first terminal can convert the first touch operation into one or more common remote control commands. For example, the first terminal obtains that the cursor needs to be moved to the right twice according to the coordinates of the second icon 4121 corresponding to the initial position of the cursor and the coordinate information of the first icon 4123, and thus, the first terminal will The first touch operation is converted into two remote control commands for moving to the right once, and each remote control command represents moving to the right once. In this way, the second terminal can use an existing TV or a set-top box, that is, the second terminal can be controlled by using the remote control method provided by the embodiment of the present invention without improving the existing second terminal. .

Another way: the first terminal may convert the first touch operation into a remote control command representing the movement and displacement of the cursor. For example, according to the coordinate information of the second icon 4121 and the coordinate information of the first icon 4123 corresponding to the initial position of the cursor, the first terminal obtains that the cursor needs to be moved to the right twice. The first touch operation is converted into a remote control command representing moving to the right twice. The second terminal receives and parses the first remote control instruction, and then controls the cursor to move to the right twice. In this manner, the same database may be preset in the first terminal and the second terminal, so that the second terminal can correctly parse the remote control command sent by the first terminal.

The third type: the first terminal may convert the first touch operation into a remote control instruction representing the coordinate change of the touch screen. For example: the first terminal sends the coordinate information of the second icon 4121 and the coordinate information of the first icon 4123 corresponding to the initial position of the cursor to the second terminal as the first remote control command; The coordinate information of the first icon 4123 relative to the second icon 4121 is sent to the second terminal as the first remote control command. The second terminal receives and parses the first remote control instruction, and then controls the cursor to move to the right twice. In this manner, the same database may also be preset in the first terminal and the second terminal, so that the second terminal can correctly parse the remote control command sent by the first terminal.

Step 204: After receiving the first touch operation, the first terminal displays a second control interface. The second control interface is consistent with the second display interface output by the second terminal, and the second display interface is a display interface output after the second terminal executes the first remote control command.

It should be noted that this embodiment does not limit the execution order of step 203 and step 204 , step 204 may be executed after step 203 , and step 204 may also be executed in parallel with step 203 .

The second terminal outputs the second display interface after executing the first remote control command, so the first terminal also switches the first control interface to the second control interface consistent with the second display interface interface, so that the first terminal can continue to control the second terminal.

The second control interface corresponding to the execution of the first touch operation may be pre-stored in the first terminal. After the first terminal receives the first touch operation, the first terminal may acquire the information of the second control interface from the memory, and then display the second control interface. For example, after the second terminal receives the first remote control command, the interface of FIG. 5(d) is switched to the second display interface 601 shown in FIG. 6 . A second control interface 610 corresponding to the second display interface 601 is pre-stored in the first terminal, and the first terminal is automatically switched to display the second control interface 610, as shown in FIG. 6 .

Further, if the second control interface is not pre-stored in the first terminal, the method may further include the following steps after step 203: the first terminal sends an interface information request to the second terminal; The first terminal receives the interface information of the second display interface sent by the second terminal. Then in step 204, the first terminal may acquire and display the second control interface according to the interface information. Specifically, as shown in FIG. 5( c ), after the first icon 4123 is selected, the user can request the interface information of the second control interface by clicking the button 420 . After detecting that the button 420 is touched, the first terminal sends an interface information request to the second terminal; after receiving the interface information request, the second terminal obtains the current output of the second terminal the interface information of the second display interface 601, for example, you can take a screenshot of the second display interface 601 or directly obtain the display data corresponding to the second display interface 601; The interface information of the display interface is sent to the first terminal; after the first terminal processes the interface information, the second control interface 610 is output.

In another optional manner, if the second control interface is not pre-stored in the first terminal, after step 203, the method may further include the following steps: the first terminal receives a message for starting taking pictures. operation; the first terminal obtains a picture with the second display interface by taking a photo; the first terminal identifies the content in the picture of the second display interface, and uses the identified content as the first The second control interface is displayed. Specifically, as shown in FIG. 5( c ), the button 420 can be replaced with a photo button. After the first icon 4123 is selected, the user can click the photo button to take a photo of the second display interface . After acquiring the picture of the second display interface, the first terminal performs icon recognition and cropping on the picture, and then outputs the second control interface.

It should be noted that, after acquiring the second control interface by taking a photo or making a request to the second terminal, the first terminal can save the second control interface, so that the The second control interface can be displayed automatically.

Consistency between the second control interface and the second display interface may specifically include any one or more of the following situations.

The first type: when the second display interface is an editable input interface, the second control interface is an input interface including a soft keyboard. Input and editing on the second display interface can be implemented through a soft keyboard in the second control interface. The interface layout of the soft keyboard in the second control interface may be the same as the interface layout of the soft keyboard in the second display interface, as shown in FIG. 8A . Alternatively, the interface layout of the soft keyboard in the second control interface and the interface layout of the soft keyboard in the second display interface may also be different, as shown in FIG. 8B .

The second type: when the second display interface includes at least two icons, for example, when the second display interface is the same as the first display interface or the second display shown in FIG. 4D , FIG. 4E , FIG. 5 or FIG. 6 When the interfaces are the same or similar, the number of icons displayed on the second control interface is the same as the number of icons displayed on the second display interface, and the arrangement of the icons displayed on the second control interface is the same as the number of icons displayed on the second display interface. The icons displayed on the interface are arranged in the same manner, and the icons displayed on the second control interface are in one-to-one correspondence with the icons displayed on the second display interface. That is, the second control interface is the same as or similar to the first control interface or the second control interface shown in FIG. 4D , FIG. 4E , FIG. 5 or FIG. 6 .

The third type: when the second display interface is an audio and video playback interface, the second control interface includes any one or more of the following: a volume control item, a playback progress control item, and a play/stop control item. As shown in FIG. 10 , when the second terminal is playing an audio and video program, the first terminal presents a second control interface 1010 , which includes a volume control item 1012 , a playback progress control item 1013 and a play/stop control item 1015 . The volume, playback progress, or program playback or pause can be controlled through the control interface of the first terminal. It should be noted that the play progress control item 1013 and the play/stop control item 1015 are generally used to control non-live audio and video programs. For live audio and video programs, the second control interface may not include the play progress control item and the play/stop control item, or the play progress control item and the play/stop control item are invalid icons.

Further, after step 204, when the second display interface is an editable input interface and the second control interface is an input interface including a soft keyboard, the method may further include the following steps 205 and 206. For example, as shown in FIG. 7 and FIG. 8A and FIG. 8B , after the search button 480 on the first control interface 410 is clicked, the first terminal presents the second control interface 810A or 810B.

Step 205: The first terminal receives a second touch operation input on the soft keyboard of the second control interface. The second touch operation includes a selection operation of a first character on a soft keyboard of the second control interface.

The second control interface displayed by the first terminal includes a soft keyboard. When the user needs to input information in the second display interface, he can directly perform touch input in the soft keyboard on the second control interface, and the touch input is the second touch operation. The second touch operation may be a touch operation on the first character on the soft keyboard, for example, a click, double-click, long-press or re-press operation on the first character.

Step 206: The first terminal generates a second remote control command according to the second touch operation, and sends the second remote control command to the second terminal, where the second remote control command is used to instruct the second remote control command. The terminal inputs the first character in the second display interface.

The first terminal may generate a variety of second remote control instructions according to the second touch operation, specifically:

The first type: the first terminal receives a plurality of the second touch operations successively, and each second touch operation corresponds to a character; the first terminal can obtain the result according to two adjacent second touch operations. The relative positions of the characters corresponding to the two second touch operations respectively are converted, and then the relative positions are converted into one or more ordinary remote control commands, and the one or more remote control commands are used to represent the displacement of the cursor movement. For example, as shown in FIG. 8A , the first terminal first receives a second touch operation 811 for character H, and then receives a second touch operation 814 for character A, and the first terminal receives a second touch operation 811 according to the second touch operation 811 and the second touch operation 814, the relative position of the two characters can be obtained, that is, when the cursor is at the character H, the character A can be obtained by moving the cursor to the left 7 times; then the relative position is converted into seven moves to the left One remote command, each remote command means to move to the right once. It should be noted that, in the process of performing this input operation, each second touch operation is based on the previous second touch operation. The first second touch operation is based on the character corresponding to the initial position of the cursor. The initial position of the cursor refers to the position of the cursor when the second terminal just enters the second display interface. The initial position of the cursor may be sent by the second terminal to the first terminal, or the first terminal may obtain a default value in the memory, or the user may first perform an input operation according to the first terminal before starting the input operation. At the initial position of the cursor presented in the second terminal, a touch operation is performed on the character corresponding to the initial position of the cursor in the second control interface.

The second type: the first terminal receives a plurality of the second touch operations successively, and each second touch operation corresponds to a character; the first terminal can obtain the result according to two adjacent second touch operations. The relative positions of the characters corresponding to the two second touch operations are converted into a remote control command for representing the movement and displacement of the cursor. For example, as shown in FIG. 8A , the second touch operation 811 received by the first terminal first and the second touch operation 814 received later, it can be concluded that the character A can be obtained by moving the cursor to the left 7 times; then Convert this relative position into a remote command to move to the left seven times. The second terminal receives and parses the second remote control command, and then controls the cursor to move to the left seven times. In this manner, the same database may be preset in the first terminal and the second terminal, so that the second terminal can correctly parse the remote control command sent by the first terminal. In the same way as the first method, in the process of executing this input operation, each second touch operation is based on the previous second touch operation, and the first second touch operation is based on the character corresponding to the initial position of the cursor as benchmark.

The third type: the first terminal can convert two adjacent second touch operations into a remote control command representing the coordinate change of the touch screen. For example, as shown in FIG. 8A , the first terminal sends the coordinate information of the character H and the coordinate information of the character A to the second terminal as the second remote control command, or sends the coordinate information of the character H relative to the character A to the second terminal. The information is sent to the second terminal as the second remote control command. The second terminal receives and parses the second remote control command, and then controls the cursor to move to the left seven times. In this manner, the same database may also be preset in the first terminal and the second terminal, so that the second terminal can correctly parse the remote control command sent by the first terminal. In the same way as the first method, in the process of executing this input operation, each second touch operation is based on the previous second touch operation, and the first second touch operation is based on the character corresponding to the initial position of the cursor as benchmark.

The fourth type: the first terminal directly sends the character corresponding to the second touch operation as a second remote control command to the second terminal, and the second terminal directly recognizes and displays the character. In this manner, the second terminal can obtain the corresponding character directly according to the current second touch operation without depending on the initial position of the cursor and the last second touch operation.

Further, after step 204, when the second display interface includes at least two icons, the layout (ie the number and arrangement) of the icons in the second control interface is the same as that of the icons in the second display interface When the layouts are the same, the method may further include the following step 207. For example, in FIG. 5( c ), after the first icon 4123 is selected, as shown in FIG. 6 , the second terminal displays the second display interface 601 , and the first terminal displays the second control interface 610 .

Step 207: The first terminal receives the touch operation input on the second control interface, generates a remote control command according to the touch operation, and then sends the remote control command to the second terminal to control the second terminal. The second display interface is the same as or similar to the aforementioned first display interface, and the second control interface is the same as or similar to the aforementioned first control interface. Therefore, the specific steps of the step 207 can refer to the above steps The descriptions in 201-step 203 are not repeated here. For example, as shown in FIG. 6, the initial position of the cursor corresponds to the icon movie 1. After the icon movie 8 is selected, the first terminal can send a remote control command to the second terminal, so that the second terminal can move the cursor to the right Move three times up and one down.

Further, after step 204, when the second display interface is an audio and video playback interface, the second control interface includes any one or more of the following: volume control item, playback progress control item, play/stop control item, the method may further include the following steps 208 and 209.

Step 208: The first terminal receives a touch operation input on the second control interface. The touch operation can be a single-click, double-click, long-press, heavy-press or slide operation. For example, as shown in FIG. 10 , when the second terminal is playing an audio and video program, the user can adjust the volume by sliding left and right on the slider 912 , or adjust the playback by sliding left and right on the slider 913 The progress can also be controlled by clicking, double-clicking, long-pressing or re-pressing the button 915 to control the play and pause of the program.

Step 209, the first terminal generates a remote control command according to the touch operation in step 208, and sends the remote control command to the second terminal, and the remote control command is used to control the volume, progress and/or volume of the currently playing program. or playback status. Specifically, the first terminal may generate a remote control command for volume or playback progress according to the direction and distance of the touch and slide, and may also generate a remote control command for controlling the playback state according to the click state of a playback button (eg, button 915 ).

It should be noted that, the first control interface in step 201 may be triggered and displayed in various ways. For example, before step 201, this embodiment of the present invention may further include the following steps 220, 230, and steps 240A, 240B, or 240C.

Step 220: After entering the remote control interface, the first terminal displays thumbnails or icons of at least two control interfaces. As shown in FIG. 3 and FIG. 4A , after clicking the remote control icon 300 in FIG. 3 , the remote control interface 430 shown in FIG. 4A can be entered. The remote control interface 430 may include thumbnails or icons of multiple control interfaces, such as menu one, menu two, and menu three. At this time, the second terminal displays the first display interface, for example, the first display interface 401 in FIG. 4D .

Step 230: The first terminal receives a third touch operation. The third touch operation may be a selection of a thumbnail image or an icon displayed on the remote control interface 430, or the third touch operation may trigger the first terminal to send interface information to the second terminal request, or the third touch operation may trigger the first terminal to take a picture of the first display interface currently output by the second terminal. When the first terminal pre-stores a control interface corresponding to the first display interface currently output by the second terminal, step 240A is performed after step 230 . When the control interface corresponding to the first display interface currently output by the second terminal is not stored in the first terminal, step 240B or step 240C is performed after step 230 .

Step 240A, the first terminal selects the first control interface from the at least two control interfaces according to the third touch operation, and then in step 201 the first terminal displays the first control interface . Since the first control interface is pre-stored in the first terminal, the user can enter the first control interface by selecting a thumbnail or icon corresponding to the first control interface from the remote control interface 430 . For example, if the menu one pre-stored in the first terminal is the first control interface, the first control interface 410 can be entered by selecting the menu one, as shown in FIG. 4A and FIG. 4D .

Step 240B: The first terminal sends an interface information request to the second terminal according to the third touch operation; the first terminal receives the interface information of the first display interface sent by the second terminal. Then in step 201, the first terminal may acquire and display the first control interface according to the interface information. For example, as shown in FIG. 4A , the remote control interface 430 includes a request button 440 . When the first terminal does not have the first control interface, that is, menu 1, menu 2, and menu 3 are not the first control interface, then the request button 440 can be clicked to request the second terminal for the first control interface. Describe the interface information of the first display interface. Then the first terminal generates the first control interface according to the interface information of the first display interface, and can display a menu 460 corresponding to the first control interface on the remote control interface 430, as shown in FIG. 4C . Then, the first control interface can be entered by selecting menu 4 460 .

Step 240C: The first terminal starts to take pictures according to the third touch operation; the first terminal acquires a picture with the first display interface by taking pictures; the first terminal identifies the picture of the first display interface and then display the identified content as the first control interface in step 201 . For example, as shown in FIG. 4B , the remote control interface 430 includes a camera button 450 . When the first control interface does not exist in the first terminal, the photographing can be started by clicking the photographing button 450 . After acquiring the information of the first control interface, the first terminal may display menu 4 corresponding to the first control interface on the remote control interface 430, as shown in FIG. 4C . Then, the first control interface can be entered by selecting menu 4.

The process of acquiring the first control interface by the first terminal in steps 240B and 240C is similar to the foregoing process of acquiring the second control interface.

It should be noted that the first control interface in this embodiment may be the first control interface after the first terminal enters the remote control program, or may be any control interface after the first terminal enters the remote control program interface.

The second terminal described in the embodiments of the present invention may be a smart TV or an Internet TV, that is, a TV that can perform on-demand audio and video programs. In this case, the first display interface and the second display interface are output and displayed by the second terminal.

In other implementations, the second terminal described in the embodiments of the present invention may be a set-top box, which cooperates with a TV or other display device to play audio and video interfaces, and can perform on-demand audio and video programs through the set-top box. In this case, the first display interface and the second display interface are output by the second terminal to an external device such as a TV, and then displayed by the external device.

In the remote control method provided by the embodiment of the present invention, the control interface displayed by the first terminal can change with the change of the display interface output by the second terminal, and the layout of the control interface is consistent with the current display interface, so that the user The display interface of the second terminal can be controlled by a small amount of touch operations on the control interface, thereby making the use of the remote control more convenient. However, in the prior art, when the TV is remotely controlled by a common remote control, it requires more operations of the direction keys or other similar keys to complete the same remote control operation as the embodiment of the present invention. In addition, in the prior art solution of using a mobile phone as a remote control, the mobile phone displays virtual keys identical to those of an ordinary remote control, and then uses the same operation method as the ordinary remote control to control the TV. Compared with the prior art, the remote control method provided by the embodiment of the present invention is simpler and more convenient.

The remote control method provided by the embodiment of the present invention is mainly applied to the remote control of a television or a similar display device. The first terminal can be used as a remote controller for various electrical appliances. That is, after the first terminal opens the remote control application, remote control interfaces corresponding to various electrical appliances can be displayed on the interface, and the user can select a remote control interface matching the electrical appliance to be remotely controlled. For example, the first terminal may display electrical appliances used for remote control of TVs, set-top boxes, DVDs, air conditioners, and electric fans.

FIG. 11 is a flowchart of another remote control method provided by an embodiment of the present invention. In this method, the first terminal takes a mobile phone as an example, and the second terminal takes a set-top box as an example. The interface output by the second terminal can be displayed on a TV, computer or other display device. This method takes a TV as an example. The method is based on the embodiments described above with reference to FIGS. 1-10 . As shown in Figure 11, the method includes:

Step 1101 , the mobile phone opens a preset remote control application.

Step 1102: Select a corresponding control interface according to a scene that needs to be operated on the TV screen. Wherein, the user can manually select the corresponding control interface in the mobile phone according to the current display content of the TV screen, or the set-top box can send the content currently displayed on the TV to the mobile phone, and the mobile phone can automatically select the corresponding control interface. If the current TV interface displays a menu interface, go to step 1103; if the current TV interface displays an editable input interface, go to step 1108; if the current TV interface is playing an audio and video interface, go to step 1110 .

Step 1103, the mobile phone enters the menu control interface of the remote control application. According to whether a control interface corresponding to the display interface of the current TV is stored in the mobile phone, after step 1103, step 1104, step 1105 and step 1106 may be executed respectively.

Step 1104: The mobile phone selects a control interface corresponding to the current TV display interface from the pre-stored control interfaces according to the user's selection operation. When a control interface corresponding to the display interface of the current TV is stored in the mobile phone, the user can directly select the control interface on the mobile phone.

Step 1105: The mobile phone takes a picture of the TV screen according to the operation triggered by the user to start taking pictures, and then automatically cuts and analyzes the taken picture, identifies the menu options in the picture, and generates a corresponding control interface.

Step 1106: The mobile phone sends an interface information request to the set-top box according to the user's trigger, and then receives the interface information of the current TV sent by the set-top box, and generates a corresponding control interface according to the interface information. After step 1105 or step 1106, the mobile phone can save the generated control interface, so that the user can directly select the control interface in the next use without taking pictures or requesting the set-top box.

After step 1104 , step 1105 and step 1106 , the method executes step 1107 .

Step 1107: The mobile phone acquires the user's touch operation on the screen of the mobile phone, and then converts the touch operation into a remote control command. For example, the mobile phone remote control application converts the touch operation F1 into the remote control instruction O1=A1(F1) through the instruction library A1. After the mobile phone generates the remote control command, the method executes step 1112 .

Step 1108, the mobile phone enters the input interface of the remote control application.

Step 1109 , the mobile phone receives characters such as English or Pinyin letters input by the user, and then converts the character F2 into a remote control command O2=A2(F2) through the command library A2, and then executes Step 1112 .

Step 1110, the mobile phone enters the volume/playback adjustment interface of the remote control application.

Step 1111: The mobile phone receives the user's sliding or clicking operation on the volume/play adjustment interface, and converts the sliding or clicking operation F3 into a remote control command O3=A3(F3) through the instruction library A3, and then executes step 1112.

Step 1112, the mobile phone sends the generated remote control instruction O1, O2 or O3 to the set-top box.

Step 1113: After receiving the remote control command, the set-top box parses the O1, O2 or O3 command into a corresponding operation through the command library matching the command library A1, and then executes the corresponding operation to realize menu jump, volume adjustment, and playback. Control or enter characters.

After step 1112, the method may further include the following steps:

Step 1114 , the mobile phone performs the jumping of the control interface according to the menu jumping of the TV. Wherein, the user may select the corresponding control interface on the mobile phone according to the interface change of the TV. The jumping of the control interface can also be automatically realized by the mobile phone. The mobile phone can also request the set-top box for the interface after the jump, or the mobile phone can take a picture of the interface after the jump. In addition, the mobile phone can record the interface before the jump, and record the corresponding relationship between the touch operation F1 and the interface after the jump, so that when the same touch operation F1 is performed next time, the mobile phone can realize the automatic jump of the interface, reducing the number of repetitions. Take pictures or reduce the number of requests to the set-top box.

After steps 1107 and 1112 , the TV may jump to an editable input interface, and at this time, correspondingly, the mobile phone may jump to the input interface described in step 1108 .

For the process of converting a touch operation into a remote control command in step 1107, step 1109 and step 1111, reference may be made to the specific descriptions in the foregoing embodiments.

On the basis of the foregoing embodiments, an embodiment of the present invention may further provide a method for remote control through gestures. The mobile phone supports a variety of gesture operations. Using the advantages of gesture operations can improve the efficiency of remote control operations and improve user operation experience. The embodiment of the present invention enumerates several gesture operations, but is not limited to these gesture operations. All gesture operations supported by mobile phones can be applied to remote control operations, and users can configure various gesture operations according to their own preferences. . In the remote control methods described in the above embodiments, the remote control by gestures may be further implemented by the method shown in FIG. 12 . FIG. 12 is a flowchart of a method for remote control through gestures provided by an embodiment of the present invention. As shown in Figure 12, the method includes:

Step 1201, the mobile phone opens the remote control application.

Step 1202: The remote control application of the mobile phone recognizes the user's gesture operation, and converts the gesture into a corresponding remote control command. The correspondence between gesture operations and remote control commands may be preset. The corresponding relationship may be the default setting of the mobile phone when it leaves the factory, or may be set and edited by the user.

Step 1203, the mobile phone sends the remote control command to the set-top box.

Step 1204: The set-top box receives the remote control instruction, converts the remote control instruction into an executable operation instruction, and executes the operation corresponding to the operation instruction. For example, the set-top box can perform operations such as returning to the previous layer, shutting down, fast forwarding or rewinding audio and video, and returning to the main menu according to the user's gesture operation. Wherein, the user's gesture operation may be various operations that are easy to recognize and not easily misjudged, such as a circle drawing operation, a polyline operation, or an operation of other specific graphics. It should be noted that the user's gesture may be a gesture of contacting the touch screen of the mobile phone, and may also be a gesture of non-contacting the touch screen of the mobile phone.

The first terminal provided in each embodiment of the present invention may also have a voice recognition function. In the process of remote control, the voice recognition function can be activated to convert the voice issued by the user into a remote control command to remotely control the second terminal.

In each embodiment of the present invention, the communication mode between the first terminal and the second terminal may be infrared, Bluetooth, Wi-Fi, or NFC. That is, the first terminal can send the remote control command to the second terminal through infrared, Bluetooth, Wi-Fi or NFC technology, and the second terminal can also use any one of these communication methods when sending interface information to the first terminal.

In the embodiment of the present invention, a widely used handheld terminal can be used to realize the operation of a touch-screen remote control of a set-top box or a TV, thereby reducing dependence on a common remote control and reducing hardware costs.

The embodiments of the present invention provide a simple and quick touch-screen remote control operation method for different usage scenarios, so as to improve the user's remote control operation efficiency and experience.

An embodiment of the present invention further provides a graphical user interface (GUI), where the GUI is stored in a first terminal, where the first terminal includes a touch screen, one or more memories, and one or more processors, the one or more A plurality of processors for executing one or more computer programs stored in the one or more memories. Wherein, the GUI includes:

The first GUI displayed on the touch screen is consistent with the first display interface currently output by the second terminal, and the second terminal is a terminal remotely controlled by the first terminal; the first display The interface displays at least two icons, the number of icons displayed on the first GUI is the same as the number of icons displayed on the first display interface, and the icons displayed on the first GUI are arranged in the same manner as the first display interface. The displayed icons are arranged in the same manner, and the icons displayed on the first GUI correspond one-to-one with the icons displayed on the first display interface;

In response to the user's first gesture on the touch screen, a first remote control instruction is generated according to the first gesture, and the first remote control instruction is sent to the second terminal; the first gesture includes a selection operation of a first icon in the first GUI, where the first remote control command is used to instruct the second terminal to select an icon corresponding to the first icon in the first display interface;

In response to the user's first gesture or third gesture on the touch screen, a second GUI is displayed on the touch screen, and the second GUI is consistent with the second display interface output by the second terminal, so The second display interface is a display interface output after the second terminal executes the first remote control command.

Wherein, the second display interface is an editable input interface, and the second GUI is an input interface including a soft keyboard. The interface layout of the soft keyboard in the second GUI is the same as or different from the interface layout of the soft keyboard in the second display interface.

Further, the GUI further includes: in response to a second gesture input by the user on the soft keyboard of the second GUI, generating a second remote control instruction according to the second gesture, and converting the second remote control instruction sent to the second terminal; the second gesture includes a selection operation on the first character on the soft keyboard of the second GUI, and the second remote control instruction is used to instruct the second terminal to The first character is input in the second display interface.

It should be noted that the first GUI described in this embodiment is equivalent to the first control interface in the foregoing embodiments, and the second GUI is equivalent to the second control interface in the foregoing embodiments. The first gesture and the second gesture in this embodiment are respectively equivalent to the first touch operation and the second touch operation in the foregoing embodiments. The third gesture in this embodiment is equivalent to the operation of requesting interface information or the operation of initiating photographing in the foregoing embodiments. For details of the GUI, reference may be made to the descriptions in the foregoing embodiments, and details are not repeated here.

An embodiment of the present invention further provides a remote control device, which is included in the first terminal, and the remote control device has a function of implementing the behavior of the first terminal in the foregoing method embodiments. The functions can be implemented by hardware, or can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions.

From the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by hardware, firmware, or a combination thereof. When implemented in software, the functions described above may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage medium can be any available medium that a computer can access. By way of example and not limitation, computer readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or be capable of carrying or storing instructions or data structures in the form of desired program code and any other medium that can be accessed by a computer. Also, any connection can be a computer-readable medium as appropriate. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable , fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the fusing of the pertinent medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and blu-ray disc, where disks usually reproduce data magnetically, and discs Lasers are used to optically copy data. Combinations of the above should also be included within the scope of computer-readable media.

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention. should be included within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (26)

1. A remote control method applied to a first terminal having a touch screen, the method comprising:

the first terminal displays a first control interface, the first control interface is consistent with a first display interface currently output by a second terminal, and the second terminal is a terminal remotely controlled by the first terminal; the first display interface displays at least two icons, the number of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, the arrangement mode of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, and the icons displayed by the first control interface are in one-to-one correspondence with the icons displayed by the first display interface;

the first terminal receives a first touch operation input on the first control interface, wherein the first touch operation comprises a selection operation of a first icon in the first control interface;

the first terminal generates a first remote control instruction according to the first touch operation;

the first terminal sends the first remote control instruction to the second terminal, wherein the first remote control instruction is used for indicating the second terminal to select the icon corresponding to the first icon in the first display interface;

the first terminal receives an operation for starting photographing;

the first terminal obtains a picture with a second display interface by photographing; the second display interface is output after the second terminal executes the first remote control instruction;

the first terminal identifies content in the picture of the second display interface and displays the identified content as a second control interface, wherein the second control interface is consistent with a second display interface output by the second terminal;

the first terminal saves the second control interface;

when the second terminal is controlled by the first terminal, the second terminal can be simultaneously controlled by other remote controllers matched with the second terminal.

2. The method of claim 1, wherein the first terminal generating a first remote control instruction according to the first touch operation comprises:

the first terminal acquires an initial position of a cursor before the first icon is selected;

and the first terminal generates the first remote control instruction according to the initial position of the cursor and the position of the first icon.

3. The method according to claim 2, wherein the first touch operation further comprises an operation of selecting a second icon corresponding to the initial position of the cursor before the first icon is selected, wherein the second icon is included in the first control interface; the initial cursor position is an initial cursor position on the first display interface.

4. A method according to any of claims 1-3, wherein the second display interface is an editable input interface and the second control interface is an input interface comprising a soft keyboard.

5. The method of claim 4, wherein the interface layout of the soft keyboard in the second control interface is the same as the interface layout of the soft keyboard in the second display interface.

6. The method of claim 4, wherein the interface layout of the soft keyboard in the second control interface is different from the interface layout of the soft keyboard in the second display interface.

7. The method of claim 4, further comprising:

the first terminal receives a second touch operation input on the soft keyboard of the second control interface, wherein the second touch operation comprises a selection operation on a first character on the soft keyboard of the second control interface;

the first terminal generates a second remote control instruction according to the second touch operation;

and the first terminal sends the second remote control instruction to the second terminal, and the second remote control instruction is used for indicating the second terminal to input the first character in the second display interface.

8. The method according to any one of claims 1-3, wherein the second control interface is consistent with a second display interface output by the second terminal, and comprises:

when the second display interface comprises at least two icons, the number of the icons displayed by the second control interface is the same as the number of the icons displayed by the second display interface, the arrangement mode of the icons displayed by the second control interface is the same as the arrangement mode of the icons displayed by the second display interface, and the icons displayed by the second control interface are in one-to-one correspondence with the icons displayed by the second display interface; or

When the second display interface is an editable input interface, the second control interface is an input interface comprising a soft keyboard; or

When the second display interface is an audio and video playing interface, the second control interface comprises any one or more of the following: volume control item, play progress control item, play/stop control item.

9. The method of any of claims 1-3, further comprising:

before the first terminal displays the first control interface, the first terminal displays thumbnails or icons of at least two control interfaces;

the first terminal receives a third touch operation;

and the first terminal selects the first control interface from the at least two control interfaces to display according to the third touch operation.

10. A method according to any one of claims 1 to 3, wherein:

the first display interface and the second display interface are displayed by the second terminal; or

And the first display interface and the second display interface are output to an external device by the second terminal for display.

11. A method according to any one of claims 1 to 3, wherein:

the first terminal sends the remote control instruction to the second terminal through any one of the following communication modes: infrared, Bluetooth, Wi-Fi, and NFC; or

The first terminal is a mobile phone or a tablet computer, and the second terminal is a television or a set-top box; or

The method further comprises the following steps: the first terminal identifies a gesture of a user, converts the gesture into a third remote control instruction and sends the third remote control instruction to the second terminal; or

The method further comprises the following steps: and the first terminal identifies the voice of the user, converts the voice into a fourth remote control instruction and sends the fourth remote control instruction to the second terminal.

12. A first terminal, wherein said first terminal is adapted to remotely control a second terminal, said first terminal comprising:

a touch screen;

one or more processors;

one or more memories having one or more computer programs stored therein, the one or more computer programs comprising instructions, which when executed by the first terminal, cause the first terminal to perform the steps of:

displaying a first control interface through the touch screen, wherein the first control interface is consistent with a first display interface currently output by the second terminal; the first display interface displays at least two icons, the number of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, the arrangement mode of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, and the icons displayed by the first control interface are in one-to-one correspondence with the icons displayed by the first display interface;

receiving a first touch operation input on the first control interface through the touch screen, wherein the first touch operation comprises a selection operation of a first icon in the first control interface;

generating a first remote control instruction according to the first touch operation;

sending the first remote control instruction to the second terminal, wherein the first remote control instruction is used for indicating the second terminal to select an icon corresponding to the first icon in the first display interface;

receiving an operation for starting photographing;

acquiring a picture with a second display interface by photographing; the second display interface is output after the second terminal executes the first remote control instruction;

identifying content in the picture of the second display interface, and displaying the identified content as a second control interface through the touch screen, wherein the second control interface is consistent with a second display interface output by the second terminal;

saving the second control interface;

when the second terminal is controlled by the first terminal, the second terminal can be simultaneously controlled by other remote controllers matched with the second terminal.

13. The first terminal according to claim 12, wherein in the step of generating a first remote control instruction according to the first touch operation, the first terminal executes the instruction to cause the first terminal to perform the steps of:

acquiring an initial position of a cursor before the first icon is selected;

and generating the first remote control instruction according to the initial position of the cursor and the position of the first icon.

14. The first terminal according to claim 13, wherein the first touch operation further comprises an operation of selecting a second icon corresponding to the initial position of the cursor before the first icon is selected, the second icon being included in the first control interface; the initial cursor position is an initial cursor position on the first display interface.

15. The first terminal according to any of claims 12-14, wherein the second display interface is an editable input interface and the second control interface is an input interface comprising a soft keyboard.

16. The first terminal of claim 15, wherein the interface layout of the soft keyboard in the second control interface is the same as the interface layout of the soft keyboard in the second display interface.

17. The first terminal of claim 15, wherein an interface layout of the soft keyboard in the second control interface is different from an interface layout of a soft keyboard in the second display interface.

18. The first terminal of claim 15, wherein execution of the instructions by the first terminal causes the first terminal to further perform the steps of:

receiving a second touch operation input on the soft keyboard of the second control interface through the touch screen, wherein the second touch operation comprises a selection operation on a first character on the soft keyboard of the second control interface;

generating a second remote control instruction according to the second touch operation;

and sending the second remote control instruction to the second terminal, wherein the second remote control instruction is used for indicating the second terminal to input the first character in the second display interface.

19. The first terminal of any of claims 12-14, wherein the second control interface is consistent with a second display interface output by the second terminal, comprising:

when the second display interface comprises at least two icons, the number of the icons displayed by the second control interface is the same as the number of the icons displayed by the second display interface, the arrangement mode of the icons displayed by the second control interface is the same as the arrangement mode of the icons displayed by the second display interface, and the icons displayed by the second control interface are in one-to-one correspondence with the icons displayed by the second display interface; or

When the second display interface is an editable input interface, the second control interface is an input interface comprising a soft keyboard; or

When the second display interface is an audio and video playing interface, the second control interface comprises any one or more of the following: volume control item, play progress control item, play/stop control item.

20. The first terminal of any of claims 12-14, wherein execution of the instructions by the first terminal causes the first terminal to further perform the steps of:

before the first control interface is displayed through the touch screen, displaying thumbnails or icons of at least two control interfaces through the touch screen;

receiving a third touch operation;

and selecting the first control interface from the at least two control interfaces for displaying according to the third touch operation.

21. The first terminal according to any of claims 12-14, characterized by:

the first display interface and the second display interface are displayed by the second terminal; or

And the first display interface and the second display interface are output to an external device by the second terminal for display.

22. A first terminal, wherein said first terminal is adapted to remotely control a second terminal, said first terminal comprising:

the display unit is used for displaying a first control interface, the first control interface is consistent with a first display interface currently output by a second terminal, and the second terminal is a terminal remotely controlled by the first terminal; the first display interface displays at least two icons, the number of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, the arrangement mode of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, and the icons displayed by the first control interface are in one-to-one correspondence with the icons displayed by the first display interface;

the input unit is used for receiving a first touch operation input on the first control interface, and the first touch operation comprises a selection operation of a first icon in the first control interface;

the control unit is used for generating a first remote control instruction according to the first touch operation received by the input unit;

the sending unit is used for sending the first remote control instruction to the second terminal, wherein the first remote control instruction is used for indicating the second terminal to select the icon corresponding to the first icon in the first display interface;

the input unit is also used for receiving the operation for starting photographing;

the control unit is further used for acquiring a picture with a second display interface by photographing, identifying content in the picture of the second display interface, using the identified content as a second control interface and storing the second control interface; the second display interface is output after the second terminal executes the first remote control instruction;

the display unit is further configured to display the second control interface, where the second control interface is consistent with a second display interface output by the second terminal;

when the second terminal is controlled by the first terminal, the second terminal can be simultaneously controlled by other remote controllers matched with the second terminal.

23. The first terminal of claim 22, wherein the second display interface is an editable input interface and the second control interface is an input interface including a soft keyboard.

24. The first terminal of claim 23,

the input unit is further used for receiving a second touch operation input on the soft keyboard of the second control interface, wherein the second touch operation comprises a selection operation on a first character on the soft keyboard of the second control interface;

the control unit is further used for generating a second remote control instruction according to the second touch operation;

the sending unit is further configured to send the second remote control instruction to the second terminal, where the second remote control instruction is used to instruct the second terminal to input the first character in the second display interface.

25. An electronic device, characterized in that the electronic device comprises means for performing the method according to any of claims 1 to 11.

26. A computer-readable storage medium comprising instructions that, when executed on an electronic device, cause the electronic device to perform the method of any of claims 1-11.

Images