CN103049190B - A kind of method of mobile communication terminal and control appliance operation - Google Patents

Abstract

The present invention provides a mobile communication terminal and a method for controlling the operation of the device, wherein the mobile communication terminal includes: a receiving module for receiving a control operation instruction for a user to control and operate a designated device through a set HTML page; the control module, It is used to send the control operation instruction to the designated device through the HTML page, so that the designated device performs the control operation indicated by the control operation instruction; wherein, the receiving module is configured to use the device to A specified HTML page monitors the control operation performed by the user on the specified device through the HTML page, obtains information about the control operation through a DOM event of the HTML page, and generates the control operation according to the information about the control operation Operating instructions. Through the present invention, the remote control of the equipment is achieved, and the effect of improving the user's operation experience is achieved.

Description

A mobile communication terminal and method for controlling device operation

technical field

The invention relates to the field of computer technology, in particular to a mobile communication terminal and a method for controlling the operation of the device.

Background technique

With the development of the Internet, computers are playing an increasingly important role in people's lives, and people's lives and work are increasingly dependent on computers. In order to control the computer, the computer has an input device. For example, a common input device is a keyboard and a mouse to operate the computer, wherein the keyboard is usually used for information input, and the mouse is used for selection or some common operations; A common input device is to use a keyboard and a touchpad to operate a computer, and is mostly used in a portable computer, wherein the keyboard is usually used for information input, and the touchpad is used for selection or some common operations.

However, when using the above-mentioned first input device to control the computer, both the keyboard and the mouse are physical media, which take up space and have requirements for placement, and can only operate the computer at close range; use the above-mentioned second input device to control the computer. When controlling, both the keyboard and the touchpad are built into the computer, and the computer can only be operated at close range, so there are limitations in operation.

Similarly, in the Internet of Things, people usually need to use dedicated controllers to control IoT devices, and there are also location requirements, which can only be operated at close range.

It can be seen that using existing input devices to control computers or Internet of Things devices has various limitations, especially it is impossible for users to control computers or Internet of Things devices at a relatively long distance, which reduces the user's ability to operate computers or Internet of Things devices. experience.

Contents of the invention

In view of the fact that the above-mentioned existing input devices cannot enable users to control computers or Internet of Things devices at a relatively long distance, which reduces the user's operating experience on computers or Internet of Things devices, the present invention is proposed to provide a method that overcomes the above problems or A mobile communication terminal and a method of controlling operation of the device at least partially address the above problems.

According to one aspect of the present invention, a method for controlling device operation is provided, including: the mobile communication terminal receives a control operation instruction for a user to control and operate a specified device through a set HTML page, wherein the HTML page is used to provide the user with a mobile communication The interface for the terminal to control and operate the specified device, receives the control operation command generated by the user on the interface operation, and sends the control operation command; the mobile communication terminal sends the control operation command to the specified device through the HTML page, so that the specified device executes the control operation command Indicated control operation; wherein, the step of the mobile communication terminal receiving the user's control operation instruction for the specified device through the set HTML page includes: the set HTML page monitors the user through the HTML page For the control operation performed by the page on the specified device, the information of the control operation is obtained through the DOM event of the HTML page, and the control operation instruction is generated according to the information of the control operation.

According to another aspect of the present invention, a mobile communication terminal is provided, including: a receiving module, configured to receive a control operation instruction for the user to control and operate a specified device through a set HTML page, wherein the HTML page is used to provide the user The interface for controlling and operating the specified device through the mobile communication terminal receives the control operation command generated by the user on the interface operation, and sends the control operation command; the control module is used to send the control operation command to the specified device through the HTML page, so that the specified The device executes the control operation indicated by the control operation instruction; wherein, the receiving module is configured to monitor the control operation performed by the user on the designated device through the HTML page through the set HTML page, and through the The DOM event of the HTML page acquires the information of the control operation, and generates the control operation instruction according to the information of the control operation.

According to the scheme of controlling device operation of the present invention, the mobile communication terminal receives the user's control operation on the designated device through the HTML page, sends a control operation instruction to the designated device, and makes the designated device perform corresponding control operation according to the control operation instruction. The HTML page is widely used and supported by almost all operating systems. The operation control of the mobile communication terminal to the designated device is realized through the HTML page. There is no need for special operation control software on the mobile communication terminal side and the designated device side, and the operation is convenient. The implementation is simple, the compatibility is good, and the implementation cost is low. Through the solution of the present invention, the operation of a specified device (such as a PC or an Internet of Things device) can be controlled through a mobile communication terminal at a long distance. The mobile communication terminal of the present invention can replace the traditional input device, realize the convenient operation of the device when it is far away from the device, monitor the device in real time, and fully control the device, thereby solving the problem that the existing input device cannot realize the user in a relatively Remote control of computers or Internet of Things devices reduces the user's operating experience on computers or Internet of Things devices, achieves remote control of devices, and improves user experience.

The above description is only an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable , the specific embodiments of the present invention are enumerated below.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiment. The drawings are only for the purpose of illustrating a preferred embodiment and are not to be considered as limiting the invention. Also throughout the drawings, the same reference numerals are used to designate the same components. In the attached picture:

FIG. 1 is a flow chart of steps of a method for controlling device operation according to Embodiment 1 of the present invention;

FIG. 2 is a flow chart of steps of a method for controlling device operation according to Embodiment 2 of the present invention;

FIG. 3 is a flow chart of steps of a method for controlling device operation according to Embodiment 3 of the present invention;

Fig. 4 is a schematic interface diagram of a touchpad HTML page in the embodiment shown in Fig. 3;

Fig. 5 is a schematic interface diagram of a shortcut operation HTML page in the embodiment shown in Fig. 3;

Fig. 6 is a schematic interface diagram of a selection terminal HTML page in the embodiment shown in Fig. 3;

FIG. 7 is a flow chart of steps of a method for controlling device operation according to Embodiment 4 of the present invention;

Fig. 8 is a structural block diagram of a mobile communication terminal according to Embodiment 5 of the present invention.

detailed description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

Embodiment one

Referring to FIG. 1 , it shows a flowchart of steps of a method for controlling device operation according to Embodiment 1 of the present invention.

The method for controlling the operation of equipment in this embodiment includes the following steps:

Step S102: the mobile communication terminal receives a control operation command for the user to control and operate the specified device through the set HTML page.

Wherein, the HTML page is used to provide an interface for the user to control and operate the designated device through the mobile communication terminal, receive control operation instructions generated by the user's operation on the interface, and send the control operation instructions.

Step S104: the mobile communication terminal sends the control operation instruction to the specified device through the HTML page, so that the specified device performs the control operation indicated by the control operation instruction.

Through this embodiment, the mobile communication terminal receives the user's control operation on the designated device through the HTML page, sends a control operation instruction to the designated device, and makes the designated device perform corresponding control operations according to the control operation instruction. The HTML page is widely used and supported by almost all operating systems. The operation control of the mobile communication terminal to the designated device is realized through the HTML page. There is no need for special operation control software on the mobile communication terminal side and the designated device side, and the operation is convenient. The implementation is simple, the compatibility is good, and the implementation cost is low. Through the solution of the present invention, the operation of a specified device (such as a PC or an Internet of Things device) can be controlled through a mobile communication terminal at a long distance. The mobile communication terminal of this embodiment can replace the traditional input device, realize the convenient operation of the device when it is far away from the device, monitor the device in real time, and fully control the device, thereby solving the problem that the existing input device cannot realize the user Controlling computers or IoT devices at a relatively long distance reduces the problem of users' operating experience on computers or IoT devices, achieves remote control of devices, and improves user operating experience.

Embodiment two

Referring to FIG. 2 , it shows a flowchart of steps of a method for controlling device operation according to Embodiment 2 of the present invention.

In this embodiment, the network control server is used to connect the mobile communication terminal and the designated device, and WebSocket connections are established between the mobile communication terminal and the network control server, and between the network control server and the designated device.

The WebSocket protocol is a new protocol in HTML5, which realizes full-duplex communication between the browser and the server. The WebSocket protocol is relatively simple. The device, like a normal browser, performs a handshake with the server through port 80 or 443. The server identifies whether it is a WebSocket request according to the HTTPHEADER. If it is, the request is upgraded to a WebSocket connection. After the handshake is successful, enter The data transmission phase of the two-way persistent connection.

When using the WebSocket protocol, NPAPI can be set on the specified device side. NPAPI (NetscapePluginApplicationProgrammingInterface, Netscape plug-in application programming interface) is a common interface for executing external applications in the browser. Each plug-in in the Netscape plug-in declares that it supports one or several types of content. When the browser encounters this type of content, it will look for a plug-in that can decode it in the plug-in library, and hand over this part of the content to It is parsed and displayed in the corresponding position of the web page. Through the NPAPI in the browser, the corresponding API (Application Programming Interface, application program interface) in the system can be called to realize corresponding functions, such as shutting down, adjusting volume, controlling mouse operation, etc. For example, through the NPAPI mechanism (currently, this mechanism has been applied to browsers such as chromium, browsers developed based on the Chromium open source project, and firefox), the browser provides the javascript scripts in the extension program web pages with SetCursorPos, Shutdown, Snapshot, and MouseDown , MouseUp, KeyDown, KeyUp and other interface functions, javascript communicates with the server through websocket, and after receiving the control command, calls these javascript interface functions expanded through NPAPI to control the cursor position, simulate mouse clicks and keyboard presses, and the computer Snapshot, computer shutdown and other functions.

Based on the above settings, the method for controlling equipment operation in this embodiment includes the following steps:

Step S202: the mobile communication terminal acquires information of at least one designated device to be controlled from the network control server.

The network control server can pre-store the information of one or more designated devices to be controlled, and send the information of these designated devices to be controlled to the mobile communication terminal when receiving a request from the mobile communication terminal; When receiving the request from the mobile communication terminal, it sends information to the designated device connected to it, obtains the information of one or more designated devices to be controlled, and then returns it to the mobile communication terminal.

In this embodiment, the mobile communication terminal sends a request to the network control server through the WebSocket connection, and obtains information of at least one designated device to be controlled from the network control server through the WebSocket connection. That is, the communication between the mobile communication terminal and the network control server follows the WebSocket protocol. Similarly, there is also a WebSocket connection between the network control server and the specified device, and the communication between the two also follows the WebSocket protocol.

Preferably, when the mobile communication terminal obtains information of at least one specified device to be controlled from the network control server, the information of the specified device includes at least one of the following: a two-dimensional code of the specified device, an identifier of the specified device, and an IP address of the specified device; The mobile communication terminal identifies and displays the designated device through the information of the designated device. Or, the network control server identifies the designated device according to the information of the designated device to be controlled, and the information of the designated device includes at least one of the following: the two-dimensional code of the designated device, the identification of the designated device, the IP address of the designated device; the mobile communication terminal The information of the designated device identified by the network control server is obtained from the network control server.

Step S204: The mobile communication terminal selects a designated device to be controlled according to user input, and sends authentication information to the network control server.

Wherein, the authentication information is used for requesting the control operation authorization of the selected designated device.

Step S206: the mobile communication terminal receives the confirmation authorization information returned by the network control server.

Through the authorization authentication, the safety of the mobile communication terminal to control the operation of the equipment can be ensured.

Step S208: the mobile communication terminal acquires the HTML page of the selected designated device.

The mobile communication terminal may acquire the HTML page of the selected specified device in various ways.

For example, a mobile communication terminal uses a WebSocket connection to send an information acquisition command to a network control server, wherein the information acquisition command is used to acquire the control operations supported by a specified device; the network control server uses a WebSocket connection to send an information acquisition command to a specified device, so that the specified device According to the information acquisition command, the control operation supported by the specified device is obtained through the NPAPI function and returned to the network control server, which is forwarded to the mobile communication terminal by the network control server; the mobile communication terminal returns the control operation information supported by the specified device according to the network control server. Controls corresponding to the control operations supported by the specified device are displayed in the HTML page, and finally an HTML page corresponding to the specified device is generated.

For another example, the mobile communication terminal uses a WebSocket connection to send an information acquisition command to the network control server, wherein the information acquisition command is used to acquire the control operations supported by the specified device; the network control server uses the WebSocket connection to send the information acquisition command to the specified device, so that the specified According to the information acquisition command, the device obtains the control operation supported by the specified device through the NPAPI function and returns it to the network control server; the network control server generates the corresponding HTML page according to the information of the control operation supported by the specified device and returns it to the mobile communication terminal.

For another example, the mobile communication terminal has pre-built HTML pages with settings, and these pre-built HTML pages include at least one of the following: a touch pad page, a shortcut operation page, and a terminal selection page, wherein the touch pad page is used to receive user touch The touch operation information on the panel page generates control operation instructions according to the touch operation information to control the cursor operation of the specified device; the shortcut operation page includes at least one of the following controls: volume adjustment control, device desktop display control, device screen capture control and device Shutdown control, the shortcut operation page is used to receive the user's operation information on the controls in the shortcut operation page, and generate control operation instructions according to the operation information of the controls, so as to control the designated device to perform the control operation indicated by the control operation instruction; select the terminal page It is used to display the information of the equipment to be controlled and the controlled status of the equipment to be controlled, so that the user can select the equipment to be controlled. The terminal selection page also includes an authorization password input control for the user to enter the authorization password for control operation authentication. After the mobile communication terminal obtains the authorization of the designated device, the mobile communication terminal correspondingly displays all or part of the HTML pages constructed in advance for the user to operate. It should be noted that in practical applications, those skilled in the art can construct one or more of the above HTML pages according to actual needs, or integrate one or more of the above HTML pages into one HTML page, using the HTML page It can realize the operation function of the specified equipment, and it is flexible and easy to expand.

Step S210: the mobile communication terminal receives a control operation instruction for the user to control and operate the designated device through the set HTML page.

In this embodiment, the HTML page in the mobile communication terminal monitors the user's control operation on the specified device through the HTML page, and obtains the information of the control operation through the DOM (DocumentObjectModel, Document Object Model) event of the HTML page. Generate control operation instructions.

DOM is a standard programming interface for processing markup languages recommended by the W3C organization. It is a platform- and language-independent application programming interface that can dynamically access programs and scripts, update their content, structure and style of www documents. Event processing is the most important part of DOM. A perfect event mechanism enables programs such as JavaScript programs to trigger different execution methods according to specific events, making programs more interactive and intelligent. For example, the mouse movement data on the HTML page can be obtained through the touchstart event, touchmove event, and touchend event.

Step S212: the mobile communication terminal uses the WebSocket connection, and sends the control operation instruction to the network control server through the HTML page.

Among them, the network control server can be built using Node.js. Node.js is a JavaScript toolkit for writing high-performance web servers, with many advantages such as single-threaded, non-blocking IO, and event-driven. Of course, it is not limited to this, and the network control server can also be built in other appropriate ways, such as being built as an Apache (Apache) server. The network control server establishes a connection between the mobile communication terminal and the specified device controlled by the mobile communication terminal through the WebSocket protocol.

Step S214: the network control server forwards the control operation instruction to the specified device through the WebSocket connection.

Step S216: the designated device receives the control operation instruction, and according to the control operation instruction, calls the system API of the designated device through NPAPI to implement the control operation indicated by the control operation instruction.

Preferably, the specified device in this embodiment is installed with a browser and a browser extension program having an NPAPI mechanism. Among them, the browser extension program includes an NPAPI plug-in (that is, a DLL file), and the browser extension program is used to realize the WebSocket communication between the designated device and the network control server, and to call the system API of the designated device through the NPAPI plug-in.

The specified device listens to and receives the control operation instructions sent by the network control server through the HTML page of the browser extension program; then, the HTML page of the browser extension program calls the NPAPI function corresponding to the control operation indicated by the control operation instruction, and through the NPAPI function Call the corresponding system API function to realize the control operation indicated by the control operation instruction. For example, when the control operation indicated by the control operation instruction is a cursor movement operation, the NPAPI function called by the HTML page of the browser extension program of the specified device is SetCursorPos, and the NPAPI function calls the mouse_event function of the system to realize the movement of the cursor.

Through this embodiment, the function of the mobile communication terminal as a remote controller is realized, such as a mobile phone remote controller. The mobile communication terminal remote controller completes the communication between a mobile communication terminal such as a mobile phone and a designated device such as a computer through the two-way communication mechanism of WebSocket, and through NPAPI The plug-in mechanism of the system calls the API of the system to execute user commands, thus realizing the full control of the specified device when the user is completely separated from the traditional input device.

Embodiment three

Referring to FIG. 3 , it shows a flow chart of steps of a method for controlling device operation according to Embodiment 3 of the present invention.

This embodiment takes the mobile communication terminal as a mobile phone and the designated device as a personal computer as an example. The user performs a touch and move operation on the touch panel page of the mobile communication terminal to control the cursor operation of the designated device as an example. The method for controlling device operation of the present invention For illustration.

The method for controlling the operation of equipment in this embodiment includes the following steps:

Step S302: Build a network control server.

In this embodiment, Node.js is used to build a network control server. As an emerging background language, Node.js has many attractive features: RESTful API, single-threaded, non-blocking IO, GoogleV8, event-driven, etc.

The construction of the network control server includes:

Step 1: Install Cygwin; then, download and install Node.js.

Step 2: Implement WebSocket communication with socket.io.js.

socket.io.js is a js library file that uses the WebSocket specification programming in the HTML5 draft to realize the two-way communication between the browser and the server. It provides such as sending messages (socket.emit()), receiving information (socket.on( )), broadcast information (socket.broadcast.emit()) and other APIs.

The third step: realize the authorization and authentication of the mobile communication terminal.

Implementation method: the specified device sends the controlled password to the network control server, and the network control server backs up the controlled password. When the control terminal, that is, the mobile communication terminal, wants to control a specified device, it needs to enter the controlled password and send it to the network control server Validate, and if the password is entered correctly, take control.

Step 4: The network control server obtains the command of the designated device, the control terminal, that is, the mobile communication terminal, and realizes the transfer.

The acquisition and transfer of this step are all completed through WebSocket communication.

Step S304: Build an HTML page on the mobile communication terminal for the user to control and operate the designated device, and send the control operation instruction generated according to the control operation.

In this embodiment, the constructed HTML page includes a touchpad page, a shortcut operation page and a terminal selection page.

The interface of the touchpad page is shown in Figure 4. The touchpad page is used to receive the information of the user's touch operation on the touchpad page, and generate control operation instructions according to the information of the touch operation to control the cursor operation of the device. Control the movement of the specified device cursor by dragging the touch point on the touchpad page (the white point in Figure 4) and calculating the dragging distance (the unit can be pixels). The method of distance conversion is equidistance conversion, that is, the cursor on the computer will move as far as the touch point is dragged on the touch panel page of a mobile communication terminal such as a mobile phone. When implemented, the control terminal, that is, the mobile communication terminal, obtains the movement information or click event of the touch point through the DOM event (including touchstart, touchmove, touchend, etc.) of the HTML page, and then uses the socket.emit() method to send the corresponding data to the network control Server, network control server listens to the data sent by the control terminal, that is, the mobile communication terminal through socket.on() and sends the data to the specified device with the socket.broadcast.emit() method, and the browser extension program of the specified device passes socket.on( ) to monitor the data sent by the network control server, and call related system APIs (such as: mouse_event, keybd_event, OpenProcessToken, LookupPrivilegeValue, AdjustTokenPrivileges, ExitWindowsEx) through the NPAPI plug-in (a DLL of the browser extension) to perform related operations. In addition, the distance conversion method may also be other appropriate methods, such as proportional conversion.

The shortcut operation page, as shown in Figure 5, is used to receive user operation information on the controls in the shortcut operation page, and generate control operation instructions according to the operation information of the controls, so as to control the device to perform the control operations indicated by the control operation instructions. In this embodiment, the controls set in the form of buttons on the shortcut operation page include: volume adjustment controls, device desktop display controls, device screen capture controls, and device shutdown controls, that is, the shortcut operations performed on the device through this page include: adjusting the volume , display the desktop, take screenshots and shut down, etc. When implemented, the control terminal, that is, the shortcut operation HTML page of the mobile communication terminal monitors the click events of each button, and then sends the corresponding information to the network control server, and the network control server sends the data to the designated device, and the designated device obtains the data according to The type of data calls the corresponding system API through the NPAPI plug-in mechanism to complete the corresponding operation.

In addition, the shortcut operation page of this embodiment is also provided with a terminal selection button control, and when the button is clicked, a terminal selection page is displayed, as shown in FIG. 6 .

The interface of the terminal selection page is shown in Figure 6, which is used to display the information of the designated device to be controlled and the controlled status of the designated device to be controlled, so that the user can select the designated device to be controlled; in addition, the selection terminal page can also include The authorization password input control is used for the user to input the authorization password for control operation authentication. The terminal selection page displays the list of specified devices obtained from the network control server and the controlled status of the specified devices. After entering the authorization password and being verified by the network control server, the user can obtain the control right of the selected specified device. When implemented, the specified device sends the information of the machine (for example, computer name, IP address, controlled password, etc.) to the network control server, and the network control server can send relevant information (information other than the controlled password) to the control terminal That is, the mobile communication terminal, the control terminal, that is, the mobile communication terminal, selects the specified device (such as according to the IP address or computer name) and enters the controlled password and sends it to the network control server for verification. After the password is correct, the network control server returns a successful message, so far the control The terminal, that is, the mobile communication terminal, obtains the control authority, and then the control terminal, that is, the mobile communication terminal, can control the designated device.

Step S306: Setting a designated device.

The specified device has a browser and browser extensions with the NPAPI mechanism, and the browser extension means that the specified device needs to rely on a browser extension (.crx, .xpi file). The background page of the extension is an HTML page, which is used to complete the communication with the network control server, and the NPAPI plug-in of the extension is a DLL, which completes the call of the system API.

When the HTML page of the control terminal, that is, the mobile communication terminal, sends the control operation command to the network control server through WebSocket, the network control server uses WebSocket to send the control operation command to the designated device, so that the designated device accepts the command of the control terminal, that is, the mobile communication terminal to operate .

After the background page of the browser extension program of the specified device obtains the control operation instructions sent by the network control server, it parses the instructions and calls NPAPI related functions, and then the NPAPI functions call system APIs (such as mouse_event, keybd_event, OpenProcessToken, LookupPrivilegeValue, AdjustTokenPrivileges , ExitWindowsEx, etc.) to complete events such as mouse click, cursor movement, shutdown, etc. This is because browser extensions cannot directly call the system API, but they can call the exported function of the NPAPI plug-in. The NPAPI plug-in is a DLL that can directly call the system API, so that the specified device can call the system API through NPAPI to execute the received function. The command.

It should be noted that, the above steps S302, S304, and S306 may be performed in any order.

After the mobile communication terminal, the network control server and the designated device have completed the above corresponding settings, the mobile communication terminal can start to control the operation of the designated device, including:

Step S308: the mobile communication terminal acquires a list of designated devices (PCs).

The specified device sends the information of the machine (such as computer name, IP address, controlled password, etc.) to the network control server, and the network control server sends the relevant information of the specified device (except for the controlled password) communication terminal.

Step S310: the mobile communication terminal selects a designated device (PC) and performs authorization authentication.

The designated device sends the controlled password to the network control server, and the control terminal, that is, the mobile communication terminal, enters the password in the authorization password input control of the HTML page, such as the input control, and sends it to the network control server for verification. If the entered password is correct, it is authorized.

Step S312: the mobile communication terminal moves the cursor through the touch area of the touch panel page, and generates a control operation instruction.

When the user drags the touch point in the touch area of the touchpad page to move the cursor, the DOM event of the touchpad page acquires the movement information of the touch point and the click event, and then generates a control operation instruction.

Similarly, when the user initiates an operation command (such as adjusting the volume, displaying the desktop, taking a screenshot, shutting down, etc.) through the button on the shortcut operation page, the DOM event on the shortcut operation page obtains the button click information, and then generates a control operation instruction.

Step S314: the mobile communication terminal communicates with the network control server through the WebSocket connection, and sends the control operation instruction to the network control server through the touch panel page.

Step S316: the network control server communicates with the designated device through the WebSocket connection, and sends the control operation instruction to the designated device.

Step S318: The designated device obtains the control operation instruction sent by the network control server through the background page of the browser extension program, and calls the relevant NPAPI function.

For example, when the control operation instruction indicates cursor operation, the device calls the NPAPI function SetCursorPos.

Step S320: the designated device calls the API function of the system through the NPAPI function, and executes the control operation indicated by the control operation instruction.

For example, the API function mouse_event corresponding to the NPAPI function in step S318 is called to realize the cursor operation of the specified device.

In this embodiment, the method for controlling the operation of the device according to the present invention is described by taking the cursor control operation of the designated device as an example. However, those skilled in the art should understand that other control operations such as volume control and shutdown operations can be performed with reference to this embodiment. Through this embodiment, mobile communication terminals such as mobile phones replace traditional computer input devices such as keyboards, touch pads, and mice, and can conveniently operate the computer and monitor the computer in real time without being far away from the computer. This embodiment utilizes new Web standards and the characteristics of mobile communication terminals such as mobile phones to be carried around, and uses mobile communication terminals such as mobile phones as input devices for computers. operating experience.

Embodiment Four

Referring to FIG. 7 , it shows a flow chart of steps of a method for controlling device operation according to Embodiment 4 of the present invention.

The method for controlling device operation in this embodiment can be applied to the control of the Internet of Things terminal device by the mobile communication terminal.

The method for controlling the operation of equipment in this embodiment includes the following steps:

Step S402: The mobile communication terminal acquires information of at least one specified device within a certain range and displays it for the user to select the specified device.

For example, a mobile communication terminal sends a corresponding request to a specified device within a certain distance through a specific connection such as Bluetooth, so as to obtain information about the specified device around. For example, a mobile communication terminal can search for multiple IoT devices (such as smart TVs, smart refrigerators, etc.) Specify the device.

Step S404: The mobile communication terminal receives the user's selection to determine a designated device, and sends a pairing request to the designated device, requesting to establish a pairing connection with the designated device.

Wherein, the pairing request includes the information of the mobile communication terminal. Preferably, the information of the mobile communication terminal includes at least one of the following: a two-dimensional code of the mobile communication terminal, an identification of the mobile communication terminal, and an IP address of the mobile communication terminal.

Step S406: The mobile communication terminal receives a pairing response carrying the information of the designated device returned by the designated device, and establishes a pairing connection with the designated device according to the pairing response.

Preferably, the information of the specified device includes at least one of the following: a QR code of the specified device, an identifier of the specified device, and an IP address of the specified device.

Step S408: the mobile communication terminal obtains the HTML page corresponding to the designated device.

For example, after the mobile communication terminal establishes a pairing connection with the designated device, it sends an information acquisition command to the designated device through the pairing connection to request to obtain the control operations supported by the designated device; furthermore, according to the information of the supported control operations returned by the designated device Generate HTML pages.

For another example, HTML pages that match one or more types of designated devices are stored in the mobile communication terminal; after obtaining the information of the designated device, the mobile communication terminal determines the type of the designated device according to the information, and then selects the matching HTML page. page.

For another example, when some servers can also participate, the mobile communication terminal sends the obtained specified device information to the server; the server determines the type of the specified device, and then generates an HTML page to send to the mobile communication terminal, or, the server According to the type of the designated device, the operations supported by the designated device of this type are returned to the mobile communication terminal, and then the mobile communication terminal generates a corresponding HTML page.

Taking a smart TV as an example, the mobile communication terminal generates a corresponding HTML page according to the information returned by the smart TV. Realize remote control of smart TV.

Step S410: the mobile communication terminal receives a control operation instruction from the user to perform a control operation on a designated device through an HTML page.

Step S412: The mobile communication terminal sends the control operation instruction to the designated device that has been paired successfully through the HTML page and using the pairing connection.

Step S414: The designated device receives the control operation instruction sent by the mobile communication terminal through the paired connection, and executes the control operation indicated by the control operation instruction.

Through this embodiment, point-to-point device control is realized. Only one mobile communication terminal is needed to realize the control of multiple designated devices without special remote control or control software, realizing the compatible control of multiple devices when they are far away from the device, and improving the effect of user operation experience.

Embodiment five

Referring to FIG. 8 , it shows a structural block diagram of a mobile communication terminal according to Embodiment 5 of the present invention.

The device for device control is provided in the mobile communication terminal of this embodiment, including: a receiving module 502, configured to receive a control operation instruction for the user to control and operate a specified device through a set HTML page, wherein the HTML page is used to provide The interface where the user controls and operates the designated device through the mobile communication terminal receives the control operation instruction generated by the user's operation on the interface, and sends the control operation instruction; the control module 504 is used to send the control operation instruction to the designated device through the HTML page, Make the specified device execute the control operation indicated by the control operation instruction.

Preferably, the control module 504 is configured to use a WebSocket connection to send the control operation instruction to the network control server through the HTML page, and the network control server uses the WebSocket connection to forward the control operation instruction to the designated device, so that the designated device can, according to the control operation instruction, Call the system API of the specified device through NPAPI to realize the control operation indicated by the control operation instruction.

Preferably, the specified device is installed with a browser with NPAPI and a browser extension program, wherein the browser extension program includes an NPAPI plug-in, and the browser extension program is used to realize the WebSocket communication between the specified device and the network control server, and through the NPAPI plug-in call Specifies the system API for the device.

Preferably, when the control module 504 makes the designated device call the system API of the designated device through NPAPI to realize the control operation indicated by the control operation instruction according to the control operation instruction, the designated device monitors and receives the network control through the HTML page of the browser extension program. The control operation instruction sent by the server; the HTML page of the browser extension program calls the NPAPI function corresponding to the control operation indicated by the control operation instruction, and calls the corresponding system API function through the NPAPI function to realize the control operation indicated by the control operation instruction .

Preferably, this embodiment further includes: a selection module 506, configured to acquire at least one designated device to be controlled from the network control server before the receiving module 504 receives a control operation instruction for the user to perform a control operation on the designated device through a set HTML page. Device information; select a designated device to be controlled, and send authentication information to the network control server, and the authentication information is used to request control operation authorization for the selected designated device; receive confirmation authorization information returned by the network control server.

Preferably, the selection module 506 is further configured to use the WebSocket connection to send an information acquisition command to the network control server after receiving the confirmation authorization information returned by the network control server, wherein the information acquisition command is used to acquire the control operations supported by the specified device; the network The control server uses the WebSocket connection to send the information acquisition command to the specified device, so that the specified device obtains the control operation supported by the specified device through the NPAPI function according to the information acquisition command and returns it to the network control server, which is forwarded to the selection module by the network control server; the selection module According to the information on the control operations supported by the specified device returned by the network control server, the controls corresponding to the control operations supported by the specified device are displayed on the HTML page.

Preferably, the selection module 506 acquires information of at least one designated device to be controlled from the network control server when obtaining information of at least one designated device to be controlled from the network control server, wherein the information of the designated device includes at least one of the following: The QR code of the designated device, the identification of the designated device, and the IP address of the designated device; the mobile communication terminal identifies and displays the designated device through the information of the designated device; or, the network control server identifies the designated device according to the information of the designated device to be controlled; The mobile communication terminal acquires the information of the designated device identified by the network control server from the network control server.

Preferably, the network control server is built using Node.js.

Preferably, the control module 504 is configured to send the control operation instruction to the specified device successfully paired with the mobile communication terminal through the HTML page, so that the specified device performs the control operation indicated by the control operation instruction.

Preferably, this embodiment further includes: a pairing module 508, configured to send a pairing request to the designated device before the receiving module 502 receives a control operation instruction for the user to perform a control operation on the designated device through a set HTML page, requesting to communicate with the designated device Establishing a pairing connection, wherein the pairing request includes the information of the mobile communication terminal; receiving a pairing response carrying the information of the designated device returned by the designated device, and establishing a pairing connection with the designated device according to the pairing response.

Preferably, the information of the mobile communication terminal includes at least one of the following: the two-dimensional code of the mobile communication terminal, the identification of the mobile communication terminal, and the IP address of the mobile communication terminal; the information of the designated device includes at least one of the following: the two-dimensional code of the designated device code, the identifier of the specified device, and the IP address of the specified device.

Preferably, this embodiment further includes: a construction module 510, configured to construct a set HTML page before the receiving module 502 receives a control operation instruction for the user to perform a control operation on a specified device through the set HTML page, and the HTML page includes the following At least one of: a touchpad page, a shortcut operation page, and a selection terminal page, wherein the touchpad page is used to receive information about the user's touch operation on the touchpad page, and generate a control operation instruction according to the information of the touch operation to control the specified device Cursor operation; the shortcut operation page includes at least one of the following controls: volume adjustment control, device desktop display control, device screenshot control, and device shutdown control. The shortcut operation page is used to receive information about the user's operation on the controls in the shortcut operation page. The control operation information generates control operation instructions to control the specified device to perform the control operation indicated by the control operation instruction; the selection terminal page is used to display the information of the device to be controlled and the controlled status of the device to be controlled for the user to choose For the equipment to be controlled, the selection terminal page also includes an authorization password input control for the user to enter the authorization password for control operation authentication.

Preferably, the receiving module 502 is configured to listen to the user's control operation on the designated device through the HTML page, obtain the information of the control operation through the DOM event of the HTML page, and generate the control operation instruction according to the information of the control operation.

The mobile communication terminal in this embodiment is used to implement the corresponding method for controlling the operation of the device in the foregoing method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

The schemes for controlling device operation provided herein are not inherently related to any particular computer, virtual system or other device. Various generic systems can also be used with the teachings based on this. The structure required to construct a system having the solution of the present invention is apparent from the above description. Furthermore, the present invention is not specific to any particular programming language. It should be understood that various programming languages can be used to implement the content of the present invention described herein, and the above description of specific languages is for disclosing the best mode of the present invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, in order to streamline this disclosure and to facilitate an understanding of one or more of the various inventive aspects, various features of the invention are sometimes grouped together in a single embodiment, figure, or its description. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art can understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. Modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore may be divided into a plurality of sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings), as well as any method or method so disclosed, may be used in any combination, except that at least some of such features and/or processes or units are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will understand that although some embodiments described herein include some features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention. and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components in the solution for controlling device operation according to the embodiment of the present invention. The present invention can also be implemented as an apparatus or an apparatus program (for example, a computer program and a computer program product) for performing a part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet site, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

This paper discloses A1. A method for controlling the operation of a device, which is characterized in that it includes: the mobile communication terminal receives a control operation instruction for the user to control and operate the specified device through a set HTML page, wherein the HTML page is used to provide The user controls and operates the specified device through the interface of the mobile communication terminal, receives the control operation instruction generated by the user's operation on the interface, and sends the control operation instruction; the mobile communication terminal through The HTML page sends the control operation instruction to the designated device, so that the designated device performs the control operation indicated by the control operation instruction; wherein, the mobile communication terminal receives the user through the HTML page set The step of specifying the control operation instruction for the device to perform the control operation includes: the set HTML page monitors the control operation performed by the user on the specified device through the HTML page, and obtains the The information of the control operation is used to generate the control operation instruction according to the information of the control operation. A2. The method according to A1, wherein the mobile communication terminal sends the control operation instruction to the designated device through the HTML page, so that the designated device executes the control operation instruction indicated The step of controlling the operation includes: the mobile communication terminal uses a WebSocket connection to send the control operation instruction to a network control server through the HTML page, and the network control server uses the WebSocket connection to forward the control operation instruction To the specified device, make the specified device call the system API of the specified device through NPAPI according to the control operation instruction to implement the control operation indicated by the control operation instruction. A3, according to the method described in A2, it is characterized in that, the specified device is installed with a browser and a browser extension program with NPAPI, wherein the browser extension program includes an NPAPI plug-in, and the browser extension program is used for Realize the WebSocket communication between the specified device and the network control server, and call the system API of the specified device through the NPAPI plug-in. A4. The method according to A3, wherein the specified device calls the system API of the specified device through NPAPI according to the control operation instruction to implement the control operation indicated by the control operation instruction, including: The designated device monitors and receives the control operation instruction sent by the network control server through the HTML page of the browser extension program; the HTML page call of the browser extension program corresponds to the control operation indicated by the control operation instruction corresponding NPAPI function, and call the corresponding system API function through the NPAPI function to realize the control operation indicated by the control operation instruction. A5. The method according to A1, characterized in that, before the mobile communication terminal receives the user’s step of controlling and operating the specified device through the set HTML page, it also includes: the mobile communication terminal from The network control server acquires information of at least one specified device to be controlled; selects one of the specified devices to be controlled, and sends authentication information to the network control server, and the authentication information is used to request the selected specified device The device authorizes the control operation; and receives the confirmation authorization information returned by the network control server. A6. The method according to A5, characterized in that, after receiving the confirmation authorization information returned by the network control server, further comprising: the mobile communication terminal sends an information acquisition command to the network control server using a WebSocket connection , wherein, the information acquisition command is used to acquire the control operation supported by the specified device; the network control server uses the WebSocket connection to send the information acquisition command to the specified device, so that the specified device according to the specified The above information acquisition command, obtains the control operation supported by the specified device through the NPAPI function and returns it to the network control server, and is forwarded to the mobile communication terminal by the network control server; The information on the control operations supported by the specified device returned by the server displays controls corresponding to the control operations supported by the specified device in the HTML page. A7. The method according to A5 or A6, wherein the step of obtaining the information of at least one designated device to be controlled by the mobile communication terminal from the network control server includes: The server acquires information about at least one designated device to be controlled, wherein the information about the designated device includes at least one of the following: a QR code of the designated device, an identifier of the designated device, and an IP of the designated device address; the mobile communication terminal identifies and displays the designated device through the information of the designated device; or, the network control server identifies the designated device according to the information of the designated device to be controlled; the mobile communication The terminal acquires the information of the designated device identified by the network control server from the network control server. A8. The method according to A2, wherein the network control server is built using Node.js. A9. The method according to A1, wherein the mobile communication terminal sends the control operation instruction to the designated device through the HTML page, so that the designated device executes the control operation instruction indicated The step of controlling the operation includes: the mobile communication terminal sends the control operation instruction to the designated device successfully paired with the mobile communication terminal through the HTML page, so that the designated device executes the control operation instruction The indicated control operation. A10. The method according to A9, characterized in that before the mobile communication terminal receives the user’s step of controlling and operating the designated device through the set HTML page, it also includes: the mobile communication terminal sending The specified device sends a pairing request, requesting to establish a pairing connection with the specified device, wherein the pairing request includes the information of the mobile communication terminal; the mobile communication terminal receives the information returned by the specified device, carrying A pairing response of the information of the specified device, establishing a pairing connection with the specified device according to the pairing response. A11. The method according to A10, wherein the information of the mobile communication terminal includes at least one of the following: a two-dimensional code of the mobile communication terminal, an identification of the mobile communication terminal, an ID of the mobile communication terminal IP address; the information of the specified device includes at least one of the following: a two-dimensional code of the specified device, an identifier of the specified device, and an IP address of the specified device. A12. The method according to A1, characterized in that before the mobile communication terminal receives the user’s step of controlling and operating the specified device through the set HTML page, it also includes: constructing the set An HTML page, where the HTML page includes at least one of the following: a touchpad page, a shortcut operation page, and a selection terminal page, wherein the touchpad page is used to receive information about the touch operation of the user on the touchpad page, Generate the control operation instruction according to the information of the touch operation to control the cursor operation of the specified device; the shortcut operation page includes at least one of the following controls: volume adjustment control, device desktop display control, device screenshot control and device Shut down the control, the shortcut operation page is used to receive the user’s operation information on the controls in the shortcut operation page, generate the control operation instruction according to the operation information of the control, so as to control the designated device to perform the control operation indicated by the control operation instruction; the selection terminal page is used to display the information of the device to be controlled and the controlled state of the device to be controlled, so that the user can select the device to be controlled, The terminal selection page also includes an authorization password input control for the user to input an authorization password for control operation authentication.

This paper discloses B13, a mobile communication terminal, which is characterized in that it includes: a receiving module, used to receive a control operation instruction for a user to control and operate a designated device through a set HTML page, wherein the HTML page is used to provide The user controls and operates the specified device through the interface of the mobile communication terminal, receives the control operation instruction generated by the user's operation on the interface, and sends the control operation instruction; the control module is used to pass The HTML page sends the control operation instruction to the designated device, so that the designated device performs the control operation indicated by the control operation instruction; wherein, the receiving module is configured to pass the set HTML The page monitors the user's control operation on the specified device through the HTML page, obtains the control operation information through the DOM event of the HTML page, and generates the control operation instruction according to the control operation information. B14, according to the described mobile communication terminal of B13, it is characterized in that, described control module is used for using WebSocket connection, sends described control operation instruction to network control server through described HTML page, is used by described network control server The WebSocket connection forwards the control operation instruction to the specified device, so that the specified device calls the system API of the specified device through NPAPI according to the control operation instruction to realize the control operation indicated by the control operation instruction . B15, according to 14 described mobile communication terminals, it is characterized in that, described designated device is installed with browser and browser extension program with NPAPI, and wherein, described browser extension program comprises NPAPI plug-in, and described browser extension program It is used to realize the WebSocket communication between the designated device and the network control server, and call the system API of the designated device through the NPAPI plug-in. B16, according to the mobile communication terminal described in B15, it is characterized in that, when the control module makes the specified device call the system API of the specified device through NPAPI according to the control operation instruction to realize the instruction of the control operation instruction During the control operation, the designated device is made to listen to and receive the control operation instruction sent by the network control server through the HTML page of the browser extension program; the HTML page call of the browser extension program is connected with the control The NPAPI function corresponding to the control operation indicated by the operation instruction is used to call the corresponding system API function through the NPAPI function to realize the control operation indicated by the control operation instruction. B17. According to the mobile communication terminal described in B13, it is characterized in that it also includes: a selection module, which is used to receive the control operation instruction from the network before the receiving module receives the control operation instruction that the user controls the designated device through the HTML page set. The control server acquires information on at least one specified device to be controlled; selects one specified device to be controlled, and sends authentication information to the network control server, and the authentication information is used to request the selected specified device Perform control operation authorization; receive confirmation authorization information returned by the network control server. B18. The mobile communication terminal according to B17, wherein the selection module is further configured to use a WebSocket connection to send information to the network control server after receiving the confirmation authorization information returned by the network control server An acquisition command, wherein the information acquisition command is used to acquire the control operation supported by the designated device; the network control server uses the WebSocket connection to send the information acquisition command to the designated device, so that the designated device According to the information acquisition command, the control operation supported by the specified device is obtained through the NPAPI function and returned to the network control server, and the network control server forwards it to the selection module; the selection module according to the network control The information on the control operations supported by the specified device returned by the server displays controls corresponding to the control operations supported by the specified device in the HTML page. B19. The mobile communication terminal according to B17 or B18, wherein the selection module obtains at least one device to be controlled from the network control server when the selection module obtains information of at least one specified device to be controlled from the network control server. The information of the designated device controlled, wherein the information of the designated device includes at least one of the following: the QR code of the designated device, the identification of the designated device, the IP address of the designated device; The communication terminal identifies and displays the designated device through the information of the designated device; or, the network control server identifies the designated device according to the information of the designated device to be controlled; The control server acquires the information of the designated device identified by the network control server. B20, according to the mobile communication terminal described in B14, it is characterized in that, described network control server uses Node.js to build. B21. The mobile communication terminal according to B13, wherein the control module is configured to send the control operation instruction to the specified device successfully paired with the mobile communication terminal through the HTML page, making the designated device execute the control operation indicated by the control operation instruction. B22. According to the mobile communication terminal described in B21, it is characterized in that it also includes: a pairing module, which is used to send a pairing module to the specified device before the receiving module receives the control operation instruction for the user to control the designated device through the set HTML page. The specified device sends a pairing request, requesting to establish a pairing connection with the specified device, wherein the pairing request includes the information of the mobile communication terminal; receiving the information returned by the specified device and carrying the specified device A pairing response, establishing a pairing connection with the specified device according to the pairing response. B23. The mobile communication terminal according to B22, wherein the information of the mobile communication terminal includes at least one of the following: the two-dimensional code of the mobile communication terminal, the identification of the mobile communication terminal, the mobile communication terminal The IP address of the terminal; the information of the specified device includes at least one of the following: a two-dimensional code of the specified device, an identifier of the specified device, and an IP address of the specified device. B24. According to the mobile communication terminal described in B13, it is characterized in that it also includes: a construction module, which is used to construct the specified device before the receiving module receives the control operation instruction that the user performs a control operation on the designated device through the HTML page set. The set HTML page, the HTML page includes at least one of the following: a touchpad page, a shortcut operation page, and a selection terminal page, wherein the touchpad page is used to receive the touch of the user on the touchpad page Operation information, generating the control operation instruction according to the touch operation information to control the cursor operation of the specified device; the shortcut operation page includes at least one of the following controls: volume adjustment control, device desktop display control, device screen capture control and device shutdown control, the shortcut operation page is used to receive information about the operation of the controls on the shortcut operation page by the user, and generate the control operation instructions according to the operation information of the controls to control all The designated device performs the control operation indicated by the control operation instruction; the selection terminal page is used to display the information of the device to be controlled and the controlled status of the device to be controlled, so that the user can select the device to be controlled. Controlled equipment, the selection terminal page further includes an authorization password input control for the user to input an authorization password for control operation authentication.

Claims (20)

1. A method of controlling operation of a device, comprising:

constructing a set HTML page, the set HTML page including at least one of: the method comprises the following steps of (1) touching a touch panel page, a shortcut operation page and a selection terminal page;

the method comprises the steps that a mobile communication terminal receives a control operation instruction of a user for controlling and operating a specified device through a set HTML page, wherein the HTML page is used for providing an interface for the user to control and operate the specified device through the mobile communication terminal, receiving the control operation instruction of the user for controlling and operating the specified device, which is generated by the user for operating the interface, and sending the control operation instruction;

the mobile communication terminal sends the control operation instruction to the specified equipment through the HTML page, so that the specified equipment executes the control operation indicated by the control operation instruction;

the step that the mobile communication terminal receives a control operation instruction of a user for performing control operation on the specified equipment through a set HTML page comprises the following steps:

and the set HTML page monitors the control operation of the user on the designated equipment through the HTML page, acquires the information of the control operation through the DOM event of the HTML page, and generates the control operation instruction according to the information of the control operation.

2. The method according to claim 1, wherein the mobile communication terminal sends the control operation instruction to the specified device through the HTML page, and the step of causing the specified device to execute the control operation indicated by the control operation instruction includes:

the mobile communication terminal uses WebSocket connection to send the control operation instruction to a network control server through the HTML page, and the network control server uses the WebSocket connection to forward the control operation instruction to the specified equipment, so that the specified equipment calls a system API of the specified equipment through an NPAPI to realize the control operation indicated by the control operation instruction according to the control operation instruction.

3. The method of claim 2, wherein the specified device is installed with a browser having an NPAPI and a browser extension program, wherein the browser extension program comprises an NPAPI plug-in, and wherein the browser extension program is configured to enable WebSocket communication between the specified device and the network control server, and to call a system API of the specified device through the NPAPI plug-in.

4. The method according to claim 3, wherein the step of the designated device calling a system API of the designated device through an NPAPI to implement the control operation indicated by the control operation instruction according to the control operation instruction comprises:

the designated equipment monitors and receives the control operation instruction sent by the network control server through an HTML page of the browser extension program;

and the HTML page of the browser extension program calls an NPAPI function corresponding to the control operation indicated by the control operation instruction, and calls a corresponding system API function through the NPAPI function to realize the control operation indicated by the control operation instruction.

5. The method according to claim 1, before the step of the mobile communication terminal receiving a control operation instruction for a user to perform a control operation on a specified device through a set HTML page, further comprising:

the mobile communication terminal acquires information of at least one designated device to be controlled from a network control server;

selecting one appointed device to be controlled, and sending authentication information to the network control server, wherein the authentication information is used for requesting control operation authorization on the selected appointed device;

and receiving the confirmation authorization information returned by the network control server.

6. The method according to claim 5, further comprising, after said receiving the confirmation authorization information returned by the network control server:

the mobile communication terminal sends an information acquisition command to the network control server by using WebSocket connection, wherein the information acquisition command is used for acquiring the control operation supported by the specified equipment;

the network control server sends the information acquisition command to the specified equipment by using the WebSocket connection, so that the specified equipment acquires the control operation supported by the specified equipment through an NPAPI function according to the information acquisition command, returns the control operation to the network control server and forwards the control operation to the mobile communication terminal by the network control server;

and the mobile communication terminal displays a control corresponding to the control operation supported by the specified equipment in an HTML page according to the information of the control operation supported by the specified equipment, which is returned by the network control server.

7. The method according to claim 5 or 6, wherein the step of the mobile communication terminal acquiring information of at least one of the designated devices to be controlled from a network control server comprises:

the mobile communication terminal acquires information of at least one specified device to be controlled from the network control server, wherein the information of the specified device comprises at least one of the following: the two-dimensional code of the designated equipment, the identifier of the designated equipment and the IP address of the designated equipment; the mobile communication terminal identifies and displays the specified equipment through the information of the specified equipment;

or,

the network control server identifies the specified equipment according to the information of the specified equipment to be controlled; the mobile communication terminal acquires information of the specified device identified by the network control server from the network control server.

8. The method of claim 2, wherein the network control server is built using node.

9. The method according to claim 1, wherein the mobile communication terminal sends the control operation instruction to the specified device through the HTML page, and the step of causing the specified device to execute the control operation indicated by the control operation instruction includes:

and the mobile communication terminal sends the control operation instruction to the specified equipment successfully matched with the mobile communication terminal through the HTML page, so that the specified equipment executes the control operation indicated by the control operation instruction.

10. The method according to claim 9, before the step of the mobile communication terminal receiving a control operation instruction for a user to perform a control operation on a specified device through a set HTML page, further comprising:

the mobile communication terminal sends a pairing request to the designated equipment to request to establish pairing connection with the designated equipment, wherein the pairing request comprises information of the mobile communication terminal;

and the mobile communication terminal receives a pairing response which is returned by the appointed equipment and carries the information of the appointed equipment, and establishes pairing connection with the appointed equipment according to the pairing response.

11. The method according to claim 10, wherein the information of the mobile communication terminal comprises at least one of: the two-dimensional code of the mobile communication terminal, the identification of the mobile communication terminal and the IP address of the mobile communication terminal; the information specifying the device includes at least one of: the two-dimensional code of the designated equipment, the identification of the designated equipment and the IP address of the designated equipment.

12. The method of claim 1,

the touch pad page is used for receiving information of touch operation of the user on the touch pad page, and generating the control operation instruction according to the information of the touch operation so as to control cursor operation of the specified equipment; the shortcut operation page includes at least one of the following controls: the shortcut operation page is used for receiving information of operation of a control in the shortcut operation page by the user, and generating the control operation instruction according to the information of the operation of the control so as to control the specified equipment to perform the control operation indicated by the control operation instruction; the selection terminal page is used for displaying information of the equipment to be controlled and the controlled state of the equipment to be controlled so that the user can select the equipment to be controlled, and the selection terminal page further comprises an authorized password input control part so that the user can input an authorized password to perform control operation authentication.

13. A mobile communication terminal, comprising:

the building module is used for building a set HTML page, and the set HTML page comprises at least one of the following: the method comprises the following steps of (1) touching a touch panel page, a shortcut operation page and a selection terminal page;

the receiving module is used for receiving a control operation instruction of a user for performing control operation on the specified equipment through a set HTML page, wherein the HTML page is used for providing an interface for the user to perform control operation on the specified equipment through the mobile communication terminal, receiving the control operation instruction for performing control operation on the specified equipment, which is generated by the user through the interface operation, and sending the control operation instruction;

the control module is used for sending the control operation instruction to the specified equipment through the HTML page so as to enable the specified equipment to execute the control operation indicated by the control operation instruction;

the receiving module is configured to monitor, through the set HTML page, a control operation performed by the user on the designated device through the HTML page, acquire information of the control operation through a DOM event of the HTML page, and generate the control operation instruction according to the information of the control operation.

14. The mobile communication terminal of claim 13, wherein the control module is configured to send the control operation instruction to a network control server through the HTML page using a WebSocket connection, and the network control server forwards the control operation instruction to the specific device using the WebSocket connection, so that the specific device calls a system API of the specific device through an NPAPI to implement the control operation indicated by the control operation instruction according to the control operation instruction.

15. The mobile communication terminal of claim 14, wherein the specified device is installed with a browser having an NPAPI and a browser extension program, wherein the browser extension program comprises an NPAPI plug-in, and the browser extension program is configured to implement WebSocket communication between the specified device and the network control server, and to call a system API of the specified device through the NPAPI plug-in.

16. The mobile communication terminal of claim 15, wherein the control module, when causing the specified device to call a system API of the specified device through an NPAPI according to the control operation instruction to implement the control operation indicated by the control operation instruction, causes the specified device to monitor and receive the control operation instruction sent by the network control server through an HTML page of the browser extension program; and the HTML page of the browser extension program calls an NPAPI function corresponding to the control operation indicated by the control operation instruction, and calls a corresponding system API function through the NPAPI function to realize the control operation indicated by the control operation instruction.

17. The mobile communication terminal according to claim 13, further comprising:

the selection module is used for acquiring information of at least one designated device to be controlled from a network control server before the receiving module receives a control operation instruction for controlling and operating the designated device by a user through a set HTML page; selecting one appointed device to be controlled, and sending authentication information to the network control server, wherein the authentication information is used for requesting control operation authorization on the selected appointed device; and receiving the confirmation authorization information returned by the network control server.

18. The mobile communication terminal of claim 17, wherein the selecting module is further configured to send an information obtaining command to the network control server by using a WebSocket connection after the receiving of the authorization confirmation information returned by the network control server, where the information obtaining command is used to obtain a control operation supported by the specified device; the network control server sends the information acquisition command to the specified equipment by using the WebSocket connection, so that the specified equipment acquires the control operation supported by the specified equipment through an NPAPI function according to the information acquisition command, returns the control operation to the network control server and forwards the control operation to the selection module by the network control server; and the selection module displays a control corresponding to the control operation supported by the specified equipment in an HTML page according to the information of the control operation supported by the specified equipment, which is returned by the network control server.

19. The mobile communication terminal according to claim 17 or 18, wherein the selection module acquires information of the at least one designated device to be controlled from a network control server when acquiring the information of the at least one designated device to be controlled from the network control server, wherein the information of the designated device comprises at least one of: the two-dimensional code of the designated equipment, the identifier of the designated equipment and the IP address of the designated equipment; the mobile communication terminal identifies and displays the specified equipment through the information of the specified equipment; or, the network control server identifies the specified device according to the information of the specified device to be controlled; the mobile communication terminal acquires information of the specified device identified by the network control server from the network control server.

20. The mobile communication terminal of claim 14, wherein the network control server is built using node.