CN115964011B

CN115964011B - Method and related device for displaying application interface based on multi-screen cooperation

Info

Publication number: CN115964011B
Application number: CN202310250876.0A
Authority: CN
Inventors: 金国春; 石彬
Original assignee: Shenzhen Xfanic Technology Co Ltd
Current assignee: Shenzhen Xfanic Technology Co Ltd
Priority date: 2023-03-16
Filing date: 2023-03-16
Publication date: 2023-06-06
Anticipated expiration: 2043-03-16
Also published as: CN115964011A

Abstract

The application provides a method and a related device for displaying an application interface based on multi-screen collaboration, wherein the method comprises the following steps: according to a first screen-throwing request of first terminal equipment, first data of the first terminal equipment are sent to second terminal equipment, the second terminal equipment displays an interface of the first terminal equipment according to the first data, a first operation event generated according to operation on a first functional module of the adapter is obtained, a first control instruction corresponding to the first operation event and a first application identifier corresponding to the first functional module are sent to the first terminal equipment, the first terminal equipment generates second data and sends the second data to the second terminal equipment, and the second terminal equipment displays the interface generated by the first control instruction executed by the first terminal equipment. In the application, the adapter for realizing the screen throwing between the first terminal equipment and the second terminal equipment comprises the functional module, the rapid control of the first terminal equipment is realized through the functional module, and the operation efficiency is improved while the stable screen throwing is realized.

Description

Method and related device for displaying application interface based on multi-screen cooperation

Technical Field

The application belongs to the technical field of communication, and particularly relates to a method and a related device for displaying an application interface based on multi-screen cooperation.

Background

At present, interaction between terminal devices is more and more frequent, and scenes used in association between terminal devices are more and more, for example, multiple terminals are mutually matched to realize multi-screen collaborative scenes of cross devices. In the existing multi-screen collaborative use scene, the first terminal equipment always projects the content displayed in the display screen of the first terminal equipment to the second terminal equipment through a network, however, the quality of the projected screen is often affected by the network, so that the quality of the projected screen is unstable and the safety is low.

Disclosure of Invention

The application interface display method and the application interface display device based on multi-screen cooperation are characterized in that a plurality of terminal devices are connected through an adapter provided with a plurality of functional modules, so that multi-screen cooperation among the plurality of terminal devices is realized, stability and safety of a screen throwing process are improved, and the application in the connected terminal devices is controlled rapidly through the functional modules, so that the intelligence of the adapter is improved.

In a first aspect, the present application provides a method for displaying an application interface based on multi-screen collaboration, applied to an adapter of a multi-screen collaboration system, the multi-screen collaboration system including at least one first terminal device and at least one second terminal device, each of the at least one first terminal device being connected to a first interface collector of the adapter, each of the at least one second terminal device being connected to a second interface collector of the adapter, the adapter being configured to transmit on-screen display data of the first terminal device to the second terminal device in real time, the adapter including at least one functional module, the at least one functional module characterizing a module for controlling applications within the first terminal device, the method comprising:

Responding to a first screen projection request of the first terminal equipment, acquiring first data of the first terminal equipment, wherein the first data represents interface display data of the first terminal equipment at a current time point;

the first data are sent to the second terminal equipment, so that the second terminal equipment displays an interface of the first terminal equipment at the current time point;

if the operation of the user on a first functional module in the at least one functional module is detected, a first operation event is generated according to the operation of the user on the first functional module;

acquiring a first control instruction set corresponding to the first functional module, and searching a first control instruction corresponding to the first operation event from the first control instruction set, wherein the first control instruction set comprises corresponding relations between different first operation events and different control instructions, and the control instructions represent instructions for controlling an application of the first terminal equipment to execute preset operations;

searching a first application identifier corresponding to the first functional module from a preset application and module corresponding relation table, wherein the application and module corresponding relation table comprises corresponding relations between different functional modules and different application identifiers;

The first application identifier and the first control instruction are sent to the first terminal equipment, so that the first terminal equipment executes the first control instruction for a first application corresponding to the first application identifier;

receiving second data of the first terminal equipment, wherein the second data represents interface display data generated by the first terminal equipment for controlling the first application according to the first control instruction;

and sending the second data to the second terminal equipment so that the second terminal equipment displays an interface generated by the first terminal equipment executing the first control instruction according to the second data.

In a second aspect, the present application provides an apparatus for displaying an application interface based on multi-screen collaboration, the apparatus including an adapter, the adapter including at least one functional module, the functional module being configured to control terminal devices connected to a first interface collector and a second interface collector of the adapter, the adapter being configured to put an interface of at least one first terminal device connected to the first interface collector on a screen of at least one second terminal device connected to the second interface collector in real time, the apparatus including:

A first obtaining unit, configured to obtain first data of a first terminal device in response to a first screen-dropping request of the first terminal device, where the first data represents interface display data of the first terminal device at a current time point, the first terminal device is a device in the at least one first terminal device, and the second terminal device is a device in the at least one second terminal device;

a first sending unit, configured to send the first data to the second terminal device, so that the second terminal device displays an interface of the first terminal device at a current time point;

the processing unit is used for generating a first operation event according to the operation of the user on a first functional module in the at least one functional module if the operation of the user on the first functional module is detected;

the second acquisition unit is used for acquiring a first control instruction set corresponding to the first functional module, searching a first control instruction corresponding to the first operation event from the first control instruction set, wherein the first control instruction set comprises corresponding relations between different first operation events and different control instructions, and the control instructions represent instructions for controlling an application of the first terminal equipment to execute preset operations;

The searching unit is used for searching a first application identifier corresponding to the first functional module from a preset application and module corresponding relation table, and the application and module corresponding relation table comprises corresponding relations between different functional modules and different application identifiers;

the execution unit is used for sending the first application identifier and the first control instruction to the first terminal equipment so that the first terminal equipment executes the first control instruction for a first application corresponding to the first application identifier;

the receiving unit is used for receiving second data of the first terminal equipment, wherein the second data represents interface display data generated by the first terminal equipment for controlling the first application according to the first control instruction;

and the second sending unit is used for sending the second data to the second terminal equipment so that the second terminal equipment displays an interface generated by the first terminal equipment executing the first control instruction according to the second data.

In a third aspect, the present application provides an adaptor, the adaptor comprising a micro control unit, a first interface collector, a second interface collector, at least one functional module, a first conversion module and a second conversion module, the first interface collector comprising at least one interface, the second interface collector comprising at least one interface, the first interface collector being connected to the first conversion module, the first conversion module being connected to the second conversion module, the second conversion module being connected to the second interface collector, the micro control unit being connected to each of the first conversion module, the second conversion module and the at least one functional module for controlling each of the first conversion module, the second conversion module and the at least one functional module, the adaptor being connected to at least one first terminal device through the first interface collector, the first conversion module being configured to convert first data received by the first interface collector to at least one first terminal device, the second conversion module being configured to convert second data received by the first conversion module into first data, the first conversion module being configured to be connected to the first terminal device, the first data collector being configured to perform the first conversion module, the first data transmission method being configured to obtain the first data collection, the method being implemented by the first conversion module.

In a fourth aspect, the present application provides an electronic device, comprising: one or more processors;

one or more memories for storing programs,

the one or more memories and the program are configured to control, by the one or more processors, the electronic device to execute instructions as steps in any of the methods of the first aspect of the embodiments of the present application.

In a fifth aspect, the present application provides a computer readable storage medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to perform part or all of the steps as described in any of the methods of the first aspect of the embodiments of the present application.

In a sixth aspect, the present application provides a computer program, wherein the computer program is operable to cause a computer to perform some or all of the steps as described in any of the methods of the first aspect of the embodiments of the present application. The computer program may be a software installation package.

The technical scheme provided by some embodiments of the present application has the beneficial effects that at least includes:

it can be seen that in the embodiment of the present application, at least one first terminal device and at least one second terminal device are connected through an adapter. Responding to a first screen throwing request of the first terminal equipment, acquiring first data of the first terminal equipment by the adapter, wherein the first data represents interface display data of the first terminal equipment at the current time point, and sending the first data to the second terminal equipment so that the second terminal equipment displays an interface of the first terminal equipment at the current time point to realize screen throwing. If the user is detected to operate on the first functional module in the at least one functional module, a first operation event is generated according to the operation of the user on the first functional module, a first control instruction corresponding to the first event is obtained from a first control instruction set, a first application identifier corresponding to the first functional module is searched from an application-module corresponding relation table, and the first application identifier and the first control instruction are sent to the first terminal device, so that the first terminal device can execute the first control instruction for the first application corresponding to the first application identifier, and control accuracy is improved. And receiving second data of the first terminal equipment and sending the second data to the second terminal equipment, so that the second terminal equipment displays an interface generated by the first terminal equipment executing the first control instruction according to the second data. In the embodiment of the application, the transmission of the screen projection data between the first terminal equipment and the second terminal equipment is realized through the adapter, and the accuracy and the stability of screen projection are improved. In addition, the adapter comprises a functional module, and the application in the first terminal equipment can be associated through the functional module, so that the efficiency of controlling the application in the first terminal equipment is improved, and the user experience is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic structural view of an adapter according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a multi-screen collaboration system provided in an embodiment of the present application;

FIG. 3 is a flowchart of a method for displaying an application interface based on multi-screen collaboration according to an embodiment of the present application;

fig. 4 is a functional unit composition block diagram of an apparatus for displaying an application interface based on multi-screen collaboration according to an embodiment of the present application.

Detailed Description

In order to make the present application solution better understood by those skilled in the art, the following description will clearly and completely describe the technical solution in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only a sub-embodiment of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

The terms first, second and the like in the description and in the claims of the present application and in the above-described figures, are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.

Referring to fig. 1, fig. 1 is a schematic structural diagram of an adapter according to an embodiment of the present application.

An adapter comprising a micro control unit (microcontroller unit, MCU), a first interface collector comprising at least one interface, a second interface collector comprising at least one interface, at least one functional module, a first conversion module and a second conversion module, said first interface collector being connected to the first conversion module, said first conversion module being connected to the second conversion module, said second conversion module being connected to the second interface collector, said micro control unit being connected to each of said first conversion module, said second conversion module and said at least one functional module for controlling each of said first conversion module, said second conversion module and said at least one functional module, the adapter is connected with at least one first terminal device through the first interface collector, the first conversion module is used for converting first data of the at least one first terminal device received by the first interface collector into second data, the second conversion module is used for converting the second data obtained by the conversion of the first conversion module into third data, and the second interface collector is used for transmitting the third data to at least one second terminal device connected with the second interface collector so as to realize screen throwing between the terminal device connected with the first interface collector and the terminal collector connected with the second interface collector. As shown in fig. 1, at least one of the functional modules may include a functional module a, functional modules b, … …, and a functional module n. It will be appreciated that the micro-control unit may be replaced with various exemplary logic blocks, units and circuits that may be implemented or performed in connection with the present disclosure, and is not limited in this regard.

In one possible example, the first interface aggregator is an aggregator including at least one of a display interface, a universal serial bus interface, and a network interface; the second interface collector is a collector comprising at least one interface of a display interface, a universal serial bus interface and a network interface.

In particular, the first interface collector may further include a universal serial bus Hub (universal serialbus Hub, USB Hub), which is a device that can expand one USB interface into a plurality of USB interfaces and can enable the interfaces to be used simultaneously. The display interface, the universal serial bus interface, and the network interface of the first interface aggregator may be aggregated by a serial bus hub. Similarly, the second interface collector may further include a universal serial bus hub, and the display interface, the universal serial bus interface, and the network interface of the second interface collector may be collected by the universal serial bus hub.

In one possible example, the adapter may also include memory, which may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an electrically Erasable EPROM (EEPROM), or a flash memory. The volatile memory may be random access memory (random access memory, RAM) which acts as an external cache. By way of example but not limitation, many forms of random access memory (random access memory, RAM) are available, such as Static RAM (SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced Synchronous Dynamic Random Access Memory (ESDRAM), synchronous Link DRAM (SLDRAM), and direct memory bus RAM (DR RAM).

In order to better understand the technical solution of the embodiments of the present application, a description is first given of a multi-screen collaboration system that may be related to the embodiments of the present application.

Referring to fig. 2, fig. 2 is a schematic diagram of a multi-screen collaboration system according to an embodiment of the present application.

The multi-screen collaboration system includes an adapter, at least one first terminal device, and at least one second terminal device. Wherein each of the at least one first terminal device is connected with a first interface collector of the adapter, each of the at least one second terminal device is connected with a second interface collector of the adapter, the adapter is used for transmitting the screen display data of the application of the first terminal device to the second terminal device in real time, the adapter comprises at least one functional module, and the at least one functional module comprises a module for controlling the application in the first terminal device. As shown in fig. 2, specifically, the at least one first terminal device may include a first terminal device a, first terminal devices b, … …, and a first terminal device m. The at least one second terminal device may comprise a second terminal device a, a second terminal device b, … ….

Each terminal device in the at least one first terminal device may be a terminal device such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a wearable device, a head-mounted device, a vehicle-mounted terminal, and the type of the terminal device is not limited herein. Similarly, each terminal device in the at least one second terminal device may be a terminal device such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a wearable device, a head-mounted device, a vehicle-mounted terminal, and the type of the terminal device is not limited herein.

Referring to fig. 3, fig. 3 is a flowchart of a method for displaying an application interface based on multi-screen collaboration according to an embodiment of the present application. A method for displaying an application interface based on multi-screen collaboration according to an embodiment of the present application will be described in detail with reference to the accompanying drawings. As shown in fig. 3, a method for displaying an application interface based on multi-screen collaboration is applied to an adapter of a multi-screen collaboration system, the multi-screen collaboration system including at least one first terminal device and at least one second terminal device, each of the at least one first terminal device being connected to a first interface collector of the adapter, each of the at least one second terminal device being connected to a second interface collector of the adapter, the adapter being configured to transmit on-screen display data of the first terminal device to the second terminal device in real time, the adapter including at least one functional module characterizing a module for controlling applications within the first terminal device, the method comprising the steps of:

step 301, responding to a first screen projection request of the first terminal equipment, and acquiring first data of the first terminal equipment.

The first data represent interface display data of the first terminal equipment at the current time point. Specifically, each of the at least one first terminal device is connected to each of the at least one second terminal device via an adapter. After receiving a first screen projection request in the first terminal equipment, the adapter acquires first data of the first terminal equipment and provides data support for subsequent screen projection.

Step 302, the first data is sent to the second terminal device, so that the second terminal device displays an interface of the first terminal device at a current time point.

After receiving the first data, the adaptor sends the first data to the second terminal equipment, so that the second terminal equipment displays an interface of the first terminal equipment at a current time point. The interface of the first terminal equipment is projected to the interface of the second terminal through the adapter, so that the projection is more stable and safer, and the projection efficiency is improved.

Step 303, if it is detected that the user performs an operation on a first function module in the at least one function module, a first operation event is generated according to the operation of the user on the first function module.

Wherein the adapter comprises at least one functional module which can be used for controlling the application in said first terminal device. For example, the function module may be a music input module, which includes piano keys for playing music, through which music can be rapidly input, and when the music input module is associated with a music class application, music scores can be rapidly input through the music input module, thereby improving the efficiency of control application and improving the use experience of a user. If the user is detected to operate on the first functional module, a first operation event is generated according to the operation, so that data support is provided for the subsequent corresponding control application.

Step 304, a first control instruction set corresponding to the first functional module is obtained, and a first control instruction corresponding to the first operation event is searched from the first control instruction set.

The first control instruction set comprises corresponding relations of different first operation events and different control instructions, and the control instructions represent instructions for controlling an application of the first terminal device to execute preset operations. Specifically, different control instruction sets are preset for different functional modules so as to improve the efficiency of searching the instruction corresponding to the operation event. After the first operation event is generated, a first control instruction set corresponding to the first functional module is obtained, and a first control instruction corresponding to the first operation event is searched from the first control instruction set, so that data support is provided for subsequent control applications.

Step 305, searching a first application identifier corresponding to the first functional module from a preset corresponding relation table of the application and the module.

The application and module corresponding relation table comprises corresponding relations between different functional modules and different application identifiers. Searching a first application identifier corresponding to the first functional module from the application-module corresponding relation table, and providing data support for the subsequent searching of the corresponding application.

Step 306, sending the first application identifier and the first control instruction to the first terminal device.

And sending the first application identifier and the first control instruction to the first terminal equipment, so that the first terminal equipment executes the first control instruction for a first application corresponding to the first application identifier. Specifically, the adapter sends the first application identifier and the first control instruction to the first terminal device, and the first terminal device determines a first application corresponding to the first functional module according to the first application identifier and executes the first control instruction to control the first application.

Step 307, receiving second data of the first terminal device.

And the second data characterizes interface display data generated by the first terminal equipment for controlling the first application according to the first control instruction. Specifically, after executing the first control instruction, the first terminal device generates interface display data, wherein the interface display data is second data, and the adapter acquires the second data to provide data support for screen projection to the second terminal device.

And 308, sending the second data to the second terminal equipment, so that the second terminal equipment displays an interface generated by the first terminal equipment executing the first control instruction according to the second data.

The adapter sends the second data to the second terminal equipment, the second terminal equipment analyzes the second data after receiving the second data, and an interface generated by executing the first control instruction by the first terminal equipment is displayed through the display screen.

In this example, the first terminal device and the second terminal device are connected through the adapter, and the screen projection data of the first terminal device is transmitted to the second terminal device, so that the stability and safety of data transmission are improved, and further the screen projection stability and safety are improved. And the application in the first terminal equipment is controlled through the functional module, so that the operation efficiency is further improved, and the use experience of a user is improved.

In a possible example, the searching for the first application identifier corresponding to the first functional module from the preset application-module correspondence table includes: when a plurality of reference application identifications corresponding to the first functional module are determined from a preset application-module corresponding relation table, acquiring a set of times of association of the application corresponding to each reference application identification in the plurality of reference application identifications with the first functional module, and determining the maximum times in the set; and setting the application identifier corresponding to the maximum number of times as the first application identifier.

In a specific example, if the first function module sets a plurality of corresponding reference application identifiers, that is, the first function module can associate a plurality of applications in the first terminal device, a plurality of corresponding reference application identifiers of the first function module are determined from a preset application-module correspondence table, the number of times that the application corresponding to each reference application identifier is associated with the first function module is obtained, and a set of the number of times that the application corresponding to the reference application identifier is associated with the first function module is generated. And determining the maximum number of times of the numerical values in the set, and setting an application identifier corresponding to the maximum number of times as a first application identifier. For example, if the first functional module is an input keyboard, the input keyboard may be used to control inputting text information, so that the input keyboard may be associated with an application such as a text editing application in the first terminal device and a chat application that needs inputting text. And if the input keyboard is associated with the text editing application and the chat application in the first terminal equipment. When the text editing application and the chat application are started in the first terminal equipment at the same time, the user starts the first functional module, and the adapter needs to judge which application in the first terminal equipment is related to the first functional module. The specific judging process is as follows: and determining a plurality of reference application identifications corresponding to the input keyboard, and if the plurality of reference application identifications comprise the application identifications of the text editing application and the application identifications of the chat application, acquiring a first number of times the text editing application is associated by the first functional module and a second number of times the chat application is associated by the first functional module. If the value of the first time is 3 and the value of the second time is 4, the application identifier of the chat application is set as the first application identifier.

In this example, the functional module may set a plurality of reference application identifiers, and enable the functional module to determine the associated application according to the historical association times, thereby improving the practicality of the functional module and the convenience of use.

In a possible example, the searching for the first application identifier corresponding to the first functional module from the preset application-module correspondence table includes: when a plurality of reference application identifiers corresponding to the first functional module are determined from a preset application-module corresponding relation table, acquiring the association times of the application corresponding to each reference application identifier in the plurality of reference application identifiers and the first functional module; sequencing the reference application identifiers in the plurality of reference application identifiers according to the sequence from large to small of the times to obtain a sequencing table; transmitting the ranking list and a preset selection prompt message to the first terminal equipment, wherein the selection prompt message characterizes and prompts the user to select a message with a reference application identifier from the ranking list; receiving a feedback message of the first terminal equipment, wherein the feedback message comprises a reference application identifier selected by the user; and setting the reference application identifier selected by the user as the first application identifier.

In a specific example, if the first functional module sets a plurality of corresponding reference application identifiers, determining a plurality of corresponding reference application identifiers of the first functional module from a preset application-module correspondence table, acquiring the number of times that the application corresponding to each reference application identifier is associated with the first functional module, and sorting the reference application identifiers in the plurality of reference application identifiers according to the number of times of each reference application identifier and the order from large to small to obtain a sorting table. And sending the ordered list and a preset selection prompt message to the first terminal equipment so as to prompt a user to select the associated application. It will be appreciated that the ordered list may also be sent to the second terminal device, through which the user may select the application of the first terminal device associated with the first functional module.

In this example, when the first functional module sets a plurality of corresponding reference application identifiers, the first functional module sends the ordered list arranged according to the number of times of each reference application identifier to the first terminal device, so as to facilitate user selection, improve autonomy of selection, and further improve use experience of the user.

In one possible example, if the operation of the user on the first functional module is detected, the application identifier corresponding to the first functional module is obtained from the application-module correspondence table. And sending a preset starting instruction to the first terminal equipment to start the application corresponding to the application identifier according to the application identifier, generating an operation event according to the operation of a user on the first functional module after the application is started, acquiring a control instruction corresponding to the operation event, and sending the control instruction and the application identifier to the first terminal equipment so as to enable the first terminal equipment to control the application corresponding to the application instruction, wherein the generated interface display data. The efficiency of starting the application is improved, and further the use experience of a user is improved.

In one possible example, after the sending the second data to the second terminal device, the method further includes: if the user is detected to perform a second operation on the first operation module of the second terminal device, a second operation event is generated according to the second operation; if the operation event which is the same as the second operation event is found from a preset association relation table, acquiring an association instruction corresponding to the same operation event, wherein the association relation table comprises at least one operation event corresponding to the association instruction, and the association instruction characterizes an association instruction for associating an operation module of the second terminal device with the first terminal device; associating an operation module of the second terminal equipment with the first terminal equipment according to the association instruction; acquiring a third operation performed by the user on a second operation module of the second terminal equipment, and generating a third operation event according to the third operation; searching a second application identifier corresponding to the second operation module from the application and module corresponding relation table; acquiring a second control instruction set corresponding to the second operation module, and searching a second control instruction corresponding to the third operation event from the second control instruction set, wherein the second control instruction set comprises the corresponding relation between different third operation events and different control instructions; the second application identifier and the second control instruction are sent to the first terminal equipment, so that the first terminal equipment executes the second control instruction to the second application corresponding to the second application identifier; acquiring third data of the first terminal equipment, wherein the third data represents interface display data generated by the first terminal equipment for controlling the second application according to the second control instruction; and sending the third data to the second terminal equipment so that the second terminal equipment displays an interface generated by the first terminal equipment executing the second control instruction according to the third data.

In a specific example, the second terminal device may include a first operation module, and if the user performs the second operation on the first operation module of the second terminal device, a second operation event is generated according to the second operation. In order to ensure the accuracy of the control, the terminal equipment corresponding to the second operation event needs to be judged. Firstly, searching whether an operation event identical to a second operation event exists in a preset association relation table, and if so, acquiring an association instruction corresponding to the identical operation event, wherein the association instruction represents an association instruction for associating an operation module of second terminal equipment with first terminal equipment. For example, if the operation module of the second terminal device is a touch screen module, and the operation event corresponding to the association instruction included in the association relation table is circling on the touch screen module. Detecting a second operation of circling the first operation module by the user, generating a second operation event according to the second operation of the user, and acquiring an associated instruction corresponding to the same operation event if the second operation event is the same as the operation event corresponding to the associated instruction. And associating the operation module of the second terminal equipment with the first terminal equipment according to the association instruction. If a third operation of the user on the second operation module is detected, a third operation event is generated according to the third operation. Searching a second application identifier corresponding to a second operation module from the application and module corresponding relation table, acquiring a second control instruction set corresponding to the second operation module, and searching a second control instruction corresponding to a third operation event from the second control instruction set. And sending the second application identifier and the second control instruction to the first terminal equipment so that the first terminal equipment executes the second control instruction aiming at the application corresponding to the second application identifier, thereby acquiring third data of the first terminal equipment. And sending the third data to the second terminal equipment so that the second terminal equipment displays an interface generated by the first terminal equipment executing the second control instruction according to the third data to realize screen projection. It is understood that the first operation module and the second operation module may be the same operation module in the second terminal device, or may be different operation modules, which is not limited herein.

In this example, when the user is detected to operate on the operation module of the second terminal device, the operation event corresponding to the operation is detected to control the terminal device, so that the user does not need to manually indicate the association relationship, and the efficiency of associating the operation module with the terminal device is improved.

In one possible example, after the sending the second data to the second terminal device, the method further includes: if the user is detected to perform fourth operation on a third operation module of the second terminal device, a third application identifier of an application started by the second terminal device in the current time point is obtained; searching a fourth application identifier corresponding to the third operation module from a preset application set, wherein the application set comprises the corresponding relation between different operation modules of the second terminal equipment and different application identifiers of the second terminal equipment; if the third application identifier is different from the fourth application identifier, associating a third operation module of the second terminal device with the first terminal device; generating a fourth operation event according to the fourth operation; searching a fifth application identifier corresponding to the third operation module from the application and module corresponding relation table; acquiring a third control instruction set corresponding to the third operation module, and searching a corresponding third control instruction from the third control instruction set according to the fourth operation event, wherein the third control instruction set comprises corresponding relations of different fourth operation events and different control instructions; the fifth application identifier and the third control instruction are sent to the first terminal equipment, so that the first terminal equipment controls a third application corresponding to the third application identifier according to the third control instruction; acquiring fourth data of the first terminal equipment, wherein the fourth data represents interface display data generated by the first terminal equipment for controlling the third application according to the third control instruction; and sending the fourth data to the second terminal equipment so that the second terminal equipment displays an interface generated by the first terminal equipment executing the third control instruction according to the fourth data.

In a specific example, the second terminal device may include a third operation module, and if the user performs the fourth operation on the third operation module, a fourth operation event is generated according to the fourth operation. In order to ensure the control accuracy, the terminal device corresponding to the fourth operation event needs to be judged. Firstly, acquiring a third application identifier of an application started by the second terminal equipment in a current time point, then acquiring a fourth application identifier corresponding to a third operation module from an application set, judging whether the third application identifier is the same as the fourth application identifier, and if so, associating the third operation module with the application corresponding to the fourth application identifier in the second terminal equipment. If the third control command set is different, a third operation module of the second terminal device is associated with the first terminal device, a fifth application identifier corresponding to the third operation module is searched from an application and module corresponding relation table, a third control command set corresponding to the third operation module is obtained, a corresponding third control command is searched from the third control command set according to a fourth operation event generated by fourth operation, and the fifth application identifier and the third control command are sent to the first terminal device, so that the first terminal device controls a fifth application corresponding to the fifth application identifier according to the third control command, and fourth data of the first terminal device are obtained. After the adapter acquires the fourth data, the fourth data is sent to the second terminal equipment, so that the second terminal equipment displays an interface generated by the first terminal equipment executing the third control instruction according to the fourth data.

In one possible example, after the sending the first data to the second terminal device, the method further includes: if the at least one functional module is detected to comprise a second functional module of the display screen, searching a fourth application identifier corresponding to the second functional module from the application and module corresponding relation table; transmitting the sixth application identifier and a preset acquisition request to the first terminal equipment, wherein the acquisition request represents a request for acquiring data of a fourth application corresponding to the sixth application identifier; receiving fifth data of the first terminal equipment, wherein the fifth data is data of the fourth application; and displaying the content corresponding to the fifth data in the second functional module.

In a specific example, if the second functional module of the adapter includes a display screen, in order to improve the display efficiency, the second functional module may directly obtain the corresponding displayed data from the first terminal device. Specifically, if it is detected that the second functional module includes a display screen, the adapter searches a sixth application identifier corresponding to the second functional module from the application-module correspondence table, and sends the sixth application identifier and a preset acquisition request to the first terminal device, so as to obtain data of a fourth application corresponding to the sixth application identifier. After the adapter acquires the data of the fourth application, the data are analyzed and displayed through the display screen of the second functional module.

In this example, if the functional module of the adapter includes a display screen, the data to be displayed corresponding to the application of the functional module can be directly obtained, so as to improve the efficiency of display, and further improve the user experience.

In one possible example, the at least one functional module may include a touch screen module for cutting video, may correspond to a video clip application, that is, may correspond to a video progress bar within the video clip application, may scale up the progress bar into a touch screen of the touch screen module, and may control progress of the corresponding video by controlling the touch screen. The at least one functional module may further include a sliding functional module for sliding a document page in the document application, a volume adjusting functional module for adjusting a volume, etc., so that a type of the functional module in the at least one functional module may be set according to actual needs, and is not limited herein.

The embodiment of the application provides a device for displaying an application interface based on multi-screen cooperation, which can be electronic equipment. Specifically, the device for displaying the application interface based on multi-screen collaboration provided in the embodiment of the application may include modules corresponding to the respective steps.

According to the embodiment of the application, the device based on the multi-screen collaborative display application interface can be divided into the functional modules according to the method example, for example, each functional module can be divided corresponding to each function, and two or more functions can be integrated into one processing module. The integrated modules may be implemented in hardware or in software functional modules. The division of the modules in the embodiment of the present application is schematic, which is merely a logic function division, and other division manners may be implemented in practice.

Referring to fig. 4, fig. 4 is a functional unit block diagram of an apparatus for displaying an application interface based on multi-screen collaboration according to an embodiment of the present application, where the apparatus includes:

an adapter comprising at least one functional module for controlling terminal devices connected to a first interface collector and a second interface collector of the adapter, the adapter being for throwing an interface of at least one first terminal device connected to the first interface collector onto a screen of at least one second terminal device connected to the second interface collector in real time, the apparatus comprising:

a first obtaining unit 401, configured to obtain, in response to a first screen-drop request of a first terminal device, first data of the first terminal device, where the first data characterizes interface display data of the first terminal device at a current point in time, the first terminal device is a device in the at least one first terminal device, and the second terminal device is a device in the at least one second terminal device;

a first sending unit 402, configured to send the first data to the second terminal device, so that the second terminal device displays an interface of the first terminal device at a current time point;

A processing unit 403, configured to generate a first operation event according to an operation performed by a user on a first function module of the at least one function module if the operation performed by the user on the first function module is detected;

a second obtaining unit 404, configured to obtain a first control instruction set corresponding to the first functional module, and find a first control instruction corresponding to the first operation event from the first control instruction set, where the first control instruction set includes correspondence between different first operation events and different control instructions, and the control instruction characterizes an instruction for controlling an application of the first terminal device to execute a preset operation;

a searching unit 405, configured to search a first application identifier corresponding to the first functional module from a preset application-module correspondence table, where the application-module correspondence table includes correspondence between different functional modules and different application identifiers;

an execution unit 406, configured to send the first application identifier and the first control instruction to the first terminal device, so that the first terminal device executes the first control instruction;

a receiving unit 407, configured to receive second data of the first terminal device, where the second data characterizes interface display data generated by the first terminal device by controlling the first application according to the first control instruction;

And a second sending unit 408, configured to send the second data to the second terminal device, so that the second terminal device displays an interface generated by the first terminal device executing the first control instruction according to the second data.

In one possible example, the search unit 405 is further configured to: when a plurality of reference application identifications corresponding to the first functional module are determined from a preset application-module corresponding relation table, acquiring a set of times of association of the application corresponding to each reference application identification in the plurality of reference application identifications with the first functional module, and determining the maximum times in the set; and setting the application identifier corresponding to the maximum number of times as the first application identifier.

In one possible example, the search unit 405 is further configured to: when a plurality of reference application identifiers corresponding to the first functional module are determined from a preset application-module corresponding relation table, acquiring the association times of the application corresponding to each reference application identifier in the plurality of reference application identifiers and the first functional module; sequencing the reference application identifiers in the plurality of reference application identifiers according to the sequence from the large number to the small number of times to obtain a sequencing table; the sorting table and a preset selection prompt message are sent to the first terminal equipment, and the selection prompt message characterizes and prompts the user to select a message with a reference application identifier from the sorting table; receiving a feedback message of the first terminal equipment, wherein the feedback message comprises a reference application identifier selected by the user; and setting the reference application identifier selected by the user as the first application identifier.

In a possible example, the apparatus further includes a generating unit, configured to generate a second operation event according to a second operation performed by the user on the first operation module of the second terminal device if the second operation is detected; if the operation event which is the same as the second operation event is found from a preset association relation table, acquiring an association instruction corresponding to the same operation event, wherein the association relation table comprises at least one operation event corresponding to the association instruction, and the association instruction represents an association instruction for associating an operation module of the second terminal equipment with the first terminal equipment; associating an operation module of the second terminal equipment with the first terminal equipment according to the association instruction; obtaining a third operation performed by the user on a second operation module of the second terminal device, and generating a third operation event according to the third operation; searching a second application identifier corresponding to the second operation module from the application and module corresponding relation table; the second control instruction set corresponding to the second operation module is obtained, and a second control instruction corresponding to the third operation event is searched from the second control instruction set, wherein the second control instruction set comprises the corresponding relation between different third operation events and different control instructions; and sending the second application identifier and the second control instruction to the first terminal device, so that the first terminal device is opposite to the second application corresponding to the second application identifier to execute the second control instruction; obtaining third data of the first terminal equipment, wherein the third data represents interface display data generated by the first terminal equipment for controlling the second application according to the second control instruction; and sending the third data to the second terminal equipment, so that the second terminal equipment displays an interface for the first terminal equipment to execute the second control instruction generation according to the third data.

In a possible example, the apparatus further includes a third obtaining unit, where the third obtaining unit is configured to obtain, if it is detected that the user performs a fourth operation on a third operation module of the second terminal device, a third application identifier of an application enabled by the second terminal device at a current time point; searching a fourth application identifier corresponding to the third operation module from a preset application set, wherein the application set comprises the corresponding relation between different operation modules of the second terminal equipment and different application identifiers of the second terminal equipment; if the third application identifier is different from the fourth application identifier, associating a third operation module of the second terminal device with the first terminal device; and generating a fourth operation event according to the fourth operation; searching a fifth application identifier corresponding to the third operation module from the application and module corresponding relation table; the third control instruction set corresponding to the third operation module is obtained, and a corresponding third control instruction is searched from the third control instruction set according to the fourth operation event, wherein the third control instruction set comprises corresponding relations of different fourth operation events and different control instructions; and sending the fifth application identifier and the third control instruction to the first terminal device, so that the first terminal device controls a third application corresponding to the fifth application identifier according to the third control instruction; the fourth data of the first terminal device is obtained, and the fourth data characterizes interface display data generated by the first terminal device for controlling the third application according to the third control instruction; and sending the fourth data to the second terminal equipment, so that the second terminal equipment displays an interface for the first terminal equipment to execute the third control instruction generation according to the fourth data.

The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present application are all or partially produced. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired or wireless means. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more sets of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.

The embodiment of the application also provides a computer storage medium, where the computer storage medium stores a computer program for electronic data exchange, where the computer program causes a computer to execute part or all of the steps of any one of the methods described in the embodiments of the method, where the computer includes an electronic device.

Embodiments of the present application also provide a computer program product comprising a computer program operable to cause a computer to perform part or all of the steps of any one of the methods described in the method embodiments above.

The computer program product may be a software installation package, said computer comprising an electronic device.

It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.

In the several embodiments provided in the present application, it should be understood that the disclosed method, apparatus, and system may be implemented in other manners. For example, the device embodiments described above are merely illustrative; for example, the division of the units is only one logic function division, and other division modes can be adopted in actual implementation; for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may be physically included separately, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in hardware plus software functional units.

The integrated units implemented in the form of software functional units described above may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Although the present invention is disclosed above, the present invention is not limited thereto. Variations and modifications, including combinations of the different functions and implementation steps, as well as embodiments of the software and hardware, may be readily apparent to those skilled in the art without departing from the spirit and scope of the invention.

Claims

1. A method of displaying an application interface based on multi-screen collaboration, characterized by an adapter applied to a multi-screen collaboration system comprising at least one first terminal device and at least one second terminal device, each of the at least one first terminal device being connected with a first interface collector of the adapter, each of the at least one second terminal device being connected with a second interface collector of the adapter, the adapter being for real-time delivery of on-screen display data of the first terminal device to the second terminal device, the adapter comprising at least one functional module characterizing a module for controlling applications within the first terminal device, the method comprising:

2. The method of claim 1, wherein the searching for the first application identifier corresponding to the first functional module from the preset application-module correspondence table includes:

when a plurality of reference application identifications corresponding to the first functional module are determined from a preset application-module corresponding relation table, acquiring a set of times of association of the application corresponding to each reference application identification in the plurality of reference application identifications with the first functional module, and determining the maximum times in the set;

And setting the application identifier corresponding to the maximum number of times as the first application identifier.

3. The method of claim 1, wherein the searching for the first application identifier corresponding to the first functional module from the preset application-module correspondence table includes:

when a plurality of reference application identifiers corresponding to the first functional module are determined from a preset application-module corresponding relation table, acquiring the association times of the application corresponding to each reference application identifier in the plurality of reference application identifiers and the first functional module;

sequencing the reference application identifiers in the plurality of reference application identifiers according to the sequence from large to small of the times to obtain a sequencing table;

transmitting the ranking list and a preset selection prompt message to the first terminal equipment, wherein the selection prompt message characterizes and prompts the user to select a message with a reference application identifier from the ranking list;

receiving a feedback message of the first terminal equipment, wherein the feedback message comprises a reference application identifier selected by the user;

and setting the reference application identifier selected by the user as the first application identifier.

4. A method according to any of claims 1-3, characterized in that after said sending of said second data to said second terminal device, the method further comprises:

If the user is detected to perform a second operation on the first operation module of the second terminal device, a second operation event is generated according to the second operation;

if the operation event which is the same as the second operation event is found from a preset association relation table, acquiring an association instruction corresponding to the same operation event, wherein the association relation table comprises at least one operation event corresponding to the association instruction, and the association instruction characterizes an association instruction for associating an operation module of the second terminal device with the first terminal device;

associating an operation module of the second terminal equipment with the first terminal equipment according to the association instruction;

acquiring a third operation performed by the user on a second operation module of the second terminal equipment, and generating a third operation event according to the third operation;

searching a second application identifier corresponding to the second operation module from the application and module corresponding relation table;

acquiring a second control instruction set corresponding to the second operation module, and searching a second control instruction corresponding to the third operation event from the second control instruction set, wherein the second control instruction set comprises the corresponding relation between different third operation events and different control instructions;

The second application identifier and the second control instruction are sent to the first terminal equipment, so that the first terminal equipment executes the second control instruction to the second application corresponding to the second application identifier;

acquiring third data of the first terminal equipment, wherein the third data represents interface display data generated by the first terminal equipment for controlling the second application according to the second control instruction;

and sending the third data to the second terminal equipment so that the second terminal equipment displays an interface generated by the first terminal equipment executing the second control instruction according to the third data.

5. A method according to any of claims 1-3, characterized in that after said sending of said second data to said second terminal device, the method further comprises:

if the user is detected to perform fourth operation on a third operation module of the second terminal device, a third application identifier of an application started by the second terminal device in the current time point is obtained;

searching a fourth application identifier corresponding to the third operation module from a preset application set, wherein the application set comprises the corresponding relation between different operation modules of the second terminal equipment and different application identifiers of the second terminal equipment;

If the third application identifier is different from the fourth application identifier, associating a third operation module of the second terminal device with the first terminal device;

generating a fourth operation event according to the fourth operation;

searching a fifth application identifier corresponding to the third operation module from the application and module corresponding relation table;

acquiring a third control instruction set corresponding to the third operation module, and searching a corresponding third control instruction from the third control instruction set according to the fourth operation event, wherein the third control instruction set comprises corresponding relations of different fourth operation events and different control instructions;

the fifth application identifier and the third control instruction are sent to the first terminal equipment, so that the first terminal equipment controls a third application corresponding to the fifth application identifier according to the third control instruction;

acquiring fourth data of the first terminal equipment, wherein the fourth data represents interface display data generated by the first terminal equipment for controlling the third application according to the third control instruction;

and sending the fourth data to the second terminal equipment so that the second terminal equipment displays an interface generated by the first terminal equipment executing the third control instruction according to the fourth data.

6. The method of claim 1, wherein after the transmitting the first data to the second terminal device, the method further comprises:

if the at least one functional module is detected to comprise a second functional module of the display screen, a sixth application identifier corresponding to the second functional module is searched from the application and module corresponding relation table;

transmitting the sixth application identifier and a preset acquisition request to the first terminal equipment, wherein the acquisition request represents a request for acquiring data of a fourth application corresponding to the sixth application identifier;

receiving fifth data of the first terminal equipment, wherein the fifth data is data of the fourth application;

and displaying the content corresponding to the fifth data in the second functional module.

7. An apparatus for displaying an application interface based on multi-screen collaboration, the apparatus comprising an adapter, the adapter comprising at least one functional module for controlling terminal devices connected to a first interface collector and a second interface collector of the adapter, the adapter for throwing an interface of at least one first terminal device connected to the first interface collector onto a screen of at least one second terminal device connected to the second interface collector in real time, the apparatus comprising:

8. An adapter, characterized in that it comprises a micro-control unit, a first interface collector, a second interface collector, at least one functional module, a first conversion module and a second conversion module, said first interface collector comprising at least one interface, said second interface collector comprising at least one interface, said first interface collector being connected to the first conversion module, said first conversion module being connected to said second conversion module, said second conversion module being connected to said second interface collector, said micro-control unit being connected to each of said first conversion module, said second conversion module and said at least one functional module for controlling each of said first conversion module, said second conversion module and said at least one functional module, said adapter being connected to at least one first terminal device via said first interface collector, said first conversion module being adapted to convert second data received by said first interface collector to said at least one first terminal device, said second conversion module being adapted to convert second data received by said first interface collector to said second terminal device, said second conversion module being adapted to obtain said second data, as per-second conversion module is adapted to said second data collector, said second conversion module being adapted to said third terminal device, said method is performed by said first conversion module 6.

9. The adapter of claim 8, wherein the first interface collector is a collector comprising at least one of a display interface, a universal serial bus interface, and a network interface;

the second interface collector is a collector comprising at least one interface of a display interface, a universal serial bus interface and a network interface.

10. A computer-readable storage medium, characterized in that a computer program for electronic data exchange is stored, wherein the computer program causes a computer to perform the method according to any one of claims 1-6.