CN106933678B

CN106933678B - Multi-device application support framework and implementation method thereof

Info

Publication number: CN106933678B
Application number: CN201511025003.1A
Authority: CN
Inventors: 梁冠宇; 王勃; 方亚芬; 周凯
Original assignee: Shanghai Advanced Research Institute of CAS
Current assignee: Shanghai Advanced Research Institute of CAS
Priority date: 2015-12-30
Filing date: 2015-12-30
Publication date: 2020-03-31
Anticipated expiration: 2035-12-30
Also published as: CN106933678A

Abstract

The invention provides a multi-device application support framework, comprising: the device communication module is used for remote interaction among different device components and data exchange among the different device components, and provides an external interface and bottom layer support for device discovery and communication; the device management module is used for managing the accessed devices, providing an interface for acquiring the device use rights, monitoring the state of each device and providing a device state change notice; and the application development suite module is used for providing components capable of adapting to the screen splitting requirement and providing a component manager for controlling the behaviors of the components. The invention can completely liberate developers from the trivial matters of bottom layer communication, component distribution and the like, only needs to care about the details of the application, and develops the multi-device application with higher quality at higher speed. The invention can improve the application interactive experience of the user and fully utilize the personal equipment of the user.

Description

Multi-device application support framework and implementation method thereof

Technical Field

The invention relates to the technical field of communication, in particular to a multi-device application support framework and an implementation method thereof.

Background

With the great emergence of intelligent hardware and the continuous development of internet technology, the interaction between users and applications and the interaction between users and the internet are not limited to be single in a traditional desktop computer, a smart phone, a tablet personal computer, a smart television and the like, so that different interaction experiences are brought to the users.

The current application is that a certain device is used as an operation platform, and the operation mode has two problems: the problem that a certain type of application is not suitable for running on the same type of platform; users have more than one type of intelligent device, and the application itself cannot fully utilize the resources of the devices. The multi-device application solves the problem that the multi-device application can be split to run on a plurality of devices, but brings user experience, and is still a complete application. To enable rapid development of such multi-device applications, a multi-device application support framework is required to solve the above-mentioned problems.

Disclosure of Invention

In view of the above drawbacks of the prior art, an object of the present invention is to provide a multi-device application support framework and an implementation method thereof, for solving the problems in the prior art that an application using a certain device as an operating platform is not suitable for operating on the same type of platform or the application itself cannot fully utilize an intelligent device.

To achieve the above and other related objects, the present invention provides a multi-device application support framework, comprising: the device communication module is used for remote interaction among different device components and data exchange among the different device components, and provides an external interface and bottom layer support for device discovery and communication; the device management module is used for managing the accessed devices, providing an interface for acquiring the device use rights, monitoring the state of each device and providing a device state change notice; and the application development suite module is used for providing components capable of adapting to the screen splitting requirement and providing a component manager for controlling the behaviors of the components.

In an embodiment of the present invention, the device communication module includes: the proxy-stub unit comprises a stub subunit defined with the external interface, a proxy subunit and a proxy engine corresponding to the stub subunit, the proxy stub and the proxy engine are used for calling an internally defined transmission interface and completing data exchange between the internal network processing unit and the external network processing unit through the isolation exchange unit, and an information exchange channel between the internal network client and the external network server is established; the RPC communication daemon unit is used for realizing peer-to-peer communication in the equipment and communication among different equipment; and the data exchange daemon unit is used for realizing point-to-point data exchange among different devices.

In an embodiment of the present invention, the application development suite module hides details of interaction with the device management module and provides an event interface to a user.

In an embodiment of the invention, the component manager manages the allocation of the adapted components and devices and distributes the components to the respective devices.

In one embodiment of the present invention, the components include basic component assemblies, custom component assemblies, and custom component and basic component assemblies.

A method for realizing a multi-device application support framework comprises the following steps: s1: providing a component capable of meeting the screen splitting requirement through the application development kit module; s2: after new equipment is found, the component manager generates a component distribution scheme by acquiring equipment characteristics and comparing the characteristics of the components with the characteristics of the equipment, then informs a user, and finally determines whether to distribute the components to specific equipment by the user; and then the component is split to run on a plurality of devices through the device management module and the device communication module.

In an embodiment of the present invention, the step S2 includes the following steps: s21: the remote interaction among different equipment components and the data exchange among the components among different equipment are realized through the equipment communication module, when the framework operates, a self-defined external interface is released in a specific network, and remote interfaces provided by other components can be conveniently accessed through the external interface; s22: and the device management module acquires the interface of the device usage right, monitors the state of each device and notifies the change of the device state.

In an embodiment of the present invention, the step S21 includes the following steps: s211, proxy-stub process: the method comprises the steps that an external interface is defined in a stub subunit, a corresponding agent subunit is defined at the same time, after a process is started, an interface of the stub is issued to an IPC component of a current running platform, other processes acquire an agent module of a target process or acquire an interface of the target process, and IPC is achieved through interface calling; s212, RPC communication daemon: the peer-to-peer communication between the devices is realized through an RPC communication daemon, and a process needing cross-device communication needs to call a corresponding interface through an agent subunit of the RPC communication daemon; s213, data exchange daemon: the client process establishes and obtains a data stream object through the interface, and operates the interface to realize data exchange among the devices and realize point-to-point data exchange among the devices.

In an embodiment of the present invention, the step S22 specifically includes the following steps: s221, the equipment management module collects the equipment information of the equipment in which the equipment is positioned when starting and generates description information; s222, when the equipment joins the equipment cluster, multicasting the equipment information of the equipment to inform all reachable equipment, synchronizing a shared information list, and sending out a corresponding event notice; when the device leaves the cluster, the multicast notification can reach the device, synchronously modifies the shared information list, and sends out a corresponding event notification; during the online period of the equipment, the heartbeat packet is multicast periodically to inform other equipment that the equipment is still online; when the heartbeat is overtime, the condition shows that the equipment is not reachable currently, and the equipment is off-line abnormally, and needs to send a corresponding event to notify an application process; s223, when the application process applies for using a certain device, a data stream channel is established, and a corresponding data stream object is returned.

As described above, the multi-device application support framework and the implementation method thereof of the present invention have the following beneficial effects:

according to the invention, the device communication module hides the implementation details of the bottom layer communication, exposes a simple communication interface to the outside, and realizes cross-device and cross-platform communication; through establishing a communication link between devices, managing information of interconnected devices and a quick application development kit, developers can conveniently and quickly realize multi-device application only by paying attention to application logic per se, abundant device resources of users are fully utilized, and optimal application experience is obtained.

Furthermore, the invention enables the developer to only process necessary event processes through the component manager, and the component manager can automatically call the developer to write the event processing process to determine the application behavior.

Furthermore, the application basic components conform to CMD specifications, each component is an independent module, the components adopt object-oriented design, all the components inherit basic classes such as input and output, the behaviors of the components are limited, and the component manager can realize scheduling management on the components according to the types of the components.

Furthermore, the invention can completely liberate developers from the trivial matters of bottom layer communication, component distribution and the like, only needs to care about the details of the application, and develops the multi-device application with higher quality at a higher speed. The method can improve the application interaction experience of the user and fully utilize the personal equipment of the user.

Drawings

Fig. 1 is a schematic diagram illustrating a multi-device application support framework and an implementation method thereof according to an embodiment of the invention.

Fig. 2 is a flowchart illustrating a method for implementing a multi-device application support framework according to an embodiment of the present invention.

Fig. 3 is a schematic diagram illustrating an implementation process of a device communication module in an embodiment of a multi-device application support framework and an implementation method thereof according to the present invention.

Fig. 4 is a schematic diagram illustrating a device management implementation process in an embodiment of the multi-device application support framework and the implementation method thereof according to the present invention.

Fig. 5 is a schematic diagram illustrating an implementation process of an application development suite in an embodiment of a multi-device application support framework and an implementation method thereof according to the present invention.

Description of the element reference numerals

1 Multi-device application support framework

11 device communication module

12 device management module

13 application development kit module

S1-S2

Detailed Description

The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.

It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.

The invention provides a multi-device application support framework, which is an application platform for interaction experience among different intelligent devices such as a user smart phone, a tablet personal computer and a smart television, can improve the application interaction experience of the user, and fully utilizes personal devices of the user.

Referring to fig. 1, a schematic diagram of a multi-device application support framework according to the present invention is shown, where as shown in the figure, the multi-device application support framework 1 includes: a device communication module 11, a device management module 12 and an application development kit module 13,

the device communication module 11 is used for remote interaction among different device components and data exchange among different device components, providing an external interface and a bottom layer support for device discovery and communication, only exposing the external interface to a user, and hiding the details of the bottom layer support for device discovery and communication; in an embodiment, the device communication module 11 includes: the proxy-stub unit comprises a stub subunit defined with the external interface, a proxy subunit and a proxy engine corresponding to the stub subunit, the transmission interface defined inside is called through the proxy stub and the proxy engine, data exchange between the internal network processing unit and the external network processing unit is completed through the isolation exchange unit, and an information exchange path between the internal network client and the external network server is established; the RPC communication daemon unit is used for realizing peer-to-peer communication in the equipment and communication among different equipment; the data exchange daemon unit is used for realizing point-to-point data exchange among different devices.

A device management module 12, configured to manage accessed devices, provide an interface for obtaining device usage rights, monitor states of the devices, and provide a device state change notification;

the application development suite module 13 is used for providing components capable of meeting the screen splitting requirement and providing a component manager for controlling the behavior of each component; in an embodiment, the application development suite module hides the interaction details with the device management module, provides an event interface to the user, and the component manager manages the allocation of adapted components and devices and distributes the components to corresponding devices, wherein the components include a basic component assembly, a custom component assembly, and a custom component and basic component assembly.

The invention provides a method for realizing a multi-device application support framework, which is used for an application platform for interaction experience among different intelligent devices of a user, such as a smart phone, a tablet personal computer, a smart television and the like, can improve the application interaction experience of the user, and fully utilizes personal devices of the user.

Please refer to fig. 2, which is a flowchart illustrating a method for implementing a multi-device application support framework according to the present invention. As shown, it comprises the following steps:

s1: providing a component capable of meeting the screen splitting requirement through the application development kit module;

s2: after new equipment is found, the component manager generates a component distribution scheme by acquiring equipment characteristics and comparing the characteristics of the components with the characteristics of the equipment, then informs a user, and finally determines whether to distribute the components to specific equipment by the user; then, the components are split and operated on a plurality of devices through a device management module and a device communication module; in an embodiment, step S2 includes the following steps: s21: the remote interaction among different equipment components and the data exchange among the components among different equipment are realized through the equipment communication module, when the framework operates, a self-defined external interface is released in a specific network, and remote interfaces provided by other components can be conveniently accessed through the external interface; s22: and the device management module acquires the interface of the device usage right, monitors the state of each device and notifies the change of the device state.

Further, step S21 specifically includes the following steps:

s211, proxy-stub process: the method comprises the steps that an external interface is defined in a stub subunit, a corresponding agent subunit is defined at the same time, after a process is started, an interface of the stub is issued to an IPC component of a current running platform, other processes acquire an agent module of a target process or acquire an interface of the target process, and IPC is achieved through interface calling;

s212, RPC communication daemon: the peer-to-peer communication between the devices is realized through an RPC communication daemon, and a process needing cross-device communication needs to call a corresponding interface through an agent subunit of the RPC communication daemon;

s213, data exchange daemon: the process can realize point-to-point data exchange among the devices, the client process establishes and obtains a data stream object through the interface, and the interface is operated on the data stream object to realize the data exchange among the devices.

Further, step S22 specifically includes the following steps:

s221, the equipment management module collects the equipment information of the equipment in which the equipment is positioned when starting and generates description information;

s222, when the equipment joins the equipment cluster, multicasting the equipment information of the equipment to inform all reachable equipment, synchronizing a shared information list, and sending out a corresponding event notice; when the device leaves the cluster, the multicast notification can reach the device, synchronously modifies the shared information list, and sends out a corresponding event notification; during the online period of the equipment, the heartbeat packet is multicast periodically to inform other equipment that the equipment is still online; when the heartbeat is overtime, the condition shows that the equipment is not reachable currently, and the equipment is off-line abnormally, and needs to send a corresponding event to notify an application process;

s223, when the application process applies for using a certain device, a data stream channel is established, and a corresponding data stream object is returned.

The following specifically describes an implementation process of the device communication module, an implementation process of device management, and an implementation process of an application development suite.

A. The implementation process of the device communication module comprises the following steps:

referring to fig. 3, fig. 3 is a schematic diagram illustrating an implementation process of a device communication module in an embodiment, where as shown, the implementation process of the inter-device communication module mainly includes three parts: proxy-process of Stub (Proxy-Stub), RPC communication daemon, data exchange daemon.

Proxy-Stub (Proxy-Stub) process: and defining the external interface of the user in the stub subunit, and defining the corresponding agent subunit at the same time, when the process is started. The framework mechanism can ensure that the interface of the stub is issued only in the IPC component of the current running platform, the IPC refers to inter-process communication, other processes can obtain the proxy subunit of the target process through the framework providing method, or the interface of the target process is obtained by using the method provided by the current platform (therefore, the process realized based on the framework is compatible with other processes, but the framework hides the bottom layer complex mechanism), and the IPC can be realized through interface calling. In FIG. 3, process A-1, process A-2, process B-1, process C-1, etc. are all such processes.

As shown in fig. 3, based on the processes implemented by the proxy-stub unit, the RPC Communication Daemon runs uniquely on each device, IPCs between devices implement peer-to-peer Communication through the Daemon, and the process requiring cross-device Communication only needs to call a corresponding interface through the proxy subunit of the CD. This achieves the following effects:

a) and the only RPC import and export is limited, and a safety verification mechanism is conveniently realized.

b) Only one communication link is maintained between every two devices, and the overhead is greatly reduced.

c) Each process does not need to separately realize the function of connecting other equipment, and the function multiplexing is realized.

As shown in fig. 3, a data exchange daemon, that is, a DataTransfer (DT for short), is uniquely operated on each device based on a process implemented by a proxy-stub unit, point-to-point data exchange between devices can be implemented by the process, a client process (e.g., process a-2) establishes and obtains a data stream object through an interface of the DT, and then operates the interface to the data stream object to implement data exchange between the devices. The DT is separated from the CD, mainly considering the different properties of the two connections, the message of the CD is usually short, and the CD needs to have the fastest response speed as possible, while the DT usually occupies the connection channel for a long time, and if the DT is put together, the DT causes serious RPC delay.

B. The device management realizes the process:

the device management module, namely Resource-Manager, runs on each device in the form of a daemon process. To implement the main responsibilities of the device management module, the following functions need to be implemented:

when the unit starts, the unit collects the device information of the device, generates accurate description information (including the information of the device itself, such as screen size, resolution and the like, and additionally including the applicable type of the device, such as display output, control input and the like), and initializes a shared information list.

Referring to fig. 4, fig. 4 is a schematic diagram illustrating a device management implementation process in an embodiment, as shown in the figure, when a device is online (joins a device cluster), a multicast device information notifies all reachable devices, performs shared information list synchronization, and sends a corresponding event notification; similarly, when the device is offline (leaves the cluster), the multicast notification can reach the device, synchronously modify the shared information list and send out a corresponding event notification; during the online period of the equipment, the heartbeat packet is multicast periodically to inform other equipment that the equipment is still online currently, and when the heartbeat is overtime, the equipment is not reachable currently, is abnormally offline and needs to send corresponding event notification to the application process.

When an application process applies for using a certain device (non-native), the establishment of the data stream channel is completed, and a corresponding data stream object is returned. Taking video output equipment as an example, the specific steps are as follows:

a) applying for using a screen of another device as an output device;

b) finishing the establishment of a data stream (stream1) channel between devices through DT;

c) the peer node Resource-Manager process receives the request, establishes a data stream (stream2) channel from the DT process to the video output process, and connects stream1 and stream 2;

d) the native node establishes an application process and a data stream (stream3) channel of the DT, and connects stream1 and stream 3.

e) Returning the stream3 object to the application process, the application process inputs the generated video data into stream3, i.e., can be processed by the video output process of another device, and outputs the processed video data to the screen.

C. The application development suite implementation process comprises the following steps:

referring to fig. 5, fig. 5 is a schematic diagram illustrating an implementation process of an application development kit according to an embodiment, and as shown in the figure, an application development kit module is submitted to a user in the form of an SDK, where the application development kit module includes a basic application framework, basic components, and related description documents to help a developer to quickly construct such applications. The application base components conform to the CMD specification, and each component is an independent module. Meanwhile, the components adopt object-oriented design, all the components inherit basic classes such as input and output, the behaviors of the components are limited, and the component manager can realize scheduling management on the components according to the types of the components.

The application framework contains several components, the most important of which is the component manager:

1) the interaction details with the equipment management module are hidden, and only an event interface is exposed to an application developer;

2) an organization and management component, a distribution and recovery component, establishes data flow paths across the equipment components.

With the component manager, as a developer, only necessary event processing procedures need to be implemented, for example, a device more suitable for display, a device more suitable for control input, and the like are found, and the component manager can automatically call the developer to write the event processing procedures to determine the behavior of the application.

In summary, the invention hides the implementation details of the underlying communication through the device communication module, exposes a simple communication interface to the outside, and implements cross-device and cross-platform communication; through establishing a communication link between devices, managing information of interconnected devices and a quick application development kit, developers only need to pay attention to application logic per se, multi-device application can be conveniently and quickly realized, abundant device resources of users are fully utilized, and optimal application experience is obtained. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.

The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.

Claims

1. a multi-device application support framework, is characterized in that, comprises:

The device communication module is used for remote interaction between different device components and data exchange between components between different devices, providing external interfaces and underlying support for device discovery and communication;

The device communication module includes:

A proxy-stub unit, which includes a stub sub-unit defined with the external interface, a proxy sub-unit and a proxy engine corresponding to the stub sub-unit, and the internally defined transmission interface is invoked through the proxy stub and the proxy engine and is exchanged through isolation unit to complete the data exchange between the internal and external network processing units, and establish an information exchange path between the internal network client and the external network server;

The RPC communication daemon unit has a corresponding unique RPC communication daemon process on each device, which is used to realize peer-to-peer communication within the device and communication between different devices;

The data exchange daemon unit has a corresponding unique data exchange daemon process on each device, which is used to realize point-to-point data exchange between different devices;

The device management module is used to manage the accessed devices, provide an interface for obtaining the right to use the device, monitor the status of each device, and provide notification of device status changes;

The application development kit module is used to provide components that can adapt to split-screen requirements, and a component manager to manage the behavior of each of the components.

2 . The multi-device application support framework according to claim 1 , wherein the application development kit module hides the interaction details with the device management module, and provides an event interface to the user. 3 .

3 . The multi-device application support framework according to claim 1 , wherein the component manager manages and distributes the adapted components and devices, and distributes the components to the corresponding devices. 4 .

4 . The multi-device application support framework according to claim 1 , wherein the components include a basic component combination, a custom component combination, and a custom component and a basic component combination. 5 .

5. The method for realizing the multi-device application support framework as described in any one of claims 1 to 4, characterized in that: comprising the following steps:

S1: Provide components that can adapt to split-screen requirements through the application development kit module;

S2: After discovering a new device, the component manager generates a component distribution scheme by acquiring the device characteristics and comparing the characteristics of the components and the device, and then notifies the user, and finally the user decides whether to distribute the component to a specific device; then The components are split and run on multiple devices through the device management module and the device communication module;

S21: Realize the remote interaction between different device components and the data exchange between different device components through the device communication module, when the framework is running, the self-defined external interface is published in a specific network, and other external interfaces can be easily accessed through the external interface. The remote interface provided by the component;

The step S21 includes the following processes:

S211, proxy-stub process: define the external interface in the stub sub-unit, and define the corresponding proxy sub-unit. After the process starts, publish the stub interface to the IPC component of the current running platform, and other processes obtain the proxy of the target process module, or obtain the interface of the target process, and implement IPC through interface calls;

S212, RPC communication daemon process: peer-to-peer communication is realized between devices by running the RPC communication daemon process uniquely on each device, and the process that needs to communicate across devices needs to call the corresponding interface through the proxy subunit of the RPC communication daemon process;

S213, data exchange daemon process: a data exchange daemon process uniquely running on each device, establishes and obtains a data flow object through the interface, realizes data exchange between devices by operating the interface of the data flow object, and realizes point-to-point data between devices exchange.

6. The implementation method of the multi-device application support framework according to claim 5, wherein the step S2 comprises the following steps:

S22: Through the device management module, the interface for obtaining the right to use the device is used to monitor the state of each device and notify the device state change.

7. The implementation method of the multi-device application support framework according to claim 5, wherein the step S22 specifically comprises the following processes:

S221, the device management module collects device information of the device where it is located at startup, and generates description information;

S222, when the device joins the device cluster, multicast its own device information to notify all reachable devices, synchronize the shared information list, and issue a corresponding event notification;

When a device leaves the cluster, multicast notifies reachable devices, synchronously modifies the shared information list, and sends out corresponding event notifications; while the device is online, it periodically multicasts heartbeat packets to inform other devices that it is still online;

When the heartbeat timeout occurs, it means that the device is currently unreachable and abnormally offline, and the corresponding event needs to be sent to notify the application process;

S223, when the application process applies for using a certain device, establish a data flow channel, and return a corresponding data flow object.