CN114647351B - Interface element display method and device - Google Patents

Abstract

The application discloses an interface element display method and device, and relates to the field of electronic devices. The problems that when the information is checked or shared across applications, the interaction process is complicated, the operation steps are multiple, and the man-machine interaction performance is poor are solved. The specific scheme is as follows: the terminal displays a first interface comprising a first element on a display screen of the terminal; after receiving a drag operation on a first element, the terminal displays an animation of the first element moving along with the drag operation on a display screen of the terminal; after the terminal receives the drag release operation, a first floating element is displayed on a display screen of the terminal, and the content of the first floating element is the same as that of the first element.

Description

Method and device for displaying interface elements

This application is a divisional application, the application number of the original application is 202011564390.7, and the original application date is December 25, 2020. The entire content of the original application is incorporated in this application by reference.

technical field

The present application relates to the field of electronic devices, in particular to a method and device for displaying interface elements.

Background technique

At present, terminals such as mobile phones have become electronic devices that are frequently used in people's daily life and work. By installing an application program (application, APP, hereinafter referred to as an application) in a mobile phone, functions of the mobile phone can be improved and a better user experience can be provided for the user. However, in the process of daily use of mobile phones by users, if they need to view or share information across applications, there will be problems with cumbersome interaction processes, many operation steps, and poor human-computer interaction performance.

For example, when user 1 is using the shopping application of the mobile phone, and wants to share the picture of the currently browsed product with user 2, he needs to save the picture of the product in the gallery application of the mobile phone first, and then open the instant messaging application of the mobile phone to find user 2. Open the chat interface with user 2, then open the gallery application through the shortcut entry of the chat interface, select the saved product picture, and finally click send to share the product picture with user 2 through the instant messaging application. For another example, user A purchases a product using a shopping application in a mobile phone, and needs to go through multiple operation steps to check whether the product is shipped, the logistics of the product, and other information.

Contents of the invention

Embodiments of the present application provide a method and device for displaying interface elements, which solve the problems of cumbersome interaction process, many operation steps, and poor human-computer interaction performance when viewing or sharing information across applications.

In order to solve the above problems, the application adopts the following technical solutions:

In a first aspect, an embodiment of the present application provides a method for displaying interface elements, which is applied to a terminal, and the method may include: the terminal displays a first interface of an application on a display screen of the terminal, and the first interface includes a first element; After receiving the drag operation on the first element, display on the display screen of the terminal the animation that the first element moves with the drag operation; after receiving the drag operation, the terminal displays the first floating element on the terminal screen element, the content of the first floated element is the same as the content of the first element.

With the above technical solution, the terminal can display any element on the interface in the form of floating elements on the terminal screen according to user operations. Afterwards, no matter whether the user switches pages in the application, exits the application or opens other applications, the floating element can continue to be displayed on the terminal screen for cross-application information viewing, file transfer, information sharing and other operations.

In a possible implementation manner, the first interface further includes a second element within a preset range of the first element; the first floating element further includes content of the second element. In this way, according to user operations, the predetermined elements in the first interface can be presented on the terminal screen in the form of floating elements, which further meets the needs of users for cross-application information viewing, file transfer, and information sharing.

In another possible implementation, the first element is a text element, a Uniform Resource Locator (URL) website, an image element, an icon element, a card element, a list contact person element, a video window element or a file One or more of resource elements.

Wherein, when the first element is a text element, the content of the first floating element can be all or part of the text in the text element; when the first element is a URL, the content of the first floating element is a link icon, and the link icon Used to link to the web page corresponding to the URL; when the first element is an image element, the content of the first floating element is the picture in the image element; when the first element is an icon element, the content of the first floating element is an icon An icon in a class element; the first element is a list element, and the content of the first floating element is the contact avatar included in the list element; when the first element is a file resource element, the content of the first floating element is a link icon, The connection icon is used to link to file resources.

In another possible implementation manner, after the terminal displays the first interface on the display screen, the method may further include: after the terminal receives the first operation, displaying elements that support floating display in the first interface in the first Format display, elements that do not support floating display are displayed in the second format; the first element is an element that supports floating display, and the first format is different from the second format. In this way, the elements that support floating display in the interface can be highlighted, which is convenient for the user to check which elements support floating display, thereby improving user experience.

In another possible implementation manner, the first operation is an operation on a predetermined function control; or the first operation is an operation on an element supporting floating display in the first interface.

In another possible implementation manner, after the terminal displays the first floating element on the display screen of the terminal, the method may further include: the terminal displays a second interface on the display screen of the terminal, and the second interface includes the third element; After receiving the second operation on the third element, the terminal displays the second floating element on the display screen of the terminal, the content of the second floating element is the same as the content of the third element, and the second floating element and the first floating element follow the The preset arrangement rules are displayed on the display screen of the terminal. Wherein, the second operation includes a drag operation and a drag-and-drop operation, or when the third element is a text element or a URL, the second operation may also be a copy operation. The technical solution supports simultaneous display of multiple floating elements on the terminal screen, meeting the needs of users. In addition, each floating element can be displayed according to a sorting rule such as the order of generation time.

In another possible implementation manner, before the terminal displays the second floating element on the display screen of the terminal, the method may further include: the terminal determines that the number of floating elements displayed on the display screen of the terminal is less than N, where N is The maximum number of floating elements allowed to be displayed on the display at the same time. The maximum number of floating elements that can be displayed on the terminal screen at the same time can be limited, which not only meets the needs of users, but also does not affect the user experience due to the excessive number of floating elements in the interface.

In another possible implementation, the method may further include: when the terminal determines that the number of floating elements displayed on the display screen of the terminal is equal to N, deleting the N floating elements ranked first according to the preset arrangement rules first floating UI element, and display the second floating element. When the upper limit of floating elements allowed to be displayed at the same time is set, when the number of floating elements on the current screen has reached the upper limit, the user triggers the generation of new floating elements, the earliest generated floating elements can be deleted, and then the newly generated elements can be displayed The floating element.

In another possible implementation manner, before the terminal displays the second floating element on the display screen of the terminal, the method may further include: the terminal determines that the floating elements displayed on the display screen of the terminal do not have the second floating element type Same for floated elements.

In another possible implementation manner, the method may further include: when the terminal determines that there is a third floating element of the same type as the second floating element among the floating elements displayed on the display screen of the terminal, the terminal sets the second floating element and the third floating element are combined into a floating element group and displayed on the display screen of the terminal.

In another possible implementation manner, the method further includes: after the terminal receives the third operation on the floating element group, displaying the second floating element and the third floating element on a display screen of the terminal.

When the user triggers the generation of a new floating element, if there is an element of the same type as the newly generated floating element in the current interface, the newly generated floating element and the element can be automatically combined into a floating element group and displayed. If not present, this new floated element is displayed. In this way, floating elements of the same type can be automatically classified and displayed, which improves user experience. It also supports expanding the group of floating elements for users to view.

In another possible implementation manner, the method may further include: after the terminal receives the fourth operation on the first floating element, deleting the first floating element; or, the terminal deletes the first floating element on the display screen of the terminal After the preset duration is displayed on, delete the first floating element.

In another possible implementation manner, one or more of the following identifiers are displayed on the first floating element: an application identifier, message reminder information, and indication information; the indication information is used to indicate the type of the first floating element. In this way, it is convenient for the user to know which application or type the floating element comes from.

In another possible implementation, when the first element is a text element, a URL address, an image element or a file resource element, after the terminal displays the first floating element on the display screen, the method may also be Including: the terminal displays an interface including an input box; after the terminal receives a drag operation on the first floating element and releases the drag operation at the position of the input box, the terminal fills the content corresponding to the first floating element into the input box . For the floating elements of text, picture, card, file resource and video window, you can drag the floating element into the input box to trigger the terminal to send it to friends through the instant messaging application. It simplifies the interaction process and operation steps when sharing information across applications, and improves the performance of human-computer interaction.

In another possible implementation manner, in the case that the first element is a URL, after the terminal displays the first floating element on the display screen, the method may further include: after receiving the user's response to the first floating element, the terminal After the operation, the webpage corresponding to the URL is displayed. In this way, floating UI elements of the URL type support direct click and jump, which improves the performance of human-computer interaction.

In another possible implementation, when the first element is an icon element or a list contact element, after the terminal displays the first floating element on the display screen, the method may further include: the terminal receives the After the fifth operation on the first floating element, the terminal executes the first function, and the first function is the same as the function executed after the terminal receives the user's operation on the first element. In this way, the floating elements representing specific contacts extracted from the contact lists of different applications support functions such as sending files and making calls according to specific business scenarios. It simplifies the interaction process and operation steps when triggering the terminal to execute functions, and improves the performance of human-computer interaction.

In another possible implementation manner, in the case where the first element is a list contact person element, after the terminal displays the first floating element on the display screen, the method may further include: after the terminal receives the text in the interface, Drag and drop the URL or picture, and after releasing the drag operation at the position of the first floating element, the terminal sends the text, the URL or the picture to the terminal of the contact corresponding to the first floating element. Files, images, text, and URLs can be directly shared with the corresponding contacts by dragging and dropping content such as files, pictures, text, and URLs to the floating elements of the contact list. It simplifies the interaction process and operation steps when sharing information across applications, and improves the performance of human-computer interaction.

In some other embodiments, other operations may also be used to trigger the terminal to display the elements in the interface in the form of floating elements on the terminal screen. For example, the terminal displays an interface on the display screen of the terminal; after receiving the user's screenshot operation, the terminal intercepts the content of the interface displayed on the display screen to generate a screenshot; after that, the terminal displays floating elements on the display screen of the terminal, The floating element includes the screenshot. Or, the terminal displays an interface on the display screen of the terminal; after receiving the user's screen capture operation, the terminal intercepts the content of the interface displayed on the display screen, generates a screenshot and displays the screenshot; after that, the terminal receives the user's After dragging the screenshot and releasing the dragging operation, a floating element is displayed on the display screen of the terminal, and the floating element includes the screenshot. For another example, an interface is displayed on the display screen of the terminal, and the interface includes a predetermined button; after receiving an operation on the predetermined button, the terminal displays a floating element on the display screen, and the content of the floating element is the same as part of the interface. elements have the same content.

In the second aspect, the embodiment of the present application provides a method for displaying interface elements, which is applied to a terminal, and the method may include: the terminal displays a first interface on the display screen of the terminal, the first interface includes a first element, and the first element is text A class element or a URL address; after receiving the copy operation of the first element, the terminal displays the first floating element on the display screen of the terminal, and the content of the first floating element is the same as that of the first element.

With the above technical solution, the terminal can display text elements or URLs on the interface in the form of floating elements on the terminal screen according to user operations. Afterwards, no matter whether the user switches pages in the application, exits the application or opens other applications, the floating element can continue to be displayed on the terminal screen for cross-application information viewing, file transfer, information sharing and other operations. Moreover, combined with the existing interaction logic of the system, for example, after the terminal is triggered to display the copy button for the element, the user can click the copy button to trigger the text element or URL to be displayed on the terminal screen in the form of a floating element In this way, the user experience is improved.

In a possible implementation manner, after the terminal displays the first floating element on the display screen, the method may further include: the terminal displays an interface including an input box; the terminal receives a drag operation on the first floating element, and After releasing the drag operation at the position of the input box, the terminal fills the content corresponding to the first floating element into the input box. It simplifies the interaction process and operation steps when sharing information across applications, and improves the performance of human-computer interaction.

In a third aspect, an embodiment of the present application provides a display device, which can be applied to a terminal, and the device can include: a display unit, configured to display a first interface of an application on a display screen of the terminal, the first interface includes a first An element; the input unit is used to receive the drag operation on the first element; the display unit is also used to display the animation of the first element moving with the drag operation on the display screen of the terminal; the input unit is also used to receive the drag operation drag and release operation; the display unit is further configured to display the first floating element on the display screen of the terminal, and the content of the first floating element is the same as the content of the first element.

In a possible implementation manner, the first interface further includes a second element within a preset range of the first element; the first floating element further includes content of the second element.

In another possible implementation manner, the first element is one of a text element, a URL address, an image element, an icon element, a card element, a list contact element, a video window element or a file resource element. one or more species.

In another possible implementation manner, the input unit is further configured to receive the first operation; the display unit is further configured to display elements in the first interface that support floating display in a first form, and elements that do not support floating display display in a first form. The second form is displayed; the first element is an element that supports floating display, and the first form is different from the second form.

In another possible implementation manner, the first operation is an operation on a predetermined function control; or the first operation is an operation on an element supporting floating display in the first interface.

In another possible implementation manner, the display unit is further configured to display a second interface on the display screen of the terminal, and the second interface includes a third element; the input unit is further configured to receive a second operation on the third element ; The display unit is also used to display the second floating element on the display screen of the terminal, the content of the second floating element is the same as the content of the third element, and the second floating element and the first floating element are displayed on the terminal according to a preset arrangement rule on the display. Wherein, the second operation may include a drag operation and a drag and drop operation, or when the first element is a text element or a URL address, the second operation may be a copy operation.

In another possible implementation manner, the display device may further include: a determination unit, configured to determine that the number of floating elements displayed on the display screen of the terminal is less than N, where N is the number of floating elements that are allowed to be displayed simultaneously on the display screen The maximum number of elements.

In another possible implementation, the display device may further include: a deleting unit, configured to delete N floating elements according to preset Set the arrangement rule to rank the first floating UI element, and display the second floating element.

In another possible implementation manner, the determining unit is further configured to determine that there is no floating element of the same type as the second floating element among the floating elements displayed on the display screen of the terminal.

In another possible implementation manner, the display unit is further configured to, when the determining unit determines that there is a third floating element of the same type as the second floating element among the floating elements displayed on the and the third floating element are combined into a floating element group and displayed on the display screen of the terminal.

In another possible implementation manner, the input unit is further configured to receive a third operation on the floating element group; the display unit is configured to display the second floating element and the third floating element on a display screen of the terminal.

In another possible implementation manner, the input unit is also used to receive the fourth operation on the first floating element, and the delete unit is also used to delete the first floating element; or, the delete unit is also used to After the floating element is displayed on the display screen of the terminal for a preset period of time, the first floating element is deleted.

In another possible implementation manner, one or more of the following identifiers are displayed on the first floating element: an application identifier, message reminder information, and indication information; the indication information is used to indicate the type of the first floating element.

In another possible implementation, when the first element is a text element, a URL, an image element or a file resource element, the display unit is also used to display an interface including an input box; the input unit, It is also used for receiving a drag operation on the first floating element, and releasing the drag operation at the position of the input box; the display device may further include: a filling unit, configured to fill the content corresponding to the first floating element into the input box.

In another possible implementation manner, when the first element is an icon element or a list contact element, the input unit is further configured to receive a fifth operation on the first floating element; the display device may further include: The execution unit is configured to execute a first function, where the first function is the same as the function executed by the terminal after receiving the user's operation on the first element.

In another possible implementation, when the first element is a list contact person element, the input unit is also used to receive a drag operation on the text, URL or picture in the interface, and Release and drag operation of the position; the display device may further include: a sending unit, configured to send text, URL or picture to the terminal of the contact corresponding to the first floating element.

In a fourth aspect, an embodiment of the present application provides a display device applied to a terminal, and the display device may include: a display unit configured to display a first interface on a display screen of the terminal, the first interface includes a first element, and the first The element is a text element or a Uniform Resource Locator URL; the input unit is used to receive the copy operation of the first element; the display unit is also used to display the first floating element on the display screen of the terminal, and the first floating element The content is the same as the content of the first element.

In a possible implementation manner, the display unit is further configured to display an interface including an input box; the input unit is further configured to receive a drag operation on the first floating element, and release the drag operation at the position of the input box; The display device may further include: a filling unit, configured to fill the input box with content corresponding to the first floating element.

In a fifth aspect, an embodiment of the present application provides a display device, including: a processor; a memory for storing processor-executable instructions; wherein, when the processor executes the instructions, the display device implements the first aspect or the second aspect. The method described in any one of the possible implementations of the first aspect, or the second aspect or the method described in any one of the possible implementations of the second aspect.

In the sixth aspect, the embodiments of the present application provide a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by the electronic device, the electronic device implements the first aspect or the possible implementation manner of the first aspect. The method described in any one, or implement the method described in any one of the second aspect or possible implementation manners of the second aspect.

In the seventh aspect, the embodiment of the present application provides an electronic device (such as the above-mentioned terminal), the electronic device includes a display screen, one or more processors, and a memory; the display screen is coupled to the processor and the memory; the memory is used to store computer programs Code, the computer program code includes computer instructions, and when the computer instructions are executed by the electronic device, the electronic device executes the method described in any one of the first aspect or possible implementations of the first aspect, or executes the method described in the second aspect Or the method described in any one of the possible implementations of the second aspect.

In an eighth aspect, an embodiment of the present application provides a computer program product, including computer readable codes, or a non-volatile computer readable storage medium carrying computer readable codes, when the computer readable codes are stored in an electronic device ( When running in the above-mentioned terminal), the processor in the electronic device executes the method described in any one of the first aspect or the possible implementation manners of the first aspect, or executes the second aspect or the possible implementation manners of the second aspect any one of the methods described.

Understandably, the display device described in the third aspect and any possible implementation thereof provided above, the display device described in the fourth aspect and any possible implementation thereof, and the display device described in the fifth aspect For the beneficial effects that can be achieved by the device, the computer-readable storage medium described in the sixth aspect, the electronic device described in the seventh aspect, and the computer program product described in the eighth aspect, you can refer to the first aspect and any one thereof. One possible implementation manner or the beneficial effects of the second aspect and any possible implementation manner thereof, which will not be repeated here.

Description of drawings

FIG. 1 is a schematic diagram of a display interface in a related art provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of a display interface in another related art provided by an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a mobile phone provided by an embodiment of the present application;

FIG. 4 is a schematic composition diagram of a software architecture of a terminal provided in an embodiment of the present application;

FIG. 5A is a schematic diagram of a display interface provided by an embodiment of the present application;

FIG. 5B is a schematic diagram of another display interface provided by the embodiment of the present application;

FIG. 6 is a schematic flowchart of a method for displaying interface elements provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of another display interface provided by the embodiment of the present application;

FIG. 8 is a schematic diagram of another display interface provided by the embodiment of the present application;

FIG. 9 is a schematic diagram of another display interface provided by the embodiment of the present application;

FIG. 10 is a schematic diagram of another display interface provided by the embodiment of the present application;

FIG. 11 is a schematic diagram of another display interface provided by the embodiment of the present application;

FIG. 12 is a schematic flowchart of another interface element display method provided by the embodiment of the present application;

FIG. 13A is a schematic diagram of another display interface provided by the embodiment of the present application;

FIG. 13B is a schematic diagram of another display interface provided by the embodiment of the present application;

FIG. 14 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 15 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 16 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 17 is a schematic diagram of another display interface provided by the embodiment of the present application;

FIG. 18 is a schematic diagram of another display interface provided by the embodiment of the present application;

FIG. 19 is a schematic diagram of another display interface provided by the embodiment of the present application;

FIG. 20 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 21 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 22 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 23 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 24 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 25 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 26 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 27 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 28 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 29 is a schematic diagram of another display interface provided by the embodiment of the present application;

FIG. 30 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 31 is a schematic diagram of another display interface provided by the embodiment of the present application;

Fig. 32 is a schematic composition diagram of a display device provided by an embodiment of the present application;

FIG. 33 is a schematic diagram of a chip system provided by an embodiment of the present application.

Detailed ways

At present, by installing applications in terminals such as mobile phones, the functions of mobile phones can be improved to provide users with a better experience. However, in the process of daily use of mobile phones, the following scenarios are often encountered: to share a picture of a certain product with a friend, you need to save it in the gallery application first, and then click on the picture in the gallery application to complete the sharing, and the interaction is very troublesome; The phone number and other content shared by friends cannot be copied directly if they are shared through pictures. If they are copied after using (optical character recognition, OCR) recognition, the OCR recognition is not accurate, and the user can only switch back and forth between the typing interface and pictures on the mobile phone. Interface; send the URL of a certain website to a friend, and after copying the URL, the copied content is only saved on the clipboard, which is not visible to the user, and it is necessary to switch multiple applications to share with friends, which is a poor experience; purchased For a product, it takes multiple steps to check whether the product is shipped, the logistics of the product and other information, the interaction process is long, and so on. These scenarios all reflect that when users use mobile phones and other terminals on a daily basis, if they need to view or share information across applications, the interaction process will be cumbersome, the operation steps will be too many, and the human-computer interaction performance will be poor.

The current technology provides some solutions to solve the above problems.

Related technology 1: Huawei has customized the side (dock) bar function. The user can perform a sliding operation from the left or right edge of the mobile phone screen to the middle area of the mobile phone screen to trigger the mobile phone to display the dock bar. The display position of the dock bar on the mobile phone screen cannot be moved. When the user clicks on other areas outside the dock bar , the dock bar is closed. The dock bar includes icons of one or more applications, providing a quick entry for the user so that the corresponding application can be quickly opened, thereby simplifying the interaction process when the user uses multiple applications and improving usage efficiency. For example, referring to FIG. 1 , as shown in (a) in FIG. 1 , the mobile phone displays a memo interface 101 . The user wants to share the contents of the memo with friends through WeChat ^TM . The user can perform a sliding operation on the mobile phone screen from the right edge of the screen to the middle area of the mobile phone screen. As a response to the sliding operation, the mobile phone may display a dock bar 102 including icons of one or more applications, such as an icon 103 including WeChat ^™ . After the user operates the icon 103 of WeChat ^TM in the dock bar 102, as shown in (b) in Fig. The form displays the interface of WeChat ^TM . Afterwards, the user selects a friend in the interface of WeChat ^TM to share the content to be shared through WeChat ^TM . However, the dock bar only provides a quick entry to open the application, and users still need to go through cumbersome steps to share content with friends, and the human-computer interaction performance has not been improved.

Related technology 2: WeChat ^TM defines the floating window (or collection bar) function. This floating window is the entry point for temporary storage of applets or article links. For example, referring to FIG. 2 , as shown in (a) in FIG. 2 , the user browses the information page 201 of a public account in WeChat ^TM . The user can trigger the mobile phone to return to the previous page of the information page 201 by performing a sliding operation from the left side of the screen of the mobile phone to the right. As shown in (b) of FIG. 2 , with the sliding of the user's finger, the mobile phone can display a partial area of the information page 201 and a partial area 202 of the upper level page of the information page 201 . In response to the user's sliding operation, the mobile phone may also display a floating window indication area 203 at the lower right corner of the mobile phone screen. After the user slides his finger into the floating window indication area 203, as shown in (c) in FIG. , the floating window 204 includes a link to the information page 201, that is to say, the information page 201 is included in the favorite column, and the user can click the favorite column to view the included information page 201. Among them, up to five mini-programs or article links can be temporarily stored (or collected) in the floating window of WeChat ^TM at the same time. When multiple applets or article links are bookmarked in the floating window, the user can click the floating window to view the list of favorites. However, the floating window is only displayed in WeChat ^TM all the time, and when WeChat ^TM exits, the floating window will no longer be displayed. That is to say, after WeChat ^TM exits, the floating window cannot serve as a quick entry, so it cannot solve the problem of poor human-computer interaction performance when viewing or sharing information across applications. In addition, the floating window function of WeChat ^TM can only temporarily store the links of the entire page in the form of a floating window, and cannot support the temporary storage of some or a certain element in the page in the form of a floating window.

Related technology 3: Android ^TM provides a bubble function. An application (such as an instant messaging application) can call the interface corresponding to this function, and after receiving the corresponding user operation (such as long pressing or left-swiping the contact list, clicking the More button, and then selecting to open the floating chat avatar), the application can be opened. The avatars of the contacts are displayed on the edge of the phone screen in the form of bubbles. However, the bubble function provided by Android ^™ is only used to provide a quick entry to the contact's chat interface. That is, after the user clicks on the contact's avatar displayed in the form of a bubble, the mobile phone can display the chat interface of the contact without other functions, and does not support the display of other content in the form of a bubble except the contact's avatar, and There are many steps involved in triggering the display of contact avatars in the form of bubbles, the interaction is complicated, and the human-computer interaction performance cannot be improved very well.

It can be seen that the solutions provided by the above related technologies cannot well solve the problem of poor human-computer interaction performance when viewing or sharing information across applications.

An embodiment of the present application provides a method for displaying an interface element, and the method can be applied to a terminal. Using the method provided in this embodiment, after the terminal receives operations such as dragging and copying any element in the interface from the user, it can convert the element into a floating user interface (user interface, UI) element and display it on the terminal screen. Such as snapping is displayed on the edge of the terminal screen. In addition, no matter whether the user switches pages in the application or exits the application, the floating UI element can continue to be displayed on the terminal screen and is displayed on the top layer of the interface. The floating UI element in this embodiment can be moved, supports interaction with other interface elements, opens the corresponding interface after being clicked, receives the content dragged in by the user and sends it out, and other functions. In this way, it can help users realize local cross-application file transfer, information sharing and other operations in a more convenient and interactive way. It simplifies the interaction process during cross-application interaction, reduces the operation steps, and improves the performance of human-computer interaction.

Exemplarily, the terminal in the embodiment of the present application may be a mobile phone, a tablet computer, a handheld computer, a PC, a cellular phone, a personal digital assistant (personal digital assistant, PDA), a wearable device (such as a smart watch), a vehicle-mounted computer, For game consoles, augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR) devices, etc., this embodiment does not specifically limit the specific form of the terminal. In addition, the technical solutions provided in this embodiment can be applied to other electronic devices, such as smart home devices (such as television sets), in addition to the above-mentioned terminals (or mobile terminals).

The implementation of the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

In this embodiment, the terminal is a mobile phone as an example. Please refer to FIG. 3 , which is a schematic structural diagram of a mobile phone provided by an embodiment of the present application. The methods in the following embodiments can be implemented in a mobile phone with the above hardware structure.

As shown in Figure 3, the mobile phone may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1. Antenna 2, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193 and display 194, etc. Optionally, the mobile phone may further include a mobile communication module 150, a subscriber identification module (subscriber identification module, SIM) card interface 195, and the like.

Among them, the sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an environmental Light sensor 180L, bone conduction sensor 180M, etc.

It can be understood that the structure shown in this embodiment does not constitute a specific limitation on the mobile phone. In other embodiments, the mobile phone may include more or fewer components than shown, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor ( image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. . Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

The controller can be the nerve center and command center of the phone. The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuitsound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver (universal asynchronous receiver) /transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input/output (general-purpose input/output, GPIO) interface, SIM interface, and/or USB interface, etc.

The charging management module 140 is configured to receive a charging input from a charger. While the charging management module 140 is charging the battery 142 , it can also provide power for the mobile phone through the power management module 141 . The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 can also receive the input of the battery 142 to provide power for the mobile phone.

The wireless communication function of the mobile phone can be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in a mobile phone can be used to cover single or multiple communication bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

When the mobile phone includes the mobile communication module 150, the mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the mobile phone. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.

A modem processor may include a modulator and a demodulator. Wherein, the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is passed to the application processor after being processed by the baseband processor. The application processor outputs sound signals through audio equipment (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In some other embodiments, the modem processor may be independent from the processor 110, and be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless local area network (wireless local area networks, WLAN) (such as Wi-Fi network), Bluetooth (bluetooth, BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation ( frequency modulation, FM), NFC, infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the mobile phone is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the mobile phone can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), wideband code wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and/or IR technology, etc. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (globalnavigation satellite system, GLONASS), a Beidou satellite navigation system (beidou navigationsatellite system, BDS), a quasi-zenith satellite system (quasi-zenith) satellite system (QZSS) and/or satellite based augmentation systems (SBAS).

The mobile phone realizes the display function through the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light emitting diode). , AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diodes (quantum dot light emitting diodes, QLED), etc. In some embodiments, the mobile phone may include 1 or N display screens 194, where N is a positive integer greater than 1.

The mobile phone can realize shooting function through ISP, camera 193 , video codec, GPU, display screen 194 and application processor. In some embodiments, the mobile phone may include 1 or N cameras 193, where N is a positive integer greater than 1.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.

The internal memory 121 may be used to store computer-executable program codes including instructions. The processor 110 executes various functional applications and data processing of the mobile phone by executing instructions stored in the internal memory 121 . The internal memory 121 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like. The storage data area can store data (such as audio data, phone book, etc.) created during the use of the mobile phone. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.

The mobile phone can realize the audio function through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playback, recording, etc.

The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, pressure sensor 180A may be disposed on display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. When a touch operation acts on the display screen 194, the mobile phone detects the intensity of the touch operation according to the pressure sensor 180A. The mobile phone can also calculate the touched position according to the detection signal of the pressure sensor 180A.

The gyroscope sensor 180B can be used to determine the motion posture of the mobile phone. The air pressure sensor 180C is used to measure air pressure. The magnetic sensor 180D includes a Hall sensor. The mobile phone can use the magnetic sensor 180D to detect the opening and closing of the flip holster. The acceleration sensor 180E can detect the acceleration of the mobile phone in various directions (generally three axes). The distance sensor 180F is used to measure the distance. The mobile phone can use the proximity light sensor 180G to detect that the user holds the mobile phone close to the ear to talk, so that the screen can be automatically turned off to save power. The proximity light sensor 180G can also be used in leather case mode, automatic unlock and lock screen in pocket mode. The ambient light sensor 180L is used for sensing ambient light brightness. The fingerprint sensor 180H is used to collect fingerprints. The mobile phone can use the fingerprint characteristics collected to unlock the fingerprint, access the application lock, take pictures with the fingerprint, answer the incoming call with the fingerprint, etc. The temperature sensor 180J is used to detect temperature.

Touch sensor 180K, also known as "touch panel". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194 . In some other embodiments, the touch sensor 180K may also be disposed on the surface of the mobile phone, which is different from the position of the display screen 194 .

The bone conduction sensor 180M can acquire vibration signals. The keys 190 include a power key, a volume key and the like. The key 190 may be a mechanical key. It can also be a touch button. The motor 191 can generate a vibrating reminder. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. The indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.

When the mobile phone includes a SIM card interface 195, the SIM card interface 195 is used for connecting a SIM card. The SIM card can be inserted into the SIM card interface 195 or pulled out from the SIM card interface 195 to realize contact and separation with the mobile phone. The mobile phone can support 1 or N SIM card interfaces, and N is a positive integer greater than 1. The mobile phone interacts with the network through the SIM card to realize functions such as calling and data communication. In some embodiments, the mobile phone employs an eSIM, ie: an embedded SIM card. The eSIM card can be embedded in the mobile phone and cannot be separated from the mobile phone.

In this embodiment of the present application, the software system of the terminal is taken as an example to illustrate the software structure of the terminal. Please refer to FIG. 4 , which is a schematic composition diagram of a software architecture of a terminal provided by an embodiment of the present application.

Wherein, the software system of the terminal may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. Take the terminal software system as an example with a layered architecture. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces.

In some embodiments, as shown in FIG. 4 , the terminal may include an application layer and a framework layer (framework, FWK). The application layer can include a series of application packages. For example, the application package may include settings, calculator, camera, navigation, shopping, SMS, video player, music player, phone and other applications. The applications included in the application layer may be system applications of the terminal or third-party applications, which are not specifically limited in this embodiment of the present application. The application layer may also include a drag-and-drop service module and an element control module (controller). This embodiment does not make specific limitations here.

The framework layer is mainly responsible for providing an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The framework layer may include a window manager (window manager, WM) and the like. Wherein, the management manager may also be called a window management module, which is not specifically limited in this embodiment.

Of course, the terminal may also include other layers, such as a kernel layer (not shown in FIG. 4 ) and so on. The kernel layer is the layer between hardware and software. The kernel layer can at least include display drivers, camera drivers, audio drivers, sensor drivers, etc.

In this embodiment of the application, based on the above software architecture, when the terminal is currently displaying an application interface, after receiving operations such as dragging or copying any element in the interface from the user, the element can be converted into a floating UI element Displayed on the terminal screen, such as snapping to the edge of the terminal screen. Afterwards, no matter whether the user switches pages in the application or exits the application or opens other applications, the floating UI element can continue to be displayed on the terminal screen for cross-application information viewing, file transfer, information sharing and other operations.

Taking the terminal as a mobile phone as an example, the method for displaying interface elements provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings. Wherein, in this embodiment, the interface elements that support display as floating UI elements may be any elements in the interface. For example, it can be text, picture, icon, card, contact list, video window, file resource and other interface elements.

The process of displaying different types of interface elements as floating UI elements on the screen of a mobile phone is described as an example below.

With reference to FIG. 5A , FIG. 5B and FIG. 6 , the process of displaying text elements on the mobile phone screen in the form of floating UI elements is illustrated.

Wherein, the operation that triggers the display of the text-type element in the form of a floating UI element may be a user's drag operation on the text-type element, or a copy operation on the text-type element.

For example, referring to FIG. 5A , it is taken as an example that the operation that triggers the display of a text-type element in the form of a floating UI element is a user's drag operation on the text-type element. As shown in (a) of FIG. 5A , the mobile phone currently displays an information page 501 of an application. When the user wants to display some text in the information page 501 on the screen of the mobile phone as a floating UI element, the user can perform a drag operation on the text. After the user releases the drag, the mobile phone can display the text as a floating UI element on the mobile phone screen.

Wherein, the above-mentioned drag operation may include one or more operations. Exemplarily, the drag operation may include an operation of triggering the floating of text and an operation of triggering the movement of the floating text.

For example, an operation that triggers the floating of the text is an operation of long pressing and selecting the text, and an operation that triggers the movement of the floating text is an operation of pressing and moving the floating text. Continuing to refer to (a) in FIG. 5A , the user can long press and select the text 502 in the information page 501 . The information page 501 may be the first interface in this embodiment of the application. Text 502 may be the first element in this embodiment of the application. The mobile phone can receive the user's long press and selection operation on the text 502 . In response to the long press and selection operation, the mobile phone can display the text 502 in a floating state. For example, as shown in (b) in FIG. 5A , the mobile phone can perform background blur processing and display floating text 502 . Afterwards, by pressing the floating text 502 and moving the finger, the user can make the text 502 move on the screen of the mobile phone following the movement of the user's finger. For example, as shown in (b) of FIG. 5A , the user may use a finger to perform a pressing operation on the floating text 502 and move the finger, such as moving the finger to the right. In response to the user's operation, the mobile phone can display an animation in which the text 502 moves with the user's finger, giving the user a visual effect of the text 502 being dragged by the user. Afterwards, the user can lift the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag and release operation. After the mobile phone receives the dragging and releasing operation, as shown in (c) in Figure 5A, the mobile phone can display a floating UI element 503 on the mobile phone screen. The selected and dragged text is displayed on the phone screen in the form of floating UI elements. For example, the floating UI element 503 can be displayed on the edge of the screen of the mobile phone (such as the left edge or the right edge of the screen, (c) in FIG. 5A is shown as an example of being displayed on the right edge of the screen). The floating UI element 503 may be the first floating element in this embodiment of the application.

For another example, referring to FIG. 5B , it is taken as an example that the operation of triggering the display of a text-type element in the form of a floating UI element is a copy operation for a text-type element. As shown in (a) of FIG. 5B , the mobile phone currently displays an information page 601 of an application. When the user wants to display some text in the information page 601 on the screen of the mobile phone in the form of a floating UI element, the user can perform a copy operation on the text. After receiving the copy operation, the mobile phone can display the text on the screen of the mobile phone in the form of a floating UI element.

Wherein, the copy operation for text may include one or more operations. Exemplarily, the copy operation for the text may include long pressing the text and an operation on the copy button. Continuing to refer to (a) in FIG. 5B , the user can long press the text 602 in the information page 601 . The information page 601 may be the first interface in this embodiment of the application. Text 602 may be the first element in this embodiment of the application. As shown in (b) of FIG. 5B , in response to the user's long press operation on the text 602 , the mobile phone can display an operation menu, which includes a copy button 603 . After the mobile phone receives the user's operation on the copy button 603, such as the click operation, as shown in (c) in Figure 5B, the mobile phone can display a floating UI element 604 on the mobile phone screen, and the content and text in the floating UI element 604 602 is the same, that is, the text copied by the user is displayed on the screen of the mobile phone in the form of floating UI elements. The floating UI element 604 may be the first floating element in this embodiment of the present application.

Wherein, the above-mentioned dragging operation and dragging and releasing operation may be the second operation in the embodiment of the present application. The above copying operation may be the second operation in the embodiment of the present application.

It should be noted that, the above embodiments shown in FIG. 5A and FIG. 5B are described by taking the text element contained in the information page of the application as an example. The application may be a system application of the mobile phone or a third-party application, which is not limited in this embodiment. In addition, the text element can also be the text contained in other pages, such as the chat content in the chat interface of the instant messaging application, and for example, the text contained in the page of the shopping application, etc. limit.

Referring to Fig. 4, as shown in Fig. 6, as an example, the specific implementation process for text elements to be presented as floating UI elements on the screen of the mobile phone may be as follows: the mobile phone, for example, the application layer of the mobile phone receives the user's drag and drop of the text After the operation, the dragging of the text can be triggered, and the dragging event can be obtained and sent to the dragging service module of the application layer of the mobile phone. For example, the process for the application layer of the mobile phone to obtain the drag event may be: after the user performs the drag operation, the framework layer of the mobile phone can monitor the drag operation (for example, it can be a long press operation on the floating text), as In response, the framework layer of the mobile phone can generate a drag event (DragEvent). Applications in the mobile phone application layer can obtain drag events generated by the framework layer through callbacks. Wherein, the dragging event may include the content of the dragged text and the type of the dragged element (such as text class). After the drag service module of the mobile phone application layer receives the drag event, it can cooperate with the element control module of the mobile phone application layer to construct a shadow according to the type of the dragged element and the content of the text in the drag event. If the type of the dragged element is text, the drag service module of the mobile phone application layer can transmit the content of the text included in the drag event to the element control module, and the element control module can generate a shadow according to the content of the text. Afterwards, the element control module of the mobile phone application layer can transmit the generated shadow to the window manager of the mobile phone frame layer, so that the window manager can display the shadow in the form of floating UI elements. For example, the window manager displays a text box with rounded corners on the screen of the mobile phone, and the text box with rounded corners is a floating UI element, which includes the text dragged by the user. Similarly, after the application of the mobile application layer receives the user's copy operation of the text, it can also trigger the framework layer of the mobile phone to send the content of the copied text to the drag service module of the mobile application layer, so that the drag service module and element control The modules cooperate to construct a shadow, and the window manager of the mobile phone frame layer displays the shadow in the form of floating UI elements, so as to realize the presentation of text elements on the mobile phone screen in the form of floating UI elements.

Using the technical solution of the embodiment shown in Figure 5A or Figure 5B above, for text interface elements, the user can display the corresponding text on the screen of the mobile phone in the form of a floating UI element by performing a drag operation or copy operation, such as adsorption Displayed at the edge of the terminal screen. That is, the text dragged or copied by the user can be explicitly presented on the screen of the mobile phone through floating UI elements. And after receiving the user's operation of switching pages in the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the screen of the mobile phone, making it more convenient to view information across applications. And the floating UI element can be moved. For example, the user may perform a drag operation on the floating UI element, such as pressing and moving a finger on the display position of the floating UI element. In response to the user's drag operation, the mobile phone can display an animation of the floating UI element moving on the mobile phone screen. In addition, when the user wants to use the text across pages or applications, such as inputting the text into the input box of other applications, there is no need to switch pages or applications back and forth, and this can be achieved by dragging the floating UI element into the input box. The text in the floating UI element fills the input box. Specifically, as an example, after the mobile phone displays text in the form of a floating UI element on its screen, the mobile phone (such as the frame layer of the mobile phone) can store the identifier of the floating UI element in association with the content of the text in the mobile phone . Afterwards, when the mobile phone is displaying an application interface including an input box, after receiving a user's drag operation on the floating UI element, it can display the animation of the floating UI element moving. After the user drags the floating UI element to the display position of the input box, hovers and releases the finger, as a response to the operation, the mobile phone may fill the text in the floating UI element into the input box. For example, after the user drags the floating UI element to the display position of the input box and hovers over it, the element control module of the mobile phone application layer can notify the frame layer of the mobile phone to transfer the data corresponding to the floating UI element, that is, the The content of the text corresponding to the identifier of the element is sent to the application. Afterwards, after the user releases the finger, that is, releases the drag, the application of the mobile phone application layer can receive the content of the text sent by the mobile phone framework layer, and fill the text into the input box.

With reference to FIG. 7 and FIG. 8 , an example is given to illustrate the process of displaying picture elements on the mobile phone screen in the form of floating UI elements.

Wherein, the operation that triggers the display of the image-type element in the form of a floating UI element may be a user's drag operation on the image-type element.

For example, referring to FIG. 7 , as shown in (a) of FIG. 7 , the mobile phone displays an interface 701 of a gallery application. The interface 701 includes multiple pictures. Interface 701 may be the first interface in this embodiment of the application. When the user wants to display a certain picture in the interface 701 on the screen of the mobile phone in the form of a floating UI element, the user can perform a drag operation on the picture. After the user releases the drag, the mobile phone can display the picture (such as a thumbnail of the picture) on the screen of the mobile phone in the form of a floating UI element.

Wherein, the drag operation may include one or more operations. Exemplarily, the dragging operation may include an operation of triggering the floating of the picture and an operation of triggering the movement of the floating picture.

For example, an operation that triggers the floating of a picture is a long press operation on the picture, and an operation that triggers movement of the floating picture is an operation of pressing and moving the floating picture as an example. Continuing to refer to (a) in FIG. 7 , the user can long press the picture 702 in the interface 701 of the gallery application. The picture 702 may be the first element in this embodiment of the application. The mobile phone may receive the user's long press operation on the picture 702 . In response to the long press operation, the mobile phone can display the picture 702 in a floating state. For example, as shown in (b) in FIG. 7 , the mobile phone can perform background blur processing and display a floating picture 702 . Afterwards, by pressing the floating picture 702 and moving the finger, the user can make the picture 702 move on the screen of the mobile phone following the movement of the user's finger. For example, as shown in (b) of FIG. 7 , the user may use a finger to perform a pressing operation on the floating picture 702 and move the finger, for example, move the finger to the right. In response to the user's operation, the mobile phone can display an animation in which the picture 702 moves with the user's finger, giving the user a visual effect of the picture 702 being dragged by the user. Afterwards, the user can lift the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag and release operation. After the mobile phone receives the dragging and releasing operation, as shown in (c) in Figure 7, the mobile phone can display a floating UI element 703 on the mobile phone screen, and the floating UI element 703 includes a picture 702, such as a thumbnail of the picture 702 , that is, the picture dragged by the user is displayed on the screen of the mobile phone in the form of a floating UI element. For example, the floating UI element 703 can be displayed on the edge of the screen of the mobile phone (such as the left edge or the right edge of the screen, (c) in FIG. 7 is shown as an example of being displayed on the right edge of the screen). The floating UI element 703 may be the first floating element in this embodiment of the present application. The drag operation and the drag and release operation may be the second operation in the embodiment of the present application.

For another example, referring to FIG. 8 , after the user performs a screenshot operation, the generated screenshot may be displayed on the screen of the mobile phone in the form of a floating UI element.

Wherein, the user triggers the operation of the mobile phone to take a screenshot, such as called a screenshot operation, which can be an operation on a screenshot button (such as a click operation), or a function key on the mobile phone (such as a volume control button: volume "+" key, volume The operation of "-" key, switch button, etc.) or function combination key (for example, the combination of volume "+" key and switch button), can also be input preset gestures, such as knuckle tapping, etc. The content to be intercepted may be all the content currently displayed on the mobile phone screen, or part of the content. If the content to be captured is part of the content currently displayed on the screen of the mobile phone, the above screenshot operation may also include the operation of selecting the capture area by drawing a circle or selecting a frame. After the mobile phone receives the screenshot operation, the mobile phone can start the screenshot function to capture all or part of the content currently displayed on the mobile phone screen, so as to generate a screenshot. After the screenshot is generated, the mobile phone can display the generated screenshot on the screen of the mobile phone in the form of a floating UI element.

For example, with reference to FIG. 8 , take the screenshot operation as an example of an operation on the screenshot button. As shown in (a) of FIG. 8 , the mobile phone displays a desktop 801 . When the user wants to use the screenshot function, he can perform a sliding operation starting from the upper edge of the mobile phone screen and pointing to the lower edge. The mobile phone can receive this swipe operation. As a response to the sliding operation, as shown in (b) in FIG. 8 , the mobile phone displays a menu bar 802 including a screenshot button 803 . The user can perform operations on the screenshot button 803, such as clicking. After the mobile phone receives the user's click operation on the screenshot button 803, it can start the screenshot function to capture the content currently displayed on the desktop 801 on the mobile phone screen, thereby generating a screenshot. After the screenshot is generated, the mobile phone can display the generated screenshot on the screen of the mobile phone in the form of a floating UI element. As an example, the mobile phone can first display the generated screenshot on the screen of the mobile phone in the form of a floating window, and then the screenshot can be changed into a floating UI element with a gradually shrinking dynamic effect (the floating UI element shown in (d) in Figure 8 Element 805) is displayed on the edge of the mobile phone screen after adsorption. As yet another example, as shown in (c) in FIG. 8 , the mobile phone may first display the generated screenshot on the screen of the mobile phone in the form of a floating window, such as display screenshot 804 . The user can press the screenshot 804 and move the finger (for example, move the finger to the right), so that the screenshot 804 moves on the screen of the mobile phone following the movement of the user's finger. Afterwards, the user can lift the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag and release operation. After the mobile phone receives the dragging and releasing operation, as shown in (d) in FIG. , that is, the generated screenshot is displayed on the mobile phone screen in the form of floating UI elements. For example, the floating UI element 805 can be adsorbed and displayed on the edge of the screen of the mobile phone (such as the left edge or the right edge of the screen, (d) in FIG. 8 is shown as an example of adsorbing and displaying on the right edge of the screen).

It should be noted that the above embodiment shown in FIG. 7 is described by taking the image element as an image in a gallery application as an example. The picture element can also be pictures contained in the pages of other applications of the mobile phone, such as the pictures shared with friends or friends in the chat interface of the instant messaging application, and the pictures contained in the pages of the browser, etc. The embodiments are not specifically limited here either.

In addition, the specific implementation process of displaying picture elements on the screen of the mobile phone as floating UI elements is similar to the specific implementation process of displaying text elements on the screen of the mobile phone as floating UI elements shown in FIG. 6 , and will not be repeated here. The difference is that the drag event includes the dragged picture and the type of the dragged element (such as picture class), and the element control module of the mobile phone application layer can generate a shadow according to the picture.

Using the technical solution of the embodiment shown in Figure 7 or Figure 8 above, for picture interface elements, the user can display the corresponding picture on the screen of the mobile phone in the form of a floating UI element by performing a drag operation or a screenshot operation, such as adsorption Displayed at the edge of the terminal screen. And after receiving the user's operation of switching pages in the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the mobile phone screen, making it more convenient to view or share information across applications. For example, after the mobile phone receives the user's click operation on the floating UI element, the mobile phone can call the application selector of the system for the user to select which application to use to view the picture displayed in the form of the floating UI element. For example, after the mobile phone displays a picture in the form of a floating UI element on its screen, the mobile phone (such as the frame layer of the mobile phone) can store the identification of the floating UI element in association with the picture in the mobile phone. After the user selects an application for viewing pictures, the application can display the pictures in the floating UI elements to the user through the application according to the stored association relationship. And floating UI elements can be moved. For another example, when the user wants to use the picture across pages or applications, such as sharing the picture with a friend, there is no need to open the gallery application again to view it. Select, drag and drop the floating UI element to the chat interface with the friend In the input box, the picture in the floating UI element can be shared with friends, etc., which improves the interaction efficiency. The specific implementation is similar to the implementation of filling the text displayed in the form of floating UI elements into the input box, and will not be described in detail here.

With reference to FIG. 9 , an example is given to illustrate the process of displaying a uniform resource locator (uniform resource locator, URL) website on a screen of a mobile phone in the form of a floating UI element.

Wherein, the process of displaying the URL on the screen of the mobile phone in the form of floating UI elements is similar to the process of displaying the text elements on the screen of the mobile phone in the form of floating UI elements, and the detailed process will not be described again. Here, an example is used to illustrate the process of displaying the URL on the screen of the mobile phone in the form of floating UI elements. Wherein, similar to the description of the embodiment shown in FIG. 5A and FIG. 5B , the operation that triggers the URL to be displayed in the form of a floating UI element may be a user's drag operation on the URL, or a copy operation on the URL. For example, referring to FIG. 9 , it is taken as an example that the operation of triggering the display of the URL in the form of a floating UI element is a copy operation of the URL. As shown in (a) in FIG. 9 , the mobile phone currently displays a webpage 901 . When the user wants to display the URL of the webpage 901 on the screen of the mobile phone in the form of a floating UI element, the user may perform a copy operation on the URL. After receiving the copy operation, the mobile phone can display the URL on the screen of the mobile phone in the form of a floating UI element. The copying operation may be the second operation in the embodiment of the present application.

Wherein, the copy operation for the URL may include one or more operations. Exemplarily, the copy operation for the URL address may include an operation of long pressing the address bar (or the URL address in the address bar) and an operation of a copy button. Continuing to refer to (a) in FIG. 9 , the user can press and hold the URL bar 902 in the web page 901 . The web page 901 may be the first interface in this embodiment of the application. The URL in the address column 902 may be the first element in this embodiment of the application. As shown in (b) of FIG. 9 , in response to the user's long press operation on the address bar 902 , the mobile phone can display an operation menu, which includes a copy button 903 . After the mobile phone receives the user's operation on the copy button 903, such as a click operation, as shown in (c) in FIG. 9 , the mobile phone can display a floating UI element 904 on the mobile phone screen, and the floating UI element 904 includes a link icon. The floating UI element 904 may be the first floating element in this embodiment of the present application. Wherein, the link icon is used to link to the web page corresponding to the URL in the address column 902, that is, the URL copied by the user is displayed on the screen of the mobile phone in the form of a floating UI element.

It should be noted that the specific implementation process of URL URLs being presented as floating UI elements on the mobile phone screen is similar to the specific implementation process of text elements as shown in Figure 6 being presented as floating UI elements on the mobile phone screen, and will not be repeated here. . The difference is that the drag event includes the URL and the element type presented as a floating UI element is URL, and the element control module of the mobile phone application layer can use the link icon as a shadow.

Using the technical solution of the embodiment shown in Figure 9 above, for the URL, the user can display the URL in the form of a floating UI element on the screen of the mobile phone by performing a drag operation or a copy operation, such as being adsorbed and displayed on the edge of the terminal screen. And after receiving the user's operation of switching pages in the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the mobile phone screen, making it more convenient to view or share information across applications. For example, after the mobile phone receives the user's click operation on the floating UI element, the mobile phone can jump to the webpage of the URL address presented by the floating UI element. For example, after the mobile phone displays the URL in the form of a floating UI element on its screen, the mobile phone (such as the frame layer of the mobile phone) can associate and store the identifier of the floating UI element with the URL in the mobile phone. After the user clicks the floating UI element, the mobile phone can open and display the URL corresponding to the identifier of the floating UI element to the user according to the stored association relationship. The application may be a default application set by the user, or an application automatically selected by the system. And floating UI elements can be moved. For another example, when the user wants to use the URL across pages or applications, such as sharing the URL with a friend, the floating UI element can be dragged and dropped into the input box of the chat interface with the friend, and the Paste the URL in the floating UI element into the input box to share with friends, etc. The specific implementation is similar to the implementation of filling the text displayed in the form of floating UI elements into the input box, and will not be described in detail here.

With reference to FIG. 10 , an example is given to illustrate the process of displaying file resource elements on the mobile phone screen in the form of floating UI elements.

Wherein, the file resource element may refer to pictures, documents, compressed packages and other types of file resources stored in the form of files in the mobile phone system. The operation that triggers the display of the file resource element in the form of a floating UI element may be a user's drag operation on the file resource element.

For example, as shown in (a) in FIG. 10 , the mobile phone displays a file resource management page 1001 . The resource management page 1001 may be the first interface in this embodiment of the application. When the user wants to display one or more file resources in the file resource management page 1001 on the screen of the mobile phone in the form of floating UI elements, the user can perform a drag operation on the file resources. After the user releases the drag, the mobile phone can display the file resource on the screen of the mobile phone in the form of a floating UI element.

Wherein, the above-mentioned drag operation may include one or more operations. Exemplarily, the drag operation may include an operation of triggering the floating of the file resource and an operation of triggering the movement of the floating file resource.

For example, an operation that triggers the floating of a file resource is an operation of long pressing and selecting a file resource, and an operation that triggers movement of a floating file resource is an operation of pressing and moving a floating file resource. Continuing to refer to (a) in FIG. 10 , the user can click the selection button 1002 on the file resource management page 1001 . In response to this operation, as shown in (b) in FIG. 10 , the mobile phone enters a mode of selecting file resources in the file resource management page 1001 . Users can select file resources in this selection mode. If the mobile phone receives the user's selection operation on the file resource "zyy" 1003, the file resource "zyy" 1003 is in the selected state. The file resource "zyy" 1003 may be the first element in this embodiment of the application. The user can long press the selected file resource “zyy” 1003 . In response to the user's operation, the mobile phone may display the file resource "zyy" 1003 as a floating state. For example, as shown in (c) in FIG. 10 , the mobile phone can perform background blur processing and display a floating file resource "zyy" 1003 . Afterwards, the user presses the floating file resource "zyy" 1003 and moves the finger (for example, moves the finger to the right), so that the file resource "zyy" 1003 can follow the movement of the user's finger on the screen of the mobile phone. Afterwards, the user can lift the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag and release operation. After the mobile phone receives the dragging and releasing operation, as shown in (d) in FIG. 10 , the mobile phone can display a floating UI element 1004 on the mobile phone screen, and the floating UI element 1004 includes a link icon. Among them, the link icon is used to link to the storage address corresponding to the file resource "zyy" 1003 in the mobile phone, that is, the file resource dragged by the user is displayed on the screen of the mobile phone in the form of floating UI elements. For example, the floating UI element 1004 can be displayed on the edge of the screen of the mobile phone (such as the left edge or the right edge of the screen, (d) in FIG. 10 is shown as an example of being displayed on the right edge of the screen). The floating UI element 1004 may be the first floating element in this embodiment of the application. The drag operation and the drag and release operation may be the second operation in the embodiment of the present application.

It should be noted that the specific implementation process of file resource elements presented as floating UI elements on the screen of the mobile phone is similar to the specific implementation process of text elements presented as floating UI elements on the screen of the mobile phone shown in Figure 6, and will not be repeated here. A repeat. The difference is that the drag event includes the storage address of the file resource in the mobile phone and the element type presented as a floating UI element is a file resource class, and the element control module of the mobile phone application layer can use the link icon as a shadow.

Similarly, using the technical solution of the embodiment shown in Figure 10 above, for the file resource element, the user can display the file resource element on the screen of the mobile phone in the form of a floating UI element by performing a drag and drop operation. Terminal screen edge. And after receiving the user's operation of switching pages in the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the mobile phone screen, making it more convenient to view or share information across applications. For example, when the user wants to use the file resource across pages or applications, such as sharing the file resource with a friend, by dragging the floating UI element into the input box of the chat interface with the friend, the floating UI element can be File resources corresponding to UI elements are shared with friends. The specific implementation is similar to the implementation of filling the text displayed in the form of floating UI elements into the input box, and will not be described in detail here.

With reference to FIG. 11 , an example is given to illustrate the process of displaying list elements on the mobile phone screen in the form of floating UI elements.

Among them, the list element is a native control type of Android ^TM , such as ListView, RecyclerView, etc. are all list elements. Specifically, it may be a list element in applications such as social networking, communication, and shopping. The operation that triggers the display of list elements in the form of floating UI elements may be the user's drag operation on elements included in the list items in the list, such as an element in the list item representing the user's avatar. For example, as shown in (a) in FIG. 11 , the mobile phone currently displays the home page 1101 of WeChat ^TM . The homepage 1101 of ^WeChatTM may be the first interface in the embodiment of this application. When the user wants to display list elements in the home page 1101 of ^WeChatTM , such as chat list items with one or more friends, on the screen of the mobile phone in the form of floating UI elements, the user can drag and drop the avatars of the contacts in the list items operate. After the user releases the drag, the mobile phone can display the contact avatar on the mobile phone screen in the form of a floating UI element.

Wherein, the above-mentioned drag operation may include one or more operations. Exemplarily, the dragging operation may include an operation of triggering the avatar of the contact to float up and an operation of triggering the movement of the avatar of the floating contact.

For example, an operation that triggers the avatar of a contact to float up is a long press operation on the avatar of the contact, and an operation that triggers the avatar of the floating contact to move is an operation of pressing and moving the avatar of the floating contact as an example. Continuing to refer to (a) shown in FIG. 11 , the user can long press the avatar 1102 of the friend BB in the home page 1101 of WeChat ^TM . The contact avatar 11102 may be the first element in this embodiment of the application. The mobile phone may receive the user's long press operation on the contact's profile picture 1102. In response to the long press operation, the mobile phone can display the contact portrait 1102 in a floating state. For example, as shown in (b) in FIG. 11 , the mobile phone can perform background blur processing and display a floating contact portrait 1102 . Afterwards, by pressing the floating contact portrait 1102 and moving the finger (for example, moving the finger to the right), the contact portrait 1102 can be moved on the screen of the mobile phone following the movement of the user's finger. Afterwards, the user can lift the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag and release operation. After the mobile phone receives the dragging and releasing operation, as shown in (c) in Figure 11, the mobile phone can display a floating UI element 1103 on the mobile phone screen. The avatar of the contact in the list item of is displayed on the screen of the mobile phone in the form of a floating UI element. The floating UI element 1103 may be the first floating element in this embodiment of the application. The drag operation and the drag and release operation may be the second operation in the embodiment of the present application.

It should be noted that the above embodiment shown in FIG. 11 is described by taking list elements as list items in ^WeChatTM as an example. The list element may also be a list item in other application interfaces, such as a list item in an address book, or a list item in a short message, etc., and this embodiment does not make specific limitations here.

Combined with Fig. 4, as shown in Fig. 12, as an example, the specific implementation process of presenting list elements as floating UI elements on the screen of the mobile phone may be as follows: the mobile phone, for example, the application layer of the mobile phone receives the user's request for a certain item in the list item. After the element is dragged and dropped, the dragging of the element can be triggered. The application in the mobile application layer can obtain the drag event and send it to the drag service module in the mobile application layer. For example, the process for the application layer of the mobile phone to obtain the drag event can be: after the user performs the drag operation, the framework layer of the mobile phone can monitor the drag operation (for example, it can be a long press operation on the floating contact avatar) , as a response, the frame layer of the mobile phone can generate a drag event (DragEvent). Applications in the mobile phone application layer can obtain drag events generated by the framework layer through callbacks. Wherein, the drag event may include the coordinates of the pressing point when the user performs a long press operation and the type of the dragged element (such as a list class). After the drag service module of the mobile application layer receives the drag event, it can cooperate with the element control module of the mobile application layer to construct a shadow according to the coordinates of the pressed point in the drag event and the type of the dragged element (such as a list class). ). If the drag service module of the mobile phone application layer determines that the type of the dragged element is a list, the drag service module of the mobile phone application layer can transmit the coordinates of the pressed point in the drag event to the window manager of the mobile phone frame layer. The window manager can determine the control within the preset range of the pressed point according to the coordinates of the pressed point, and capture a screenshot of the control and transmit it to the element control module of the mobile phone application layer, so that the element control module can generate a shadow according to the screenshot. Wherein, the preset range of the pressing point may be within a distance of predetermined pixels (such as 1 pixel, or greater than 1 pixel) centered on the pressing point. Afterwards, the element control module of the mobile phone application layer can transmit the generated shadow to the window manager of the mobile phone frame layer, so that the window manager can display the shadow in the form of floating UI elements. For example, the window manager displays a circular element on the screen of the mobile phone, and the circular element is a floating UI element, which includes a screenshot of the control pressed by the user, such as the avatar of the contact.

Using the technical solution of the embodiment shown in FIG. 11 above, for list interface elements, the user can display the list items, such as the elements included in the list items, on the screen of the mobile phone in the form of floating UI elements by performing a drag operation. That is, the elements in the list item dragged by the user can be presented on the screen of the mobile phone through the floating UI element. And after receiving the user's operation of switching pages in the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the screen of the mobile phone. The floating UI element supports the operations supported by the original list item when it is clicked. That is, the user performs a corresponding operation (this operation may be the fifth operation in the embodiment of the present application), such as the response made by the mobile phone after clicking the floating UI element (or the function executed, which may be the fifth operation in the embodiment of the application) first function), which is the same as the phone's response when the user taps the original list item. Its specific implementation can be: after the user performs a drag operation on an element in the list item, such as a contact's avatar, the application of the mobile phone application layer can send the intent (intent) that needs to be triggered after clicking the contact's avatar to the mobile phone application layer After the mobile phone displays the avatar of the contact in the form of a floating UI element on its screen, the mobile phone can associate and store the identity of the floating UI element and the above-mentioned intent that needs to be triggered in the mobile phone. Afterwards, after the mobile phone receives the user's operation on the floating UI element, such as a click operation, the mobile phone can call the corresponding intent that needs to be triggered to make a corresponding response.

For example, in combination with the example shown in Figure 11, the user triggers the mobile phone to exit the WeChat ^TM application by dragging the contact avatar of friend BB to trigger it to be displayed on the screen of the mobile phone in the form of a floating UI element, as shown in Figure 13A As shown in (a), the mobile phone displays a desktop 1301 . The contact avatar of friend BB continues to be displayed on the screen of the mobile phone in the form of a floating UI element, such as the floating UI element 1302 shown in (a) in FIG. 13A . In addition, the mobile phone associates and stores the identifier of the floating UI element 1302 with the intent that needs to be triggered in the mobile phone. If the intent that needs to be triggered is to display the chat interface with the friend BB. After the mobile phone receives the user's operation on the floating UI element 1302, such as a click operation, as a response to the operation, as shown in (b) in FIG. 13A, the mobile phone displays a chat interface 1303 with the friend BB.

For another example, referring to FIG. 14 , as shown in (a) in FIG. 14 , the mobile phone displays an address book interface 1401 . After the mobile phone receives the user's drag operation on the avatar 1402 of the contact Sam in the address book interface 1401, as shown in (b) in Figure 14, the mobile phone displays a floating UI element 1403, which includes the contact Sam The avatar 1402 of the user, that is, the avatar 1402 of the contact Sam in the address book interface 1401 dragged by the user is displayed on the screen of the mobile phone in the form of a floating UI element. In addition, the mobile phone associates and stores the identifier of the floating UI element 1403 with the intent that needs to be triggered in the mobile phone. If the intent to be triggered is to make a call to the corresponding contact. As shown in (c) in FIG. 14 , the user triggers the mobile phone to exit the address book, and the mobile phone displays a desktop 1404 . The floating UI element 1403 continues to be displayed on the phone screen. Afterwards, after the mobile phone receives the user's operation on the floating UI element 1403, such as a click operation, as a response to the operation, the mobile phone can make a call to the contact Sam. Also as shown in (d) in FIG. 14 , the mobile phone can also display a dial interface 1405 .

In addition, for a contact avatar displayed in the form of a floating UI element, when content such as a picture, URL or text is dragged onto the floating UI element, the mobile phone can directly send the content to the corresponding contact. Wherein, the operation of dragging the content may include an operation of triggering the floating of the content and an operation of triggering the movement of the floating content. Take the operation that triggers the floating of the content as a long press operation on the content, and the operation that triggers the movement of the floating content as an operation of pressing and moving the floating content as an example. For example, referring to the example shown in FIG. 11 , after the user drags the contact avatar of the friend BB to trigger it to be displayed on the screen of the mobile phone in the form of a floating UI element, the user opens the gallery application. As shown in (a) of FIG. 13B , the mobile phone displays an interface 1304 of the gallery application. The contact avatar of the friend BB continues to be displayed on the screen of the mobile phone in the form of a floating UI element, such as the floating UI element 1305 shown in (a) in FIG. 13B . After the mobile phone receives the user's long press operation on the picture 1306 in the interface 1304 of the gallery application, as shown in (b) in FIG. 13B , the mobile phone floats the picture 1306 for display. Afterwards, the mobile phone receives the user's operation of pressing and moving the floating picture 1306, and the mobile phone displays the moving animation of the picture 1306. If the mobile phone receives the operation of releasing the drag when the user moves the picture 1306 to the position of the floating UI element 1305, the mobile phone sends the picture 1306 to the friend BB. In addition, as shown in (c) in FIG. 13B , the mobile phone can also automatically open the chat interface 1307 with the friend BB, and display a message of successful sending. Among them, the specific implementation of the mobile phone sending the picture to the friend may be that the mobile phone associates and stores the identifier of the floating UI element with the intent that needs to be triggered (for example, the intent that needs to be triggered is to display the chat interface with the friend BB) in the mobile phone After receiving the user's operation of dragging content (such as picture 1306 ) to the position of the floating UI element and releasing the drag, the mobile phone can call the corresponding intent that needs to be triggered to start the chat interface. The mobile phone can also construct material data (ClipData), and store the content dragged and dropped by the user into the ClipData. Afterwards, the mobile phone releases the ClipData in the started chat interface, so that the application will process the ClipData, that is, send the content in the ClipData to the corresponding contact friend. It should be noted that, the example shown in FIG. 13B is illustrated by dragging the picture directly onto the avatar of the contact presented as a floating UI element, which can trigger the mobile phone to directly send the picture to the corresponding contact as an example. It can be understood that, in this embodiment, when the user drags text, pictures, URLs, files, etc. to the floating UI element that can respond to the sending event (such as the above-mentioned contact avatar presented by the floating UI element) and releases it, it will trigger Send operation, which can improve the interaction efficiency of common operations such as sharing and sending content.

The above example uses the mobile phone to associate and store the identifier of the floating UI element with the intent that needs to be triggered in the mobile phone to trigger the mobile phone to make a corresponding response after receiving the user's operation on the floating UI element. In some other embodiments, the mobile phone may not associate and store the identifier of the floating UI element with the intent that needs to be triggered in the mobile phone, but after receiving the user's operation on the floating UI element, send the corresponding operation event to the application , to trigger the app to make a corresponding response according to the operation event. For example, after receiving the user's operation on the floating UI element, the mobile phone may determine whether an intent corresponding to the identifier of the floating UI element is stored. If the intent corresponding to the identifier of the floating UI element is stored, the mobile phone can activate the intent to respond to the user's operation. If it is not stored and the application is still displayed in the foreground, the mobile phone can transmit the corresponding operation event to the main screen, so as to trigger the application to make a corresponding response according to the operation event. If the application retreats to the background, the mobile phone can move the interface of the application to the virtual screen for drawing. After determining that the intent corresponding to the identifier of the floating UI element is not stored, the mobile phone may send a corresponding operation event to the virtual screen on which the application interface is drawn, so as to trigger the application to make a corresponding response according to the operation event.

With reference to FIG. 15 , an example is given to illustrate the process of displaying icon elements on the mobile phone screen in the form of floating UI elements.

Wherein, the icon-type element may refer to a graphic element in an interface for which a click event is defined. The operation that triggers the icon-type element to be displayed in the form of a floating UI element may be a user's drag operation on the icon-type element.

For example, as shown in (a) in FIG. 15 , the mobile phone displays an interface 1501 of Taobao ^™ . The Taobao ^TM interface 1501 may be the first interface in the embodiment of this application. The interface 1501 includes multiple controls. When the user wants to display one or more controls in the interface 1501 of Taobao ^TM on the mobile phone screen in the form of floating UI elements, he can perform a drag operation on the controls. After the user releases the drag, the mobile phone can display the control on the mobile phone screen in the form of a floating UI element. Wherein, the above-mentioned drag operation may include one or more operations. Exemplarily, the drag operation may include an operation of triggering the floating of the control and an operation of triggering the movement of the floating control.

For example, an operation that triggers the floating of the control is a long press operation on the control, and an operation that triggers the movement of the floating control is an operation of pressing and moving the floating control as an example. Continuing to refer to (a) in FIG. 15 , the user can perform a long-press operation on the control 1502 to be received in the interface 1501 of ^TaobaoTM . The goods to be received control 1502 may be the first element in this embodiment of the application. In response to the long press operation, the mobile phone may display the control 1502 to be received in a floating state. For example, as shown in (b) in FIG. 15 , the mobile phone can perform background blur processing and display a floating control 1502 to be received. By pressing the floating control 1502 and moving the finger (such as moving the finger to the right), the user can make the control 1502 move on the screen of the mobile phone following the movement of the user's finger. Afterwards, the user can lift the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag and release operation. After the mobile phone receives the dragging and releasing operation, as shown in (c) in FIG. 15 , the mobile phone can display a floating UI element 1503 on the mobile phone screen, and the floating UI element 1503 includes an icon corresponding to the control to be received 1502, That is, the control dragged by the user is displayed on the screen of the mobile phone in the form of a floating UI element. The floating UI element 1503 may be the first floating element in this embodiment of the application. The drag operation and the drag and release operation may be the second operation in the embodiment of the present application.

It should be noted that the specific implementation process of icon elements presented as floating UI elements on the mobile phone screen is similar to the specific implementation process of list elements presented as floating UI elements on the mobile phone screen shown in Figure 12. repeat. The difference is that the mobile phone receives the user's drag operation on the control, for example, after a long press operation, the drag can be triggered. The type of the dragged element in the drag event is icon class.

It can be understood that, with the technical solution of the above embodiment shown in FIG. 15 , for icon elements, the user can display the icon elements on the screen of the mobile phone in the form of floating UI elements by performing a drag operation. And after receiving the user's operation of switching pages in the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the screen of the mobile phone. The floating UI element supports the operations supported by the control after it is clicked. That is, the user performs a corresponding operation (this operation may be the fifth operation in the embodiment of the present application), such as the response made by the mobile phone after clicking the floating UI element (or the function executed, which may be the fifth operation in the embodiment of the application) The first function), which is the same response as the user clicks on the original control phone. The specific implementation can be: if the application is still displayed in the foreground, after receiving the user's operation on the floating UI element, the mobile phone can transmit the corresponding operation event to the main screen, so as to trigger the application to make a corresponding response according to the operation event. If the application retreats to the background, the mobile phone can move the application to the virtual screen for drawing. After receiving the user's operation on the floating UI element, the mobile phone can send the corresponding operation event to the virtual screen on which the application interface is drawn, so as to trigger the application to make a corresponding response according to the operation event. That is to say, referring to the example shown in FIG. 15 , after the mobile phone receives the user's operation on the floating UI element 1503 , such as a click operation, the mobile phone can display the detailed interface of the goods to be received.

With reference to FIG. 16 and FIG. 17 , the process of displaying card elements in the form of floating UI elements on the screen of the mobile phone is described by way of example.

Wherein, the card element may refer to an element for displaying information. Which elements in the interface are card-type elements may be pre-defined. For example, the corresponding configuration file may be pre-stored in the mobile phone, and the configuration file includes the identification of the card element. For example, it can be elements in navigation and shopping applications. In some embodiments, the operation that triggers the card element to be displayed as a floating UI element may be a user's drag operation on the card element.

For example, as shown in (a) in FIG. 16 , the mobile phone displays a navigation interface 1601 . The navigation interface 1601 may be the first interface in this embodiment of the application. When the user wants to display one or more elements in the navigation interface 1601 on the screen of the mobile phone in the form of floating UI elements, the user can perform a drag operation on the elements. After the user releases the drag, the mobile phone can display the element on the screen of the mobile phone in the form of a floating UI element. Wherein, the above-mentioned drag operation may include one or more operations. Exemplarily, the dragging operation may include an operation of triggering the floating of the element and an operation of triggering the movement of the floating element.

For example, an operation that triggers the floating of an element is a long press operation on the display area where the element is located, and an operation that triggers the movement of the floating element is an operation of pressing and moving the floating element. Continuing to refer to (a) in FIG. 16 , the user can perform a long press operation on the display area where the element 1602 in the navigation interface 1601 is located. Element 1602 may be the first element in this embodiment of the application. In response to the long press operation, the mobile phone can extract some card elements in the navigation interface 1601 according to the configuration file, for example, extract card elements within a preset range of the pressing point and display the extracted card elements in a floating state. For example, as shown in (a) in FIG. 16 , the mobile phone extracts elements 1602 and 1603 in the navigation interface 1601 . Element 1603 may be the second element in this embodiment of the application. As shown in (b) of FIG. 16 , the mobile phone performs background blurring and displays a floating element 1604 , which includes the extracted element 1602 and element 1603 . By pressing the floating element 1604 and moving the finger (such as moving the finger to the right), the user can make the element 1604 move on the screen of the mobile phone following the movement of the user's finger. Afterwards, the user can lift the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag and release operation. After the mobile phone receives the drag-and-drop operation, as shown in (c) in Figure 16, the mobile phone can display a floating UI element 1605 on the mobile phone screen, and the floating UI element 1605 includes the extracted element 1602 and element 1603, namely The elements dragged by the user are displayed on the screen of the mobile phone in the form of floating UI elements. The floating UI element 1605 may be the first floating element in this embodiment of the application. Wherein, when the extracted element is displayed in the form of a floating UI element, its layout may be the same as or different from the layout of the element in the interface. When the layout of the extracted elements displayed in the form of floating UI elements is the same as that in the interface, after the mobile phone extracts the elements, it can arrange the extracted elements according to their layout in the current interface, Displayed on the phone screen as a floating UI element. When the layout of the extracted elements displayed in the form of floating UI elements is different from the layout in the interface, as an example, a corresponding layout file may be pre-stored in the mobile phone, and the layout file includes the extracted elements , such as the layout configuration including element 1602 and element 1603 . After the mobile phone extracts the elements, such as the above-mentioned element 1602 and element 1603, the element 1602 and element 1603 can be laid out according to the layout file, and then displayed in the form of floating UI elements, such as the display effect shown in (c) in Figure 16 The floating UI element 1605 of . The drag operation and the drag and release operation may be the second operation in the embodiment of the present application.

It should be noted that, in the example shown in Figure 16, the specific implementation process of the card element being presented as a floating UI element on the screen of the mobile phone is similar to the specific implementation process of the list element shown in Figure 12 being presented as a floating UI element on the screen of the mobile phone , which will not be repeated here. The difference is that the mobile phone may trigger the drag operation after receiving the user's drag operation on the element, for example, after receiving a long press operation on the display area where the element is located. The type of the dragged element in the drag event is a card class. In addition, during specific implementation, the mobile phone may use the card elements within the preset range of the pressed point as elements subsequently presented as floating UI elements according to the coordinates of the user's pressed point.

In some other embodiments, the operation that triggers the display of the card element in the form of a floating UI element may be the user's operation on a predetermined button in the interface.

For example, as shown in (a) in FIG. 17 , the mobile phone displays a shopping details interface 1701 of a shopping application. When the user wants to display one or more elements in the shopping details interface 1701 on the mobile phone screen in the form of floating UI elements, he can operate (eg, click) a predetermined button, such as the share button 1706 . After the mobile phone receives the user's click operation on the sharing button, it can display some elements in the shopping details interface 1701, such as card elements, on the screen of the mobile phone in the form of floating UI elements.

For example, continuing to refer to (a) in FIG. 17 , the user operates, such as clicking, the share button 1706 in the shopping details interface 1701 of the shopping application. In response to the click operation, the mobile phone can extract card elements in the shopping details interface 1701 , such as item preview 1702 , item price 1707 , item profile 1703 , add to cart button 1704 and buy now button 1705 . Afterwards, as shown in (b) in Figure 17, the mobile phone can display a floating UI element 1708 on the mobile phone screen, which includes the extracted product preview 1702, product price 1707, product introduction 1703, and a button to add to the shopping cart 1704 and a buy now button 1705.

Wherein, after the mobile phone receives the user's operation on the predetermined button in the interface, it can extract the card elements in the interface according to the corresponding configuration file, so that it can be presented to the user in the form of floating UI elements. The configuration file defines which elements in the interface will be presented in the form of floating UI elements after receiving the user's operation on the predetermined button.

It can be understood that, by adopting the technical solutions of the embodiments shown in Figure 16 and Figure 17 above, for card-type elements, the user can display the card-type elements in the form of floating UI elements by performing a drag operation or operating a predetermined button on the phone screen. And after receiving the user's operation of switching pages in the application, or exiting the application, or opening other applications, the floating UI element can continue to be displayed on the mobile phone screen, which is convenient for the user to continue viewing. It also makes it easier to share information across applications. For example, when the user wants to share the card element with a friend, by dragging the floating UI element (such as the floating UI element 1708 shown in (b) in Figure 17) into the input box of the chat interface with the friend , you can share the floating UI element with your friends. The specific implementation is similar to the implementation of filling the text displayed in the form of floating UI elements into the input box and sharing it with friends, and will not be described in detail here. In addition, after receiving the floating UI element, the friend can click on the received floating UI element in the chat interface displayed on the mobile phone to trigger the mobile phone to display the corresponding shopping details interface.

With reference to FIG. 18 , an example is given to illustrate the process of displaying video window elements on the mobile phone screen in the form of floating UI elements.

Among them, the video window element is a native control type of Android ^TM , such as TextureView, SurfaceView, or video elements in webview are all video window elements. The operation that triggers the display of the video window type element in the form of a floating UI element may be a user's drag operation on the video window type element.

For example, as shown in (a) in FIG. 18 , the mobile phone displays a video detail interface 1801 . The video details interface 1801 may be the first interface in this embodiment of the application. When the user wants to display the video element in the video detail interface 1801 on the screen of the mobile phone in the form of a floating UI element, the user can perform a drag operation on the video element. After the user releases the drag, the mobile phone can display the video element on the screen of the mobile phone in the form of a floating UI element. Wherein, the above-mentioned drag operation may include one or more operations. Exemplarily, the drag operation may include an operation of triggering the floating of the video element and an operation of triggering the movement of the floating video element.

For example, an operation that triggers the floating of the video element is a long press operation on the video element, and an operation that triggers the movement of the floating video element is an operation of pressing and moving the floating video element. Continuing to refer to (a) in FIG. 18 , the user can perform a long press operation on the video element 1802 in the video detail interface 1801 . The video element 1802 may be the first element in this embodiment of the application. In response to the long press operation, the mobile phone may display the video element 1802 in a floating state. For example, as shown in (b) in FIG. 18 , the mobile phone can perform background blur processing and display a floating video element 1802 . By pressing the floating video element 1802 and moving the finger (for example, moving the finger to the right), the user can make the video element 1802 move on the screen of the mobile phone following the movement of the user's finger. Afterwards, the user can lift the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag and release operation. After the mobile phone receives the drag and release operation, as shown in (c) in Figure 18, the mobile phone can display a floating UI element 1803 on the mobile phone screen, and the floating UI element 1803 includes a video element 1802, which is dragged by the user. The video element is displayed on the phone screen as a floating UI element. The floating UI element 1803 may be the first floating element in this embodiment of the application. The drag operation and the drag and release operation may be the second operation in the embodiment of the present application.

Among them, the specific implementation process for the video window elements to be presented as floating UI elements on the screen of the mobile phone can be as follows: the mobile phone, such as the application layer of the mobile phone, receives the user's drag operation on the video window element, and can trigger the video window for the video window. Drag and drop of class elements. The application in the mobile application layer can obtain the drag event and send it to the drag service module in the mobile application layer. For example, the process for the application layer of the mobile phone to obtain the drag event can be: after the user performs the drag operation, the frame layer of the mobile phone can monitor the drag operation (for example, it can be a long-press operation on the video window element), In response, the framework layer of the mobile phone may generate a drag event (DragEvent). Applications in the mobile phone application layer can obtain drag events generated by the framework layer through callbacks. Wherein, the drag event may include the coordinates of the pressed point, the identification of the application (such as the package name), the identification of the video window class element and the type of the dragged element (such as the video window class). The drag-and-drop service module of the mobile phone application layer can start the virtual screen, and transmit the identification of the virtual screen and the package name of the application to the window manager of the mobile phone frame layer. According to the identification of the virtual screen and the package name of the application, the window manager of the mobile phone frame layer can move the interface of the application indicated by the package name, that is, the above-mentioned video detail interface, to the virtual screen for drawing. The drag service module of the mobile application layer can also cooperate with the element control module of the mobile application layer to construct a shadow according to the drag event. If the drag service module of the mobile phone application layer determines that the type of the dragged element is a video window, the drag service module of the mobile phone application layer can transmit the coordinates of the pressed point in the drag event to the window manager of the mobile phone frame layer. The window manager can determine the elements near the pressed point according to the coordinates of the pressed point, and take a screenshot of the element and transmit it to the element control module of the mobile phone application layer, so that the element control module can generate a shadow according to the screenshot, and transmit the generated shadow to the mobile phone The window manager of the framework layer, so that the window manager can add the shadow to the main screen of the mobile phone. Later, after the user releases the drag, the drag service module of the mobile phone application layer can extract the corresponding video window elements from the interface drawn on the virtual screen according to the identification of the video window elements in the drag event, such as the video element , and map the extracted video window elements to the shadow of the main screen, that is, to achieve the effect that the video window elements are displayed in the form of floating UI elements.

Using the technical solution of the embodiment shown in Figure 18 above, for the video window element, the user can display the video window element on the screen of the mobile phone in the form of a floating UI element by performing a drag operation. And after receiving the operation that the user switches pages in the application, or exits the application (display desktop 1804 as shown in (d) in FIG. 18 ), or opens other applications, the floating UI element (as shown in FIG. 18 The floating UI element 1803 shown in (d) can continue to be displayed on the screen of the mobile phone for the user to continue viewing. In this way, it is convenient for the user to operate other interfaces of the mobile phone while continuing to view the video displayed by the video window element. In addition, when the video carried in the video window element is a local video, the user can drag the floating UI element (such as the floating UI element 1803 shown in (c) in Figure 18) to the chat interface with the friend In the input box, you can share the video with your friends. The specific implementation is similar to the implementation of filling the text displayed in the form of floating UI elements into the input box and sharing it with friends, and will not be described in detail here.

It can be understood that an interface may include multiple interface elements. The interface elements that support display in the form of floating UI elements in the interface may be customized by the third-party application or recommended by the system. That is to say, the third-party application or the system can define one or more interface elements in the interface to support the presentation of floating UI elements. For example, a developer of a third-party application may predefine which elements in the interface of the application support presentation in the form of floating UI elements, and configure the corresponding configuration files in the mobile phone. For another example, the system developer can configure a corresponding recommendation model in the mobile phone, and the recommendation model is used to identify which elements in the interface support being presented in the form of floating UI elements. Based on this, in order to facilitate the user to know which elements in the interface support to be presented in the form of floating UI elements, in this embodiment, when the mobile phone displays the interface, after receiving the corresponding user operation, the interface (the interface can be In the first interface in the embodiment of the present application), elements that support the presentation of floating UI elements are displayed in the first form, such as highlighted, and elements that do not support the presentation of floating UI elements are displayed in the second form. Wherein, the user operation may be a user's preset operation on an element in the interface, such as a long-press operation, a double-click operation, and the like. The user operation may also be the user's operation on the corresponding switch button. The user operation may be the first operation in this embodiment of the application. The switch button may be a predetermined function control in the embodiment of the present application.

For example, take the user operation as an example that the user performs a long press operation on an element in the interface. With reference to FIG. 19 , take the mobile phone displaying the Taobao ^TM interface 1901 as an example. As shown in (a) of FIG. 19 , the user can perform a long press operation on the pending payment control 1902 in the interface 1901 . After the mobile phone receives the long-press operation of the control to be paid 1902, it can determine whether the control to be paid 1902 supports being presented in the form of a floating UI element. If the pending payment control 1902 supports presentation in the form of floating UI elements, the mobile phone can identify all elements in the interface 1901 that support presentation in the form of floating UI elements according to the corresponding configuration file or recommendation model. As shown in (b) in Figure 19, the mobile phone recognizes the control to be paid 1902, the control to be shipped 1903, the control to be received 1904, the evaluation control 1905, the My Express control 1906 and the collection center control 1907 to support floating UI elements presented in the form. Afterwards, the mobile phone may highlight these elements that support being presented in the form of floating UI elements, such as in a floating state. For example, as shown in (b) in Figure 19, the mobile phone can perform background blur processing, and display the floating pending payment control 1902, pending delivery control 1903, pending delivery control 1904, evaluation control 1905, my express control 1906 and favorite center control 1907. If the pending payment control 1902 does not support being presented as a floating UI element, the mobile phone may not respond.

For another example, it is assumed that the user operation is the user's operation on the corresponding switch button. With reference to FIG. 20 , continue to take the mobile phone displaying Taobao ^TM interface 2001 as an example. As shown in (a) in FIG. 20 , when the user wants to check which elements in the interface 2001 support being presented as floating UI elements, the user can perform a sliding operation starting from the upper edge of the mobile phone screen and pointing to the lower edge. As a response to the sliding operation, as shown in (b) in FIG. 20 , the mobile phone displays a menu bar 2002, which includes a switch button 2003 for the floating UI element function. The user can operate the switch button 2003, such as clicking. After the mobile phone receives the user's click operation on the switch button 2003, the mobile phone can identify all elements in the interface 2001 that support the presentation of floating UI elements according to the corresponding configuration file or the recommended model, and will support all elements presented in the form of floating UI elements. Elements are displayed in a floated state. For example, as shown in (c) in FIG. 20, the mobile phone can perform background blur processing, and will support elements presented in the form of floating UI elements: pending payment control 2004, pending delivery control 2005, and pending delivery control 2006 , Evaluation Control 2007, My Express Control 2008 and Collection Center Control 2009 are displayed as floating.

By highlighting elements in the interface that support being presented as floating UI elements, and displaying other elements in a blurred manner, the user is prompted which elements in the interface support being presented as floating UI elements. It should be noted that, in addition to displaying the elements that support the form of floating UI elements in a floating state, the display can also be enlarged, or only the elements that support the form of floating UI elements can be displayed in a zoomed-in manner. This embodiment is here There is no specific limitation on the form of highlighting. For example, as shown in FIG. 21 , continue to take the mobile phone displaying the Taobao ^TM interface as an example. After the mobile phone receives the user's preset operation on the elements in the interface or the operation on the corresponding switch button, as a response, the mobile phone can add a dark mask to the current interface, such as 2101 in Figure 21, and supports floating UI elements. The elements presented in the form are placed on the dark mask (such as the control to be paid 2102, the control to be shipped 2103, the control to be received 2104, the evaluation control 2105, my express control 2106 and the collection center control 2107 are placed in a dark color mask), other elements are placed under the dark mask. Of course, elements presented in the form of floating UI elements are supported, and can also be enlarged for presentation. Afterwards, the user can trigger the corresponding element to be displayed on the screen of the mobile phone in the form of a floating UI element by performing a corresponding operation (such as a drag operation). After the user performs the corresponding operation (such as a drag operation) or closes the corresponding switch button, the interface can restore the initial display state.

According to the description of the above embodiments, it can be seen that by performing corresponding operations, the user can trigger the elements in the interface to be presented on the screen of the mobile phone in the form of floating UI elements. In some embodiments, only one floating UI element may be allowed to be displayed on the phone screen. In some other embodiments, multiple floating UI elements can be displayed on the mobile phone screen at the same time. When multiple floating UI elements are allowed to be displayed at the same time, the maximum number of floating UI elements allowed to be displayed at the same time (for example, the maximum number is N) can be limited. For example, whether to allow multiple floating UI elements to be displayed on the mobile phone screen at the same time can be configured by default by the system. For example, only one floating UI element is allowed to be displayed on the mobile phone screen by default. For another example, the mobile phone may provide a configuration interface for the user to choose whether to allow multiple floating UI elements to be displayed on the mobile phone screen at the same time, such as the user selects to allow multiple floating UI elements to be displayed on the mobile phone screen at the same time. In the case that the user chooses to allow multiple floating UI elements to be displayed at the same time, the user can also set the maximum number of floating UI elements that are allowed to be displayed at the same time. For another example, the user can choose whether to allow multiple floating UI elements to be displayed on the mobile phone screen at the same time. default allocation.

When multiple floating UI elements are allowed to be displayed on the mobile phone screen at the same time, the user can trigger the mobile phone to display multiple elements in the form of floating UI elements on the mobile phone screen by performing corresponding operations (such as drag operation, copy operation, etc.) superior. In some embodiments, the mobile phone can arrange the plurality of floating UI elements according to preset arrangement rules. For example, the mobile phone arranges and displays multiple floating UI elements on the screen of the mobile phone according to the order in which the floating UI elements are generated, for example, arranges and displays them on the edge of the screen of the mobile phone from top to bottom. For example, referring to FIG. 22, as shown in (a) in FIG. 22, the mobile phone displays a floating UI element 2201 on the left edge of the mobile phone screen. The user triggers the phone to generate a floating UI element 2202 . As shown in (b) of FIG. 22 , the mobile phone may arrange and display the floating UI element 2202 behind the floating UI element 2201 . The user triggers the mobile phone to generate a floating UI element 2203. As shown in (c) in FIG. That is to say, the mobile phone can display the earliest generated floating UI element at the top, and the subsequent generated floating UI elements are arranged in sequence according to the generation time.

In some embodiments, when multiple floating UI elements are allowed to be displayed on the screen of the mobile phone at the same time, and the maximum number of floating UI elements allowed to be displayed at the same time is set, if the number of floating UI elements already displayed on the screen of the mobile phone is If the maximum number has been reached, when the user triggers the mobile phone to display interface elements in the form of floating UI elements again, the mobile phone can automatically delete the earliest generated floating UI element and display the floating UI element generated this time on the mobile phone screen . For example, referring to FIG. 23 , the mobile phone currently displays 5 floating UI elements, and the maximum number of floating UI elements allowed to be displayed at the same time is 5. As shown in (a) in FIG. 23 , the mobile phone currently displays five floating UI elements, namely floating UI element 2302 , floating UI element 2303 , floating UI element 2304 , floating UI element 2305 and floating UI element 2306 . Among them, the floating UI element 2302 is the earliest generated floating UI element. As shown in (a) of FIG. 23 , the mobile phone displays an interface 2301 of a gallery application, and the interface 2301 may be the second interface in the embodiment of the present application. The user drags the picture 2307 in the interface 2301 to trigger the picture 2307 to be displayed on the screen of the mobile phone in the form of a floating UI element. The picture 2307 may be the third element in the embodiment of this application. Since the number of floating UI elements displayed on the mobile phone screen has reached the upper limit at this time, the mobile phone can delete (or not display ) floating UI element 2302, and displaying floating UI element 2308, floating UI element 2303, floating UI element 2304, floating UI element 2305 and floating UI element 2306 on the screen. For example, the mobile phone can automatically arrange and display the newly generated floating UI element 2308 (the floating UI element 2308 can be the second floating element in this embodiment of the application) at the bottom of the displayed floating UI element queue, and the displayed floating UI The elements move up sequentially, and the floating UI element 2302 at the top of the queue is automatically replaced and disappears. In some embodiments, before the newly generated floating UI element automatically replaces the earliest generated floating UI element and is displayed on the screen, the mobile phone may also display a prompt message to remind the user that the newly generated floating UI element will replace the earliest generated floating UI Elements are displayed on the screen. After the prompt, the newly generated floating UI element is automatically displayed on the screen instead of the earliest generated floating UI element.

Wherein, in the above-mentioned embodiment shown in FIG. 23 , when the number of floating UI elements already displayed on the screen of the mobile phone has reached the limit of the maximum number, when the user triggers the mobile phone to display interface elements in the form of floating UI elements again, Whether to allow newly generated floating UI elements to automatically replace the earliest generated floating UI elements to be displayed on the screen can be the default configuration of the system, or a setting interface can be provided for users to set. For example, when the user sets to allow newly generated floating UI elements to automatically replace the earliest generated floating UI elements on the screen, the number of floating UI elements already displayed on the mobile phone screen has reached the maximum number limit, and the user again When the mobile phone is triggered to display interface elements in the form of floating UI elements, the mobile phone can automatically replace the earliest generated floating UI elements and display them on the screen. If the user setting does not allow newly generated floating UI elements to automatically replace the earliest generated floating UI elements on the screen, the number of floating UI elements already displayed on the mobile phone screen has reached the maximum limit, and the user triggers the mobile phone again When the interface elements are displayed in the form of floating UI elements, the mobile phone displays a prompt message. For example, the prompt message can be "The number has reached the upper limit, please delete the existing floating UI elements", prompting the user that the number of floating UI elements on the screen has reached the maximum number The limitation of , cannot allow to add and display new floating UI elements. Of course, the automatic replacement rule in the embodiment shown in FIG. 23 above, such as automatically replacing the earliest generated floating UI element is only an example. In some other embodiments, the automatic replacement rule can also be that the automatic replacement is the latest generation Floating UI elements, or automatically replacing the least frequently used floating UI elements, etc., are not limited in this embodiment. The automatic replacement rule may be predefined by the system, or a setting interface may be provided for the user to set.

As described in the above-mentioned embodiment, the user can trigger the mobile phone to display various types in the interface (such as text, picture, icon, card, list contacts, etc.) Video window class, file resource class) elements are displayed on the mobile phone screen in the form of floating UI elements. Therefore, it can be considered that the correspondingly generated floating UI element also has a corresponding type, for example, the type of the floating UI element is the same as the type of the element. Based on this, in some other embodiments, when multiple floating UI elements are allowed to be displayed simultaneously on the mobile phone screen, and the maximum number of floating UI elements that are allowed to be displayed at the same time is set, if the floating UI elements already displayed on the mobile phone screen If the number of elements has reached the maximum limit, when the user triggers the mobile phone to display interface elements in the form of floating UI elements again, the mobile phone can first determine whether there is a newly generated floating UI element among the currently displayed floating UI elements type (or whether there is a floating UI element of the same type as the interface element), if not, then the floating UI element can be displayed on the screen of the mobile phone according to the embodiment shown in FIG. 23 . If it exists, such as the floating UI element 1 in the currently displayed floating UI element is the same type as the newly generated floating UI element, then the newly generated floating UI element (the floating UI element can be the second floating UI element in the embodiment of the application) element) and the floating UI element 1 (the floating UI element 1 may be the third floating element in the embodiment of the present application) are automatically merged into a floating UI element group and displayed on the screen of the mobile phone.

For example, in combination with Figure 24, as shown in (a) in Figure 24, the mobile phone currently displays five floating UI elements, namely floating UI element 2402, floating UI element 2403, floating UI element 2404, floating UI element 2405 and floating UI element 2406. Wherein, the floating UI element 2402 is a floating UI element of list contacts. The floating UI element 2403 is a floating UI element of the file resource class. The floating UI element 2404 is a floating UI element of icon type. The floating UI element 2405 is a floating UI element of picture type. The floating UI element 2406 is a card type floating UI element. As shown in (a) in Figure 24, the mobile phone displays the interface 2401 of the gallery application, and the user drags the picture 2407 in the interface 2401 to trigger the picture 2407 to be a floating UI element (the type of the newly generated floating UI element is a picture class ) is displayed on the phone screen. Since the number of floating UI elements displayed on the mobile phone screen has reached the upper limit at this time, after the mobile phone receives the user's drag and drop operation, the mobile phone can determine whether there is a newly generated floating UI element among the currently displayed floating UI elements. Floating UI elements with the same element type (image class). 24, it can be seen that the mobile phone can determine that the currently displayed floating UI element 2405 is the same type as the newly generated floating UI element, both of which are pictures, and then the mobile phone will display the newly generated floating UI element and the floating UI element 2405 together. , as shown in (c) in FIG. 24 , the mobile phone displays a floating UI element group 2408 . In some embodiments, when the mobile phone determines that among the currently displayed floating UI elements, there is a floating UI element of the same type as the newly generated floating UI element, the mobile phone can also use the displayed floating UI element of the same type as the newly generated floating UI element. The floating UI element of is highlighted to prompt the user that the newly generated floating UI element will be displayed merged with it. For example, as shown in (b) of FIG. 24 , during the process of the user dragging the picture 2407 , the mobile phone may perform background blur processing to display the floating picture 2407 . The mobile phone may also clearly display the floating UI element 2405 of the same type as the newly generated floating UI element among the displayed floating UI elements, and display other floating UI elements indistinctly. Afterwards, after the user releases the drag, as shown in (c) in FIG. 24 , the mobile phone displays a floating UI element group 2408 . In addition, after the mobile phone combines and displays multiple (two or more) floating UI elements, the mobile phone can also display prompt information on the floating UI element group to prompt the individual floating UI elements included in the floating UI element group. number. As shown in (c) in Figure 24, the mobile phone displays a subscript 2409 on the floating UI element group 2408 (such as the upper right corner), and the number "2" included in the subscript 2409 is used to indicate that the floating UI element group 2408 includes 2 A floating UI element, or a combination of two floating UI elements.

It should be noted that, the above example is described by taking as an example that there is one floating UI element of the same type as the newly generated floating UI element among the displayed floating UI elements. In some other embodiments, when there are multiple floating UI elements of the same type as the newly generated floating UI elements among the displayed floating UI elements, the mobile phone may automatically select one or more of the multiple floating UI elements to match with the newly generated floating UI elements. Newly generated floating UI elements are merged into a floating UI element group. The mobile phone may also combine the floating UI element selected by the user among the plurality of floating UI elements and the newly generated floating UI element into a floating UI element group according to the user's operation.

In the above embodiment shown in FIG. 24 , when the number of floating UI elements displayed on the screen reaches the upper limit, and a new floating UI element is generated, the mobile phone will merge the floating UI elements of the same type into a group of floating UI elements and display them on the screen. The mobile phone screen is used as an example to illustrate. In some other embodiments, in the process of generating floating UI elements, if an operation of the user triggers the mobile phone to generate multiple floating UI elements, the mobile phone can also generate floating UI elements of the same type among the multiple floating UI elements. Elements are automatically merged into a floating UI element group presented to the user. That is to say, when the user simultaneously triggers multiple elements of the same type to be presented in the form of floating UI elements, the mobile phone will automatically combine the generated floating UI elements of these multiple elements into a floating UI element group and present it to the user.

For example, referring to FIG. 25 , it is taken as an example that the user triggers multiple picture elements to be presented as floating UI elements at the same time, and the operation of triggering picture elements to be presented as floating UI elements is a drag operation as an example. For example, the drag operation includes an operation of triggering the floating of the picture and an operation of triggering the movement of the floating picture. The operation that triggers the floating of the picture is a long press operation on the picture, and the operation that triggers the movement of the floating picture is the operation of pressing and moving the floating picture.

As shown in (a) of FIG. 25 , the mobile phone displays an interface 2501 of a gallery application, and the interface 2501 of the gallery application includes multiple pictures. The user can long press the picture 2502 in the interface 2501 of the gallery application. As shown in (b) in FIG. 25 , the user presses and holds the picture 2503 in the interface 2501 of the gallery application. The mobile phone may receive the user's long press operation on pictures 2502 and 2503. In response to the long press operation, the mobile phone may display the pictures 2502 and 2503 in a floating state. For example, as shown in (c) in FIG. 25 , the mobile phone can perform background blur processing and display floating pictures 2502 and 2503 . Afterwards, the user presses the floating picture 2502 and picture 2503 and moves the finger (eg, moves the finger to the right), so that the picture 2502 and the picture 2503 can move along with the movement of the user's finger on the screen of the mobile phone. Afterwards, the user can lift the finger to release the drag. After the user releases the finger, the mobile phone can receive the user's drag and release operation. After the mobile phone receives the drag-and-drop operation, the mobile phone can generate the floating UI element of the picture 2502 and the floating UI element of the picture 2503. Since the two floating UI elements are of the same type, both are pictures, as shown in Figure 25 As shown in (d), the mobile phone may display a floating UI element group 2504 on the mobile phone screen. The floating UI element group 2504 includes the floating UI elements of the picture 2502 and the floating UI elements of the picture 2503 , or is formed by merging the floating UI elements of the picture 2502 and the floating UI elements of the picture 2503 . As shown in (d) in Figure 25, the mobile phone displays a subscript 2505 on the floating UI element group 2504 (such as the upper right corner), and the number "2" included in the subscript 2505 is used to indicate that the floating UI element group 2504 includes 2 A floating UI element, or a combination of two floating UI elements.

Similarly, when the user simultaneously triggers multiple elements of the same type, such as file resources, text, icons, cards, list contacts, video windows, etc. to be presented as floating UI elements, the mobile phone can also display the The generated multiple floating UI elements are automatically merged into a floating UI element group and displayed on the screen of the mobile phone. The mobile phone may also display prompt information on the floating UI element group to prompt the number of floating UI elements included in the floating UI element group. In addition, in this embodiment, the number of floating UI elements allowed to be included in a floating UI element group may not be limited, or an upper limit (for example, 9) may be set. Wherein, the upper limit may provide a setting interface for the user to set, and may also be a default configuration of the system.

For floating UI elements presented in a floating UI element group, the mobile phone may also expand the floating UI element group according to user operations, so that the user can view each floating UI element in the floating UI element group. For example, as shown in (a) in FIG. 26 , the mobile phone currently displays: floating UI element 2601 , floating UI element 2602 , floating UI element 2603 , floating UI element group 2604 and floating UI element 2605 . Wherein, the floating UI element group 2604 includes two floating UI elements. The mobile phone receives the user's operation on the floating UI element group 2604 (this operation may be the third operation in the embodiment of the present application), such as a click operation, a double-click operation or a long-press operation. In response to this operation, the handset may display the floating UI elements included in the group 2604 of floating UI elements. For example, as shown in (b) in FIG. 26 , the mobile phone hides other floating UI elements, namely floating UI element 2601, floating UI element 2602, floating UI element 2603 and floating UI element 2605, expands floating UI element group 2604, and displays Floating UI element 2604-1 and floating UI element 2604-2. Afterwards, after receiving the user's operation, such as operations on other areas outside the area where the floating UI elements are displayed, the mobile phone can automatically close the floating UI element group and redisplay other floating UI elements, and the display effect is restored Figure 26 In (a) shown. In addition, as described in the above embodiments, there may be an upper limit requirement for the number of floating UI elements that are allowed to be displayed on the screen of the mobile phone at the same time. When the number of floating UI elements included in the floating UI element group is greater than the upper limit, after receiving the user's operation on the floating UI element group, the mobile phone can display some floating UI elements in the floating UI element group on the mobile phone screen , other floating UI elements are temporarily not displayed. Wherein, the number of displayed floating UI elements is the same as the upper limit. Afterwards, when the mobile phone receives the user's sliding operation up and down, it can trigger the mobile phone to switch the floating UI elements in the displayed floating UI element group.

For the floating UI elements presented in floating UI element groups, the mobile phone can also split (or separate) the floating UI element groups according to user operations (this operation is different from the user operations in the embodiment shown in Figure 26) so that the floating UI elements can be floated. Each floating UI element in the UI element group is independently displayed on the screen of the mobile phone. For example, as shown in (a) in FIG. 27 , the mobile phone currently displays a floating UI element group 2701 . Wherein, the floating UI element group 2701 includes two floating UI elements. The mobile phone receives the user's operation on the floating UI element group 2701, such as a click operation, a double-click operation or a long-press operation. As a response to this operation, as shown in (b) in FIG. 27 , the mobile phone independently displays the floating UI element 2702 and the floating UI element 2703 included in the floating UI element group 2701 on the screen of the mobile phone. In addition, as described in the above embodiments, there may be an upper limit requirement for the number of floating UI elements that are allowed to be displayed on the screen of the mobile phone at the same time. After receiving the user's operation for separating the floating UI element group, the mobile phone can first determine whether all the floating UI element groups displayed on the screen exceed the upper limit after the floating UI element group is separated and displayed, and if the upper limit is not exceeded, respond to An operation for separating a floating UI element group, and displaying the separated floating UI element group on the mobile phone screen. If the upper limit is exceeded, the mobile phone may not perform the operation of separating the floating UI element group, and the mobile phone may temporarily expand and display the floating UI element group, and then automatically retract and continue to display the floating UI element group.

In this embodiment, for the floating UI elements displayed on the screen of the mobile phone, the mobile phone can automatically or according to the user operation (the user operation can be the fourth operation in the embodiment of the present application), delete it, and no longer display it on the mobile phone. on the screen.

For example, after receiving the user's operation for deleting a floating UI element, the mobile phone may delete the corresponding floating UI element and no longer display it on the screen of the mobile phone. The operation for deleting the floating UI element may be a long-press operation, a double-click operation, or a drag-and-drop operation on the floating UI element. The dragging operation may be an operation of pressing and holding a floating UI element and quickly sliding in one direction (such as upward or downward) (such as the sliding distance within a preset time meets a predetermined condition).

For another example, the display duration of the floating UI element on the mobile phone screen can be unlimited (that is, it will always be displayed on the mobile phone screen if the user does not actively delete it), or it can be limited . For example, it may be to set (user setting or system default setting) the maximum existence time of floating UI elements. If the maximum existence time of the floating UI element can be set to be X hours, then after the floating UI element is generated and displayed on the mobile phone screen, the mobile phone can start timing to determine the existence time of the floating UI element on the mobile phone screen. After the timing duration is equal to the maximum existence time X hours, the mobile phone can automatically delete the floating UI element and no longer display it on the mobile phone screen. Alternatively, after the floating UI element is generated and displayed on the screen of the mobile phone, the mobile phone can start timing. If no user operation on the floating UI element is received within X hours, the mobile phone will automatically set The floating UI element is removed and no longer displayed on the phone screen. Restart the timer if a user action on the floating UI element is received within X hours. In this embodiment, in order to facilitate the user to know the existence time of the floating UI element on the screen, after the floating UI element is generated and displayed on the screen of the mobile phone, the mobile phone can display corresponding prompt information during the timing process of the mobile phone to remind the user The lifetime of the floating UI element. For example, as shown in (a) in FIG. 28 , the mobile phone displays a floating UI element 2801 . The mobile phone can also display a circular bar 2802 that grows as the mobile phone's timing for the floating UI element 2801 increases. When the timing duration is equal to the maximum existence time, the ring bar 2802 becomes a closed ring. And when the timing duration is equal to the maximum existence time, the mobile phone deletes the floating UI element 2801, and is no longer displayed on the mobile phone screen, such as the animation that the floating UI element 2801 is blown up (such as, similar to the animation that the bubble is blown up) can be displayed on the mobile phone. ), the floating UI element 2801 is deleted from the mobile phone screen.

For another example, the area for deleting the floating UI element can be preset, and after the mobile phone receives that the user drags the floating UI element to the area and releases the drag (this operation can be the fourth operation in the embodiment of the present application), The mobile phone can delete the floating UI element and no longer display it on the mobile phone screen. For example, as shown in (b) in FIG. 28 , the mobile phone displays a floating UI element 2803 . After receiving the user's operation of dragging the floating UI element 2803, the mobile phone may display an animation of the floating UI element 2803 moving with the movement of the user's finger on the screen of the mobile phone. In addition, the mobile phone may also display a delete button 2804, and the area where the delete button 2804 is located is an area for deleting floating UI elements. If the mobile phone determines that the user drags the floating UI element 2803 to the area where the delete button 2804 is located, and releases the drag, the mobile phone can delete the floating UI element 2803 and no longer display it on the mobile phone screen.

It should be noted that the above embodiments are illustrated by taking the style of the floating UI element as a circle as an example. In some other embodiments, the style of the floating UI element can also be rectangle, rounded rectangle, ellipse, polygon and other For irregular UI shapes, this embodiment does not specifically limit the styles of floating UI elements. The content in the floating UI element (or the element that can be presented as a floating UI element) is not limited to the text class, picture class, icon class, card class, list contact class, video window class, file resource class, etc. in this embodiment. element. Of course, the content in the floating UI element may also be a combination of these types of elements. In addition, in the above embodiments, the floating UI elements are displayed on the left or right edge of the screen as an example. In some other embodiments, the floating UI elements can also be displayed on other positions of the screen, such as the upper edge. The embodiment here does not specifically limit the display position (or layout position) of the floating UI element. The display position of the floating UI element may be specifically configured by the system by default, or a setting interface may be provided for the user to set. It can be seen that, in the above embodiment, the floating UI element is completely displayed on the edge of the screen in an opaque manner as an example. In some other embodiments, with reference to FIG. 29, the floating UI element, such as the floating UI element 2901 shown in (a) in FIG. 29 can also be displayed in a semi-transparent manner, or as shown in (b) in FIG. , the floating UI element 2902 can also be partially hidden at the edge of the screen, or as shown in (c) in FIG. 29 , the floating UI element 2903 can be partially hidden at the edge of the screen. The way in which the floating UI elements are displayed can be configured by the system by default, or a setting interface can be provided for the user to set.

In addition, as shown in the examples shown in FIGS. 24-27 above, prompt information, such as corner marks, may be displayed on the floating UI element group to prompt the number of floating UI elements included in the floating UI element group. Whether to display the corner mark on the floating UI element group can be the default configuration of the system, or a setting interface can be provided for the user to set. For example, a setting interface may be provided for the user to set whether to display a corner mark on the floating UI element group for prompting the number of floating UI elements included in the floating UI element group, that is, to set whether to display the mark. In some other embodiments, for floating UI elements, other subscripts may also be displayed.

For example, as shown in (a) in Figure 30, for a floating UI element that lists contacts, such as the floating UI element 3001 shown in (a) in Figure 30, when the mobile phone receives a message from the contact, The message reminder information is displayed on the floating UI element, as shown in (a) in FIG. 30 , the message reminder information can be displayed in the form of a subscript 3002, and the number in the subscript 3002 can be used to indicate the number of received messages. Of course, whether to display the message reminder information may be the default configuration of the system, or a setting interface may be provided for the user to set. If a message reminder is displayed on a floating UI element when a message from a contact is received, the message reminder may disappear and no longer be displayed after the user views it.

For another example, an application identifier indicating the source of the floating UI element, such as an application icon, may be displayed on the floating UI element, so that the user can easily identify the original source of the floating UI element, that is, which application it comes from. For example, continuing to take the example of the floating UI element of the contact list, the mobile phone displays a floating UI element 3003 as shown in (b) in FIG. The icon 3004 of the application. Wherein, whether to display the identification of the application used to indicate the source of the floating UI element may be a default configuration of the system, or a setting interface may be provided for the user to set. For example, if the system is configured by default, the floating UI element of the contact list will display the logo of the application by default. For another example, a setting interface may be provided for the user to set the display application identifier of one or more floating UI elements, and other types of floating UI elements are not displayed.

For another example, a symbol used to indicate the type of the floating UI element (such as a link) may be displayed on the floating UI element. For example, taking the floating UI element of the file resource class as an example, the mobile phone displays a floating UI element 3005 as shown in (c) in FIG. link) symbol 3006.

In some other embodiments, the subtitles in the above example may appear simultaneously, for example, a floating UI element displays message reminder information and an application identification indicating the source of the floating UI element. As another example, the floating UI element group has message reminder information (as shown in (a) in FIG. 30 ) and prompt information for prompting the number of floating UI elements included in the floating UI element group (as shown in ((a) in FIG. 27 ). a) as shown). When the floating UI element group contains message reminder information and prompt information at the same time, in order to distinguish them, the two kinds of information can be displayed at different positions of the floating UI element group, for example, the message reminder information is displayed in the upper left corner, and the prompt information is displayed in the lower left corner. Further, different display forms can also be used to distinguish the two types of information. Taking the two types of information as an example in the form of subtitles, for example, the subtitles corresponding to the message reminder information and the subtitles corresponding to the prompt information have different colors or different shapes ( For example, a corner mark is a circle, and a corner mark is a square), etc. In the scenario where the two kinds of information do not appear at the same time, the two kinds of information can also be distinguished, for details, refer to the description in the scenario where the two kinds of information appear at the same time.

In addition, as described in the above embodiments, the floating UI elements displayed on the screen can be dragged by the user to move on the screen. In some other embodiments, when the floating UI element is moving on the screen of the mobile phone, the mobile phone can display a corresponding prompt (such as a prompt in the form of a corner mark) according to the area where the floating UI element is currently located, to prompt the user whether the current area supports interact. For example, referring to FIG. 31 , take the floating UI element as an example of a text-type floating UI element. As shown in (a) of FIG. 31 , the mobile phone displays a floating UI element 3101 . The user can press and move the floating UI element 3101 to trigger the floating UI element 3101 to move with the movement of the user's finger. As shown in (b) of FIG. 31 , when the floating UI element 3101 moves to a non-interactive area, a superscript 3102 is displayed on the floating UI element 3101, and the superscript 3102 is used to prompt the user that the current area does not support interaction. As shown in (c) in Figure 31, when the floating UI element 3101 moves to an interactive area, such as the area where the input box 3103 is located, a superscript 3104 is displayed on the floating UI element 3101, and the superscript 3104 is used to prompt the user The current locale supports interaction. After the user releases the drag, the text carried by the floating UI element 3101 can be pasted into the input box 3103 . As shown in (d) in Figure 31, when the floating UI element 3101 moves to the deletion area (this area may be predefined) 3106, a superscript 3105 is displayed on the floating UI element 3101, and the superscript 3105 is used to prompt the user After releasing the drag in this area, the floating UI element 3101 will be deleted.

It can be known from the description of the above embodiment that, in the scenario where the element is triggered to be presented as a floating UI element by dragging, after the user drags and drops the element and releases the drag, the mobile phone can be triggered to display the element as a floating UI element. The form of the element is rendered on the phone screen. In some other embodiments, the trigger area can also be predefined. When the user drags the element to the trigger area and releases the drag in the trigger area, the mobile phone can present the element as a floating UI element on the screen of the mobile phone . If the user drags the element to another area and releases the drag, the mobile phone will not be triggered to perform the operation of presenting the element as a floating UI element.

With the above technical solution, the mobile phone can display any element on the interface in the form of floating UI elements on the screen of the mobile phone according to user operations. Afterwards, no matter whether the user switches pages in the application, exits the application or opens other applications, the floating UI element can continue to be displayed on the mobile phone screen for cross-application information viewing, file transfer, information sharing and other operations. For example, the floating UI elements of the text class, picture class, card class, file resource class, and video window class support interactions such as sending to friends through instant messaging applications. Files, images, text, and URLs can be directly shared with the corresponding contacts by dragging and dropping them to the floating UI elements of the contact list. Floating UI elements of the URL type support direct click and jump. A floating UI element representing a specific contact extracted from the contact list of different applications supports functions such as sending files and making calls according to specific business scenarios. In addition, the floating UI elements are generated in combination with the existing interactive logic of the system.

FIG. 32 is a schematic composition diagram of a display device provided by an embodiment of the present application. The device can be applied to a terminal. As shown in FIG. 32 , the device may include: a display unit 3201 and an input unit 3202 .

The display unit 3201 is configured to display a first interface of the application on a display screen of the terminal, where the first interface includes a first element.

The input unit 3202 is configured to receive a drag operation on the first element.

The display unit 3201 is further configured to display an animation of the first element moving with the drag operation on the display screen of the terminal.

The input unit 3202 is also configured to receive a dragging and releasing operation.

The display unit 3201 is further configured to display the first floating element on the display screen of the terminal, the content of the first floating element is the same as the content of the first element.

Further, the first interface may also include a second element within a preset range of the first element; the first floating element may also include the content of the second element.

Further, the first element may be one or more of text elements, URL URLs, image elements, icon elements, card elements, contact list elements, video window elements or file resource elements.

Further, the input unit 3202 is also configured to receive the first operation.

The display unit 3201 is further configured to display elements in the first interface that support floating display in a first form, and elements that do not support floating display in a second form; the first element is an element that supports floating display, and the first form and the second form The two forms are different.

Further, the first operation may be an operation on a predetermined function control; or the first operation may be an operation on an element supporting floating display in the first interface.

Further, the display unit 3201 is further configured to display a second interface on the display screen of the terminal, where the second interface includes a third element.

The input unit 3202 is further configured to receive a second operation on the third element.

The display unit 3201 is further configured to display a second floating element on the display screen of the terminal, the content of the second floating element is the same as that of the third element, and the second floating element and the first floating element are displayed on the terminal according to a preset arrangement rule on the display. Wherein, the second operation may include a drag operation and a drag-and-drop operation, or when the first element is a text element or a URL, the second operation may also be a copy operation.

Further, the display device may further include: a determination unit 3203, configured to determine that the number of floating elements displayed on the display screen of the terminal is less than N, where N is the maximum number of floating elements allowed to be displayed simultaneously on the display screen.

Further, the display device may further include: a deleting unit 3204, configured to delete N floating elements ranked according to a preset arrangement rule when the determining unit determines that the number of floating elements displayed on the display screen of the terminal is equal to N. One floated UI element and displays the second floated element.

Further, the determining unit 3203 is further configured to determine that there is no floating element of the same type as the second floating element among the floating elements displayed on the display screen of the terminal.

Further, the display unit 3201 is further configured to combine the second floating element and the third floating element when the determining unit 3203 determines that there is a third floating element of the same type as the second floating element among the floating elements displayed on the display screen of the terminal. Combined into groups of floating elements and displayed on the terminal's display.

Further, the input unit 3202 is further configured to receive a third operation on the floating element group; the display unit 3201 is configured to display the second floating element and the third floating element on the display screen of the terminal.

Further, the input unit 3202 is further configured to receive a fourth operation on the first floating element, and the deleting unit 3204 is further configured to delete the first floating element. Or, the deleting unit 3204 is further configured to delete the first floating element after the first floating element is displayed on the display screen of the terminal for a preset period of time.

Further, one or more of the following identifications are displayed on the first floating element: application identification, message reminder information, and indication information; the indication information is used to indicate the type of the first floating element.

Further, when the first element is a text element, a URL address, an image element or a file resource element, the display unit 3201 is further configured to display an interface including an input box.

The input unit 3202 is further configured to receive a drag operation on the first floating element, and release the drag operation at the position of the input box.

The display device may further include: a filling unit 3205, configured to fill the input box with content corresponding to the first floating element.

Further, in the case that the first element is an icon element or a list contact element, the input unit 3202 is further configured to receive a fifth operation on the first floating element.

The display device may further include: an executing unit 3206, configured to execute a first function, the first function being the same as the function executed by the terminal after receiving the user's operation on the first element.

In another possible implementation, when the first element is a list contact person element, the input unit 3202 is also configured to receive a drag operation on the text, URL or picture in the interface, and The release drag operation of the element position.

The display device may further include: a sending unit 3207, configured to send text, a URL or a picture to a terminal of a contact corresponding to the first floating element.

In another possible implementation manner, the display unit 3201 is configured to display a first interface on a display screen of the terminal, where the first interface includes a first element, and the first element is a text element or a URL. The input unit 3202 is configured to receive a copy operation on the first element. The display unit 3201 is further configured to display the first floating element on the display screen of the terminal, the content of the first floating element is the same as the content of the first element.

The embodiment of the present application also provides a display device, which can be applied to electronic equipment, such as the terminal in the foregoing embodiment. The device may include: a processor; a memory for storing processor-executable instructions; wherein, when the processor is configured to execute the instructions, the display device implements various functions or steps performed by the terminal (such as a mobile phone) in the above method embodiments .

An embodiment of the present application provides an electronic device (such as the above-mentioned terminal), the electronic device includes a display screen, one or more processors and a memory; the display screen, the processor and the memory are coupled; the memory is used to store computer program codes, computer programs The code includes computer instructions, and when the computer instructions are executed by the electronic device, the electronic device is made to implement various functions or steps performed by the terminal (such as a mobile phone) in the above method embodiments. Of course, the electronic device includes, but is not limited to, the above-mentioned display screen, memory, and one or more processors. For example, the structure of the electronic device may refer to the structure of the mobile phone shown in FIG. 3 .

The embodiment of the present application further provides a chip system, which can be applied to the terminal in the foregoing embodiments. As shown in FIG. 33 , the chip system includes at least one processor 3301 and at least one interface circuit 3302 . The processor 3301 may be the processor in the aforementioned terminal. The processor 3301 and the interface circuit 3302 can be interconnected through wires. The processor 3301 may receive and execute computer instructions from the memory of the terminal above through the interface circuit 3302 . When the computer instructions are executed by the processor 3301, the terminal can be made to perform various steps performed by the mobile phone in the above-mentioned embodiments. Of course, the chip system may also include other discrete devices, which is not specifically limited in this embodiment of the present application.

An embodiment of the present application further provides a computer-readable storage medium, configured to store computer instructions executed by the above-mentioned terminal.

An embodiment of the present application further provides a computer program product, including computer instructions executed by the above-mentioned terminal.

Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above functional modules is used as an example for illustration. In practical applications, the above functions can be assigned by Completion of different functional modules means that the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be Incorporation or may be integrated into another device, or some features may be omitted, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may be one physical unit or multiple physical units, that is, it may be located in one place, or may be distributed to multiple different places . Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium Among them, several instructions are included to make a device (which may be a single-chip microcomputer, a chip, etc.) or a processor (processor) execute all or part of the steps of the methods described in the various embodiments of the present application. The above-mentioned storage medium includes: U disk, mobile hard disk, read only memory (read only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk, and other various media that can store program codes.

The above content is only the specific implementation of the application, but the protection scope of the application is not limited thereto, and any changes or replacements within the technical scope disclosed in the application shall be covered within the protection scope of the application. Therefore, the protection scope of the present application should be determined by the protection scope of the claims.

Claims (16)

1. An interface element display method is applied to a terminal, and is characterized by comprising the following steps:

the terminal displays a first interface of an application on a display screen of the terminal, wherein the first interface comprises a first element;

after the terminal receives a drag operation on the first element, displaying an animation of the first element moving along with the drag operation on a display screen of the terminal;

after receiving a drag release operation, the terminal displays a first floating element on a display screen of the terminal, wherein the content of the first floating element is the same as that of the first element;

the terminal displays a second interface on a display screen of the terminal, wherein the second interface comprises a third element;

after the terminal receives the dragging operation of the third element, displaying the animation of the third element moving along with the dragging operation on a display screen of the terminal;

after the terminal receives a drag release operation, when the terminal determines that a third floating element with the same type as a second floating element exists in floating elements displayed on a display screen of the terminal, the terminal combines the second floating element and the third floating element into a floating element group and displays the floating element group on the display screen of the terminal; wherein the content of the second floating element is the same as the content of the third element;

the terminal displays a third interface;

after the terminal receives a drag operation of moving the first floating element to a first area of the third interface, displaying a first corner mark on the first floating element, wherein the first corner mark is used for prompting a user that the first area supports interaction, and the first area is an area supporting interaction;

and after the terminal receives the drag releasing operation in the first area, filling the content corresponding to the first floating element into the first area by the terminal.

2. The method of claim 1, further comprising: and when the terminal determines that floating elements with the same type as the second floating elements do not exist in the floating elements displayed on the display screen of the terminal, displaying the second floating elements on the display screen of the terminal, wherein the second floating elements and the first floating elements are displayed on the display screen of the terminal according to a preset arrangement rule.

3. The method of claim 1, wherein the terminal displays a third interface comprising:

and the terminal exits from the first interface and displays the third interface.

4. The method of claim 1, wherein the first element is one or more of a text class element, a Uniform Resource Locator (URL) address, a picture class element, an icon class element, a card class element, a list contact human element, a video window class element, or a file resource class element.

5. The method of claim 1, the third interface being an interface comprising an input box, the first element being a text-type element, a URL web address, a picture-type element, or a file resource-type element;

the terminal receives a drag operation of moving the first floating element to the first area of the third interface, and the drag operation comprises the following steps: the terminal receives a dragging operation of moving the first floating element to the position of the input box;

after the terminal receives the drag and drop releasing operation in the first area, the terminal fills the content corresponding to the first floating element into the first area, and the method comprises the following steps: and after the terminal receives the drag releasing operation at the position of the input box, filling the content corresponding to the first floating element into the input box by the terminal.

6. The method according to any one of claims 1-5, wherein the first interface further comprises a second element, and the terminal displays a first floating element on a display screen of the terminal after receiving a drag operation on the first element, including:

the terminal displays a first floating element group on a display screen of the terminal after receiving dragging operation of the first element and the second element, wherein the first floating element group comprises the first floating element and the second floating element, second prompt information is displayed on the first floating element group, and the second prompt information is used for prompting the number of the floating elements included in the first floating element group.

7. The method according to claim 6, wherein the terminal displays the first floating element and the second floating element according to the precedence order of the generation time of the first floating element and the second floating element.

8. The method of claim 1, wherein after the terminal displays the first interface of the application on a display screen of the terminal, the method further comprises:

and after receiving a first operation, the terminal displays the elements supporting floating display in the first interface in a first form, wherein the first elements are the elements supporting floating display.

9. The method of claim 8, wherein the first operation is an operation on an element of the first interface that supports a floating display.

10. The method of claim 1, wherein the first interface and the third interface are interfaces of different applications.

11. The method of claim 1, further comprising:

the terminal deletes the first floating element after receiving the second operation on the first floating element; or the like, or a combination thereof,

and the terminal deletes the first floating element after the first floating element is displayed on a display screen of the terminal for a preset time.

12. The method of claim 1, wherein the first floating element has one or more of the following identifiers displayed thereon: the application identification, message reminding information and indication information; the indication information is used for indicating the type of the first floating element.

13. A display device, comprising: a processor; a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to cause the display apparatus to implement the method of any one of claims 1-12.

14. A computer readable storage medium having computer program instructions stored thereon, which, when executed by an electronic device, cause the electronic device to implement the method of any one of claims 1-12.

15. An electronic device comprising a display screen, one or more processors, and memory;

the one or more memories coupled with the one or more processors for storing computer program code, the computer program code comprising computer instructions that, when executed by the one or more processors, cause the electronic device to perform the method of any of claims 1-12.

16. A computer program product comprising instructions for causing a computer to perform the method according to any one of claims 1-12 when the computer program product is run on the computer.

Images