CN114827343B - Method and device for screen sharing - Google Patents

Abstract

A method and apparatus for screen sharing are provided. The method includes: when an application uses a camera service, the camera service provides an image capture function, a screen sharing function, or an image capture and screen sharing function option for a user to select; the camera service acquires data based on the function selected by the user, and provides the acquired data to the application.

Description

Method and apparatus for screen sharing

Technical Field

The present application relates to the field of display, and more specifically, to a method and device for screen sharing.

Background Art

In the prior art, the screen sharing function is implemented inside the application, and the application mainly obtains the screen content by recording the screen.

Therefore, the screen sharing function of the prior art can only be used separately in a specific application with this function. For example, screen sharing cannot be performed when using the application for a video call. In addition, since the application obtains the screen content by recording the screen, the original high-quality image data cannot be obtained.

Summary of the invention

According to an exemplary embodiment of the present invention, a method for screen sharing is provided, including: when an application uses a camera service, the camera service provides options of an image capture function, a screen sharing function, or an image capture and screen sharing function for a user to select; the camera service acquires data based on the function selected by the user, and provides the acquired data to the application.

When the image capture function is selected by the user, the acquired data may be the image captured by the camera; when the screen sharing function is selected by the user, the acquired data may be the screen frame displayed by another application; when the image capture and screen sharing functions are selected by the user, the acquired data may be the image captured by the camera and the screen frame displayed by the other application.

The camera service can obtain screen frames from the display rendering service (Surface Flinger).

When the image capture and screen sharing functions are selected by the user, the step of the camera service acquiring data based on the function selected by the user may include: determining whether the current electronic device has a folding screen; and acquiring data assigned to the corresponding screen based on whether the current electronic device has a folding screen.

When it is determined that the current electronic device has a folding screen, the step of obtaining data assigned to the corresponding screen may include: determining whether the folding screen of the current electronic device is folded up and down or folded left and right; and obtaining data assigned to the corresponding screen based on whether the folding screen of the current electronic device is folded up and down or folded left and right.

When it is determined that the current electronic device does not have a foldable screen, the step of obtaining data assigned to the corresponding screen may include: obtaining a camera captured image and a screen frame; reducing the brightness difference between the camera captured image and the screen frame; and determining a layout style of the camera captured image and the screen frame on the screen that reduces the brightness difference.

The step of determining a layout pattern of the camera captured images and the screen frames on the screen that reduces the brightness difference may include determining the layout pattern of the camera captured images and the screen frames on the screen that reduces the brightness difference through a decision tree model based on predetermined attributes.

The predetermined attributes may include whether the current electronic device has a folding screen, whether the current electronic device is in a folding state, camera captured image sharpness, screen frame sharpness, the type of the application, and at least one of the type of the other application.

According to another exemplary embodiment of the present invention, a camera service device for screen sharing is provided, including: an option providing module, configured to provide options of an image capturing function, a screen sharing function, or both an image capturing and a screen sharing function for user selection when an application uses the camera service; and a data acquisition module, configured to acquire data based on a function selected by the user, and provide the acquired data to the application.

When the image capture function is selected by the user, the acquired data may be the image captured by the camera; when the screen sharing function is selected by the user, the acquired data may be the screen frame displayed by another application; when the image capture and screen sharing functions are selected by the user, the acquired data may be the image captured by the camera and the screen frame displayed by the other application.

The camera service device for screen sharing may further include: a screen management module configured to obtain a screen frame from a display rendering service (Surface Flinger).

When the image capture and screen sharing functions are selected by the user, the data acquisition module may also be configured to perform the following operations: determine whether the current electronic device has a folding screen; and acquire data assigned to the corresponding screen based on whether the current electronic device has a folding screen.

When it is determined that the current electronic device has a folding screen, the data acquisition module can also be configured to perform the following operations: determine whether the folding screen of the current electronic device is folded up and down or left and right; and acquire data assigned to the corresponding screen based on whether the folding screen of the current electronic device is folded up and down or left and right.

When it is determined that the current electronic device does not have a folding screen, the data acquisition module can also be configured to perform the following operations: acquire camera captured images and screen frames; reduce the brightness difference between the camera captured images and the screen frames; determine the layout style of the camera captured images and screen frames on the screen that reduce the brightness difference.

The data acquisition module may be further configured to perform the following operation: determining a layout pattern of the camera-captured image and the screen frame on the screen that reduces the brightness difference through a decision tree model based on predetermined attributes.

The predetermined attributes may include whether the current electronic device has a folding screen, whether the current electronic device is in a folding state, camera captured image sharpness, screen frame sharpness, the type of the application, and at least one of a type of another application for outputting a screen frame.

According to another exemplary embodiment of the present invention, an electronic device is provided, including a memory and a processor, wherein computer executable instructions of a camera service are stored in the memory, and when the instructions are executed by the processor, the aforementioned method is executed.

According to another exemplary embodiment of the present invention, a computer-readable medium is provided, on which computer-executable instructions of a camera service are stored, and when the instructions are executed, the aforementioned method is performed.

According to an exemplary embodiment of the present invention, the functionality of the camera service can be expanded, and applications using the camera service can perform screen sharing. The extended functionality of the camera service as a system process can be used to obtain original high-quality screen frames for screen sharing, and the camera-captured images and screen frames can be displayed simultaneously, so that the application does not need to implement the screen sharing function itself. In addition, a better layout can be provided for screen sharing and better adaptation to folding screens.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and features of the present invention will become more apparent from the following detailed description in conjunction with the accompanying drawings, in which:

FIG1 is a flow chart illustrating a method for screen sharing according to an exemplary embodiment of the present invention;

FIG2 is a schematic diagram showing providing options according to an exemplary embodiment of the present invention;

FIG3 is a flow chart showing step S120 of FIG1 according to an exemplary embodiment of the present invention;

4 is a schematic diagram illustrating allocation of camera captured images and screen frames according to an exemplary embodiment of the present invention;

FIG5 is a schematic diagram illustrating brightness fusion and image stitching according to an exemplary embodiment of the present invention;

FIG. 6 is a block diagram illustrating a camera service device for screen sharing according to an exemplary embodiment of the present invention.

DETAILED DESCRIPTION

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like parts throughout. The embodiments will be described below in order to explain the present invention by referring to the drawings.

FIG. 1 is a flowchart illustrating a method for screen sharing according to an exemplary embodiment of the present invention.

1, in step S110, when the application uses the camera service, the camera service may provide an image capture function, a screen sharing function, or an image capture and screen sharing function options for the user to select. It should be noted that the camera service here may be a system camera service. Here, when the image capture function is selected by the user, the acquired data may be a camera captured image, wherein the camera captured image may be a front camera captured image, a rear camera captured image, or another camera captured image; when the screen sharing function is selected by the user, the acquired data may be a screen frame for screen sharing displayed by another application; when the image capture and screen sharing functions are selected by the user, the acquired data may be a camera captured image and a screen frame displayed by the other application. In addition, by way of example only and not limitation, when the system is Android, the camera service may obtain a screen frame from SurfaceFlinger, and here, since the camera service is a system process, it may call Surface Flinger. In addition, by way of example only and not limitation, the camera service may provide an image capture function, a screen sharing function, or an image capture and screen sharing function options through a pop-up icon, a pop-up window, etc. In addition, when the user uses the camera service from the application, the image capture function, the screen sharing function, or the image capture and screen sharing function options can be provided, or when the user uses the camera service from the application, a floating icon can be provided, and when the user clicks or touches the floating icon, the image capture function, the screen sharing function, or the image capture and screen sharing function options are provided. FIG. 2 is a diagram showing the provision of options according to an exemplary embodiment of the present invention. As shown in FIG. 2, when the user clicks the floating icon, the image capture function, the screen sharing function, or the image capture and screen sharing function options are provided.

In step S120, the camera service may acquire data based on the function selected by the user and provide the acquired data to the application. Step S120 will be described in more detail below with reference to FIG.

FIG. 3 is a flowchart illustrating step S120 of FIG. 1 according to an exemplary embodiment of the present invention.

3 , in step S310 , when an image capturing function is selected by a user, the camera service may acquire a camera captured image.

In step S320 , when the screen sharing function is selected by the user, the camera service may acquire a screen frame displayed by another application.

In step S330, when the image capture and screen sharing functions are selected by the user, the camera service may determine whether the current electronic device has a foldable screen, and acquire data allocated to the corresponding screen based on whether the current electronic device has a foldable screen.

When it is determined that the current electronic device has a folding screen, in step S340, the camera service may determine whether the folding screen of the current electronic device is folded up and down or folded left and right, and obtain the data assigned to the corresponding screen based on whether the folding screen of the current electronic device is folded up and down or folded left and right, such as obtaining the camera capture image and screen frame, and assigning the camera capture image and screen frame to the upper screen and the lower screen respectively, or assigning the camera capture image and screen frame to the left screen and the right screen respectively. More specifically, the camera capture image and screen frame may be assigned to the upper screen and the lower screen or the left screen and the right screen respectively according to the user's usage habits, such as when the user's dominant hand is the right hand, the camera capture image may be assigned to the left screen and the screen frame may be assigned to the right screen. Here, it should be noted that the above-mentioned method of assigning camera capture images and screen frames is only an example and not a limitation, and those skilled in the art may assign camera capture images and screen frames to the folding screen according to actual conditions. FIG. 4 is a diagram showing the assignment of camera capture images and screen frames according to an exemplary embodiment of the present invention. As shown in FIG. 4, when the user chooses to share a PowerPoint (PPT) application as another application, the camera capture image and screen frame (displayed by the PPT application) may be displayed according to the folding screen.

In addition, when it is determined that the current electronic device does not have a folding screen, in step S350, the camera service may obtain the camera captured image and the screen frame, and reduce the brightness difference between the camera captured image and the screen frame (such as, making the brightness of the two tend to be consistent). Here, by way of example only and not limitation, the brightness difference between the camera captured image and the screen frame may be reduced by performing brightness fusion. In step S360, the camera service may determine the layout style of the camera captured image and the screen frame on the screen that reduces the brightness difference for image stitching. More specifically, the step of determining the layout style of the camera captured image and the screen frame on the screen that reduces the brightness difference may include: determining the layout style of the camera captured image and the screen frame on the screen that reduces the brightness difference based on a predetermined attribute through a decision tree model, wherein, by way of example only and not limitation, the predetermined attribute may include whether the current electronic device has a folding screen, whether the current electronic device is in a folded state, the camera captured image sharpness, the screen frame sharpness, the type of the application, and at least one of the type of the other application. Here, for example only and not limitation, whether the current electronic device has a foldable screen may include having (the value may be 1, the same below) and not having (0); whether the current electronic device is in a folded state may include a non-foldable device (null, the value may be 0), a folded state (1) and a non-folded state (0); the camera capture image sharpness may include high (0), medium (1) and low (2); the screen frame sharpness may include high (0), medium (1) and low (2); the type of the application may include business (0), entertainment (1) and general (2); the type of the other application may include business (0), entertainment (1) and general (2). In addition, the decision tree model can be trained by using the data of screen sharing completed by existing applications or the data of spliced images on mainstream websites. For example, the data of screen sharing completed by online teaching applications can be x={whether it has a folding screen, whether it is in a folded state, camera captured image sharpness, screen frame sharpness, application type, type of another application}={0, 0, 1, 0, 0, 0}, and the corresponding layout style is y=type 2 (such as, the screen frame is displayed on the entire screen, and the camera captured image is covered in the upper right corner), so that a training data {x, y}={0, 0, 1, 0, 0, 0, 2} can be obtained, and multiple such training data can be obtained to train the decision tree model. After the training is completed, the data x can be used to obtain the type y of the layout style through the decision tree model. Figure 5 is a schematic diagram showing brightness fusion and image splicing according to an exemplary embodiment of the present invention. As shown in Figure 5, the user expects to record a short video of making food with a recipe at the same time, and the food video as the camera captured image and the recipe text as the screen frame can be subjected to brightness fusion and image splicing as described above.

In addition, after steps S310, S320, S340, and S360, in step S370, the camera service may provide the acquired data to the application. More specifically, after step S310, the camera service may provide the camera captured image to the application; after step S320, the camera service may provide the screen frame to the application; after step S340, the camera service may provide the camera captured image and the screen frame assigned to the folding screen to the application; after step S360, the camera service may provide the camera captured image and the screen frame with the determined layout style to the application.

FIG. 6 is a block diagram illustrating a camera service device for screen sharing according to an exemplary embodiment of the present invention.

6 , a camera service device 600 for screen sharing according to an exemplary embodiment of the present invention may include an option providing module 610 and a data acquiring module 620 .

The option providing module 610 may be configured to provide options of an image capturing function, a screen sharing function, or both an image capturing function and a screen sharing function for user selection when an application uses a camera service.

The data acquisition module 620 may be configured to acquire data based on the function selected by the user and provide the acquired data to the application. Here, when the image capture function is selected by the user, the acquired data may be a camera captured image; when the screen sharing function is selected by the user, the acquired data may be a screen frame for screen sharing displayed by another application; when the image capture and screen sharing functions are selected by the user, the acquired data may be a camera captured image and a screen frame displayed by the other application.

When the image capture and screen sharing functions are selected by the user, the data acquisition module 620 may also be configured to perform the following operations: determine whether the current electronic device has a foldable screen; and acquire data assigned to the corresponding screen based on whether the current electronic device has a foldable screen.

When it is determined that the current electronic device has a folding screen, the data acquisition module 620 may also be configured to perform the following operations: determine whether the folding screen of the current electronic device is folded up and down or folded left and right; and acquire data assigned to the corresponding screen based on whether the folding screen of the current electronic device is folded up and down or folded left and right. When it is determined that the current electronic device does not have a folding screen, the data acquisition module 620 may also be configured to perform the following operations: acquire camera-captured images and screen frames; reduce the brightness difference between the camera-captured images and the screen frames; and determine the layout style of the camera-captured images and screen frames on the screen with reduced brightness difference.

The data acquisition module 620 can also be configured to perform the following operations: determine the layout style of the camera captured image and the screen frame on the screen that reduces the brightness difference through a decision tree model based on predetermined attributes, wherein, as an example only and not a limitation, the predetermined attributes include whether the current electronic device has a folding screen, whether the current electronic device is in a folded state, the sharpness of the camera captured image, the sharpness of the screen frame, the type of the application, and at least one of the type of the other application.

In addition, the camera service device 600 for screen sharing may further include: a screen management module 630 , which may be configured to obtain a screen frame from Surface Flinger.

It should be understood that the camera service device 600 for screen sharing according to an exemplary embodiment of the present invention in FIG. 6 may correspond to the camera service described above.

On the other hand, according to an exemplary embodiment of the present invention, there is also provided an electronic device, including a memory and a processor, wherein the memory stores computer executable instructions for camera services, and when the instructions are executed by the processor, the aforementioned method is executed. According to an exemplary embodiment of the present invention, there is also provided a computer readable medium, wherein computer executable instructions for camera services are stored, and when the instructions are executed, the aforementioned method is executed.

According to an exemplary embodiment of the present invention, the functionality of the camera service can be expanded, and applications using the camera service can perform screen sharing. The extended functionality of the camera service as a system process can be used to obtain original high-quality screen frames for screen sharing, and the camera-captured images and screen frames can be displayed simultaneously, so that the application does not need to implement the screen sharing function itself. In addition, a better layout can be provided for screen sharing and better adaptation to folding screens.

While the invention has been shown and described with reference to the particular exemplary embodiments, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the invention as defined by the claims and their equivalents.

Claims (18)

1. A method for screen sharing, comprising: When an app uses the camera service, the camera service provides options for users to choose between screen sharing, or image capture and screen sharing. The camera service acquires data based on the function selected by the user and provides the acquired data to the application. The camera service is a system camera service that is a system process, so as to obtain the original screen frame for screen sharing. 2. The method for screen sharing according to claim 1, wherein: When the screen sharing function is selected by the user, the data obtained is the screen frame displayed by another application. When the image capture and screen sharing functions are selected by the user, the acquired data are the camera captured image and the screen frame displayed by the other application. 3. The method for screen sharing of claim 2, wherein the camera service obtains the screen frame from the display rendering service SurfaceFlinger. 4. The method for screen sharing as claimed in claim 2, wherein when the image capture and screen sharing functions are selected by the user, the step of the camera service acquiring data based on the function selected by the user comprises: Determine whether the current electronic device has a foldable screen; Data allocated to the corresponding screen is acquired based on whether the current electronic device has a foldable screen. 5. The method for screen sharing according to claim 4, wherein, when it is determined that the current electronic device has a folding screen, the step of acquiring data assigned to the corresponding screen comprises: Determine whether the foldable screen of the current electronic device is foldable up and down or left and right; The data assigned to the corresponding screen is obtained based on whether the folding screen of the current electronic device is folded up and down or left and right. 6. The method for screen sharing according to claim 4, wherein, when it is determined that the current electronic device does not have a foldable screen, the step of acquiring data assigned to the corresponding screen comprises: Get camera captured images and screen frames; Reduce the brightness difference between the camera captured image and the screen frame; Determines the layout pattern of camera captured images and screen frames on the screen to reduce brightness differences. 7. The method for screen sharing according to claim 6, wherein the step of determining a layout pattern of the camera captured image and the screen frame on the screen that reduces the brightness difference comprises: A layout pattern of camera-captured images and screen frames on a screen that reduces brightness differences is determined through a decision tree model based on predetermined attributes. 8. The method for screen sharing as claimed in claim 7, wherein the predetermined attributes include whether the current electronic device has a folding screen, whether the current electronic device is in a folded state, camera captured image sharpness, screen frame sharpness, the type of the application, and at least one of the type of the other application. 9. A camera service device for screen sharing, comprising: An option providing module, configured to provide options of a screen sharing function, or image capture and screen sharing functions for user selection when an application uses a camera service; a data acquisition module, configured to acquire data based on a function selected by a user, and provide the acquired data to the application, The camera service is a system camera service that is a system process, so as to obtain the original screen frame for screen sharing. 10. The camera service device for screen sharing according to claim 9, wherein: When the screen sharing function is selected by the user, the data obtained is the screen frame displayed by another application. When the image capture and screen sharing functions are selected by the user, the acquired data are the camera captured image and the screen frame displayed by the other application. 11. The camera service device for screen sharing according to claim 10, further comprising: The screen management module is configured to obtain the screen frame from the display rendering service Surface Flinger. 12. The camera service device for screen sharing according to claim 10, wherein when the image capture and screen sharing functions are selected by the user, the data acquisition module is further configured to perform the following operations: Determine whether the current electronic device has a foldable screen; Data allocated to the corresponding screen is acquired based on whether the current electronic device has a foldable screen. 13. The camera service device for screen sharing according to claim 12, wherein, when it is determined that the current electronic device has a foldable screen, the data acquisition module is further configured to perform the following operations: Determine whether the foldable screen of the current electronic device is foldable up and down or left and right; The data assigned to the corresponding screen is obtained based on whether the folding screen of the current electronic device is folded up and down or left and right. 14. The camera service device for screen sharing according to claim 12, wherein when it is determined that the current electronic device does not have a foldable screen, the data acquisition module is further configured to perform the following operations: Get camera captured images and screen frames; Reduce the brightness difference between the camera captured image and the screen frame; Determines the layout pattern of camera captured images and screen frames on the screen to reduce brightness differences. 15. The camera service device for screen sharing according to claim 14, wherein the data acquisition module is further configured to perform the following operations: A layout pattern of camera-captured images and screen frames on a screen that reduces brightness differences is determined through a decision tree model based on predetermined attributes. 16. A camera service device for screen sharing as described in claim 15, wherein the predetermined attributes include whether the current electronic device has a folding screen, whether the current electronic device is in a folded state, camera captured image sharpness, screen frame sharpness, the type of the application, and at least one of the type of the other application. 17. An electronic device comprising a memory and a processor, wherein the memory stores computer executable instructions for a camera service, and when the instructions are executed by the processor, the method according to any one of claims 1 to 8 is executed. 18. A computer-readable medium having computer-executable instructions for a camera service stored thereon, wherein when the instructions are executed, the method according to any one of claims 1 to 8 is executed.