CN111669462B - Method and related device for displaying image - Google Patents

Abstract

A method for displaying images is disclosed, including: an electronic device displays a preview interface, the preview interface includes a first image collected by a first camera, and the first image is displayed in a first area of a display screen; a first user is detected in the preview interface operation, the first image is captured by the first camera, and the second image is captured by the second camera; the first horizontal and vertical screen state of the electronic device is determined according to the second image, and the first image and the first horizontal and vertical screen state are stored in association; The second user operation of the first image determines the second horizontal and vertical screen states of the electronic device, and displays the first image in the second area of the display screen; the first image displayed in the second area on the display screen is displayed relative to the first area The first rotation angle of the second first image is determined by the first horizontal and vertical screen state and the second horizontal and vertical screen state, and the size ratio of the second area is the same as that of the first area. In this way, the horizontal and vertical screen states of the electronic device can be accurately identified.

Description

A method and related device for displaying images

technical field

The present application relates to the field of electronic technology, and in particular, to a method for displaying images and related devices.

Background technique

With the improvement of living standards, the camera function of smartphones is more and more accepted and loved by ordinary users. Smartphones are lighter and easier to control than professional cameras. Users can easily change various camera positions and angles when taking pictures with smartphones to meet different shooting needs.

At present, most mobile devices have built-in gravitational acceleration sensors, and use the gravitational acceleration sensor to detect the horizontal and vertical screen states of the mobile phone, so as to determine the display direction of the photos taken by the user. However, due to technical limitations, the sensor sometimes cannot accurately capture the changes in the horizontal and vertical screen states of the mobile phone. At this time, the problem of inconsistency between the photos displayed by the electronic device and the preview images displayed on the shooting interface seen by the user occurs. For example, when a smartphone user places the mobile phone parallel to the ground plane, or nearly parallel to the ground plane, when taking an overhead shot, the built-in sensor of the mobile phone cannot accurately identify the horizontal and vertical screen status of the mobile phone, which brings trouble to the photographer and causes problems in use. trouble. Of course, the above problems are not limited to aerial shots. The built-in sensor of the mobile phone has a certain probability of being unable to accurately identify the horizontal and vertical screen status of the mobile phone in any scene.

To sum up, in the prior art, the mobile phone cannot accurately identify the horizontal and vertical screen states of the electronic device.

SUMMARY OF THE INVENTION

The embodiments of the present application provide a method and a related apparatus for displaying an image, which can accurately identify the horizontal and vertical screen states of an electronic device.

In a first aspect, the present application provides a method for displaying images, including: an electronic device displays a preview interface, the preview interface includes a first image captured by a first camera, and the first image is displayed in a first area of a display screen; In response to the first user operation, a first user operation is detected, a first image is captured by a first camera, and a second image is captured by a second camera; the second image includes a human face; the first image of the electronic device is determined according to the second image. The horizontal and vertical screen states are stored in association with the first image and the first horizontal and vertical screen states; a second user operation for viewing the first image is detected, and in response to the second user operation, the second horizontal and vertical screen states of the electronic device are determined, and displayed on the display screen The first image is displayed in the second area of It is determined that the size ratio of the second area is the same as that of the first area.

Implementing the embodiment of the present application, when the electronic device captures the first image through the first camera, the second image can be captured through the second camera, and the second image is used to determine the first horizontal and vertical screen state of the electronic device when the first image is captured. When the user views the first image in the second horizontal and vertical screen state through the electronic device, the electronic device determines the rotation angle of the first image displayed by the electronic device according to the first horizontal and vertical screen state and the second horizontal and vertical screen state. The implementation of the method for displaying images provided by the embodiments of the present application is not limited by the performance limitations of traditional sensors, and the horizontal and vertical screen states conforming to the user's behavior intention are acquired by detecting the user's behavior. Therefore, the horizontal and vertical screen states of the electronic device can be accurately identified, and the user's shooting experience can be improved.

In an implementation manner, determining the second horizontal and vertical screen state of the electronic device includes: collecting a third image through a second camera; the third image includes a human face; and determining the second horizontal and vertical screen state of the electronic device according to the third image.

In an implementation manner, the electronic device uses the first parameter to represent the first horizontal and vertical screen state, the first parameter with the first value represents the vertical vertical screen, the first parameter with the second value represents the left horizontal screen, and the third The first parameter of the value represents the downward vertical screen, and the first parameter of the fourth value represents the right horizontal screen; the associated storage of the first image and the first horizontal and vertical screen state includes: associated storage of the first image and the first parameter.

In an implementation manner, determining the first horizontal and vertical screen state of the electronic device according to the second image includes: when the angle between the screen of the electronic device and the horizontal plane is less than a preset angle, determining the first horizontal and vertical screen state of the electronic device according to the second image. Horizontal and vertical screen status.

In an implementation manner, the above-mentioned determining the first horizontal and vertical screen state of the electronic device according to the second image includes: determining the first horizontal and vertical screen state according to the first angle between the standard reference line of the second image and the face reference line corresponding to the second image. Horizontal and vertical screen status.

In an implementation manner, when the second image contains multiple faces, the face reference line corresponding to the second image is the face reference line of the face with the largest area among the multiple faces in the second image.

In an implementation manner, the above-mentioned determining the first horizontal and vertical screen state according to the first angle between the standard reference line of the second image and the face reference line corresponding to the second image includes: when the first included angle is within the first range , it is determined that the first horizontal and vertical screen state is the upward vertical screen; when the first included angle is within the second range, the first horizontal and vertical screen state is determined to be the left horizontal screen; when the first included angle is within the third range, the first horizontal and vertical screen is determined. The state is the downward vertical screen; when the first included angle is within the fourth range, it is determined that the first horizontal and vertical screen state is the right horizontal screen.

In an implementation manner, the orientation of the standard reference line of the second image is from the bottom of the second image to the top of the second image along the side of the second image; the orientation of the face reference line corresponding to the second image It is along the vertical direction of the connecting line between the eyes of the face in the second image, from the chin to the top of the head; the first range is greater than -45° and less than or equal to 45°, the second range is greater than -135° and less than or equal to -45°, and the first range is greater than -45° and less than or equal to 45°. The third range is greater than 135° and less than or equal to 225°, and the fourth range is greater than 45° and less than or equal to 135°.

In an implementation manner, the above-mentioned first parameter is equal to the second rotation angle when the second horizontal screen state is the upward vertical screen; when the second horizontal screen state is the left horizontal screen, the first rotation angle is equal to the first parameter plus 90 °; When the second landscape state is vertical downward, the first rotation angle is equal to the first parameter plus 180°; when the second landscape state is right landscape, the first rotation angle is equal to the first parameter plus -90 °.

In an implementation manner, when the first horizontal and vertical screen state is the upward vertical screen, and the second horizontal screen state is the upward vertical screen, the first rotation angle is equal to 0°; the first horizontal screen state is the left horizontal screen, and the second horizontal screen state is the left horizontal screen. When the screen state is vertical screen up, the first rotation angle is equal to -90°; when the first horizontal screen state is vertical screen down, and the second horizontal screen state is vertical screen up, the first rotation angle is equal to 180°; When the screen state is the right horizontal screen, and the second horizontal screen state is the upward vertical screen, the first rotation angle is equal to 90°.

In an implementation manner, the above-mentioned first camera is a rear camera, and the second camera is a front camera.

In a second aspect, the present application provides an electronic device, comprising: a first camera, a second camera, a display screen, one or more processors, and one or more memories; the one or more memories are used to store a computer Program code, the computer program code includes computer instructions; when the computer instructions are executed on the processor, the electronic device causes the electronic device to execute the method for displaying an image in any possible implementation manner of any of the above aspects.

In a third aspect, an embodiment of the present application provides a computer storage medium, including computer instructions, when the computer instructions are executed on an electronic device, the communication apparatus is made to perform any of the possible implementations of any of the above aspects of displaying an image. method.

In a fourth aspect, the embodiments of the present application provide a computer program product that, when the computer program product runs on a computer, causes the computer to execute the method for displaying an image in any possible implementation manner of any one of the foregoing aspects.

Description of drawings

1 is a schematic diagram of a user interface provided by an embodiment of the present application;

2 is a schematic diagram of a shooting interface provided by an embodiment of the present application;

3 is a schematic diagram of a photo album interface provided by an embodiment of the present application;

4A to 4D are schematic diagrams of a horizontal and vertical screen state according to an embodiment of the present application;

FIG. 5 is a schematic diagram of a coordinate system of an electronic device according to an embodiment of the present application;

6 is a schematic diagram of another horizontal and vertical screen state provided by an embodiment of the present application;

7 to 9 are schematic diagrams of user interfaces for capturing and displaying according to an embodiment of the present application;

10 is a schematic diagram of a standard reference line provided by an embodiment of the present application;

FIG. 11A and FIG. 11B are schematic diagrams of face baselines provided by the embodiments of the present application;

FIG. 12 is a schematic diagram of a reference image provided by an embodiment of the present application;

13 and 14 are schematic diagrams of a first included angle provided by an embodiment of the present application;

15 is a schematic flowchart of a method for displaying an image provided by an embodiment of the present application;

16 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

FIG. 17 is a schematic structural diagram of a software system provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and in detail below with reference to the accompanying drawings. Wherein, in the description of the embodiments of the present application, unless otherwise specified, “/” means or, for example, A/B can mean A or B; “and/or” in the text is only a description of an associated object The association relationship indicates that there can be three kinds of relationships, for example, A and/or B can indicate that A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application , "plurality" means two or more than two.

Hereinafter, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as implying or implying relative importance or implying the number of indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, the "multiple" The meaning is two or more.

The following introduces some exemplary graphical user interfaces (user interfaces, UI) provided by the embodiments of the present application. The term "user interface" in the description, claims and drawings of this application is a medium interface for interaction and information exchange between application programs or operating systems and users, which realizes the internal form of information and the form acceptable to users. conversion between. A commonly used form of user interface is a graphical user interface (graphic user interface, GUI), which refers to a user interface related to computer operations displayed in a graphical manner. It can be an icon, window, control and other interface elements displayed on the display screen of the electronic device, wherein the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, Widgets, etc. visual interface elements.

FIG. 1 illustrates an exemplary user interface 10 on an electronic device 100 for presenting applications installed by the electronic device 100 .

User interface 10 may include a status bar 101, a navigation bar 102, a calendar indicator 103, a weather indicator 104, a tray 105 with icons of frequently used applications, and other application icons. in:

Status bar 101 may include: one or more signal strength indicators 101A of mobile communication signals (also known as cellular signals), operator name (eg, "China Mobile") 101B, wireless fidelity (Wi-Fi) ) signal of one or more signal strength indicators 101C, battery status indicator 101D, time indicator 101E.

The navigation bar 102 may include system navigation keys such as a back key 102A, a home screen key 102B, and a multitasking key 102C. When detecting that the user clicks the back key 102A, the electronic device 100 may display the previous page of the current page. When detecting that the user taps the home key 102B, the electronic device 100 may display a home interface. When detecting that the user clicks on the multitasking key 102C, the electronic device 100 may display the most recently opened tasks of the user. The names of the navigation keys may also be other, which is not limited in this application. Not limited to virtual keys, each navigation key in the navigation bar 102 can also be implemented as physical keys.

Calendar indicator 103 may be used to indicate the current time, such as date, day of the week, hour and minute information, and the like.

The weather indicator 104 can be used to indicate the weather type, such as cloudy to sunny, light rain, etc., and can also be used to indicate information such as temperature.

A tray 105 with icons of frequently used applications may display: a phone icon 105A, a contact icon 105B, a text message icon 105C, a camera icon 105D.

Other application icons can be, for example, the icon 106 of mutual sharing, the icon 107 of gallery, the icon 108 of music, the icon 109 of application, the icon 110 of mailbox, the icon 111 of cloud sharing, the icon 112 of memo, and the icon 113 of settings. User interface 10 may also include page indicators 114 . Other application icons may be distributed across multiple pages, and the page indicator 106 may be used to indicate which page of the application the user is currently viewing. Users can swipe left and right in the area of other application icons to view application icons in other pages.

In some embodiments, the user interface 10 exemplarily shown in FIG. 1 may be a Home screen.

In some other embodiments, the electronic device 100 may further include a front camera. The front camera may also be referred to as a sub-camera, which is mainly located above the screen of the electronic device 100. The front camera may be used for selfies, video calls, and the like.

It can be understood that, FIG. 1 only exemplarily shows a user interface on the electronic device 100, and should not constitute a limitation to the embodiments of the present application.

Exemplarily, as shown in FIG. 2 , the user may click the camera icon 105D on the user interface 10 , the electronic device 100 detects the above-mentioned user operation, and in response to the above-mentioned user operation, the electronic device 100 displays the camera's shooting interface 11 .

The user interface 11 may include: a navigation bar 201 , a shooting control 202 , an album control 203 , a camera switching control 204 , a shooting mode 205 , an image preview area 206 , a flash icon 207 , and a setting icon 208 . in:

The shooting control 202 may receive a user operation (eg, a touch operation), and in response to the detected user operation, the electronic device 100 may use a camera to collect image data and display the above-mentioned image.

The album control 203 can receive a user operation (such as a touch operation), and in response to the detected user operation, the electronic device 100 can display the latest saved photo in the album, and the expression form of the album control 203 can be a thumbnail of the above-mentioned latest photographed photo .

The camera switch control 204 is used to switch cameras. The electronic device 100 may detect a touch operation (eg, a click operation on the camera switching control 204 ) acting on the camera switching control 204 , and in response to the operation, the electronic device 100 may switch cameras. For example, the camera used by the electronic device 100 for shooting is switched from the front camera to the rear camera, or the camera used by the electronic device 100 for shooting is switched from the rear camera to the front camera.

The shooting mode 205 may include: a night scene mode 205A, a professional mode 205B, a photographing mode 205C, a video recording mode 205D, a portrait mode 205E, and the like. The night scene mode 205A, the professional mode 205B and the portrait mode 205E are all photographing modes optimized for specific scenes. Any photographing mode in the above photographing modes 205 can receive a user operation (eg, a touch operation), and in response to the detected user operation, the electronic device 100 can display a photographing interface in the photographing mode.

It can be understood that if the current shooting mode is the shooting mode, the shooting control 204 can be used to shoot a photo; if the current shooting mode is the video recording mode, the shooting control 204 can be used to start or stop the video recording.

The image preview area 206 may be used to display images captured by the front or rear camera of the electronic device 100 (ie, the camera currently used for shooting).

It can be understood that if the camera currently used by the electronic device 100 for shooting is the front camera, the image preview area 206 is used to display the image captured by the front camera of the electronic device 100; if the camera currently used by the electronic device 100 for shooting is the rear camera , the image preview area 206 is used to display the image captured by the rear camera of the electronic device 100 .

The flash icon 207 may receive a user operation (eg, a touch operation), and in response to the detected user operation, the electronic device 100 may display setting options for the flash.

The settings icon 208 may receive a user operation (eg, a touch operation), and in response to the detected user operation, the electronic device 100 may display a settings interface for the camera.

Exemplarily, as shown in FIG. 3 , the user can click on the album control 205 of the camera on the user interface 11, the electronic device 100 detects the above-mentioned user operation, and in response to the above-mentioned user operation, the electronic device 100 displays the latest photographed photos on the album interface 12. .

The album interface 12 may include: a status bar 301 , a navigation bar 302 , an image display area 303 , a back button 304 , a property control 305 , an album control 306 , a share icon 307 , an edit icon 308 , a delete icon 309 , and a more icon 310 . in:

The image display area 303 may be used to display the latest photograph of the electronic device 100 , and may also be used to display other images in the album of the electronic device 100 .

The back key 304 may receive a user operation (eg, a touch operation), and in response to the detected user operation, the electronic device 100 may display a previous page of the current page.

The property control 305 may receive a user operation (eg, a touch operation), and in response to the detected user operation, the electronic device 100 may display the file properties of the image displayed in the image display area 304 . The file properties of an image can include creation time, dimensions, file size, storage path, etc.

In some embodiments of the present application, the file attribute of the above-mentioned image may further include a rotation parameter, and the above-mentioned rotation parameter is used to represent that when the user views the image displayed in the image display area 303 of the display screen in an upward vertical screen, the display screen of the electronic device The orientation of the image displayed in the image display area 303 is a second rotation angle required relative to the orientation of the image displayed in the image preview area 206 during shooting.

In some embodiments of the present application, the file attribute of the above-mentioned image may include an orientation parameter, and the above-mentioned orientation parameter is used to represent the horizontal and vertical screen states of the electronic device 100 when the electronic device 100 captures an image.

It can be understood that the file attribute of the above-mentioned image may only include one of the rotation parameter and the orientation parameter.

The storing of the image in association with the rotation parameter/the orientation parameter may be storing the rotation parameter/the orientation parameter in a file attribute of the image.

The album control 306 may receive a user operation (eg, a touch operation), and in response to the detected user operation, the electronic device 100 may display more images in the album of the electronic device 100 .

The sharing icon 307 may receive a user operation (eg, a touch operation), and in response to the detected user operation, the electronic device 100 may display a sharing interface of the image displayed in the image display area 303 .

The editing icon 308 may receive a user operation (eg, a touch operation), and in response to the detected user operation, the electronic device 100 may display an editing interface of the image displayed in the image display area 303 .

The delete icon 308 may receive a user operation (eg, a touch operation), and in response to the detected user operation, the electronic device 100 may delete the image displayed in the image display area 303 .

The more icons 310 may receive a user operation (eg, a touch operation), and in response to the detected user operation, the electronic device 100 may display one or more function options of the image displayed in the image display area 303 . For example, set as wallpaper, move, amplitude, rename, etc.

In the embodiment of the present application, in the user interface shown in FIG. 2 , the current shooting mode is the shooting mode. The capture control 202 can receive a user operation (eg, a touch operation), and in response to the detected user operation, the electronic device 100 captures a photo and can display the user album interface 12, and the image display area 303 of the user album interface 12 is used to display the above-mentioned photos . Alternatively, the capture control 204 may receive a user operation (eg, a touch operation), and in response to the detected user operation, the electronic device 100 captures and saves a photo, and the electronic device 100 may continue to display the capture interface 11 , and the user can click the capture interface 11 On the album control 203, view the photos taken. This embodiment of the present application does not specifically limit this.

It can be understood that, FIG. 1 to FIG. 3 merely illustrate the user interface on the electronic device 100 by way of example, and should not constitute a limitation to the embodiments of the present application.

The image data acquired by the application camera of the electronic device 100 comes from an image sensor (ImageSensor) of the camera. The user can rotate the electronic device 100 at any angle when taking pictures. The electronic device displays the image captured by the rear camera in the image preview area of the shooting interface; after the electronic device 100 receives the user's shooting operation, in response to the above-mentioned shooting operation, the electronic device 100 saves the above-mentioned image by using. In order to ensure that the orientation of the image displayed by the electronic device 100 is consistent with the orientation of the image that the user sees in the preview image area when the user is viewing the photo, the electronic device 100 usually has a horizontal and vertical screen state of the image when the electronic device 100 takes the photo. The above-mentioned image is displayed on the display screen, so that no matter how the user rotates the electronic device 100 when taking the photo, the above-mentioned image displayed on the display screen can be the same photo as the preview image the user sees in the image preview area when taking the photo. The landscape and portrait screen states of the electronic device 100 may include four states of up portrait, right landscape, down portrait, and left landscape.

Exemplarily, FIGS. 4A to 4D are schematic diagrams of photographing the electronic device 100 in a vertical upward screen, a left horizontal screen, a downward vertical screen, and a right horizontal screen according to an embodiment of the present application. As shown in FIG. 4A to FIG. 4D , when the electronic device 100 takes a photo, the first row of the image data collected by the camera is the first row near the top of the electronic device 100 in the above-mentioned image data, and the first row of the above-mentioned image data is the above-mentioned image data. in the first column near the left of the electronic device 100 . The top, bottom, left, and right sides of the electronic device 100 are fixed, that is, the top, bottom, left, and right sides of the electronic device 100 when the electronic device 100 is vertically screened upward, and the top of the electronic device 100 when the electronic device 100 is vertically screened downward or horizontally. , bottom, left, and right remain unchanged. For example, the top of the electronic device 100 is provided with a camera. Regardless of whether the electronic device has a vertical screen or a horizontal screen, the top of the electronic device 100 refers to the end with the camera.

As shown in FIG. 4A , the electronic device 100 takes a photo in a vertical screen upward, and when a user views the photo in a vertical screen upward, the first row of image data is at the top of the photo, and the first column of image data is at the left of the photo.

As shown in FIG. 4B , the electronic device 100 takes a photo in the left horizontal screen. In order to ensure that the photo displayed by the electronic device 100 is consistent with the image that the user sees in the preview image area when shooting, when the user views the photo vertically upward, the first line of the image data of the photo is located on the left side of the image displayed by the electronic device 100. The first column of image data of the photo is located at the bottom of the image displayed by the electronic device 100. At this time, the above-mentioned image displayed on the display screen of the electronic device 100 is rotated by -90° relative to the image displayed in the preview image area, that is, rotated 90° counterclockwise. .

As shown in FIG. 4C , the electronic device 100 takes a picture with the downward vertical screen. In order to ensure that the photo displayed by the electronic device 100 is consistent with the image that the user sees in the preview image area when the photo is taken, when the user views the photo vertically upwards, the first line of the image data of the photo is located at the bottom of the image displayed by the electronic device 100. The first column of image data of the photo is located to the right of the image displayed by the electronic device 100 . At this time, the above-mentioned image displayed on the display screen of the electronic device 100 is rotated by 180° relative to the image displayed in the preview image area.

As shown in FIG. 4D , the electronic device 100 takes a photo in the right horizontal screen. In order to ensure that the photo displayed by the electronic device 100 is consistent with the image that the user sees in the preview image area when shooting, when the user views the photo vertically upwards, the first line of the image data of the photo is located to the right of the image displayed by the electronic device 100. The first column of image data of the photo is at the top of the image displayed by the electronic device 100 . At this time, the above-mentioned image displayed on the display screen of the electronic device 100 is rotated by 90° relative to the image displayed in the preview image area.

To sum up, when the user is viewing the captured image, the electronic device 100 determines the first rotation angle required by the image to be displayed on the display screen relative to the image displayed in the preview image area according to the horizontal and vertical screen states when the image is captured, Further, the electronic device 100 determines a display area of the image on the display screen according to the first rotation angle, draws the image, and displays the drawn image in the display area.

Regarding how to determine the horizontal and vertical screen states of the electronic device 100, a commonly used technical solution is to use a three-axis gravitational acceleration sensor to determine the gravitational acceleration in the three directions of the X-axis, the Y-axis, and the Z-axis of the electronic device 100. The gravitational acceleration in the three directions determines whether the electronic device 100 is in a horizontal or vertical screen state; alternatively, the angle between the X-axis, Y-axis, and Z-axis of the electronic device 100 and the direction of gravity is determined by using a direction sensor, and the angle is determined by the above-mentioned angle. It is determined whether the electronic device 100 is in a landscape or portrait state.

Exemplarily, as shown in FIG. 5 , the coordinate system of the electronic device 100 is defined in the following manner: the X axis is parallel to the short side direction of the screen of the electronic device 100, from the left side of the screen to the right side of the screen; the Y axis is parallel to the screen The long side direction of the screen points from the bottom of the screen to the top of the screen; the Z axis is perpendicular to the plane formed by the X axis and the Y axis, that is, the Z axis is perpendicular to the plane where the screen is located. When the electronic device 100 is placed horizontally with the screen facing upward, the Z axis is opposite to the direction of gravity, and the gravitational acceleration values of the X, Y, and Z axes are 0, 0, and -9.8 respectively (the gravitational acceleration is 9.8). When the electronic device 100 is turned over, the corresponding gravitational acceleration values of the X-axis, the Y-axis and the Z-axis will also change accordingly.

For example, as shown in FIGS. 4A to 4D , when the electronic device 100 is vertically screened upward, the corresponding gravitational acceleration values of the X-axis, Y-axis, and Z-axis are 0, -9.8, and 0, respectively; when the electronic device 100 is vertically screened, the corresponding The gravitational acceleration values for the X-axis, Y-axis, and Z-axis are 0, 9.8, and 0, respectively. When the electronic device 100 is in the left horizontal screen, the corresponding gravitational acceleration values of the X-axis, Y-axis and Z-axis are -9.8, 0, and 0 respectively; when the electronic device 100 is in the right horizontal screen, the corresponding X-axis, Y-axis and Z-axis The gravitational acceleration values are 9.8, 0, and 0, respectively. It can be understood that when the user holds the electronic device 100, it is not completely consistent with FIG. 4A to FIG. 4D. According to the above-mentioned three-axis gravitational acceleration value belonging to the preset range of the three-axis gravitational acceleration value corresponding to the horizontal screen or the vertical screen, the electronic The device 100 may determine the landscape or portrait state of the electronic device 100 .

As shown in FIGS. 4A to 4D , when the electronic device 100 is vertically screened upward, the angles between the corresponding gravitational acceleration direction and the X-axis, Y-axis, and Z-axis are 90°, -180°, and 90°, respectively; , the angles between the corresponding gravitational acceleration direction and the X-axis, Y-axis and Z-axis are -90°, 180°, and 90°, respectively. When the electronic device 100 is in the left horizontal screen, the corresponding angles of the gravitational acceleration direction and the X axis, the Y axis and the Z axis are -180°, -90° and 90° respectively; when the electronic device 100 is in the right horizontal screen, the corresponding gravitational acceleration The included angles between the direction and the X, Y and Z axes are 180°, 90°, and 90°, respectively. It can be understood that when the user holds the electronic device 100, it is not completely consistent with FIG. 4A to FIG. 4D. According to the angle between the gravitational acceleration direction and the above-mentioned three axes, it belongs to the preset range of the angle between the three axes corresponding to the horizontal screen or the vertical screen. , the electronic device 100 may determine the landscape or portrait state of the electronic device 100 .

Exemplarily, as shown in FIG. 6 , the included angle between the direction of the gravitational acceleration and the Z axis is 90°. When the angle between the gravitational acceleration direction and the Y-axis is between -135° and -180° or between 135° and 180°, the electronic device 100 is in a vertical screen-up state; when the angle between the gravitational acceleration direction and the Y-axis is -45° When between ° and -135°, the electronic device 100 is in the left horizontal screen state; when the angle between the direction of gravitational acceleration and the Y axis is between -45° and 0° or between 0° and 180°, the electronic device 100 is in the horizontal direction. The lower vertical screen state; when the included angle between the gravitational acceleration direction and the Y axis is between 45° and 135°, the electronic device 100 is in the right horizontal screen state.

In addition to the gravitational acceleration sensor, the electronic device 100 can also acquire the gravitational acceleration on the three axes of the electronic device 100 through the magnetic sensor and the acceleration sensor.

When the electronic device 100 is placed horizontally (or nearly horizontally), the direction of the gravitational acceleration g is coincident or nearly coincident with the z-axis in the aforementioned coordinate system, and the component values of the gravitational acceleration vector on the X-axis and the Y-axis are 0 (or close to 0). In the above situation, the electronic device 100 cannot obtain effective gravitational acceleration components of the X-axis and Y-axis accelerations, so that the horizontal and vertical screen states of the electronic device 100 cannot be accurately determined. Therefore, when a user views a photographed photo, the direction of the photograph displayed by the electronic device 100 may be inconsistent with the direction of the preview image seen by the user when photographing, which reduces user experience.

A relatively common shooting scenario in which the above problem occurs is that the user takes an overhead shot of the electronic device 100 parallel to, or nearly parallel to, the ground plane. In fact, in addition to the above-mentioned shooting scenes, under the limitation of the prior art, the built-in gravitational acceleration sensor of the electronic device 100 has a certain probability that it cannot accurately identify the horizontal and vertical screen states of the electronic device 100 in any scene, thereby bringing the user experience Trouble and reduce the user experience.

In order to solve the above problem, an embodiment of the present application provides a method for displaying an image, which can accurately acquire the first horizontal and vertical screen state when the electronic device 100 takes a photo. Furthermore, when the user views the photo, the first rotation angle required for the photo displayed on the display screen relative to the image displayed in the preview image area when the photo is taken can be determined.

When the user wants to use the rear camera of the electronic device 100 to take a photo, the user can open a shooting interface through a camera application (eg, the camera application shown in FIG. 2 ), and the shooting interface can display the first image captured by the rear camera. At this time, the electronic device 100 is in the first horizontal and vertical screen state (for example, the left horizontal screen as shown in FIG. 7 ).

Exemplarily, as shown in FIG. 7 , the user holds the electronic device 100 horizontally to the left, and the image preview area 206 of the shooting interface of the electronic device 100 is used to display the image captured by the rear camera. The current photographing mode of the electronic device 100 shown in FIG. 7 is photographing.

In addition to opening the shooting interface through the camera application, users can also open the shooting interface in other ways. For example, the user can open the shooting interface through the camera function in other applications, and the other applications can be instant messaging software, Meitu software and so on.

In some embodiments of the present application, the electronic device 100 receives a user's shooting operation, and in response to detecting the above shooting operation, the electronic device 100 uses the rear camera to capture the first image, and uses the front camera to capture the first reference image. The first reference image is used to determine the first horizontal and vertical screen states of the electronic device, and the first horizontal and vertical screen states are the horizontal and vertical screen states of the electronic device 100 when shooting. After the electronic device 100 determines the above-mentioned first horizontal and vertical screen state, it saves the first image and the first parameter of the first image, and the first parameter can be used to represent the horizontal and vertical screen state of the electronic device 100 when taking a photo.

Exemplarily, as shown in FIG. 8 , the above shooting operation may be that the user clicks the shooting control 202 on the shooting interface 11 . As shown in FIG. 8 , in response to the user's shooting operation, the front camera captures the first reference image. The electronic device 100 may determine, according to the first reference image, that the first horizontal and vertical screen state is the left horizontal screen.

In some embodiments of the present application, the display screen is configured with a touch panel, which can be used to receive a user's shooting operation. In response to the above shooting operation, the electronic device 100 uses the rear camera to shoot a picture. The above-mentioned photographing operation may refer to a touch operation of a user's hand, elbow, stylus, or the like touching the display screen. For example, click, double-click, long-press, draw specific shapes (like Z, rectangles, circles, etc.) with knuckles, etc. In addition to the touch operation, in a specific implementation, there may also be other ways to implement the above-mentioned user operation. For example, the user can turn on the rear camera for shooting by pressing a button; or turn on the rear camera for shooting by detecting the voice input by the user; or turn on the rear camera for shooting by using a specific gesture. This is not specifically limited in the embodiments of the present application.

In some embodiments of the present application, the electronic device 100 receives a user's viewing operation, and in response to the detection of the above-mentioned viewing operation, the electronic device 100 uses the front camera to capture a second reference image, and determines the position of the electronic device 100 according to the second reference image. The second horizontal and vertical screen state; then, the first rotation angle required for the first image to be displayed is determined according to the first parameter and the second horizontal and vertical screen state, and the above-mentioned first image is displayed according to the above-mentioned first rotation angle.

Exemplarily, as shown in FIG. 9 , after the user takes a photo in the left horizontal screen shown in FIG. 8 , the user wants to view the captured image, the user holds the electronic device in the vertical screen upward, and clicks the album control 203 of the shooting interface 11 . , in response to the above viewing operation, the electronic device 100 determines that the second horizontal and vertical screen state of the electronic device 100 is the left horizontal screen according to the reference image collected by the front camera, and determines the location of the photo according to the first parameter of the image and the second horizontal and vertical screen state. The required first rotation angle is -90°. The electronic device 100 determines the display area 304 of the image on the display screen according to the first rotation angle, draws the image, and then displays the image in the display area 304 of the album interface 12 .

In addition to viewing photos by clicking the album control 203 of the shooting interface 11, the user can also view photos in other ways. There is no specific limitation here. For example, the user can also view photos through the gallery application corresponding to the gallery icon 107 shown in FIG. 1 , and the user can also call the gallery through other applications such as instant messaging software and shopping software, and then view the photos in the gallery.

In some embodiments of the present application, the above-mentioned first parameter may be a direction parameter or a rotation parameter. For example, the orientation parameter may be the parameter "Orientation" in the exchangeable image file format (Exif) of the photo.

The solution proposed in the implementation of the present application is not limited by the performance defects of the built-in sensor of the electronic device 100, and the horizontal and vertical screen states conforming to the user's behavior intention are obtained by detecting the user's behavior. Therefore, by implementing the proposed solution in the present application, the possibility of inaccurate horizontal and vertical screen states can be reduced, and the user's shooting experience can be improved.

The following describes how to determine the horizontal and vertical screen states of the electronic device 100 in the embodiments of the present application.

In this embodiment of the present application, the horizontal and vertical screen states when the electronic device 100 is photographed may be determined according to the first reference image, and the horizontal and vertical screen states when the electronic device 100 receives a viewing operation may be determined according to the second reference image.

In some embodiments of the present application, the electronic device 100 may determine the horizontal and vertical screen states of the electronic device 100 according to a first included angle, where the first included angle is the clip between the standard reference line of the reference image and the face reference line in the reference image horn. The reference image is the first reference image or the second reference image.

In some embodiments of the present application, the standard reference line may be oriented along the side of the first column of the reference image to the top of the reference image (that is, the side of the first row of the reference image), for example, as shown in FIG. 10 shown.

In some embodiments of the present application, the face reference line may be determined according to one or more of the position of the eyes, the position of the nose, and the position of the mouth of the above-mentioned face. For example, the above-mentioned face reference line includes at least two of the middle point of the eyes, the middle point of the nose, and the middle point of the mouth of the human face.

Exemplarily, as shown in FIG. 11A , the orientation of the face reference line may be a vertical line along the connecting line of the eyes, from the chin to the top of the head. For example, the face reference line is a vertical straight line connecting the eyes of the face.

Exemplarily, as shown in FIG. 11B , the orientation of the face reference line may be from the right eye to the left eye along the connection line between the eyes. For example, the face reference line may also be a straight line parallel to the connection line between the eyes.

It should be noted that the orientation of the above-mentioned standard reference line and the orientation of the face reference line are exemplary orientations provided by the embodiments of the present application, and should not be construed as limitations to the embodiments of the present application.

The following describes how to determine the horizontal and vertical screen states of the electronic device 100 according to the reference image in the embodiments of the present application.

(1) First, it is determined whether the reference image contains a human face.

In some embodiments of the present application, the electronic device 100 performs face recognition on the reference image to determine whether the reference image contains a human face. When the reference image includes a human face, the face reference line of the above-mentioned face is determined as the face reference line of the reference image; when the reference image does not include a human face, the electronic device 100 determines the horizontal and vertical screen of the electronic device 100 in a conventional manner. state.

In some embodiments of the present application, the reference image is input into the trained face detection model, and the classification information output by the face detection model is obtained. The classification information is the processing result of the neural network and is used to indicate whether the reference image contains a human face. .

Specifically, the electronic device 100 extracts the face features on the reference image through the face detection model, matches the extracted face features with the face features of the face template, and obtains the correlation between the face image and the face template. value. Then, according to the above correlation value, the classifier is used to judge whether there is a face.

In some embodiments of the present application, the electronic device 100 may use the Viola-Jones face detection algorithm to determine whether the reference image includes a human face. The Viola-Jones face detection algorithm uses Haar features to describe the common features of human faces. For example, in a frontal image, the eye area will be darker than the cheek area, the lip area will also be darker than the surrounding area, but the nose area will be darker than both sides. Cheeks should be bright.

In addition to the above face recognition scheme, other schemes may also be used to determine whether the reference image contains a face, which is not specifically limited in this embodiment of the present application.

In some embodiments of the present application, the electronic device 100 performs face recognition on the reference image to determine whether the reference image contains the face of a preset user. When the reference image contains the face of the preset user, determine that the above-mentioned preset user's face baseline in the reference image is the face baseline of the reference image; when the reference image does not contain the preset user's face, the electronic The device 100 follows the landscape and portrait state of the electronic device 100 in a conventional manner.

It can be understood that the above-mentioned preset user's face image is stored in the electronic device 100 .

Specifically, when the electronic device 100 recognizes that the above-mentioned reference image includes one or more faces, the electronic device 100 can match the above-mentioned one or more face images with the preset user's face image, and when the first face image When the facial feature matching degree with the preset user's face image exceeds a certain threshold, it is determined that the reference image includes the preset user's face, and the one or more face images include the first face image.

(2) When the reference image contains a human face, determine the face reference line of the reference image.

In some embodiments of the present application, when the electronic device 100 recognizes that the reference image includes one or more faces, the electronic device 100 determines the face reference line of the face with the largest area among the one or more faces as the reference The face baseline of the image.

Exemplarily, as shown in FIG. 12 , the reference image includes the face of user 1 and the face of user 2, the face area of user 2 in the reference image is larger than the face area of user 1, and the face benchmark of user 2 is determined. The line is the face reference line of the reference image.

In some embodiments of the present application, the reference image is input into the trained face detection model, and the classification information and fixed-point position information output by the face detection model are obtained, and the classification information and fixed-point position information are the processing results of the neural network. The fixed point position information can be used to indicate the positions of the eyes, nose and mouth in the reference image in the reference image.

In some embodiments of the present application, the reference image is input into the trained face detection model, and the classification information and alignment position information output by the face detection model are obtained. The classification information and position information are the processing results of the neural network. The position information is used to indicate the position of the face reference line in the reference image in the reference image.

The face detection model can be a convolutional neural network (convolutional neuronnetwork, CNN), a multi-task convolutional neural network (Multi-task convolutional neural network, MTCNN), a deep convolutional neural network (deep convolutional neuralnetworks, DCNN), region-based CNN or regions with CNN (RCNN), generative adversarial networks (GAN), super-resolution generative adversarial networks (SRGAN) and other models. There is no specific limitation here.

In some embodiments of the present application, the face detection model may include all or part of the MTCNN model. MTCNN includes four sub-models: face classification model, face frame regression model, face key point location model and face key line location model. Each sub-model is a CNN model. The face classification model outputs two values, which are the probability of a face and the probability of a non-face. The face detection model can judge whether the reference image contains a face according to the output value of the face classification model. The face frame regression model outputs a four-dimensional vector, including the upper left corner coordinates and width and height of the face area. The facial key point location model outputs a ten-dimensional vector, which are the coordinates of the left eye, right eye, nose, left mouth, and right mouth. The face detection model may determine the coordinates of the midpoint of the eyes, the midpoint of the nose and/or the midpoint of the mouth of the face in the reference image according to the output value of the facial key point location model. The face key point location model outputs a four-dimensional vector, the coordinates of any two different points in the face baseline.

It is not limited to the above-mentioned way of determining the face reference line, and other ways may also be used to determine the face reference line in the embodiment of the present application, which is not specifically limited here.

(3) Finally, the electronic device 100 determines the horizontal and vertical screen states of the electronic device 100 according to the face reference line in the reference image.

The following describes how to determine the horizontal and vertical screen states of the electronic device 100 with respect to the two types of face reference lines shown in FIG. 11A and FIG. 11B respectively.

In some embodiments of the present application, the orientation of the face reference line may be from the chin to the top of the head along the vertical line of the eye-connecting line. When the first included angle is between -45° and 45°, the horizontal and vertical screen state of the electronic device 100 is the upward vertical screen. When the first included angle is between 45° and 135°, the horizontal and vertical screen state of the electronic device 100 is the right horizontal screen. When the first included angle is between 135° and 180° or -135° (225°) and -180° (180°), the horizontal and vertical screen state of the electronic device 100 is the downward vertical screen. When the first included angle is between -45° (225°) and -135° (315°), the horizontal and vertical screen state of the electronic device 100 is the left horizontal screen.

Exemplarily, as shown in FIG. 13 , the first included angle is equal to -88°, and the first included angle is between -45° and -135°. When the electronic device 100 determines that when the electronic device 100 collects the reference image, the electronic device 100 The horizontal and vertical screen status is left horizontal screen.

In some embodiments of the present application, the orientation of the face reference line may be from the right eye to the left eye along the binocular connecting line. When the first included angle is between 45° and 135°, the horizontal and vertical screen state of the electronic device 100 is the upward vertical screen. When the first included angle is between 135° and 180° or -135° (225°) and -180° (180°), the horizontal and vertical screen state of the electronic device 100 is the right horizontal screen. When the first included angle is between -45° (225°) and -135° (315°), the horizontal and vertical screen state of the electronic device 100 is the downward vertical screen. When the first included angle is between -45° and 45°, the horizontal and vertical screen state of the electronic device 100 is the left horizontal screen.

Exemplarily, as shown in FIG. 14 , the first included angle is equal to 2°, and the first included angle is between -45° and 45°. The screen status is left horizontal screen.

The following describes how to determine the first parameter of the image according to the horizontal and vertical screen states when the electronic device 100 captures the image, and the first parameter may be an orientation parameter or a rotation parameter.

In some embodiments of the present application, after the electronic device 100 determines the first horizontal and vertical screen state when capturing an image, the orientation parameter of the image is determined according to the first horizontal and vertical screen state. The electronic device 100 stores the above-mentioned image and the orientation parameters of the image.

In some embodiments of the present application, when the horizontal and vertical screen state when the electronic device 100 takes an image is the vertical screen up, the orientation parameter of the photo is equal to 1; when the horizontal and vertical screen state when the electronic device 100 takes an image is the left horizontal screen, the The orientation parameter of the image is equal to 8; the orientation parameter of the image is equal to 3 when the horizontal and vertical screen state when the electronic device 100 captures the image is that the vertical screen is down; The direction parameter is equal to 6.

In this embodiment of the present application, in addition to the above-mentioned representation methods, the direction parameter may also represent the horizontal and vertical screen states of the electronic device 100 by other numerical values or characters, which are not specifically limited here.

In some embodiments of the present application, after the electronic device 100 determines the first horizontal and vertical screen state of the electronic device 100 when capturing an image, the rotation parameter of the image is determined according to the first horizontal and vertical screen state. The electronic device 100 stores the above-mentioned image and the rotation parameters of the image. The above-mentioned rotation parameters are used to represent the second rotation angle required for the electronic device 100 to display the photo when the user views the image in the vertical screen upward.

As shown in FIG. 4A , if the horizontal and vertical screen state of the electronic device 100 is the upward vertical screen when taking a photo, the rotation parameter of the photo should be equal to 0°; as shown in FIG. 4B , if the above-mentioned horizontal and vertical screen state is the left horizontal screen, the photo. The rotation parameter of the photo should be equal to -90°; as shown in Figure 4C, if the above-mentioned horizontal and vertical screen state is the downward vertical screen, the rotation parameter of the photo should be equal to 180°; as shown in Figure 4D, if the above-mentioned horizontal and vertical screen state is left horizontal screen, the rotation parameter of the photo should be equal to 90°.

It can be understood that the face reference line of the first reference image may be the projection of the actual face reference line of the user on the screen of the electronic device 100 , and the standard reference line may be parallel to the side of the electronic device 100 . If the face standard line is perpendicular to the eye connecting line, when the standard reference line is rotated by a first angle relative to the face reference line of the first reference image, in order to ensure that the image viewed by the user through the vertical screen of the electronic device 100 is the same as the time when the image was shot The preview images seen by the user are the same, and the image displayed on the display screen of the electronic device 100 is the image after the preview image is rotated by the first included angle. At this time, the image displayed by the electronic device 100 and the preview image have the same aspect ratio. Since there are only four rotation angles of the image displayed by the electronic device 100, 0°, 90°, 180° and -90° (270°), therefore, when the first included angle is between -45° and -135°, The rotation parameter is equal to -90°; when the first angle is between 135° and 225°, the rotation parameter is equal to 180°; when the first angle is between 45° and 135°, the rotation parameter is equal to 90°.

To sum up, the orientation of the face reference line is from the chin to the top of the head along the vertical line of the connecting line between the eyes, as shown in Table 1, is the first angle, the first horizontal and vertical screen state, Correspondence between orientation parameters and rotation parameters.

Table 1

first angle The first horizontal and vertical screen state Orientation parameter Rotation parameter -45° to 45° vertical screen up 1 0° -135° to -45° left horizontal screen 8 -90° 135° to 225° vertical screen down 3 180° 45° to 135° Right horizontal screen 6 90°

To sum up, when the orientation of the face reference line is from the bottom of the first reference image to the top of the first reference image along the eye-connecting line, as shown in Table 2, it is the first included angle provided by the embodiment of the present application. , the first horizontal and vertical screen state, the corresponding relationship between the orientation parameter and the rotation parameter.

Table 2

first angle The first horizontal and vertical screen state Orientation parameter Rotation parameter 45° to 135° vertical screen up 1 0° -45° to 45° left horizontal screen 8 -90° -135° to -45° vertical screen down 3 180° 135° to 225° Right horizontal screen 6 90°

The following describes how the electronic device 100 determines the first rotation angle required for the first image to be displayed according to the second horizontal and vertical screen state of the electronic device 100 and the first parameter of the image when the user views the captured first image. In the following, the first parameters are respectively introduced as a direction parameter and a rotation parameter.

(1) The first parameter is a direction parameter.

In this embodiment of the present application, the electronic device 100 receives a user's viewing operation, and in response to the detected viewing operation, the electronic device 100 determines the orientation of the first image according to the current horizontal and vertical screen state of the electronic device 100 (ie, the second horizontal and vertical screen state) and the orientation of the first image. parameters, determine the first rotation angle required for the image, and display the first image according to the first rotation angle. The following describes the four possible orientations of the second horizontal and vertical screen state respectively.

In some embodiments of the present application, when the user views the first image in the vertical screen, the electronic device 100 determines that the second horizontal and vertical screen state of the electronic device 100 is the vertical screen up. Referring to FIG. 4A , when the orientation parameter of the image the user intends to view represents the upward vertical screen, the required first rotation angle of the above image is 0°; with reference to FIG. 4B , when the orientation parameter of the image the user intends to view represents the left horizontal screen, the above The first rotation angle required for the image is -90°; with reference to FIG. 4C , when the orientation parameter of the image the user intends to view represents the downward vertical screen, the first rotation angle required for the above image is 180°; with reference to FIG. 4D , the user When the orientation parameter of the image intended to be viewed represents the right horizontal screen, the first rotation angle required for the above image is 90°. When the second horizontal and vertical screen state is the upward vertical screen, the first rotation angle is equal to the second rotation angle.

In some embodiments of the present application, when the user performs a viewing operation on the left horizontal screen, the electronic device 100 determines that the second horizontal and vertical screen state of the electronic device 100 is the left horizontal screen, and the first rotation angle required for the first image may be the user On the basis of the first rotation angle required when viewing the image in vertical screen, rotate it by 90°. Therefore, when the orientation parameter of the first image represents the vertical screen up, the required first rotation angle of the above image is 90°; when the orientation parameter of the first image represents the left horizontal screen, the required first rotation angle of the above image is 0 °; when the orientation parameter of the first image represents the downward vertical screen, the required first rotation angle of the above image is -90°; when the orientation parameter of the first image represents the right horizontal screen, the required first rotation angle of the above image is is 180°.

In some embodiments of the present application, in the same way, when the user performs a viewing operation in the downward vertical screen, the electronic device 100 determines that the second horizontal and vertical screen state of the electronic device 100 is the downward vertical screen, and the first rotation required for the first image The angle may be rotated by 180° on the basis of the first rotation angle required when the user views the image in the vertical screen. Therefore, when the orientation parameter of the first image represents the upward vertical screen, the required first rotation angle of the above image is 180°; when the orientation parameter of the first image represents the left horizontal screen, the required first rotation angle of the above image is 90° °; when the orientation parameter of the first image represents the downward vertical screen, the required first rotation angle of the above image is 0°; when the orientation parameter of the first image represents the right horizontal screen, the required first rotation angle of the above image is -90°.

In some embodiments of the present application, similarly, when the user performs a viewing operation on the right horizontal screen, the electronic device 100 determines that the second horizontal and vertical screen state of the electronic device 100 is the right horizontal screen, and the first rotation angle required for the first image can be It is based on the first rotation angle required by the user to view the image in the vertical screen, and then rotates it by -90°. Therefore, when the orientation parameter of the first image represents the upward vertical screen, the required first rotation angle of the above image is -90°; when the orientation parameter of the first image represents the left horizontal screen, the required first rotation angle of the above image is 180°; when the orientation parameter of the first image represents the downward vertical screen, the required first rotation angle of the above image is 90°; when the orientation parameter of the first image represents the right horizontal screen, the required first rotation angle of the above image is is 0°.

(2) The first parameter is a rotation parameter.

It can be understood that the rotation parameter, that is, the second horizontal and vertical screen state is the second rotation angle required for the image displayed by the electronic device 100 when the vertical screen is upward. The rotation parameter is also essentially used to represent the horizontal and vertical screen state when the photo is taken, and the orientation parameter is different from the rotation parameter.

In this embodiment of the present application, the electronic device 100 receives a user's viewing operation for the first image, and in response to the detected viewing operation, the electronic device 100 determines the second horizontal and vertical screen state of the electronic device 100 and the rotation parameter of the first image to determine The first rotation angle required for the image, and the first image is displayed according to the first rotation angle.

In some embodiments of the present application, when the user views the first image in the vertical screen upward (that is, the second horizontal and vertical screen state is the vertical screen upward), the first rotation angle required for the first image is equal to the rotation parameter. For the rotation parameters, reference may be made to the foregoing embodiments, and details are not described herein again.

Similar to the orientation parameter, in the embodiment of the present application, when the user performs the viewing operation on the left horizontal screen (that is, the second horizontal and vertical screen state is the left horizontal screen), the first rotation angle required for the first image may be the rotation parameter directly plus 90 °. When the user performs a viewing operation on the downward vertical screen (that is, the second horizontal and vertical screen state is the downward vertical screen), the first rotation angle required for the first image may be the rotation parameter directly plus 180°. When the user performs a viewing operation on the right horizontal screen (that is, the second horizontal and vertical screen state is the right horizontal screen), the first rotation angle required for the first image may be the rotation parameter directly plus -90°.

To sum up, as shown in Table 3, it is a correspondence between the second horizontal and vertical screen state, the orientation parameter, the rotation parameter and the first rotation angle required for the photo to be displayed provided in the embodiment of the present application.

table 3

In some embodiments, when the second horizontal and vertical screen state is the preset horizontal and vertical screen state, the first rotation angle required by the electronic device 100 to display the first image is zero. That is, when the second horizontal and vertical screen state is the preset horizontal and vertical screen state, the orientation of the first image displayed by the electronic device 100 is consistent with the orientation of the preview image when the first image is captured. For example, the preset horizontal and vertical screen state is the downward vertical screen, and when the user views the first image in the downward vertical screen, the first rotation angle required for the first image is zero. The preset horizontal and vertical screen states may be set by the electronic device 100 by default, or may be preset by the user.

The method for displaying images provided by the embodiments of the present application will be described in detail below.

FIG. 15 is a schematic flowchart of a method for displaying an image provided by an embodiment of the present application. As shown in FIG. 15 , the method for displaying an image provided by this embodiment of the present application includes, but is not limited to, steps S101 to S108 . A possible implementation manner of this method embodiment is further described below.

S101. The electronic device 100 receives a user's photographing operation.

Exemplarily, as shown in FIG. 8 or FIG. 9 , the electronic device 100 may receive a user's photographing operation through a camera application, and the above photographing operation may be that the user clicks on the photographing control 202 of the photographing interface 11 .

S102. In response to the detected shooting operation, the electronic device 100 uses the first camera to shoot the first image, and obtains the first reference image collected by the second camera.

In some embodiments of the present application, the first camera is a rear camera, and the second camera may be a front camera; in other embodiments, the first camera is a front camera, and the second camera may be a front camera. There is no specific limitation here.

It is not limited to controlling the first camera to perform shooting by clicking the shooting control 202 with the user's finger, and the embodiment of the present application may also control the first camera to perform shooting through other user operations. There is no specific limitation here.

Exemplarily, the first reference image is shown in FIG. 9 .

In some embodiments of the present application, in response to a user's shooting operation, the electronic device 100 uses the first camera to capture the first image, and uses the second camera to capture the first reference image.

In some embodiments of the present application, after the electronic device 100 is powered on, the electronic device 100 uses the second camera to periodically capture images. The above-mentioned first reference image may be an image newly collected by the second camera when the electronic device 100 receives the user's shooting operation.

In some embodiments of the present application, when the electronic device 100 turns on the camera and the camera currently used by the electronic device 100 for taking pictures is the first camera, the electronic device 100 uses the second camera to periodically capture images. The above-mentioned first reference image may be an image newly collected by the second camera when the electronic device 100 receives the user's shooting operation.

In some embodiments of the present application, when the electronic device 100 turns on the camera and the electronic device 100 satisfies the preset condition, the electronic device 100 uses the second camera to capture the first reference image. The above-mentioned first reference image may be an image newly collected by the second camera when the electronic device 100 receives the user's shooting operation.

In some embodiments of the present application, the above-mentioned preset condition may be that the angle between the screen of the electronic device 100 and the horizontal plane is smaller than the preset angle. The above preset condition may also be that the gravitational acceleration of the three axes of the electronic device 100 is within the first preset range. The above preset condition may also be that the angle between the direction of the gravitational acceleration and the three axes of the electronic device 100 is within the second preset range.

For example, the above-mentioned preset angle is equal to 10°, 20° or 30°.

It can be understood that when the electronic device 100 does not meet the preset conditions, the horizontal and vertical screen states of the electronic device 100 can be determined in a conventional manner. For example, the horizontal and vertical screen states of the electronic device 100 are determined by using a gravitational acceleration sensor or an orientation sensor. When the electronic device 100 satisfies the preset condition, that is, when the electronic device 100 is placed nearly horizontally, the solution proposed in the embodiments of the present application can be used to determine the horizontal and vertical screen states of the electronic device 100 .

It should be noted that, the second camera mentioned in the embodiment of the present application may be a low-power camera.

Exemplarily, as shown in FIG. 8 , the user clicks the shooting control 202, and in response to the above-mentioned user operation, the electronic device 100 uses the first camera to capture an image, and uses the second camera to capture the first reference image.

S103. When the first reference image includes a human face, determine the human face reference line of the first reference image.

In some embodiments of the present application, the face reference line may be determined according to one or more of the position of the eyes, the position of the nose, and the position of the mouth of the above-mentioned face. For example, the above-mentioned face reference line includes at least two of the middle point of the eyes, the middle point of the nose, and the middle point of the mouth of the human face.

Exemplarily, as shown in FIG. 11A , the orientation of the face reference line may be from the chin to the top of the head along the above-mentioned vertical line connecting the eyes. Exemplarily, as shown in FIG. 11B , the orientation of the face reference line may also be from the bottom of the first reference image to the top of the first reference image along the eye connecting line.

It can be understood that the orientations of the face baselines shown in FIGS. 11A and 11B are exemplary orientations provided by the embodiments of the present application, and should not be limited to the embodiments of the present application. The face baselines in the embodiments of the present application may also have other orientations.

In some embodiments of the present application, when the first reference image includes a preset user's face, it is determined that the preset user's face reference line in the first reference image is the face reference line of the first reference image.

In some embodiments of the present application, when the first reference image includes one or more faces, the electronic device 100 determines that the face reference line of the face with the largest area among the one or more faces is the person in the first reference image face baseline. Illustratively, as shown in FIG. 12 .

In this embodiment of the present application, if the first reference image does not include a human face, the electronic device 100 may determine the current horizontal and vertical screen states of the electronic device 100 in a conventional manner.

It is not limited to the above-mentioned way of determining the face reference line, and other ways may also be used to determine the face reference line in the embodiment of the present application, which is not specifically limited here. How to determine whether the first reference image contains a human face may refer to the foregoing embodiments, and details will not be repeated here.

S104: Determine the first horizontal and vertical screen state when the electronic device 100 takes a picture according to the first included angle between the standard reference line of the first reference image and the face reference line in the first reference image.

The standard reference line may be oriented along the side of the first column of the first reference image to the top of the first reference image (the first row of the first reference image). Exemplarily, the standard reference line may be as shown in FIG. 10 . The first included angle is an included angle between the standard reference line and the face reference line in the first reference image. Exemplarily, the first included angle may be as shown in FIG. 13 or FIG. 14 .

It can be understood that the first reference image is the first reference image newly collected when the electronic device 100 receives the user's shooting operation, and the horizontal and vertical screen states when the electronic device 100 collects the first reference image can be regarded as the electronic device 100 responding to the above shooting operation. The horizontal and vertical screen state when shooting.

In some embodiments of the present application, when the first included angle is within the first range, it is determined that the first horizontal and vertical screen state is the upward vertical screen; when the first included angle is within the second range, it is determined that the first horizontal and vertical screen state is left Horizontal screen; when the first included angle is within the third range, it is determined that the first horizontal and vertical screen state is the downward vertical screen; when the first included angle is within the fourth range, the first horizontal and vertical screen state is determined to be the right horizontal screen.

In some embodiments of the present application, when the orientation of the face reference line is from the chin to the top of the head along the above-mentioned vertical line connecting the eyes, the corresponding relationship between the first angle and the horizontal and vertical screen states can refer to Table 1. In this case , the first range may be greater than -45° and less than or equal to 45°, the second range may be greater than -135° and less than or equal to -45°, the third range may be greater than 135° and less than or equal to 225°, and the fourth range may be greater than 45° ° is less than or equal to 135°. When the orientation of the face reference line is from the bottom of the first reference image to the top of the first reference image along the eye connecting line, the corresponding relationship between the first angle and the horizontal and vertical screen states can refer to Table 2. It will not be repeated here.

S105 , the electronic device 100 saves the first image and the first parameter corresponding to the image, where the first parameter is used to represent the first horizontal and vertical screen state when the image is captured.

The image saved by the electronic device 100 generally consists of two parts. One part is the image data itself, which records the color value of each pixel in the image; the other part is the file header of the image, which records the attribute information of the image, such as the width and height of the image.

In some embodiments of the present application, the electronic device uses the first parameter to represent the first horizontal and vertical screen state, the first parameter with the first value represents the vertical up screen, the first parameter with the second value represents the left horizontal screen, and the first parameter with the first value represents the left horizontal screen. The first parameter of three values represents the downward vertical screen, and the first parameter of the fourth value represents the right horizontal screen.

In some embodiments of the present application, the first parameter may be a direction parameter. For example, the first parameter is the parameter "Orientation" of Exif in the image file header. In some embodiments, an orientation parameter of 1 indicates a vertical screen up; an orientation parameter of 8 indicates a right landscape screen; an orientation parameter of 3 indicates a vertical screen down; and an orientation parameter of 6 indicates a left landscape screen.

In some embodiments of the present application, the first parameter may be a rotation parameter. 4A to 4D , the rotation parameter is equal to 0°, indicating that the first horizontal and vertical screen state is vertical upward screen; the rotation parameter is equal to -90°, indicating that the first horizontal and vertical screen state is the left horizontal screen; A horizontal and vertical screen state is a downward vertical screen; the rotation parameter is equal to 90°, indicating that the first horizontal and vertical screen state is a right horizontal screen.

In some embodiments of the present application, when the electronic device 100 is turned on, or when the electronic device 100 turns on the camera and the camera currently used by the electronic device 100 to take pictures is the first camera, or when the electronic device 100 turns on the camera and the electronic device 100 satisfies the Under the aforementioned preset conditions, the electronic device 100 uses the second camera to periodically capture images. In the above case, in an implementation manner, the electronic device 100 may generate the first parameter of the image when the second camera periodically captures each image. When the electronic device 100 receives the user's photographing operation, the electronic device 100 directly obtains the first parameter generated by the latest captured image. In this way, the acquisition speed of the first parameter can be improved. In the above case, in another implementation manner, the electronic device 100 generates the first parameter of the latest image captured by the second camera only when the electronic device 100 receives the user's shooting operation. In this way, the power consumption of the electronic device 100 can be saved.

Different from the traditional method for determining the first parameter, the method for determining the first parameter provided by the embodiment of the present application accurately acquires the horizontal and vertical screen states of the electronic device 100 by detecting the user's behavioral intention, and then determines the first parameter. Therefore, when a user views a photo, a photo that conforms to the user's behavioral intent can be displayed.

S106, the electronic device 100 receives the user's viewing operation.

Exemplarily, as shown in FIG. 15 , the above-mentioned viewing operation may be that the user clicks on the album control 203 on the shooting interface 11 .

S107. In response to the detected viewing operation, the electronic device 100 determines a first rotation angle required for the first image to be displayed according to the first parameter of the electronic device 100 and the current horizontal and vertical screen states of the electronic device 100.

The current horizontal and vertical screen states of the electronic device 100 may refer to the second horizontal and vertical screen states in the foregoing embodiments.

In some embodiments of the present application, when the electronic device 100 receives the user's viewing operation, if the electronic device 100 satisfies the foregoing preset conditions, the electronic device 100 determines the second horizontal and vertical orientation of the electronic device 100 through the reference image collected by the second camera. screen state; if the electronic device 100 does not meet the aforementioned preset conditions, the electronic device 100 determines the second horizontal and vertical screen state of the electronic device 100 in a conventional manner. In some embodiments, the electronic device 100 determines the second horizontal and vertical screen state of the electronic device 100 in a conventional manner, without considering the aforementioned preset conditions.

In some embodiments of the present application, in response to the detected viewing operation, the electronic device 100 acquires a second reference image captured by the second camera, and determines a second horizontal and vertical screen state of the electronic device 100 according to the second reference image. Then, the electronic device 100 determines the first rotation angle required for the image to be displayed according to the first parameter of the electronic device 100 and the second horizontal and vertical screen states of the electronic device 100 .

Similar to the first horizontal and vertical screen state, the electronic device 100 may determine the second horizontal and vertical screen state of the electronic device 100 according to the angle between the standard reference line of the second reference image and the face reference line of the second reference image. It will not be repeated here.

Referring to the foregoing embodiment, when the first parameter is an orientation parameter or a rotation parameter, the corresponding relationship between the first parameter, the second horizontal and vertical screen state of the electronic device 100 and the first rotation angle required for the photo to be displayed may be as shown in Table 3.

S108 , the electronic device 100 displays the first image according to the first rotation angle required by the first image.

The electronic device 100 determines a display area of the first image on the display screen according to the above-mentioned first rotation angle, draws the first image, and displays the drawn first image in the above-mentioned display area.

In this embodiment of the present application, the shooting interface may also be referred to as a preview interface, and the preview image area of the shooting interface may also be referred to as the first area; the shooting operation may also be referred to as the first user operation; the viewing operation may also be referred to as The second user operation; the first reference image may also be referred to as the second image; the image display area of the album interface may also be referred to as the second area; the second reference image may also be referred to as the third image. There is no specific limitation here.

Next, the electronic devices involved in the embodiments of the present application are introduced. This application does not specifically limit the types of electronic devices mentioned, and the electronic device 100 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, and a laptop computer (laptop). . Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices powered by iOS, android, microsoft, or other operating systems. The portable electronic device described above may also be other portable electronic devices, such as a laptop or the like having a touch-sensitive surface (eg, a touch panel). It should also be understood that, in some other embodiments of the present application, the electronic device may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (eg, a touch panel), or a smart TV or the like.

Referring to FIG. 16 , FIG. 16 shows a schematic structural diagram of an exemplary electronic device 100 provided by an embodiment of the present application.

The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, and Subscriber identification module (SIM) card interface 195 and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.

It can be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or less components than shown, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor ( image signal processor, ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and/or neural-network processing unit (NPU), etc. . Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

The controller may be the nerve center and command center of the electronic device 100 . The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.

The NPU can perform artificial intelligence operations using Convolutional Neural Networks (CNN) processing. For example, using the CNN model to do a lot of information identification and information screening can realize the training and identification of situational intelligence.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby increasing the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuitsound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver (universal asynchronous receiver) interface /transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and/or Universal serial bus (universal serial bus, USB) interface, etc.

The charging management module 140 is used to receive charging input from the charger. The charger may be a wireless charger or a wired charger.

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives input from the battery 142 and/or the charging management module 140 and supplies power to the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 .

The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be provided in the processor 110 . In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.

The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, the baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide wireless communication solutions including 2G/3G/4G/5G etc. applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and then turn it into an electromagnetic wave for radiation through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the same device as at least part of the modules of the processor 110 .

The modem processor may include a modulator and a demodulator. Wherein, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and passed to the application processor. The application processor outputs sound signals through audio devices (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or videos through the display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processor 110, and may be provided in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), and global navigation satellite systems applied on the electronic device 100 . (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , perform frequency modulation on it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2 .

In some embodiments, the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code Wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM , and/or IR technology, etc.

The electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light-emitting diode). , AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diodes (quantum dot light emitting diodes, QLED) and so on. In some embodiments, the electronic device 100 may include one or N display screens 194 , where N is a positive integer greater than one.

In some embodiments of the present application, the display screen 194 displays the interface content currently output by the system. For example, the interface content is an interface provided by an instant messaging application.

The electronic device 100 may implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like. The ISP is used to process the data fed back by the camera 193 .

Camera 193 is used to capture still images or video. The object is projected through the lens to generate an optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

A digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy and so on.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, for example, moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.

The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process the input information, and can continuously learn by itself. Applications such as intelligent cognition of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

Internal memory 121 may be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121 . The internal memory 121 may include a storage program area and a storage data area. The storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like. The storage data area may store data (such as audio data, phone book, etc.) created during the use of the electronic device 100 and the like. In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.

The electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playback, recording, etc.

The audio module 170 is used for converting digital audio information into analog audio signal output, and also for converting analog audio input into digital audio signal. Audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110 , or some functional modules of the audio module 170 may be provided in the processor 110 .

Speaker 170A, also referred to as a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.

The receiver 170B, also referred to as "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a call or a voice message, the voice can be answered by placing the receiver 170B close to the human ear.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.

The earphone jack 170D is used to connect wired earphones.

The pressure sensor 180A is used to sense pressure signals, and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be provided on the display screen 194 . In some optional embodiments of the present application, the pressure sensor 180A can be used to capture the pressure value generated when the user's finger part touches the display screen, and transmit the pressure value to the processor, so that the processor can identify which finger part the user inputs through User action.

There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, and the like. The capacitive pressure sensor may be comprised of at least two parallel plates of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example, when a touch operation whose intensity is less than the first pressure threshold acts on the short message application icon, the instruction for viewing the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

In some optional embodiments of the present application, the pressure sensor 180A can transmit the detected capacitance value to the processor, so that the processor can identify which finger part (knuckle or finger pad, etc.) the user uses to input the user operation. In some optional embodiments of the present application, the pressure sensor 180A may also calculate the number of touch points according to the detected signal, and transmit the calculated value to the processor, so that the processor recognizes that the user inputs the user through a single finger or multiple fingers operate.

The gyro sensor 180B may be used to determine the motion attitude of the electronic device 100 . In some embodiments, the angular velocity of electronic device 100 about three axes (ie, x, y, and z axes) may be determined by gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse motion to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenarios.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The terminal 100 can detect the opening and closing of the flip holster using the magnetic sensor 180D. In some embodiments, when the terminal 100 is a flip machine, the terminal 100 can detect the opening and closing of the flip according to the magnetic sensor 180D. Further, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, characteristics such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal 100 in various directions (generally three axes). When the terminal 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc. In some optional embodiments of the present application, the acceleration sensor 180E can be used to capture the acceleration value generated when the user's finger part touches the display screen, and transmit the acceleration value to the processor, so that the processor can identify which finger part the user inputs through User action.

Distance sensor 180F for measuring distance. The terminal 100 can measure the distance through infrared or laser. In some embodiments, when shooting a scene, the terminal 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The terminal 100 emits infrared light to the outside through light emitting diodes. The terminal 100 detects infrared reflected light from nearby objects using a photodiode. When sufficient reflected light is detected, it can be determined that there is an object near the terminal 100 . When insufficient reflected light is detected, the terminal 100 may determine that there is no object near the terminal 100 . The terminal 100 can use the proximity light sensor 180G to detect that the user holds the terminal 100 close to the ear to talk, so as to automatically turn off the display screen to save power. Proximity light sensor 180G can also be used in holster mode, pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense ambient light brightness. The terminal 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal 100 is in a pocket, so as to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The terminal 100 can use the collected fingerprint characteristics to unlock the fingerprint, access the application lock, take a picture with the fingerprint, answer the incoming call with the fingerprint, and the like.

The temperature sensor 180J is used to detect the temperature. In some embodiments, the terminal 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold value, the terminal 100 reduces the performance of the processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the terminal 100 heats the battery 142 to avoid abnormal shutdown of the terminal 100 due to low temperature. In some other embodiments, when the temperature is lower than another threshold, the terminal 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

Touch sensor 180K, also called "touch panel". The touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation acting on or near it, and the touch touch operation refers to an operation of a user's hand, elbow, stylus, etc. touching the display screen 194 . The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations may be provided through display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the location where the display screen 194 is located.

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the pulse of the human body and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined with the bone conduction earphone. The audio module 170 can analyze the voice signal based on the vibration signal of the vocal vibration bone block obtained by the bone conduction sensor 180M, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M, and realize the function of heart rate detection.

The keys 190 include a power-on key, a volume key, and the like. Keys 190 may be mechanical keys. It can also be a touch key. The terminal 100 may receive key input and generate key signal input related to user settings and function control of the terminal 100 .

Motor 191 can generate vibrating cues. The motor 191 can be used for vibrating alerts for incoming calls, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. The motor 191 can also correspond to different vibration feedback effects for touch operations on different areas of the display screen 194 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, which can be used to indicate the charging state, the change of the power, and can also be used to indicate a message, a missed call, a notification, and the like.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be contacted and separated from the terminal 100 by inserting into the SIM card interface 195 or pulling out from the SIM card interface 195 .

The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take an Android system with a layered architecture as an example to exemplarily describe the software structure of the electronic device 100 .

FIG. 17 is a block diagram of the software structure of the electronic device 100 according to the embodiment of the present application.

The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, which are, from top to bottom, an application layer, an application framework layer, an Android runtime (Android runtime) and system libraries, and a kernel layer.

The application layer can include a series of application packages.

As shown in Figure 17, the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and so on.

In the present application, a floating launcher component may be added to the application layer, which is used to display the application in the above-mentioned floating window by default, and provides the user with an entrance to other applications.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for the applications of the application layer. The application framework layer includes some predefined functions.

As shown in FIG. 17, the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, an activity manager, and the like.

A window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the display screen, take a screenshot of the display screen, etc. In this application, a FloatingWindow can be extended based on Android's native PhoneWindow, which is specially used to display the above-mentioned floating window, so as to be different from an ordinary window, and the window has the property of being displayed floating at the top level of a series of windows. In some optional embodiments, the window size can be given an appropriate value according to the size of the actual screen and an optimal display algorithm. In some possible embodiments, the aspect ratio of the window may be the screen aspect ratio of a conventional mainstream mobile phone by default. At the same time, in order to facilitate the user to close and exit and hide the floating window, an additional close button and a minimize button can be drawn in the upper right corner.

Content providers are used to store and retrieve data and make these data accessible to applications. The data may include video, images, audio, calls made and received, viewing history and bookmarks, phone book, etc.

The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. View systems can be used to build applications. A display interface can consist of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures. In the present application, a button view for closing, minimizing and other operations on the floating window can be correspondingly added, and bound to the FloatingWindow in the above-mentioned window manager.

The phone manager is used to provide the communication function of the electronic device 100 . For example, the management of call status (including connecting, hanging up, etc.).

The resource manager provides various resources for the application, such as localization strings, icons, pictures, layout files, video files and so on.

The notification manager enables the application to display notification information in the status bar 207, which can be used to convey notification-type messages, and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also display notifications in the status bar at the top of the system in the form of graphs or scroll bar text, such as notifications from applications running in the background, and notifications on the display in the form of dialog windows. For example, text information is prompted in the status bar, a prompt sound is issued, the electronic device vibrates, and the indicator light flashes.

The activity manager is used to manage the activities running in the system, including process, application, service, task information and so on. In this application, an activity task stack dedicated to managing the application activity displayed in the above floating window can be added to the activity manager module, so as to ensure that the application activity and task in the floating window will not be generated from the application displayed in full screen on the screen. conflict.

In the present application, a motion detector may be added to the application framework layer, which is used to perform logical judgment on the acquired input events and identify the type of the input events. For example, it is determined that the input event is a knuckle touch event or a finger pad touch event, etc., according to the touch coordinates included in the input event, the time stamp of the touch operation, and other information. At the same time, the motion detection component can also record the trajectory of the input event, determine the gesture rule of the input event, and respond to different operations according to different gestures.

Android Runtime includes core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

The core library consists of two parts: one is the function functions that the java language needs to call, and the other is the core library of Android.

The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, safety and exception management, and garbage collection.

A system library can include multiple functional modules. For example: input manager (input manager), input dispatcher (input dispatcher), surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg : SGL) etc.

The input manager is responsible for obtaining event data from the underlying input driver, parsing and encapsulating it, and passing it to the input dispatch manager.

The input scheduling manager is used to store window information. After it receives the input event from the input manager, it will look for a suitable window in its managed window and dispatch the event to this window.

The Surface Manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications.

The media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer contains at least display drivers, camera drivers, audio drivers, and sensor drivers.

In the following, the workflow of the software and hardware of the electronic device 100 is exemplarily described in conjunction with the capturing and photographing scene.

When the touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes touch operations into raw input events (including touch coordinates, timestamps of touch operations, etc.). Raw input events are stored at the kernel layer. The application framework layer obtains the original input event from the kernel layer, and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation, and the control corresponding to the click operation is the control of the camera application icon, for example, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer. The camera 193 captures still images or video.

From the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above functional modules is used as an example for illustration. In practical applications, the above functions can be allocated as required. It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. For the specific working process of the system, apparatus and unit described above, reference may be made to the corresponding process in the foregoing method embodiments, and details are not described herein again.

Each functional unit in each of the embodiments of the embodiments of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage The medium includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: flash memory, removable hard disk, read-only memory, random access memory, magnetic disk or optical disk and other media that can store program codes.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this, and any changes or substitutions within the technical scope disclosed in the present application should be covered within the protection scope of the present application. . Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims (13)

1. A method for displaying an image, comprising: The electronic device displays a preview interface, the preview interface includes a first image captured by a first camera, and the first image is displayed in a first area of the display screen; A first user operation is detected in the preview interface, and in response to the first user operation, the first image is captured by the first camera, and the second image is captured by the second camera; When the second image includes the face of the preset user, determine the first horizontal and vertical screen state of the electronic device according to the second image; when the second image does not include the face of the preset user , using the sensor to determine the first horizontal and vertical screen state of the electronic device; storing the first image and the first horizontal and vertical screen state in association, and the first horizontal and vertical screen state includes: left horizontal screen or right horizontal screen; Detecting a second user operation for viewing the first image, determining a second horizontal and vertical screen state of the electronic device in response to the second user operation, and displaying the first image in a second area of the display screen image; the first rotation angle of the first image displayed in the second area on the display screen relative to the first image displayed in the first area is determined by the first horizontal and vertical screen state and the first Two horizontal and vertical screen states are determined, the second horizontal and vertical screen state includes: right horizontal screen or left horizontal screen, the first rotation angle includes 180°, the size ratio of the second area and the size ratio of the first area same. 2. The method according to claim 1, wherein the determining the second horizontal and vertical screen state of the electronic device comprises: A third image is collected by the second camera; the third image includes a human face; A second horizontal and vertical screen state of the electronic device is determined according to the third image. 3 . The method according to claim 1 , wherein the electronic device represents the first horizontal and vertical screen state by a first parameter, the first parameter with a first value represents the vertical screen upward, and the second The first parameter of the value represents the left horizontal screen, the first parameter of the third value represents the downward vertical screen, and the first parameter of the fourth value represents the right horizontal screen; The associated storage of the first image and the first horizontal and vertical screen state includes: The first image and the first parameter are stored in association. 4. The method according to claim 1, wherein the determining the first horizontal and vertical screen state of the electronic device according to the second image comprises: When the included angle between the screen of the electronic device and the horizontal plane is smaller than a preset angle, the first horizontal and vertical screen state of the electronic device is determined according to the second image. 5. The method according to claim 1, wherein the determining the first horizontal and vertical screen state of the electronic device according to the second image comprises: The first horizontal and vertical screen state is determined according to the first included angle between the standard reference line of the second image and the face reference line corresponding to the second image. 6 . The method according to claim 5 , wherein when the second image includes multiple faces, the face reference line corresponding to the second image is the multiple faces of the second image. 7 . The face baseline of the face with the largest area. 7 . The method according to claim 5 or 6 , wherein the first angle is determined according to a standard reference line of the second image and a face reference line corresponding to the second image. 8 . A horizontal and vertical screen state, including: When the first included angle is within the first range, it is determined that the first horizontal and vertical screen state is the upward vertical screen; When the first included angle is within the second range, determine that the first horizontal and vertical screen state is the left horizontal screen; When the first included angle is within the third range, it is determined that the first horizontal and vertical screen state is the downward vertical screen; When the first included angle is within the fourth range, it is determined that the first horizontal and vertical screen state is the right horizontal screen. 8 . The method according to claim 7 , wherein the orientation of the standard reference line of the second image is from the bottom of the second image to the second image along the side of the second image. 9 . The top of the image; the orientation of the face reference line corresponding to the second image is along the vertical direction of the connecting line between the eyes of the face in the second image, from the chin to the top of the head; the first range is greater than -45 ° is less than or equal to 45°, the second range is greater than -135° and less than or equal to -45°, the third range is greater than 135° and less than or equal to 225°, and the fourth range is greater than 45° and less than or equal to 135°. 9 . The method according to claim 3 , wherein the first parameter is equal to the second rotation angle when the second horizontal and vertical screen state is the upward vertical screen; the second horizontal and vertical screen state is the left horizontal screen. 10 . , the first rotation angle is equal to the first parameter plus 90°; when the second horizontal and vertical screen state is the downward vertical screen, the first rotation angle is equal to the first parameter plus 180°; When the second horizontal and vertical screen state is the right horizontal screen, the first rotation angle is equal to the first parameter plus -90°. 10. method according to claim 1, is characterized in that, described first horizontal and vertical screen state is upward vertical screen, when described second horizontal and vertical screen state is upward vertical screen, described first rotation angle is equal to 0 °; When the first horizontal and vertical screen state is the left horizontal screen, and the second horizontal and vertical screen state is the upward vertical screen, the first rotation angle is equal to -90°; The first horizontal and vertical screen state is a downward vertical screen, and when the second horizontal and vertical screen state is an upward vertical screen, the first rotation angle is equal to 180°; When the first horizontal and vertical screen state is the right horizontal screen, and the second horizontal and vertical screen state is the upward vertical screen, the first rotation angle is equal to 90°. 11. The method of claim 1, wherein the first camera is a rear camera, and the second camera is a front camera. 12. An electronic device, comprising: a first camera, a second camera, a display screen, one or more processors, and one or more memories, wherein the one or more memories are used to store computer programs Code, the computer program code comprising computer instructions; when executed on the one or more processors, the computer instructions cause the electronic device to perform the method of any of claims 1-11. 13. A computer-readable storage medium, characterized by comprising computer instructions that, when the computer instructions are executed on an electronic device, cause the electronic device to perform the method according to any one of claims 1-11 .

Images