CN116074624B - A focusing method and device - Google Patents

Abstract

Embodiments of the present application provide a focusing method and device, which relate to the field of terminals and can focus more accurately and reasonably, making the focus of the picture more prominent and improving user experience. The method is: display the current viewfinding picture and determine whether the current viewfinder picture belongs to a multi-depth-of-field scene; if it is determined that the current viewfinder picture belongs to a multi-depth-of-field scene, focus according to the attention position of the target face in the current viewfinder picture. The current viewfinder picture belongs to a multi-depth-of-field scene. A depth-of-field scene indicates that multiple subjects in the current viewfinder are at different depth positions; if it is determined that the current viewfinder does not belong to a multi-depth-of-field scene, the photographer's attention position is detected, and the focus is performed based on the photographer's attention position. Not belonging to a multi-depth-of-field scene means that multiple subjects in the current viewfinder are at the same or similar depth positions.

Description

A focusing method and device

Technical field

The present application relates to the field of terminals, and in particular, to a focusing method and device.

Background technique

The traditional focusing system focuses based on the central area of the picture. When the subject of interest deviates from the central area of the picture, out-of-focus problems often occur. For example, as shown in Figure 1, during actual shooting, if the focus frame is in the center of the screen, the subject of interest (flowers) will be out of focus. In order to solve this problem, an intelligent focusing system based on subject object detection has emerged.

The implementation principle of the intelligent focus system based on subject object detection is: detect the subject in the viewfinder (such as flowers, faces), and then send the subject object frame to the focusing system for focusing. Since the subject can be in different positions in the image The position and therefore the focus area also changes.

However, for content-rich scenes, there may be multiple subject object frames in the viewfinder, and how to focus becomes a new problem.

Contents of the invention

Embodiments of the present application provide a focusing method and device, which can focus more accurately and reasonably, making the focus of the picture more prominent and improving user experience.

In the first aspect, embodiments of the present application provide a focusing method, which is applied to electronic devices, including: displaying the current viewfinding picture, and determining whether the current viewfinding picture belongs to a multi-depth-of-field scene; if it is determined that the current viewfinder picture belongs to a multi-depth-of-field scene, based on the current viewfinder picture Focus on the target face's attention position in the scene. If the current viewfinder picture belongs to a multi-depth-of-field scene, it means that multiple subjects in the current viewfinder picture are at different depth positions; if it is determined that the current viewfinder picture does not belong to a multi-depth-of-field scene, detect the photographer's Attention position, focus is performed based on the photographer's attention position. The current viewfinder does not belong to a multi-depth scene, which means that multiple subjects in the current viewfinder are at the same or similar depth positions.

Based on the focusing method provided by the embodiment of the present application, different focusing schemes can be adopted according to whether the current viewing image is a multi-depth-of-field scene. For example, when the current viewing image belongs to a multi-depth-of-field scene, the direction of attention of the person (target face) in the image can be used as the focus direction. Due to the complex content of multi-depth scenes, objects in the picture (for example, people or objects) have different depths of field. It is difficult for people to understand complex scenes and quickly find the key points in the picture. In this case, it is not enough to use the photographer's attention position as the focus position. Accurate and fast. Therefore, the focus direction can be based on the attention direction of the person (target face) in the picture, which is more accurate and reasonable, can make the focus of the picture more prominent, be more in line with the expression of the lens language, and improve the user experience. On the contrary, when the current view frame does not belong to a multi-depth-of-field scene (that is, a non-multi-depth-of-field scene), focusing can be performed based on the photographer's attention position. Since the content of the non-multi-depth-of-field scene is simple and the objects (for example, people or objects) in the picture are at similar or similar depths, it is easier for people to understand simple scenes, and it is easier to find the key points in the picture. In this case, based on the photographer Focusing on the focus position can make the focus of the picture more prominent, more consistent with the expression of the lens language, and improve the user experience.

In a possible implementation, if the photographer's attention position cannot be detected, focusing is performed based on the target face in the current viewfinder, or focusing is performed based on the central area in the current viewfinder.

In one possible implementation, if the attention position of the target face is not within the current viewing frame, focus is performed based on the area where the target face is located. That is, when the convergence area of the sight direction of the target face in the viewfinder exceeds the viewfinder, a face focusing solution can be used, that is, focusing based on the target face. The target face can be the face with the largest area in the viewfinder. If the target face's attention position is not within the current viewing frame, the target face is likely to be an object of greater interest to the user, so it is reasonable and accurate to focus based on the area where the target face is located.

In a possible implementation, determining whether the current view frame belongs to a multi-depth scene includes: obtaining the phase difference sequence of the current view frame; determining the maximum value and minimum value in the phase difference sequence; and determining the difference between the maximum value and the minimum value. The absolute value of the difference is greater than the preset threshold; if the absolute value of the difference is greater than the preset threshold, it is determined that the current viewing image belongs to a multi-depth scene; if the absolute value of the difference is less than or equal to the preset threshold, it is determined that the current viewing image does not belong to a multi-depth scene. Depth of field scene. Since most current cameras are equipped with phase detection pixels, they can support phase detection autofocus (PDAF) technology. That is, the image captured by the camera can be processed by the PDAF algorithm to obtain a phase difference (PD) sequence, and focusing can be performed based on the phase difference sequence. Moreover, the imaging effect using PDAF technology for focusing is good, the cost is lower than laser imaging, and it is not limited by distance. Therefore, judging whether an image is a multi-depth-of-field image based on the phase difference sequence obtained by PDAF technology has a high degree of adaptability and can be adapted to cameras or imaging products from most manufacturers on the market.

In a possible implementation, obtaining the phase difference sequence of the current viewing image includes: obtaining the phase difference original (PD raw) image corresponding to the current viewing image from the camera; inputting the PD raw image into the phase detection library PDLIB; PDLIB through the gain The picture shows the multiple phase difference PD pixels corresponding to the PD raw image to increase the gain; multiple PD values are calculated based on the multiple PD pixels after the gain is increased, and the multiple PD values form a phase difference sequence.

In a possible implementation, the target face is determined based on at least one of the size of the face frame in the current viewing frame, the weight ratio of the face to the center position, and the distance between the face and the camera.

In a possible implementation, if the current viewfinder includes multiple faces, the target face is the face with the largest area in the current viewfinder.

In a second aspect, the present application provides a chip system, which includes one or more interface circuits and one or more processors. The interface circuit and the processor are interconnected through lines. The above chip system can be applied to electronic devices including communication modules and memories. The interface circuit is configured to receive signals from the memory of the electronic device and send the received signals to the processor, the signals including computer instructions stored in the memory. When the processor executes the computer instructions, the electronic device can perform the method described in the first aspect and any possible design manner thereof.

In a third aspect, the present application provides a computer-readable storage medium that includes computer instructions. When the computer instructions are run on an electronic device (such as a mobile phone), the electronic device is caused to execute the method described in the first aspect and any possible design manner thereof.

In a fourth aspect, embodiments of the present application provide a focusing device, including a processor. The processor is coupled to a memory. The memory stores program instructions. When the program instructions stored in the memory are executed by the processor, the device achieves the first step described above. Aspects and methods described in any possible design method. The device may be an electronic device; or may be a component of the electronic device, such as a chip.

In the fifth aspect, embodiments of the present application provide a focusing device. The device can be divided into different logical units or modules according to functions, and each unit or module performs different functions, so that the device performs the above first aspect and any possible design method.

In a sixth aspect, the present application provides a computer program product, which when the computer program product is run on a computer, causes the computer to execute the method described in the first aspect and any possible design manner thereof.

It can be understood that the chip system described in the second aspect, the computer-readable storage medium described in the third aspect, the devices described in the fourth and fifth aspects, and the computer program product described in the sixth aspect are provided above. The beneficial effects that can be achieved can be referred to the beneficial effects in the first aspect and any possible design method thereof, which will not be described again here.

Description of the drawings

Figure 1 is a schematic diagram of a center focusing provided by an embodiment of the present application;

Figure 2 is a schematic diagram of a depth of field provided by an embodiment of the present application;

Figure 3 is a schematic diagram of a non-multi-depth-of-field scene provided by an embodiment of the present application;

Figure 4 is a schematic diagram of a multi-depth-of-field scene provided by an embodiment of the present application;

Figure 5 is a pixel diagram of phase detection focusing provided by an embodiment of the present application;

Figure 6A is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application;

Figure 6B is a schematic diagram of the software architecture of an electronic device provided by an embodiment of the present application;

Figure 6C is a schematic flowchart of a focusing method provided by an embodiment of the present application;

Figure 7 is a schematic display diagram provided by an embodiment of the present application;

Figure 8 is a schematic flowchart of calculating a PD value provided by an embodiment of the present application;

Figure 9 is a schematic diagram of face focusing provided by an embodiment of the present application;

Figure 10 is a schematic diagram of focusing based on the attention position of a character in the picture provided by an embodiment of the present application;

Figure 11 is a schematic diagram of focusing based on the photographer's attention position provided by an embodiment of the present application;

Figure 12 is a schematic structural diagram of a chip system provided by an embodiment of the present application.

Detailed ways

In order to describe the following embodiments clearly and concisely, a brief introduction to related concepts or technologies is first given:

Depth of field (DOF): refers to the distance range in front and behind the subject measured by imaging that can obtain a clear image at the front edge of the camera lens or other imager.

For example, when the camera focuses on object A and takes a picture, not only object A is clear, but also object B which is some distance (b) in front of object A (between object A and the camera) is also clear, and which is some distance (c) behind object A. Object C is also clear, and b+c can be called depth of field.

As shown in (1) in Figure 2, the shooting device (for example, a mobile phone) is shooting a television (the object being photographed). The clear part in the picture is the depth of field (the clear part (for example, the television) is represented by a solid line. Unclear parts (e.g., bed and speakers) are indicated by dotted lines). Among them, the distance between the front lens of the lens and the photographed object is the focusing distance. As shown in (2) in Figure 2, the shooting device (for example, a mobile phone) is shooting the TV (the object being photographed), and the clear part of the picture is the depth of field. The depth of field in (1) in Figure 2 is shallower (smaller) than the depth of field in (2) in Figure 2 .

Non-multi-depth-of-field scene (simple scene): Within the depth-of-field range, the objects (for example, people or objects) in the picture are at similar or the same depth. For example, as shown in Figure 3, objects (for example, people) in the picture are basically at the same depth position in the picture, and the depth is single.

Multiple depth of field scenes (complex scenes): Within the depth of field range, objects (for example, people or objects) in the picture are at different depth positions, and the depth distribution is uneven. For example, as shown in Figure 4, the characters in the picture are at depth positions one behind the other in the picture, and the depth distribution is uneven.

PDAF: used to use the phase difference collected by the photosensitive chip as the basis for focusing. The part of the camera module that senses images is a photosensitive chip, and each pixel of the photosensitive chip senses an image. As shown in Figure 5, two symmetrical pixels separated by a certain distance can be covered with the left half and the right half of the pixel respectively (in this way, the left half-shielded pixel (left-half-shield pixel) can only be Accept the light from the right, and the right-half-shield pixel that is blocked can only accept the light from the left). In this way, the two symmetrical pixels are equivalent to "the left and right eyes of a person". When the "left and right eyes" of PDAF "see" the same object, there will be a certain visual difference, that is, the phase difference PD (value). Based on the PD value, whether the focus is accurate can be calculated.

Current focus systems can automatically focus based on the subject in the viewfinder. However, when there are multiple subject object frames in the viewfinder, how to focus more accurately and reasonably becomes a new problem.

In one possible design, the focus can be based on the direction of the photographer's attention, or the focus can be based on the direction of the attention of the person in the picture. Under what circumstances which attention focusing scheme is more reasonable is an issue that needs to be solved urgently.

Embodiments of the present application provide a focusing method that can adopt different attention focusing solutions according to whether the current viewfinder is a multi-depth scene, and can focus more accurately and reasonably, making the focus of the picture more prominent and improving the user experience.

Since the content of the non-multi-depth-of-field scene is simple and the objects (for example, people or objects) in the picture are at similar or the same depth, it is easier for people to understand simple scenes, and it is easier to find the key points in the picture. In this case, based on the photographer Focusing on the attention position can make the focus of the picture more prominent and improve the user experience. On the contrary, the content of the scene with multiple depths of field is complex, and the objects in the picture (for example, people or objects) have different depths of field. It is difficult for people to understand the complex scene and find the key points in the picture quickly. In this case, the focus position is taken as the focus position of the photographer. If it is not accurate enough, the focus direction can be based on the direction of attention of the characters in the picture. It is more accurate and reasonable, which can make the focus of the picture more prominent and improve the user experience.

The focusing method provided by the embodiments of the present application can be applied to electronic devices. Electronic devices can be, for example, mobile phones, tablet computers, cameras, video cameras (video cameras), desktop computers (desktop computers), handheld computers, and notebook computers (laptop computers). , ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), augmented reality (AR)\virtual reality (VR) equipment, etc., this application The embodiment does not place any special restrictions on the specific form of the electronic device.

FIG. 6A is a schematic structural diagram of an electronic device 100 provided by an embodiment of the present application. As shown in Figure 6A, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, and a battery 142. Antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone interface 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, Display 194, and subscriber identification module (subscriber identification module, SIM) card interface 195, etc.

Among them, the sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and an environment. Light sensor 180L, bone conduction sensor 180M, etc.

It can be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100 . In other embodiments, the electronic device 100 may include more or fewer components than illustrated, some components may be combined, some components may be separated, or components may be arranged differently. The components illustrated may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), an image signal processor ( image signal processor (ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU), etc. . Among them, different processing units can be independent devices or integrated in one or more processors.

The controller may be the nerve center and command center of the electronic device 100 . The controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.

The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have been recently used or recycled by processor 110 . If the processor 110 needs to use the instructions or data again, it can be called directly from the memory. Repeated access is avoided and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuitsound, I2S) interface, a pulse code modulation (PCM) interface, and a universal asynchronous receiver (universal asynchronous receiver) /transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and/or Universal serial bus (USB) interface, etc.

It can be understood that the interface connection relationships between the modules illustrated in this embodiment are only schematic illustrations and do not constitute a structural limitation of the electronic device 100 . In other embodiments, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

The charging management module 140 is used to receive charging input from the charger. While the charging management module 140 charges the battery 142, it can also provide power to the electronic device through the power management module 141.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, wireless communication module 160, etc. In some other embodiments, the power management module 141 may also be provided in the processor 110 . In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.

The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example: Antenna 1 can be reused as a diversity antenna for a wireless LAN.

The mobile communication module 150 can provide solutions for wireless communication including 2G/3G/4G/5G applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, perform filtering, amplification and other processing on the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves through the antenna 1 for radiation.

A modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194.

The wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (bluetooth, BT), and global navigation satellite system. (global navigation satellite system, GNSS), frequency modulation (FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (codedivision multiple access, CDMA), broadband code Wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM , and/or IR technology, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou satellite navigation system (beidounavigation satellite system, BDS), quasi-zenith satellite system (quasi- zenith satellitesystem (QZSS) and/or satellite based augmentation systems (SBAS).

The electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like. The GPU is an image processing microprocessor and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can use a liquid crystal display (LCD), a light-emitting diode (LED), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix Organic light-emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diode (QLED) )wait.

The electronic device 100 can implement the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like. The ISP is used to process the data fed back by the camera 193. Camera 193 is used to capture still images or video. Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

The cameras 193 may include 1 to N cameras. In this embodiment of the present application, at least one camera is configured with PD pixels, including left shielded pixels and right shielded pixels. Focusing can be quickly performed based on the phase difference of the images output by left shielded pixel and right shielded pixel.

NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can continuously learn by itself. Intelligent cognitive applications of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, etc.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100 . The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. Such as saving music, videos, etc. files in external memory card. Internal memory 121 may be used to store computer executable program code, which includes instructions. The processor 110 executes instructions stored in the internal memory 121 to execute various functional applications and data processing of the electronic device 100 . For example, in the embodiment of the present application, the processor 110 can execute instructions stored in the internal memory 121, and the internal memory 121 can include a program storage area and a data storage area. Among them, the stored program area can store an operating system, at least one application program required for a function (such as a sound playback function, an image playback function, etc.). The storage data area may store data created during use of the electronic device 100 (such as audio data, phone book, etc.). In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), etc.

The electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor. Such as music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. Audio module 170 may also be used to encode and decode audio signals. Speaker 170A, also called "speaker", is used to convert audio electrical signals into sound signals. Receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. The headphone interface 170D is used to connect wired headphones.

The buttons 190 include a power button, a volume button, etc. Key 190 may be a mechanical key. It can also be a touch button. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 . The motor 191 can generate vibration prompts. The motor 191 can be used for vibration prompts for incoming calls and can also be used for touch vibration feedback. The indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, or may be used to indicate messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 . The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card, etc.

The methods in the following embodiments can all be implemented in the electronic device 100 with the above hardware structure.

The software system of the above-mentioned electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. This embodiment of the present invention takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100 .

The layered architecture divides the software into several layers, and each layer has clear roles and division of labor. The layers communicate through interfaces. In some embodiments, the Android system may include an application layer, an application framework layer, an Android runtime and system libraries, a hardware abstraction layer (HAL), and a kernel layer. It should be noted that the embodiments of the present application are illustrated by taking the Android system as an example. In other operating systems (such as Hongmeng system, IOS system, etc.), as long as the functions implemented by each functional module are similar to those of the embodiments of the present application, the present application can also be implemented. plan.

Among them, the application layer can include a series of application packages.

As shown in Figure 6B, the Android system may include an application layer (which may be referred to as an application layer), an application framework layer (which may be referred to as a framework layer), and a kernel layer. In addition, the Android system can also include Android runtime and system libraries, which are not limited in this application.

The application layer may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc. This embodiment of the present application does not impose any restrictions on this.

The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

The application framework layer may include an activity manager, a window manager, a content provider, a view system, a resource manager, a notification manager, etc. This embodiment of the present application does not impose any restrictions on this.

The kernel layer is the layer between hardware and software. The kernel layer can include display drivers, Wi-Fi drivers, camera drivers, audio drivers, sensor drivers, etc.

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Among them, in the description of this application, unless otherwise specified, "at least one" refers to one or more, and "plurality" refers to two or more than two. In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as “first” and “second” are used to distinguish identical or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the number and execution order, and words such as "first" and "second" do not limit the number and execution order.

For ease of understanding, the focusing method provided by the embodiment of the present application will be introduced in detail below with reference to the accompanying drawings.

As shown in Figure 6C, this embodiment of the present application provides a focusing method, including:

601. Determine whether the current viewfinder belongs to a multi-depth-of-field scene.

When a user uses a terminal device to take pictures (photograph or video), the display screen of the terminal device can display the current viewfinder image. For example, as shown in (a) of Figure 7 , in response to the user's operation (for example, a click operation) of opening the camera application on the main interface 701 of the terminal device, as shown in (b) of Figure 7 , the display of the terminal device The screen can display the picture 703 at the current moment (that is, the current viewfinder picture).

When determining whether the current viewing image belongs to a multi-depth-of-field scene, the phase difference sequence of the current viewing image can be calculated first. For example, as shown in Figure 8, the PD raw image corresponding to the viewfinder is obtained from the camera, and the PD raw image includes the left PD raw image and the right PD raw image. Among them, RAW images can be understood as "unprocessed and uncompressed image data". Then, the PD raw image is input into the phase detection library (PDLIB) to calculate the PD value. Among them, PDLIB can increase the gain (gain) for the PD pixels corresponding to the PD raw image through the gain map. Then, the PD value is calculated based on the PD pixels after the gain is increased. Multiple PD values can form a PD sequence.

Then, based on the phase difference sequence (PD sequence) and the preset threshold, it is determined whether the viewing image belongs to a multi-depth scene. Determine the maximum value and minimum value in the phase difference sequence, and determine the absolute value of the difference between the maximum value and the minimum value and the preset threshold. If the absolute value of the difference is greater than the preset threshold, the scene is determined to be a multi-depth scene. , if the absolute value of the difference is less than the preset threshold, the scene is judged to be a non-multi-depth-of-field scene. The value of the preset threshold may be 80, for example.

For example, assuming that the pd value sequence is {pd1, pd2, pd3, pd4,..., pdn}, the multi-depth judgment threshold (ie, the preset threshold) is multidepth_thr, in the pd sequence {pd1, pd2, pd3, pd4,..., pdn}, select the maximum pd value max_pd and the minimum pd value min_pd; when |max_pd-min_pd|>multidepth_thr, the scene is judged to be a multi-depth scene. When |max_pd-min_pd|<multidepth_thr, the scene is judged to be a non-multidepth scene.

It should be noted that most current cameras are equipped with PD pixels and have PDAF function (support PDAF technology). That is, the image captured by the camera can be processed by the PDAF algorithm to obtain a phase difference sequence, and focusing can be performed based on the phase difference sequence. Moreover, the imaging effect using PDAF technology for focusing is good, the cost is lower than laser imaging, and it is not limited by distance. Therefore, judging whether an image is a multi-depth-of-field image based on the phase difference sequence obtained by PDAF technology has a high degree of adaptability and can be adapted to cameras or imaging products from most manufacturers on the market.

602. If the current viewfinder is a multi-depth-of-field scene, determine whether there are people in the current viewfinder.

Whether there is a person in the current view frame (that is, whether there is a face or a human body) can be determined based on the face recognition algorithm or the human body recognition algorithm. The process of face recognition based on the face recognition algorithm and the process of identifying the human body based on the human body recognition algorithm can refer to the existing technology and will not be described in detail here.

603. If there is a person in the current viewfinder, determine whether the attention position of the target face in the current viewfinder is within the viewfinder.

If the viewfinder includes a character, the target face is the face of the character, and it is determined whether the attention position of the character's face is within the viewfinder. Among them, the attention position of the human face is the convergence area (convergence point) of the line of sight direction of the human eye.

If the viewfinder includes multiple people, determine whether the attention position of the target face is within the viewfinder. The target face can be the face with the largest area in the viewfinder.

In one possible design, the target face can be determined based on parameters such as the size of the face frame, the deviation of the face from the center position, and the distance between the face and the lens. Among them, the weight ratio of parameters such as the size of the face frame, the deviation of the face from the center position, and the distance between the face and the camera can be adjusted based on the machine learning algorithm so that the target face can be determined more accurately.

604. If there are no people in the current viewfinder, perform center focus or smart focus.

That is, when it is determined that there are no people in the current viewfinder, you can focus based on the center area of the current viewfinder, or you can detect the subject in the current viewfinder (such as flowers, faces), and focus based on the location of the subject.

605. If the attention position of the target face in the current viewfinder is outside the frame, focus on the face.

That is, when the convergence area of the sight direction of the target face in the viewfinder exceeds the viewfinder, a face focusing solution can be used, that is, focusing based on the target face. The target face can be the face with the largest area in the viewfinder.

As shown in FIG. 9 , the attention position 902 of the character 901 in the viewfinder 903 is outside the viewfinder 903 , and a face focusing solution can be adopted, for example, the face of the character 901 is selected for focus.

606. If the attention position of the target face in the current viewfinder is within the viewfinder, focus is performed based on the attention position of the target face in the current viewfinder.

That is, when the convergence area of the human eyesight direction of the characters in the framing picture is in the framing picture, focusing is performed with the attention position of the characters in the framing picture as the focus position.

As shown in FIG. 10 , the attention position 1002 of the character 1001 in the viewfinder 1003 is within the viewfinder 1003 , and focusing can be performed based on the attention position 1002 . That is, the attention position 1002 can be used as the focus position.

607. If the viewfinder is a non-multi-depth-of-field scene, determine whether the photographer is detected.

The electronic device can detect whether there is a photographer through the front-facing camera. For example, if a user takes a photo with a handheld electronic device, the photographer can be detected. If the user uses a camera stick or other equipment to support the electronic device for shooting, or if the electronic device is placed on a table, grass, etc. for shooting, the photographer may not be detected.

608. If the photographer is detected, determine whether the photographer's attention position can be detected.

If the photographer is detected, the LED light can be used to illuminate the cornea, and the sensor detects the infrared reflected light of the cornea to identify the sight position of the user's pupils, thereby determining the photographer's attention position.

609. If the photographer is not detected, perform center focus or smart focus.

That is, when it is determined that there are no people in the current viewfinder, you can focus based on the center area of the current viewfinder, or you can detect the subject in the current viewfinder (such as flowers, faces), and focus based on the location of the subject.

610. If the photographer's attention position can be detected, focus will be performed based on the photographer's attention position.

That is, the photographer's eyes (eyeballs) can be used to focus, and the changes in the focus area can be controlled by the movement of the photographer's eyes.

As shown in Figure 11, the front attention detection system can detect the gaze position of the photographer's eyes in the current viewfinder. Afterwards, the coordinates of the gaze position can be sent to the post-focus algorithm. The post-focus algorithm determines the position of the gaze according to the pupil. The focus window 1101 is generated based on the line of sight position, and then the motor is driven to complete focusing based on the phase difference information of the pixels in the focus window 1101.

611. If the photographer's attention position cannot be detected, focus on the face.

That is, when the photographer's attention position is detected, the face focusing solution can be used. For example, you can select the face with the largest area in the current viewfinder to focus.

In one possible design, the final focused face can be obtained based on parameters such as the size of the face frame, the weight ratio of the face to the center position, and the distance between the face and the lens. For example, you can select the face closest to the center of the frame to focus, or you can select the face closest to the lens to focus, or you can focus on the front face in the viewfinder (the face closest to the lens is not the front face).

Based on the focusing method provided by the embodiment of the present application, different attention focusing solutions can be adopted according to whether the current viewing frame is a multi-depth-of-field scene. For example, when the current viewfinder is a non-multi-depth-of-field scene, focusing can be based on the photographer's attention position. Since the content of the non-multi-depth-of-field scene is simple and the objects (for example, people or objects) in the picture are at similar or similar depths, it is easier for people to understand simple scenes, and it is easier to find the key points in the picture. In this case, based on the photographer Focusing on the attention position can make the focus of the picture more prominent and improve the user experience. On the contrary, when the current viewing frame is a scene with multiple depths of field, the direction of attention of the person (target face) in the frame can be used as the focus direction. Due to the complex content of multi-depth scenes, objects in the picture (for example, people or objects) are at different depths. It is difficult for people to quickly understand complex scenes and find the key points in the picture. At this time, the focus is based on the photographer's attention position. If the position is not accurate and fast enough, the focus direction can be based on the attention direction of the person in the picture (target face), which is more accurate and reasonable. It can make the focus of the picture more prominent, more consistent with the expression of the lens language, and improve the user experience.

Some embodiments of the present application provide an electronic device, which may include a touch screen, a memory, and one or more processors. The touch screen, memory and processor are coupled. The memory is used to store computer program code, which includes computer instructions. When the processor executes computer instructions, the electronic device may perform various functions or steps performed by the electronic device in the above method embodiments. The structure of the electronic device may refer to the structure of the electronic device 100 shown in FIG. 6A.

An embodiment of the present application also provides a chip system (for example, a system on a chip (SoC)). As shown in FIG. 12 , the chip system includes at least one processor 1201 and at least one interface circuit 1202 . The processor 1201 and the interface circuit 1202 may be interconnected by wires. For example, interface circuit 1202 may be used to receive signals from other devices, such as memory of an electronic device. As another example, the interface circuit 1202 may be used to send signals to other devices, such as the processor 1201 or a touch screen of an electronic device. For example, the interface circuit 1202 can read instructions stored in the memory and send the instructions to the processor 1201. When the instructions are executed by the processor 1201, the electronic device can be caused to perform various steps in the above embodiments. Of course, the chip system may also include other discrete devices, which are not specifically limited in the embodiments of this application.

Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium includes computer instructions. When the computer instructions are run on the above-mentioned electronic device, the electronic device causes the electronic device to execute the execution of the above-mentioned method embodiment. Each function or step.

An embodiment of the present application also provides a computer program product. When the computer program product is run on an electronic device, it causes the electronic device to perform each function or step performed by the electronic device in the above method embodiment.

Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In practical applications, the above functions can be allocated according to needs. Different functional modules are completed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, for example, multiple units or components may be The combination can either be integrated into another device, or some features can be omitted, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated. The components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place, or they may be distributed to multiple different places. . Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application are essentially or contribute to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium , including several instructions to cause a device (which can be a microcontroller, a chip, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes.

The above contents are only specific implementation modes of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be covered by the protection scope of the present application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims (10)

1. A focusing method, characterized in that it is applied to electronic equipment, including: Display the current viewfinding picture, obtain the phase difference sequence of the current viewfinding picture, and determine whether the current viewfinder picture belongs to a multi-depth-of-field scene according to the phase difference sequence; If it is determined that the current viewfinding picture belongs to the multi-depth-of-field scene, focus is performed according to the attention position of the target face in the current viewfinder picture. The fact that the current viewfinder picture belongs to the multi-depth-of-field scene means that the current viewfinder picture belongs to the multi-depth-of-field scene. Multiple subjects at different depth positions; If it is determined that the current viewfinding picture does not belong to the multi-depth-of-field scene, the photographer's attention position is detected, and focusing is performed according to the photographer's attention position. The fact that the current viewfinder picture does not belong to the multi-depth-of-field scene indicates that the Multiple subjects in the current viewfinder are at the same or similar depth positions. 2. The method according to claim 1, characterized in that, If the photographer's attention position cannot be detected, focusing is performed based on the target face in the current viewing frame, or focusing is performed based on the central area in the current viewing frame. 3. The method according to claim 1 or 2, characterized in that, If the attention position of the target face is not within the current viewing frame, focus is performed according to the area where the target face is located. 4. The method according to claim 1 or 2, wherein determining whether the current view frame belongs to a multi-depth-of-field scene according to the phase difference sequence includes: Determine the maximum value and minimum value in the phase difference sequence; Determine the absolute value of the difference between the maximum value and the minimum value and the size of the preset threshold; If the absolute value of the difference is greater than the preset threshold, it is determined that the current viewfinder picture belongs to a multi-depth-of-field scene; if the absolute value of the difference is less than or equal to the preset threshold, it is determined that the current viewfinder picture is not It belongs to a multi-depth-of-field scene. 5. The method according to claim 4, wherein said obtaining the phase difference sequence of the current viewing frame includes: Obtain the phase difference original PD raw image corresponding to the current viewing frame from the camera; Enter the PD raw image into the phase detection library PDLIB; The PDLIB uses the gain map to increase the gain for the multiple phase difference PD pixels corresponding to the PD raw map; Multiple PD values are calculated based on the multiple PD pixels after the gain is increased, and the multiple PD values constitute the phase difference sequence. 6. The method according to any one of claims 1, 2 or 5, characterized in that, The target face is determined based on at least one of the size of the face frame in the current viewing frame, the weight ratio between the face and the center position, and the distance between the face and the lens. 7. The method according to any one of claims 1, 2 or 5, characterized in that, If the current view frame includes multiple faces, the target face is the face with the largest area in the current view frame. 8. An electronic device, characterized in that the electronic device includes: a wireless communication module, a memory and one or more processors; the wireless communication module, the memory and the processor are coupled; Wherein, the memory is used to store computer program code, and the computer program code includes computer instructions; when the computer instructions are executed by the processor, the electronic device is caused to perform any one of claims 1-7. the method described. 9. A computer-readable storage medium, characterized in that it includes computer instructions; When the computer instructions are run on the electronic device, the electronic device is caused to perform the method according to any one of claims 1-7. 10. A chip system, characterized in that the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through lines; The chip system is applied to electronic equipment including a communication module and a memory; the interface circuit is used to receive signals from the memory and send the signals to the processor, where the signals include computer data stored in the memory. Instructions; when the processor executes the computer instructions, the electronic device performs the method of any one of claims 1-7.