CN110213502B - Image processing method, device, storage medium and electronic device - Google Patents

Abstract

The present application discloses an image processing method, device, storage medium and electronic device, wherein the image processing method includes: continuously acquiring a first YUV image of a target scene based on a first exposure value; Stored in the first image cache queue; when previewing or photographing or video recording is required, the newly stored multi-frame first YUV images are obtained from the first image cache queue; the multi-frame first YUV images are synthesized to obtain high-quality images. The first composite image of the dynamic range; according to the first composite image, the target scene is previewed or photographed or recorded. In this solution, when previewing, photographing or video recording is required, the ready-made first YUV image is directly called for synthesis processing, which can shorten the operation time of previewing, photographing or recording, so that the image processing solution provided by this embodiment can obtain the Images can be used for image preview, photography and video recording.

Description

Image processing method, device, storage medium and electronic device

technical field

The present application belongs to the field of image technology, and in particular, relates to an image processing method, device, storage medium and electronic device.

Background technique

Compared with ordinary images, High-Dynamic Range (HDR) images can provide more dynamic range and image details. For example, the electronic device may acquire multiple frames of images shot in the same scene, and perform high dynamic range synthesis on the multiple frames of images to obtain one frame of HDR image. However, the images processed by the related HDR technology are difficult to be used for preview, photography and video recording at the same time.

SUMMARY OF THE INVENTION

Embodiments of the present application provide an image processing method, device, storage medium, and electronic device, and the image obtained by the processing can be suitable for previewing, photographing, and video recording.

In a first aspect, the embodiments of the present application provide an image processing method, including:

Based on the first exposure value, continuously obtain the first YUV image of the target scene;

storing the first YUV image in the first image buffer queue according to the sequence of exposure;

When previewing or photographing or video recording is required, obtain the newly stored multi-frame first YUV images from the first image cache queue;

performing synthesis processing on the multiple frames of the first YUV images to obtain a first synthesized image with a high dynamic range;

According to the first composite image, a preview or photographing or video recording operation is performed on the target scene.

In a second aspect, an embodiment of the present application provides an image processing apparatus, including:

a first acquisition module, configured to continuously acquire the first YUV image of the target scene based on the first exposure value;

a storage module, configured to store the first YUV image in the first image buffer queue according to the sequence of exposure;

The second acquisition module is used to acquire the newly stored multi-frame first YUV images from the first image cache queue when previewing or taking pictures or videos is required;

a synthesis module, configured to perform synthesis processing on the multiple frames of the first YUV images to obtain a first synthesized image with a high dynamic range;

The processing module is configured to perform a preview or photographing or video recording operation on the target scene according to the first composite image.

In a third aspect, a storage medium provided by an embodiment of the present application stores a computer program thereon, and when the computer program runs on a computer, the computer causes the computer to execute the image processing method provided by any embodiment of the present application.

In a fourth aspect, an electronic device provided by an embodiment of the present application includes a processor and a memory, the memory has a computer program, and the processor is configured to execute the image provided by any embodiment of the present application by invoking the computer program. Approach.

In this embodiment, when previewing or photographing or video recording is required, the electronic device can directly obtain the newly stored multi-frame first YUV images from the first image buffer queue and perform composite processing to obtain a first composite image with a high dynamic range. , and then use the first composite image to perform image preview or photographing or video recording operations, thereby shortening the operation time of previewing, photographing or video recording, so that the generated first composite image can be suitable for previewing, photographing and video recording.

Description of drawings

The technical solutions of the present application and the beneficial effects thereof will be apparent through the detailed description of the specific embodiments of the present application in conjunction with the accompanying drawings.

FIG. 1 is a first schematic flowchart of an image processing method provided by an embodiment of the present application.

FIG. 2 is a schematic diagram of a first scenario of an image processing method provided by an embodiment of the present application.

FIG. 3 is a schematic diagram of a second scenario of the image processing method provided by the embodiment of the present application.

FIG. 4 is a schematic diagram of a third scenario of an image processing method provided by an embodiment of the present application.

FIG. 5 is a second schematic flowchart of the image processing method provided by the embodiment of the present application.

FIG. 6 is a schematic diagram of a fourth scenario of the image processing method provided by the embodiment of the present application.

FIG. 7 is a schematic diagram of a synthesis process of an image processing method provided by an embodiment of the present application.

FIG. 8 is a third schematic flowchart of the image processing method provided by the embodiment of the present application.

FIG. 9 is a fourth schematic flowchart of the image processing method provided by the embodiment of the present application.

FIG. 10 is a fifth schematic flowchart of the image processing method provided by the embodiment of the present application.

FIG. 11 is a schematic diagram of a first structure of an image processing apparatus provided by an embodiment of the present application.

FIG. 12 is a schematic diagram of a second structure of an image processing apparatus provided by an embodiment of the present application.

FIG. 13 is a schematic diagram of a third structure of an image processing apparatus provided by an embodiment of the present application.

FIG. 14 is a schematic diagram of a first structure of an electronic device provided by an embodiment of the present application.

FIG. 15 is a schematic structural diagram of an image processing circuit provided by an embodiment of the present application.

FIG. 16 is a schematic diagram of a second structure of an electronic device provided by an embodiment of the present application.

Detailed ways

The following description is based on illustrated specific embodiments of the present application and should not be construed as limiting other specific embodiments of the present application not detailed herein. As used herein, the term "module" can be thought of as a software object that executes on the computing system. The different modules, engines and services in this paper can be regarded as implementation objects on the computing system.

The embodiment of the present application provides an image processing method, and the execution body of the image processing method may be the image processing apparatus provided by the embodiment of the present application, or an electronic device integrated with the image processing apparatus. Wherein, the electronic device includes a camera and a memory, and the electronic device may be a smart phone, a tablet computer, a PDA (Personal Digital Assistant), and the like.

The specific analysis is described below.

An embodiment of the present application provides an image processing method, which is applied to an electronic device. The electronic device includes a camera assembly. Please refer to FIG. 1 . FIG. 1 is a first schematic flowchart of the image processing method provided by the embodiment of the present application. The image processing method The following steps can be included:

101. Based on the first exposure value, continuously acquire a first YUV image of the target scene.

Among them, the exposure value, also known as the EV value, is a measurement value that reflects the amount of incoming light. Generally, if the scene is too dark, you can make positive compensation by increasing the exposure value; if the scene is too bright, you can make negative compensation by reducing the exposure value. The exposure value is associated with the sensitivity, aperture, and exposure duration, and the electronic device can change the exposure value by changing at least one parameter of the sensitivity, aperture, and exposure duration. Regardless of the combination of sensitivity, aperture, and exposure duration, as long as the EV value is the same, an image with the same exposure can be obtained in the end. For example, when the sensitivity remains unchanged, the exposure value is obtained by combining different exposure durations and apertures. At this time, if you want to reduce the EV value, you can reduce the exposure duration or reduce the aperture coefficient. At this time, if If you want to increase the EV value, you can take measures to increase the exposure time or increase the aperture factor.

The target scene refers to a scene to be photographed by the user through the camera assembly, and the target scene can be presented on the image obtained through the camera assembly. In addition, the content of the target scene is not specifically limited in the embodiments of the present application, for example, the target scene may be a character scene, the target scene may also be a night sky scene, and so on.

YUV images refer to images whose image format is YUV format. YUV format is a common color image recording format, usually presented in Y:UV format, where Y represents the brightness of the image, represented by Cr; UV represents the chroma of the image, represented by Cb. The higher the value of Y, the brighter the image; the higher the value of UV, the more saturated the color of the image.

In the embodiment of the present application, the electronic device keeps the exposure value constant at a certain value during a certain period of time, and continuously exposes the target scene through the camera component during the period of time, and obtains the same or similar exposure amount. The first YUV image. It should be noted that, because the first YUV image is to be used for high dynamic range synthesis processing, the first YUV image obtained based on the first exposure value in this embodiment of the present application includes at least two frames.

In some embodiments, based on the first exposure value, a first YUV image of the target scene is continuously acquired, and the first YUV image is a short-exposure image with low brightness.

102. Store the first YUV image in the first image buffer queue according to the exposure sequence.

In the embodiment of the present application, the first YUV image is stored in the first image buffer queue according to the sequence of exposure, which may specifically refer to: every time a frame of the first YUV image of the target scene is obtained based on the first exposure value, immediately The first YUV image of the frame is stored in the first image buffer queue. For example, as shown in FIG. 2 , if a frame of the first YUV image is generated at time T _i , the frame of the first YUV image is immediately stored in the first image buffer queue at time T _i .

The electronic device is equipped with a dedicated memory for storing images, and the dedicated memory can divide the cache queue according to a certain strategy to classify and store the images. The first image cache queue of the present application may be a queue of fixed length or a queue of indefinite length, which is not specifically limited in this embodiment.

Because the electronic device is equipped with a dedicated memory for saving images, the electronic device can continuously expose the target scene before previewing or taking pictures or recordings, and obtain multiple frames of the first YUV image of the target scene. The sequence of exposure is to store the multiple frames of the first YUV images in the first image cache queue. When previewing or taking pictures or recordings is required, the first YUV image newly stored in the first image cache queue can be directly called, thereby Shorten the operation time of previewing or taking photos or videos, and slow down the screen delay to ensure the frame rate of image preview and recording.

103. When previewing, photographing or video recording is required, acquire the newly stored multi-frame first YUV images from the first image buffer queue.

104. Perform synthesis processing on the multiple frames of the first YUV images to obtain a first synthesized image with a high dynamic range.

105. Perform a preview or photographing or video recording operation on the target scene according to the first composite image.

When a preview needs to be performed, the newly stored multi-frame first YUV images are obtained from the first image buffer queue for synthesis processing to obtain a first synthesized image with a high dynamic range, and a preview operation is performed on the target scene according to the first synthesized image. . Wherein, the multi-frame refers to at least two frames, and the specific number of frames is not specifically limited, for example, the multi-frame may be three frames, six frames, nine frames, and the like.

For example, as shown in Fig. 2, when a preview needs to be performed at time T _i , three newly stored first YUV images are obtained from the first image cache queue, that is, the image stored at time T _i and the image stored at time T _i-1 . The image and the image stored at time T _i-2 are combined with the three frames of the first YUV images to obtain a first combined image with a high dynamic range, and a preview of the target scene is performed according to the first combined image. where i is an integer greater than 0.

In some embodiments, for an electronic device with a real-time preview function, such as an electronic device such as a terminal, the preview needs to be performed without interruption, that is, for the first time, the electronic device "obtains the latest stored image data from the first image cache queue for the first time." Frame the first YUV image, perform composite processing on multiple frames of the first YUV image to obtain a first composite image with high dynamic range, and perform a preview operation on the target scene according to the first composite image. A new frame of image is stored in the queue, and the electronic device executes once "obtain the newly stored multi-frame first YUV image from the first image cache queue, and perform composite processing on the multi-frame first YUV image to obtain the first YUV image with high dynamic range. Synthesize the image, and perform a preview operation on the target scene according to the first synthetic image.”

When a photo needs to be taken, the newly stored multi-frame first YUV images are obtained from the first image buffer queue, and the multi-frame first YUV images are synthesized to obtain a first composite image with a high dynamic range. image to take pictures of the target scene.

For example, as shown in FIG. 3 , after the time T _i-1 and before the time T _i , the electronic device receives the photographing instruction, and at this time, it is necessary to take a picture, and obtain the newly stored 3 frames of the first YUV from the first image cache queue. Images, that is, the image stored at time T _i-1 , the image stored at time T _i-2 , and the image stored at time T _i-3 , the three frames of the first YUV images are synthesized to obtain the first YUV image with high dynamic range. a composite image, outputting the first composite image in response to the photographing instruction. where i is an integer greater than 0.

When video recording is required, the newly stored multi-frame first YUV images are obtained from the first image buffer queue, and the multi-frame first YUV images are synthesized to obtain a first composite image with high dynamic range. image to record the target scene.

For example, as shown in FIG. ₄ , after time T3 and before time _T4 , the electronic device receives the recording start command, after time _T7 and before time _T8 , the electronic device receives the recording end command, and since receiving the video recording From the moment of the start instruction to the moment of receiving the recording end instruction, it is necessary to continuously acquire pictures for synthesis. That is, i is 4, 5, 6, 7, or 8 in sequence, and when video recording is required at time T _i , the newly stored 3 frames of the first YUV images are obtained from the first image buffer queue, that is, the first YUV images stored at time T _i-3 are obtained. The image, the image stored at time T _i-2 , and the image stored at time T _i-1 are combined with the three frames of the first YUV images to obtain a first combined image with a high dynamic range, and the first combined image Perform video encoding processing to obtain a video corresponding to the target scene.

Among them, the first composite image with high dynamic range can provide more dynamic range and image details than ordinary images. The dynamic range of an image refers to the difference between light and dark in the image. It should be noted that the obtained first composite image is also an image in YUV format.

In some embodiments, the electronic device may use a multi-frame short-exposure synthesis method to perform synthesis processing on multiple frames of the first YUV image to obtain a first synthesized image with a high dynamic range. The method of synthesizing multiple frames of short exposure specifically refers to: synthesizing N frames of short exposure images to obtain a frame of HDR image, and the pixels of each position of the frame of HDR image are approximately equal to N times of pixels corresponding to each position of each frame of short exposure image. For example, 3 frames of short-exposure images captured by electronic device A include the sky and buildings. In each frame of short-exposure images, the sky pixel value is 10, and the building pixel value is 3. The 3 frames of short-exposure images are synthesized to obtain an HDR image. In the image, the sky pixel value is about 10+10+10=30, and the building pixel value is about 3+3+3=9, so the difference between the sky and the building in the HDR image is larger, that is, the dynamic range is higher.

During specific implementation, the present application is not limited by the execution order of the described steps, and certain steps may also be performed in other sequences or simultaneously under the condition of no conflict. For example, exposure according to the first exposure value and storage in the first image buffer queue and acquisition of multiple frames of images from the buffer queue for synthesis and display may be performed synchronously. .

It can be seen from the above that, in the image processing method provided in this embodiment, due to the existence of the first image cache queue, the electronic device can directly obtain the latest stored images from the first image cache queue when previewing, taking pictures or recordings is required. The first YUV image of the frame is synthesized to obtain a first synthesized image with a high dynamic range, and then the first synthesized image is used to perform image preview or photographing or video recording operations, so that there is no need to wait for the generation of multiple frames of the first YUV image, which can shorten the The operation time of previewing or taking pictures or recordings, slowing down the screen delay to ensure the frame rate of image previewing and recording. Therefore, the image obtained by the image processing solution provided in this embodiment can be suitable for previewing, photographing and video recording.

Please refer to FIG. 5 , which is a second schematic flowchart of the image processing method provided by the embodiment of the present application, which will be described in detail below.

In some embodiments, before performing 101, it further includes:

Obtaining a frame of a second RAW image of the target scene through the image sensor;

Obtain the light ratio of the second RAW image;

Determine the reference exposure value according to the light ratio;

According to the reference exposure value, at least one parameter of exposure duration, aperture, and sensitivity is adjusted to determine the first exposure value.

Among them, the light ratio is the ratio of light reception between the bright part and the dark part of the target scene. The exposure time is the time that the light shines on the film or the image sensor. By adjusting the exposure time, the light input of the image sensor can be adjusted. The longer the exposure time, the more light input; the shorter the exposure time, the less light input, which affects the exposure value. When the other parameters of , the exposure time is proportional to the exposure value. The aperture is an important component of the camera assembly. The amount of light entering the image sensor can be adjusted by adjusting the aperture. The larger the aperture, the greater the amount of light entering; the smaller the aperture, the less the amount of light entering. Exposure value is proportional. Sensitivity is a parameter that measures the sensitivity of the "film" to light. By adjusting the sensitivity, the amount of light entering the image sensor can be adjusted. The weaker the photosensitive ability of the camera assembly, the less the amount of light entering, that is, when other parameters affecting the exposure value remain unchanged, the sensitivity is proportional to the exposure value.

The reference exposure value refers to the exposure value setting parameters of the electronic device at the current moment (the moment when the reference exposure value is determined according to the light ratio): reference exposure duration, reference aperture, and reference sensitivity. Acquiring one frame of the second RAW image of the target scene by using the image sensor specifically refers to: acquiring one frame of the second RAW image of the target scene by using the image sensor based on the basic exposure value. Therefore, the light ratio of the second RAW image is inversely proportional to the reference exposure value.

It should be noted that the first exposure value is smaller than the reference exposure value. Adjusting at least one parameter of exposure duration, aperture, and sensitivity according to the reference exposure value, and determining the first exposure value specifically refers to: reducing at least one parameter of exposure duration, aperture, and sensitivity to make the electronic device's The exposure value is decreased from the reference exposure value to the first exposure value (eg, from the reference exposure value -1EV to the first exposure value -3EV).

Specifically, the acquiring the light ratio of the second RAW image may include: detecting the second RAW image by a light meter, and obtaining the value of the bright part of the second RAW image and the value of the dark part of the second RAW image; according to the value of the bright part and The dark part value calculates the light ratio of the second RAW image.

For example, based on 0EV, the electronic device obtains a frame of a second RAW image of the target scene through an image sensor; obtains the light ratio of the second RAW image; exposure duration, reference aperture, and reference sensitivity), and according to the reference exposure value 0EV (the reference exposure value 0EV specifically sets parameters: reference exposure duration, reference aperture, and reference sensitivity), perform at least one of the following operations: On the basis of the reference exposure time, reduce the exposure time, reduce the aperture on the basis of the reference aperture, and reduce the sensitivity on the basis of the reference sensitivity, so that the exposure value of the electronic device is reduced from the reference exposure value 0EV to 0EV. The first exposure value -2EV.

In some embodiments, 101 specifically includes: continuously acquiring the first RAW image of the target scene through the image sensor according to the first exposure value; and performing format conversion on the first RAW image to obtain the first YUV image.

It should be noted that the camera component of the electronic device includes a lens and an image sensor. The lens is used to collect external light source signals and provide them to the image sensor. The image sensor senses the light source signal from the lens and converts it into digitized raw image data. That is, RAW image data. RAW images, which are unprocessed and uncompressed formats, can be aptly referred to as "digital negatives".

In some embodiments, step 101 specifically includes: continuously acquiring the first RAW image of the target scene through the image sensor according to the first exposure value; performing preset processing on the first RAW image, wherein the preset processing includes dead pixel correction, black at least one of level correction, noise reduction, lens brightness correction, and automatic white balance; and performing format conversion on the processed first RAW image to obtain a first YUV image. At this time, the first YUV image obtained through the preset processing occupies a lot of memory, so it is necessary to appropriately increase the dedicated memory for storing the first YUV image.

In some embodiments, after performing the preview operation on the target scene, the method further includes:

If a photographing instruction is received, the photographing instruction is responded to according to the first composite image.

In some embodiments, after performing the preview operation on the target scene, the method further includes:

If it is currently in the video recording mode, video coding processing is performed on the first composite image to obtain a video corresponding to the target scene.

As shown in FIG. 6 , for an electronic device with a real-time preview function, such as an electronic device such as a terminal, the electronic device obtains the newly stored 3 frames of the first YUV image from the first image buffer queue, that is, the first YUV image stored at time T _i-1 The image, the image stored at the time T _i-2 , and the image stored at the time T _i-3 , the three frames of the first YUV images are synthesized to obtain a first synthesized image with a high dynamic range, i=4, 5, 6, 7, 8..., that is, since the electronic device performs 103, 104, and 105 for the first time (previewing the target scene according to the first composite image), every time the first image buffer queue stores a new image When the first YUV image is framed, the electronic device executes 103, 104, and 105 once (previewing the target scene according to the first composite image). Therefore, when a photographing instruction is received at a certain time from time T _i-1 to time T _i , it is not necessary to repeat the same multi-frame first YUV images: the image stored at time T _i-1 , the image stored at time T _i-2 The image stored at time and the image stored at time T _i-3 are synthesized, and the latest first synthesized image for preview operation can be directly called to respond to the photographing instruction; when a video recording start instruction is received at a certain time.

In the same way, when a video recording start command is received from time T _i-1 to time T _i , there is no need to repeat the same multi-frame first YUV images: the image stored at time T _i-1 , the image stored at time T i The image stored at time _i-2 and the image stored at time T _i-3 are synthesized, and the latest first synthesized image for preview operation can be directly called as the first frame of the video. Next, whenever the first image cache queue is updated To store one frame of image, the electronic device can perform 103 (when video recording is required, obtain the newly stored multiple frames of the first YUV image from the first image cache queue), 104, 105 (according to the first composite image, perform a Recording operation) to obtain other frames other than the first frame of the video, until the end command of recording is received.

In some embodiments, referring to FIG. 5, 104 may include: 1041 and 1042:

1041. Divide the first YUV image into multiple target areas, and determine area information of the target areas.

1042. Combine multiple frames of the first YUV images according to the area information of the target area to obtain a first combined image with a high dynamic range.

The region information includes region location information, region pixel mean, and target pixel mean. The target area of the first YUV image in each frame is the same, and each target area corresponds to one or more area location information. Through the area location information, the electronic device can determine the corresponding target area on each frame of the first YUV image. Specifically, the area location information may be area coordinate information. For example, a coordinate system is established with the center of the first YUV image of each frame as the origin, and the area coordinate information corresponding to the target area is obtained. For the multiple frames of the first YUV image obtained from the first image cache queue, the target area of each frame of the first YUV image has its own pixel average value, which may be the same or different. Calculate the average value of all the pixel average values. , the obtained value is the regional pixel mean of the target area. Each target area corresponds to an average value of target pixels, and the average value of target pixels is an expected average value of pixels corresponding to the target area of the first synthesized image obtained after the synthesis process.

In some embodiments, the electronic device may pre-store a first mapping relationship, where the first mapping relationship is a corresponding relationship between the target pixel mean and the target object. Specifically, image recognition is performed on the first YUV image to obtain the recognition object of the first YUV image; according to the recognition object, the first YUV image is divided into a plurality of target areas, wherein each target area includes a recognition object; The area acquires the area location information, calculates the area pixel mean value of the target area, and searches the target pixel mean value corresponding to the target area from the first mapping relationship according to the identification object to obtain the area information of the target area.

In the embodiment of the present application, after the same target area division is performed on multiple frames of the first YUV image, for example, the first YUV image is divided into 4 target areas A, B, C, and D, taking the target area A as an example, according to The area information of the target area A, the target area A of the multiple frames of the first YUV image is synthesized, and the target areas B, C, and D of the multiple frames of the first YUV image are synthesized in the same way, and finally the first YUV image with high dynamic range is obtained. Composite image.

In some embodiments, before performing 1042, the method may further include: performing noise reduction processing on the multiple frames of the first YUV image.

Specifically, 1042 may include:

Determine the number of synthetic frames corresponding to the target area according to the average value of the area pixels and the average value of the target pixels;

Based on the regional position information of the target area and the number of synthesized frames, multiple frames of the first YUV images are synthesized to obtain a first synthesized image with a high dynamic range.

Wherein, the composite frame number is the value obtained by performing integer wavelet transformation on the average pixel value of the region and the target pixel average value. For example, the composite frame number 3 is the value obtained by performing integer wavelet transformation on the regional pixel average value 2 and the target pixel average value 5.

It should be noted that, by first dividing the image into multiple target regions, and then synthesizing each target region, the dynamic range of the first composite image can be better controlled than that of synthesizing the entire image. For example, the electronic device acquires 3 frames of the first YUV image from the first image cache queue. As shown in FIG. 7 , the first YUV image is divided into a sky area and a building area, wherein the sky area in each frame of the short-exposure image is in the sky area. The area pixel mean is 10, the building area pixel mean is 3, and the sky area target pixel mean is 30, and the building area target pixel mean is 6. ①If it is synthesized according to the target area, as shown in Figure 7, according to the regional pixel average value of the sky area and the target pixel average value, determine the number of composite frames in the sky area 30/10=3, and for the sky in the first YUV image of the 3 frames The area is synthesized to obtain the sky area of the first synthesized image. Similarly, according to the regional pixel average value of the building area and the target pixel average value, the number of synthesized frames in the building area is determined to be 6/3=2. For the first YUV image of the two frames The building area in the image is synthesized to obtain the building area of the first synthesized image. At this time, the average pixel value of the sky region in the first composite image is about 10+10+10=30, and the pixel average value of the building region is about 3+3=6. The execution of "obtaining the sky area of the first composite image" is not affected by "obtaining the building area of the first composite image", and both can be performed simultaneously. ②If the synthesis is not performed according to the target area, for example, the entire area of the first YUV image of 3 frames is synthesized to obtain the first synthesized image. The average pixel value of the sky area in the first synthesized image is about 10+10+10=30, The average pixel value of the building area is about 3+3+3=9. For example, the entire area of the 2 frames of the first YUV image is synthesized to obtain a first composite image. The average pixel value of the sky area in the first composite image is approximately 10+10=20, the average pixel value of the building area is about 3+3=6. It can be seen that by first dividing the image into multiple target regions, and then synthesizing each target region, the dynamic range of the synthesized image is better than that of synthesizing the entire image.

It can be seen from the above that the dynamic range of the synthesized image obtained is better than that of synthesizing the entire image by first dividing the image into multiple target regions, and then synthesizing each target region.

Please refer to FIG. 8 , which is a third schematic flowchart of the image processing method provided by the embodiment of the present application, which will be described in detail below.

In some embodiments, the image processing method, after 104 and before 105, further includes:

106. If the brightness of the first composite image is lower than the target brightness, perform a brightening process on the first composite image to obtain a first composite image with the target brightness.

The target brightness is the brightness to be achieved by the first composite image, and the electronic device can adjust the brightness of the first composite image according to the target brightness.

In some embodiments, the electronic device may preset a second mapping relationship, where the second mapping relationship may be a correspondence between target brightness and other elements, for example, other elements may be target textures, target colors, target objects, etc. , which is not specifically limited here. The electronic device may search for the target brightness corresponding to the first composite image in the second mapping relationship according to the texture, color, or object of the first composite image, and so on. Specifically, the value of the target brightness may be represented by a pixel value, and if the brightness of the first composite image is lower than the target brightness, it may be: the pixel value of the first composite image is lower than the target brightness value. It should be noted that the mapping relationship can be updated and modified.

After synthesizing multiple frames of first YUV images and obtaining a first composite image with a high dynamic range, there may be cases where the brightness of the first composite image does not reach the target brightness, for example, 3 frames of the first YUV images are composited The first composite image is obtained, but the brightness of the first composite image may not reach the target brightness. In this case, incremental processing may be performed on the first composite image.

In some embodiments, the target brightness may correspond to the entire frame of the first composite image, and if the brightness of the first composite image is lower than the target brightness, the brightness of the entire frame of the first composite image is increased to the target brightness.

Or, when the first composite image is composited in sub-regions, the brightness of some regions may not reach the target brightness. For example, the first composite image includes a sky area and a building area. In this case, the brightness of the sky area in the first composite image may reach the target brightness, but the brightness of the building area does not reach the target brightness. In view of this situation, after obtaining a first composite image with a high dynamic range, the present application performs a brightening process on an area where the brightness of the first composite image is lower than the target brightness.

In some other implementations, the target brightness may also be the target brightness corresponding to the partial area in the first composite image. In this case, there may be multiple target brightnesses. If the brightness of the first composite image is lower than the target brightness, the brightness enhancement process is performed. is the partial area corresponding to the target brightness in the first composite image. For example, image recognition is performed on the first composite image to obtain that the first composite image includes the sky and buildings; the target brightness of the sky and the target brightness of the building are obtained; the sky brightness in the first composite image and the target brightness of the sky are obtained By comparison, if the brightness of the sky in the first composite image is less than the target brightness of the sky, then brighten the sky in the first composite image; compare the brightness of the buildings in the first composite image with the target brightness of the buildings, if the first composite image is If the brightness of the buildings in the composite image is lower than the target brightness of the buildings, then the buildings in the first composite image are brightened to obtain a first composite image with the target brightness.

There are various ways of brightening processing. For example, the brightening processing may be specifically: improving the brightness of the first composite image through color filtering processing, and for example, the highlighting processing may be specifically: adjusting the color level to improve the brightness of the first composite image. the brightness of the first composite image, and so on. The present application does not specifically limit the manner of brightening processing.

It can be seen from the above that the image processing method provided in this embodiment can perform brightening processing on the first composite image when the brightness of the first composite image is lower than the target brightness, thereby improving the brightness of the first composite image.

Please refer to FIG. 9 , which is a fourth schematic flowchart of the image processing method provided by the embodiment of the present application, which will be described in detail below.

In some embodiments, 101 includes 1011:

1011. Perform alternate exposures based on the first exposure value and the second exposure value to obtain a first YUV image of the target scene, wherein for every m consecutive exposures based on the first exposure value, one exposure is performed based on the second exposure value, and m Greater than 1, the second exposure value is greater than the first exposure value.

It should be noted that the embodiments of the present application are applicable to an electronic device including a camera. Because the second exposure value is greater than the first exposure value, the first YUV image obtained based on the second exposure value has more highlight details than the first YUV image obtained based on the first exposure value. Wherein, m can be any integer value greater than 1, for example, m can be 3, m can be 7, and so on.

For example, the first exposure value is -2EV, the second exposure value is 1EV, and the target scene is exposed in the form of "-2EV, -2EV, -2EV, 1EV, -2EV, -2EV, -2EV, 1EV..." , get the first YUV image.

In some embodiments, 103 includes 1031:

1031. When previewing or taking pictures or videos is required, acquire the newly stored m+1 frames of the first YUV image from the first image buffer queue.

It should be noted that the newly stored first YUV image is obtained from the first image cache queue, because in principle it must be guaranteed that there are at least two frames of the first YUV image obtained based on the first exposure value, and one frame based on the second exposure value. Therefore, the number of frames of the first YUV image obtained from the first image buffer queue each time is the same as the number of frames of the first YUV image obtained from each round of alternate exposure.

In order to ensure that the multi-frame first YUV images obtained from the first image cache queue each time include at least one frame of the first YUV image obtained based on the second exposure value, the first YUV image obtained from the first image cache queue of the present application each time The number of image frames may be greater than or equal to the number of first YUV image frames obtained in each round of alternate exposure. For example, assuming that m is equal to 3, the first image buffer queue successively stores the first YUV images obtained based on the exposure values "-2EV, -2EV, -2EV, 1EV, -2EV, -2EV, -2EV, 1EV...". If it is performed according to the acquisition of the newly stored m+1 frames of the first YUV image from the first image buffer queue, such as the acquisition of 4 frames of the first YUV image based on the exposure values "-2EV, -2EV, -2EV, 1EV", Acquire 4 frames of first YUV images based on exposure values "-2EV, -2EV, 1EV, -2EV", and obtain 4 frames of first YUV images based on exposure values "-2EV, 1EV, -2EV, -2EV", Four first YUV images of "1EV, -2EV, -2EV, -2EV" are acquired, and so on. This ensures that three frames of the first YUV image obtained based on the first exposure value and one frame of the first YUV image obtained based on the second exposure value can be acquired each time.

In some embodiments, 104 includes:

1043. Perform synthesis processing on the m frames of the first YUV images obtained based on the first exposure value to obtain a second synthesized image with a high dynamic range.

1044. Perform synthesis processing on a frame of the first YUV image obtained based on the second exposure value and the second composite image to obtain a first composite image with a high dynamic range.

In this embodiment of the present application, after the newly stored m+1 frames of the first YUV image are acquired from the first image cache queue, m frames obtained based on the first exposure value are extracted from the acquired m+1 frames of the first YUV image. For the first YUV image, the m frames of the first YUV image are synthesized to obtain a second synthesized image with a high dynamic range. After the second composite image with high dynamic range is obtained, 1 frame of the first YUV image obtained based on the second exposure value can also be extracted from the acquired m+1 frames of the first YUV image, and the second composite image and the The 1 frame of the first YUV image is synthesized to obtain a first synthesized image with a high dynamic range.

For example, obtain the first YUV image based on the exposure values "-2EV, -2EV, 1EV, -2EV" of the newly stored 3+1 frames from the first image buffer queue; In the first YUV image obtained by 1EV, -2EV", extract 3 frames of first YUV images obtained based on the first exposure value "-2EV", and extract 1 frame of the first YUV image obtained based on the second exposure value "1EV"; Perform synthesis processing on the 3 frames of the first YUV image to obtain a second composite image with high dynamic range, and perform composite processing on the second composite image and the 1 frame of the first YUV image to obtain a first composite image with a specific high dynamic range image.

In some embodiments, 1044 may specifically include: determining a first pixel weight ratio of the second composite image; determining a second pixel weight ratio of a frame of the first YUV image obtained based on the second exposure value; according to the first pixel weight ratio The second composite image and a frame of the first YUV image are composited according to the second pixel weight ratio to obtain a first composite image with a high dynamic range.

In some embodiments, 1044 may specifically include: determining a first replacement area of the second composite image; obtaining, according to the first replacement area, a second replacement area corresponding to a frame of the first YUV image obtained based on the second exposure value; using The second replacement area replaces the first replacement area of the second composite image to obtain a first composite image with a high dynamic range.

It can be seen from the above that the image processing method provided in this embodiment can also obtain a second composite image by synthesizing m frames of short-exposure images, and then synthesizing the second composite image with a long-exposure image to obtain a second composite image. This increases the details of the highlights of the second composite image to obtain a better first composite image.

Please refer to FIG. 10. FIG. 10 is a fifth schematic flowchart of the image processing method provided by the embodiment of the present application, which will be described in detail below.

In some embodiments, referring to Figure 10, 101 includes 1012:

1012. Based on the first exposure value, continuously obtain the first YUV image of the target scene through the first camera, and continuously obtain the second YUV image of the target scene through the second camera based on the third exposure value, wherein the third exposure value is greater than first exposure value.

In the embodiment of the present application, the electronic device includes at least a first camera and a second camera. When acquiring the image of the target scene, based on the first exposure value, the first YUV image of the target scene is continuously acquired by the first camera; and based on the third exposure value, the second YUV image of the target scene is continuously acquired by the second camera, wherein , the third exposure value is greater than the first exposure value.

In some embodiments, the electronic device may continuously expose the target scene through the first camera and the second camera simultaneously, to obtain a first YUV image obtained based on the first exposure value and a second YUV image obtained based on the third exposure value .

For example, the electronic device uses the first camera and the second camera to continuously expose the target scene, and obtains the first RAW image with exposure values of "-2EV, -2EV, -2EV, -2EV..." through the first camera. And perform format conversion on the first RAW image to obtain a first YUV image with exposure values of "-2EV, -2EV, -2EV, -2EV...", and obtain exposure values of "1EV, 1EV" through the second camera , 1EV, 1EV, .

In some embodiments, see 10, 102 includes 1021:

1021. Store the second YUV image in the second image buffer queue according to the exposure order of the first camera, and store the second YUV image in the second image buffer queue according to the exposure order of the second camera.

In the embodiment of the present application, after continuously acquiring the first YUV image of the target scene through the first camera based on the first exposure value, the acquired first YUV image needs to be stored in the first image cache queue in the order of exposure; and After the second YUV image of the target scene is continuously acquired by the second camera based on the third exposure value, the second YUV image needs to be stored in the second image buffer queue according to the exposure sequence.

It should be noted that, in the embodiments of the present application, the first YUV image and the second YUV image are stored separately, but in some embodiments, the first YUV image and the second YUV image may also be stored together.

In some embodiments, see 10, 103 includes 1032:

1032. When previewing, photographing or video recording is required, acquire the newly stored multiple frames of the first YUV image from the first image cache queue, and acquire the newly stored one frame of the second YUV image from the second image cache queue.

In the embodiment of the present application, when previewing or taking pictures or video recording is required, it is necessary to obtain newly stored multiple frames of first YUV images from the first image cache queue, and obtain a newly stored one-frame image from the second image cache queue. Frame the second YUV image. It should be noted that the acquisition of the newly stored multiple frames of the first YUV image from the first image cache queue may be performed at the same time as the acquisition of the latest stored one frame of the second YUV image from the second image cache queue, or may not be performed simultaneously. Carry out: firstly acquire the newly stored multiple frames of the first YUV image from the first image cache queue, and then acquire the newly stored one frame of the second YUV image from the second image cache queue.

In some embodiments, see 10, 104 includes 1045 and 1046:

1045. Perform synthesis processing on the multiple frames of the first YUV images to obtain a third synthesized image with a high dynamic range.

In the embodiment of the present application, after the newly stored multiple frames of the first YUV images are acquired from the first image buffer queue, it is necessary to perform synthesis processing on the multiple frames of the first YUV images to obtain a third synthesized image with a high dynamic range. Among them, 1045, the specific implementation can refer to the above-mentioned embodiment, and details are not repeated here.

1046. Perform a composite process on the third composite image and a frame of the second YUV image to obtain a first composite image with a high dynamic range.

In this embodiment of the present application, after acquiring a newly stored frame of a second YUV image from the second image buffer queue and obtaining a third composite image with a high dynamic range, the third composite image and the 1 frame The second YUV composite image is subjected to composite processing to obtain a first composite image with a high dynamic range.

In some embodiments, 1046 may specifically include: determining a third pixel weight ratio of the third composite image; determining a fourth pixel weight ratio of a frame of the second YUV image; according to the third pixel weight ratio and the fourth pixel weight ratio, A first composite image with a high dynamic range is obtained by performing composite processing on a frame of the second YUV image and the third composite image.

In some embodiments, 1046 may specifically include: determining a third replacement area of the third composite image; obtaining a fourth replacement area corresponding to a frame of the second YUV image according to the third replacement area; using the fourth replacement area to replace the Three alternate regions of the composite image, resulting in a first composite image with high dynamic range.

It can be seen from the above that the image processing method provided in this embodiment can obtain a third composite image by synthesizing multiple frames of short exposure images, and then synthesizing the third composite image with a long exposure image. , so as to increase the details of the bright part of the third composite image, and obtain the first composite image with better effect.

In order to facilitate better implementation of the image processing method provided by the embodiment of the present application, the embodiment of the present application further provides an image processing apparatus based on the above-mentioned image processing method. The meanings of the nouns are the same as those in the above image processing method, and the specific implementation details can refer to the description in the method embodiment.

Please refer to FIG. 11 , which is a schematic diagram of a first structure of an image processing apparatus provided by an embodiment of the present application. Specifically, the image processing apparatus 200 includes: a first acquisition module 201 , a storage module 202 , a second acquisition module 203 , a synthesis module 204 , and a processing module 205 .

The first acquisition module 201 is configured to continuously acquire the first YUV image of the target scene based on the first exposure value;

The storage module 202 is configured to store the first YUV image in the first image buffer queue according to the exposure sequence;

The second acquisition module 203 is configured to acquire the newly stored multi-frame first YUV images from the first image cache queue when previewing or taking pictures or videos is required;

A synthesis module 204, configured to perform synthesis processing on the multiple frames of the first YUV images to obtain a first synthesized image with a high dynamic range;

The processing module 205 is configured to preview or photograph or record the target scene according to the first composite image.

In some embodiments, the first acquisition module 201 is specifically configured to: continuously acquire the first RAW image of the target scene through the image sensor according to the first exposure value; perform format conversion on the first RAW image to obtain the first RAW image. A YUV image.

In some embodiments, the processing module 205, after performing the preview operation on the target scene, further includes: if a photographing instruction is received, responding to the photographing instruction according to the first composite image.

In some embodiments, the processing module 205, after performing the preview operation on the target scene, further includes: if the target scene is currently in a video recording mode, performing video encoding processing on the first composite image to obtain a corresponding image of the target scene. video.

In some embodiments:

The first acquisition module 201 can also be configured to perform alternate exposures based on the first exposure value and the second exposure value to obtain a first YUV image of the target scene, wherein m consecutive exposures are performed each time based on the first exposure value , an exposure is performed based on the second exposure value, and m is greater than 1, and the second exposure value is greater than the first exposure value.

The second obtaining module 203 may also be configured to obtain the newly stored m+1 frames of the first YUV image from the first image buffer queue when previewing, taking pictures or recordings are required.

The synthesis module 204 may also be configured to perform synthesis processing on m frames of the first YUV images obtained based on the first exposure value to obtain a second composite image with a high dynamic range; The frame of the first YUV image is combined with the second combined image to obtain a first combined image with a high dynamic range.

In some embodiments:

The first acquisition module 201 can also be used to continuously acquire the first YUV image of the target scene through the first camera based on the first exposure value, and continuously acquire the second image of the target scene through the second camera based on the third exposure value. YUV image, wherein the third exposure value is greater than the first exposure value.

The storage module 202 can also be configured to store the first YUV image in the first image buffer queue in the order of exposure, and store the second YUV image in the second image buffer queue in the order of exposure.

The second obtaining module 203 can also be used to obtain the newly stored multi-frame first YUV images from the first image cache queue when previewing or taking pictures or videos is required, and obtain from the second image cache queue Obtain the latest stored second frame of the YUV image.

The synthesizing module 204 can also be used for synthesizing the multiple frames of the first YUV images to obtain a third synthesized image with a high dynamic range; synthesizing the third synthesized image and the one frame of the second YUV image processing to obtain a first composite image with a high dynamic range.

Please refer to FIG. 12. FIG. 12 is a schematic diagram of a second structure of an image processing apparatus provided by an embodiment of the present application.

In some embodiments, referring to FIG. 12 , the synthesis module 204 may include:

A dividing submodule 2041, for dividing the first YUV image into multiple target areas, and determining the area information of the target area;

The synthesis sub-module 2042 is configured to synthesize the multiple frames of the first YUV images according to the area information of the target area to obtain a first synthesized image with a high dynamic range.

In some embodiments, the area information includes area position information, area pixel mean value, and target pixel mean value, and the synthesis sub-module 2042 may be specifically configured to determine the target pixel value according to the area pixel mean value and the target pixel mean value The number of synthesized frames corresponding to the area; based on the area position information of the target area and the number of synthesized frames, the multiple frames of the first YUV images are synthesized to obtain a first synthesized image with a high dynamic range.

Please refer to FIG. 13 . FIG. 13 is a schematic diagram of a third structure of an image processing apparatus provided by an embodiment of the present application.

In some embodiments, referring to FIG. 13 , the image processing apparatus 200 may further include:

A third acquisition module 206, configured to acquire a frame of a second RAW image of the target scene through an image sensor;

a fourth acquisition module 207, configured to acquire the light ratio of the second RAW image;

a first determining module 208, configured to determine a reference exposure value according to the light ratio;

The second determination module 209 is configured to adjust at least one parameter of exposure duration, aperture, and sensitivity according to the reference exposure value, and determine the first exposure value.

In some embodiments, referring to FIG. 13 , the image processing apparatus 200 may further include:

The brightening processing module 210 is configured to perform brightening processing on the first composite image if the brightness of the first composite image is lower than the target brightness, so as to obtain the first composite image with the target brightness.

It can be seen from the above that, in the image processing device provided by the embodiment of the present application, when previewing or taking pictures or videos is required, the second obtaining module directly obtains the newly stored multi-frame first YUV images from the first image buffer queue, and the synthesizing module directly obtains the first YUV images of multiple frames from the first image cache queue. Combining multiple frames of the first YUV image can shorten the operation time of previewing, photographing, or recording, so that the image obtained by the image processing solution provided in this embodiment can be suitable for previewing, photographing, and recording images. In addition, after the HDR image is obtained by synthesizing multiple frames of short exposure, the HDR image can be brightened when the brightness of the HDR image is lower than the target brightness, or the HDR image can be synthesized with a long-exposure image, thereby Get a brighter image.

An embodiment of the present application further provides an electronic device. Please refer to FIG. 14 , which is a schematic diagram of a first structure of the electronic device provided by the embodiment of the present application. The electronic device 300 includes a processor 301 , a memory 302 , and a camera assembly 303 . The processor 301 is electrically connected to the memory 302 and the camera assembly 303 .

The processor 300 is the control center of the electronic device 300, uses various interfaces and lines to connect various parts of the entire electronic device, executes the electronic device by running or loading the computer program stored in the memory 302, and calling the data stored in the memory 302. Various functions of the device 300 are processed and data is processed to monitor the electronic device 300 as a whole.

The memory 302 can be used to store software programs and modules, and the processor 301 executes various functional applications and data processing by running the computer programs and modules stored in the memory 302 . The memory 302 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, a computer program required for at least one function, and the like; the storage data area may store data created according to the use of the electronic device, and the like. Additionally, memory 302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, memory 302 may also include a memory controller to provide processor 301 access to memory 302 .

The camera component 303 may include an image processing circuit, and the image processing circuit may be implemented by hardware and/or software components, and may include various processing units that define an image signal processing (Image Signal Processing) pipeline. The image processing circuit may at least include: a camera, an image signal processor (Image Signal Processor, ISP processor), a control logic, an image memory, a display, and the like. The camera may at least include one or more lenses and an image sensor. The image sensor may include an array of color filters (eg, Bayer filters). The image sensor can acquire light intensity and wavelength information captured with each imaging pixel of the image sensor and provide a set of raw image data that can be processed by an image signal processor.

Image signal processors can process raw image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the image signal processor may perform one or more image processing operations on the raw image data, collecting statistical information about the image data. Among them, the image processing operations can be performed with the same or different bit depth precision. The raw image data can be stored in the image memory after being processed by the image signal processor. The image signal processor may also receive image data from the image memory.

The image memory may be a part of a memory device, a storage device, or an independent dedicated memory in an electronic device, and may include a DMA (Direct Memory Access, Direct Memory Access) feature.

When receiving image data from the image memory, the image signal processor may perform one or more image processing operations, such as temporal filtering. The processed image data can be sent to the image memory for additional processing before being displayed. The image signal processor may also receive processed data from the image memory and process the processed data as image data in the raw domain and in the RGB and YCbCr color spaces. The processed image data may be output to a display for viewing by a user and/or further processed by a graphics engine or a GPU (Graphics Processing Unit, graphics processor). In addition, the output of the image signal processor can also be sent to an image memory, and the display can read image data from the image memory. In one embodiment, the image memory may be configured to implement one or more frame buffers.

Statistics determined by the image signal processor may be sent to the control logic. For example, the statistics may include statistics of the image sensor for auto exposure, auto white balance, auto focus, flicker detection, black level compensation, lens shading correction, etc.

Control logic may include a processor and/or microcontroller executing one or more routines (eg, firmware). One or more routines may determine camera control parameters and ISP control parameters based on the received statistics. For example, camera control parameters may include camera flash control parameters, lens control parameters (eg, focal length for focusing or zooming), or a combination of these parameters. ISP control parameters may include gain levels and color correction matrices, etc. for automatic white balance and color adjustment (eg, during RGB processing).

Please refer to FIG. 15 , which is a schematic structural diagram of an image processing circuit in this embodiment. As shown in FIG. 15 , for the convenience of description, only various aspects of the image processing technology related to the embodiments of the present invention are shown.

For example, the image processing circuit may include a camera, an image signal processor, a control logic, an image memory, and a display. Among them, the camera may include one or more lenses and image sensors. In some embodiments, the camera can be either a telephoto camera or a wide-angle camera.

The first image collected by the camera is transmitted to the image signal processor for processing. After the image signal processor processes the first image, the statistical data of the first image (such as the brightness of the image, the contrast value of the image, the color of the image, etc.) can be sent to the control logic. The control logic can determine the control parameters of the camera according to the statistical data, so that the camera can perform automatic focusing, automatic exposure and other operations according to the control parameters. The first image can be stored in the image memory after being processed by the image signal processor. The image signal processor can also read the image stored in the image memory for processing. In addition, the first image can be directly sent to the display for display after being processed by the image signal processor. The display can also read the image in the image memory for display.

In addition, not shown in the figures, the electronic device may further include a CPU and a power supply module. The CPU is connected to the logic controller, the image signal processor, the image memory and the display, and the CPU is used for global control. The power supply module is used to supply power to each module.

In this embodiment of the present application, the processor 301 in the electronic device 300 loads the instructions corresponding to the processes of one or more computer programs into the memory 302 according to the following steps, and is executed by the processor 301 and stored in the memory 302 The computer program in , so as to realize various functions, as follows:

Based on the first exposure value, continuously obtain the first YUV image of the target scene;

storing the first YUV image in the first image buffer queue according to the sequence of exposure;

When previewing or photographing or video recording is required, obtain the newly stored multi-frame first YUV images from the first image cache queue;

performing synthesis processing on the multiple frames of the first YUV images to obtain a first synthesized image with a high dynamic range;

According to the first composite image, a preview or photographing or video recording operation is performed on the target scene.

In some embodiments, when continuously acquiring the first YUV image of the target scene based on the first exposure value, the processor 301 specifically executes:

According to the first exposure value, the image sensor continuously acquires the first RAW image of the target scene;

Perform format conversion on the first RAW image to obtain the first YUV image.

In some embodiments, when performing composite processing on the multiple frames of the first YUV images to obtain a first composite image with a high dynamic range, the processor 301 specifically executes:

dividing the first YUV image into a plurality of target areas, and determining the area information of the target area;

According to the area information of the target area, the multiple frames of the first YUV images are synthesized to obtain a first synthesized image with a high dynamic range.

In some embodiments, when continuously acquiring the first YUV image of the target scene based on the first exposure value, the processor 301 executes:

Alternate exposure is performed based on the first exposure value and the second exposure value to obtain a first YUV image of the target scene, wherein for every m consecutive exposures based on the first exposure value, one exposure is performed based on the second exposure value exposure, and m is greater than 1, and the second exposure value is greater than the first exposure value;

When executing the acquisition of the newly stored multi-frame first YUV image from the first image cache queue, the processor 301 executes:

Obtain the newly stored m+1 frames of the first YUV image from the first image cache queue;

When performing synthesis processing on the first YUV images of the multiple frames to obtain a first synthesized image with a high dynamic range, the processor 301 executes:

performing synthesis processing on the m frames of the first YUV images obtained based on the first exposure value to obtain a second composite image with a high dynamic range;

A frame of the first YUV image obtained based on the second exposure value and the second composite image are composited to obtain a first composite image with a high dynamic range.

In some embodiments, when continuously acquiring the first YUV image of the target scene based on the first exposure value, the processor 301 executes:

Based on the first exposure value, the first YUV image of the target scene is continuously acquired by the first camera;

Before performing synthesis processing on the multiple frames of the first YUV images to obtain a first synthesized image with a high dynamic range, the processor 301 executes:

continuously acquiring a second YUV image of the target scene through a second camera based on a third exposure value, wherein the third exposure value is greater than the first exposure value;

storing the second YUV image in the second image buffer queue according to the sequence of exposure;

When it is necessary to preview or take a picture or record a video, obtain a newly stored second frame of the second YUV image from the second image cache queue;

When performing synthesis processing on the first YUV images of the multiple frames to obtain a first synthesized image with a high dynamic range, the processor 301 executes:

performing synthesis processing on the multiple frames of the first YUV images to obtain a third synthesized image with a high dynamic range;

The third composite image and the one frame of the second YUV image are composited to obtain a first composite image with a high dynamic range.

Please also refer to FIG. 16 . FIG. 16 is a schematic diagram of a second structure of an electronic device provided by an embodiment of the present application. In some embodiments, the electronic device 300 may further include: a display 304 , a radio frequency circuit 305 , an audio circuit 306 , and a power supply 307 . Among them, the display 304 , the radio frequency circuit 305 , the audio circuit 306 and the power supply 307 are respectively electrically connected to the processor 301 .

Display 304 may be used to display information entered by or provided to a user and various graphical user interfaces, which may be comprised of graphics, text, icons, video, and any combination thereof.

The radio frequency circuit 305 can be used to send and receive radio frequency signals, so as to establish wireless communication with the network device or other electronic devices through wireless communication, and to send and receive signals with the network device or other electronic devices.

Audio circuitry 306 may be used to provide an audio interface between the user and the electronic device through speakers, microphones.

Power supply 307 may be used to power various components of electronic device 300 . In some embodiments, the power supply 307 may be logically connected to the processor 301 through a power management system, so as to implement functions such as managing charging, discharging, and power consumption through the power management system.

Although not shown in FIG. 16 , the electronic device 300 may further include a Bluetooth module, etc., which will not be described herein again.

It can be seen from the above that when the electronic device provided in this embodiment needs to preview or take pictures or videos, the electronic device directly obtains the newly stored multi-frame first YUV images from the first image buffer queue for synthesis processing, which can shorten the preview or The operation time of photographing or video recording, so that the image obtained by the image processing solution provided in this embodiment can be suitable for image preview, photographing and video recording. In addition, after the HDR image is obtained by synthesizing multiple frames of short exposure, the HDR image can be brightened when the brightness of the HDR image is lower than the target brightness, or the HDR image can be synthesized with a long-exposure image, thereby Get a brighter image.

An embodiment of the present application further provides a storage medium, where a computer program is stored in the storage medium, and when the computer program runs on a computer, the computer is made to execute the image processing method in any of the foregoing embodiments, for example: based on the first exposure value, continuously obtain the first YUV image of the target scene; store the first YUV image in the first image cache queue according to the exposure sequence; Obtain the newly stored first YUV images of multiple frames; perform synthesis processing on the first YUV images of the multiple frames to obtain a first synthesized image with a high dynamic range; Preview or take pictures or video operations.

In this embodiment of the present application, the storage medium may be a magnetic disk, an optical disk, a read only memory (Read Only Memory, ROM), or a random access memory (Random Access Memory, RAM), or the like.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

It should be noted that, for the image processing method of the embodiment of the present application, ordinary testers in the art can understand that all or part of the process of realizing the image processing method of the embodiment of the present application can be completed by controlling the relevant hardware through a computer program , the computer program can be stored in a computer-readable storage medium, such as a memory of an electronic device, and executed by at least one processor in the electronic device, and the execution process can include, for example, the implementation of an image processing method example process. The storage medium may be a magnetic disk, an optical disk, a read-only memory, a random access memory, or the like.

For the image processing apparatus of the embodiment of the present application, each functional module thereof may be integrated in one processing chip, or each module may exist physically alone, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk or an optical disk.

The image processing method, device, storage medium, and electronic device provided by the embodiments of the present application have been described in detail above. The principles and implementations of the present application are described with specific examples. The descriptions of the above embodiments are only It is used to help understand the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there will be changes in the specific embodiments and application scope. In summary, this specification The content should not be construed as a limitation on this application.

Claims (8)

1. An image processing method, comprising:

acquiring a frame of second RAW image of a target scene through an image sensor;

acquiring the light ratio of the second RAW image;

determining a base exposure value from the light ratio, the light ratio being inversely proportional to the base exposure value;

adjusting at least one parameter of exposure duration, aperture and sensitivity according to the benchmark exposure value, and determining a first exposure value, wherein the first exposure value is smaller than the benchmark exposure value;

continuously acquiring a first RAW image of the target scene through the image sensor based on the first exposure value; performing preset processing on the first RAW image, and performing format conversion on the preprocessed first RAW image to obtain a first YUV image;

storing the first YUV image to a first image buffer queue according to the sequence of exposure;

when previewing or photographing or video recording is needed, obtaining a plurality of frames of first YUV images which are newly stored from the first image cache queue;

carrying out noise reduction processing on the multiple frames of the first YUV images;

dividing the first YUV image into a plurality of target areas, and determining area information of the target areas, wherein the area information comprises area position information, an area pixel mean value and a target pixel mean value;

determining a synthesis frame number corresponding to the target area according to the area pixel average value and the target pixel average value;

synthesizing the multiple frames of first YUV images based on the region position information and the synthesis frame number of the target region to obtain a first synthesis image with a high dynamic range;

if the brightness of the first composite image is smaller than the target brightness, performing brightening treatment on the first composite image to obtain the first composite image reaching the target brightness;

and previewing or photographing or recording the target scene according to the first synthetic image reaching the target brightness.

2. The image processing method according to claim 1, wherein after the synthesizing of the plurality of frames of the first YUV images to obtain the first synthesized image with a high dynamic range, before the previewing, photographing, or recording the target scene according to the first synthesized image, the method further comprises:

and if the brightness of the first synthetic image is not less than the target brightness, performing preview or photographing or video recording operation on the target scene according to the first synthetic image.

3. The image processing method of claim 1, wherein obtaining a first YUV image based on the first exposure value comprises:

alternately exposing based on the first exposure value and the second exposure value to obtain a first YUV image of a target scene, wherein each time m times of continuous exposure are carried out based on the first exposure value, one time of exposure is carried out based on the second exposure value, m is larger than 1, and the second exposure value is larger than the first exposure value;

the acquiring of the latest stored multi-frame first YUV image from the first image buffer queue includes:

acquiring a first YUV image of a newly stored m +1 frame from the first image cache queue;

the synthesizing of the multiple frames of the first YUV images to obtain a first synthesized image with a high dynamic range includes:

synthesizing the m frames of first YUV images obtained based on the first exposure value to obtain a second synthesized image with a high dynamic range;

and synthesizing the frame of the first YUV image obtained based on the second exposure value and the second synthetic image to obtain a first synthetic image with a high dynamic range.

4. The image processing method according to any one of claims 1 to 3, wherein after the preview operation is performed on the target scene, the method further comprises:

and if a photographing instruction is received, responding to the photographing instruction according to the first synthetic image.

5. The image processing method according to any one of claims 1 to 3, wherein after the preview operation is performed on the target scene, the method further comprises:

and if the current video mode is in the video recording mode, performing video coding processing on the first synthetic image to obtain a video corresponding to the target scene.

6. An image processing apparatus characterized by comprising:

the first acquisition module is used for acquiring a frame of second RAW image of a target scene through the image sensor; acquiring the light ratio of the second RAW image; determining a base exposure value from the light ratio, the light ratio being inversely proportional to the base exposure value; adjusting at least one parameter of exposure duration, aperture and sensitivity according to the benchmark exposure value, and determining a first exposure value, wherein the first exposure value is smaller than the benchmark exposure value; continuously acquiring a first RAW image of the target scene through the image sensor based on the first exposure value; performing preset processing on the first RAW image, and performing format conversion on the preprocessed first RAW image to obtain a first YUV image;

the storage module is used for storing the first YUV image to a first image cache queue according to the sequence of exposure;

the second acquisition module is used for acquiring a plurality of frames of first YUV images which are newly stored from the first image cache queue when previewing, photographing or video recording is required;

the synthesis module is used for carrying out noise reduction processing on the multi-frame first YUV image, dividing the first YUV image into a plurality of target areas and determining area information of the target areas, wherein the area information comprises area position information, an area pixel mean value and a target pixel mean value; determining a synthesis frame number corresponding to the target area according to the area pixel mean value and the target pixel mean value; synthesizing the multiple frames of first YUV images based on the region position information and the synthesis frame number of the target region to obtain a first synthesis image with a high dynamic range; if the brightness of the first composite image is smaller than the target brightness, performing brightening treatment on the first composite image to obtain the first composite image reaching the target brightness;

and the processing module is used for performing preview or photographing or video recording operation on the target scene according to the first composite image reaching the target brightness.

7. A storage medium having stored thereon a computer program which, when run on a computer, causes the computer to execute an image processing method according to any one of claims 1 to 5.

8. An electronic device comprising a processor and a memory, said memory having a computer program, wherein said processor is adapted to perform the image processing method of any of claims 1 to 5 by invoking said computer program.

Images