CN118158538B

CN118158538B - Wide dynamic automatic exposure control method and image acquisition equipment

Info

Publication number: CN118158538B
Application number: CN202410565805.4A
Authority: CN
Inventors: 储燎原; 王天臣
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2024-05-09
Filing date: 2024-05-09
Publication date: 2024-08-02
Anticipated expiration: 2044-05-09
Also published as: CN118158538A

Abstract

The embodiment of the application provides a wide dynamic automatic exposure control method and image acquisition equipment, wherein the method comprises the following steps: determining a long frame adjustment amplitude according to the actually measured exposure degree of the long frame and a preset long frame target exposure degree; determining a short frame adjustment amplitude according to the short frame actual measurement exposure degree and a preset short frame target exposure degree; determining a long frame initial shutter and a long frame initial gain of a second long frame image, and a short frame initial shutter and a short frame initial gain of a second short frame image; and according to the corresponding relation between the signal-to-noise ratio of the target image frame and the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain, and according to the direction for increasing the signal-to-noise ratio, the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain, the long frame target shutter, the long frame target gain, the short frame target shutter and the short frame target gain are obtained. By applying the technical scheme provided by the embodiment of the application, the quality of the image can be improved.

Description

Wide dynamic automatic exposure control method and image acquisition equipment

Technical Field

The application relates to the technical field of video monitoring, in particular to a wide dynamic automatic exposure control method and image acquisition equipment.

Background

When a camera collects an image of a target scene, if the target scene is illuminated by a strong light source (such as sunlight, a lamp or light reflection, etc.), a high-brightness area and a low-brightness area (such as a shadow, a backlight, etc. with relatively low brightness) may exist in the image at the same time, so that the bright area of the image becomes white due to overexposure, and the dark area becomes black due to underexposure, which seriously affects the quality of the image.

Disclosure of Invention

An object of an embodiment of the present application is to provide a wide dynamic automatic exposure control method and an image capturing apparatus, so as to improve the quality of a captured image when a high-brightness area and a low-brightness area exist in a target scene at the same time. The specific technical scheme is as follows:

A wide dynamic auto-exposure control method, the method comprising: responding to the wide dynamic synthesis to obtain a current image frame, and counting the exposure degree of a first long frame image to be used as the exposure degree of a long frame actual measurement; counting the exposure degree of a first short frame image as a short frame actual measurement exposure degree, wherein the first long frame image is a long frame image for widely and dynamically synthesizing the current image frame, and the first short frame image is a short frame image for widely and dynamically synthesizing the current image frame; determining the corresponding adjustment amplitude of the first long frame image according to the measured exposure degree of the long frame and the preset target exposure degree of the long frame, and taking the adjustment amplitude as the long frame adjustment amplitude; determining the corresponding adjustment amplitude of the first short frame image according to the short frame actual measurement exposure degree and the preset short frame target exposure degree, and taking the adjustment amplitude as the short frame adjustment amplitude; determining initial long frame exposure of a second long frame image according to the exposure of the first long frame image and the long frame adjustment amplitude; determining initial short frame exposure of a second short frame image according to the exposure of the first short frame image and the short frame adjustment amplitude, wherein the second long frame image is a long frame image for widely and dynamically synthesizing a target image frame, the second short frame image is a short frame image for widely and dynamically synthesizing the target image frame, and the target image frame is an image frame obtained by shooting the current image frame in a follow-up mode; determining a long frame initial shutter and a long frame initial gain of the second long frame image according to the initial long frame exposure and the initial short frame exposure, wherein the long frame initial gain is equal to the short frame initial gain; according to the corresponding relation between the signal-to-noise ratio of the target image frame and the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain, the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain are adjusted according to the direction for increasing the signal-to-noise ratio, and a long frame target shutter, a long frame target gain, a short frame target shutter and a short frame target gain are obtained; shooting according to the long frame target shutter and the long frame target gain to obtain a second long frame image, and shooting according to the short frame target shutter and the short frame target gain to obtain a second short frame image; and obtaining the target image frame by wide dynamic synthesis according to the second long frame image and the second short frame image.

The embodiment of the application has the beneficial effects that:

In the technical scheme provided by the embodiment of the application, the image acquisition equipment can compare the measured exposure degree of the long frame with the preset long frame target exposure degree, and compare the measured exposure degree of the short frame with the preset short frame target exposure degree to determine the long frame adjustment amplitude and the short frame adjustment amplitude, so that the adjusted long frame image and short frame image, namely the exposure degree of the second long frame image and the second short frame image, are positioned in the target exposure degree, and therefore, the second long frame image can better present the image of a darker area in a scene, and the second short frame image can better present the image of a lighter area in the scene, therefore, the dark area and the bright area in the target image frame synthesized based on the second long frame image and the second short frame image are clearer, and the image quality is better; on the basis, the influence of gain on the exposure ratio is not considered, only a shutter is used as a means for adjusting the exposure ratio, and then the initial shutter and the initial gain of the second long frame image are determined according to the initial long frame exposure and the initial short frame exposure; according to the corresponding relation between the signal-to-noise ratio of the target image frame and the initial shutter, the initial gain, the initial shutter and the initial gain, the initial shutter, the initial gain and the initial gain are adjusted according to the direction of increasing the signal-to-noise ratio, so as to obtain the target shutter, the target gain, the target shutter and the target gain, thereby, the second long frame image is obtained by shooting the target shutter and the target gain, the signal-to-noise ratio of the second short frame image is obtained by shooting the target shutter and the target gain, and the signal-to-noise ratio of the target image frame is better, and thus, the signal-to-noise ratio of the target image frame is better.

Of course, it is not necessary for any one product or method of practicing the application to achieve all of the advantages set forth above at the same time.

Drawings

In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the application, and other embodiments may be obtained according to these drawings to those skilled in the art.

FIG. 1 is a schematic flow chart of a wide dynamic automatic exposure control method according to an embodiment of the present application;

FIG. 2 shows the present application provided by the examples first step of S15 a seed refinement schematic diagram;

Fig. 3 is a second detailed schematic diagram of step S15 according to an embodiment of the present application;

fig. 4 is a third detailed schematic diagram of step S15 according to an embodiment of the present application;

FIG. 5 is a flowchart illustrating an embodiment of determining an average signal-to-noise ratio of an image frame according to the present application;

FIG. 6 is a schematic diagram of a correspondence relationship between pixel values of a long and short frame positions according to an embodiment of the present application;

FIG. 7 is another schematic diagram of a correspondence relationship between pixel values at a long and short frame positions according to an embodiment of the present application;

FIG. 8 is a schematic overall flow chart of the wide dynamic automatic exposure control method using the example of FIG. 2 to adjust exposure parameters;

FIG. 9 is a detailed schematic diagram of step S85 in FIG. 8;

FIG. 10 is a detailed schematic diagram of step S87 in FIG. 8;

FIG. 11 is a schematic overall flow chart of a wide dynamic automatic exposure control method using the example of FIG. 3 to adjust exposure parameters;

FIG. 12 is a detailed schematic diagram of step S1111-step S1113 in FIG. 11;

fig. 13 is a schematic structural diagram of an image capturing device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by the person skilled in the art based on the present application are included in the scope of protection of the present application.

The terms appearing in the embodiments of the present application are explained below:

Dynamic range: the camera has limitations on the appearance of brightest and darker areas in the same scene, such as short exposure times, which can result in dark areas that are too dark, long exposure times, loss of bright area information due to overflow, etc., which is commonly referred to as the "dynamic range", i.e., the ratio of the maximum to minimum light values in the scene.

Wide dynamic mode: the technology is that a plurality of frames of images are obtained by shooting the same scene with different exposure amounts, and a frame of image is obtained by synthesizing the plurality of frames of images; the description herein mainly deals with the case of synthesizing two frames of images into one frame of image, wherein, among the two frames of images, an image with longer exposure time is called a long frame image, and an image with shorter exposure time is called a short frame image; because the exposure time of the long frame image is longer, the darker area in the long frame image is not underexposed, and the exposure time of the short frame image is shorter, and the brighter area in the short frame image is not overexposed, so that the dark area is not excessively dark, the bright area is not overexposed and the image quality is better in the image frame synthesized by the long frame image and the short frame image.

Wide dynamic synthesis threshold: in the wide dynamic mode, when two frames of images are combined into one frame of image, a long frame threshold value and a short frame threshold value are generally set when a long frame of image and a short frame of image are combined, wherein pixels with luminance values within the long frame threshold value are all used as pixels of the combined image frame, pixels with luminance values greater than the short frame threshold value are all used as pixels of the combined image frame, and values of other pixels in the combined image frame are proportionally determined between the long frame of image and the short frame of image.

Linear mode: refers to a camera that does not use a wide dynamic mode, where the sensor outputs a single frame image.

As can be seen from the foregoing description of the wide dynamics, in the case of synthesizing two frames of images into one frame of image, the sensor needs to output two frames of images, i.e., a long frame of image and a short frame of image, when a single image frame is photographed using the wide dynamics mode, and thus, the operation modes of the sensor are different in the linear mode and the wide dynamics mode.

Exposure amount: indicating the integral of the illuminance received by the sensor pixel in time t; the magnitude of the exposure may be determined by the shutter (i.e., exposure time), the gain, and the aperture together, that is, the exposure=shutter×gain×aperture, the greater any one of the three is, the greater the exposure is; the external environment illumination is the same, and if the exposure is the same, the image brightness is the same. The following mainly describes the manner of adjusting the exposure by adjusting the gain and the shutter, and the principle of the manner of adjusting the shutter, the gain and the aperture to adjust the exposure is the same, so that the description is omitted; in the case of adjusting the exposure amount by adjusting the gain and the shutter, the influence of the aperture on the exposure amount can be ignored, that is, at this time, the calculation formula of the exposure amount can be regarded as: exposure = shutter x gain.

Exposure ratio: in the wide dynamic mode, when two frames of images are combined into one frame of image, the exposure ratio of the image frame is the ratio of the exposure amounts of a long frame image and a short frame image which are combined into the image frame; however, since the gains of the long frame image and the short frame image are generally kept the same when the long frame image and the short frame image are photographed in the related art, the exposure ratio of the image frame in the related art can be regarded as the shutter ratio of the long frame image and the short frame image that compose the image frame, that is, the shutter ratio of the long frame image and the short frame image. Since the following description will be mainly directed to a manner of adjusting the exposure amount by adjusting the gain and the shutter, the exposure ratio hereinafter refers to a ratio of the product of the shutter and the gain of the long frame image and the short frame image, and thus, in the present application, the manner of adjusting the exposure ratio may be a gain-same variable shutter, a shutter-same variable gain, or a combination of both.

Image signal-to-noise ratio: referring to the ratio of image signals to noise, the larger the signal-to-noise ratio is, the smaller the noise influence is, and the better the image quality is.

In the related art, a fixed exposure ratio is generally used in the wide dynamic mode, so that a scene with a larger dynamic range can be handled, a dark area is lightened, and a bright area is darkened, but because more nonlinear processing is adopted, compared with an image shot in the linear mode, the image shot by the wide dynamic mode has the defects of poor permeability, inaccurate color, large noise, unnaturalness and the like compared with the image shot by the fixed wide dynamic technology. To overcome these drawbacks, the related art generally makes judgment according to the scene, and switches between the linear mode and the wide dynamic mode dynamically, that is, to the linear mode when the wide dynamic mode is not required, thereby making the resulting image more natural. However, this process needs to switch the sensor working mode, which has the problem of long time consumption and easily causes frame loss of the video stream.

In order to solve the above problems, the industry also proposes a technical solution for dynamically changing the exposure ratio, where the exposure ratio is generally adjusted by the dynamic range of the scene, that is, in a wide dynamic scene, a larger exposure ratio is used, both a bright area and a dark area are considered, in a scene with a small dynamic range, a smaller exposure ratio is used to have an effect similar to that of a linear mode, thereby avoiding the working mode of a hard switching sensor, that is, avoiding the problems of time consuming and current cutting of switching.

The technical scheme of dynamic change of exposure ratio avoids switching the working mode of a sensor, but focuses more on calculation of the exposure ratio, ignores influence of distribution of exposure parameters (such as a shutter and a gain) on image effects, namely, the scheme only considers that the exposure ratio is a ratio of a long frame image to a short frame image shutter, and gains of the long frame image and the short frame image are the same, so that the long frame image and the short frame image cannot obtain optimal signal to noise ratio due to simple distribution of the exposure parameters.

For example, the gain of the long frame image and the short frame image is 30 decibels (dB); the frame rate required for the wide dynamic technique is 25 frames per second (FRAMES PER seconds, fps), that is, 25 image frames are synthesized per second, and therefore, the sum of shutters of long frame images and short frame images is 40 milliseconds (ms) at the maximum, at this time, the following two cases can be considered:

case one: when the exposure ratio is 16, the shutters of the long frame image and the short frame image are allocated according to the above scheme, and the shutters of the long frame image and the short frame image can be calculated by the following formula (1) and formula (2), respectively,

shutter_long=40ms×ratio/( ratio+1)≈37.65ms （1）

shutter_short=40ms×1/( ratio+1)≈2.35ms （2）

Wherein, the timer_long is the shutter of the long frame image, the timer_short is the shutter of the short frame image, and the ratio is the exposure ratio.

At this time, the short frame image signal-to-noise ratio is poor because the gain is high, the ambient illuminance is low, and the short frame shutter is low. Also, since the exposure is relatively high at this time, that is, the proportion of pixels of the short frame image in the synthesized image frame to the total pixels is relatively high, the signal-to-noise ratio of the short frame image has a large influence on the effect of the synthesized image frame, and therefore, such distribution may result in poor effect of the synthesized image frame.

And a second case: when the exposure ratio is 1, the shutters of the long frame image and the short frame image are allocated according to the above scheme, and the shutters of the long frame image and the short frame image can be calculated by the following formulas (3) and (4), respectively,

shutter_long=40ms×ratio/( ratio+1)=20ms （3）

shutter_short=40ms×1/( ratio+1)=20ms （4）

At this time, the exposure ratio is the lowest, that is, the proportion of the pixels of the short frame image to the total pixels in the synthesized image frame is relatively low, the proportion of the pixels of the long frame image to the total pixels is relatively high, so that the signal-to-noise ratio of the short frame image has a small influence on the effect of the synthesized image frame, and the signal-to-noise ratio of the long frame image has a large influence on the effect of the synthesized image frame, but at this time, the long frame image can only be allocated to half of the maximum shutter sum, that is, the shutter time of the long frame image and the short frame image is the same, so that the effect of the long frame image and the short frame image is almost the same, and the effect of the image frame synthesized by using the long frame image and the short frame image is the same as that of the long frame image and the short frame image, that is, that the shutter allocated in this way results in half of the time being wasted by the short frame image.

In order to solve the above-mentioned problems, the embodiment of the present application provides a wide dynamic automatic exposure control method, and the following describes the wide dynamic automatic exposure control method provided by the embodiment of the present application in detail through a specific embodiment.

The wide dynamic automatic exposure control method provided by the embodiment of the application can be applied to image acquisition equipment, and particularly, the image acquisition equipment can be a camera, an intelligent mobile terminal and the like.

The wide dynamic synthesis in the present application may be based on two frames of images to synthesize one image frame, or may be based on three or more frames of images to synthesize one image frame. Since the principle of the method of combining three or more images into one image frame is the same as that of combining two images into one image frame, the method of combining two images into one image frame will be mainly described herein, and when one image frame is combined from two images, the two images are respectively referred to as a long frame image and a short frame image, and at this time, a flow diagram of the above-described wide dynamic auto-exposure control method is shown in fig. 1 below.

Referring to fig. 1, fig. 1 is a schematic flow chart of a wide dynamic automatic exposure control method according to an embodiment of the present application, including the following steps:

Step S11: responding to the wide dynamic synthesis to obtain a current image frame, and counting the exposure degree of a first long frame image to be used as the exposure degree of a long frame actual measurement; counting the exposure degree of the first short frame image, and taking the exposure degree as the short frame actual measurement exposure degree;

Step S12: according to the measured exposure degree of the long frame and the preset target exposure degree of the long frame, determining an adjustment amplitude corresponding to the first long frame image as a long frame adjustment amplitude; determining the corresponding adjustment amplitude of the first short frame image according to the short frame actual measurement exposure degree and the preset short frame target exposure degree, and taking the adjustment amplitude as the short frame adjustment amplitude;

step S13: determining initial long frame exposure of the second long frame image according to the exposure of the first long frame image and the long frame adjustment amplitude; determining initial short frame exposure of the second short frame image according to the exposure of the first short frame image and the short frame adjustment amplitude;

Step S14: determining a long frame initial shutter and a long frame initial gain of a second long frame image according to the initial long frame exposure and the initial short frame exposure, wherein the long frame initial gain is equal to the short frame initial gain;

Step S15: according to the corresponding relation between the signal-to-noise ratio of the target image frame and the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain, according to the direction for increasing the signal-to-noise ratio, the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain are adjusted to obtain a long frame target shutter, a long frame target gain, a short frame target shutter and a short frame target gain;

Step S16: shooting according to the long frame target shutter and the long frame target gain to obtain a second long frame image, and shooting according to the short frame target shutter and the short frame target gain to obtain a second short frame image;

Step S17: and obtaining the target image frame by wide dynamic synthesis according to the second long frame image and the second short frame image.

That is, the application provides a method for analyzing the dynamic range of the current scene and dynamically adjusting the exposure ratio and the exposure parameters (i.e. shutter and gain) of the long frame image and the short frame image based on the exposure degree of the first long frame image and the first short frame image, so that the bright area and the dark area of the synthesized image frame can be seen clearly and the signal to noise ratio of the image is good.

In the step S11, the current image frame is a wide dynamic image based on wide dynamic synthesis, which may be understood as an image frame acquired at the current time, where the current time may be understood as a time when the wide dynamic automatic exposure control method shown in the present application starts to be executed, and the image frame acquired at the current time may be an image frame acquired at the current time, or may be a frame previous to the image frame acquired at the current time, which is not limited thereto. The current image frame is synthesized by a first long frame image and a first short frame image, wherein the first long frame image is a long frame image for wide dynamic synthesis of the current image frame, and the first short frame image is a short frame image for wide dynamic synthesis of the current image frame.

In the embodiment of the application, the measured exposure degree of the long frame can be the whole exposure degree of the first long frame image, and can also be the local exposure degree of the first long frame image, such as the underexposure degree of a dark area in the first long frame image; the exposure degree of the short frame actually measured can be the whole exposure degree of the first short frame image, and can also be the local exposure degree of the first short frame image, such as the overexposure degree of a bright area in the first short frame image.

It will be appreciated that the image acquisition device may determine the exposure level of the image by means of histogram statistics, blocking statistics, etc. Since the expected goal of the wide dynamic is to make the dark area of the long frame image not too dark and the bright area of the short frame image not too exposed, after the wide dynamic image is synthesized according to the long frame image and the short frame image, the bright area and the dark area in the wide dynamic image are clear, so the exposure degree needs to be capable of representing the brightness condition of the dark area of the long frame image and the brightness condition of the bright area of the short frame image.

In the above step S12, the preset target exposure level of the long frame may be understood as the desired exposure level of the long frame image, and the dark area in the long frame image should be made to be not too dark; the preset target exposure level of the short frame can be understood as the desired exposure level of the short frame image, and the bright area in the short frame image should not be overexposed. The preset long frame target exposure degree and the preset short frame target exposure degree may be specific values or a range. The long frame adjustment amplitude is determined by the measured exposure degree of the long frame and the preset target exposure degree of the long frame, and can be understood as the ratio of the exposure amount of the long frame image to the exposure amount of the first long frame image, which is needed to be obtained when the long frame image is shot later; the short frame adjustment amplitude is the same as the principle. The adjustment amplitude can be expressed by Δg, which is set to a value greater than 1 when the exposure amount needs to be increased, and to a value less than 1 when the exposure amount needs to be decreased; meanwhile, when the adjustment amplitude is determined, if the actually measured exposure degree is larger than the preset target exposure degree, the exposure adjustment step length can be increased, namely the adjustment amplitude is set larger; similarly, if the difference between the actually measured exposure degree and the preset target exposure degree is smaller, the exposure adjustment step length can be reduced, i.e. the adjustment amplitude is set smaller.

In the embodiment of the application, if the measured exposure degree of the long frame is matched with the preset target exposure degree of the long frame and the measured exposure degree of the short frame is matched with the preset target exposure degree of the short frame, the first long frame image and the first short frame image are indicated to reach the target exposure degree, namely, the exposure degree is stable at the moment, so that the exposure ratio and the exposure parameter do not need to be adjusted, and the next long frame image and the next short frame image can be shot directly according to the exposure ratio and the exposure parameter of the first long frame image and the first short frame image. Here, the matching of the measured exposure degree with the preset target exposure degree means that when the preset target exposure degree is a specific value, the measured exposure degree is the same as the specific value; when the preset target exposure degree is a range, the measured exposure degree is within the range of the preset target exposure degree.

If the measured exposure degree of the long frame is not matched with the preset target exposure degree of the long frame, or if the measured exposure degree of the short frame is not matched with the preset target exposure degree of the short frame, the image acquisition equipment can determine the adjustment amplitude of the long frame according to the measured exposure degree of the long frame and the preset target exposure degree of the long frame; and determining the short frame adjustment amplitude according to the short frame actual measurement exposure degree and the preset short frame target exposure degree.

In the step S13, the target image frame is an image frame captured after the current image frame is captured and the new exposure parameter is applied, for example, the target image frame may be understood as a first image frame captured when the long frame target shutter, the long frame target gain, the short frame target shutter, and the short frame target gain configured in the image capturing device are in effect. The long frame target shutter, the long frame target gain, the short frame target shutter, and the short frame target gain are explained in the following S15, and are not described here.

The initial long frame exposure of the second long frame image may be understood as an exposure obtained by adjusting the exposure of the first long frame image by a long frame adjustment range, that is, the exposure of the second long frame image is determined by the exposure of the first long frame image and the long frame adjustment range; the initial short frame exposure of the second short frame image may be understood as an exposure obtained by adjusting the exposure of the first short frame image by the short frame adjustment range, that is, the exposure of the second short frame image is determined by the exposure of the second short frame image and the short frame adjustment range.

Since the exposure amount is determined by the shutter and the gain, the image pickup apparatus can acquire the shutter and the gain of the first long frame image, and determine the exposure amount of the first long frame image based on the shutter and the gain of the first long frame image; the image pickup apparatus may further acquire a shutter and a gain of the first short frame image, and determine an exposure amount of the first short frame image based on the shutter and the gain of the first short frame image.

For example, the calculation formula of the exposure amount is shown in the following formula (5),

EV=shutter×Gain （5）

Where EV represents the exposure amount, shutter represents the shutter, and Gain represents the Gain.

The image pickup apparatus may calculate the exposure amount of the first long frame image by the following formula (6),

EV_{cur_l}= shutter_{cur_1}×Gain_{cur_1} （6）

Where EV _{cur_1} denotes the exposure amount of the first long frame image, timer _{cur_1} denotes the shutter of the first long frame image, and Gain _{cur_1} denotes the Gain of the first long frame image.

The image pickup apparatus may also calculate the exposure amount of the first short frame image by the following formula (7),

EV_{cur_s}= shutter_{cur_s}×Gain_{cur_s} （7）

Where EV _{cur_s} denotes the exposure amount of the first short-frame image, timer _{cur_s} denotes the shutter of the first short-frame image, and Gain _{cur_s} denotes the Gain of the first short-frame image.

Based on the exposure of the first long frame image and the exposure of the first short frame image, the image pickup apparatus may determine the total exposure of the current image frame according to the following formula (8),

EV_{cur_t}= EV_{cur_1}+EV_{cur_s} （8）

Where EV _{cur_t} represents the total exposure of the first long frame image and the first short frame image, which may also be referred to as the total exposure of the current image frame.

Taking the long frame adjustment amplitude Δg ₁ as an example, the image capturing apparatus may calculate the initial long frame exposure of the second long frame image by the following formula (9) based on the exposure of the first long frame image and the long frame adjustment amplitude,

EV_{nxt_ori_l}= EV_{cur_1}×△G₁ （9）

Where EV _{nxt_ori_l} denotes an initial long frame exposure amount of the second long frame image, and Δg ₁ denotes a long frame adjustment amplitude, that is, a relative ratio calculated by a difference of the exposure degree of the first long frame image with respect to a preset long frame target exposure degree.

Taking the example of the short frame adjustment amplitude being ΔG _s, the image capturing apparatus may calculate the initial short frame exposure of the second short frame image by the following formula (10) based on the exposure of the first short frame image and the short frame adjustment amplitude,

EV_{nxt_ori_s}= EV_{cur_s}×△G_s （10）

Where EV _{nxt_ori_s} denotes an initial short frame exposure amount of the second short frame image, and Δgs denotes a short frame adjustment amplitude, that is, a relative ratio calculated by a difference of the exposure degree of the first short frame image with respect to a preset short frame target exposure degree.

Based on the initial long frame exposure of the second long frame image, the initial short frame exposure of the second short frame image, the image pickup apparatus may calculate the total exposure of the target image frames by the following formula (11),

EV_{nxt_ori_t}=EV_{nxt_ori_l}+EV_{nxt_ori_s}=shutter_{nxt_ori_t}×Gain_{nxt_ori_t} （11）

Where EV _{nxt_ori_t} denotes the total exposure of the target image frames, and timer _{nxt_ori_t} denotes the shutter of the target image frames when only the shutter is the means of exposure ratio, that is, the shutter sum value of the second long frame image and the second short frame image, and Gain _{nxt_ori_t} denotes the Gain of the target image frames.

It can be understood that the initial long frame exposure, the initial short frame exposure, and the initial shutter and the initial gain calculated hereinafter are the shutter, the gain, and the exposure of the target image frame calculated primarily in the process of calculating the contents of the target shutter, the target gain, and the like of the first priority frame and the second priority frame, and do not represent the shutter, the gain, and the exposure used when capturing the current image frame; meanwhile, the initial long frame exposure amount, the initial short frame exposure amount, and the initial shutter, the initial gain calculated hereinafter are used only for the content such as the target shutter and the target gain for the subsequent determination of the first priority frame and the second priority frame, and are not used for photographing the image frame, that is, the long frame image and the short frame image are not photographed using the initial shutter and the initial gain and the image frame is obtained.

In the step S14, the long frame initial gain and the short frame initial gain are the gain of the second long frame image and the gain of the second short frame image determined when the gain of the second long frame image and the gain of the second short frame image are the same, and therefore, the long frame initial gain and the short frame initial gain are the same; the long frame initial shutter and the short frame initial shutter refer to a shutter of the second long frame image and a shutter of the second short frame image calculated in the case where the gain of the second long frame image is the same as the gain of the second short frame image.

After the image acquisition device determines the initial long frame exposure and the initial short frame exposure, calculating a long frame initial shutter of the second long frame image and a short frame initial shutter of the second short frame image under the condition that the gain of the second long frame image is the same as the gain of the second short frame image.

For example, based on the initial long frame exposure of the second long frame image and the initial short frame exposure of the second short frame image, the image pickup apparatus may calculate the initial exposure ratio ExpRatio _{nxt_ori_t} of the target image frame by the following formula (12),

ExpRatio_{nxt_ori_t}=EV_{nxt_ori_l}/EV_{nxt_ori_s} （12）

Taking the maximum value (i.e., the maximum shutter and value) of the shutters and values of the long frame image and the short frame image as an example of shutterMAX, the relationship between the long frame image shutter shunt _l, the short frame image shutter shunt _s, the shutters and values of the long frame image and the short frame image Shutter _t, and the maximum shutter and value shutterMAX can be expressed by the following formula (13),

Shutter_t=shutter_l+shutter_s≤shutterMAX （13）

In the case where only the shutter is used as a parameter for determining the exposure ratio, that is, in the case where the long-frame initial gain is equal to the short-frame initial gain, the image capturing apparatus may further determine the long-frame initial shutter and the long-frame initial gain of the second long-frame image, and the short-frame initial shutter and the short-frame initial gain of the second short-frame image in combination with the influence of the shutter and the gain on the signal-to-noise ratio (that is, the gain needs to be reduced to increase the image signal-to-noise ratio).

For example, the image capturing apparatus may determine the long-frame initial shutter and the long-frame initial gain of the second long-frame image, and the short-frame initial shutter and the short-frame initial gain of the second short-frame image by:

(a) When EV _{nxt_ori_t} is equal to or less than ShutterMAX ×1.0, the image capture device can calculate a long-frame initial shutter, a long-frame initial gain, a short-frame initial shutter, and a short-frame initial gain by the following formulas (14) - (17):

shutter_{nxt_ori_t}=EV_{nxt_ori_t}/Gain_{nxt_ori_t}=EV_{nxt_ori_t}/1.0 （14）

Gain_{nxt_ori_l}=Gain_{nxt_ori_s}=Gain_{nxt_ori_t}=1.0 （15）

shutter_{nxt_ori_l}=EV_{nxt_ori_l}/Gain_{nxt_ori_l}= EV_{nxt_ori_l}/1.0 （16）

shutter_{nxt_ori_s}=EV_{nxt_ori_s}/Gain_{nxt_ori_s}=EV_{nxt_ori_s}/1.0 （17）

(b) When EV _{nxt_ori_t} > ShutterMAX ×1.0, shutternxt _ori_t= ShutterMAX, the image capturing apparatus can calculate a long frame initial shutter, a long frame initial gain, a short frame initial shutter, and a short frame initial gain by the following formulas (18) - (20):

Gain_{nxt_ori_l}=Gain_{nxt_ori_s}=Gain_{nxt_ori_t}=EV_{nxt_ori_t}/ShutterMAX （18）

shutter_{nxt_ori_l}=EV_{nxt_ori_l}/Gain_{nxt_ori_l} （19）

shutter_{nxt_ori_s}=EV_{nxt_ori_s}/Gain_{nxt_ori_s} （20）

Wherein, the shutter _{nxt_ori_t} represents the shutter and the value of the second long frame image and the second short frame image, that is, the shutter of the target image frame; EV _{nxt_ori_t} represents the total exposure of the target image frames; gain _{nxt_ori_t} represents the Gain of the target image frame; gain _{nxt_ori_l} represents the long frame initial Gain; gain _{nxt_ori_s} represents the short frame initial Gain; EV _{nxt_ori_l} represents the initial long frame exposure; the shutter _{nxt_ori_l} represents a long frame initial shutter; EV _{nxt_ori_s} represents the initial short frame exposure; gain _{nxt_ori_s} represents the short frame initial Gain; the shutter _{nxt_ori_s} short frame initial shutter.

In some embodiments, for (a) above, a smaller gain may also be introduced, and when the introduced gain is smaller, less impact on the signal-to-noise ratio is acceptable; similarly, for (b) above, instead of using the maximum shutter sum value as the shutter sum value of the second long frame image and the second short frame image, a value slightly smaller than the maximum shutter sum value may be used as the sum value of the second long frame image and the second short frame image, so that the influence on the signal-to-noise ratio is small, which is acceptable.

In the embodiment of the application, the image acquisition device combines the influence of the shutter and the gain on the signal to noise ratio of the image in the process of determining the long frame initial shutter and the long frame initial gain of the second long frame image and the short frame initial shutter and the short frame initial gain of the second short frame image, so that the signal to noise ratio of the image shot by the long frame target shutter, the long frame target gain, the short frame target shutter and the short frame target gain which are determined according to the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain is better.

In the above step S15, it is understood that the initial long frame exposure, the initial short frame exposure, the long frame initial shutter, the short frame initial shutter, the long frame initial gain, and the short frame initial gain are all determined when the gains of the second long frame image and the second short frame image are the same, that is, the gain adjustment is not used as a means for realizing the exposure ratio adjustment.

After the shutter and the gain are distributed in the above manner, when the exposure ratio is larger, the shutter of the long frame image is larger, the shutter of the short frame image is smaller, that is, the dynamic range of the scene is larger, and other areas except for the dark area in the long frame image, especially the brighter area, can have the condition of overexposure, so that the short frame image is more dependent when the image frames are synthesized, that is, the occupation ratio of the short frame image in the synthesized image frames is higher, but because the shutter of the short frame is smaller, the signal-to-noise ratio of the short frame is poorer, and therefore, the occupation ratio of the short frame in the synthesized image is higher, so that the signal-to-noise ratio of the synthesized image is poorer; similarly, when the exposure is smaller, the difference between the shutter of the long frame image and the shutter of the short frame image is not large, that is, the dynamic range of the scene is smaller, the effect of the long frame image and the effect of the short frame image are close, but compared with the effect of the short frame image, the shutter of the long frame image is larger, so that the effect is relatively better, the long frame image is more depended on when the image frames are synthesized, that is, the occupation of the long frame image in the synthesized image is relatively higher, but because the shutters of the short frame image and the long frame image are close at the moment, that is, the long frame image is not distributed to enough shutters, and a part of the shutters are wasted by the short frame image, so that the signal to noise ratio of the long frame image is not optimal, and the signal to noise ratio of the synthesized image frame is not optimal.

In summary, after the shutter and the gain are allocated in the above manner, when the second short frame image occupies a relatively high area in the target image frame, the shutter is mostly allocated to the second long frame image, which results in poor signal-to-noise ratio of the short frame image, that is, poor signal-to-noise ratio of the target image frame; when the second long frame image occupies a relatively high proportion in the target image frame, the second long frame image cannot be allocated to a sufficient number of shutters, resulting in an insufficient signal-to-noise ratio of the target image frame.

Therefore, the image acquisition device needs to adjust the obtained long frame initial shutter, short frame initial shutter, long frame initial gain and short frame initial gain according to the signal-to-noise ratio increasing direction to obtain a long frame target shutter, a long frame target gain, a short frame target shutter and a short frame target gain, and obtain a second long frame image and a second short frame image through shooting of the long frame target shutter, the long frame target gain, the short frame target shutter and the short frame target gain to synthesize the target image frame, so that the signal-to-noise ratio of the target image frame is better. The adjustment manner is not limited in any way, and the step S15 will be exemplarily described in three different examples hereinafter, and is not repeated here, as shown in fig. 2, 3 and 4.

In the step S16, the image capturing device may capture a second long frame image according to the long frame target shutter and the long frame target gain; the image acquisition device can also shoot according to the short frame target shutter and the short frame target gain to obtain a second short frame image.

In some embodiments, after determining the long frame target shutter, the long frame target gain, the short frame target shutter, and the short frame target gain, the image capturing apparatus may determine whether the obtained long frame target shutter, long frame target gain, short frame target shutter, and short frame target gain are the same as the shutter and gain of the first long frame image, and the shutter and gain of the first short frame image, if so, it indicates that the exposure level has reached the limit when the first long frame image and the first short frame image are captured, for example, when the capturing scene is too dark, the shutter and gain have reached the maximum limit, at which time there is no means to make the image brighter, so the image capturing apparatus may not configure the long frame target shutter, the long frame target gain, the short frame target shutter, and the short frame target gain to the image capturing apparatus, and still capture the second long frame image and the second short frame image according to the shutter and gain of the first long frame image, and the shutter and the gain of the first short frame image; if the frame target shutter, the long frame target gain, the short frame target shutter and the short frame target gain are different, the frame target shutter, the long frame target gain, the short frame target shutter and the short frame target gain are required to be configured to the image acquisition equipment, so that the acquisition equipment shoots according to the long frame target shutter and the long frame target gain to obtain a second long frame image, and shoots according to the short frame target shutter and the short frame target gain to obtain the second short frame image.

In the above step S17, the image pickup apparatus may synthesize the second long frame image and the second short frame image into the target image frame according to the wide dynamics.

Next, three examples of the above step S15 are given by fig. 2, 3, and 4, and the contents of the above step S15 have been described.

Referring to fig. 2, fig. 2 is a first detailed schematic diagram of step S15 provided in the embodiment of the present application, which may include the following steps:

step S21: determining a shutter corresponding to the initial gain of the first priority frame and the initial exposure ratio of the target image frame as a maximum shutter of the first priority frame according to a preset corresponding relation between gain-exposure ratio-shutter;

Step S22: determining, as a target shutter and a target gain of the first priority frame, a shutter and a gain of the first priority frame when the shutter is not greater than the maximum shutter such that a difference between a target exposure of the first priority frame and an initial exposure of the first priority frame is less than a first preset exposure difference;

Step S23: determining a maximum shutter of the second priority frame according to the target shutter of the first priority frame;

step S24: determining a value range of the target exposure of the second priority frame according to the target exposure of the first priority frame and the exposure ratio value range of the target image frame;

Step S25: and determining the shutter and the gain of the second priority frame when the shutter of the second priority frame is not greater than the maximum shutter of the second priority frame, wherein the difference value between the target exposure of the second priority frame and the initial exposure of the second priority frame is smaller than a second preset exposure threshold value, the target exposure of the second priority frame is within the value range of the target exposure of the second priority frame, and the shutter and the gain of the second priority frame are used as the target shutter and the target gain of the second priority frame.

In the technical scheme provided by the embodiment of the application, the image acquisition device determines the maximum shutter of the first priority frame according to the initial gain of the first priority frame, the initial exposure ratio of the target image frame and the preset corresponding relation between the gain and the exposure ratio and the shutter, the corresponding relation between the gain and the exposure ratio and the shutter is set based on the principle of improving the signal to noise ratio of the image, and the signal to noise ratio of the first priority frame shot according to the obtained target shutter and the target gain of the first priority frame and the target shutter and the target gain of the second priority frame can be higher based on the limitation of the maximum shutter and other constraint conditions of the first priority frame, and the exposure is close to the initial exposure. In the embodiment of the application, the image acquisition equipment can determine the maximum shutter of the first priority frame according to the corresponding relation between the preset gain-exposure ratio-shutter, and then determine the target shutter and the target gain of the first priority frame and the target shutter and the target gain of the second priority frame based on the maximum shutter and other constraint conditions, so that the signal-to-noise ratio of the current image frame is not required to be calculated, the signal-to-noise ratio of the middle image frame is not required to be counted when the middle image frame is shot, and the calculation complexity is reduced.

In the above step S21, the first priority frame refers to a frame that preferentially allocates an exposure parameter, and the first priority frame is one of the second long frame image and the second short frame image, that is, the first priority frame may be a long frame image or a short frame image, and specifically, the long frame image or the short frame image depends on whether the exposure parameter is preferentially allocated to the long frame image or the short frame image in the image capturing device, if the exposure parameter is preferentially allocated to the long frame image in the image capturing device, the first priority frame is the long frame image, and if the exposure parameter is preferentially allocated to the short frame image in the image capturing device, the first priority frame is the short frame image. When calculating the exposure parameters (i.e. the shutter and gain of the long frame image and the short frame image), the shutter requirement of the first priority frame needs to be preferentially met, for example, when the second short frame image is the first priority frame, the target shutter of the second short frame image needs to be determined first during exposure, and the shutter of the second long frame image can only be allocated in the rest shutters after the shutter allocation of the second short frame image is finished.

After the corresponding relation between the gain-exposure ratio-shutter is preset, the image acquisition device determines the initial exposure ratio, the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain of the target image frame, if the first priority frame is the second long frame image, the image acquisition device can determine the maximum shutter of the first priority frame, namely the maximum shutter of the second long frame image according to the corresponding relation between the long frame initial gain, the initial exposure ratio of the target image frame and the gain-exposure ratio-shutter; if the first priority frame is the second short frame image, the image acquisition device may determine a maximum shutter of the first priority frame, that is, a maximum shutter of the second short frame image, according to the initial gain of the short frame, the initial exposure ratio of the target image frame, and the correspondence between gain-exposure ratio-shutter. Wherein, the correspondence relationship between gain-exposure ratio-shutter may be established in advance according to the following rule:

(a) When the exposure ratio is smaller than the exposure ratio threshold, the larger the short frame initial gain is, the smaller the maximum shutter of the second short frame image is; the larger the initial gain of the long frame is, the larger the maximum shutter of the second long frame image is;

(b) When the exposure ratio is larger than the exposure ratio threshold, the larger the short frame initial gain is, the larger the maximum shutter of the second short frame image is; the larger the initial gain of the long frame is, the smaller the maximum shutter of the second long frame image is;

(c) When the exposure ratio is equal to the exposure ratio threshold, the maximum shutter of the second short frame image is not changed along with the gain change, and the values of the maximum shutter are 1/(ExpRatio +1); the maximum shutter of the second long frame image does not change along with the gain change, and the values of the maximum shutter are ExpRatio/(ExpRatio +1);

(d) When the short frame initial gain is 0dB, the maximum shutter of the second short frame image is 1/(ExpRatio +1); when the long frame initial gain is 0dB, the maximum shutter of the second long frame image is ExpRatio/(ExpRatio +1).

The exposure ratio threshold can be set according to actual conditions; expRatio is the exposure ratio of the wide dynamic composite image frame, here ExpRatio can be understood as ExpRatio _{nxt_ori_t} calculated in equation (12) above, i.e., the initial exposure ratio of the target image frame.

Here, the correspondence relationship between gain-exposure ratio-shutter may be linear or nonlinear.

The above rule setting principle for establishing the correspondence relationship between gain-exposure ratio-shutter is as follows:

(a) When the exposure is smaller (i.e., the exposure ratio is smaller than the exposure ratio threshold), it indicates that the long frame image occupies a higher position in the composite image frame (i.e., the long frame image and the short frame image are based on the image frame of the wide dynamic composite), the short frame image occupies a lower position in the composite image frame, and when the gain becomes larger, more shutters should be allocated to the frames occupying a higher position in the composite image, and less shutters should be allocated to the frames occupying a lower position in the composite image, i.e., more shutters are allocated to the long frame image, and fewer shutters should be allocated to the short frame image;

(b) When the exposure is larger (namely the exposure ratio is larger than the exposure ratio threshold value), the short frame image is higher in the composite image, when the gain is larger, the signal to noise ratio loss of a plurality of shutters to the long frame image is smaller because the proportion of the shutter of the long frame to the total shutter (namely the maximum shutter and the value) is larger, but after the shutter is added to the short frame, the signal to noise ratio of the short frame image is greatly improved, and finally the overall signal to noise ratio of the composite image is better, so that more shutters are required to be allocated to the short frame image at the moment;

(c) To ensure a smooth transition of the intermediate exposure ratio, when the exposure ratio is an exposure ratio threshold, only the shutter is used as a parameter for calculating the exposure ratio, and shutter allocation of the long frame image and the short frame image is determined according to the exposure ratio.

(D) The signal to noise ratio of the image frame is improved, the gain is required to be reduced, the shutter is required to be increased, when the gain is 0dB, the exposure can be achieved only by the shutter, the gain is not required to be started, at this time, the gain is minimum, and the signal to noise ratio can not be improved by adjusting the gain, so that the shutter is only used as a parameter for calculating the exposure ratio, and the shutter is distributed for the long frame image and the short frame image according to the exposure ratio.

In some embodiments, in the above-described correspondence between gain-exposure-shutter, the shutter may be understood as the first priority frame shutter duty cycle, i.e., the duty cycle of the maximum shutter of the first priority frame in the total shutter. After the image acquisition device determines the initial Gain of the first priority frame and the initial exposure ratio of the target image frame, the maximum shutter of the first priority frame may be determined according to the preset corresponding relationship between Gain-exposure ratio-shutter, where when the first priority frame is the second long frame image, the initial Gain of the first priority frame is the long frame initial Gain, that is, gain _{nxt_ori_l} calculated in step S14 above; when the first priority frame is the second short frame image, the initial Gain of the first priority frame is the short frame initial Gain, that is, gain _{nxt_ori_s} calculated in step S14.

For example, if f (ExpRatio, gain) is used to represent the correspondence between Gain-exposure ratio-shutter, then f (ExpRatio, gain) can be understood as the exposure ratio ExpRatio, the shutter duty ratio of the first priority frame when Gain is Gain, the image capturing apparatus can calculate the maximum shutter of the first priority frame by the following formula (21),

ShutterMAX_{adj_1st}= f(ExpRatio，Gain)× ShutterMAX （21）

Wherein ShutterMAX _{adj_1st} denotes a maximum shutter of the first priority frame, f (ExpRatio, gain) denotes an exposure ratio of ExpRatio, and ShutterMAX denotes a maximum shutter sum value.

For another example, by substituting the initial exposure ratio ExpRationxt _ori_t of the target image frame calculated by the above formula (12), the gain of the target image frame calculated by the above formula (15) or the formula (18), that is, the long frame initial gain or the short frame initial gain, into the above formula (21), the maximum shutter ShutterMAX _{adj_1st} of the first priority frame can be obtained, as shown in the following formula (22),

ShutterMAX_{adj_1st}= f(ExpRatio_{nxt_ori_t},Gain_{nxt_ori_t})× ShutterMAX （22）

In some embodiments, when the first priority frame is a short frame image, the correspondence between the gain-exposure ratio-shutter set in advance is as shown in table 1 below,

TABLE 1 correspondence between preset gain-exposure ratio-shutter

；

In table 1, the first row represents the initial gain of the first priority frame in dB; as can be seen from the contents of the first row in table 1, when the exposure ratio of the target image frame is 1, if the initial gain of the first priority frame is 0dB, the shutter ratio of the first priority frame is 1/2; if the initial gain of the first priority frame is 6dB, the shutter ratio of the first priority frame is 1/2; if the initial gain of the first priority frame is 12dB, the shutter ratio of the first priority frame is 1/3; if the initial gain of the first priority frame is 18dB, the shutter ratio of the first priority frame is 1/4; if the initial gain of the first priority frame is 24dB, the shutter ratio of the first priority frame is 1/4; if the initial gain of the first priority frame is 30dB, the shutter ratio of the first priority frame is 1/5; if the initial gain of the first priority frame is 36dB, the shutter ratio of the first priority frame is 1/5; the meaning of the other rows in Table 1 is the same.

In the above step S22, the target exposure of the first priority frame may be understood as the exposure of the first priority frame captured in the ideal state, the target exposure of the first priority frame being determined by the target shutter and the target gain of the first priority frame. Since only the limitation of a part of the exposure parameters is considered in determining the parameters of the long frame initial exposure amount, the short frame initial exposure amount, and the first priority frame maximum shutter in the foregoing steps, in practice, both the exposure parameters and the exposure ratio are limited, and the limitation is externally configured. Accordingly, the target shutter and target gain of the first priority frame may be determined according to the exposure parameters and the limits of the exposure ratio by:

(a) Setting a minimum shutter of the first priority frame, and then the value range of the target shutter of the first priority frame is shown in the following formula (23);

ShutterMIN_1st≤Shutter_{adj_1st}≤ShutterMAX_{adj_1st} （23）

Wherein ShutterMIN _1st denotes the minimum shutter of the set first priority frame; shutter _{adj_1st} denotes a target shutter of the first priority frame; shutterMAX _{adj_1st} denotes the maximum shutter of the first priority frame, which can be calculated by the above formula (22).

(B) Setting the minimum gain and the maximum gain of the first priority frame, the range of the target gain of the first priority frame is as shown in the following formula (24),

GainMIN_1st≤Gain_{adj_1st}≤GainMAX_1st （24）

Wherein GainMIN _1st denotes a minimum gain of the set first priority frame; gain _{adj_1st} represents the target Gain for the first priority frame; gainMAX _1st denotes the maximum gain of the set first priority frame.

(C) The target shutter and target gain of the first priority frame also need to satisfy the following conditions under the conditions set in (a) and (b) above:

(d) The target shutter of the first priority frame is as large as possible, so that the shutter is exposed as much as possible;

(e) The gain of the first priority frame is as small as possible, so that the gain is as small as possible;

(f) The difference between the target exposure amount of the first priority frame and the initial exposure amount of the first priority frame is as small as possible so that the error of the actual exposure amount and the expected exposure amount is minimized. That is, in an ideal state, |ev _{nxt_ori_1st}-EV_{adj_1st}|=|EV_{nxt_ori_1st}-Shutter_{adj_1st}×Gain_{adj_1st} | is made as small as possible, where EV _{nxt_ori_1st} represents the initial exposure of the first priority frame, which can be calculated from the initial shutter and the initial gain of the first priority frame; EV _{adj_1st} represents the target exposure of the first priority frame, which can be calculated from the target shutter Shutter _{adj_1st} and the target Gain _{adj_1st} of the first priority frame. In some embodiments, the difference between the target exposure of the first priority frame and the initial exposure of the first priority frame is as small as possible when the difference is smaller than the first preset exposure difference.

In the step S23, the second priority frame is the other of the second long frame image and the second short frame image, that is, if the image acquisition device preferentially allocates the exposure parameter to the second long frame image, that is, the first priority frame is the second long frame image, the second priority frame is the second short frame image; if the exposure parameters are preferentially allocated to the second short frame image in the image acquisition device, that is, the first preferential frame is the second short frame image, the second preferential frame is the second long frame image. The maximum shutter of the second priority frame may be determined by the target shutter and the maximum shutter sum value of the first priority frame, i.e., the maximum shutter of the second priority frame may be the difference of the maximum shutter sum value and the target shutter of the first priority frame.

In the above step S24, the target exposure amount of the first priority frame is determined by the target shutter and the target gain of the first priority frame. The exposure ratio value range of the target image frame is preset, for example, the minimum exposure ratio of the target image frame is ExpRatioMIN, the maximum exposure ratio of the target image frame is ExpRatioMAX, the exposure ratio ExpRatio _adj of the target image frame is set as shown in the following formula (25),

ExpRatioMIN≤ExpRatio_adj≤ExpRatioMAX （25）

Since the exposure ratio of the target image frame is determined by the exposure amount of the first priority frame and the exposure amount of the second priority frame, the value range of the target exposure amount of the second priority frame can be determined based on the target exposure amount of the first priority frame, which is, for example, a short frame image, and the preset exposure ratio value range of the second priority frame, which is shown in the above-described formula (25), as shown in the following formula (26),

ExpRatioMIN×EV_{adj_1st}≤EV_{adj_2nd}≤ExpRatioMAX×EV_{adj_1st} （26）

Where EV _{adj_2nd} represents the target exposure of the second priority frame, EV _{adj_1st} represents the target exposure of the first priority frame. The target exposure amount of the first priority frame may be calculated by the following formula (27),

EV_{adj_1st}=Shutter_{adj_1st}×Gain_{adj_1st} （27）

Wherein Shutter _{adj_1st} denotes a target shutter of the first priority frame, and Gain _{adj_1st} denotes a target Gain of the first priority frame; the target shutter and target gain of the first priority frame may be determined by step S22 described above.

In step S25, the maximum shutter of the second priority frame is associated with the target shutter of the first priority frame, and in general, the maximum shutter of the second priority frame is determined by the difference between the maximum shutter sum value and the target shutter of the first priority frame. The image acquisition device may determine the target shutter and the target gain of the second priority frame by the following constraint conditions, where the following constraint conditions are the constraint conditions when the second priority frame is the second long frame image, and when the second priority frame is the second short frame image, the principle is the same as that of the second priority frame, and will not be described here again:

(a) ShutterMIN _2nd≤Shutter_{adj_2nd}≤ShutterMAX_{adj_2nd}, wherein ShutterMIN _2nd represents a preset minimum shutter of the second priority frame, shutterMAX _{adj_2nd} represents a maximum shutter of the second priority frame, and Shutter _{adj_2nd} represents a target shutter of the second priority frame; wherein the maximum shutter ShutterMAX _{adj_2nd} of the second priority frame can be calculated by the following formula (28),

Shutter_{adj_2nd}=ShutterMAX－ShutterMAX_{adj_1st} （28）

Where ShutterMAX denotes the maximum shutter sum value and ShutterMAX _{adj_1st} denotes the maximum shutter of the first priority frame.

(B) GainMIN _2nd≤Gain_{adj_2nd}≤GainMAX_2nd, wherein GainMIN _2nd represents a preset minimum Gain for the second priority frame, gainMAX _2nd represents a preset maximum Gain for the second priority frame, and Gain _{adj_2nd} represents a target Gain for the second priority frame;

(c) ExpRatioMIN xEV _{adj_1st}≤EV_{adj_2nd}≤ExpRatioMAX×EV_{adj_1st}; i.e. equation (26) above.

(D) The target shutter Shutter _{adj_2nd} of the second priority frame is made as large as possible, and the shutter is made as full as possible; making the target gain of the second priority frame as small as possible;

(e) The difference between the target exposure amount and the initial exposure amount of the second priority frame is made as small as possible, that is, in an ideal state, |EV _{nxt_ori_2nd}-EV_{adj_2nd}|=|EV_{nxt_ori_2nd}-Shutter_{adj_2nd}×Gain_{adj_2nd} | is made as small as possible, wherein EV _{nxt_ori_2nd} represents the initial exposure amount of the second priority frame and can be obtained through calculation of an initial shutter and an initial gain of the second priority frame; EV _{adj_2nd} represents the target exposure for the second priority frame, which can be calculated from the target shutter Shutter _{adj_2nd} and the target Gain _{adj_2nd} for the second priority frame.

In some embodiments, the difference between the target exposure of the second priority frame and the initial exposure of the second priority frame is as small as possible when the difference is smaller than the second preset exposure difference.

The second preset exposure amount difference may be the same as the first preset exposure amount difference, or may be different from the first preset exposure amount difference.

In some embodiments, referring to fig. 3, fig. 3 is a second detailed schematic diagram of step S15 provided in the embodiment of the present application, which may include the following steps:

step S31: determining an initial exposure ratio of the target image frame according to the initial long frame exposure and the short frame exposure;

step S32: determining a maximum shutter of the first priority frame according to the initial exposure ratio of the target image frame;

Step S33: determining, as an intermediate shutter and an intermediate gain of the first priority frame, a shutter and a gain of the first priority frame when the shutter is not greater than the maximum shutter such that a difference between an intermediate exposure of the first priority frame and an initial exposure of the first priority frame is less than a first preset exposure difference;

Step S34: determining a maximum shutter of the second priority frame according to the intermediate shutter of the first priority frame;

Step S35: determining a value range of the target exposure of the second priority frame according to the target exposure of the first priority frame and a preset value range of the exposure ratio of the target image frame;

Step S36: determining, according to the maximum shutter of the second priority frame, a shutter and a gain of the second priority frame when the target exposure of the second priority frame is less than the second preset exposure difference and the target exposure of the second priority frame is within a value range of the target exposure of the second priority frame, and the shutter of the second priority frame is not greater than the maximum shutter of the second priority frame, as an intermediate shutter and an intermediate gain of the second priority frame;

step S37: shooting according to the middle shutter of the first priority frame and the middle gain of the first priority frame to obtain the first priority frame, and shooting according to the middle shutter of the second priority frame and the middle gain of the second priority frame to obtain the second priority frame;

step S38: according to the first priority frame and the second priority frame, obtaining an intermediate image frame by wide dynamic synthesis;

step S39: counting the average signal-to-noise ratio of the middle image frame;

Step S310: and adjusting the middle shutter of the first priority frame, the middle gain of the first priority frame, the middle shutter of the second priority frame and the middle gain of the second priority frame according to the average signal-to-noise ratio of the middle image frame to obtain the target shutter of the first priority frame, the target gain of the first priority frame, the target shutter of the second priority frame and the target gain of the second priority frame.

According to the technical scheme provided by the embodiment of the application, the image acquisition equipment determines the middle shutter and the middle gain of the second long frame image and the middle shutter and the middle gain of the second short frame image according to the initial exposure ratio of the target image frame under the condition that the gains of the second long frame image and the second short frame image are the same, shoots the first priority frame and the second priority frame according to the middle shutter and the middle gain of the second long frame image and the second short frame image, synthesizes the middle image frame based on the second priority frame and the first priority frame, adjusts the middle shutter and the middle gain of the first priority frame and the middle shutter and the middle gain of the second priority frame by taking the average signal-to-noise ratio of the middle image frame as a reference, and further determines the target shutter and the target gain of the first priority frame and the target shutter and the target gain of the second priority frame, so that the signal-to-noise ratio direction of the adjusted target image frame is more definite, and the signal-to-noise ratio of the target image frame is more favorable for improving.

The steps S34 to S35 are the same as the steps S23 to S24, and will not be described in detail.

In the above step S31, the exposure ratio is the ratio of the long frame exposure amount to the short frame exposure amount, and therefore, here, the initial exposure amount of the target image frame can be understood as the ratio of the long frame initial exposure amount of the second long frame image to the short frame initial exposure amount of the second short frame image. Such as the initial exposure ratio of the target image frame calculated according to the above equation (12).

In the above step S32, since the long frame initial gain of the second long frame image and the short frame initial gain of the second short frame image are the same, the initial exposure ratio of the target image frame may be regarded as the ratio of the long frame initial shutter of the second long frame image to the short frame initial shutter of the second short frame image, and thus the maximum shutter of the first priority frame may be determined from the exposure ratio of the target image frame and the maximum shutter sum value.

For example, when the first priority frame is the second long frame image, the image pickup device may calculate the maximum shutter of the first priority frame by the following formula (29),

ShutterMAX_{adj_1st}=ExpRatio/(ExpRatio+1)× ShutterMAX （29）

Wherein ShutterMAX _{adj_1st} denotes the maximum shutter of the first priority frame; expRatio denotes an exposure ratio of the target image frame; shutterMAX denotes the maximum shutter sum value of the second long frame image and the second short frame image.

In some embodiments, the exposure ratio used to determine the maximum shutter of the first priority frame in the present application may also be an exposure ratio determined by the initial exposure ratio of the target image frame and the preset exposure ratio value range of the target image frame, that is, if the initial exposure ratio of the target image frame is within the preset exposure ratio value range of the target image frame, the initial exposure ratio of the target image frame may be used to determine the maximum shutter of the first priority frame, that is, at this time, expRatio in the above formula (29) is the initial exposure ratio of the target image frame, which may be calculated by the above formula (12); if the initial exposure ratio of the target image frame is outside the preset exposure ratio value range of the target image frame, a value close to the initial exposure ratio of the target image frame and within the preset exposure ratio value range of the target image frame is required to be taken as ExpRatio in the above formula (29).

The first priority frame is one of the second long frame image and the second short frame image, and when the first priority frame is the second short frame image, the image pickup apparatus may calculate a maximum shutter of the first priority frame by the following formula (30),

ShutterMAX_{adj_1st}=1/（ExpRatio+1）×ShutterMAX （30）

Wherein ShutterMAX _{adj_1st} denotes the maximum shutter of the first priority frame; expRatio denotes an exposure ratio of the target image frame; shutterMAX denotes the maximum shutter sum value of the second long frame image and the second short frame image. Here, expRatio has the same value as ExpRatio in the above formula (29).

In the step S33, the manner of determining the intermediate shutter and the intermediate gain of the first priority frame according to the maximum shutter of the first priority frame is the same as the manner of determining the target shutter and the target gain of the first priority frame according to the maximum shutter of the first priority frame in the step S22, and the intermediate shutter and the intermediate gain of the first priority frame in the embodiment of the present application may be regarded as the target shutter and the target gain of the first priority frame in the step S22 during the calculation, and the maximum shutter of the first priority frame in the embodiment of the present application is the maximum shutter of the first priority frame determined by the step S32. The specific calculation process may be referred to the description in step S22, and will not be repeated here.

In the above step S34: the principle of determining the maximum shutter of the second priority frame according to the intermediate shutter of the first priority frame is the same as that of determining the maximum shutter of the second priority frame according to the target shutter of the first priority frame in the above step S23, and the intermediate shutter of the first priority frame in the embodiment of the present application may be regarded as the target shutter of the first priority frame in the above step S22 in the calculation process, and the specific calculation process may be referred to the description in the above step S23 and will not be repeated here.

In the step S35, the manner of determining the range of the target exposure of the second priority frame according to the intermediate exposure of the first priority frame and the exposure ratio range of the target image frame is the same as the manner of determining the range of the target exposure of the second priority frame according to the target exposure of the first priority frame and the exposure ratio range of the target image frame in the step S24, and the calculation process may be regarded as the target exposure of the first priority frame in the step S24 according to the intermediate exposure of the first priority frame in the embodiment of the present application, and the specific calculation process may be referred to the description in the step S24 and is not repeated herein.

In the above step S36, the principle of determining the intermediate shutter and the intermediate gain of the second priority frame according to the maximum shutter of the second priority frame is the same as that of determining the target shutter and the target gain of the second priority frame according to the maximum shutter of the second priority frame in the above step S25, and in the calculating process, the intermediate shutter and the intermediate gain of the second priority frame in the embodiment of the present application may be regarded as the target shutter and the target gain of the second priority frame in the above step S25, and the specific calculating process may be referred to the description in the above step S25 and will not be repeated here.

In the above step S37, the image capturing apparatus may configure the intermediate shutter and the intermediate gain of the first priority frame and the intermediate shutter or the intermediate gain of the second priority frame to the image capturing apparatus, so that the image capturing apparatus captures the first priority frame according to the intermediate shutter and the intermediate gain of the first priority frame and captures the second priority frame according to the intermediate shutter and the intermediate gain of the second priority frame.

In some embodiments, before configuring the intermediate shutter and the intermediate gain of the first priority frame, and the intermediate shutter and the intermediate gain of the second priority frame to the image capturing apparatus to cause the image capturing apparatus to capture the intermediate image frame, the image capturing apparatus may determine whether the resulting intermediate shutter and the intermediate gain of the first priority frame, and the intermediate shutter and the intermediate gain of the second priority frame are the same as those of the first long frame image, and the shutter and the gain of the first short frame image, for example, when the first priority frame is a long frame image, determine whether the intermediate shutter and the intermediate gain of the first priority frame are the same as those of the first long frame image, and whether the intermediate shutter and the intermediate gain of the second priority frame are the same as those of the first short frame image. If the first frame image and the second frame image are the same, the exposure degree reaches the limit when the first long frame image and the first short frame image are shot, so that the middle shutter and the middle gain of the first priority frame and the middle shutter and the middle gain of the second priority frame are not configured to the image acquisition equipment, and the image acquisition equipment still shoots the next image frame according to the shutter and the gain of the first long frame image and the shutter and the gain of the first short frame image; if the first and second priority frames are different, the middle shutter and the middle gain of the first priority frame and the middle shutter and the middle gain of the second priority frame are required to be configured to the image acquisition device, so that the acquisition device shoots the first priority frame according to the middle shutter and the middle gain of the first priority frame and shoots the second priority frame according to the middle shutter and the middle gain of the second priority frame.

In some embodiments, the image capturing apparatus may further determine whether the exposure amounts of the first priority frame and the second priority frame captured by the intermediate shutter and the intermediate gain of the first priority frame and the intermediate shutter and the intermediate gain of the second priority frame are the same as the exposure amount of the first long frame image and the exposure amount of the first short frame image, for example, when the first priority frame is a long frame image, whether the exposure amount of the first priority frame captured by the intermediate shutter and the intermediate gain of the first priority frame is the same as the exposure amount of the first long frame image, and whether the exposure amount of the second priority frame captured by the intermediate shutter and the intermediate gain of the second priority frame is the same as the exposure amount of the first short frame image. If the two types of the images are the same, the exposure of the images is not changed after the adjustment, so that the influence of the improvement of the signal to noise ratio on the image quality is not great, and therefore, the middle shutter and the middle gain of the first priority frame and the middle shutter and the middle gain of the second priority frame can not be configured to the image acquisition equipment, the steps S37-S310 are not executed, and the image acquisition equipment still shoots the next image frame according to the shutter and the gain of the first long frame image and the shutter and the gain of the first short frame image; if the images are different, the exposure of the images is changed, the bright area and the dark area in the shot images can be more clear, and on the basis, the image acquisition equipment obtains the first priority frame and the second priority frame through shooting by configuring the middle shutter and the middle gain of the first priority frame and the middle shutter and the middle gain of the second priority frame to the image acquisition equipment, so that the target shutter and the target gain of the first priority frame and the target shutter and the target gain of the second priority frame are conveniently determined according to the first priority frame and the second priority frame shot at present, and the signal to noise ratio of the images is further improved.

In the above step S38, the intermediate image frame may be understood as the first frame image captured while the intermediate shutter and the intermediate gain of the first priority frame configured in the image capturing apparatus and the intermediate shutter and the intermediate gain of the second priority frame are in effect. That is, the image acquisition device shoots according to the intermediate shutter and the intermediate gain of the first priority frame to obtain the first priority frame; and shooting according to the intermediate shutter and the intermediate gain of the second priority frame to obtain the second priority frame, and combining the first priority frame and the second priority frame based on wide dynamic state at the moment to obtain the intermediate image frame.

In the step S39, the image acquisition device may determine the signal-to-noise ratio of each pixel point by counting the mean value and the variance of each pixel point in the intermediate image frame, and further determine the average signal-to-noise ratio of the third image frame according to the signal-to-noise ratio of each pixel point and the number of the pixel points in the intermediate image frame. The method for determining the average signal-to-noise ratio of the image frame is not limited in any way, and the step S39 will be described in an exemplary manner hereinafter, which will not be repeated here.

In the above-described step S310, the image capturing apparatus may calculate the exposure amount of the first priority frame and the exposure amount of the second priority frame for synthesizing the intermediate image frame by the following formula (31) and formula (32),

EV_{last_l}=Shutter_{last_l}×Gain_last （31）

EV_{last_s}=Shutter_{last_s}×Gain_last （32）

In the above-described formulas (31) and (32), EV _{last_l} represents the exposure amount of the first priority frame that synthesizes the intermediate image frame, and the intermediate exposure amount of the first priority frame, shutter _{last_l} represents the intermediate shutter of the first priority frame, gain _last represents the intermediate Gain of the first priority frame; EV _{last_s} denotes an exposure amount of the second priority frame that synthesizes the intermediate image frame, that is, an intermediate exposure amount of the second priority frame, shutter _{last_s} denotes an intermediate shutter of the second priority frame, and Gain _last denotes an intermediate Gain of the second priority frame.

The target shutter and target gain of the first priority frame, and the target shutter and target gain of the second priority frame should satisfy the following constraints:

(a) Keeping the exposure unchanged; that is to say ,EV_l=shutter_l×Gain_l=EV_{last_l},EV_s=shutter_s×Gain_s=EV_{last_s}, where EV _l represents the target exposure of the first priority frame, determined by the product of the filter _l and Gain _l, filter _l represents the target shutter of the first priority frame, gain _l represents the target Gain of the first priority frame, and EV _{last_l} represents the intermediate exposure of the first priority frame, which can be determined by the above formula (31), that is, where the target exposure of the first priority frame needs to be kept the same as the intermediate exposure of the first priority frame; EV _s represents the target exposure of the second priority frame, determined by the product of the blend _s and Gain _s, blend _S represents the target shutter of the second priority frame, gain _s represents the target Gain of the second priority frame, and EV _{last_s} represents the intermediate exposure of the second priority frame, as determined by equation (32) above, i.e., where the target exposure of the second priority frame needs to be maintained the same as the intermediate exposure of the second priority frame.

(B) The shutter of the long frame image needs to be in a preset long frame shutter range, and the gain of the long frame image needs to be in a preset long frame gain range; that is, ,ShutterMIN_l≤Shutter_l≤ShutterMAX_l,GainMIN_l≤Gain_l≤GainMAX_l, where ShutterMIN _l denotes a preset long frame minimum shutter, shutterMAX _l denotes a preset long frame maximum shutter, and Shutter _l denotes a long frame target shutter; gainMIN _l denotes a preset long frame minimum Gain, gainMAX _l denotes a preset long frame maximum Gain, gain _l denotes a long frame target Gain;

(c) The shutter of the short frame image needs to be in a preset short frame shutter range, and the gain of the short frame image needs to be in a preset short frame gain range; that is, ,ShutterMIN_s≤Shutter_s≤ShutterMAX_s,GainMIN_s≤Gain_s≤GainMAX_s, where ShutterMIN _s denotes a preset short frame minimum shutter, shutterMAX _s denotes a preset short frame maximum shutter, and Shutter _s denotes a short frame target shutter; gainMIN _s denotes a preset short frame minimum Gain, gainMAX _s denotes a preset short frame maximum Gain, and Gain _s denotes a short frame target Gain;

(d) The shutter and the shutter value of the long frame image and the short frame image are not more than the maximum shutter and value, that is Shutter _l+ Shutter_s.ltoreq. ShutterMAX.

Under the condition that the constraint conditions are met, different combinations are traversed, so that the signal to noise ratio of the target image frames shot according to the target shutter and the target gain is as large as possible, and the exposure parameters of the long frames and the exposure parameters of the short frames obtained at the moment are approximate optimal exposure parameters.

In some embodiments, a threshold may be set, and when the difference between the signal-to-noise ratio of the target image frame and the signal-to-noise ratio of the intermediate image frame exceeds the threshold, the exposure parameters of the long frame image and the short frame image may be considered to be approximately optimal, thereby stopping the solution process. In some embodiments, if the signal-to-noise ratio of the traversed target image frame is not greater than the signal-to-noise ratio of the intermediate image frame, the exposure parameters of the long frame image and the short frame image may not be adjusted, that is, the intermediate shutter and the intermediate gain of the long frame image and the short frame image are directly used as the target shutter and the target gain, so as to capture the subsequent image.

It will be understood that, after the above steps S31-S38, the exposure parameters are within the limit and as close as possible to the exposure requirement, and the gain of the long frame image is consistent with the gain of the short frame image only by using the shutter as the exposure ratio means, and the ratio of the long frame image and the short frame shutter is the exposure ratio.

In some embodiments, referring to fig. 4, fig. 4 is a third detailed schematic diagram of step S15 provided in the embodiment of the present application, which may include the following steps:

step S41: determining an initial exposure ratio of the target image frame according to the initial long frame exposure and the initial short frame exposure;

Step S42: determining a maximum shutter of the first priority frame according to the initial exposure ratio of the target image frame;

Step S43: determining, as an intermediate shutter and an intermediate gain of the first priority frame, a shutter and a gain of the first priority frame when the shutter is not greater than the maximum shutter such that a difference between an intermediate exposure of the first priority frame and an initial exposure of the first priority frame is less than a first preset exposure difference;

step S44: determining a maximum shutter of the second priority frame according to the intermediate shutter of the first priority frame;

Step S45: determining a value range of the target exposure of the second priority frame according to the target exposure of the first priority frame and a preset value range of the exposure ratio of the target image frame;

Step S46: determining, as an intermediate shutter and an intermediate gain of the second priority frame, a shutter and a gain of the second priority frame when the shutter of the second priority frame is not greater than the maximum shutter of the second priority frame such that a difference between a target exposure amount of the second priority frame and an initial exposure amount of the second priority frame is less than a second preset exposure amount difference and the exposure amount of the second priority frame is within a target exposure amount range of the second priority frame;

step S47: counting the average signal-to-noise ratio of the current image frame;

Step S48: and adjusting the middle shutter of the first priority frame, the middle gain of the first priority frame, the middle shutter of the second priority frame and the middle gain of the second priority frame according to the average signal-to-noise ratio of the current image frame to obtain the target shutter of the first priority frame, the target gain of the first priority frame, the target shutter of the second priority frame and the target gain of the second priority frame.

In the technical scheme provided by the embodiment of the application, the image acquisition equipment counts the signal-to-noise ratio of the current image frame, takes the signal-to-noise ratio of the current image frame as a reference, adjusts the intermediate shutter and the intermediate gain of the first priority frame and the intermediate shutter and the intermediate gain of the second priority frame to obtain the target shutter and the target gain of the first priority frame and the target shutter and the target gain of the second priority frame, avoids the step of shooting the first priority frame and the second priority frame according to the intermediate shutter and the intermediate gain to obtain the intermediate image frame, and reduces the processing complexity while ensuring the signal-to-noise ratio.

The steps S41 to S46 are the same as the steps S31 to S36 in fig. 3, and are not repeated here.

In the step S47, the image capturing device may determine the signal to noise ratio of each pixel by counting the mean and variance of each pixel in the current image frame, and further determine the average signal to noise ratio of the first image frame according to the signal to noise ratio of each pixel and the number of pixels in the current image frame. The method for determining the average signal-to-noise ratio of the image frame is not limited in any way, and the step S39 will be described in an exemplary manner hereinafter, which will not be repeated here.

In the above step S48: in the same way as in step S310, the intermediate image frame may be regarded as the current frame in the process of determining the target shutter of the first priority frame, the target gain of the first priority frame, the target shutter of the second priority frame, and the target gain of the second priority frame.

Three exemplary descriptions have been made above for the step S15, and in the examples shown in fig. 3 and 4, the average signal-to-noise ratio of the intermediate image frame or the current image frame needs to be calculated, so how to calculate the signal-to-noise ratio of the image frame is described below by way of example shown in fig. 5. It will be appreciated that the example shown in fig. 5 below is merely one way in which the signal-to-noise ratio of an image may be calculated, and that the application may also be used to calculate the average signal-to-noise ratio of an image frame in other ways, without limitation.

Referring to fig. 5, fig. 5 is a flowchart illustrating a process for determining an average signal-to-noise ratio of an image frame according to an embodiment of the present application, which may include the following steps:

step S51: determining long frame pixel points, short frame pixel points and fusion pixel points in an image frame;

Step S52: counting the signal-to-noise ratio of a long frame pixel point in a long frame image, and taking the signal-to-noise ratio as a long frame signal-to-noise ratio;

step S53: counting the signal-to-noise ratio of short frame pixel points in the short frame image, and taking the signal-to-noise ratio as the short frame signal-to-noise ratio;

step S54: weighting the signal-to-noise ratio of the fused pixel points in the long frame image and the short frame image to be used as the fused signal-to-noise ratio;

step S55: and determining the average signal-to-noise ratio of the image frame according to the long frame signal-to-noise ratio, the short frame signal-to-noise ratio and the fusion signal-to-noise ratio.

In the technical scheme provided by the embodiment of the application, the image acquisition equipment can determine the signal-to-noise ratio of the image frame by respectively counting the signal-to-noise ratio of the long frame pixel point, the signal-to-noise ratio of the short frame pixel point and the signal-to-noise ratio of the fusion pixel point in the image frame, and a specific mode for determining the signal-to-noise ratio of the image frame is provided.

In the above step S51, the image frame may be understood as one frame image obtained by synthesizing the long frame image and the short frame image based on the wide dynamics. It can be understood that the pixels in the image frame of the wide dynamic synthesis include long frame pixels, short frame pixels and fusion pixels, wherein the long frame pixels are pixels in which the pixel value in the wide dynamic synthesis is completely taken from the long frame image, the short frame pixels are pixels in the wide dynamic synthesis which are completely taken from the short frame image, and the fusion pixels are pixels in which the pixel value in the wide dynamic synthesis is determined by the pixel value of the long frame image and the pixel value of the short frame image.

According to the wide dynamic synthesis principle, a long frame threshold and a short frame threshold can be determined in a long frame image, and are respectively marked as a first brightness threshold and a second brightness threshold, and a short frame threshold is also required to be determined in a short frame image for calculating the signal-to-noise ratio of an image frame, and is marked as a third brightness threshold. It can be understood that there is a correspondence between luminance values of the long frame image and the short frame image, and the correspondence can be understood as: in the histogram of the long frame image (i.e., pixel value-pixel number histogram) and the histogram of the short frame image, the pixel values of the pixel points at the same position correspond on the axis where the pixel values are located. Referring to fig. 6, fig. 6 (a) is a histogram (hist_l) of a long frame image, and fig. 6 (b) is a histogram (hist_s) of a short frame image; in the histogram, the horizontal axis represents the pixel value, the vertical axis represents the number of pixels corresponding to the pixel value, BIN represents the number of segments of the histogram, and in this context, the number of pixels corresponding to the pixel value is the number of pixels in the image for which the pixel value is the pixel value. As can be seen from fig. 6, the proportion of the pixels with x-axis values in the long frame histogram in the long frame image is the same as the proportion of the pixels with y-axis values in the short frame histogram in the short frame image, and therefore the pixels with x-axis values in the long frame image have y-axis values in the short frame image.

Referring to fig. 7, if a pixel with a luminance value (i.e., a pixel value) of x exists in a long frame image, a pixel with a pixel value of y exists in a short frame image, and a corresponding relationship exists between the pixel with a pixel value of x in the long frame image and the pixel with a pixel value of y in the short frame image, the pixel with a pixel value of x and the pixel with a pixel value of y satisfyThe minimum, i.e. the proportion of the luminance value x to the left and right in the long frame histogram is approximately the same as the proportion of the luminance value y to the left and right in the short frame histogram. Similarly, the pixel value x of the pixel point at the corresponding position in the long frame image at the same position can be obtained by knowing the pixel point with the pixel value y in the short frame image.

In the above step S52, the image capturing device counts the histogram of the long frame image, and determines the signal-to-noise ratio of the long frame pixels in the image frame synthesized dynamically and widely, that is, the long frame signal-to-noise ratio, and the number of the long frame pixels in the image frame based on the histogram of the long frame image.

Image noise is typically represented by variance σ ², first, the image noise variance σ ² is established as a function of image mean μ and Gain: σ ² (μ, gain). And then, calibrating the image noise and the average value according to the sequence of the gains of the sensors, and covering the image noise and the average value to the maximum gain required by the user. Calibration procedures and steps are not described in detail herein, and subsequent steps of the application are all to process sigma ² (mu, gain) according to known parameters.

Signal-to-noise ratio, i.e., signal-to-noise ratio, in embodiments of the present application the image signal-to-noise ratio is defined as the ratio of the mean to the variance, as shown in equation (33) below,

SNR=μ/σ （33）

Wherein SNR represents the signal-to-noise ratio; μ represents the mean value to characterize the signal intensity of the image frame; sigma represents variance.

If the pixel value of the pixel is x, the pixel value of the pixel may be taken as the average μ, and the signal to noise ratio of the pixel under the Gain may be expressed as the following formula (34):

（34）

Where SNR (x, gain) represents the signal-to-noise ratio of a pixel with a pixel value of x, x represents the mean of pixels with a pixel value of x, σ ² (μ, gain) represents the variance of pixels with a pixel value of x.

According to the histogram of the long frame image or the short frame image, the sum of signal to noise ratios of all pixel points with the brightness value x in the long frame image or the short frame image can be determined, and a specific calculation formula is shown in the following formula (35):

SNR(x,Gain)× Hist()，x∈[0，BIN] （35）

Wherein SNR (x, gain) represents the signal-to-noise ratio of a pixel point with a pixel value of x; hist () represents a histogram of a long frame image or a short frame image; hist (x) represents the number of pixels with the pixel value x in the long frame image or the short frame image; the value range of x is [0, BIN ].

As can be seen from the above calculation principle, in the image frame of wide dynamic synthesis, the signal-to-noise ratio of the long frame pixel point, that is, the long frame signal-to-noise ratio, can be calculated by the following formula (36):

（36）

Wherein: SNR _{sum_l} represents the signal-to-noise ratio of a long frame pixel point in an image frame, namely the long frame signal-to-noise ratio; BLC (Gain) represents the black level at Gain, which is generally only related to Gain, where subtraction is required and thus does not start from x=0; th_l represents a long frame threshold value of wide dynamic synthesis, i.e., the above-mentioned first luminance threshold value; hist_l (x) represents a histogram of a long frame image; x represents a pixel value, SNR represents a signal-to-noise ratio of a pixel point having a pixel value of x, where the value range of x is [ BLC (Gain), th_l ].

The image acquisition device may determine the number of pixels of the long frame pixel point in the image frame according to the following formula (37):

（37）

Wherein Sum _l represents the number of long frame pixels in the image frame, hist_l (x) represents the histogram of the long frame image, and can be understood as the number of pixels with the pixel value x in the long frame image; th_l represents a long frame threshold for wide dynamic synthesis; BLC (Gain) represents the black level at Gain.

In the step S53, the image acquisition device counts the histogram of the short frame image, and determines the signal-to-noise ratio of the short frame pixels in the image frame synthesized in a wide dynamic manner, that is, the short frame signal-to-noise ratio, and the number of the short frame pixels in the image frame based on the histogram of the short frame image.

In a wide dynamic synthesized image frame, the signal-to-noise ratio of the short frame pixels, i.e., the short frame signal-to-noise ratio, can be calculated by the following equation (38):

（38）

Wherein: SNR _{sum_s} represents the signal-to-noise ratio of the short frame pixel point in the image frame, namely the short frame signal-to-noise ratio; th_s_s represents a third luminance threshold, which can also be understood as a short frame threshold; hist_s (y) represents the histogram of the short frame image and can be understood as the number of pixel points with the pixel value of y in the short frame image; y represents the pixel value, SNR (y, gain) represents the signal-to-noise ratio of the pixel point with the pixel value y under the Gain, where the value range of y is [ th_s_s, BIN ], and BIN represents the number of segments of the histogram.

The image acquisition device may determine the number of pixels of the short frame pixel point in the image frame according to the following equation (39):

（39）

Wherein Sum _s represents the number of pixels of the short frame in the image frame, hist_s (y) represents the histogram of the short frame image, and can be understood as the number of pixels with the pixel value of y in the short frame image; th_s_s represents a short frame threshold for wide dynamic synthesis; BIN represents the number of BINs of the histogram.

In the step S54, the pixel values of the fused pixels in the image frame are determined by the image capturing device by weighting the pixels in the corresponding areas in the long frame image and the short frame image proportionally. Taking the ratio of the pixel value of the long frame image to the fusion pixel as a ratio _l (x), the ratio of the pixel value of the short frame image to the fusion pixel as a ratio _s (x) as an example, wherein the ratio _l (x) and the ratio _s (x) need to satisfy: ratio _l(x)+ratio_s (x) =1, and. It can be understood that the embodiment of the application calculates the duty ratio of the long frame image and the short frame image by adopting a linear interpolation mode, and can also determine the duty ratio of the long frame image and the short frame image by other modes, which is not limited.

According to the wide dynamic synthesis principle, in the fusion area, the embodiment of the application performs fusion calculation according to the short frame multiplied by the exposure ratio and then the long frame in proportion, or adopts a mode of dividing the long frame by the exposure ratio, so that the result of final calculation of the signal to noise ratio is not affected, and the method is not limited.

The image acquisition device may determine the signal-to-noise ratio of the fused pixel points in the image frame, i.e., the fused signal-to-noise ratio, by the following equation (40):

（40）

Wherein, SNR (x, gain _l, Gain_s) represents the fusion signal-to-noise ratio, that is, the signal-to-noise ratio of each pixel point of the fusion area; x×ratio _l (x) represents the average value provided by the pixels in the long frame image as the fused pixels; σ ²(x,Gain_l)×ratio_l ² (x) represents the variance provided by the pixels in the long frame image for the fused pixels; y×ratio _s (x) × ExpRatio represents the average value provided by the pixels in the short frame image for the fused pixels, and ExpRatio represents the exposure ratio of the image frame; σ ²(y,Gain_s)×ratio_s ²(x)×ExpRatio² represents the variance provided by the pixels in the short frame image for the fused pixels; x×ratio _l(x)+ y×ratio_s (x) × ExpRatio represents the mean ,σ²(x, Gain_l)×ratio_l ²(x)+σ²(y, Gain_s)×ratio_s ²(x)×ExpRatio² of the fused pixel points, y is calculated from the corresponding relationship between x and the pixel points in the long frame image and the short frame image, gain _l represents the Gain of the long frame image, and Gain _s represents the Gain of the short frame image.

Based on the fused signal-to-noise ratio calculated by the above formula (40), the image capturing apparatus can determine the sum of the signal-to-noise ratios of the fused regions by the following formula (41):

（41）

Where SNR _{Sum_fusion} represents the sum of the signal-to-noise ratios of the fused regions, SNR (x, gain _l, Gain_s) represents the fused signal-to-noise ratio, hist_l (x) represents the histogram of the long frame image, th_l represents the first luminance threshold, and th_s represents the second luminance threshold.

The image acquisition device can determine the number Sum _fusion of the fusion pixel points in the image frame through the following formula (42);

（42）

it can be understood that in the embodiment of the present application, the above formula (42) is adopted to calculate the number of the fused pixels by taking the long frame image as a reference; similarly, the number of the fused pixels may be calculated using the short frame image as a reference, and this is not limited thereto.

In the step S55, the average signal-to-noise ratio of the image frame may be understood as the average signal-to-noise ratio of each pixel point in the image frame; the image acquisition device may determine the average signal-to-noise ratio of the image frame according to the long frame signal-to-noise ratio, the number of long frame pixels, the short frame signal-to-noise ratio, the number of short frame pixels, the fusion signal-to-noise ratio, and the number of fusion pixels calculated in the steps S52-S54, and specifically,

The image acquisition device may calculate the signal-to-noise ratio of the image frame by the following formula (43):

SNR_{sum_total}=SNR_{sum_l}+SNR_{sum_s}+SNR_{sum_fusion} （43）

Where SNR _{sum_total} represents the signal-to-noise ratio of the image frame, SNR _{sum_l} represents the signal-to-noise ratio of the long frame pixels in the image frame, SNR _{sum_s} represents the signal-to-noise ratio of the short frame pixels in the image frame, and SNR _{sum_fusion} represents the sum of the signal-to-noise ratios of the fused regions, i.e., the signal-to-noise ratio of the fused pixels in the image frame.

The image acquisition device may calculate the number of pixels of the image frame by the following formula (44):

Sum_total=Sum_l+Sum_s+Sum_fusion （44）

Where Sum _total represents the number of pixels in the image frame, sum _l represents the number of long frame pixels in the image frame, sum _s represents the number of short frame pixels in the image frame, and Sum _fusion represents the number of fused pixels in the image frame.

The image acquisition device may determine the average signal-to-noise ratio of the image frames by the following equation (45):

SNR_avrg(hist_l,hist_s,Gain_l,Gain_s)= SNR_{sum_total}/Sum_total （45）

SNR _avrg(hist_l,hist_s,Gain_l,Gain_s) represents the average signal-to-noise ratio of the image frame, SNR _{sum_total} represents the signal-to-noise ratio of the image frame, A histogram representing a long frame image,A histogram representing a short frame image,Indicating the gain of the long-frame image,Representing the short frame image gain.

In some embodiments, the image acquisition device can match the exposure degree of the adjusted long frame image with the preset long frame target exposure degree only by one time adjustment, and the exposure degree of the short frame image is matched with the preset short frame target exposure degree, so that the dark area is not too dark, the bright area is too bright in the synthesized image frame, and the signal-to-noise ratio is better, namely the image quality is better.

In other embodiments, when the exposure level of the second long frame image of the composite target image frame does not match the preset long frame target exposure level, or the exposure level of the second short frame image of the composite target image frame does not match the preset short frame target exposure level, the image capturing apparatus may repeatedly perform the above-mentioned statistics of the exposure level of the first long frame image as the long frame actual measurement exposure level with the target image frame as the new current image frame; and counting the exposure degree of the first short frame image, wherein the first short frame image is used as a step of short frame actual measurement exposure degree until the long frame actual measurement exposure degree is matched with the preset long frame target exposure degree, and the short frame actual measurement exposure degree is matched with the preset short frame image exposure degree.

Although three exemplary descriptions of the step S15 are provided above, the step S15 is separately described, and for more clearly describing the wide dynamic automatic exposure control method of the present application, the overall flow of the wide dynamic automatic exposure control method of the present application will be described below with reference to the examples shown in fig. 2 and 3, respectively.

Referring to fig. 8, fig. 8 is a schematic overall flow chart of a wide dynamic automatic exposure control method using the example of fig. 2 to adjust exposure parameters, which may include the following steps:

step S81: start to

Step S82: initializing exposure;

In the embodiment of the application, exposure initialization can be understood as that the image acquisition device shoots a first long frame image and a first short frame image and dynamically synthesizes a current image frame according to the first long frame image and the first short frame image.

Step S83: obtaining and processing statistical information to obtain an exposure state;

in the embodiment of the present application, the exposure state value may be understood as the measured exposure degree of the long frame and the measured exposure degree of the short frame, and the specific content may be referred to the description in step S11.

Step S84: determining whether the exposure state is within the target exposure range, if the exposure state is not within the target exposure range, executing step S85, and if the exposure state is within the target range, executing step S811;

In the embodiment of the present application, the target range may be understood as a preset long frame target exposure level and a preset short frame target exposure level, and the specific content may be referred to the description in step S12.

Step S85: obtaining the next original exposure and exposure parameters according to the exposure state and the target exposure range;

In the embodiment of the application, the next original exposure can be understood as the initial long frame exposure, the initial short frame exposure, and the total exposure of the target image frames, and the next exposure parameters can be understood as the long frame initial shutter, the long frame initial gain, the short frame initial shutter, and the short frame initial gain. For details, see the description in step S13.

Step S86: determining a maximum shutter of the first priority frame according to a pre-established corresponding relation between gain and exposure ratio and shutter; for details, see the description in step S21.

Step S87: calculating the exposure parameters to be configured actually according to the exposure ratio and the limitation of the exposure parameters;

In the embodiment of the present application, the exposure parameters to be actually configured may be understood as the target shutter and the target gain of the second long frame image, and the target shutter and the target gain of the second short frame image, and the specific details may be found in the descriptions in the above steps S22 to S25.

Step S88: determining whether the exposure parameter is changed, and if the exposure parameter is not changed, executing step S811; if the exposure parameters change, executing step S89; see for details the description in step S16 above.

Step S89: updating exposure parameters and waiting for the exposure parameters to take effect; see for details the description in step S16 above.

Step S810: and shooting the long frame image and the short frame image by using the updated exposure parameters, and returning to the step S83 until the exposure degree of the long frame image and the short frame image is within the target exposure range.

Step S811: the next round of exposure is performed.

When the exposure state is within the target range, or the exposure parameter is not changed, the image capturing apparatus may directly perform the next round of capturing according to the current exposure parameter, such as directly capturing the long frame image and the short frame image of the next frame image according to the shutter and the gain of the first long frame image and the shutter and the gain of the first short frame image, without adjusting the exposure parameter, which is described in the above steps S12 and S16.

Step S812: and (5) ending.

In the embodiment shown in fig. 8, referring to fig. 9, fig. 9 is a detailed schematic diagram of step S85 in the embodiment of the present application, which may include the following steps:

Step S91: distributing the ratio of the next exposure to the current exposure according to the difference between the exposure state and the target exposure range;

In the embodiment of the application, the next exposure can be understood as the initial long frame exposure of the second long frame image and the initial short frame exposure of the second short frame image; the current exposure amount may be understood as an exposure amount of the first long frame image and an exposure amount of the first short frame image; the ratio of the next exposure to the current exposure can be understood as the adjustment amplitude. See for details the description in step S12 above.

Step S92: calculating the exposure of the first long frame image and the exposure of the first short frame image; see for details the description in step S13 above.

Step S93: calculating initial long frame exposure of the second long frame image, initial short frame exposure of the second short frame image and total exposure of the target image frames; see for details the description in step S13 above.

Step S94: calculating an original target exposure ratio;

in the embodiment of the present application, the original exposure ratio may be understood as the initial exposure ratio of the target image frame, and may be specifically referred to the description in step S14.

Step S95: calculating exposure parameters of original targets of long and short frames;

In the embodiment of the application, the original target exposure parameters of the long and short frame images can be understood as a long frame initial shutter, a long frame initial gain, a short frame initial shutter and a short frame initial gain. For details, see the description in step S13.

In the embodiment shown in fig. 10, referring to fig. 10, fig. 10 is a detailed schematic diagram of step S87 provided in the embodiment of the present application, which may include the following steps,

Step S101: acquiring and calculating a first priority frame shutter gain limit;

the method comprises the steps of obtaining a maximum shutter of a first priority frame, and determining a target shutter value range of the first priority frame and a value range of a target gain of the first priority frame.

Step S102: calculating actual exposure parameters of the first priority frame according to the exposure limit and a preset strategy;

the actual exposure parameters of the first priority frame may be understood as the target shutter and target gain of the first priority frame.

Step S103: acquiring and calculating a second priority frame shutter gain limit;

and determining a shutter maximum value of the second priority frame, and a target shutter value range of the second priority frame and a value range of a target gain of the first priority frame.

Step S104: calculating a second priority frame exposure limit according to the exposure ratio limit;

step S105: and calculating the actual exposure parameters of the second priority frame according to the exposure limit and the preset strategy.

The actual exposure parameters of the second priority frame may be understood as the target shutter and target gain of the second priority frame.

The details of the above steps S101 to S105 can be seen from the description of fig. 2.

Referring to fig. 11, fig. 11 is a schematic overall flow chart of a wide dynamic automatic exposure control method using the example of fig. 3 to adjust exposure parameters, which may include the following steps:

Step S111: starting;

Step S112: initializing exposure; see the description in step S82 above.

Step S113: obtaining and processing statistical information to obtain an exposure state; see the description in step S83 above.

Step S114: determining whether the exposure state is within the target exposure range, if the exposure state is not within the target exposure range, executing step S115, and if the exposure state is within the target range, executing step S1115; see the description in step S84 above.

Step S115: obtaining the next original exposure and exposure parameters according to the exposure state and the target exposure range; see the description in step S85 above.

Step S116: determining a maximum shutter of the first priority frame according to the exposure ratio of the target image frame; see the description in step S32 above.

Step S117: calculating the exposure parameters to be configured actually according to the exposure ratio and the limitation of the exposure parameters; see the description in step S31 to step S36 above.

Step S118: determining whether the exposure parameter changes, if not, executing step S1115; if the exposure parameter changes, executing step S119;

step S119: determining whether the exposure amount changes, if not, executing step S1115; if the exposure parameters change, step S1110 is performed; see the description in step S37 above.

Step S1110: updating exposure parameters and waiting for the exposure parameters to take effect; see the description in step S37-step S38 above.

Step S1111: statistical information is obtained and processed, and a noise curve and a valid exposure parameter of the last round are obtained; see the description in step S39 above.

Step S1112: searching an optimal signal-to-noise ratio exposure parameter combination according to exposure parameter constraint; see description in step S310 above.

Step S1113: updating exposure parameters;

Here, it can be understood that the exposure parameter is updated from the intermediate gain of the first priority frame, the intermediate shutter of the first priority frame, the intermediate gain of the second priority frame, and the intermediate shutter of the second priority frame to the target gain of the first priority frame, the target shutter of the first priority frame, the target gain of the second priority frame, and the target shutter of the second priority frame. .

Step S1114: and shooting the long frame image and the short frame image by using the updated exposure parameters, and returning to execute the step S113 until the exposure degree of the long frame image and the short frame image is within the target exposure range.

Step S1115: performing the next round of exposure; see in particular the description in step S16 above

Step S1116: and (5) ending.

In the embodiment shown in fig. 11, referring to fig. 12, fig. 12 is a detailed schematic diagram of step S1111-step S1113 in fig. 11, and may include the following steps:

step S121: acquiring the relation between image noise and an average value; see the description in step S52 above.

Step S122: acquiring a long and short frame histogram in which the previous exposure is effective; see the description in step S52 above.

Step S123: calculating the signal-to-noise ratio of the wide dynamic image fully selected long frame;

the signal-to-noise ratio of the long frame pixels in the image frame is calculated, see in particular the description in step S52 above.

Step S124: calculating the signal-to-noise ratio of the completely selected short frames of the wide dynamic image;

the signal-to-noise ratio of the short frame pixels in the image frame is calculated, see in particular the description in step S53 above.

Step S125: calculating the signal-to-noise ratio of the wide dynamic image fusion area;

the signal-to-noise ratio of the fused pixel points in the image frame is calculated, see in particular the description in step S54 above.

Step S126: calculating the average signal-to-noise ratio of the wide dynamic image;

The average signal-to-noise ratio of the image frames is calculated, see in particular the description in step S55 above.

Step S127: searching an exposure parameter with optimal average signal-to-noise ratio; see in particular the description of step S310 above.

Step S128: the exposure parameters are validated.

Taking a specific example as an example, after the exposure parameters are determined in the manner shown in fig. 8 in the embodiment of the present application, compared with the traditional method, the gain of the long frame image is increased from 30dB to 31.2dB, which results in slightly deteriorated signal-to-noise ratio of the long frame image, but the amplitude of the decrease of the signal-to-noise ratio of the long frame image is not large; the gain of the short frame image is greatly reduced from 30dB to 20.5dB, and the exposure is relatively large, so that the duty ratio of the short frame image is higher during wide dynamic synthesis, and the signal-to-noise gain of the short frame image exceeds the signal-to-noise loss of the long frame image, thereby greatly improving the signal-to-noise ratio of the synthesized image frame. See in particular the following table 2.

TABLE 2 comparison of the image SNR of the conventional method with the image SNR of the method of FIG. 8 of the present application

；

Taking a specific example as an example, after the exposure parameters are determined by using the method shown in fig. 11 in the embodiment of the application, compared with the traditional method, the gain of the long frame image is reduced from 30dB to 25.9dB, so that the signal-to-noise ratio of the long frame image is better, and the gain of the short frame image is increased from 30dB to 41.5dB, but because the exposure is smaller, the duty ratio of the long frame image is higher during wide dynamic synthesis, and the signal-to-noise ratio gain of the long frame image exceeds the signal-to-noise ratio loss of the short frame image, so that the signal-to-noise ratio of the synthesized image can be effectively improved.

TABLE 3 comparison of the image SNR of the conventional method with the image SNR of the method of FIG. 11 of the present application

；

In some embodiments, the present application further provides an image capturing device, referring to fig. 13, fig. 13 is a schematic structural diagram of the capturing device provided in the embodiment of the present application, where the image capturing device includes a capturing unit 131 and a processing unit 132,

An acquisition unit 131 for capturing a first long frame image and a first short frame image, and transmitting to a processing unit 132;

The processing unit 132 receives the first long frame image and the first short frame image, obtains a current image frame in response to the wide dynamic synthesis, and counts the exposure degree of the first long frame image as the exposure degree of the long frame actual measurement; counting the exposure degree of a first short frame image as a short frame actual measurement exposure degree, wherein the first long frame image is a long frame image for widely dynamic synthesis of a current image frame, and the first short frame image is a short frame image for widely dynamic synthesis of the current image frame; according to the measured exposure degree of the long frame and the preset target exposure degree of the long frame, determining an adjustment amplitude corresponding to the first long frame image as a long frame adjustment amplitude; determining the corresponding adjustment amplitude of the first short frame image according to the short frame actual measurement exposure degree and the preset short frame target exposure degree, and taking the adjustment amplitude as the short frame adjustment amplitude; determining initial long frame exposure of the second long frame image according to the exposure of the first long frame image and the long frame adjustment amplitude; determining initial short frame exposure of a second short frame image according to the exposure of the first short frame image and the short frame adjustment amplitude, wherein the second long frame image is a long frame image for wide dynamic synthesis target image frames, the second short frame image is a short frame image for wide dynamic synthesis target image frames, and the target image frames are image frames obtained by subsequent shooting of the current image frames; determining a long frame initial shutter and a long frame initial gain of a second long frame image according to the initial long frame exposure and the initial short frame exposure, wherein the long frame initial gain is equal to the short frame initial gain; according to the corresponding relation between the signal-to-noise ratio of the target image frame and the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain, according to the direction for increasing the signal-to-noise ratio, the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain are adjusted to obtain a long frame target shutter, a long frame target gain, a short frame target shutter and a short frame target gain;

the processing unit 132 is further configured to send the long frame target shutter, the long frame target gain, the short frame target shutter, and the short frame target gain to the acquisition unit 131;

The acquisition unit 131 is further configured to receive a long frame target shutter, a long frame target gain, a short frame target shutter, and a short frame target gain; shooting according to the long frame target shutter and the long frame target gain to obtain a second long frame image, and shooting according to the short frame target shutter and the short frame target gain to obtain a second short frame image; transmitting the second long frame image and the second short frame image to the processing unit 132;

The processing unit 132 is further configured to receive a second long frame image and a second short frame image; and obtaining the target image frame by wide dynamic synthesis according to the second long frame image and the second short frame image.

In yet another embodiment of the present application, there is also provided a computer readable storage medium having stored therein a computer program which, when executed by a processor, implements the steps of any of the wide dynamic automatic exposure control methods described above.

In yet another embodiment of the present application, there is also provided a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the wide dynamic automatic exposure control methods of the above embodiments.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a Solid state disk (Solid STATE DISK, SSD), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the image acquisition apparatus, the storage medium, i.e. the computer program product embodiments, the description is relatively simple, as it is substantially similar to the method embodiments, as relevant see the partial description of the method embodiments.

The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A wide dynamic automatic exposure control method, characterized in that the method comprises:

Responding to the wide dynamic synthesis to obtain a current image frame, and counting the exposure degree of a first long frame image to be used as the exposure degree of a long frame actual measurement; counting the exposure degree of a first short frame image as a short frame actual measurement exposure degree, wherein the first long frame image is a long frame image for widely and dynamically synthesizing the current image frame, and the first short frame image is a short frame image for widely and dynamically synthesizing the current image frame;

Determining the corresponding adjustment amplitude of the first long frame image according to the measured exposure degree of the long frame and the preset target exposure degree of the long frame, and taking the adjustment amplitude as the long frame adjustment amplitude; determining the corresponding adjustment amplitude of the first short frame image according to the short frame actual measurement exposure degree and the preset short frame target exposure degree, and taking the adjustment amplitude as the short frame adjustment amplitude;

Determining initial long frame exposure of a second long frame image according to the exposure of the first long frame image and the long frame adjustment amplitude; determining initial short frame exposure of a second short frame image according to the exposure of the first short frame image and the short frame adjustment amplitude, wherein the second long frame image is a long frame image for widely and dynamically synthesizing a target image frame, the second short frame image is a short frame image for widely and dynamically synthesizing the target image frame, and the target image frame is an image frame obtained by shooting the current image frame in a follow-up mode;

determining a long frame initial shutter and a long frame initial gain of the second long frame image according to the initial long frame exposure and the initial short frame exposure, wherein the long frame initial gain is equal to the short frame initial gain;

According to the corresponding relation between the signal-to-noise ratio of the target image frame and the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain, the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain are adjusted according to the direction for increasing the signal-to-noise ratio, and a long frame target shutter, a long frame target gain, a short frame target shutter and a short frame target gain are obtained;

Shooting according to the long frame target shutter and the long frame target gain to obtain a second long frame image, and shooting according to the short frame target shutter and the short frame target gain to obtain a second short frame image;

And obtaining the target image frame by wide dynamic synthesis according to the second long frame image and the second short frame image.

2. The method of claim 1, wherein said adjusting said long frame initial shutter, said long frame initial gain, said short frame initial shutter, and said short frame initial gain in a direction that increases said signal-to-noise ratio results in a long frame target shutter, a long frame target gain, a short frame target shutter, and a short frame target gain, comprising:

determining a shutter corresponding to the initial gain of a first priority frame and the initial exposure ratio of the target image frame as a maximum shutter of the first priority frame according to a preset corresponding relation between gain-exposure ratio-shutter; if the first priority frame is a short frame image, the corresponding relation is as follows: when the exposure ratio is greater than the exposure ratio threshold, the gain is positively correlated with the corresponding shutter, and when the exposure ratio is less than the exposure ratio threshold, the gain is negatively correlated with the corresponding shutter; if the first priority frame is a long frame image, the corresponding relation is as follows: when the exposure ratio is greater than the exposure ratio threshold, the gain is inversely related to the corresponding shutter, and when the exposure ratio is less than the exposure ratio threshold, the gain is positively related to the corresponding shutter; the first priority frame is one of the second long frame image and the second short frame image;

Determining, as a target shutter and a target gain of a first priority frame, a shutter and a gain of the first priority frame when a difference between a target exposure of the first priority frame and an initial exposure of the first priority frame is smaller than a first preset exposure difference and the shutter is not larger than the maximum shutter, according to a maximum shutter of the first priority frame; wherein the target exposure of the first priority frame is determined by the target shutter and the target gain of the first priority frame;

Determining a maximum shutter of a second priority frame according to the target shutter of the first priority frame, wherein the second priority frame is the other of the second long frame image and the second short frame image;

determining a value range of the target exposure of the second priority frame according to the target exposure of the first priority frame and a preset value range of the exposure ratio of the target image frame;

And determining, as the target shutter and the target gain of the second priority frame, according to the maximum shutter of the second priority frame, the shutter and the gain of the second priority frame when the target exposure of the second priority frame is less than the initial exposure of the second priority frame and the target exposure of the second priority frame is within the range of the target exposure of the second priority frame, and the shutter of the second priority frame is not greater than the maximum shutter of the second priority frame.

3. The method of claim 1, wherein said adjusting said long frame initial shutter, said long frame initial gain, said short frame initial shutter, and said short frame initial gain in a direction that increases said signal-to-noise ratio results in a long frame target shutter, a long frame target gain, a short frame target shutter, and a short frame target gain, comprising:

Determining an initial exposure ratio of the target image frame according to the initial long frame exposure and the initial short frame exposure;

Determining a maximum shutter of a first priority frame according to the initial exposure ratio of the target image frame; the first priority frame is one of the second long frame image and the second short frame image;

determining, as an intermediate shutter and an intermediate gain of a first priority frame, a shutter and a gain of the first priority frame when a difference between an intermediate exposure of the first priority frame and an initial exposure of the first priority frame is smaller than a first preset exposure difference and the shutter is not larger than the maximum shutter, according to a maximum shutter of the first priority frame; wherein the intermediate exposure of the first priority frame is determined by an intermediate shutter and an intermediate gain of the first priority frame;

determining a maximum shutter of a second priority frame according to the intermediate shutter of the first priority frame, wherein the second priority frame is the other of the second long frame image and the second short frame image;

Determining, according to the maximum shutter of the second priority frame, a shutter and a gain of the second priority frame when the shutter of the second priority frame is not greater than the maximum shutter of the second priority frame so that a difference between a target exposure amount of the second priority frame and an initial exposure amount of the second priority frame is smaller than a second preset exposure amount difference and the target exposure amount of the second priority frame is within a value range of the target exposure amount of the second priority frame, as an intermediate shutter and an intermediate gain of the second priority frame;

Shooting according to the middle shutter of the first priority frame and the middle gain of the first priority frame to obtain the first priority frame, and shooting according to the middle shutter of the second priority frame and the middle gain of the second priority frame to obtain the second priority frame;

According to the first priority frame and the second priority frame, obtaining an intermediate image frame through wide dynamic synthesis;

counting the average signal-to-noise ratio of the intermediate image frames;

and adjusting the middle shutter of the first priority frame, the middle gain of the first priority frame, the middle shutter of the second priority frame and the middle gain of the second priority frame according to the average signal-to-noise ratio of the middle image frame to obtain the target shutter of the first priority frame, the target gain of the first priority frame, the target shutter of the second priority frame and the target gain of the second priority frame.

4. The method of claim 1, wherein said adjusting said long frame initial shutter, said long frame initial gain, said short frame initial shutter, and said short frame initial gain in a direction that increases said signal-to-noise ratio results in a long frame target shutter, a long frame target gain, a short frame target shutter, and a short frame target gain, comprising:

Determining, as an intermediate shutter and an intermediate gain of the second priority frame, a shutter and a gain of the second priority frame when the shutter of the second priority frame is not greater than the maximum shutter of the second priority frame such that a difference between a target exposure amount of the second priority frame and an initial exposure amount of the second priority frame is smaller than a second preset exposure amount difference and the target exposure amount of the second priority frame is within a value range of the target exposure amount of the second priority frame;

Counting the average signal-to-noise ratio of the current image frame;

And adjusting the middle shutter of the first priority frame, the middle gain of the first priority frame, the middle shutter of the second priority frame and the middle gain of the second priority frame according to the average signal-to-noise ratio of the current image frame to obtain the target shutter of the first priority frame, the target gain of the first priority frame, the target shutter of the second priority frame and the target gain of the second priority frame.

5. A method according to claim 3 or 4, characterized in that determining the average signal-to-noise ratio of the image frames by:

Determining long frame pixel points, short frame pixel points and fusion pixel points in an image frame, wherein the long frame pixel points are pixel points with pixel values completely taken from long frame images in wide dynamic synthesis, the short frame pixel points are pixel points with pixel values completely taken from short frame images in wide dynamic synthesis, and the fusion pixel points are pixel points with pixel values jointly determined by the pixel values of the long frame images and the pixel values of the short frame images in wide dynamic synthesis;

counting the signal-to-noise ratio of the long frame pixel points in the long frame image to be used as a long frame signal-to-noise ratio;

Counting the signal-to-noise ratio of the short frame pixel points in the short frame image to be used as a short frame signal-to-noise ratio;

Weighting the signal-to-noise ratio of the fused pixel points in the long frame image and the short frame image to be used as a fused signal-to-noise ratio;

and determining the average signal-to-noise ratio of the image frame according to the long frame signal-to-noise ratio, the short frame signal-to-noise ratio and the fusion signal-to-noise ratio.

6. The method of claim 5, wherein determining long frame pixels, short frame pixels, and fused pixels in an image frame comprises:

Determining a first brightness threshold and a second brightness threshold according to the brightness value of the long frame image; determining a third brightness threshold according to the second brightness threshold and the corresponding relation between the long frame image and the pixel points in the short frame image; the corresponding relation between the pixel points in the long frame image and the short frame image is as follows: in the pixel value-pixel number histogram of the long frame image and the pixel value-pixel number histogram of the short frame image, the pixel values of the pixel points positioned at the same position of the axis where the pixel values are positioned correspond;

Determining a pixel point with a brightness value not greater than the first brightness threshold value in the long frame image as the long frame pixel point;

determining a pixel point with the brightness value not smaller than the third brightness threshold value in the short frame image as the short frame pixel point;

And determining the pixel points except the long frame pixel point and the short frame pixel point in the image frame as the fusion pixel point.

7. The method according to claim 2, wherein determining, as the maximum shutter of the first priority frame, a shutter corresponding to an initial gain of the first priority frame and an initial exposure ratio of the target image frame according to a preset gain-exposure ratio-shutter correspondence relation, comprises:

Determining a first priority frame shutter duty ratio according to the preset corresponding relation between gain and exposure ratio and shutter, the initial gain of the first priority frame and the initial exposure ratio of the target image frame; the first priority frame shutter duty ratio represents a duty ratio of a maximum shutter of the first priority frame in a maximum shutter sum value, the maximum shutter sum value being a maximum value of shutter sum values of the second long frame image and the second short frame image;

And determining the maximum shutter value of the first priority frame according to the shutter duty ratio of the first priority frame, the maximum shutter and the value.

8. The method of claim 7, wherein the shutter corresponding to the gain is a shutter corresponding to a range of values to which the gain belongs, the gain being positively correlated with the corresponding shutter, comprising: the value range of the gain is positively correlated with the corresponding shutter;

The gain is inversely related to the corresponding shutter, including: the value range to which the gain belongs is inversely related to the corresponding shutter.

9. The method according to any one of claims 1-4, 7-8, wherein,

Repeatedly executing the statistics of the exposure degree of the first long frame image by taking the target image frame as a new current image frame, and taking the statistics of the exposure degree of the first long frame image as a long frame actual measurement exposure degree; and counting the exposure degree of the first short frame image, wherein the first short frame image is used as a step of short frame actual measurement exposure degree until the long frame actual measurement exposure degree is matched with the preset long frame target exposure degree, and the short frame actual measurement exposure degree is matched with the preset short frame image exposure degree.

10. An image acquisition device, characterized in that the image acquisition device comprises an acquisition unit and a processing unit,

The acquisition unit is used for shooting a first long frame image and a first short frame image and sending the first long frame image and the first short frame image to the processing unit;

The processing unit is used for receiving the first long frame image and the first short frame image, responding to wide dynamic synthesis to obtain a current image frame, and counting the exposure degree of the first long frame image to be used as the actual exposure degree of the long frame; counting the exposure degree of a first short frame image as a short frame actual measurement exposure degree, wherein the first long frame image is a long frame image for widely and dynamically synthesizing the current image frame, and the first short frame image is a short frame image for widely and dynamically synthesizing the current image frame; determining the corresponding adjustment amplitude of the first long frame image according to the measured exposure degree of the long frame and the preset target exposure degree of the long frame, and taking the adjustment amplitude as the long frame adjustment amplitude; determining the corresponding adjustment amplitude of the first short frame image according to the short frame actual measurement exposure degree and the preset short frame target exposure degree, and taking the adjustment amplitude as the short frame adjustment amplitude; determining initial long frame exposure of a second long frame image according to the exposure of the first long frame image and the long frame adjustment amplitude; determining initial short frame exposure of a second short frame image according to the exposure of the first short frame image and the short frame adjustment amplitude, wherein the second long frame image is a long frame image for widely and dynamically synthesizing a target image frame, the second short frame image is a short frame image for widely and dynamically synthesizing the target image frame, and the target image frame is an image frame obtained by shooting the current image frame in a follow-up mode; determining a long frame initial shutter and a long frame initial gain of the second long frame image according to the initial long frame exposure and the initial short frame exposure, wherein the long frame initial gain is equal to the short frame initial gain; according to the corresponding relation between the signal-to-noise ratio of the target image frame and the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain, the long frame initial shutter, the long frame initial gain, the short frame initial shutter and the short frame initial gain are adjusted according to the direction for increasing the signal-to-noise ratio, and a long frame target shutter, a long frame target gain, a short frame target shutter and a short frame target gain are obtained;

The processing unit is further configured to send the long frame target shutter, the long frame target gain, the short frame target shutter, and the short frame target gain to the acquisition unit;

The acquisition unit is further used for receiving the long frame target shutter, the long frame target gain, the short frame target shutter and the short frame target gain; shooting according to the long frame target shutter and the long frame target gain to obtain a second long frame image, and shooting according to the short frame target shutter and the short frame target gain to obtain a second short frame image; transmitting the second long frame image and the second short frame image to the processing unit;

The processing unit is further configured to receive the second long frame image and the second short frame image; and obtaining the target image frame by wide dynamic synthesis according to the second long frame image and the second short frame image.