CN115589515A - Camera module, electronic equipment and image acquisition method - Google Patents

Abstract

The application discloses a camera module, electronic equipment and an image acquisition method, which belong to the field of camera technology. The camera module includes: a housing and a lens, a filter, a first image sensor, a mirror, a second image sensor and a polarization assembly arranged in the housing; the mirror is arranged correspondingly to the lens, and The included angle between the reflective plane of the reflective mirror and the optical axis direction of the lens is greater than 0° and less than 90°; the polarizing component is arranged between the reflective mirror and the second image sensor; the filter A sheet is arranged between the reflective mirror and the first image sensor; after passing through the lens, part of the incident light passes through the reflective mirror to reach the second image sensor, and part of the incident light reaches the second image sensor after being reflected by the reflective mirror. The first image sensor mirror mirror.

Description

Camera module, electronic equipment and image acquisition method

technical field

The application belongs to the technical field of photography, and in particular relates to a camera module, electronic equipment and an image collection method.

Background technique

With the development of electronic technology, people have higher and higher requirements for image acquisition, but some smooth non-metallic surfaces reflect at a certain angle to form dazzling light, called polarized light, which makes the captured image unclear.

In some instances, using a polarizing filter can filter out the interference of polarized light, making the captured image more saturated in color, but there are also some problems with using a polarizing filter, such as using a polarizing filter at sunrise and sunset times or on a wide-angle lens may make the sky blur Gradients and unevenness appear, and the tone of the image is greatly affected, and the polarizer may also add more ghosting and flare to the image, making the image quality of the shot poor.

Contents of the invention

The embodiment of the present application provides a camera module, electronic equipment, and an image acquisition method, which can solve the problem of affecting the tone of the image, increasing ghosting and light spots when using a polarizer to filter out polarized light in the related art, so that the quality of the captured image is relatively poor. bad question.

In a first aspect, an embodiment of the present application provides a camera module, including: a housing, a lens disposed in the housing, a filter, a first image sensor, a mirror, a second image sensor, and a polarization component;

The reflective mirror is arranged corresponding to the lens, and the included angle between the reflective plane of the reflective mirror and the optical axis direction of the lens is greater than 0° and less than 90°;

The polarizing component is disposed between the mirror and the second image sensor;

The filter is arranged between the mirror and the first image sensor;

After passing through the lens, part of the incident light passes through the reflective mirror to reach the second image sensor, and part of the incident light reaches the first image sensor after being reflected by the reflective mirror.

In a second aspect, an embodiment of the present application provides an electronic device, including the camera module described in the first aspect.

In a third aspect, an embodiment of the present application provides an image acquisition method, which is applied to the electronic device described in the second aspect, and the image acquisition method includes:

Controlling the first image sensor of the camera module of the electronic device to collect at least one first image, and controlling the second image sensor of the camera module to collect at least one second image, wherein the second image is polarized light image;

A target image is obtained based on at least one of the first images and at least one of the second images.

In a fourth aspect, an embodiment of the present application provides an image acquisition device, including the camera module described in the first aspect, and the image acquisition device further includes:

An acquisition module, configured to control the first image sensor of the camera module to acquire at least one first image, and control the second image sensor of the camera module to acquire at least one second image, wherein the second The image is a polarized light image;

A fusion module, configured to obtain a target image based on at least one of the first images and at least one of the second images.

In the fifth aspect, the embodiment of the present application provides an electronic device, the electronic device includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction is executed by The processor implements the steps of the method described in the third aspect when executed.

In a sixth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the third aspect are implemented .

In the seventh aspect, the embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions, to achieve the third aspect the method described.

In an eighth aspect, the embodiment of the present application provides a computer program product/program product, the computer program/program product is stored in a non-transitory storage medium, and the computer program/program product is executed by at least one processor To realize the steps of the method as described in the third aspect.

In the embodiment of the present application, the camera module includes a housing and a lens, a filter, a first image sensor, a mirror, a second image sensor, and a polarization assembly arranged in the housing. The included angle between the reflective plane of the mirror and the optical axis direction of the lens is greater than 0° and less than 90°; the polarization component is arranged between the reflective mirror and the second image sensor; the filter is arranged between the reflective mirror and the first image sensor; After the incident light passes through the lens, part of it passes through the mirror to reach the second image sensor, and part of it is reflected by the mirror to reach the first image sensor. In the embodiment of the present application, a high-definition color image can be obtained by arranging the first image sensor and a filter, and by arranging the second image sensor and a polarization component, the polarized light can be filtered out to obtain an image after the polarized light is filtered out, without the need for a camera module A polarizing filter is added outside the group, so it will not affect the tone of the image, nor will it increase ghosting and flare, and improve the quality of the captured image.

In the embodiment of the present application, the images obtained by the first image sensor and the second image sensor of the camera module are processed by image fusion, image signal processing, etc., and clear, high-definition images with more image details can be obtained. A polarizing filter is added outside the group, so it will not affect the tone of the image, nor will it increase ghosting and flare, and improve the quality of the captured image.

Description of drawings

Fig. 1 is a schematic structural diagram of a camera module provided by an embodiment of the present application;

Fig. 2 is a schematic structural diagram of a polarizing component provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of external light passing through a polarizing component provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a pixel signal processing circuit provided by an embodiment of the present application;

FIG. 5 is a flowchart of an image acquisition method provided by an embodiment of the present application;

Fig. 6 is a schematic diagram of divisions of image brightness collected by the camera module provided by an embodiment of the present application;

Fig. 7 is a schematic diagram of an offset angle when the brightness of each region is the same provided by an embodiment of the present application;

Fig. 8 is a structural block diagram of an image acquisition device provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of an electronic device provided by an embodiment of the present application;

Fig. 10 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application.

in,

100-housing;

200-lens;

310-first image sensor, 320-optical filter;

400 - mirror;

510-second image sensor, 520-polarization component, 521-first polarizer, 522-second polarizer, 523-first drive mechanism, 5231-first drive member, 5232-first universal wheel, 524- The second driving mechanism, 5241-the second driving member, 5242-the second universal wheel.

detailed description

The following will clearly describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this application belong to the protection scope of this application.

The terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the application can be practiced in sequences other than those illustrated or described herein. In addition, "and/or" in the specification and claims means at least one of the connected objects, and the character "/" generally means that the related objects are an "or" relationship.

A camera module, an electronic device, and an image acquisition method provided in the embodiments of the present application will be described in detail below through specific embodiments and application scenarios with reference to the accompanying drawings 1-10.

As shown in Figures 1-4, the embodiment of the present application provides a camera module. As shown in Figure 1, the camera module may include: a housing 100 and a lens 200 disposed in the housing 100, a filter 320, a first image sensor 310, a mirror 400, a second image sensor 510 and a polarization assembly 520.

Wherein, the reflective mirror 400 is arranged corresponding to the lens 200, and the included angle between the reflective plane of the reflective mirror 400 and the optical axis direction of the lens 200 is greater than 0° and less than 90°; the polarization component 520 is arranged between the reflective mirror 400 and the second image sensor 510 Between; the filter 320 is arranged between the mirror 400 and the first image sensor 310; after the incident light passes through the lens 200, part of the incident light passes through the mirror 400 and reaches the second image sensor 510, and part of it is reflected by the mirror 400 and reaches the second image sensor 510. An image sensor 310 .

Wherein, the mirror 400 is a light-transmitting mirror that can both transmit light and reflect light, and the ratio of transmitted light and reflected light can be determined according to actual applications, which is not limited in this embodiment.

For example, the reflector 400 can be a semi-transparent mirror, that is, a 50% reflector, the incident light is irradiated on the reflector 400 through the lens 200, half of the light is irradiated on the polarizing component 520 through the reflector 400, and is filtered by the polarizing component 520 After that, it is irradiated on the second image sensor 510 to form a second image, and the other half of the light is reflected by the mirror 400 and irradiates on the filter 320, and after being filtered by the filter 320, it is irradiated on the first image sensor 310 to form a first image .

It is worth noting that the filter 320 can be an infrared filter, which is used to filter out infrared light, so that the first image sensor 310 has a better light-sensing effect, and obtains a high-definition color first image.

In the embodiment of the present application, the camera module includes a housing 100, a lens 200, a filter 320, a first image sensor 310, a mirror 400, a second image sensor 510 and a polarization assembly 520 disposed in the housing 100, The reflective mirror 400 is arranged corresponding to the lens 200, and the included angle between the reflective plane of the reflective mirror 400 and the optical axis direction of the lens 200 is greater than 0° and less than 90°; the polarization component 520 is arranged between the reflective mirror 400 and the second image sensor 510 The optical filter 320 is arranged between the reflective mirror 400 and the first image sensor 310; after the incident light passes through the lens 200, part of it passes through the reflective mirror 400 to reach the second image sensor 510, and part of it reaches the first image after being reflected by the reflective mirror 400 sensor 310 . In the embodiment of the present application, by setting the first image sensor 310 and the filter 320, a high-definition color image can be obtained, and by setting the second image sensor 510 and the polarization component 520, the polarized light can be filtered to obtain an image after the polarized light is filtered, Since there is no need to add a polarizer outside the camera module, it will not affect the tone of the image, nor will it increase ghosting and flare, improving the quality of the captured image.

As shown in Figure 2, in a possible implementation of the present application, the polarization assembly 520 may include a first polarizer 521, a second polarizer 522, a first drive mechanism 523, and a second drive mechanism 524; the second polarizer 522 is arranged between the mirror 400 and the second image sensor 510, the first polarizer 521 is arranged between the second polarizer 522 and the second image sensor 510, the first driving mechanism 523 is connected with the first polarizer 521, and uses In order to drive the first polarizer 521 to rotate in the first plane, the second driving mechanism 524 is connected to the second polarizer 522 for driving the second polarizer 522 to rotate in the second plane.

Wherein, the first plane is parallel to the second plane.

The first plane is the plane where the mirror surface of the first polarizer 521 is located, and the second plane is the plane where the mirror surface of the second polarizer 522 is located.

That is to say, the second polarizer 522 is used to determine the polarization direction of the incident light, and the first polarizer 521 is used to determine the amount of light entering the second image sensor 510 .

It is worth noting that the first polarizer 521 and the second polarizer 522 are arranged in parallel, and the first drive mechanism 523 and the second drive mechanism 524 can respectively drive the first polarizer 521 and the second polarizer 522 to rotate, so that the second polarizer The first polarizer 521 and the second polarizer 522 present different angles to obtain different polarized images, which can further improve the clarity of the captured image.

In a possible implementation manner of the present application, the first driving mechanism 523 includes a first driving member 5231 and a first universal wheel 5232 , and the second driving mechanism 524 includes a second driving member 5241 and a second universal wheel 5242 .

Wherein, the first universal wheel 5232 is rollingly connected with the first polarizer 521, and the first driving member 5231 drives the first universal wheel 5232 to move; the second universal wheel 5242 is rollingly connected with the second polarizer 522, and the second driving member 5241 drives the second universal wheel 5242 to move.

Wherein, there may be multiple first driving members 5231 , first universal wheels 5232 , second driving members 5241 and second universal wheels 5242 .

It should be noted that the first driver 5231 and the second driver 5241 are respectively arranged on different supports, and the supports are supported on the casing 100, so that the polarization component 520 is located at the side of the second image sensor 510 away from the casing 100. Above, the light entering the second image sensor 510 can be filtered by the first polarizer 521 and the second polarizer 522 .

It is worth noting that the first polarizer 521 and the second polarizer 522 may rotate synchronously or asynchronously, and may be controlled according to practical applications. When rotating synchronously, polarization patterns in different directions can be obtained, and when rotating asynchronously, the amount of incoming light can be controlled by controlling the angle between the two polarizers. When the angle between the first polarizer 521 and the second polarizer 522 is 0, it has the same function as only one polarizer, and only the polarized light in the corresponding direction passes through. As shown in FIG. 3 , the included angle between the first polarizer 521 and the second polarizer 522 is not zero, and images with different brightness can be obtained with different angles, and the brightness formula is as follows.

I＝I ₀ cos ² θ

Wherein, I is the brightness of light when passing through the first polarizer 521; _I0 is the brightness of light when passing through the second polarizer 522; θ is the angle between the first polarizer 521 and the second polarizer 522.

As shown above, when the angle between the first polarizer 521 and the second polarizer 522 is 90°, the light passing through is 0, that is, a completely black image is obtained.

In the embodiment of the present application, the first polarizer 521 and the second polarizer 522 are driven to move through the universal wheel and the driving member. Since the universal wheel can move in different directions, the polarizer can be driven to rotate at any angle, so that the first polarizer Any polarization angle can be obtained between the mirror 521 and the second polarizer 522, so that the final fused image has more details, avoiding the influence of polarized light on the captured image, and improving the shooting quality.

It is worth noting that, in addition to the above-mentioned structure, the first driving mechanism 523 and the second driving mechanism 524 can also be realized by using a limit slideway, a driving member and a supporting member, specifically, the first end of a part of the supporting member and the first end A polarizer 521 is connected, and the second end is arranged in the first limit slideway, and the driver can drive the support to slide in the first limit slideway, so that the first polarizer 521 rotates. Correspondingly, the other part of the support The first end is connected to the second polarizer 522 , the second end is arranged in the second limit slideway, and the driving member can drive the support to slide in the second limit slideway, so that the second polarizer 522 is driven. Other structures are also possible, as long as the rotation of the first polarizer 521 and the second polarizer 522 can be realized, and details will not be repeated in this embodiment of the present application.

Both the first image sensor 310 and the second image sensor 510 in the embodiment of the present application are composed of a plurality of photosensitive imaging pixels, as shown in FIG. 4 , each pixel, that is, the signal processing circuit structure of the photodiode PD is as follows .

The signal processing circuit structure of the pixel provided in the present application is a pinned photodiode pixel structure (Pinned Photodiode Pixel, PPD) pixel structure. The PPD pixel structure includes a photosensitive area of a photodiode (Photodiode, PD), that is, a photodiode, and four transistors, that is, a reset transistor RST, a floating switch TG, a row selector SET, and a signal amplifier SF. The structure is also called 4T pixel structure. PPD allows the introduction of correlated double sampling (Correlated Double Sampling, CDS) circuit, which eliminates thermal noise (kTC noise) introduced by reset and flicker noise (1/f noise) introduced by oxide semiconductor (metal-oxide-semiconductor, mos) transistor ) and dark noise (offset noise). Its working principle is as follows, where VDD is the external voltage; DC is the constant current, which is the reference voltage of the circuit; Vout is the output voltage.

Expose first. RST and TG are turned on at the same time to clear the PD, and then the two are disconnected to start exposure. The electron-hole pairs generated by light irradiation will be separated due to the existence of the PD electric field, and the electrons will move to the n region and the holes will move to the p region.

Next reset. At the end of the exposure, RST is activated, which resets the capacitor FD to a high level.

Then reset level readout. After the reset is completed, read the FD reset level, which includes the offset noise of the mos tube, 1/f noise and kTC noise introduced by the reset, and store the read signal in the first capacitor.

charge transfer again. Activation of the TG completely transfers the charge from the photosensitive region into the FD.

Then the signal level is read out. Next, read the voltage signal from the FD to the second capacitor. The signals here include: signals generated by photoelectric conversion, offset generated by operational amplifiers, 1/f noise, and kTC noise introduced by reset.

The final signal output. The signals stored in the two capacitors are subtracted (such as using CDS, which can eliminate the main noise in the pixel), and the obtained signal is amplified by analog, and then sampled by an analog-to-digital converter (Analog-to DigitalConverter, ADC). Digital signal output can be carried out.

It is worth noting that the first image sensor 310 and the second image sensor 510 can be Bayer array sensors, four-in-one (four pixels of the same color are arranged together) or multi-in-one (such as nine-in-one, sixteen-in-one etc.) sensors, specifically, there is no limitation in the embodiments of the present application, and the actual application shall prevail.

An embodiment of the present application also provides an electronic device, including the camera module provided in any one of the foregoing embodiments. The above-mentioned functions can be realized, and in consideration of the brevity of the text, details will not be described in this embodiment.

The embodiment of the present application also provides an image acquisition method, as shown in FIG. 5 , the image acquisition method is applied to the electronic device provided in the above embodiment, and the image acquisition method may include: the contents shown in S501 to S502 .

In S501, control the first image sensor of the camera module of the electronic device to collect at least one first image, and control the second image sensor of the camera module to collect at least one second image.

Wherein, the second image is a polarized light image. That is to say, after the camera module of the electronic device is powered on, the first image sensor can adjust auto exposure, auto focus, auto white balance, etc. and start to collect at least one first image at a preset frame rate, and the second image sensor And the polarization component above the second image sensor also starts working to collect at least one second image.

Wherein, the preset frame rate may be 30fps (Frame Per Second, the number of frames transmitted per second), or other frame rates, subject to actual application, which is not specifically limited in this application.

In the embodiment of the present application, when collecting images, the first image sensor is used to collect high-definition color images, and the second image sensor is used to collect polarized light images. The polarized light images can be obtained without installing a polarizer outside the camera module. Operation Simple.

In S502, a target image is obtained based on at least one first image and at least one second image.

In the embodiment of the present application, at least one first image captured by the first image sensor and at least one second image captured by the second image sensor may be processed by image fusion, image signal processing, etc., to obtain a clear, Target image with more image details.

Wherein, the image signal processing may include automatic exposure control (Automatic Exposure Control, AEC), automatic gain control (Automatic Gain Control, AGC), automatic white balance (Auto White Balance, AWB), color correction, lens shading, gamma (Gamma ) correction, get rid of dead pixels and so on. Specifically, the processing may be performed by an image signal processing chip, which is not specifically described in this embodiment of the present application.

In the embodiment of the present application, first, the first image sensor of the camera module of the electronic device is controlled to capture at least one first image, and the second image sensor of the camera module is controlled to capture at least one second image, the second image is The polarized light image, and then based on at least one first image and at least one second image, an object image is obtained. In the embodiment of the present application, the first image sensor can be controlled to acquire at least one high-definition color image, the second image sensor can be controlled to acquire at least one polarized light image, and the images acquired by the first image sensor and the second image sensor can be fused to obtain a For high-definition images with more image details, since there is no need to add a polarizer outside the camera module, it will not affect the image tone, nor will it increase ghosting and flare, and improve the quality of the captured image.

In a possible implementation manner of the present application, controlling the second image sensor of the camera module to capture at least one second image may include the following steps:

Step 1. Control the first driving mechanism and the second driving mechanism of the camera module to respectively drive the first polarizer and the second polarizer in the polarization assembly to move, so that the polarization direction of the first polarizer is the first polarization direction, so that the second polarizer The polarization direction of the two polarizers is the second polarization direction.

That is to say, the second image sensor can not only obtain polarized light images, but also can obtain multiple polarization images with different polarization directions through the movement of the first polarizer and the second polarizer.

Specifically, the first polarizer starts to rotate under the drive of the first drive mechanism, and the second polarizer starts to rotate under the drive of the second drive mechanism. At this time, the polarization angles of the first polarizer and the second polarizer are the same, that is, The two polarizers rotate at the same time, and the second image sensor collects multiple polarized light images at a certain frame rate (which can be 120fps, or other frame rates). Polarization images for four polarization directions of °, 45°, 90° and 135°. It is also possible to collect polarization images of more polarization directions, and the principle of collecting polarization images of four polarization directions is the same, that is, to drive the first polarizer and the second polarizer to rotate corresponding angles at the same time.

Step 2, controlling the second image sensor to acquire at least one second image.

Wherein, when the number of the second images is at least two, each second image corresponds to a different polarization direction.

It should be noted that, since the polarized light in the environment will affect the normal light sensitivity of the first image sensor, the first image sensor will be in an overexposed state in the polarized light region, so that image details cannot be obtained. In the embodiment of the present application, by controlling the second image sensor to collect at least one second image with different polarization directions, the at least one second image is used to display detailed information that cannot be obtained by the first image sensor.

In the embodiment of the present application, the first driving mechanism and the second driving mechanism of the camera module are controlled to respectively drive the first polarizer and the second polarizer in the polarization assembly to move, so that the polarization direction of the first polarizer is the first polarization direction, Make the polarization direction of the second polarizer be the second polarization direction, control the second image sensor to collect at least one second image, and when the number of second images is at least two, each second image corresponds to The direction of polarization is different. Multiple image details that cannot be acquired by the first image sensor can be displayed through the second image with different polarization directions, so as to supplement the content that cannot be displayed in the image captured by the first image sensor, so that the finally obtained image is clearer.

Since the fusion of complex polarized imaging images is not just a simple image overlap, but needs to obtain comprehensive image information in the fused image and extract more valuable information, that is, the information obtained from the fused image is better than the source image. The image information is more comprehensive.

In a possible implementation manner of the present application, obtaining the target image based on at least one first image and at least one second image may include the following steps:

Step 1. Perform wavelet decomposition on two second target images whose polarization directions are perpendicular to each other in at least one second image to obtain two multi-scale decomposed images.

For example, an image with a polarization direction of 0° and an image with a polarization direction of 90° are two second target images whose polarization directions are perpendicular to each other; an image with a polarization direction of 45° and an image with a polarization direction of 135° are also two images whose polarization directions are perpendicular to each other The second target image.

In the embodiment of the present application, wavelet decomposition is used for image decomposition, that is, the process of two-dimensional filtering and extraction, that is, according to the multi-resolution theory, a fast algorithm for wavelet decomposition and reconstruction is determined, namely the Mallat algorithm. Specifically, for any two-dimensional image signal f(m, n), the Mallat algorithm can be expressed as:

分别为低频子图像及水平、垂直和对角方向上的子图像，大小均为原来图像的1/4；h(x)和g(x)分别为同一小波的低通滤波器和高通滤波器。in,

are low-frequency sub-images and sub-images in horizontal, vertical and diagonal directions respectively, and the size is 1/4 of the original image; h(x) and g(x) are the low-pass filter and high-pass filter of the same wavelet respectively .

In the embodiment of the present application, the above-mentioned wavelet decomposition is used to decompose the two second target images whose polarization directions are perpendicular to each other, and two multi-scale decomposition images are obtained. The multi-scale decomposition images are the low-frequency sub-image and the horizontal, vertical and diagonal directions sub image.

Step 2: Perform weighted fusion processing on the two multi-scale decomposed images to obtain a first polarization fusion image.

That is to say, the multi-scale decomposition image obtained above, that is, each sub-image is subjected to image subtraction and addition processing according to the corresponding layer after decomposition, and an XOR image (subtraction) and an OR image (addition) can be obtained , and then perform fusion processing to obtain the first polarization fusion image.

Specifically, the two second target images with 0° polarization direction and 90° polarization direction can be respectively subjected to wavelet decomposition to obtain respective low-frequency sub-images and sub-images in horizontal, vertical and diagonal directions, and then the corresponding low-frequency Subtraction of subimages, subtraction of subimages in the corresponding horizontal direction, subtraction of subimages in the corresponding vertical direction, subtraction of subimages in the corresponding diagonal direction, and then fuse the four corresponding subtracted images , to obtain the XOR images of the two second target images of the 0° polarization direction and the 90° polarization direction; add the corresponding low-frequency sub-images, add the corresponding sub-images in the horizontal direction, and add the corresponding sub-images in the vertical direction Image addition, sub-image addition on the corresponding diagonal direction, and then four images corresponding to the addition are fused to obtain two second target images or images of 0° polarization direction and 90° polarization direction; The XOR image and the OR image are fused to obtain a first polarization fused image.

Correspondingly, the above calculation may also be performed on the two second target images with a polarization direction of 45° and a polarization direction of 135° to obtain a corresponding first polarization fusion image.

Step 3: performing image fusion on at least one first image and the first polarization fusion image to obtain the target image.

That is to say, at least one first image captured by the first image sensor can be fused with the first polarization fusion image obtained above to obtain the target image.

In the embodiment of the present application, decomposing and then merging the polarized light images can make the final image have more image information and improve the performance of the polarized image.

Since the image acquired by the camera module not only has polarized light, but also has uneven brightness, that is, the central area of the image is brighter, and the surrounding area is darker, and the uneven brightness makes the image quality poor. The polarization structure can correct the shadow of the lens.

In a possible implementation manner of the present application, before controlling the second image sensor of the camera module to acquire at least one second image, the image acquisition method may further include the following steps:

Step 1. Control the second image sensor of the camera module to collect at least two third images.

Since the lens characteristics of each camera module are different and the assembly methods are also different, there may be differences in the lenses of different camera modules, but when the camera module is assembled, the zero-degree uniformity of the lens is also fixed. Example First, the second image sensor of the camera module is controlled to collect at least two third images. In order to determine the uniformity of the image captured by the lens of the current camera module.

Specifically, a uniform gray card can be used, and the gray card can be placed under a uniform light source, and at least two third images can be collected by the second image sensor of the camera module.

Step 2. Perform image fusion on at least two third images to obtain a second polarization fusion image.

Image fusion may be performed by using the method for obtaining the first polarization fusion image above, or may be performed by other methods, which are not limited in this embodiment of the present application.

Step 3: Determine offset angles corresponding to at least two image areas based on brightness information corresponding to at least two image areas in the second polarization fusion image.

Wherein, the offset angle is the angle between the polarization direction of the first polarizer and the polarization direction of the second polarizer.

In this embodiment of the present application, brightness information may be determined by measuring shadow characteristics of the second polarization fusion image, and each image region corresponds to corresponding brightness information. Specifically, the test can be performed by using the color restoration test software, for example, by using the imatest test software to obtain contour lines with decreasing brightness from the center to the edge.

Divide the contour lines determined above, that is, label each area with a number, as shown in Figure 6, and record the central brightness value and the brightness values of each annular area at the same time, specifically, preset the brightness gradient first, for example, 10 is the lowest Effective bit (Least Significant Bit, lsb). Let's say the center brightness is 100lsb and the edges are 40lsb. Then it will be divided into 1~7 areas from the center to the edge, corresponding to 100lsb~40lsb respectively.

In the embodiment of the present application, according to the brightness value determined above, combined with the brightness formula I=I ₀ cos ² θ, the ratio between the polarization direction of the first polarizer and the polarization direction of the second polarizer can be calculated when the brightness of each area is equal. The included angle is the offset angle corresponding to the two image areas.

For example, if area 1 and area 7 are to be equal, area 1 needs to be changed from 100lsb to 40lsb. According to the above formula, 40/100=cos ² θ can be obtained, and θ is 50.77°. Therefore, when area 1 is exposed, the first The included angle between the polarization direction of the polarizer and the polarization direction of the second polarizer is adjusted to 50.77°, that is, the offset angle corresponding to the region 1 and region 7 is 50.77°. Corresponding adjustments between other areas are also the same as the above adjustments, which will not be described one by one in the embodiment of the present application, and finally obtain the polarization angle when the brightness of each area is the same, as shown in FIG. 7 .

Correspondingly, controlling the second image sensor of the camera module to collect at least one second image may include: when the polarization direction of the second polarizer is the second polarization direction, controlling the first polarizer according to the offset angle Offset to the target angle; controlling the second image sensor to collect at least one second image.

That is to say, the first polarizer can be controlled to shift to a target angle according to the above-mentioned calculated shift angle, and then the second image sensor can be controlled to capture at least one second image.

For example, area 1 and area 7 are equal, and area 1 needs to be changed from 100lsb to 40lsb. According to the above formula, 40/100=cos ² θ can be obtained, and θ is 50.77°. Therefore, when area 1 is exposed, the first polarizer needs to be The angle between the polarization direction of the second polarizer and the polarization direction of the second polarizer is adjusted to 50.77°, that is, the offset angle corresponding to the region 1 and region 7 is 50.77°. Assume that the initial offset direction of the second polarizer is 5°, and the offset direction of the first polarizer is 30°. At this time, the offset direction of the first polarizer needs to be adjusted to 55.77°, so that the first polarizer The angle between the polarization direction of the second polarizer and the polarization direction of the second polarizer is 50.77°.

Through the above adjustments, the images with the same brightness corresponding to each area are cropped and then fused, that is, the image corresponding to area 1 is obtained by exposing area 1 and cropped to obtain the image corresponding to area 1, and the image corresponding to area 2 is obtained by exposing area 2 until all areas are obtained For the corresponding image, the images in several areas are finally fused to obtain an image with consistent brightness in each area, which can make the brightness of the image captured by the camera module uniform, and improve the quality of the captured image.

As shown in FIG. 8 , the embodiment of the present application also provides an image acquisition device, which may include the camera module provided in the above embodiment, and the image acquisition device may further include: an acquisition module 801 and a fusion module 802 .

Wherein, the acquisition module 801 is configured to control the first image sensor of the camera module to acquire at least one first image, and control the second image sensor of the camera module to acquire at least one second image, wherein the second image is a polarization Light image; a fusion module 802, configured to obtain a target image based on at least one first image and at least one second image.

In this embodiment of the present application, first, the acquisition module 801 controls the first image sensor of the camera module to acquire at least one first image, and controls the second image sensor of the camera module to acquire at least one second image, wherein the second The images are polarized images, and then the fusion module 802 obtains the target image based on at least one first image and at least one second image. In the embodiment of the present application, the first image sensor is controlled to acquire at least one high-definition color image, the second image sensor is controlled to acquire at least one polarized light image after the polarized light is filtered, and the images collected by the first image sensor and the second image sensor are Image fusion can obtain non-polarized high-definition images. Since there is no need to add a polarizer outside the camera module, it will not affect the image tone, nor will it increase ghosting and flare, and improve the quality of the captured image.

In a possible implementation manner of the present application, the acquisition module 801 can be used to drive the first polarizer and the second polarizer to move respectively through the first drive mechanism and the second drive mechanism of the camera module to obtain multiple different images. Polarization images with polarization angles, wherein the polarization angle is the angle between the first polarizer and the second polarizer; performing polarization fusion processing on the polarization images to obtain multiple second images with different polarization angles.

In a possible implementation manner of the present application, the acquisition module 801 may be used to: control the first driving mechanism and the second driving mechanism of the camera module to respectively drive the movement of the first polarizer and the second polarizer in the polarization assembly, Make the polarization direction of the first polarizer be the first polarization direction, make the polarization direction of the second polarizer be the second polarization direction; Control the second image sensor to collect and obtain at least one second image; Wherein, the number of second images In the case of at least two images, each second image corresponds to a different polarization direction.

In a possible implementation of the present application, the fusion module 802 may be configured to: respectively perform wavelet decomposition on two second target images whose polarization directions are perpendicular to each other in at least one second image to obtain two multi-scale decomposition images ; performing weighted fusion processing on two multi-scale decomposed images to obtain a first polarization fusion image; performing image fusion on at least one first image and the first polarization fusion image to obtain a target image.

In a possible implementation manner of the present application, the acquisition module 801 may be configured to: control the second image sensor of the camera module to acquire at least two third images; perform image fusion on the at least two third images to obtain the second Polarization fusion image; based on brightness information corresponding to at least two image areas in the second polarization fusion image, determine offset angles corresponding to at least two image areas, where the offset angle is the polarization direction of the first polarizer and the polarization direction of the second polarizer The angle between the polarization directions.

Correspondingly, the acquisition module 801 can be used for: when the polarization direction of the second polarizer is the second polarization direction, according to the offset angle, control the first polarizer to shift to the target angle; control the second image sensor to acquire At least one second image.

The image acquisition device in the embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal. The device may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, tablet computer, notebook computer, palmtop computer, vehicle electronic device, wearable device, ultra-mobile personal computer (ultra-mobile personal computer, UMPC), netbook or personal digital assistant (personal digital assistant) , PDA), etc., the non-mobile electronic device can be a server, a network attached storage (NetworkAttached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., the embodiment of the present application Not specifically limited.

The image acquisition device in the embodiment of the present application may be a device with an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in this embodiment of the present application.

The image acquisition device provided by the embodiment of the present application can realize each process realized by the method embodiment in Fig. 5-Fig. 7 and achieve the same technical effect. To avoid repetition, details are not repeated here.

Optionally, as shown in FIG. 9 , the embodiment of the present application further provides an electronic device 900, including a processor 901, a memory 902, and programs or instructions stored in the memory 902 and operable on the processor 901, When the program or instruction is executed by the processor 901, the various processes of the above-mentioned image acquisition method embodiment can be achieved, and the same technical effect can be achieved. To avoid repetition, details are not repeated here.

It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

FIG. 10 is a schematic diagram of a hardware structure of an electronic device implementing various embodiments of the present application,

The electronic device 1000 includes but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010. . The electronic device also includes a camera module.

The camera module may include: a housing, a lens, a filter, a first image sensor, a reflective mirror, a second image sensor and a polarization component disposed in the housing.

Wherein, the reflective mirror is arranged correspondingly to the lens, and the included angle between the reflective plane of the reflective mirror and the optical axis direction of the lens is greater than 0° and less than 90°; the polarizing component is arranged between the reflective mirror and the second image sensor; the optical filter is set Between the reflective mirror and the first image sensor; after the incident light passes through the lens, part of it passes through the reflective mirror to reach the second image sensor, and part of it is reflected by the reflective mirror to reach the first image sensor.

In an example of the present application, the polarization assembly may include a first polarizer, a second polarizer, a first drive mechanism, and a second drive mechanism; the second polarizer is arranged between the mirror and the second image sensor, and the first The polarizer is arranged between the second polarizer and the second image sensor, the first drive mechanism is connected to the first polarizer, and is used to drive the first polarizer to rotate in the first plane, and the second drive mechanism is connected to the second polarizer connected to drive the second polarizer to rotate in the second plane.

Wherein, the first plane is parallel to the second plane.

In an example of the present application, the first driving mechanism includes a first driving member and a first universal wheel, and the second driving mechanism includes a second driving member and a second universal wheel.

Wherein, the first universal wheel is rollingly connected with the first polarizer, and the first driving member drives the first universal wheel to move; the second universal wheel is rollingly connected with the second polarizer, and the second driving member drives the second universal wheel sports.

Those skilled in the art can understand that the electronic device 1000 can also include a power supply (such as a battery) for supplying power to various components, and the power supply can be logically connected to the processor 1010 through the power management system, so that the management of charging, discharging, and function can be realized through the power management system. Consumption management and other functions. The structure of the electronic device shown in FIG. 10 does not constitute a limitation to the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange different components, and details will not be repeated here. .

Wherein, the processor 1010 is configured to control the first image sensor of the camera module of the electronic device to collect at least one first image, and control the second image sensor of the camera module to collect at least one second image, wherein the second The images are polarized light images; based on at least one first image and at least one second image, the target image is obtained.

In the embodiment of the present application, firstly, the first image sensor of the camera module of the electronic device is controlled to capture at least one first image, and the second image sensor of the camera module is controlled to capture at least one second image, the second image is The polarized light image, and then based on at least one first image and at least one second image, an object image is obtained. In the embodiment of the present application, the first image sensor can be controlled to acquire at least one high-definition color image, the second image sensor can be controlled to acquire at least one polarized light image after filtering the polarized light, and the images collected by the first image sensor and the second image sensor Image fusion can obtain high-definition images without polarization. Since there is no need to add a polarizer outside the camera module, it will not affect the image tone, nor will it increase ghosting and flare, and improve the quality of the captured image.

In a possible implementation manner of the present application, the processor 1010 is configured to control the first driving mechanism and the second driving mechanism of the camera module to respectively drive the movement of the first polarizer and the second polarizer in the polarization assembly, Make the polarization direction of the first polarizer be the first polarization direction, make the polarization direction of the second polarizer be the second polarization direction; control the second image sensor to acquire at least one second image; Wherein, when the number of the second images is at least two, each second image corresponds to a different polarization direction.

In a possible implementation manner of the present application, the processor 1010 is configured to respectively perform wavelet decomposition on the two second target images whose polarization directions are perpendicular to each other in the at least one second image to obtain two multi-scale decomposed images ; performing weighted fusion processing on the two multi-scale decomposition images to obtain a first polarization fusion image; performing image fusion on the at least one first image and the first polarization fusion image to obtain the target image.

In a possible implementation manner of the present application, the processor 1010 is configured to control the second image sensor of the camera module to collect at least two third images; perform image fusion on the at least two third images to obtain A second polarization fusion image; based on brightness information corresponding to at least two image areas in the second polarization fusion image, determine an offset angle corresponding to the at least two image areas, where the offset angle is the first polarization The included angle between the polarization direction of the second polarizer and the polarization direction of the second polarizer; when the polarization direction of the second polarizer is the second polarization direction, according to the offset angle, control the first A polarizer is shifted to a target angle; the second image sensor is controlled to collect at least one second image.

The technical effects of the foregoing embodiments have been described in detail in the foregoing embodiments of the image acquisition method, and will not be repeated here to avoid repetition.

It should be understood that, in this embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042. Camera) to process the image data of still pictures or videos. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes a touch panel 10071 and other input devices 10072 . The touch panel 10071 is also called a touch screen. The touch panel 10071 may include two parts, a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, and joysticks, which will not be repeated here. The memory 1009 can be used to store software programs as well as various data, including but not limited to application programs and operating systems. Processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly processes operating systems, user interfaces, and application programs, and the modem processor mainly processes wireless communications. It can be understood that the foregoing modem processor may not be integrated into the processor 1010 .

The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, each process of the image acquisition method embodiment provided in any of the foregoing embodiments is implemented. And can achieve the same technical effect, in order to avoid repetition, no more details here.

Wherein, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.

The embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the above image acquisition method embodiment Each process can achieve the same technical effect, so in order to avoid repetition, it will not be repeated here.

It should be understood that the chips mentioned in the embodiments of the present application may also be called system-on-chip, system-on-chip, system-on-a-chip, or system-on-a-chip.

The embodiment of the present application also provides a computer program product, the computer program product is stored in a non-transitory storage medium, and when the computer program product is executed by the processor, each process of the above image acquisition method embodiment is realized, And can achieve the same technical effect, in order to avoid repetition, no more details here.

The embodiment of the present application further provides a processing device configured to execute each process of the above image acquisition method embodiment, and can achieve the same technical effect. To avoid repetition, details are not repeated here.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions are performed, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of computer software products, which are stored in a storage medium (such as ROM/RAM, magnetic disk, etc.) , optical disc), including several instructions to enable a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Under the inspiration of this application, without departing from the purpose of this application and the scope of protection of the claims, many forms can also be made, all of which belong to the protection of this application.

Claims (10)

1. A camera module, comprising: a housing and a lens, a filter, a first image sensor, a reflective mirror, a second image sensor and a polarization assembly arranged in the housing; The reflective mirror is arranged corresponding to the lens, and the included angle between the reflective plane of the reflective mirror and the optical axis direction of the lens is greater than 0° and less than 90°; The polarizing component is disposed between the mirror and the second image sensor; The filter is arranged between the mirror and the first image sensor; After passing through the lens, part of the incident light passes through the reflective mirror to reach the second image sensor, and part of the incident light reaches the first image sensor after being reflected by the reflective mirror. 2. The camera module according to claim 1, wherein the polarization assembly comprises a first polarizer, a second polarizer, a first driving mechanism and a second driving mechanism; The second polarizer is disposed between the mirror and the second image sensor; The first polarizer is disposed between the second polarizer and the second image sensor; The first driving mechanism is connected with the first polarizer for driving the first polarizer to rotate in the first plane; the second driving mechanism is connected with the second polarizer for driving the first polarizer The second polarizer rotates in the second plane; Wherein, the first plane is parallel to the second plane. 3. The camera module according to claim 2, wherein the first driving mechanism includes a first driving member and a first universal wheel; the second driving mechanism includes a second driving member and a second universal wheel. to the wheel; The first universal wheel is rollingly connected with the first polarizer, and the first driving member drives the first universal wheel to move; The second universal wheel is rollingly connected with the second polarizer, and the second driving parts drive the second universal wheel to move. 4. An electronic device, comprising: the camera module according to any one of claims 1-3. 5. An image acquisition method, applied to the electronic device as claimed in claim 4, wherein the image acquisition method comprises: Controlling the first image sensor of the camera module of the electronic device to collect at least one first image, and controlling the second image sensor of the camera module to collect at least one second image, wherein the second image is polarized light image; A target image is obtained based on at least one of the first images and at least one of the second images. 6. The image acquisition method according to claim 5, wherein the controlling the second image sensor of the camera module to acquire at least one second image comprises: Controlling the first drive mechanism and the second drive mechanism of the camera module respectively drives the first polarizer and the second polarizer in the polarization assembly to move, so that the polarization direction of the first polarizer is the first polarization direction, so that The polarization direction of the second polarizer is the second polarization direction; controlling the second image sensor to acquire at least one second image; Wherein, when the number of the second images is at least two, each second image corresponds to a different polarization direction. 7. The image acquisition method according to claim 6, wherein said obtaining a target image based on at least one said first image and at least one said second image comprises: performing wavelet decomposition on the two second target images whose polarization directions are perpendicular to each other in the at least one second image to obtain two multi-scale decomposition images; performing weighted fusion processing on the two multi-scale decomposition images to obtain a first polarization fusion image; performing image fusion on the at least one first image and the first polarization fusion image to obtain the target image. 8. The image acquisition method according to claim 6, characterized in that, before the second image sensor controlling the camera module acquires at least one second image, the method further comprises: controlling the second image sensor of the camera module to collect at least two third images; performing image fusion on the at least two third images to obtain a second polarization fusion image; Based on brightness information corresponding to at least two image areas in the second polarization fusion image, determine an offset angle corresponding to the at least two image areas, where the offset angle is the polarization direction of the first polarizer and the polarization direction of the first polarizer. The angle between the polarization directions of the second polarizer; The second image sensor controlling the camera module collects at least one second image, including: When the polarization direction of the second polarizer is the second polarization direction, controlling the first polarizer to shift to a target angle according to the offset angle; The second image sensor is controlled to capture at least one second image. 9. An image acquisition device, characterized in that it comprises the camera module according to any one of claims 1-3, the image acquisition device further comprising: An acquisition module, configured to control the first image sensor of the camera module to acquire at least one first image, and control the second image sensor of the camera module to acquire at least one second image, wherein the second The image is a polarized light image; A fusion module, configured to obtain a target image based on at least one of the first images and at least one of the second images. 10. An electronic device, characterized in that it includes a processor, a memory, and a program or instruction stored on the memory and operable on the processor, and the program or instruction is implemented when executed by the processor. The steps of the image acquisition method as described in any one of claims 5-8.

Images