CN115988335A - Image white balance correction method, device and equipment for multiple light sources and storage medium - Google Patents

Abstract

The application belongs to the technical field of image processing, and discloses a multi-light-source-oriented image white balance correction method, device, equipment and storage medium, which can be suitable for white balance correction in a multi-light-source environment. Calculating the influence weight of the environment light sources at different positions on each pixel position in the shot image, and calculating an equivalent light source cone cell response value equivalent to the multiple light sources based on the influence weight and the cone cell response value of the multiple light sources; calculating an influence factor of the equivalent light source on each pixel position on the image, and calculating the adaptation degree of the equivalent light source based on the influence factor; calculating a gain coefficient of each viewing cone channel based on the adaptation degree, the viewing cone cell response values of the equivalent light source and the reference light source; and calculating corresponding colors of the shot images under the reference light source by using the gain coefficients, and converting the corresponding colors into RGB (red, green and blue) output of the corrected images to realize white balance correction of the shot images.

Description

Image white balance correction method, device, equipment and storage medium for multiple light sources

Technical Field

The present invention relates to the technical field of camera white balance correction, and in particular to a method, device, equipment and storage medium for multi-light source image white balance correction.

Background Art

The human visual system has the ability to adjust color adaptation, and can adjust the sensitivity curve of cone cells according to the color and brightness of the adapted light environment, so as to keep the color appearance of objects relatively stable under different light sources. The white balance module in the digital camera signal processing flow assumes the function of adjusting the color adaptation reference light source as the correction target, so that the image after white balance correction is consistent with the human eye's visual perception of the real scene. The camera's white balance correction algorithm includes light source estimation and color adaptation adjustment: based on the image color information, the light source estimation module can estimate the color and brightness of the light source in the adapted environment; based on the estimated light source information, the color adaptation adjustment module calculates the gain coefficient of each channel, and uses this coefficient to make independent linear adjustments to the three-channel response values of the original image.

Different from the ideal single-light source environment, there may be multiple different light sources in the real adaptation environment. However, the current white balance algorithm is mainly aimed at the adaptation environment of a single light source. For the complex adaptation field of multiple light sources, the color adaptation adjustment method in the white balance algorithm has a series of problems and challenges: (1) The influence of the spatial distribution of the light source color on the adaptation state is unknown. Under the theoretical framework of the Walkers chromatic adaptation transform, the test light source used for gain coefficient calculation is the estimated ambient light source. However, for the multi-light source environment, the current white balance algorithm does not consider the mutual influence between different light sources, so the calculation method of the test light source is still unknown; (2) The existing white balance algorithm assumes that the human eye has reached a state of complete adaptation, and does not consider the influence of factors such as the color space distribution, brightness space distribution, and adaptation field size of the multi-light source environment on the degree of adaptation, resulting in the color restoration result deviating from the human eye perception.

Summary of the invention

In view of this, the embodiments of the present application provide a method, device, equipment and storage medium for image white balance correction for multiple light sources, which can be applicable to multiple light source environments.

In a first aspect, an embodiment of the present application provides a method for image white balance correction for multiple light sources, and the specific steps include:

Calculating the influence weight of ambient light sources at different positions on each pixel position in the captured image, and calculating the cone cell response value of an equivalent light source equivalent to the multiple light sources based on the influence weight and the cone cell response value of the multiple light sources;

Calculating an influence factor of an equivalent light source on each pixel position on the image, and calculating an adaptability of the equivalent light source based on the influence factor;

Calculating a gain coefficient of each cone channel based on the adaptation degree, the cone cell response values of the equivalent light source and the reference light source;

The gain coefficient is used to calculate the corresponding color of the captured image under the reference light source, and the corresponding color is converted into the RGB output of the correction image to achieve white balance correction of the captured image.

Optionally, based on the actual spacing corresponding to adjacent pixels of the image and Gaussian distribution functions in the horizontal and vertical directions, the weight of influence of light sources at different positions on each pixel position in the captured image is set.

Optionally, for a certain pixel (i, j) in the captured image, the influence weight is set to w _s (m, n, i, j):

Where, d ₀ is the actual spacing between adjacent pixels in the image; σ and k are the unknown constants of the Gaussian distribution curves in the horizontal and vertical directions, respectively; (m, n) is the spatial coordinate of the light source; M and N represent the number of pixels in the horizontal and vertical directions of the image, respectively; 0≤i≤M, 0≤j≤N;

所述基于所述影响权重与多光源的视锥细胞响应值，计算像素(i,j)的等效光源视锥细胞响应值

为：Based on the characteristic matrix of the image acquisition device and the cone response transformation matrix, the cone cell response values of multiple light sources are obtained.

The equivalent light source cone cell response value of pixel (i, j) is calculated based on the influence weight and the cone cell response values of multiple light sources.

for:

Optionally, the calculation of the influence factor of the equivalent light source for each pixel position on the image and the degree of adaptability of the equivalent light source calculated based on the influence factor are:

The chromaticity factor and brightness factor of the equivalent light source for each pixel on the image are calculated, and the scale factor of the equivalent light source is calculated; and the product of the chromaticity factor, the brightness factor and the scale factor is used as the adaptability of the equivalent light source.

Optionally, the gain coefficient of each cone channel is calculated based on the adaptation degree, the cone cell response value of the equivalent light source and the reference light source as follows:

为参考光源的锥体细胞响应值，

为等效光源的锥体细胞响应值，D(i,j)为等效光源的适应程度。in,

is the cone cell response value of the reference light source,

is the cone cell response value of the equivalent light source, and D(i,j) is the adaptation degree of the equivalent light source.

Optionally, the corresponding color of the captured image under the reference light source is calculated by using the gain coefficient:

Based on the characteristic matrix of the image acquisition device and the cone response transformation matrix, the cone cell response value of the captured image is obtained, and the gain coefficient is multiplied by the cone cell response value of the captured image to calculate the corresponding color of the captured image under the reference light source;

为拍摄图像中坐标为(i,j)的像素点在参考光源下对应色的锥体细胞响应值，

为拍摄图像中坐标为(i,j)的像素点锥体细胞响应值；in,

is the cone cell response value of the corresponding color of the pixel with coordinate (i, j) in the captured image under the reference light source,

is the cone cell response value of the pixel with coordinates (i, j) in the captured image;

The corresponding color is converted into the RGB of the corrected image:

Through the inverse matrix of cone response transformation and the inverse matrix of camera characterization, the cone cell response value of the corresponding color of the image under the reference light source is converted back to the RGB space of the camera used:

为锥体响应变换逆矩阵。Among them, A ^-1 is the camera characterization inverse matrix,

is the inverse matrix of the cone response transformation.

Optionally, the reference light source is an isoenergetic white light source, and its tristimulus values are scaled so that its brightness is the same as that of an equivalent light source at a corresponding position, thereby obtaining the tristimulus values of the reference light source, which are then converted into cone cell response values using a cone response transformation matrix.

In a second aspect, an embodiment of the present application provides an image white balance correction device for multiple light sources, including: an equivalent light source module, an adaptation degree module, a gain coefficient module and a correction module;

An equivalent light source module is used to calculate the influence weight of the ambient light sources at different positions on each pixel position in the captured image, and calculate the cone cell response value of the equivalent light source equivalent to the multiple light sources based on the influence weight and the cone cell response value of the multiple light sources;

An adaptability module, used to calculate an influence factor of an equivalent light source on each pixel position on an image, and calculate an adaptability degree of the equivalent light source based on the influence factor;

A gain coefficient module, used for calculating the gain coefficient of each cone channel based on the adaptation degree, the cone cell response value of the equivalent light source and the reference light source;

The correction module calculates the corresponding color of the captured image under the reference light source using the gain coefficient to achieve white balance correction of the captured image.

In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the computer program.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, which implement the steps of any of the above methods when executed by a processor.

Beneficial Effects

First, the embodiment of the present application sets the influence weights corresponding to each pixel in the captured image based on the position information of multiple light sources, and uses the influence weights to equate the multiple light sources to one equivalent light source, thereby ensuring that the method can be applied to situations where there are multiple light sources in the environment.

Second, based on the research result that the color adaptation state is mainly affected by the central field of view, the example of this application uses a two-dimensional Gaussian distribution function to simulate the influence weight of light sources in different positions on the equivalent light source, thereby ensuring that the method is consistent with the visual perception of the human eye.

Third, the embodiment of the present application fully considers the influence of factors such as light source color, brightness, size, etc. on the degree of adaptation, and uses the product of light source color, brightness, size and environmental factors to calculate the gain coefficient, which can avoid the deviation between the image after white balance correction and the human eye vision.

Fourth, the present application can be applied to different imaging electronic devices and has a good applicability prospect.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a flow chart of the method of the present invention.

FIG. 2 is a schematic diagram of a specific characterization process in the method of the present invention.

3 is a schematic diagram showing the corresponding relationship between the position weight in the equivalent light source calculation formula and the distance between the relative position of the light source in the horizontal/vertical direction when the parameter σ has different values in an embodiment of the present invention.

4 is a schematic diagram of the correspondence between the light source position weight and the two-dimensional space pixel coordinates of a captured image, taking the pixel point located at the position (1000, 1000) of the image as a reference object in an embodiment of the present invention.

DETAILED DESCRIPTION

The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

It should be noted that the following embodiments and features in the embodiments may be combined with each other in the absence of conflict; and, based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making any creative work are within the scope of protection of the present disclosure.

It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and/or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and/or practice the method. In addition, other structures and/or functionalities other than one or more of the aspects described herein may be used to implement this device and/or practice this method.

The design ideas of this application are: (1) For shooting environments with multiple light sources, influence weights are set based on the positions of the light sources, and multiple light sources at different positions are equivalent to one equivalent light source. White balance correction is performed based on the relevant information of the equivalent light source, so that the method can be applied to shooting environments with multiple light sources; (2) The captured image is converted into the LMS space, and the processing process is closer to the change process of the spectral sensitivity of the cone cells of the human eye under different light environments; (3) A new variable, the degree of adaptation D, is introduced into the calculation of the gain coefficient to quantify the influence of parameters such as the color, brightness, and size of the light environment on the degree of adaptation, forming an accurate, comprehensive, and widely applicable degree of adaptation evaluation model, and combining it with the camera color characterization model and the light source estimation algorithm; Based on the above three design ideas, white balance correction of images shot in a multi-light source environment is realized, and the white balance correction of the color camera is consistent with the real perception of the human eye.

The embodiment of the present application does not consider the light source estimation problem, and assumes that the light source color in the actual shooting scene has been calculated by the existing light source estimation module.

As shown in FIG1 , an embodiment of the present application is a white balance correction method for a multi-light source environment, and the specific process is as follows:

The weights of influence of ambient light sources at different positions on each pixel position in the captured image are set, and based on the influence weights and the cone cell response values of the multiple light sources, the cone cell response values of the equivalent light source equivalent to the multiple light sources are calculated; the influence factor of the equivalent light source on each pixel position on the image is calculated, and the adaptation degree of the equivalent light source is calculated based on the influence factor; the gain coefficient of each cone channel is calculated based on the adaptation degree, the cone cell response values of the equivalent light source and the reference light source; the corresponding color of the captured image under the reference light source (i.e., the image LMS cone response) is calculated using the gain coefficient, and the corresponding color is converted into the RGB output of the corrected image to achieve white balance correction of the captured image.

This embodiment sets the influence weights corresponding to each pixel in the captured image based on the position information of multiple light sources, and uses the influence weights to equate the multiple light sources to one equivalent light source, thereby ensuring that the method can be applied to situations where there are multiple light sources in the environment.

Optionally, in combination with the above embodiment, based on the actual spacing between adjacent pixels of the image and Gaussian distribution functions in the horizontal and vertical directions, the weight of influence of light sources at different positions on each pixel position in the captured image is set.

The example of this application is based on the research result that the color adaptation state is mainly affected by the central field of view, and uses a two-dimensional Gaussian distribution function to simulate the influence weight of light sources in different positions on the equivalent light source, thereby ensuring that the method is consistent with the visual perception of the human eye.

In combination with the above embodiment, for a certain pixel (i, j) in the captured image, the influence weight is set to w _s (m, n, i, j):

Where, d ₀ is the actual spacing between adjacent pixels in the image; σ and k are the unknown constants of the Gaussian distribution curves in the horizontal and vertical directions, respectively; (m, n) is the spatial coordinate of the light source; M and N represent the number of pixels in the horizontal and vertical directions of the image, respectively; 0≤i≤M, 0≤j≤N;

所述基于所述影响权重与多光源的视锥细胞响应值，计算像素(i,j)的等效光源视锥细胞响应值

为：Based on the characteristic matrix of the image acquisition device and the cone response transformation matrix, the cone cell response values of multiple light sources are obtained.

The equivalent light source cone cell response value of pixel (i, j) is calculated based on the influence weight and the cone cell response values of multiple light sources.

for:

In combination with the above embodiment, optionally, the influence factor of the equivalent light source for each pixel position on the image is calculated, and the adaptability of the equivalent light source is calculated based on the influence factor as follows:

The chromaticity factor and brightness factor of the equivalent light source for each pixel on the image are calculated, and the scale factor of the equivalent light source is calculated; and the product of the chromaticity factor, the brightness factor and the scale factor is used as the adaptability of the equivalent light source.

This embodiment fully considers the influence of factors such as light source color, brightness, size, etc. on the degree of adaptation, and uses the product of light source color, brightness, size and environmental factors to calculate the gain coefficient, which can avoid the deviation between the image after white balance correction and human vision, and jointly forms an accurate, comprehensive, and color adaptation adjustment method suitable for various multi-light source environments, thereby realizing the white balance correction of color cameras consistent with the real perception of the human eye.

In combination with the above embodiment, optionally, based on the adaptation degree, the cone cell response values of the equivalent light source and the reference light source, the gain coefficient of each cone channel is calculated as:

为参考光源的锥体细胞响应值，

为等效光源的锥体细胞响应值，D(i,j)为等效光源的适应程度。in,

is the cone cell response value of the reference light source,

is the cone cell response value of the equivalent light source, and D(i,j) is the adaptation degree of the equivalent light source.

In combination with the above embodiment, the corresponding color of the captured image under the reference light source calculated by using the gain coefficient is:

Based on the characteristic matrix of the image acquisition device and the cone response transformation matrix, the cone cell response value of the captured image is obtained, and the gain coefficient is multiplied by the cone cell response value of the captured image to calculate the corresponding color of the captured image under the reference light source;

为拍摄图像中坐标为(i,j)的像素点在参考光源下对应色的锥体细胞响应值，

为拍摄图像中坐标为(i,j)的像素点锥体细胞响应值；in,

is the cone cell response value of the corresponding color of the pixel with coordinate (i, j) in the captured image under the reference light source,

is the cone cell response value of the pixel with coordinates (i, j) in the captured image;

The corresponding color is converted into the RGB of the corrected image:

Through the inverse matrix of cone response transformation and the inverse matrix of camera characterization, the cone cell response value of the corresponding color of the image under the reference light source is converted back to the RGB space of the camera used:

为锥体响应变换逆矩阵。Among them, A ^-1 is the camera characterization inverse matrix,

is the inverse matrix of the cone response transformation.

In this embodiment, the characteristic matrix and cone response transformation matrix of the image acquisition device are used to obtain the cone cell response value of each light source in a multi-light source environment in advance, and the cone cell response value of an equivalent light source equivalent to the multi-light source is obtained by combining the influence weight, so as to ensure that the obtained cone cell response value of the equivalent light source can accurately represent the characteristics of the multi-light source in the environment. In this embodiment, the characteristic matrix and cone response transformation matrix of the image acquisition device are used to convert the captured image into the LMS space for correction. Compared with the prior art based on the device-related RGB space or R/G, B/G space, this embodiment can more accurately solve the problem caused by the change of cone cell spectral sensitivity with the light environment, thereby ensuring that the color correction result is consistent with the perception of the human eye.

In combination with the above embodiment, optionally, the reference light source is an isoenergetic white light source, and its tristimulus values are scaled so that its brightness is the same as that of an equivalent light source at a corresponding position, thereby obtaining the tristimulus values of the reference light source, which are then converted into cone cell response values using a cone response transformation matrix.

Examples:

In this embodiment, the image acquisition device is a digital camera, and white balance correction is performed on the image acquired by the digital camera.

(1) Using simulated D65 light source, obtain the camera’s characterization matrix A;

As shown in Figure 2, the specific process of this step is as follows:

计算标准色卡第i个色块在该光源下的色度值XYZ：101, for a simulated D65 light source with a spectral power distribution of P(λ), use the color matching functions

Calculate the chromaticity value XYZ of the i-th color block of the standard color card under the light source:

Where R ⁱ (λ) is the spectral reflectance of the i-th color block in the standard color card. There are N color blocks in the standard color card, which are combined to obtain an N×3 XYZ tristimulus value matrix X, where each row corresponds to a color block in the color card. The color matching function used can be selected from the CIE 1931 2°, CIE 1964 10°, CIE 2006 2°, and CIE 200610° standard observer color matching functions according to the size of the object in the actual scene.

102, using the digital camera to be calibrated to obtain the RGB response value of each color block in the standard color card under the D65 light source, wherein the response value of the i-th color block is recorded as R _i G _i B _i . The N color blocks in the standard color card are combined to obtain an N×3 RGB camera response value matrix V ₀ , where each row corresponds to a color block in the color card.

103, expand the camera response value matrix to N×11, denoted as V, where s≥3, represents the number of selected polynomial terms, when s=11, the i-th row corresponds to:

104. Calculate the camera characterization matrix:

A＝(V ^T V) ^-1 V ^T X

Among them, the superscript T represents the transposed matrix, and the superscript -1 represents the inverse matrix.

(2) Based on the camera's characterization matrix and cone response transformation matrix, cone cell response values of multiple ambient light sources and cone cell response values of captured images are obtained; the specific process of this step is:

Based on the characteristic matrix of the camera, the chromaticity value of the light source in the actual environment and the camera response value of the captured image are converted to the XYZ color space:

[X ^I ,Y ^I ,Z ^I ]＝V ^I ·A

Wherein, V ^I is the camera response value of the captured image (i.e., the pixel value of the image), and X ^I , Y ^I , and Z ^I are the chromaticity values of the captured image obtained after characteristic transformation (i.e., the tristimulus values).

本步骤中光源色度值可以采用现有技术获得，本申请不在此做进一步描述。The chromaticity value of the light source in the actual environment can be converted in the above way to obtain the tristimulus value

The chromaticity value of the light source in this step can be obtained using existing technology, and this application will not describe it further here.

Based on the cone response transformation matrix, the spatial distribution of the ambient light source and the XYZ tristimulus values of the captured image are converted to the LMS space to obtain the cone cell response values of multiple ambient light sources and the cone cell response values of the captured image:

Among them, M _xyz2lms is the selected cone response transformation matrix. The selectable transformation matrices include HPE matrix, CAT02 matrix, CAT16 matrix, Sharp matrix, BFD matrix, CIE 2006 LMS conversion matrix, etc.

(3) Based on the design concept of equating multiple ambient light sources to one equivalent light source, the cone cell response values of the equivalent light source are calculated using the set light source position weights and the cone cell response values of multiple ambient light sources, and converted to the XYZ color space; the specific process is as follows:

为针对空间坐标为(i,j)的图像像素点对应的等效光源锥体细胞响应值；w_s(m,n,i,j)为空间坐标为(m,n)的环境光源对于空间坐标为(i,j)的图像像素点适应状态的影响权重；d₀为图像相邻像素点对应的实际间距；σ和k分别为水平和垂直方向高斯分布曲线的待定常数，二元高斯分布中各个待定常数的具体数值可以通过实验结果优化获取，可能会随着适应环境和观察方式的变化而变化。其中参数σ在不同数值下，等效光源计算公式中的位置权重与光源相对刺激物体在水平/垂直方向上距离之间的对应关系示意图如图3所示；以位于(1000,1000)的像素点为参考对象，拍摄所得图片的光源位置权重与二维空间像素坐标之间的对应关系示意图如图4所示。Where M and N represent the number of pixels in the horizontal and vertical directions of the image, respectively, 0≤i≤M, 0≤j≤N;

is the cone cell response value of the equivalent light source corresponding to the image pixel with spatial coordinates (i, j); _ws (m,n,i,j) is the influence weight of the ambient light source with spatial coordinates (m,n) on the adaptation state of the image pixel with spatial coordinates (i,j); _d0 is the actual spacing corresponding to adjacent pixels in the image; σ and k are the undetermined constants of the Gaussian distribution curve in the horizontal and vertical directions respectively. The specific values of each undetermined constant in the binary Gaussian distribution can be obtained through optimization of experimental results and may change with changes in the adaptation environment and observation methods. The corresponding relationship between the position weight in the equivalent light source calculation formula and the horizontal/vertical distance of the light source relative to the stimulus object under different values of the parameter σ is shown in Figure 3; the corresponding relationship between the light source position weight and the two-dimensional space pixel coordinates of the captured image with the pixel at (1000,1000) as the reference object is shown in Figure 4.

转换到XYZ色度空间上

The cone cell response value of the equivalent light source

Convert to XYZ color space

(4) Obtain the chromaticity tristimulus values of the reference light source and convert them into LMS space to obtain the cone cell response values of the reference light source. The specific process of this step is as follows:

并且满足

The three stimulus values of the isoenergetic white as the reference light source are scaled so that its brightness is the same as the equivalent light source at the corresponding (i, j) position. The chromaticity three stimulus values of the reference light source are

And satisfy

Next, convert the reference light source chromaticity tristimulus values to the LMS space:

(5) Calculate the gain coefficient of each cone channel based on the cone cell response value of the equivalent light source and the cone cell response value of the reference light source; the specific process of this step is:

First, based on the tristimulus values on the XYZ chromaticity space corresponding to the equivalent light source, the chromaticity factor and brightness factor of the image pixel with the spatial coordinate (i, j) are calculated; the field of view of the adapted light environment of the equivalent light source is obtained as the scale factor, and the chromaticity factor (f _color ), size factor (f _fov ), and brightness factor (f _La ) are multiplied to obtain the degree of adaptation of the human eye to the equivalent light source D(i, j):

Where fov refers to the field of view of the adapted light environment. D(i,j) ranges from 0 (no adaptation) to 1 (full adaptation) The gain coefficient of each cone channel is:

为参考光源的锥体细胞响应值，

为等效光源的锥体细胞响应值。in,

is the cone cell response value of the reference light source,

is the cone cell response value of the equivalent light source.

(6) Use the gain coefficient calculated in step (5) to calculate the corresponding color of the image under the reference light source:

为图片中坐标为(i,j)的像素点在参考光源下对应色的锥体细胞响应值。in,

It is the cone cell response value of the corresponding color of the pixel with coordinates (i, j) in the image under the reference light source.

(7) Through the inverse matrix of cone response transformation and the inverse matrix of camera characterization, the cone cell response value of the corresponding color of the image under the reference light source is converted back to the RGB space of the camera used:

为图片中坐标为(i,j)的像素点在参考光源下对应色的相机响应值，完成了考虑光源颜色空间分布的、更符合人眼感受的多光源彩色相机白平衡校正。in,

The camera response value of the corresponding color of the pixel with coordinates (i, j) in the image under the reference light source is used to complete the multi-light source color camera white balance correction that takes into account the color space distribution of the light source and is more in line with the human eye's perception.

In this embodiment, a binary Gaussian distribution is used to calculate the influence weights of light sources at different positions on the adaptation state, the tristimulus values of the equivalent light source are calculated by weighted integration of the spatial distribution of the light source, and the degree of adaptation of the human visual system to the equivalent light source is calculated in combination with the brightness of the equivalent light source and the size of the adaptation field. This embodiment innovatively proposes the concept of an equivalent light source, solves the problem of color adaptation adjustment in a multi-light source environment, and is of great significance for improving the processing effect of the white balance algorithm and expanding its application scope.

Based on the same inventive concept as the above-mentioned white balance correction method for a multi-light source environment, the embodiment of the present application further provides a white balance correction device for a multi-light source environment, including: an equivalent light source module, an adaptation degree module, a gain coefficient module and a correction module;

An equivalent light source module, used to calculate the influence weight of ambient light sources at different positions on each pixel position in the captured image, and calculate the cone cell response value of the equivalent light source equivalent to the multiple light sources based on the influence weight and the cone cell response value of the multiple light sources;

An adaptability module, used to calculate an influence factor of an equivalent light source on each pixel position on an image, and calculate an adaptability degree of the equivalent light source based on the influence factor;

A gain coefficient module, used for calculating the gain coefficient of each cone channel based on the adaptation degree, the cone cell response value of the equivalent light source and the reference light source;

The correction module calculates the corresponding color of the captured image under the reference light source using the gain coefficient to achieve white balance correction of the captured image.

The white balance correction device for a multi-light source environment in the embodiment of the present application and the white balance correction method for a multi-light source environment described above adopt the same inventive concept and can achieve the same beneficial effects, which will not be described in detail here.

Based on the same inventive concept as the above-mentioned white balance correction method for a multi-light source environment, an embodiment of the present application also provides an electronic device, which may be a mobile phone, a digital camera, a tablet computer, etc. The electronic device includes a memory and a processor.

The processor may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and may implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

As a non-volatile computer-readable storage medium, the memory can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory can include at least one type of storage medium, for example, it can include flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), programmable read-only memory (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic memory, disk, CD, etc. The memory is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and/or data.

A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; the above-mentioned computer storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to: mobile storage devices, random access memory (RAM, Random Access Memory), magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor memory (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)) and other media that can store program codes.

Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly reflected in the form of a software product, which is stored in a storage medium and includes a number of instructions for an electronic device to execute all or part of the methods described in each embodiment of the present application.

The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (10)

1. A multi-light-source-oriented image white balance correction method is characterized by comprising the following specific steps:

calculating the influence weight of the environment light sources at different positions on each pixel position in the shot image, and calculating an equivalent light source cone cell response value equivalent to the multiple light sources based on the influence weight and the cone cell response value of the multiple light sources;

calculating an influence factor of the equivalent light source on each pixel position on the image, and calculating the adaptation degree of the equivalent light source based on the influence factor;

calculating a gain coefficient of each viewing cone channel based on the adaptation degree, the viewing cone cell response values of the equivalent light source and the reference light source;

and calculating the corresponding color of the shot image under the reference light source by using the gain coefficient, and converting the corresponding color into RGB (red, green and blue) output of a corrected image to realize white balance correction of the shot image.

2. The method for correcting the white balance of the image facing multiple light sources according to claim 1, wherein the weight of the influence of the light sources at different positions on each pixel position in the captured image is set based on the actual distance, horizontal and vertical Gaussian distribution functions corresponding to adjacent pixels in the image.

3. The multi-light-source-oriented image white balance correction method according to claim 2, wherein the influence weight is set to w for a certain pixel (i, j) in the captured image _s (m,n,i,j)：

Wherein d is ₀ Actual distances corresponding to adjacent pixel points of the image are obtained; sigma and k are undetermined constants of Gaussian distribution curves in the horizontal direction and the vertical direction respectively, (M, N) are space coordinates of the light source, M, N respectively represent the number of pixels of the image in the horizontal direction and the vertical direction, i is more than or equal to 0 and less than or equal to M, and j is more than or equal to 0 and less than or equal to N;

obtaining response values of multi-light source pyramidal cells based on characterization matrix and pyramidal response transformation matrix of image acquisition equipment

Calculating an equivalent light source cone cell response value ^ based on the impact weight and the cone cell response value of the multiple light sources>

Comprises the following steps:

4. the method for correcting the white balance of the image facing multiple light sources according to claim 1, wherein the method for calculating the influence factor of the equivalent light source on each pixel position on the image comprises the following steps:

calculating the chromaticity factor and the brightness factor of the equivalent light source for each pixel point on the image, and calculating the scale factor of the equivalent light source; and taking the product of the chrominance factor, the luminance factor and the scale factor as the adaptation degree of the equivalent light source.

5. The multi-light-source-oriented image white balance correction method according to claim 4, wherein the gain coefficient of each viewing cone channel is calculated based on the adaptation degree, the cone cell response values of the equivalent light source and the reference light source as follows:

wherein,

is a pyramidal cell response value of the reference light source, <' >>

D (i, j) is the adaptation degree of the equivalent light source.

6. The method for correcting the white balance of the image facing multiple light sources according to claim 5, wherein the corresponding colors of the shot image under the reference light source are calculated by using the gain coefficients as follows:

obtaining a cone cell response value of the shot image based on a characterization matrix and a cone response transformation matrix of the image acquisition equipment, multiplying the gain coefficient by the cone cell response value of the shot image, and calculating the corresponding color of the shot image under a reference light source;

wherein,

for the cone cell response value of the corresponding color of the pixel point with the coordinate (i, j) in the shot image under the reference light source, the value is judged>

The cone cell response value of a pixel point with coordinates (i, j) in a shot image is obtained;

the RGB for converting the corresponding color into a corrected image is:

and (3) converting the response value of the cone cell of the corresponding color of the picture under the reference light source into the RGB space of the used camera again through the cone response transformation inverse matrix and the camera characterization inverse matrix:

wherein, A ^-1 The inverse matrix is characterized for the camera and,

the inverse matrix is transformed for the pyramidal response.

7. The multi-light-source-oriented image white balance correction method according to claim 1 or 2, wherein the reference light source is an isoenergetic white light source, the tristimulus values of the reference light source are obtained by scaling the tristimulus values of the reference light source so that the brightness of the tristimulus values is the same as that of the equivalent light source at the corresponding position, and the tristimulus values of the reference light source are converted into pyramidal cell response values by using a pyramidal response transformation matrix.

8. An image white balance correction apparatus for multiple light sources, comprising: the device comprises an equivalent light source module, an adaptation degree module, a gain coefficient module and a correction module;

the equivalent light source module is used for calculating the influence weight of the environment light sources at different positions on each pixel position in the shot image, and calculating an equivalent light source cone cell response value equivalent to the multiple light sources based on the influence weight and the cone cell response values of the multiple light sources;

the adaptive degree module is used for calculating an influence factor of the equivalent light source on each pixel position on the image and calculating the adaptive degree of the equivalent light source based on the influence factor;

the gain coefficient module is used for calculating the gain coefficient of each viewing cone channel based on the adaptive degree, the viewing cone cell response values of the equivalent light source and the reference light source;

and the correction module calculates the corresponding color of the shot image under the reference light source by using the gain coefficient to realize the white balance correction of the shot image.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the method of any of claims 1-7 are performed when the computer program is executed by the processor.

10. A computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the steps of the method of any one of claims 1 to 7.

Images