CN115701131A - A method, device and readable storage medium for removing image artifacts - Google Patents

Abstract

The disclosure provides a method, device, and readable storage medium for removing image artifacts, which are applied in the field of image processing. The method includes: acquiring N long-exposure images collected by a camera; The image calculates the probability of each pixel in each non-reference frame image being located in the artifact area; calculates the weight of the corresponding pixel in the corresponding non-reference frame image according to the probability of each pixel in each non-reference frame image being located in the artifact area ; Synthesizing weighted images of the N long exposure images according to the weight of each pixel in the N-1 non-reference frame images. In this disclosure, the corresponding noise variance coefficient of the camera is set according to the performance of different cameras, and the corresponding noise variance coefficient is used to calculate the probability that each pixel in each non-reference frame image is located in the artifact area, thereby effectively removing the noise in the synthesized image. Artifact areas to improve image quality.

Description

A method, device and readable storage medium for removing image artifacts

technical field

The present disclosure relates to the technical field of image processing, and in particular to a method, device and readable storage medium for removing image artifacts.

Background technique

In the high dynamic range image synthesis technology of the camera, multiple frames of long-exposure images with the same brightness are synthesized into one image to improve image quality, increase details and reduce noise. However, the long-exposure method is more prone to camera shake, which causes displacement and blurring of the subject in the picture, and cannot be corrected by image registration. These images directly participating in fusion will produce artifacts in the synthesized image.

In the method of detecting artifact regions through image differences, first select the clearest frame among all frames as a reference frame, use the reference frame to register other frames, calculate the difference between other frames and the reference frame, and set a The threshold is used to filter out the artifact area, and the difference greater than this threshold is considered to be an artifact area, and this area does not participate in subsequent image fusion, thereby alleviating the artifact to a certain extent.

The method for detecting artifact regions by image differences is more sensitive to the setting of the threshold. If the threshold is set lower, it is easy to detect flat regions without information as artifact regions, and these regions do not participate in fusion, affecting The effect of multi-frame noise reduction; if the threshold is set higher, the artifact area with a smaller difference will be missed, resulting in artifacts on the final film.

Contents of the invention

In view of this, the present disclosure provides a method, device and readable storage medium for removing image artifacts.

According to a first aspect of an embodiment of the present disclosure, a method for removing image artifacts is provided, including:

Obtain N long-exposure images collected by the camera; the N long-exposure images include a reference frame image and N-1 non-reference frame images;

According to the noise variance coefficient of the camera and the N long-exposure images, the probability of each pixel in each non-reference frame image being located in the artifact area is calculated;

Calculate the weight of the corresponding pixel in the corresponding non-reference frame image according to the probability that each pixel in each non-reference frame image is located in the artifact area;

The weighted images of the N long-exposure images are synthesized according to the weight of each pixel in the N-1 non-reference frame images.

In one embodiment, the calculation of the probability that each pixel in each non-reference frame image is located in the artifact area according to the noise variance coefficient of the camera and the N long-exposure images includes:

Determine the noise characterization map corresponding to each non-reference frame image according to the noise variance coefficient of the camera and each non-reference frame image;

calculating the probability that each pixel in each non-reference frame image is located in the artifact region according to the noise characterization map and the reference frame image.

In one embodiment, calculating the probability that each pixel in each non-reference frame image is located in the artifact area according to the noise characterization map and the reference frame image includes:

determining a disparity map between each non-reference frame image and the reference frame image;

Calculate the probability that each pixel in each non-reference frame image is located in an artifact area according to the noise representation map and the gap map.

In one embodiment, the determining the noise characterization map corresponding to each non-reference frame image according to the noise variance coefficient of the camera and each non-reference frame image includes:

According to the gray value of the pixel in each non-reference frame image and the noise variance coefficient, the value of the corresponding pixel in the noise representation map is determined.

In one embodiment, the noise variance coefficient includes a first coefficient and a second coefficient;

The determining the value of the corresponding pixel in the noise characterization map according to the gray value of the pixel in each non-reference frame image and the noise variance coefficient includes:

calculating the product of the gray value of the pixel in the non-reference frame image and the first coefficient, and the sum of the product and the second coefficient;

The sum is used as the value of the corresponding pixel in the noise representation map.

In one embodiment, the calculation of the probability that each pixel in the non-reference frame image is located in the artifact region according to the noise representation map and the gap map includes:

Inputting the value corresponding to the same pixel in the noise characterization map and the gap map into a probability density function, and using the output value of the probability density function as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area.

In one embodiment, the calculation of the probability that each pixel in the non-reference frame image is located in the artifact area according to the noise representation map and the gap map further includes:

Inputting the value corresponding to the same pixel in the noise characterization map and the gap map into a probability density function, and determining an output value of the probability density function;

When the output value is greater than or equal to the preset probability, the difference between the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area;

When the output value is less than the preset probability, the sum of the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area;

Wherein, the adjustment value is negatively correlated with the imaging quality of the camera.

In one embodiment, the probability of each pixel in each non-reference frame image being located in the artifact area is negatively correlated with the weight of the corresponding pixel in the corresponding non-reference frame image.

According to a second aspect of an embodiment of the present disclosure, a device for removing image artifacts is provided, including:

An acquisition module, configured to acquire N long-exposure images collected by the camera; the N long-exposure images include a reference frame image and N-1 non-reference frame images;

The first calculation module is used to calculate the probability of each pixel in each non-reference frame image being located in the artifact area according to the noise variance coefficient of the camera and the N long-exposure images;

The second calculation module is used to calculate the weight of the corresponding pixel in the corresponding non-reference frame image according to the probability of each pixel in each non-reference frame image being located in the artifact area;

A synthesis module, configured to synthesize weighted images of the N long-exposure images according to the weight of each pixel in the N-1 non-reference frame images.

In one embodiment, the first calculation module includes:

The first determination module is used to determine the noise characterization map corresponding to each non-reference frame image according to the noise variance coefficient of the camera and each non-reference frame image;

The third calculating module calculates the probability that each pixel in each non-reference frame image is located in the artifact area according to the noise characterization map and the reference frame image.

In one embodiment, the third computing module further includes:

The second determination module is used to determine the difference map between each non-reference frame image and the reference frame image;

The fourth calculation module is further configured to calculate the probability that each pixel in each non-reference frame image is located in the artifact area according to the noise representation map and the gap map.

In one embodiment, the first determining module is further configured to use the following method to determine a noise characterization map corresponding to each non-reference frame image according to the noise variance coefficient of the camera and each non-reference frame image:

According to the gray value of the pixel in each non-reference frame image and the noise variance coefficient, the value of the corresponding pixel in the noise representation map is determined.

In one embodiment, the noise variance coefficient includes a first coefficient and a second coefficient;

The determining the value of the corresponding pixel in the noise characterization map according to the gray value of the pixel in each non-reference frame image and the noise variance coefficient includes:

calculating the product of the gray value of the pixel in the non-reference frame image and the first coefficient, and the sum of the product and the second coefficient;

The sum is used as the value of the corresponding pixel in the noise representation map.

In one embodiment, the fourth calculation module includes:

The first function calling module is used to input the value corresponding to the same pixel in the noise representation map and the gap map into the probability density function, and use the output value of the probability density function as the corresponding pixel in the non-reference frame image The probability that a point is located in the artifact region.

In one embodiment, the fourth calculation module includes:

The second function calling module is used to input the value corresponding to the same pixel in the noise representation map and the gap map into the probability density function, and determine the output value of the probability density function;

An adjustment module, configured to use the difference between the output value and an adjustment value as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area when the output value is greater than or equal to a preset probability;

When the output value is less than the preset probability, the sum of the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area;

Wherein, the adjustment value is negatively correlated with the imaging quality of the camera.

In one embodiment, the probability of each pixel in each non-reference frame image being located in the artifact area is negatively correlated with the weight of the corresponding pixel in the corresponding non-reference frame image.

The present disclosure provides a device for removing image artifacts, including:

processor;

memory for storing processor-executable instructions;

Wherein, the processor is configured to execute executable instructions in the memory to implement the steps of the method.

The present disclosure provides a non-transitory computer-readable storage medium on which executable instructions are stored, and the executable instructions implement the steps of any one of the methods when executed by a processor.

The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: set the corresponding noise variance coefficient of the camera according to the performance of different cameras, and use the corresponding noise variance coefficient to calculate that each pixel in each non-reference frame image is located in the artifact area Probability, so as to determine the weight of each pixel in each non-reference frame image, and use the weight to weight the gray value of the same pixel to obtain a more accurate artifact area, thereby effectively removing the artifacts in the synthesized image area to improve image quality.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

FIG. 1 is a flowchart of a method for removing image artifacts according to an example embodiment;

Fig. 2 is a flow chart of another method for removing image artifacts according to an example embodiment;

Fig. 3 is a flowchart of another method for removing image artifacts according to an example embodiment;

Fig. 4 is an effect diagram of removing image artifacts provided according to an example embodiment;

Fig. 5 is a structural diagram of an apparatus for removing image artifacts according to an example embodiment;

Fig. 6 is a structural diagram of an apparatus for removing image artifacts according to an example embodiment.

Detailed ways

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the examples in this disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of the embodiments of the present disclosure as recited in the appended claims.

Terms used in the embodiments of the present disclosure are for the purpose of describing specific embodiments only, and are not intended to limit the embodiments of the present disclosure. As used in the examples of this disclosure and the appended claims, the singular forms "a", "said" and "the" are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

Considering that the imaging quality of different cameras (sensors) is different, the signal-to-noise ratio of images collected by cameras with poor imaging quality is low, and the noise in the image is large. When calculating the difference between the reference frame and the non-reference frame, These two frames of images will have a large difference in the non-artifact area due to the influence of noise, so that the non-artifact area is misjudged as the artifact area.

An embodiment of the present disclosure provides a method for removing image artifacts, which is applied to a mobile terminal. The mobile terminal has a camera, and the mobile terminal may be a mobile phone, a tablet computer, a smart device, and the like.

Referring to FIG. 1 , FIG. 1 is a flow chart showing a method for removing image artifacts according to an exemplary embodiment. As shown in Figure 1, this method includes:

Step S11, acquiring N long-exposure images collected by the camera; the N long-exposure images include a reference frame image and N-1 non-reference frame images;

Step S12, calculating the probability of each pixel in each non-reference frame image being located in the artifact area according to the noise variance coefficient of the camera and the N long-exposure images;

Step S13, calculating the weight of the corresponding pixel in the corresponding non-reference frame image according to the probability of each pixel in each non-reference frame image being located in the artifact area;

Step S14, compositing weighted images of the N long-exposure images according to the weight of each pixel in the N-1 non-reference frame images.

In one embodiment, before step S11, step S10 is also included to determine the noise variance coefficient of the camera, which specifically includes: determining the noise variance coefficient of the camera through a noise calibration method, so that the noise variance coefficient reflects the imaging quality of the camera.

In one embodiment, the N long-exposure images in step S11 are N long-exposure images with the same brightness. The N long-exposure images with the same brightness are images that have undergone image registration.

In the embodiment of the present disclosure, the corresponding noise variance coefficient of the camera is set according to the performance of different cameras, and the probability of each pixel in each non-reference frame image being located in the artifact area is calculated using the corresponding noise variance coefficient, thereby determining each non-reference frame The weight of each pixel in the image is used to weight the gray value of the same pixel to obtain more accurate artifact areas, thereby effectively removing the artifact areas in the synthesized image and improving the quality of the image.

An embodiment of the present disclosure provides a method for removing image artifacts, which is applied to a mobile terminal. Referring to FIG. 2 , FIG. 2 is a flowchart of a method for removing image artifacts according to an exemplary embodiment. As shown in Figure 2, this method includes:

Step S11, acquiring N long-exposure images collected by the camera; the N long-exposure images include a reference frame image and N-1 non-reference frame images;

Step S12-1, determining a noise characterization map corresponding to each non-reference frame image according to the noise variance coefficient of the camera and each non-reference frame image;

Step S12-2, calculating the probability that each pixel in each non-reference frame image is located in the artifact area according to the noise representation map and the reference frame image.

Step S13, calculating the weight of the corresponding pixel in the corresponding non-reference frame image according to the probability of each pixel in each non-reference frame image being located in the artifact area;

Step S14, compositing weighted images of the N long-exposure images according to the weight of each pixel in the N-1 non-reference frame images.

In the embodiment of the present disclosure, the noise variance coefficient of the camera is used to determine the noise representation map corresponding to each non-reference frame image, which is used to represent the noise distribution in the non-reference frame image, so that each non-reference frame image is calculated through the noise representation map. The probability that each pixel in the frame image is located in the artifact area.

In one embodiment, step S12-1, determining the noise characterization map corresponding to each non-reference frame image according to the noise variance coefficient of the camera and each non-reference frame image, includes: according to the pixels in each non-reference frame image The gray value of the point and the noise variance coefficient are used to determine the value of the corresponding pixel in the noise representation map.

For example: the noise variance coefficient includes a first coefficient a and a second coefficient b.

According to the gray value of the pixel point in each non-reference frame image and the noise variance coefficient, determine the value of the corresponding pixel point in the noise representation map, including: calculating the gray value of the pixel point in the non-reference frame image and the first A coefficient product, and a sum of the product and the second coefficient; using the sum as the value of a corresponding pixel in the noise representation map.

For example: the grayscale value corresponding to the pixel at (x, y) position in the i-th non-reference frame image is Y _i (x, y), and the noise representation map corresponding to the i-th non-reference frame image is N _i (x ,y), calculate the value of the corresponding pixel in the noise representation map N _i (x,y) according to the formula (1):

N _i (x, y) = a · Y _i (x, y) + b (1)

In this embodiment, the noise variance value is represented by the linear transformation result of the gray value in the pixel, and the noise variance value is used to form a noise representation map, so as to effectively reflect the distribution of noise.

An embodiment of the present disclosure provides a method for removing image artifacts, which is applied to a mobile terminal. Referring to FIG. 3 , FIG. 3 is a flow chart showing a method for removing image artifacts according to an exemplary embodiment. As shown in Figure 3, this method includes:

Step S11, acquiring N long-exposure images collected by the camera; the N long-exposure images include a reference frame image and N-1 non-reference frame images;

Step S12-1', determine the noise representation map corresponding to each non-reference frame image according to the noise variance coefficient of the camera; determine the difference map between each non-reference frame image and the reference frame image;

Step S12-2', calculating the probability that each pixel in each non-reference frame image is located in the artifact area according to the noise representation map and the gap map.

Step S13, calculating the weight of the corresponding pixel in the corresponding non-reference frame image according to the probability of each pixel in each non-reference frame image being located in the artifact area;

Step S14, compositing weighted images of the N long-exposure images according to the weight of each pixel in the N-1 non-reference frame images.

Wherein, in step S12-1', the method for determining the noise representation map corresponding to each non-reference frame image according to the noise variance coefficient of the camera is the same as the method in step S12-1, and will not be repeated here.

In one embodiment, determining the difference map between each non-reference frame image and the reference frame image in step S12-1' includes: calculating the gray value of the same pixel in each non-reference frame image and the reference frame image For example, calculating the absolute value of the difference between each non-reference frame image and the gray value of the same pixel in the reference frame image.

For example: the gray value corresponding to the pixel at (x, y) position in the i-th non-reference frame image is Y _i (x, y), and the gray value corresponding to the pixel at (x, y) position in the reference frame image The value is Y _ref (x,y),

Calculate the difference map D _i (x, y) between the i-th non-reference frame image and the reference frame image according to formula (2).

D _i (x,y)＝|Y _ref (x,y)-Y _i (x,y)| (2)

In one embodiment, in step S12-2', according to the noise representation map and the gap map, the probability that each pixel in the non-reference frame image is located in the artifact area is calculated, including:

Inputting the value corresponding to the same pixel in the noise characterization map and the gap map into a probability density function, and using the output value of the probability density function as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area.

For example: Calculate the probability _P i (x, y) corresponding to the pixel at the (x, y) position in the i-th non-reference frame image located in the artifact area according to formula (3):

P _i (x, y) = R (N _i (x, y), D _i (x, y)) (3)

Among them, R is the probability density function.

In one embodiment, in step S12-2', according to the noise representation map and the gap map, the probability that each pixel in the non-reference frame image is located in the artifact area is calculated, including:

Inputting values corresponding to the same pixel in the noise characterization map and the gap map to a probability density function to determine an output value of the probability density function.

When the output value is greater than or equal to the preset probability, the difference between the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area;

When the output value is less than the preset probability, the sum of the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area;

Wherein, the adjustment value is negatively correlated with the imaging quality of the camera.

For example:

When the output value is greater than or equal to the preset probability, as shown in formula (4), the difference between the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area:

P _i (x,y)'=P _i (x,y)-k (4)

When the output value is less than the preset probability, as shown in formula (5), the sum of the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area:

P _i (x,y)'=P _i (x,y)+k (5)

Wherein, k is an adjustment value, for example, k is a value between 0 and 0.5.

The value of the adjustment value k is negatively correlated with the imaging quality of the camera, that is, when the imaging quality of the camera is good, the value of k is small, and when the imaging quality of the camera is poor, the value of k is large. For example: a camera has a better image quality, and when removing image artifacts from N long-exposure images collected by the camera, the value of k is set to 0.1. The imaging quality of a camera is poor, and the value of k is set to 0.4 when performing image artifact removal processing on N long-exposure images collected by the camera. Wherein, the imaging quality of the camera may be represented by an imaging quality index.

In this embodiment, formula (4) and formula (5) are used to adjust the probability calculated by using the probability density function, so as to make corresponding corrections for different cameras, so as to be applicable to various cameras with different imaging qualities.

An embodiment of the present disclosure provides a method for removing image artifacts, which is applied to a mobile terminal. This method includes the method shown in Figure 1, Figure 2 or Figure 3, and:

The probability that each pixel in each non-reference frame image is located in the artifact area is negatively correlated with the weight of the corresponding pixel in the corresponding non-reference frame image.

For example: the probability P i (x, y) corresponding to the pixel at the (x, y) position in the i-th non-reference frame image is located in the artifact area, and the probability P _i (x, y) at the (x, y) position in the i-th non-reference frame image The weight W _i (x, y) corresponding to the pixel points = 1-P _i (x, y).

In the embodiment of the present disclosure, a value negatively correlated with the probability of a pixel in a non-reference frame image being located in an artifact area is used as the weight of the pixel in the non-reference frame image, so that the probability of this pixel being located in an artifact area The larger the value, the smaller the weight, and the smaller the probability that the pixel is located in the artifact area, the larger the weight. Therefore, when N long-exposure images are subsequently used to synthesize an image, the component values of the pixel weights with a lower probability of being located in the artifact area are increased, and the component values of the pixel weights of the less likely area of the non-artifact area are reduced, thereby Efficiently removes artifact regions in composited images.

An embodiment of the present disclosure provides a method for removing image artifacts, which is applied to a mobile terminal. This method includes the method shown in Figure 1, Figure 2 or Figure 3, and:

In step S14, the weighted images of the N long-exposure images are synthesized according to the weight of each pixel in the N-1 non-reference frame images, including:

Set the weight of each pixel in the reference frame image to 1.

For example: the weight W _i (x, y)=1-P _i (x, y) corresponding to the pixel at (x, y) position in the i-th non-reference frame image, and the value of i is 1 to N-1 . The weight W _N (x, y) corresponding to the pixel at the position (x, y) in the reference image frame. Calculate the weighted image Y _f (x, y) of N long-exposure images according to formula (6):

In one example, in the schematic diagram of the processing results shown in FIG. 4 , the lower figure is a synthetic figure obtained by using the processing method of the embodiment of the present disclosure. Compared with the synthetic figure obtained by using the processing method of the prior art in the above figure, The clarity of the character's legs has been significantly improved.

An embodiment of the present disclosure provides a device for removing image artifacts, which is applied to a mobile terminal. Referring to FIG. 5 , FIG. 5 is a structural diagram of an apparatus for removing image artifacts according to an exemplary embodiment. As shown in Figure 5, this device includes:

An acquisition module 51, configured to acquire N long-exposure images collected by the camera; the N long-exposure images include a reference frame image and N-1 non-reference frame images;

The first calculation module 52 is used to calculate the probability of each pixel in each non-reference frame image being located in the artifact area according to the noise variance coefficient of the camera and the N long-exposure images;

The second calculation module 53 is used to calculate the weight of the corresponding pixel in the corresponding non-reference frame image according to the probability of each pixel in each non-reference frame image being located in the artifact area;

The synthesis module 54 is configured to synthesize the weighted images of the N long-exposure images according to the weight of each pixel in the N-1 non-reference frame images.

An embodiment of the present disclosure provides a device for removing image artifacts, which is applied to a mobile terminal, including the device shown in FIG. 5 , and:

The first computing module 52 includes:

The first determination module is used to determine the noise characterization map corresponding to each non-reference frame image according to the noise variance coefficient of the camera and each non-reference frame image;

The third calculating module calculates the probability that each pixel in each non-reference frame image is located in the artifact area according to the noise characterization map and the reference frame image.

An embodiment of the present disclosure provides a device for removing image artifacts, which is applied to a mobile terminal, including the device shown in FIG. 5 , and:

The third calculation module also includes:

The second determination module is used to determine the difference map between each non-reference frame image and the reference frame image;

The fourth calculation module is further configured to calculate the probability that each pixel in each non-reference frame image is located in the artifact area according to the noise representation map and the gap map.

In one embodiment, the first determining module is further configured to use the following method to determine a noise characterization map corresponding to each non-reference frame image according to the noise variance coefficient of the camera and each non-reference frame image:

According to the gray value of the pixel in each non-reference frame image and the noise variance coefficient, determine the value of the corresponding pixel in the noise characterization map.

In one embodiment, the noise variance coefficient includes a first coefficient and a second coefficient;

The determining the value of the corresponding pixel in the noise characterization map according to the gray value of the pixel in each non-reference frame image and the noise variance coefficient includes:

calculating the product of the gray value of the pixel in the non-reference frame image and the first coefficient, and the sum of the product and the second coefficient;

The sum is used as the value of the corresponding pixel in the noise representation map.

In one embodiment,

The fourth computing module includes:

The first function calling module is used to input the value corresponding to the same pixel in the noise representation map and the gap map into the probability density function, and use the output value of the probability density function as the corresponding pixel in the non-reference frame image The probability that a point is located in the artifact region.

An embodiment of the present disclosure provides a device for removing image artifacts, which is applied to a mobile terminal, including the device shown in FIG. 5 , and:

The fourth computing module includes:

The second function calling module is used to input the value corresponding to the same pixel in the noise representation map and the gap map into the probability density function, and determine the output value of the probability density function;

An adjustment module, configured to use the difference between the output value and an adjustment value as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area when the output value is greater than or equal to a preset probability;

When the output value is less than the preset probability, the sum of the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area;

Wherein, the adjustment value is negatively correlated with the imaging quality of the camera.

An embodiment of the present disclosure provides a device for removing image artifacts, which is applied to a mobile terminal, including the device shown in FIG. 5 , and:

The probability that each pixel in each non-reference frame image is located in the artifact area is negatively correlated with the weight of the corresponding pixel in the corresponding non-reference frame image.

An embodiment of the present disclosure provides a device for removing image artifacts, including:

processor;

memory for storing processor-executable instructions;

Wherein, the processor is configured to execute the executable instructions in the memory to implement the steps of the method for removing image artifacts.

An embodiment of the present disclosure provides a non-transitory computer-readable storage medium, on which executable instructions are stored, and the steps of the method for removing image artifacts are implemented when the executable instructions are executed by a processor.

Fig. 6 is a block diagram of an apparatus 600 for removing image artifacts according to an exemplary embodiment. For example, the apparatus 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

6, device 600 may include one or more of the following components: processing component 602, memory 604, power supply component 606, multimedia component 608, audio component 610, input/output (I/O) interface 612, sensor component 614, and communication component 616 .

The processing component 602 generally controls the overall operations of the device 600, such as those associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 602 may include one or more modules that facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602 .

The memory 604 is configured to store various types of data to support operations at the device 600 . Examples of such data include instructions for any application or method operating on device 600, contact data, phonebook data, messages, pictures, videos, and the like. The memory 604 can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

The power supply component 606 provides power to various components of the device 600 . Power components 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for device 600 .

The multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 608 includes a front camera and/or a rear camera. When the device 600 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.

The audio component 610 is configured to output and/or input audio signals. For example, the audio component 610 includes a microphone (MIC) configured to receive external audio signals when the device 600 is in operation modes, such as call mode, recording mode and voice recognition mode. Received audio signals may be further stored in memory 604 or sent via communication component 616 . In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

The I/O interface 612 provides an interface between the processing component 602 and a peripheral interface module. The peripheral interface module may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.

Sensor assembly 614 includes one or more sensors for providing status assessments of various aspects of device 600 . For example, the sensor component 614 can detect the open/closed state of the device 600, the relative positioning of components, such as the display and keypad of the device 600, and the sensor component 614 can also detect a change in the position of the device 600 or a component of the device 600 , the presence or absence of user contact with the device 600 , the device 600 orientation or acceleration/deceleration and the temperature change of the device 600 . The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 614 may also include optical sensors, such as CMOS or CCD image sensors, for use in imaging applications. In some embodiments, the sensor component 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

The communication component 616 is configured to facilitate wired or wireless communication between the apparatus 600 and other devices. The device 600 can access wireless networks based on communication standards, such as WiFi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, apparatus 600 may be programmed by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation for performing the methods described above.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium including instructions, such as the memory 604 including instructions, which can be executed by the processor 620 of the device 600 to implement the above method. For example, the non-transitory computer readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

Other implementations of the embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the embodiments in the present disclosure, which follow the general principles of the embodiments in the present disclosure and include those in the technical field not disclosed in the embodiments. Common knowledge or common technical means. It is intended that the specification and examples be considered exemplary only, with a true scope and spirit of the disclosed embodiments being indicated by the following claims.

It should be understood that the embodiments of the present disclosure are not limited to the precise structures described above and shown in the accompanying drawings, and various combinations of method steps or terminal components disclosed in the present application can be made without departing from the scope thereof. , substitutions, modifications and changes, and these combinations, substitutions, modifications and changes are considered to be included within the scope of the present disclosure. It is intended that the scope of protection claimed by the present disclosure be limited by the appended claims.

It should be noted that in this disclosure, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements , but also includes other elements not expressly listed, or also includes elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element.

Claims (18)

1. A method for removing image artifacts, comprising: Obtain N long-exposure images collected by the camera; the N long-exposure images include a reference frame image and N-1 non-reference frame images; According to the noise variance coefficient of the camera and the N long-exposure images, the probability of each pixel in each non-reference frame image being located in the artifact area is calculated; Calculate the weight of the corresponding pixel in the corresponding non-reference frame image according to the probability that each pixel in each non-reference frame image is located in the artifact area; The weighted images of the N long-exposure images are synthesized according to the weight of each pixel in the N-1 non-reference frame images. 2. The method of claim 1, wherein The calculation of the probability that each pixel in each non-reference frame image is located in the artifact region according to the noise variance coefficient of the camera and the N long-exposure images includes: Determine the noise characterization map corresponding to each non-reference frame image according to the noise variance coefficient of the camera and each non-reference frame image; calculating the probability that each pixel in each non-reference frame image is located in the artifact region according to the noise characterization map and the reference frame image. 3. The method of claim 2, wherein The calculation of the probability that each pixel in each non-reference frame image is located in the artifact area according to the noise characterization map and the reference frame image includes: determining a disparity map between each non-reference frame image and the reference frame image; Calculate the probability that each pixel in each non-reference frame image is located in an artifact area according to the noise representation map and the gap map. 4. The method of claim 2 or 3, wherein, The determining the noise characterization map corresponding to each non-reference frame image according to the noise variance coefficient of the camera and each non-reference frame image includes: According to the gray value of the pixel in each non-reference frame image and the noise variance coefficient, the value of the corresponding pixel in the noise representation map is determined. 5. The method of claim 4, wherein, The noise variance coefficient includes a first coefficient and a second coefficient; The determining the value of the corresponding pixel in the noise characterization map according to the gray value of the pixel in each non-reference frame image and the noise variance coefficient includes: calculating the product of the gray value of the pixel in the non-reference frame image and the first coefficient, and the sum of the product and the second coefficient; The sum is used as the value of the corresponding pixel in the noise representation map. 6. The method of claim 3, wherein, The calculation of the probability that each pixel in the non-reference frame image is located in the artifact region according to the noise representation map and the gap map includes: Inputting the value corresponding to the same pixel in the noise characterization map and the gap map into a probability density function, and using the output value of the probability density function as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area. 7. The method of claim 3, wherein, The calculating the probability that each pixel in the non-reference frame image is located in the artifact region according to the noise representation map and the gap map further includes: Inputting the value corresponding to the same pixel in the noise characterization map and the gap map into a probability density function, and determining an output value of the probability density function; When the output value is greater than or equal to the preset probability, the difference between the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area; When the output value is less than the preset probability, the sum of the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area; Wherein, the adjustment value is negatively correlated with the imaging quality of the camera. 8. The method of claim 1, wherein, The probability that each pixel in each non-reference frame image is located in the artifact area is negatively correlated with the weight of the corresponding pixel in the corresponding non-reference frame image. 9. A device for removing image artifacts, comprising: An acquisition module, configured to acquire N long-exposure images collected by the camera; the N long-exposure images include a reference frame image and N-1 non-reference frame images; The first calculation module is used to calculate the probability of each pixel in each non-reference frame image being located in the artifact area according to the noise variance coefficient of the camera and the N long-exposure images; The second calculation module is used to calculate the weight of the corresponding pixel in the corresponding non-reference frame image according to the probability of each pixel in each non-reference frame image being located in the artifact area; A synthesis module, configured to synthesize weighted images of the N long-exposure images according to the weight of each pixel in the N-1 non-reference frame images. 10. The apparatus of claim 9, wherein The first computing module includes: The first determination module is used to determine the noise characterization map corresponding to each non-reference frame image according to the noise variance coefficient of the camera and each non-reference frame image; The third calculating module calculates the probability that each pixel in each non-reference frame image is located in the artifact area according to the noise characterization map and the reference frame image. 11. The apparatus of claim 10, wherein The third calculation module also includes: The second determination module is used to determine the difference map between each non-reference frame image and the reference frame image; The fourth calculation module is used to calculate the probability that each pixel in each non-reference frame image is located in the artifact area according to the noise representation map and the gap map. 12. Apparatus as claimed in claim 10 or 11, characterized in that, The first determination module is further configured to determine a noise characterization map corresponding to each non-reference frame image according to the noise variance coefficient of the camera and each non-reference frame image using the following method: According to the gray value of the pixel in each non-reference frame image and the noise variance coefficient, the value of the corresponding pixel in the noise representation map is determined. 13. The apparatus of claim 12, wherein The noise variance coefficient includes a first coefficient and a second coefficient; The determining the value of the corresponding pixel in the noise characterization map according to the gray value of the pixel in each non-reference frame image and the noise variance coefficient includes: calculating the product of the gray value of the pixel in the non-reference frame image and the first coefficient, and the sum of the product and the second coefficient; The sum is used as the value of the corresponding pixel in the noise representation map. 14. The apparatus of claim 11, wherein: The fourth computing module includes: The first function calling module is used to input the value corresponding to the same pixel in the noise representation map and the gap map into the probability density function, and use the output value of the probability density function as the corresponding pixel in the non-reference frame image The probability that a point is located in the artifact region. 15. The apparatus of claim 11, wherein The fourth computing module includes: The second function calling module is used to input the value corresponding to the same pixel in the noise representation map and the gap map into the probability density function, and determine the output value of the probability density function; An adjustment module, configured to use the difference between the output value and an adjustment value as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area when the output value is greater than or equal to a preset probability; When the output value is less than the preset probability, the sum of the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact area; Wherein, the adjustment value is negatively correlated with the imaging quality of the camera. 16. The apparatus of claim 9, wherein: The probability that each pixel in each non-reference frame image is located in the artifact area is negatively correlated with the weight of the corresponding pixel in the corresponding non-reference frame image. 17. A device for removing image artifacts, comprising: processor; memory for storing processor-executable instructions; Wherein, the processor is configured to execute the executable instructions in the memory to realize the steps of the method for removing image artifacts in any one of claims 1-8. 18. A non-transitory computer-readable storage medium, on which executable instructions are stored, characterized in that, when the executable instructions are executed by a processor, the removal of image artifacts according to any one of claims 1 to 8 is implemented steps of the method.

Images