CN113643198B - Image processing method, device, electronic device and storage medium - Google Patents

Abstract

The application provides an image processing method, an image processing device, electronic equipment and a storage medium, wherein the image processing method comprises the following steps: acquiring a central image block and a neighborhood image block in at least one region of the image, wherein the neighborhood image block is adjacent to the central image block; calculating the distance between the neighborhood image block and the center image block; the method comprises the steps of searching a weight corresponding to the distance from a preset table, and taking the weight as the weight of a neighborhood image block, wherein the preset table is used for representing the corresponding relation between different distances and weights; the pixels in the center image block are filtered based on the weight of the neighborhood image block, the pixel value of the neighborhood image block, the weight of the center image block and the pixel value of the center image block, so that the complexity of image noise reduction calculation can be simplified, and the cost of a hardware circuit is further reduced.

Description

Image processing method, device, electronic device and storage medium

Technical Field

The present application relates to the field of image processing technology, and in particular to an image processing method, device, electronic device and storage medium.

Background technique

With the continuous development of semiconductor chips in digital image processing technology, people can use various shooting devices (such as digital cameras, mobile phones, etc.) to shoot and obtain high-resolution pictures or videos. Among various shooting devices, complementary metal oxide semiconductor (CMOS) image sensors are mainly used to obtain high-resolution pictures or videos.

Due to the inherent hardware limitations of CMOS image sensors, the pictures taken by the camera in many cases have serious brightness and chromaticity noise. In order to improve the quality of the picture, generally speaking, the noise of the original picture (i.e., Bayer image) inside the camera is directly suppressed. In this way, a higher quality image can be obtained. For example, the non-local means filtering method (NLM) is used to reduce the noise of the Bayer image to obtain a higher quality image.

However, the computational complexity of existing image noise reduction algorithms is relatively high. Accordingly, when implemented by an application specific integrated circuit (ASIC), the hardware requirements for the ASIC are relatively high, which will undoubtedly increase the cost of the ASIC.

Summary of the invention

The purpose of the embodiments of the present application is to provide an image processing method, device, electronic device, and storage medium to simplify the computational complexity of image denoising, thereby reducing the cost of ASIC.

In order to solve the above technical problems, the embodiments of the present application provide the following technical solutions:

In a first aspect, the present application provides an image processing method, the method comprising: obtaining a central image block and a neighborhood image block in at least one area of an image, wherein the neighborhood image block is adjacent to the central image block; calculating the distance between the neighborhood image block and the central image block; finding out a weight corresponding to the distance from a preset table and using the weight as the weight of the neighborhood image block, wherein the preset table is used to characterize the correspondence between different distances and weights; and filtering pixels in the central image block based on the weight of the neighborhood image block, the pixel value of the neighborhood image block, the weight of the central image block, and the pixel value of the central image block.

According to a second aspect of the present application, there is provided an image processing device, the device comprising: a receiving module, used to obtain a central image block and a neighborhood image block in at least one area of an image, wherein the neighborhood image block is adjacent to the central image block; a calculating module, used to calculate the distance between the neighborhood image block and the central image block; a searching module, used to find out a weight corresponding to the distance from a preset table and use it as the weight of the neighborhood image block, wherein the preset table is used to characterize the correspondence between different distances and weights; and a filtering module, used to filter pixels in the central image block based on the weight of the neighborhood image block, the pixel value of the neighborhood image block, the weight of the central image block and the pixel value of the central image block.

The third aspect of the present application provides an electronic device, comprising: a processor, a memory, and a bus; wherein the processor and the memory communicate with each other through the bus; and the processor is used to call program instructions in the memory to execute the method in the first aspect.

A fourth aspect of the present application provides a computer-readable storage medium, comprising: a stored program; wherein, when the program is executed, the device where the storage medium is located is controlled to execute the method in the first aspect.

Compared with the prior art, the image processing method provided in the first aspect of the present application, after acquiring the central image block and the neighborhood image block in at least one area of the image, calculates the distance between the neighborhood image block and the central image block, and then finds the weight of the neighborhood image block from a preset table based on the distance between the neighborhood image block and the central image block, and finally filters the pixels in the central image block based on the weight of the neighborhood image block, the pixel value of the neighborhood image block, the weight of the central image block and the pixel value of the central image block. When determining the corresponding weight of each image block based on the distance between each image block, the weight of each image block is calculated one by one based on the Gaussian function, and a table lookup method is used to directly find the weight corresponding to the distance of each image block from the preset table. Since the table lookup is simpler than the function calculation method, it can simplify the complexity of the image denoising calculation, thereby reducing the cost of the hardware circuit.

The image processing device provided in the second aspect, the electronic device provided in the third aspect, and the computer-readable storage medium provided in the fourth aspect of the present application have the same or similar beneficial effects as the image processing method provided in the first aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

FIG1 is a schematic diagram of a flow chart of an image processing method in an embodiment of the present application;

FIG2 is a schematic diagram 1 of each image block in an embodiment of the present application;

FIG3 is a second schematic diagram of each image block in an embodiment of the present application;

FIG4 is a schematic diagram 1 of a preset table in an embodiment of the present application;

FIG5 is a second flow chart of the image processing method in an embodiment of the present application;

FIG6 is a schematic diagram showing that the sliding window is not completely located within the image in an embodiment of the present application;

FIG7 is a schematic diagram of a neighborhood image block and a central image block in an embodiment of the present application;

FIG8 is a schematic diagram 1 of several pixel position templates in an embodiment of the present application;

FIG9 is a second schematic diagram of a preset table in an embodiment of the present application;

FIG10 is a third schematic diagram of each image block in an embodiment of the present application;

FIG11 is a fourth schematic diagram of each image block in an embodiment of the present application;

FIG12 is a second schematic diagram of several pixel position templates in an embodiment of the present application;

FIG13 is a first structural diagram of an image processing device according to an embodiment of the present application;

FIG14 is a second structural diagram of the image processing device in an embodiment of the present application;

FIG. 15 is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

Detailed ways

The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the exemplary embodiments set forth herein. On the contrary, these exemplary embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.

In the prior art, when it is necessary to perform noise reduction processing on an image, generally, an image noise reduction algorithm is used to process the image to obtain a high-quality image. However, the operation process of the image noise reduction algorithm is complicated. Moreover, the image noise reduction algorithm ultimately needs to rely on hardware circuit implementation. As a result, a high-performance hardware circuit is required to support the image noise reduction processing of the image by the image noise reduction algorithm, thereby increasing the cost of the hardware circuit.

After extensive research, the applicant discovered that the reason why the existing image denoising algorithms, especially the NLM algorithm, have a complicated operation process is that the Euclidean distance is used to calculate the distance between each image block in the image. The calculation of the Euclidean distance is relatively cumbersome and requires multiple multiplication operations, which undoubtedly requires the performance of the hardware circuit to be enhanced. In addition, when calculating the corresponding weight of each image block based on the distance between each image block, the Gaussian function is used. The Gaussian function involves the calculation of exponents, which undoubtedly also requires the performance of the hardware circuit to be enhanced, which leads to an increase in the cost of the hardware circuit.

In view of this, an embodiment of the present application provides an image processing method. When determining the corresponding weight of each image block based on the distance between each image block, the weight of each image block is not calculated one by one using a Gaussian function, but a table lookup method is adopted to find the corresponding weight of each image block from the table. In the table, the corresponding weight is calculated in advance based on various distances. In the actual process of image denoising, when it is necessary to obtain the weight corresponding to a certain image block, the weight corresponding to the distance of the image block can be found from the table. Since table lookup is simpler than function calculation, the image processing method provided by the embodiment of the present application can simplify the complexity of image denoising calculation, thereby reducing the cost of hardware circuits.

It should be noted that, in practical applications, the image processing method provided in the embodiment of the present application is mainly applied to the processing of Bayer images. Of course, the image processing method provided in the embodiment of the present application can also be applied to the processing of other images. The types of other images are not limited here.

Next, the image processing method provided in the embodiment of the present application is described in detail.

FIG. 1 is a flow chart of an image processing method in an embodiment of the present application. Referring to FIG. 1 , the method may include:

S101: Acquire a central image block and a neighborhood image block in at least one region of an image.

Among them, the neighborhood image block is adjacent to the central image block.

When it is necessary to perform noise reduction processing on an image, first, obtain the image to be processed. Then, determine at least one area in the image. Since when performing noise reduction processing on an image, not all areas of the image are processed together, but it is divided into multiple areas and processed separately, it is necessary to determine at least one area in the image so that each area can be processed separately. Finally, obtain the central image block and the neighborhood image block in at least one area. Since each area is processed separately later, it is also necessary to divide each area, and each divided area contains multiple image blocks, namely the central image block and the neighborhood image block.

FIG2 is a schematic diagram of each image block in an embodiment of the present application. Referring to FIG2 , in image X, a sliding window is used to determine region Y ₁ . The size of the sliding window is 7×7, that is, the size of region Y ₁ is also 7×7. In region Y ₁ , region Y ₁ is divided into 9 image blocks according to a size of 3×3, namely, image blocks A ₀ , A ₁ , A ₂ , A ₃ , A ₄ , A ₅ , A ₆ , A ₇ , and A ₈ . Among them, there is an overlap of 1 pixel between adjacent image blocks in the horizontal and vertical directions. Here, image block A ₄ is the central image block, and image blocks A ₀ , A ₁ , A ₂ , A ₃ , A ₅ , A ₆ , A ₇ , and A ₈ are neighborhood image blocks. Acquiring each image block is equivalent to acquiring the pixel value of each pixel in each image block, and the pixel value may be a grayscale value. Of course, the pixel value may also be a pixel value corresponding to a red (Red, R), green (Green, G), or blue (Blue, B) channel. The specific category of the pixel value is not limited here.

In practical applications, the number of central image blocks and neighborhood image blocks in an area of an image is 1, and the number of neighborhood image blocks can be 1 or more. The number of neighborhood image blocks and the position of the neighborhood image blocks relative to the central image block are not specifically limited here and can be set according to actual needs. In FIG2 , the number of neighborhood image blocks A ₀ , A ₁ , A ₂ , A ₃ , A ₅ , A ₆ , A ₇ , and A ₈ is 8, which are evenly distributed outside the central image block A ₄ .

S102: Calculate the distance between the neighborhood image block and the central image block.

In other words, it is equivalent to calculating the similarity between the pixel values in the neighborhood image block and the pixel values in the central image block.

Taking the calculation of the similarity between a certain neighborhood image block and the central image block as an example, since there is not only one pixel point in the neighborhood image block and the central image block, but multiple pixels points, it is necessary to determine the similarity between the neighborhood image block and the central image block based on the similarity between each pixel point in the neighborhood image block and the pixel point at the corresponding position in the central image block.

FIG3 is a second schematic diagram of each image block in the embodiment of the present application. Referring to FIG3 , when calculating the similarity between the neighborhood image block _A0 and the central image block _A4 , the neighborhood image block _A0 includes 9 pixels, namely, pixels _q0 , _q1 , _q2 , _q3 , _q4 , _q5 , _q6 , _q7 , _q8 , and the central image block _A4 also includes 9 pixels, namely, pixels _p0 , _p1 , _p2 , _p3 , _p4 , _p5 , _p6 , _p7 , _p8 . Therefore, the difference between the pixel values of pixel _q0 and pixel _p0 is calculated, the difference between the pixel values of pixel _q1 and pixel _p1 is calculated, and so on. By summing the squares of the differences and taking the average, we can get the similarity between the neighborhood image block _A0 and the central image block _A4 , that is, the distance between the neighborhood image block _A0 and the central image block _A4 . The method for calculating the similarity between the neighborhood image blocks _A1 , _A2 , _A3 , _A5 , _A6 , _A7 , _A8 and the central image block _A4 is the same as the calculation method of the neighborhood image block _A0 , which will not be repeated here.

The distance between the neighborhood image block and the center image block is calculated by the square of the Euclidean distance. The specific calculation formula is shown in the following formula (1):

Among them, d represents the Euclidean distance, p represents the central image block, q represents the neighborhood image block, and k represents the pixel point in the image block.

Of course, other distance calculation methods may also be used to calculate the distance between the neighborhood image block and the central image block. The specific distance calculation method used to calculate the distance between the neighborhood image block and the central image block is not limited here.

S103: Find out the weight corresponding to the distance from a preset table and use it as the weight of the neighborhood image block.

The preset table is used to represent the corresponding relationship between different distances and weights.

The fundamental purpose of calculating the distance between the neighborhood image block and the central image block is to determine the weight of the neighborhood image block relative to the central image block. Therefore, after calculating the distance between the neighborhood image block and the central image block, it is necessary to determine the weight of the neighborhood image block according to the distance between the neighborhood image block and the central image block.

In the prior art, based on the distance between the neighborhood image block and the central image block, a Gaussian function is used to calculate the weight of the neighborhood image block. The specific calculation formula is shown in the following formula (2):

Among them, w represents the weight, i represents the neighborhood image block, and d represents the distance between the neighborhood image block and the center image block. σ represents the standard deviation of the noise. For different image sensors, σ needs to be calibrated using standard images under different illumination. In other words, the σ in the above formula uses the σ corresponding to the image sensor through which the image to be processed is obtained. h represents the filter coefficient, which is positively correlated with σ, that is, h = kσ, where k is generally a coefficient between (0.3, 1).

From the above formula, we can know that the smaller the distance between the neighborhood image block and the central image block, the greater the weight of the neighborhood image block relative to the central image block. When the square of the distance between the neighborhood image block and the central image block is less than or equal to 2σ ² , the weight of the neighborhood image block relative to the central image block is 1. The weight corresponding to the central image block is also 1.

Since the prior art calculates the weight of the neighboring image blocks relative to the central image block one by one based on the distance between the neighboring image blocks and the central image block, which includes exponential calculation, the calculation process is complicated. Therefore, in the embodiment of the present application, the weight of the neighboring image is no longer calculated one by one based on the distance between the neighboring image blocks and the central image block, but the weight corresponding to the distance between the neighboring image block and the central image block is directly searched in a preset table that pre-stores the weights corresponding to various distances, and used as the weight of the neighboring image. Compared with exponential calculation, the calculation process of table lookup is simpler.

FIG4 is a schematic diagram of a preset table in an embodiment of the present application. Referring to FIG4 , the preset table stores a plurality of distances d ₁ , d ₂ , ..., d _n , and corresponding weights w ₁ , w ₂ , ..., w _n . After obtaining the distance d ₂ between the neighborhood image block and the center image block, the weight w ₂ of the neighborhood image block can be obtained by searching the preset table. This avoids repeated index calculations and simplifies the weight calculation process.

It should be noted that the weights corresponding to various distances in the preset table are pre-calculated and stored. The weights can be calculated by distance using various existing methods for calculating image block weights, such as Gaussian function, etc. The specific method for calculating the weights is not limited here.

S104: Filtering the pixels in the central image block based on the weights of the neighboring image blocks, the pixel values of the neighboring image blocks, the weight of the central image block, and the pixel values of the central image block.

After obtaining the weight of the neighborhood image block relative to the central image block, and combining the pixel value of the neighborhood image block, as well as the pixel value and weight (generally 1) of the central image block, the pixel value of the target pixel in the central image block can be obtained by weighted averaging, that is, filtering of the target pixel is achieved. The specific calculation formula is shown in the following formula (3):

Wherein, u'(A _n ) represents the pixel value of the central image block after filtering, u(A _i ) represents the pixel value of the central image block and all neighboring image blocks before filtering, and w _i represents the weights corresponding to the central image block and all neighboring image blocks. C=∑w _i is the weight normalization value.

As can be seen from the above content, the image processing method provided by the embodiment of the present application, after obtaining the central image block and the neighborhood image block in at least one area of the image, calculates the distance between the neighborhood image block and the central image block, and then finds the weight of the neighborhood image block from the preset table based on the distance between the neighborhood image block and the central image block, and finally filters the pixels in the central image block based on the weight of the neighborhood image block, the pixel value of the neighborhood image block, the weight of the central image block and the pixel value of the central image block. When determining the corresponding weight of each image block based on the distance between each image block, the weight of each image block is calculated one by one based on the Gaussian function, and a table lookup method is used to directly find the weight corresponding to the distance of each image block from the preset table. Since the table lookup is simpler than the function calculation method, it can simplify the complexity of the image denoising calculation, thereby reducing the cost of the hardware circuit.

Further, as a refinement and extension of the method shown in FIG1 , the embodiment of the present application also provides an image processing method. FIG5 is a second flow chart of the image processing method in the embodiment of the present application. Referring to FIG5 , the method may include:

S501: Determine a target area of an image.

When performing noise reduction on an image, since noise reduction is not performed directly on the entire image, but on each area in the image, it is necessary to first determine an area in the image to perform noise reduction on the area. When noise reduction is completed on all areas in the image, the noise reduction of the image is completed.

Specifically, the target area can be determined in the image by using a sliding window. Still referring to FIG. 2 , in the image X, area Y ₁ is a target area determined by using a sliding window. Of course, by continuing to move the sliding window, areas Y ₂ , Y ₃ , etc. can also be determined. The specific number of target areas can be determined according to the size of the image and the size of the sliding window, as long as all the determined target areas can cover the image. In order to process the image quickly and comprehensively, the target areas determined by using the sliding window may not overlap.

When the pixels that need to be filtered are located at the boundary of the image, that is, when the pixels to be processed are located at the center of the sliding window, and the sliding window is not completely located on the image, in order to continue to perform noise reduction processing on the image in the sliding window, the missing image in the sliding window can be filled according to the entire image to obtain the target area to be processed.

FIG6 is a schematic diagram of a sliding window that is not completely located in an image in an embodiment of the present application. Referring to FIG6 , an area 6011 in the sliding window 601 is located in the image X, while an area 6012 in the sliding window 601 is not located in the image X. In order to perform noise reduction processing on the target area in the sliding window 601, it is necessary to fill the missing image in the area 6012.

In practical applications, the area in the sliding window that is not on the image can be filled in a mirroring manner, that is, the image is flipped with the edge of the image in the sliding window as the flip axis, and the missing part in the sliding window is filled. Of course, other methods can also be used to fill the area in the sliding window that is not on the image. The specific filling method is not limited here.

S502: Acquire a central image block and a neighborhood image block in the target area.

The specific implementation method of step S502 is the same as that of step S101, which will not be repeated here.

S503: Determine whether the target area is a texture detail area; if so, execute S504; if not, it can be considered that the target area is a flat area, and execute S505.

Of course, in other embodiments, it may also be determined whether the target area is a flat area; if so, S505 is executed; if not, the target area may be considered to be a texture detail area, and S504 is executed.

Different types of regions in the image have different emphases when denoising. For texture detail regions, it is more necessary to retain the detail information in the image while denoising, so the denoising intensity should not be too large. For flat regions, there is not much detail information, so denoising can be focused on.

When determining whether the target area is a flat area or a texture detail area, still referring to FIG. 2 , the specific steps are as follows:

Step 1: Determine the pixel values of the central image block A ₄ and the area image blocks A ₀ , A ₁ , A ₂ , A ₃ , A ₅ , A ₆ , A ₇ , and A ₈ in the target area;

Step 2: Select the maximum pixel value max_val and the minimum pixel value min_val from the above 9 pixel values;

Step 3: Calculate the difference between the maximum pixel value max_val and the minimum pixel value min_val, diff=max_val-min_val;

Step 4: Compare the difference diff with the preset value diff_threshold; if the difference is less than the preset value, that is, diff＜diff_threshold, the target area is determined to be a flat area; if the difference is greater than or equal to the preset value, that is, diff≥diff_threshold, the target area is determined to be a texture detail area.

The above method for determining the type of the target region is simple and convenient. Of course, other methods can also be used to determine whether the target region is a flat region or a texture detail region, and the specific determination method is not limited here.

The central image block does not contain only one pixel, but generally contains multiple pixels. There are as many central pixels as there are pixels in the central image block. Similarly, the neighborhood image block does not contain only one pixel, but also contains multiple pixels, and the number and distribution of pixels in the neighborhood image block are generally the same as those in the central image block. Therefore, there are as many neighborhood pixels as there are pixels in the neighborhood image block.

S504: Calculate pixel differences between a plurality of central pixels and neighboring pixels at corresponding pixel positions.

That is to say, when the target area is a texture detail area, it is necessary to consider the image detail preservation. Therefore, when calculating the distance between the central image block and the neighboring image block, it is necessary to calculate the pixel difference between each central pixel point in the central image block and the neighboring pixel points at the corresponding pixel position in the neighboring image block. As for why the texture detail information in the central image block can be retained when the central image block is denoised by calculating the pixel difference between each central pixel point in the central image block and the neighboring pixel points at the corresponding pixel position in the neighboring image block, the specific reason will be explained later (in the Chebyshev formula).

FIG7 is a schematic diagram of a neighborhood image block and a central image block in an embodiment of the present application. Referring to FIG7 , in the neighborhood image block, there are 9 pixels, namely, neighborhood pixel points q ₀ , q ₁ , q ₂ , q ₃ , q ₄ , q ₅ , q ₆ , q ₇ , q ₈ . In the central image block, there are 9 pixels, namely, center pixel points p ₀ , p ₁ , p ₂ , p ₃ , p ₄ , p ₅ , p ₆ , p ₇ , p ₈ . When calculating the distance between the neighborhood image block and the central image block, since the target area in which the neighborhood image block is located is a texture detail area, it is necessary to calculate the pixel difference between each center pixel in the central image block and the neighborhood pixel at the corresponding pixel position in the neighborhood image block. That is, p ₀ -q ₀ , p ₁ -q ₁ , p ₂ -q ₂ , p ₃ -q ₃ , p ₄ -q ₄ , p ₅ -q ₅ , p ₆ -q ₆ , p ₇ -q ₇ , and p ₈ -q ₈ are calculated.

S505: Calculate pixel differences between N central pixels and neighboring pixels at corresponding pixel positions.

The value of N is smaller than the total number of central pixels in the central image block.

That is to say, when the target area is a flat area, there is no need to consider the image details too much, and the focus can be on denoising. Therefore, when calculating the distance between the central image block and the neighboring image block, it is sufficient to calculate the pixel difference between the finite number of central pixels in the central image block and the neighboring pixels at the corresponding pixel positions in the neighboring image blocks. As for why the central image block can be better denoised by calculating the pixel difference between the finite number of central pixels in the central image block and the neighboring pixels at the corresponding pixel positions in the neighboring image blocks, the specific reason will be explained later (in the Chebyshev formula).

Still referring to FIG. 7 , when calculating the distance between the neighborhood image block and the central image block, since the target area in which it is located is a flat area, the pixel difference between several central pixels in the central image block and the neighborhood pixels at the corresponding pixel positions in the neighborhood image block can be calculated. For example: calculate p ₁ -q ₁ , p ₃ -q ₃ , p ₄ -q ₄ , p ₅ -q ₅ , p ₇ -q ₇ .

There are many different options for selecting which central pixels to select from the central image block, which are not limited here.

FIG8 is a schematic diagram of several pixel position templates in an embodiment of the present application. Referring to FIG8 , an image block containing 9 pixels, i.e., 3×3, is taken as an example. In 8a, all pixels in the image block (pixels where the shadows are located) are selected to participate in the distance calculation. 8a is applicable to step S504. In 8b and 8c, some pixels in the image block (pixels where the shadows are located) are selected to participate in the distance calculation. 8b and 8c are applicable to step S505.

Of course, the positions of some of the pixels involved in the calculation are not limited to 8b and 8c, but may also be a combination of other positions, such as: pixels at four corners, 3 pixels in the first column, and so on.

S506: Determine a target pixel difference with the largest pixel difference from pixel differences between at least one central pixel point and neighboring pixel points at corresponding pixel positions, and use the target pixel difference as the distance between the neighboring image block and the central image block.

In the prior art, the Euclidean distance is used to calculate the distance between the neighborhood image block and the center image block. In the Euclidean distance, more multiplications are involved. For example, referring to Figures 2 and 3, in order to calculate the distance between the neighborhood image block _A0 and the center image block _A4 , it is necessary to calculate the Euclidean distance between the neighborhood pixel point _q0 and the center pixel point _p0 , the Euclidean distance between the neighborhood pixel point _q1 and the center pixel point _p1 , ..., the Euclidean distance between the neighborhood pixel point _q8 and the center pixel point _p8 , which involves 9 multiplication operations. In the target area _Y1 , there are a total of 8 domain image blocks, namely, the neighborhood image blocks _A0 , _A1 , _A2 , _A3 , _A5 , _A6 , _A7 , _A8 . In this way, a total of 9×8=72 multiplication operations are involved. This is just calculating the distance between the neighborhood image block and the central image block in a region of image X. There are multiple such regions in image X, and the amount of multiplication operations is huge.

In order to simplify the distance calculation, in the embodiment of the present application, the Euclidean distance is abandoned and the Chebyshev distance is used to approximate the Euclidean distance. That is to say, from the pixel differences between the multiple central pixels and the neighboring pixels of the corresponding pixel positions calculated above, a pixel difference with the largest value is selected as the distance between the neighboring image block and the central image block. The specific calculation formula is shown in the following formula (4):

d _Chebyshev = max _k | p _k - q _k | (4)

Where d _Chebyshev represents the Chebyshev distance, that is, the distance between the neighborhood image block and the central image block. p represents the central image block, q represents the neighborhood image block, and k represents the sequence number of the pixel in the image block.

Here, based on the calculation formula of Euclidean distance and Chebyshev distance, the relationship between the two can be obtained as shown in the following formula (5):

Where i represents an image block.

Assuming that the noise follows a Gaussian distribution, in the flat area of the image, most of the values of the 9 pixel differences |p _k -q _k | obtained above are much smaller than d _Chebyshev , which can be approximated as shown in the following equation (6):

In other words, the Chebyshev distance is used to approximate the Euclidean distance to determine the distance between the neighborhood image block and the central image block. The error is not large and does not reduce the accuracy of denoising.

Moreover, it can be known from the calculation formula of Chebyshev distance that the distance between image blocks depends on the pixel position with the largest pixel difference. The fewer pixels involved in the calculation, the greater the chance that the image block distance will obtain a smaller value, the larger the weight obtained, and the greater the noise reduction intensity. Therefore, in the flat area, some pixels in the image block can be selected for distance calculation, and the image block distance obtained will be relatively small, and the corresponding weight will be relatively large, thereby achieving a better noise reduction effect. In the detailed texture area, all pixels in the image block are selected for distance calculation, and the image block distance obtained will be relatively large, and the corresponding weight will be relatively small, thereby avoiding a large degree of noise reduction to ensure that the texture details in the image block are not lost. It can be seen that the image processing method provided by the embodiment of the present application can not only reduce the complexity of calculation, thereby reducing the cost of hardware circuits, but also can strike a balance between denoising and retaining texture information, thereby obtaining better image quality.

S507: Determine the index number of the neighborhood image block based on the distance between the neighborhood image block and the central image block.

Compared with the square of the Euclidean distance, the range of Chebyshev distance is much smaller, that is, the range of Chebyshev distance is limited. Therefore, all foreseeable Chebyshev distances can be calculated in advance, and then the weights corresponding to each Chebyshev distance can be calculated respectively, so that after obtaining the distance between the neighborhood image block and the central image block, the corresponding weight can be directly obtained based on the obtained distance to find the alternative index calculation.

In order to further reduce the length of the preset table, the corresponding relationship between the distance and the weight may not be saved in the preset table, but the corresponding relationship between the index number and the weight may be saved. After obtaining the distance between the neighborhood image block and the central image block, the index number corresponding to the distance is calculated by the index formula, and then the weight corresponding to the distance is found in the preset table according to the index number.

Specifically, after obtaining the distance between the neighborhood image block and the central image block, first, the distance between the neighborhood image block and the central image block is subtracted from the preset distance to obtain the distance difference. The preset distance is determined based on the noise standard deviation of the image sensor that acquires the image data and the noise reduction strength input by the user. Then, the distance difference is right-shifted according to the preset number of bits to obtain the index number of the neighborhood image block, wherein the preset number of bits is determined based on the noise standard deviation and the maximum length of the preset table. The index formula can be specifically shown as follows (7):

idx＝(d _Chebyshev - d _thr )＞＞rshift (7)

Among them, idx represents the index number, d _Chebyshev represents the Chebyshev distance, beta represents the noise reduction strength and can be input by the user. The user can adjust the noise reduction strength of the image by inputting beta. σ represents the standard deviation of the noise. >>rshift represents right shifting by a preset number of bits. Here, the bits of the pixel value are shifted right. rshift is used to reduce the value range of the index and can be determined based on the noise standard deviation and the maximum length of the preset table.

For example, assuming that d _thr = 110010, rshift = 4. After determining that d _Chebyshev = 1100100, according to the index formula, 1100100-110010=110010, 110010 is shifted right by 4 bits to obtain 11. 11 is the index number corresponding to 1100100.

S508: Find out the weight corresponding to the index number of the neighborhood image block from a preset table.

FIG9 is a second schematic diagram of a preset table in an embodiment of the present application. Referring to FIG9 , the preset table stores index numbers idx ₁ , idx ₂ , ..., idx _n and their corresponding weights w ₁ , w ₂ , ..., w _n . After obtaining the index number idx ₂ of the neighborhood image block, the weight of the neighborhood image block can be found to be w ₂ through the preset table.

Since the index numbers and their corresponding weights saved in the preset table are calculated in advance, before performing noise reduction on the image, it is necessary to prepare the weights in the preset table. The specific steps for calculating the weights are as follows:

Step 1: Use Gaussian function to calculate the initial weights corresponding to different distances.

Step 2: Adjust the initial weight based on the noise reduction strength input by the user, obtain the weights corresponding to different distances, and store them in a preset table.

In fact, when calculating the weight corresponding to the distance, the above steps 1 and 2 can be performed simultaneously. That is, the noise reduction strength is introduced into the Gaussian function, and then the weight corresponding to each distance is calculated by introducing the Gaussian function of the noise reduction strength. The specific calculation formula is shown in the following formula (8):

Where w represents the weight, i represents the neighborhood image block, d _Chebyshev represents the Chebyshev distance between the neighborhood image block and the central image block, beta represents the noise reduction strength input by the user, σ represents the standard deviation of the noise, and h represents the filter coefficient.

S509: Filtering the pixels in the central image block based on the weights of the neighboring image blocks, the pixel values of the neighboring image blocks, the weight of the central image block, and the pixel values of the central image block.

After obtaining the weights of each neighborhood image block, the pixel values of each neighborhood image block, the weight of the center image block (default is 1) and the pixel value of the center image block are combined, and a weighted average calculation method is adopted to filter the center pixel of the center image block. The specific calculation method has been described in detail in the above step S104 and will not be repeated here.

The above are all explanations of filtering processing for the central pixel point of the central image block, that is, the pixel point p ₄ in Figure 3. However, the image processing method provided in the embodiment of the present application does not only perform filtering processing on the central pixel point of the central image block, but can also perform filtering processing on the pixel point at any position in the central image block, and can even perform filtering processing on multiple pixel points in the central image block at the same time. The overall processing process can be referred to the above steps S501-S509. Only the differences are described below.

FIG10 is a third schematic diagram of each image block in an embodiment of the present application. Referring to FIG10 , the size of the sliding window is 7×6. The sliding window includes pixel points A ₀ , B ₀ , A ₁ , B ₁ , A ₂ , B ₂ , A ₃ , B ₃ , A ₄ , B ₄ , A ₅ , B 5 , A ₆ , B ₆ , A ₇ , B ₇ , A ₈ , _{B 8 and} surrounding unmarked pixel points.

FIG11 is a schematic diagram of each image block in an embodiment of the present application. Referring to FIG11 , when the image in the sliding window of FIG10 needs to be divided into 3×3 image blocks, the size of each image block is 2×3. Among them, each image block has no overlap in the horizontal direction and has an overlap of 1 pixel in the vertical direction. In this way, A ₀ , B ₀ and the 4 pixels above and below it constitute an image block, and so on, a total of 9 image blocks. Among the 9 image blocks here, A ₄ , B ₄ and the 4 pixels above and below it constitute the central image block, which can be referred to as the central image block A ₄ B ₄ for convenience. Correspondingly, the aforementioned central image block A ₄ does not only refer to the existence of only one pixel A ₄ in the central image block, but also includes the 8 pixels around it. And A ₀ , B ₀ and the 4 pixels above and below it, A ₁ , B ₁ and the 4 pixels above and below it, ..., A ₈ , B ₈ and the 4 pixels above and below it constitute 8 corresponding neighborhood image blocks.

In this sliding window, A ₄ and B ₄ can be filtered at the same time. The specific calculation formulas are shown in the following equations (9) and (10):

Among them, u( _A4 ) and u( _B4 ) represent the pixel values of _A4 and _B4 after filtering respectively, _Ai and _Bi represent the pixel values of all neighborhood image blocks of _A4 and _B4 respectively, w _i represents the weights of all neighborhood image blocks relative to _A4 and _B4 , _Ai and _Bi use the same weights, which can save calculation amount, C = _∑wi is the weight normalization value.

FIG12 is a second schematic diagram of several pixel position templates in an embodiment of the present application, as shown in FIG12 . Since the pixel blocks divided in FIG11 are not 3×3 image blocks, but 2×3 image blocks, after determining that the target area belongs to a flat area, when calculating the distance between the neighborhood image block and the central image block in the target area, the pixel position template used will change slightly, but the calculation idea remains unchanged, and some pixels in the image block are still used to participate in the distance calculation. In 12a, all pixels in the image block (pixels where the shadow is located) are selected to participate in the distance calculation. 12a is suitable for distance calculation of image blocks in detail texture areas. In 12b and 12c, some pixels in the image block (pixels where the shadow is located) are selected to participate in the distance calculation. 12b and 12c are suitable for distance calculation of image blocks in flat areas.

Based on the same inventive concept, as an implementation of the above method, the present application embodiment further provides an image processing device. FIG13 is a structural schematic diagram of an image processing device in an embodiment of the present application. Referring to FIG13 , the device may include:

The receiving module 1301 is configured to obtain a central image block and a neighborhood image block in at least one region of an image, wherein the neighborhood image block is adjacent to the central image block.

The calculation module 1302 is used to calculate the distance between the neighborhood image block and the central image block.

The search module 1303 is used to search for the weight corresponding to the distance from a preset table and use it as the weight of the neighborhood image block. The preset table is used to characterize the corresponding relationship between different distances and weights.

The filtering module 1304 is used to perform filtering processing on the pixels in the central image block based on the weights of the neighborhood image blocks, the pixel values of the neighborhood image blocks, the weight of the central image block, and the pixel values of the central image block.

Further, as a refinement and extension of the device shown in FIG13, the embodiment of the present application further provides an image processing device. FIG14 is a second structural schematic diagram of the image processing device in the embodiment of the present application. Referring to FIG14, the device may include:

The storage module 1401 includes:

The first calculation unit 1401a is used to calculate initial weights corresponding to different distances by using a Gaussian function.

The storage unit 1401b is used to adjust the initial weight based on the noise reduction strength input by the user, obtain weights corresponding to different distances, and store them in the preset table.

The first determining module 1402 includes:

The sliding window unit 1402a is used to determine the target area in the image using a sliding window.

The filling unit 1402b is used to fill the area of the target area outside the image based on the image when the target area is not completely located in the image, so as to obtain the at least one filled area.

The receiving module 1403 is configured to obtain a central image block and a neighborhood image block in at least one region of an image, wherein the neighborhood image block is adjacent to the central image block.

The central image block includes a plurality of central pixel points, and the neighborhood image block includes a plurality of neighborhood pixel points.

The second determining module 1404 includes:

The first determining unit 1404a is configured to determine the pixel value of the central image block and the pixel value of the neighborhood image block.

The selection unit 1404b is used to select the maximum pixel value and the minimum pixel value from the pixel values of the central image block and the pixel values of the neighborhood image blocks.

The second calculating unit 1404c is used to calculate the difference between the maximum pixel value and the minimum pixel value.

The second determining unit 1404d is configured to determine that the at least one region is a flat region when the difference is less than a preset value; and to determine that the at least one region is a detail texture region when the difference is greater than or equal to the preset value.

The computing module 1405 includes:

The third calculation unit 1405a is used to calculate the pixel difference between at least one central pixel point and the neighboring pixel points at the corresponding pixel position.

The third calculation unit 1405a is specifically used to calculate the pixel differences between N central pixels and the neighborhood pixels of the corresponding pixel positions when the at least one area is a flat area, and the value of N is less than the total number of central pixels in the central image block; when the at least one area is a texture detail area, calculate the pixel differences between the multiple central pixels and the neighborhood pixels of the corresponding pixel positions.

The third determining unit 1405b is used to determine a target pixel difference with the largest pixel difference from the pixel differences between the at least one central pixel point and the neighborhood pixel points at the corresponding pixel position, and use the target pixel difference as the distance between the neighborhood image block and the central image block.

The preset table contains a correspondence between multiple index numbers and weights, and one index number in the preset table corresponds to at least one distance.

The search module 1406 includes:

The fourth determining unit 1406a is configured to determine the index number of the neighborhood image block based on the distance between the neighborhood image block and the central image block.

The fourth determination unit 1406a is specifically used to subtract the distance between the neighborhood image block and the central image block from a preset distance to obtain a distance difference, where the preset distance is determined based on the noise standard deviation of the image sensor that obtains the image data and the noise reduction strength input by the user; right-shift the distance difference according to a preset number of bits to obtain an index number of the neighborhood image block, where the preset number of bits is determined based on the noise standard deviation and the maximum length of the preset table.

The searching unit 1406b is used to search the preset table for the weight corresponding to the index number of the neighborhood image block.

The filtering module 1407 is used to perform filtering processing on the pixels in the central image block based on the weights of the neighborhood image blocks, the pixel values of the neighborhood image blocks, the weight of the central image block, and the pixel values of the central image block.

In addition, FIG14 shows the signal flows between the modules and the units in the modules.

It should be noted here that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

Based on the same inventive concept, the embodiment of the present application also provides an electronic device. FIG15 is a schematic diagram of the structure of an electronic device in the embodiment of the present application. Referring to FIG15 , the electronic device may include: a processor 1501, a memory 1502, and a bus 1503; wherein the processor 1501 and the memory 1502 communicate with each other through the bus 1503; the processor 1501 is used to call the program instructions in the memory 1502 to execute the method in one or more of the above embodiments.

It should be noted that the description of the above electronic device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the electronic device embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium, which may include: a stored program; wherein, when the program is running, the device where the storage medium is located is controlled to execute the method in one or more of the above embodiments.

It should be noted here that the description of the above storage medium embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the storage medium embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims (9)

1. An image processing method, the method comprising:

acquiring a central image block and a neighborhood image block in at least one region of an image, wherein the neighborhood image block is adjacent to the central image block;

calculating the distance between the neighborhood image block and the central image block;

the weight corresponding to the distance is found out from a preset table and used as the weight of the neighborhood image block, and the preset table is used for representing the corresponding relation between different distances and weights;

filtering pixels in the center image block based on the weight of the neighborhood image block, the pixel value of the neighborhood image block, the weight of the center image block and the pixel value of the center image block;

The central image block comprises a plurality of central pixel points, and the neighborhood image block comprises a plurality of neighborhood pixel points; the calculating the distance between the neighborhood image block and the center image block comprises:

determining pixel values of the central image block and pixel values of the neighborhood image blocks;

Selecting a maximum pixel value and a minimum pixel value from the pixel value of the central image block and the pixel value of the neighborhood image block;

calculating a difference between the maximum pixel value and the minimum pixel value;

when the difference value is smaller than a preset value, determining that the at least one area is a flat area;

When the difference value is greater than or equal to a preset value, determining the at least one area as a detail texture area;

when the at least one area is a flat area, calculating pixel differences between N central pixel points and neighborhood pixel points corresponding to pixel positions, wherein the value of N is smaller than the total number of the central pixel points in the central image block;

when the at least one region is a texture detail region, calculating pixel differences between the plurality of center pixel points and neighborhood pixel points corresponding to pixel positions;

And determining the distance between the neighborhood image block and the central image block according to the calculated pixel difference.

2. The method of claim 1, wherein the preset table includes a plurality of index numbers corresponding to the weights, and one index number in the preset table corresponds to at least one distance; the searching the weight corresponding to the distance from the preset table comprises the following steps:

determining an index number of the neighborhood image block based on a distance between the neighborhood image block and the center image block;

and searching the weight corresponding to the index number of the neighborhood image block from the preset table.

3. The method of claim 2, wherein the determining the index number of the neighborhood image block based on the distance of the neighborhood image block from the center image block comprises:

Subtracting the distance between the neighborhood image block and the central image block from a preset distance to obtain a distance difference, wherein the preset distance is determined based on the noise standard deviation of an image sensor for acquiring the image data and the noise reduction intensity input by a user;

And right shifting the distance difference according to a preset bit number to obtain an index number of the neighborhood image block, wherein the preset bit number is determined based on the noise standard deviation and the maximum length of the preset table.

4. The method of claim 1, wherein before the finding the weight corresponding to the distance from the preset table, the method further comprises:

Calculating initial weights corresponding to different distances by adopting a Gaussian function;

And adjusting the initial weight based on the noise reduction intensity input by the user, obtaining weights corresponding to different distances, and storing the weights in the preset table.

5. The method of any one of claims 1 to 4, wherein the central image block comprises a plurality of central pixel points and the neighborhood image block comprises a plurality of neighborhood pixel points; the calculating the distance between the neighborhood image block and the center image block comprises:

calculating pixel differences between at least one central pixel point and the neighborhood pixel points corresponding to the pixel positions;

And determining a target pixel difference with the largest pixel difference from the pixel differences of the at least one central pixel point and the neighborhood pixel points of the corresponding pixel positions, and taking the target pixel difference as the distance between the neighborhood image block and the central image block.

6. The method of any of claims 1 to 4, wherein prior to the center image block and the neighborhood image block in at least one region of the acquired image, the method further comprises:

a sliding window is adopted in the image to determine a target area;

And when the target area is not completely positioned in the image, filling the area, positioned outside the image, in the target area based on the image, so as to obtain the at least one filled area.

7. An image processing apparatus, characterized in that the apparatus comprises:

A receiving module, configured to acquire a central image block and a neighborhood image block in at least one region of an image, where the neighborhood image block is adjacent to the central image block;

the calculating module is used for calculating the distance between the neighborhood image block and the center image block;

The searching module is used for searching the weight corresponding to the distance from a preset table and taking the weight as the weight of the neighborhood image block, and the preset table is used for representing the corresponding relation between different distances and weights;

The filtering module is used for carrying out filtering processing on pixels in the central image block based on the weight of the neighborhood image block, the pixel value of the neighborhood image block, the weight of the central image block and the pixel value of the central image block;

The central image block comprises a plurality of central pixel points, and the neighborhood image block comprises a plurality of neighborhood pixel points;

A second determination module comprising:

a first determining unit, configured to determine a pixel value of the center image block and a pixel value of the neighborhood image block;

A selecting unit, configured to select a maximum pixel value and a minimum pixel value from the pixel value of the center image block and the pixel value of the neighborhood image block;

A second calculation unit configured to calculate a difference value between the maximum pixel value and the minimum pixel value;

a second determining unit configured to determine that the at least one area is a flat area when the difference value is smaller than a preset value; when the difference value is greater than or equal to a preset value, determining the at least one area as a detail texture area;

A computing module, comprising:

A third calculation unit, configured to calculate, when the at least one area is a flat area, pixel differences between N central pixel points and neighboring pixel points corresponding to pixel positions, where a value of N is smaller than a total number of central pixel points in the central image block; when the at least one region is a texture detail region, calculating pixel differences between the plurality of center pixel points and neighborhood pixel points corresponding to pixel positions; and determining the distance between the neighborhood image block and the central image block according to the calculated pixel difference.

8. An electronic device, comprising: a processor, a memory, a bus;

The processor and the memory complete communication with each other through the bus; the processor is configured to invoke program instructions in the memory to perform the method of any of claims 1 to 6.

9. A computer-readable storage medium, comprising: a stored program; wherein the program, when run, controls a device in which the storage medium is located to perform the method of any one of claims 1 to 6.