CN107317959B - Image filtering device and image filtering method thereof - Google Patents

Abstract

An image filtering device is used to filter an image, and comprises: a pixel difference calculation module, an adaptive brightness adjustment module, a weight calculation module and a filter calculation module. The pixel difference calculation module uses any pixel as the central pixel in the pixel window and calculates the absolute pixel difference with all pixels in the pixel window. The adaptive brightness adjustment module multiplies the absolute pixel difference by an adjustment parameter to generate an adjusted absolute pixel difference. The weight calculation module generates a weight value according to the adjusted absolute pixel difference. The filter calculation module convolves the pixel value of each pixel in the pixel window with the corresponding weight value to generate a filter result of the central pixel.

Description

Image filtering device and image filtering method therefor

technical field

The present invention relates to an image processing technology, and more particularly, to an image filtering device and an image filtering method thereof.

Background technique

In the process of image data processing, a filter of a specific kernel (kernel) is often required to filter the input image to generate an optimized output image. However, when the noise intensity of the image is very high, the effect of edge preservation filter is often not ideal. Because the value of the center pixel is usually not representative, the filter weights dynamically generated by the edge-preserving filter are not appropriate. Moreover, under the same numerical difference, the general filter often generates the same set of weights, which cannot reflect the human eye's perception of different environmental brightness. Furthermore, the computational complexity of the general edge preserving filter is very high, which is very unfavorable for hardware implementation.

Therefore, how to design a new image filtering device and an image filtering method to solve the above problems is an urgent problem to be solved in the industry.

SUMMARY OF THE INVENTION

Therefore, an aspect of the present invention is to provide an image filtering apparatus for filtering an image, wherein the image includes a plurality of pixels (pixels), each of which has a pixel value. The image filtering device includes: a pixel difference calculation module, an adaptive brightness adjustment module, a weight calculation module and a filter calculation module. The pixel difference calculation module is configured so that any pixel is used as the center pixel in the pixel window, and the absolute difference value of the complex number of pixels is calculated according to the pixel value of the center pixel and each pixel value of all the pixels in the pixel window. The adaptive brightness adjustment module is configured to multiply the pixel absolute difference by the adjustment parameter to generate a corresponding complex adjusted pixel absolute difference. When the reference brightness of a pair of pixels corresponding to each pixel absolute difference is smaller, the adjustment parameter is larger. , when the reference brightness of a pair of pixels corresponding to the absolute difference of each pixel is larger, the adjustment parameter is smaller. The weight calculation module is configured to generate a corresponding complex weight value according to the adjusted absolute pixel difference value. When the size of the adjusted pixel absolute difference value is smaller, the generated weight value is larger. When the value is larger, the generated weight value is smaller. The filtering calculation module is configured to perform convolution between the pixel value of each pixel in the pixel window and the corresponding weight value, so as to generate the filtering result of the central pixel.

Another aspect of the present invention is to provide an image filtering method for filtering an image from a camera device and including a plurality of pixels, the image filtering method comprising: making any pixel as a center pixel in a pixel window, and Calculate the absolute difference value of complex pixels according to the pixel value of the central pixel and each pixel value of all pixels in the pixel window; multiply the absolute pixel difference value by the adjustment parameter to generate the corresponding complex adjusted pixel absolute difference value. The smaller the reference brightness of a pair of pixels corresponding to the difference value is, the larger the adjustment parameter is. When the reference brightness of a pair of pixels corresponding to the absolute difference of each pixel is larger, the adjustment parameter is smaller; The complex weight value of , when the size of the adjusted pixel absolute difference is smaller, the generated weight value is larger, when the adjusted pixel absolute difference value is larger, the generated weight value is smaller; The pixel value of each pixel is convolved with the corresponding weight value to generate the filtering result of the central pixel.

The advantage of applying the present invention is that the image filtering device of the present invention can calculate the absolute pixel difference value by the pixel difference calculation module, and adjust the pixel absolute difference value by the adaptive brightness adjustment module. Different content of the image generates weight values. On the one hand, the absolute difference of the adjusted pixels can reflect the human eye's perception of brightness, so as to improve the filtering quality and achieve the purpose of edge preservation, which greatly reduces the complexity of calculation and hardware implementation.

Description of drawings

FIG. 1 is a block diagram of an image filtering apparatus according to an embodiment of the present invention;

2A is a schematic diagram of an image according to an embodiment of the present invention;

2B is a schematic diagram of an image after preprocessing in an embodiment of the present invention;

2C is a schematic diagram of a pixel absolute difference calculation result generated by a pixel difference calculation module according to pre-processed pixel calculation according to an embodiment of the present invention;

2D is a schematic diagram of a calculation result of the adjusted absolute pixel difference value generated by the adaptive brightness adjustment module according to the calculation of the pixel absolute difference value according to an embodiment of the present invention;

2E is a schematic diagram of a weight value calculation result generated by the weight calculation module according to the adjusted absolute pixel difference according to an embodiment of the present invention; and

FIG. 3 is a flowchart of an image filtering method according to an embodiment of the present invention.

Symbol Description

1: Image filtering device 100: Pre-processing module

101, 101’: Image 102: Pixel Difference Calculation Module

103: Calculation result of pixel absolute difference 104: Adaptive brightness adjustment module

105: Calculation of absolute pixel difference after adjustment 106: Weight calculation module

Result 107: Weight value calculation result

108: Filter calculation module 109: Filter result

2: Camera device 300: Image filtering method

301-305: Steps

Detailed ways

Please refer to Figure 1. FIG. 1 is a block diagram of an image filtering apparatus 1 according to an embodiment of the present invention. The image filtering device 1 can be used for filtering an image 101 from the camera device 2 and comprising a plurality of pixels to generate a filtered image. Wherein, each pixel has a pixel value, and the pixel value can be, for example, but not limited to, the Y value in the YUV color space, or the red pixel value, the green pixel value and the blue pixel value in the RGB color space converted into The Y value in the YUV color space is generated.

In this embodiment, the image filtering device 1 includes: a preprocessing module 100 , a pixel difference calculation module 102 , an adaptive brightness adjustment module 104 , a weight calculation module 106 and a filter calculation module 108 .

In different embodiments, the camera device 2 may be, for example, but not limited to, a digital camera device including a charge-coupled device (CCD) or a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor (CMOS) photosensitive element), to Image 101 is generated.

The preprocessing module 100 is configured to receive the image 101 of the camera device 2 and perform preprocessing on the image 101 according to the noise characteristics of the camera device 2 . The noise characteristics of the camera device 2 can be learned through training. In one embodiment, the noise characteristic of the camera device 2 is Gaussian or random. At this time, the preprocessing performed by the preprocessing module 100 may be, for example, but not limited to, Gaussian filtering. In another embodiment, the noise characteristic of the camera device 2 is in the form of an impulse. At this time, the preprocessing performed by the preprocessing module 100 is, for example, but not limited to median filtering.

Please also refer to FIG. 2A . FIG. 2A is a schematic diagram of an image 101 according to an embodiment of the present invention. It should be noted that, in FIG. 2A , 25 pixels are exemplarily shown. However, the number of pixels of the image 101 in the present invention is not limited to this.

In one embodiment, the pre-processing performed on any pixel in the image 101 takes the pixel as the center pixel, and performs operations on the surrounding pixels, such as, but not limited to, the pixels included in the pixel window of the size of 3×3. .

For example, image 101 in FIG. 2A includes pixels P ₀₀ -P ₂₂ . The pixels P ₀₀ -P ₂₂ form a pixel window with the pixel P ₁₁ as the center pixel. The pixel values of the pixels P ₀₀ -P ₂₂ are, for example, but not limited to, 200, 220, 240, 150, 70, 140, 180, 100, and 120, respectively.

When the current processing is Gaussian filtering, the pre-processing module 100 averages or weights the pixel values of the pixels P ₀₀ -P ₂₂ , and the generated value is the pre-processed pixel value of the pixel P ₁₁ . Taking the averaging operation as an example, if the pre-processing module 100 directly averages the pixel values of the pixels P ₀₀ -P ₂₂ , the generated value 157 will be the pre-processed pixel value of the pixel P ₁₁ .

When the current processing is median filtering, the pre-processing module 100 will take the median of the pixel values of the pixels P ₀₀ -P ₂₂ , and the median will be the pre-processed pixel value of the pixel P ₁₁ . Among them, median filtering can effectively remove the spurious noise that cannot be removed by Gaussian filtering.

In one embodiment, the pre-processing module 100 may be implemented by a circuit (not shown) such as, but not limited to, a comparison network. The comparison network circuit can simultaneously compare the pixels P ₀₀ -P ₂₂ in two groups in parallel, so that the median is found under a fixed number of comparisons, which is beneficial to parallel processing on the hardware.

Please also refer to FIG. 2B . FIG. 2B is a schematic diagram of an image 101' after preprocessing according to an embodiment of the present invention.

Since the median of pixels P ₀₀ -P ₂₂ is 150, the pixel value of pixel P ₁₁ in image 101 ′ will be 150 after median filtering. Similarly, pixels P ₀₀ -P ₁₀ and P ₁₂ -P ₂₂ can also be median-filtered according to their surrounding pixels, respectively. In this embodiment, the pixel values of the pixels P ₀₀ -P ₁₀ and P ₁₂ -P ₂₂ are respectively 200, 220, 220, 150, 130, 120, 120 and 110 after preprocessing.

The pixel difference calculation module 102 receives the pre-processed image 101 ′, makes any pixel P ₀₀ -P ₂₂ as the central pixel in the pixel window, and according to the pixel value of the central pixel and each pixel of all the pixels in the pixel window value to calculate pixel absolute difference. For example, when the pixel P ₁₁ is used as the center pixel and is calculated with the pixels in the 3×3 pixel window, the pixel absolute difference calculation result 103 can be obtained, including, for example, but not limited to, a plurality of pixel absolute difference values. Difference D ₀₀ -D ₂₂ . Each pixel absolute difference value D ₀₀ -D ₂₂ is the absolute difference value between the central pixel P ₁₁ in the pixel window and each pixel P ₀₀ -P ₂₂ in the pixel window, respectively.

Please also refer to FIG. 2C. FIG. 2C is a schematic diagram of the pixel absolute difference calculation result 103 generated by the pixel difference calculation module 102 according to the pre-processed pixel calculation according to an embodiment of the present invention.

In this embodiment, the pixel absolute difference values D ₀₀ -D ₂₂ are the absolute values of the differences between the central pixel P ₁₁ and the pixels P ₀₀ -P ₂₂ in the pixel window. Therefore, according to the pre-processed pixel values in FIG. 2B , the absolute pixel differences D ₀₀ -D ₂₂ will be 50, 70, 80, 0, 0, 20, 30, 30, and 40, respectively. It should be noted that, in different embodiments, the pixel difference calculation module 102 may be in a one-norm (or Manhattan distance) or two-norm (or Euclidean distance) manner The pixel absolute difference values D ₀₀ -D ₂₂ are calculated to calculate the pixel absolute difference values, and are not limited to the above-mentioned calculation methods.

Next, the adaptive brightness adjustment module 104 multiplies the pixel absolute difference values D ₀₀ -D ₂₂ by the adjustment parameters to generate a corresponding adjusted pixel absolute difference value calculation result 105 , including, for example, but not limited to, a plurality of adjusted pixel absolute difference values. Difference D ₀₀ '-D ₂₂ '.

Please also refer to Figure 2D. FIG. 2D is a schematic diagram of a calculation result 105 of the adjusted pixel absolute difference calculated by the adaptive brightness adjustment module 104 according to the pixel absolute difference D ₀₀ -D ₂₂ according to an embodiment of the present invention.

In one embodiment, when the reference luminance of a pair of pixels corresponding to the absolute pixel difference values D ₀₀ -D ₂₂ is smaller, the adjustment parameter is larger. When the reference brightness of the pixel corresponding to the pixel absolute difference is larger, the adjustment parameter is smaller. In one embodiment, the smallest one can be selected from the two pixel values of the pair of pixels as the reference luminance. Also, the adjustment parameter may be determined by a function related to the reference luminance.

For example, if the value of a pixel is between 0 and 255, for a pair of pixels P _i,j and P _i+m,j+n , the corresponding adjustment parameter can be a power of two, For example, 2 ^K , and the size of this power K is determined by functions such as, but not limited to, the following:

Wherein, F ₁ [0]=0, F ₁ [1]=1, F ₁ [2]=1, F ₁ [3]=2, F ₁ [4]=2, F ₁ [5]=3, F ₁ [6]=3, F ₁ [7]=3.

Taking a pair of pixels P ₀₀ and P ₁₁ corresponding to the pixel absolute difference value D ₀₀ as an example, the reference luminance is the smallest pixel value between them, that is, 150. The power K corresponding to the adjustment parameter will be calculated by the following formula:

Therefore, the adjusted pixel absolute difference value D ₀₀ ′ will be the multiplication result of the pixel absolute difference value D ₀₀ and the adjustment parameter 2 ^K , that is, D ₀₀ ′=D ₀₀ ×2 ^K =D ₀₀ ×2 ² =50×4 = 200. In one embodiment, if the calculated absolute difference value of the adjusted pixel exceeds 255, it will directly take the value of 255.

The adjusted absolute pixel difference values D ₀₁ ′-D ₂₂ ′ can be calculated in the same way, and are 255, 255, 0, 0, 80, 120, 120 and 160, respectively.

The above method is equivalent to dividing the size of the pixel value into eight intervals from 0 to 31, 32 to 63, ..., 224 to 255, so as to reflect the brightness of the pixel, and according to the brightness of the pixel, the absolute difference of the corresponding pixel is calculated. The size of the value is adjusted to different degrees. In general, the human eye is more sensitive to changes in brightness in areas of low brightness. Therefore, by the above method, the absolute pixel difference value of the area with low brightness can be adjusted to a large extent, and the absolute difference value of the pixel in the area with high brightness can be adjusted only slightly, so as to meet the needs of human vision. . In addition, the adjustment parameter realized by the power of 2 will make the absolute difference value of the pixel directly shift the bit to the left to obtain the absolute difference value of the pixel after adjustment, which greatly saves the computational complexity. When the adaptive brightness adjustment module 104 is implemented by hardware, there is no need to set up a large number of multipliers, which can greatly save hardware cost and reduce operation time.

It should be noted that the above-mentioned calculation method of the adjustment parameter is only an example. In other embodiments, the reference luminance may be determined by, for example, but not limited to, the average pixel value or the weighted average pixel value of a corresponding pair of pixels. Moreover, the design of the function may also vary according to actual needs, and is not limited by the above examples.

Next, the weight calculation module 106 determines the size of the adjusted pixel absolute difference value D ₀₀ ′-D ₂₂ ′ to generate a corresponding weight value calculation result 107 , including multiple weight values such as, but not limited to, multiple weight values W ₀₀ ′-W ₂₂ .

Please also refer to FIG. 2E. 2E is a schematic diagram of a weight value calculation result 107 generated by the weight calculation module 106 according to the adjusted absolute pixel difference values D ₀₀ ′-D ₂₂ ′ according to an embodiment of the present invention.

In one embodiment, when the size of the adjusted absolute pixel difference value D ₀₀ ′-D ₂₂ ′ is smaller, the weight calculation module 106 generates a larger weight value. When the adjusted pixel absolute difference value D ₀₀ ′-D ₂₂ ′ is smaller When the size of ' is larger, the weight calculation module 106 generates a smaller weight value. In one embodiment, the weight values are respectively powers of two. In other embodiments, the weight value can also be generated in other ways, which is not limited by the above-mentioned embodiments.

For example, if the value of an adjusted pixel absolute difference value D ₀₀ ′ is between 0 and 255, the corresponding weight value W ₀₀ can be, for example, 2 ^S , and the size of the power S is determined by, for example, but Not limited to the following functions:

Wherein, F ₂ [0]=0, F ₂ [1]=1, F ₂ [2]=2, F ₂ [3]=3, F ₂ [4]=4, F ₂ [5]=5, F ₂ [6]=6, F ₂ [7]=7.

Taking the absolute difference value D ₀₀ ′ corresponding to the adjusted pixel as an example, the corresponding weight value W ₀₀ will be calculated by the following formula:

Therefore, the weight value W ₀₀ will be 2 ^S =2 ¹ =2.

The weight values W ₀₁ -W ₂₂ can be calculated in the same way, and are respectively 2 ⁰ , 2 ⁰ , 2 ⁷ , 2 ⁷ , 2 ⁵ , 2 ⁴ , 2 ⁴ and 2 ² .

The above method is equivalent to dividing the size of the adjusted pixel absolute difference into eight intervals of 0-31, 32-63, . . . , 224-255, and generating different weight values according to the adjusted pixel absolute difference. It should be noted that the above calculation method of the weight value is only an example. In other embodiments, the design of the function may vary according to actual requirements, and is not limited by the above examples.

Next, the filter calculation module 108 performs calculation according to the image 101 ′ and the weight value calculation result 107 . More specifically, the filter calculation module 108 convolves each pixel in the pixel window with the corresponding weight value to generate the filter result 109 of the center pixel. Taking the aforementioned pixel P ₁₁ as an example of the central pixel, the result of convolving each pixel P ₀₀ -P ₂₂ with the corresponding weight value is:

(200×2 ¹ +220×2 ⁰ +240×2 ⁰ +150×2 ⁷ +70×2 ⁷ +140×2 ⁵ +180×2 ⁴ +100×2 ⁴ +120×2 ² )/(2 ¹ +2 ⁰ +2 ⁰ +2 ⁷ +2 ⁷ +2 ⁵ +2 ⁴ +2 ⁴ +2 ² )=38460/328=117

Therefore, the filtered result of 109, ie the filtered value of pixel P ₁₁ , will be 117.

Since the weight value is realized by a power of 2, the calculation of the filtering result FR does not require a large number of multiplication operations, but is replaced by a bit left shift, and finally only one division is required, which greatly saves the computational complexity. When the filtering calculation module 108 is implemented by hardware, there is no need to set up a large number of multipliers, which can greatly save hardware cost and reduce operation time.

It should be noted that, for each image 101', each pixel must undergo the above-mentioned operation process to complete the filtering of each pixel.

The image filtering device 1 of the present invention can calculate the absolute pixel difference value by the pixel difference calculation module 102, and adjust the pixel absolute difference value by the adaptive brightness adjustment module 104. On the one hand, the weight calculation module 106 can dynamically target the image 101. Different content generates weight values. On the one hand, the absolute difference of the adjusted pixels can reflect the perception ability of the human eye for brightness, so as to improve the filtering quality and achieve the purpose of edge preservation. Further, by generating the weight value of the power of two by the weight calculation module 106, the complexity of the filtering process in calculation and hardware implementation can be greatly reduced. Furthermore, with the preprocessing mechanism of the preprocessing module 100 , the image filtering device 1 can remove noise in an appropriate manner according to the noise characteristics of the camera device 2 , thereby reducing the degree of influence of different types of noise on the image.

Please refer to Figure 3. FIG. 3 is a flowchart of an image filtering method 300 according to an embodiment of the present invention. The image filtering method 300 can be applied to, for example, but not limited to, the image filtering apparatus 1 shown in FIG. 1 . The image filtering method 300 includes the following steps (it should be understood that the steps mentioned in this embodiment, unless the sequence is specifically stated, can be adjusted according to actual needs, and can even be executed simultaneously or partially simultaneously) .

In step 301 , the image 101 is pre-processed by the pre-processing module 100 according to the noise characteristics of the camera device 2 .

In step 302, the pixel difference calculation module 102 makes any pixel in the image 101, such as pixel _P11 in FIG. 2A, as the center pixel in the pixel window, and according to the pixel value of the center pixel and all the pixels in the pixel window Each pixel value of P ₀₀ -P ₂₂ is used to calculate the pixel absolute difference value D ₀₀ -D ₂₂ .

In step 303, the adaptive brightness adjustment module 104 multiplies the pixel absolute difference values D ₀₀ -D ₂₂ by the adjustment parameters to generate the corresponding adjusted pixel absolute difference values D ₀₀ ′-D ₂₂ ′. Wherein, when the reference brightness of a pair of pixels corresponding to the absolute difference values D ₀₀ '-D ₂₂ ' of each pixel is smaller, the adjustment parameter is larger, when the absolute difference values D ₀₀ '-D ₂₂ ' of each pixel correspond to a pair of pixels. The larger the reference brightness, the smaller the adjustment parameter.

In step 304 , the weight calculation module 106 generates corresponding weight values W ₀₀ -W ₂₂ according to the adjusted absolute pixel difference values D ₀₀ ′-D ₂₂ ′. Wherein, when the size of the adjusted pixel absolute difference value D ₀₀ '-D ₂₂ ' is smaller, the generated weight value is larger, and when the adjusted pixel absolute difference value D ₀₀ '-D ₂₂ ' is larger, the The resulting weight value is smaller. In one embodiment, the weight values are respectively powers of two.

In step 305, the filtering calculation module 108 convolves each pixel P ₀₀ -P ₂₂ in the pixel window with the corresponding weight values W ₀₀ -W ₂₂ to generate the filtering result 109 of the center pixel P ₁₁ .

Although the content of this case has been disclosed as above in an embodiment, it is not configured to limit the content of this case. Anyone who is familiar with this technique can make various changes and modifications without departing from the spirit and scope of the content of this case. Therefore, the content of this case is The scope of protection shall be determined by the scope of the appended patent application.

Claims (10)

1. An image filtering device for filtering an image, wherein the image comprises a plurality of pixels (pixels) each having a pixel value, the image filtering device comprising: A pixel difference calculation module configured so that any one of the pixels is used as a center pixel in a pixel window, and based on the pixel value of the center pixel and the pixel values of all the pixels in the pixel window to calculate the absolute difference of complex pixels; an adaptive brightness adjustment module, configured to multiply the absolute pixel difference values by an adjustment parameter to generate a corresponding complex adjusted pixel absolute difference value. The smaller the luminance is, the larger the adjustment parameter is, and the larger the reference luminance of a pair of pixels corresponding to the absolute difference of the pixels is, the smaller the adjustment parameter is; A weight calculation module configured to generate corresponding complex weight values according to the adjusted absolute pixel difference values. When the size of the adjusted pixel absolute difference values is smaller, the generated weight values are larger. When the size of the adjusted pixel absolute difference is larger, the generated weight value is smaller; and A filtering calculation module configured to perform convolution of the pixel value of each of the pixels in the pixel window with the corresponding weight value to generate a filtering result of the central pixel. 2. The image filtering device according to claim 1, further comprising: A preprocessing module configured to perform preprocessing on the image according to a noise characteristic of a camera device, so that the pixel difference calculation module calculates the absolute pixel difference values according to the preprocessed image. 3 . The image filtering device of claim 2 , wherein when the noise characteristic of the camera device is in the form of an impulse, the preprocessing is a median filter. 4 . 4. The image filtering device of claim 3, wherein the pre-processing module performs the median filtering by a comparison network. 5 . The image filtering device of claim 2 , wherein when the noise characteristic of the camera device is a Gaussian form or a random form, the preprocessing is a Gaussian filter. 6 . 6 . The image filtering device according to claim 1 , wherein the pixel difference calculation module calculates the absolute difference values of the pixels in a one-norm or two-norm manner. 7 . 7. The image filtering device of claim 1, wherein the reference luminance is a minimum pixel value of a corresponding pair of the pixels. 8. The image filtering device of claim 1, wherein the adjustment parameter is a power of two. 9. An image filtering method for filtering an image, wherein the image comprises a plurality of pixels, each of the pixels having a pixel value, the image filtering method comprising: make any one of the pixels a central pixel in a pixel window, and calculate a complex pixel absolute difference based on the pixel value of the central pixel and each of the pixel values of all the pixels in the pixel window; Multiply the absolute difference values of these pixels by an adjustment parameter to generate a corresponding complex adjusted pixel absolute difference value. When a reference brightness of a pair of pixels corresponding to the absolute difference values of these pixels is smaller, the adjustment parameter is larger. When the reference luminance of a pair of pixels corresponding to the absolute difference values of the pixels is larger, the adjustment parameter is smaller; The corresponding complex weight values are generated according to the adjusted absolute pixel difference values. When the size of the adjusted pixel absolute difference values is smaller, the generated weight values are larger. When the adjusted pixel absolute difference value is larger The larger the value, the smaller the resulting weight value; and The pixel values of the pixels in the pixel window are convolved with the corresponding weight values to generate a filtering result of the center pixel. 10 . The image filtering method of claim 9 , further comprising performing a pre-processing on the image according to a noise characteristic of a camera device, so as to calculate the pixel absolute difference according to the pre-processed image. 11 .

Images