CN108288267B - A no-reference evaluation method for SEM image clarity based on dark channel - Google Patents

Abstract

The invention provides a dark channel-based SEM image clarity evaluation method without reference. It uses an edge-preserving filter based on the weighted square method for enhanced denoising, and finally, according to human visual characteristics, the weighting of the maximum gradient and the average gradient is used as the quality score of the image. The present invention applies dark channel to SEM image quality evaluation for the first time, and the performance of the proposed method is better than the current typical fuzzy image quality evaluation method.

Description

A no-reference evaluation method for SEM image clarity based on dark channel

technical field

The invention relates to the field of image quality evaluation, in particular to a dark channel-based scanning electron microscope image clarity evaluation method without reference.

Background technique

SEM images broaden human vision and provide access to fine structural information. In the process of acquiring SEM images, various types of distortion will be caused, and these distortions directly affect the judgment of human beings on research items, so the quality evaluation of SEM images is of great significance. Not getting attention. Image clarity is one of the most important technical indicators in the imaging process of SEM images. Usually, it is necessary to repeatedly adjust imaging parameters and settings to obtain clear images, which is time-consuming and labor-intensive.

There are many existing methods for evaluating image sharpness, and some of these methods are introduced below.

No-reference quality evaluation of sharpness: Some image blur evaluation algorithms have been proposed in recent years. The literature Marziliano [1] and others first proposed to use the Sobel operator to detect the edge of the image, and then calculate the edge width. Ferzli [2] et al. proposed the Just Noticeable Blur (JNB) method, which combines the concept of JNB into a probability summation model, which can predict the relative blurring of images with different contents. Niranjan [3] et al., based on the research of human blur perception with different contrast values, calculated the blur probability of each edge and proposed the cumulative probability of detection blur (CPBD) method, which improved the JNB method. In the literature, Bahrami[4] defined the maximum local variance (MLV) of each pixel as the maximum intensity difference between this pixel and its 8-neighbor, and the standard deviation of the MLV distribution of each pixel is the representation of sharpness.

None of the above methods are designed for SEM images, and it is found that these methods are not ideal in evaluating the quality of SEM images, so a special method for evaluating SEM image clarity is urgently needed.

[1]Marziliano P,Dufaux F,Winkler S,et al.Perceptual blur and ringingmetrics:application to JPEG2000[J].Signal processing:Image communication,2004,19(2):163-172.

[2] Ferzli R, Karam L J.A no-reference objective image sharpness metricbased on the notion of just noticeable blur(JNB)[J].IEEE Transactions onImage Processing,2009,18(4):717-728.

[3]Narvekar N D,Karam L J.A no-reference image blur metric based on the cumulative probability of blur detection(CPBD)[J].IEEE Transactions onImage Processing,2011,20(9):2678-2683.

[4]Bahrami K, Kot A C.A fast approach for no-reference image sharpnessassessment based on maximum local variation[J].IEEE Signal Processing Letters,2014,21(6):751-755.

SUMMARY OF THE INVENTION

Purpose of the invention: The present invention proposes a method for evaluating the sharpness of SEM images, which is a dark channel-based method for evaluating the sharpness of SEM images without reference.

Technical scheme: The technical scheme proposed by the present invention is:

A no-reference evaluation method for scanning electron microscope image clarity based on a dark channel, the method comprising the steps of:

(1) obtain scanning electron microscope blurred image I, carry out dark channel preprocessing to I;

(2) Extract the edge image of the image preprocessed by the dark channel with the Sobel edge operator;

(3) Filter the edge image with an edge-preserving smoothing filter based on the weighted least squares method;

(4) Use the weight of the maximum gradient and the average gradient of the filtered image as the objective quality score for evaluating the blurred image I of the SEM.

Further, the expression that carries out dark channel preprocessing to image I in the described step (1) is:

In the formula, s(I) represents the image I after dark channel preprocessing, s(I)(t) represents the pixel value at the pixel point t in s(I); Ω(t) represents the pixel point t as An m×m window in the center, where m is the width of the window; I(z) represents the pixel value at the pixel point z in the image I.

Further, the edge image expression extracted in the step (2) is:

g(x,y)＝| _Δx h(x,y)|+|Δy v(x, _y )| (2)

Δ _x h(x,y)=s(x+1,y-1)+2s(x+1,y)+s(x+1,y+1)-s(x-1,y-1) -2s(x-1,y)-s(x-1,y+1)

Δy v(x, _y )=s(x-1,y+1)+2s(x,y+1)+s(x+1,y+1)-s(x-1,y-1) -2s(x,y-1)-s(x+1,y-1)

In the formula, g represents the edge image, g(x, y) represents the pixel value at the pixel point (x, y) in the edge image g, Δ _x h(x, y) and Δ _y v(x, y) represent respectively The first derivative of the image s(I) in the horizontal direction and the first derivative in the vertical direction.

Further, in the described step (3), the specific steps of filtering the edge image with an edge-preserving smoothing filter based on the weighted least squares method include:

(4-1) Build the objective function:

in,

In the formula, u represents the target image, u _p represents the pixel value at point p on the image u; λ is the regularization parameter, a _x , p(g) and a _{y, p} (g) respectively represent the smoothing weight in the horizontal direction and the smoothing weight in the vertical direction; l represents the logarithmic luminance channel of the edge image g; α is an exponential, and ε is a small constant;

(4-2) Convert the objective function to matrix form:

In the formula, A _x is a diagonal matrix containing smooth weights in the horizontal direction, A _y is a diagonal matrix containing smooth weights in the vertical direction; D _x and _Dy are the horizontal discrete difference operator and the vertical discrete difference operator, respectively. son;

(4-3) Find u that minimizes the objective function.

Further, the step of solving u that minimizes the objective function is:

Let the objective function equal to 0, and convert the objective function to:

(Q+λL _g )u=g (5)

为与D_x方向相反的后向差分算子，

为与D_y方向相反的后向差分算子；Q为单位矩阵；where L _g is a five-point spatially anisotropic Laplacian matrix for deriving a piecewise smooth adjustment map from a sparse constraint set,

is the backward difference operator opposite to the direction of D _x ,

is the backward difference operator opposite to the direction of _Dy ; Q is the identity matrix;

According to formula (5), u can be calculated.

Further, the maximum gradient and average gradient of the filtered image in the step (4) are respectively:

MG=max(u(x,y))

AG=(∑ _x,y u(x,y)/(w×h))

where MG and AG are the maximum gradient and average gradient of the filtered image, respectively, and w and h are the length and width of the filtered image, respectively;

The objective quality score of the SEM blurred image I is: IQA=MG×AG ^-α .

Beneficial effects: Compared with the existing method for evaluating the quality of an image without a reference for sharpness, the present invention uses the dark channel for the image quality evaluation of the scanning electron microscope for the first time, and the performance is obviously improved.

Description of drawings

FIG. 1 is a flow chart of the present invention.

Detailed ways

The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

The invention relates to a method for evaluating the sharpness of scanning electron microscope images without reference based on a dark channel. Since there is no SEM image database, this embodiment first builds a database of 650 SEM images, and obtains the subjective quality scores of the images through subjective experiments, and extracts 150 blurred and distorted images from the 650 SEM images and Its corresponding subjective quality score, secondly, dark channel preprocessing is performed on the original SEM blurred image, and its edge is calculated for the preprocessed image again. Finally, according to the human visual characteristics, the weight of the maximum gradient and the average gradient is used as the objective quality score of the image. The present invention applies the dark channel to the image quality evaluation for the first time, and the performance of the proposed method is better than the current typical fuzzy image quality evaluation method.

The present invention will be further described below in conjunction with FIG. 1 .

Step 1: 150 SEM blurred images I _i , i=1, 2, ... 150, perform dark channel preprocessing on I _i . For image I _i , the dark channel is defined as:

表示图像I_i在色度分量c上的像素点z处的像素值。由于扫面电子显微图像都是灰度图像，只有一个通道，所以本发明中

即暗通道的公式为：s(I _i )(x) is the image preprocessed by the dark channel; Ω(t) represents an m×m window centered on the pixel point t, and m is the most suitable value of 15 through experiments; in the formula, min represents the value of Min operation;

represents the pixel value of the image I _i at the pixel point z on the chrominance component c. Since scanning electron microscope images are all grayscale images with only one channel, in the present invention,

That is, the formula for the dark channel is:

Step 2: Use the Sobel edge operator to extract the edge image of the dark channel preprocessed image:

g _i (x,y)＝|Δ _x h _i (x,y)|+|Δ _y v _i (x,y)|

Δ _x h _i (x,y)=s _i (x+1,y-1)+2s _i (x+1,y)+s _i (x+1,y+1)-s _i (x-1 ,y-1)-2s _i (x-1,y)-s _i (x-1,y+1)

Δ _y v _i (x,y)=s _i (x-1,y+1)+2s _i (x,y+1)+s _i (x+1,y+1)-s _i (x-1 ,y-1)-2s _i (x,y-1)-s _i (x+1,y-1)

In the formula, g _i (x, y) represents the pixel value at the pixel point (x, y) in the edge image g _i , and Δ _x h _i (x, y) and Δ _y v _i (x, y) represent the dark The first-order derivative in the horizontal direction and the first-order derivative in the vertical direction of the image _si (I) after channel preprocessing.

Step 3: The edge image g _i (x, y) is enhanced with an edge-preserving smoothing filter based on the weighted least squares method, and noise is removed at the same time. For the input image g _i , we set the target image to be _ui . On the one hand, we want it to be as close to _gi as possible, and at the same time, _ui should be as smooth as possible, except for some places where the gradient of _gi is relatively large. Therefore, the specific steps for filtering the edge image by the edge-preserving smoothing filter based on the weighted least squares method include:

Find a solution that minimizes the following objective function.

in,

In the formula, the first term of the objective function (u _ip -g _ip ) ² is used to make the input image and the output image as similar as possible, and the second term is the regular term, which minimizes the partial derivative of u _i to make the output image smoother the better. λ is a regular term parameter, which balances the proportions of the two. The larger the λ, the smoother the image. The smoothing weights are a _x,p ( _gi ) and a _y,p ( _gi ), respectively. where _li is the logarithmic luminance channel of the input image _gi , α is the exponential, and ε is a small constant (usually 0.0001).

Write the above equation in matrix form:

Here, A _ix and A _iy are diagonal matrices containing smoothing weights, and Di _ix and D _iy are discrete difference operators.

Let the objective function equal to 0, and convert the objective function to:

(Q+λL _ig )u _i = _gi

主要用于从稀疏约束集导出分段平滑调整图；

和

是后向差分算子。where L _ig is a five-point spatially anisotropic Laplace matrix,

Mainly used to derive piecewise smooth adjustment maps from sparse constraint sets;

and

is the backward difference operator.

Step 4: Finally, use the weight of the maximum gradient and the average gradient of the filtered image as the quality score of the evaluation image. The maximum gradient and average gradient of an image are defined as:

MG _i =max(u _i (x,y))

AG _i =(∑ _x,y u _i (x,y)/( _wi × _hi ))

where MG _i and AG _i are the maximum gradient and average gradient of the filtered image, respectively, and w and h are the length and width of the filtered image.

The objective quality score of the SEM blurred image I _i is:

IQA _i =MG _i ×AG _i ^-α

Through experimental tests, it can be concluded that when α is 0.4366, the result is optimal.

Experimental results and performance:

In order to verify the performance of the method proposed by the present invention, the objective quality score predicted in this embodiment is compared with the subjective quality score, that is, it is judged whether the result obtained by the above technical solution is consistent with human visual sense. Since the objective quality score and subjective quality score obtained in this implementation are nonlinear, a suitable nonlinear transformation should be performed on the objective quality score, that is, the objective quality score of the image is mapped to the subjective quality score through a nonlinear fitting function. on the same scale. Usually a five-parameter fitting function is used, and v is set to represent the objective quality score, and the expression of f(v) is as follows:

Among them, τ _i , i=1, 2, 3, 4, 5 are parameters of fitting.

After nonlinear fitting, three common indexes can be selected to determine the performance of the present invention. The subjective quality score of the _ith SEM image and the objective quality score after nonlinear transformation are defined as _ki and qi respectively, and the calculation process of the three indicators is described below.

(1) Pearson Linear Correlation Coefficient (PLCC):

和

分别表示N幅扫描电镜图像的主观质量分数均值和客观质量分数均值。where N represents the number of SEM images,

and

represent the mean subjective quality score and the mean objective quality score of N SEM images, respectively.

(2) Root Mean Square Error (RMSE):

(3) Spearman correlation coefficient (SRCC):

Among them, d _i represents the ranking difference between the subjective quality score and the objective quality score of the SEM image I _i .

The above three performance indicators judge the performance of the technical solution of the present invention from the perspectives of prediction accuracy and monotonicity, wherein PLCC and RMSE are indicators of algorithm prediction accuracy, and higher PLCC and SRCC represent higher algorithm prediction accuracy. The prediction monotonicity index of the test algorithm is RMSE. The lower the RMSE, the better the algorithm performance is.

Table 1 is a comparison of the performance of the present invention and 11 typical no-reference evaluation methods for sharpness. For the convenience of viewing, the best performance has been shown in bold.

Table 1 Performance comparison between the method of the present invention and the existing sharpness no-reference image quality evaluation algorithm

From the above table, we can obtain that the method proposed by the present invention has obvious advantages compared with the existing typical definition no-reference image quality evaluation methods, that is, the value of PLCC/SRCC is significantly higher than all methods, and the RMSE is the lowest.

The above is only the preferred embodiment of the present invention, it should be pointed out: for those skilled in the art, under the premise of not departing from the principle of the present invention, several improvements and modifications can also be made, and these improvements and modifications are also It should be regarded as the protection scope of the present invention.

Claims (3)

1. A non-reference evaluation method for the definition of a scanning electron microscope image based on a dark channel is characterized by comprising the following steps:

(1) acquiring a scanning electron microscope blurred image I, and performing dark channel pretreatment on the I;

wherein, s (I) represents the image after the image I is subjected to dark channel preprocessing, and s (I), (t) represents the pixel value at the pixel point t in s (I); Ω (t) represents a window of m × m with the pixel point t as the center, and m is the width of the window; i (z) represents the pixel value at pixel point z in image I;

(2) extracting an edge image of the image subjected to dark channel preprocessing by using a Sobel edge operator;

(3) filtering the edge image by using an edge-preserving smoothing filter based on a weighted least square method; the method comprises the following specific steps:

31) constructing an objective function:

wherein,

wherein u represents the target image, u_pRepresents the pixel value at a point p on the image u; λ is a regularization term parameter, a_x,p(g) And a_y,p(g) Respectively representing the smoothing weight in the horizontal direction and the smoothing weight in the vertical directionWeighing; l represents the logarithmic luminance channel of the edge image g; alpha is an exponent, and epsilon is a very small constant;

32) converting the objective function to a matrix form:

in the formula, A_xFor diagonal matrices containing smooth weights in the horizontal direction, A_yIs a diagonal matrix containing smooth weights in the vertical direction; d_xAnd D_yRespectively a horizontal direction discrete difference operator and a vertical direction discrete difference operator;

is a reaction of with D_xThe backward difference operator in the opposite direction is,

is a reaction of with D_yBackward difference operators in opposite directions;

33) solving for u that minimizes the objective function;

(4) weighting the maximum gradient and the average gradient of the filtered image to be used as objective quality scores for evaluating the scanning electron microscope fuzzy image I; the maximum gradient and the average gradient of the filtered image are respectively:

MG＝max(u(x,y))

AG＝(∑_x,yu(x,y)/(w×h))

in the formula, MG and AG are the maximum gradient and average gradient of the filtered image respectively, and w and h are the length and width of the filtered image respectively;

the objective quality fraction of the scanning electron microscope blurred image I is as follows: iQA ═ MG × AG^-α。

2. The method for non-reference evaluation of the definition of a scanning electron microscope image based on a dark channel according to claim 1, wherein the edge image extracted in the step (2) has an expression as follows:

g(x,y)＝|Δ_xh(x,y)+|Δ_yv(x,y)| (2)

Δ_xh(x,y)＝s(x+1,y-1)+2s(x+1,y)+s(x+1,y+1)-s(x-1,y-1)-2s(x-1,y)-s(x-1,y+1)

Δ_yv(x,y)＝s(x-1,y+1)+2s(x,y+1)+s(x+1,y+1)-s(x-1,y-1)-2s(x,y-1)-s(x+1,y-1)

wherein g represents an edge image, g (x, y) represents a pixel value at a pixel point (x, y) in the edge image g, and Δ_xh (x, y) and Δ_yv (x, y) represents the first derivative of the image s (i) in the horizontal direction and the first derivative in the vertical direction, respectively.

3. The method for non-reference evaluation of the definition of a scanning electron microscope image based on a dark channel according to claim 2, wherein the step of solving u that minimizes the objective function is:

let the objective function equal to 0, convert the objective function to:

(Q+λL_g)u＝g (5)

wherein L is_gIs a five-point space anisotropic Laplacian matrix which is used for deriving a piecewise smooth adjustment graph from a sparse constraint set,

is a reaction of with D_xThe backward difference operator in the opposite direction is,

is a reaction of with D_yBackward difference operators in opposite directions; q is an identity matrix;

u is calculated according to equation (5).

Images