CN118570213B - A method for detecting defects in optical wafers - Google Patents

Abstract

The present invention discloses a method for detecting defects of a light pass sheet, comprising the following steps: step 1: converting an original image into a grayscale image for preprocessing to obtain a binary image; step 2: obtaining the vertex coordinates of the light pass sheet in the binary image, namely, the contour detection coordinates, as well as the size of the light pass sheet, the inclination angle of the light pass sheet, and the center coordinates of the light pass sheet through contour detection; step 3: using Blob detection to obtain the center coordinates of the light pass sheet in the grayscale image; step 4: performing vertex indentation on the coordinates after the contour detection coordinates and the Blob detection coordinates are fused to remove the influence of the burrs of the light pass sheet in the binary image on the final detection result and to prepare a mask map; step 5: performing defect detection on the circumscribed rectangular area of the binary image, and then marking the defects in the original image. The present invention compares the Blob center coordinates with the contour center coordinates, combines the Blob center coordinates with the contour information, supplements the contour vertex coordinates, prevents missed detection, and obtains more accurate coordinates.

Description

A method for detecting defects in optical wafers

Technical Field

The invention relates to a detection method, in particular to a method for detecting defects in optical pass sheets.

Background Art

Optical film, also known as optical thin film, has a complex and precise production process and plays an important role in many high-tech applications, such as lasers, solar cells, and fiber-optic communications. During the production and transportation of optical film, it is easy to cause surface scratches, edge collapse, pitting and other small defects that are difficult to find. These defects may cause product performance degradation or even make the product unusable.

Since the optical sheet module contains multiple optical sheet units, the length and width of each optical sheet unit are between 1.4-2.5 mm. For such tiny surface defects, high-power eyepieces are often used for observation during manual inspection, and the position of the optical sheet is manually adjusted in small increments to observe the defects of each optical sheet unit. This inspection method not only has low inspection efficiency and large workload, but also causes visual damage to inspectors during long working hours. In addition, the inspection levels of inspectors are uneven, which is prone to missed inspections and wrong inspections, which has a great impact on product quality.

In summary, it is necessary to introduce an automated defect detection method for optical wafers with high-precision detection effects, which can effectively identify various defects, replace manual detection and quickly execute it on the production line to improve detection efficiency.

Summary of the invention

In view of the technical problems existing in the prior art, the purpose of the present invention is to provide a method for detecting defects in optical pass sheets.

The object of the present invention is achieved by the following technical solution: A method for detecting defects in a light-transmitting sheet comprises the following steps:

Step 1: Obtain the original image with the optical pass film through an image acquisition device, and copy the original image into a grayscale image;

Step 2: Preprocess the grayscale image, obtain the coordinates of the contour vertices through contour detection and angle cosine value screening, and calculate the contour center coordinates, contour width, contour height and contour inclination angle;

Step 3: Use Blob detection to obtain the center coordinates of the Blob in the grayscale image, compare the center coordinates of the Blob with the center coordinates of the contour, combine the center coordinates of the Blob with the contour information, supplement the coordinates of the contour vertices, obtain the supplementary coordinates, and use the indentation algorithm on the supplementary coordinates to obtain the indentation coordinates;

Step 4: Create a black image with the same resolution as the grayscale image, use the polygonal circumscribed rectangle algorithm for the indented coordinates of the black image to obtain the circumscribed rectangular area of the indented coordinates, i.e., the black image mask area, perform integral binarization and preprocessing on the grayscale image mask area with the same coordinates as the black image mask area, and generate a mask edge map;

Step 5: Perform defect detection on the mask edge image to obtain defect information, and then mark the defects in the original image.

Preferably, obtaining the original image: obtaining the original image with the optical pass film by an image acquisition device with a high-definition camera, and copying the original image into a grayscale image;

Obtain contour information: Use Gaussian blur processing on the grayscale image to reduce the noise of the grayscale image and make the grayscale image smooth in preparation for feature extraction; Use the Sobel operator to calculate the total gradient of the grayscale image and generate the total gradient map of the grayscale image, so that the grayscale image only displays the pixel intensity change rate, and then use the parameter threshold method to binarize the total gradient map of the grayscale image, and divide the total gradient map of the grayscale image into foreground and background, that is, obtain the parameter contour map; then use the OpenCV adaptive threshold method to binarize the parameter contour map, divide the parameter contour map into edge areas and non-edge areas, so that the contour in the parameter contour map becomes a single pixel display, that is, obtain the edge contour map; calculate the area of the connected domain detected in the edge contour map and compare it with the first set area, and delete the area in the edge contour map where the connected domain area is smaller than the first set area; Use the Sobel operator to calculate the total gradient of the edge contour map and binarize it by the parameter threshold method to obtain the parameter edge contour Figure; use the edge search algorithm to obtain the edge of the connected area in the parameter edge contour map, perform polygonal approximation on the edge of the connected area, and obtain the contour of the connected area; remove the connected area contours except the convex quadrilateral contour in the connected area contour, calculate the cosine value of the angle between the vertex coordinates of the convex quadrilateral contour, select the maximum value of the cosine value of the angle formed by the vertex coordinates of the convex quadrilateral contour and compare it with the second set value, and if it is less than the second set value, it is judged that the shape of the convex quadrilateral is close to a rectangle, and the convex quadrilateral contour with a shape close to a rectangle is screened out, and the circumscribed inclined rectangle algorithm is used to calculate the circumscribed inclined rectangle of the convex quadrilateral contour, that is, the contour vertex coordinates; calculate the contour center coordinates through the contour vertex coordinates, compare the contour vertex coordinates with the contour center coordinates, and rearrange them clockwise; perform inverse trigonometric function calculation on the angle between the first two points sorted by the contour vertex coordinates and the X-coordinate axis to obtain the contour inclination angle, and calculate the contour width and contour height;

Obtain the Blob center coordinates and supplement the contour vertex coordinates: the Blob detector sets the detection style so that the Blob detector only detects quadrilateral style patches; the Blob detector sets the detection area so that the Blob detector screens out patches outside the third set area range; the configured Blob detector is used to directly detect the grayscale image to obtain the Blob center coordinates; the distance dis between the Blob center coordinates and the contour center coordinates is calculated using the Euclidean distance formula. When dis is less than or equal to the fourth set value, the Blob center coordinates and the contour center coordinates are the center points of the same optical pass sheet and no supplement is required; when dis is greater than the fourth set value, the Blob center coordinates are considered inconsistent with the contour center coordinates. At this time, the optical pass sheet obtained by the Blob detection needs to be supplemented to the optical pass sheet obtained by the contour detection to obtain supplementary coordinates; in order to prevent the optical pass sheet at the edge of the image from being misdetected, the supplementary coordinates that exceed the edge of the grayscale image are removed; and the supplementary coordinates within the edge of the grayscale image are polygonally indented to obtain the indented coordinates;

Obtain the mask area and process it: create a black image with the same resolution as the grayscale image, draw a simulated light pass film on the black image with the indented coordinates as vertices, and fill the indented coordinate area of the black image with white; use the polygonal circumscribed rectangle algorithm for the indented coordinates to obtain the four vertices of the circumscribed rectangle of the indented coordinates, and the circumscribed rectangular area surrounded by the four vertices is the black image mask area, traverse the pixels in the black image mask area, and if the ratio of the sum of black pixels in the black image mask area to the total sum of pixels is less than the fifth set value, the black image mask area is judged to be qualified; for the grayscale image mask area with the same coordinates as the black image mask area, use the calculated integral map and square integral map to calculate the pixel sum, square pixel sum, pixel mean and pixel variance of all positions, obtain information of multiple pixels in one calculation, and use the accumulated pixel values for calculation to complete the integral binarization to obtain Mask integral binary image; using a connected domain algorithm to obtain the area of each connected domain on the mask integral binary image, and the connected domain area smaller than the first set area is a minimal connected domain and the minimal connected domain is removed, that is, impurities are removed; setting the pixels in the mask integral binary image except the indented coordinate area to 0, that is, removing interference factors; using the Sobel operator to obtain the total gradient of the mask integral binary image and generate a mask gradient image, so that the mask integral binary image only displays the pixel intensity change rate; using the parameter threshold method to binarize the mask gradient image, the mask gradient image is divided into the foreground and the background, that is, the mask parameter contour image is obtained; using the adaptive threshold method to binarize the mask parameter contour image, the mask parameter contour image is divided into the edge area and the non-edge area, and the mask edge contour image is obtained; using the Sobel operator to calculate the total gradient of the mask edge contour image again and binarize it by the parameter threshold method to obtain the mask edge image;

Obtain defect information and mark it: Use the connected domain algorithm to obtain the internal connected domain information of the mask edge map. If the connected domain information exceeds the corresponding set parameters, it is determined that there is a defect in the mask edge map, and the coordinates of the defective mask edge map, that is, the defect coordinates, are recorded, and marked in the original map according to the defect coordinates.

Preferably, step 2 further comprises the following steps:

According to the approximation algorithm, polygonal approximation is performed on the edge of the connected domain to obtain the connected domain contour, and the connected domain contour except the convex quadrilateral contour is removed from the connected domain contour, and the coordinates of the four vertices surrounding the convex quadrilateral contour are calculated, that is, the coordinates of the four contour vertices of the convex quadrilateral are obtained;

The cosine values of the angles between the vertices of the convex quadrilateral outline are calculated, and the cosine values of the angles between the two vectors formed by three adjacent points are calculated and the maximum value is selected to determine whether the shape of the convex quadrilateral is close to a rectangle. The specific formula is as follows:

In the formula is the coordinate of the ith vertex of the convex quadrilateral, is the x- coordinate of the ith vertex of the convex quadrilateral, is the y coordinate of the i- th vertex of the convex quadrilateral; are the coordinates of the i +1th vertex of the convex quadrilateral, is the x- coordinate of the i +1th vertex of the convex quadrilateral, is the y coordinate of the i +1th vertex of the convex quadrilateral; , are the coordinates of the i +2th vertex of the convex quadrilateral, is the x- coordinate of the i +2th vertex of the convex quadrilateral, is the y coordinate of the i +2th vertex of the convex quadrilateral; vector For point Pointing Point vector, vector For point Pointing Point The vector of For vector and vector The angle between two vectors, is the maximum cosine value calculated, point With point for the same point;

like If the value of is less than the second set value, it is considered that the shape of the convex quadrilateral is close to a rectangle, and the contour of the convex quadrilateral is the contour of the light pass sheet, which meets the requirements.

Preferably, the contour inclination angle is calculated according to the contour vertex coordinates, specifically as follows:

In the formula, is the profile inclination angle, is the number of contours, is the coordinate of the first point of the i -th contour, is the x- coordinate of the first point of the i -th contour, is the y coordinate of the first point of the i -th contour; is the coordinate of the second point of the i -th contour, is the x- coordinate of the second point of the i - th contour, is the y coordinate of the second point of the i - th contour;

The contour width and height are calculated according to the contour vertex coordinates using the Euclidean distance formula, as shown below:

In the formula, is the outline width, is the profile height, is the number of contours, is the coordinate of the first point of the i -th light channel contour, is the x- coordinate of the first point of the i -th contour, is the y coordinate of the first point of the i -th contour; is the coordinate of the second point of the i - th light channel contour, is the x- coordinate of the second point of the i - th contour, is the y coordinate of the second point of the i - th contour; is the coordinate of the third point of the i - th light channel contour, is the x- coordinate of the third point of the i - th contour, is the y coordinate of the third point of the i - th contour.

Preferably, obtaining the coordinates of four contour vertices in step 2 includes the following steps:

The cosine value of the angle is calculated using three adjacent points of the detected vertices of the convex quadrilateral outline, and the maximum cosine value of the three or four corners of the convex quadrilateral outline is compared with the second set value. If the maximum cosine value of the convex quadrilateral outline is less than the second set value, it is judged that the shape of the convex quadrilateral is close to a rectangle; the inclination angle of each convex quadrilateral outline is calculated by using the inverse trigonometric function of the upper left corner vertex and the upper right corner vertex of each convex quadrilateral outline, the sum of the inclination angles of all convex quadrilateral outlines is calculated, and then the average value is calculated.

Preferably, in step 3, the light flux sheet obtained by Blob detection is added to the light flux sheet obtained by contour detection, specifically: the center coordinates of the Blob are used as the center of the rectangle, and the contour inclination angle is used as the center of the rectangle. The tilt angle of the rectangle and the width of the outline is the rectangle width, and the outline height Draw a Blob rectangle for the rectangle height and get the coordinates of the Blob rectangle vertices;

Sort the Blob rectangle vertex coordinates clockwise, and add the sorted Blob rectangle vertex coordinates to the contour vertex coordinate points to obtain the supplementary coordinates;

The supplementary coordinates within the edge of the grayscale image are indented to obtain the indented coordinates, as shown in the following formula:

in

In the formula are supplementary coordinates, is the indentation coordinate, L is the indentation distance, , is a vector consisting of a supplementary coordinate and two adjacent supplementary coordinates. for The modulus of a vector, for The magnitude of a vector, is the angle between the two sides of the supplementary coordinates, Representation vector The component in the x direction, Representation vector The component in the y direction, Representation vector The component in the x direction, Representation vector Component in the y direction.

Preferably, the polygonal circumscribed rectangle algorithm is used to obtain the four vertices of the circumscribed rectangle of the indented coordinates of the black image, as shown in the following formula:

Where left is the left boundary of the bounding rectangle, right is the right boundary of the bounding rectangle, top is the upper boundary of the bounding rectangle, and bottom is the lower boundary of the bounding rectangle. is the minimum x- coordinate value of all coordinates in the indented coordinate area. is the minimum y coordinate value of all coordinates in the indented coordinate area. is the maximum x- coordinate value of all coordinates in the indented coordinate area. The maximum y coordinate value among all coordinates in the indented coordinate area;

The black image mask area is a black image mask area without an inclination angle, pixels in the black image mask area are traversed, the traversed black pixels are accumulated, and a ratio of the sum of black pixels in the black image mask area to the sum of total pixels is obtained. If the ratio is less than a fifth set value, the mask area is judged to be qualified;

The grayscale image mask area image processing uses an integral threshold algorithm to perform Gaussian blur processing on the grayscale image mask area, sets the Gaussian blur standard deviation value sigma, and uses the boundary extension algorithm to copy the grayscale image mask area boundary;

Calculate the pixel sum of the integral kernel in the integral image: Use the anti-boundary mode to expand the grayscale mask area boundary. sum represents the integral area of the image after boundary expansion. sqsum represents the integral square area of the image after boundary expansion. Get the cumulative sum of the pixels in the area enclosed by the four vertices of the integral area and the origin of the background image coordinates, as shown in the following formula:

In the formula, and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper left corner of the integral area and the origin of the coordinates. and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper right corner of the integral area and the origin of the coordinates. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower left corner of the integration area and the coordinate origin. and All represent the sum of pixels in the area enclosed by the coordinates of the lower right corner of the integration area and the coordinate origin, i is the sum of the first i pixels in the X direction of the integration area, j is the sum of the first j pixels in the Y direction of the integration area, ker is the size of the integration kernel, is the pixel sum;

Similarly, calculate the cumulative sum of pixels in the area enclosed by the four vertices of the integral square area and the origin of the background coordinate system, as follows:

In the formula, and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper left corner of the integral square area and the coordinate origin. and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper right corner of the integral square area and the origin of the coordinate system. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower left corner of the integral square area and the origin of the coordinate system. and All represent the sum of pixels in the area enclosed by the coordinates of the lower right corner of the integral square area and the coordinate origin, i is the sum of the first i pixels in the X direction of the integral square area, j is the sum of the first j pixels in the Y direction of the integral square area, ker is the integral kernel size, is the sum of square pixels;

By integrating the pixels in the area and obtaining the pixel mean , as follows:

And get the pixel variance var as follows:

Calculate the integral binarization threshold th as follows:

The grayscale image mask area is binarized pixel by pixel through the integral binarization threshold to obtain the mask integral binary image, as shown in the following formula:

in, is the grayscale mask area The pixel value of the coordinate point, is the integral binarization threshold calculated for the corresponding pixel value, is the pixel value after binarization;

The removing of the extremely small connected domain whose area is smaller than the first set area is to set the pixels in the extremely small connected domain whose area is smaller than the first set area to 0.

Preferably, the resolution of the black image is the same as that of the grayscale image, that is, the coordinates of the grayscale image mask area and the black image mask area are the same, and an integral binarization threshold calculation is performed on the grayscale image mask area with no tilt angle having the same coordinates as the black image mask area with no tilt angle. When calculating the pixel variance, a value greater than 0 is taken. If the pixel variance is less than 0, the pixel variance is taken as 0. During binarization, the calculated integral binarization threshold is used to binarize the grayscale image mask area pixel by pixel.

Preferably, performing defect detection on the mask edge image and then marking the defects in the original image comprises the following steps:

5.1. Using a connected domain algorithm to obtain the connected domain information inside the mask edge map, the connected domain information inside the mask edge map includes the area, number, width and height of the connected domain inside the mask edge map;

5.2. Point defect detection: compare the area of the connected domain obtained by the mask edge map detection with the area calculated by the sixth set diameter. When the area of the connected domain is larger than the area calculated by the sixth set diameter, it is considered that there is a point defect in the mask edge map;

5.3. Pockmark defect detection: compare the number of connected domains detected by the mask edge map with the set number. If the number of connected domains is greater than the seventh set number of pockmarks, it is considered that the mask edge map has a pockmark defect;

5.4. Slender defect detection, step 1: whether the ratio of the width to the height of the connected domain of the mask edge map is greater than the eighth set value or less than the ninth set value, wherein the eighth set value is greater than the ninth set value, if the ratio of the width to the height of the connected domain of the mask edge map is greater than the eighth set value or less than the ninth set value, it is a slender connected domain, step 2: according to the comparison between the maximum edge value of the connected domain and the tenth set value, when the maximum edge value of the connected domain is greater than the tenth set value, it is considered that the mask edge map has a slender defect; the larger value of the width and height of the connected domain is the maximum edge value of the connected domain;

5.5. After detection, store the center coordinates of all mask edge images that are judged to be defects, and mark the corresponding coordinate points in the original image.

Preferably, the kernel for defining Gaussian blur is 1, 3, 5 or 7, and the standard deviation value of Gaussian blur is 0.5-15. More preferably, the kernel for defining Gaussian blur is 3, and the standard deviation value of Gaussian blur is 15.

Compared with the prior art, the present invention has the following advantages and effects:

1. The present invention uses the calculated angle cosine value to screen the vertex coordinates of the convex quadrilateral contour to prevent false detection; uses the Blob center coordinates to compare with the contour center coordinates, and combines the Blob center coordinates with the contour information to supplement the contour vertex coordinates to prevent missed detection, and the acquired coordinates are more accurate.

2. The present invention uses a polygon shrinking algorithm to obtain shrinking coordinates and remove the influence of the burrs of the optical pass sheet on defect detection.

3. The present invention uses a polygon circumscribed rectangle algorithm to obtain the circumscribed rectangle of the indented coordinates, namely the mask area, and subsequently only processes the mask area, thereby reducing the calculation range of the integral threshold algorithm and improving the detection efficiency.

4. The present invention expands the inward coordinates with tilt angles into a mask area without tilt angles through a polygon circumscribed rectangle algorithm, which can more simply perform integral threshold calculation on the grayscale mask area without tilt angles, thereby accelerating the calculation speed.

5. The grayscale image mask area integral threshold binarization processing module of the present invention uses an integral image and a square integral image to pre-calculate the grayscale image mask area image, which accelerates the calculation of local statistical information, improves performance, avoids repeated calculations, corrects variance, avoids numerical instability, and enhances accuracy.

6. The present invention sets pixels outside the indented coordinate region of the mask integral binary image to 0, thereby eliminating the influence of factors outside the indented coordinate region of the grayscale image on image processing, and making image defects more accurately judged.

7. When judging defects, the present invention uses multiple factors such as defect area, defect quantity, and defect aspect ratio to make a joint judgment, which can more accurately judge multiple defects.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic flow chart of a method for detecting defects in a light-transmitting sheet according to the present invention;

Fig. 2 is the original image detected by the present invention;

FIG3 is a total gradient diagram of a grayscale image calculated using a Sobel operator according to the present invention;

FIG4 is a contour vertex graph obtained by contour detection of the present invention;

FIG5 is a Blob center coordinate diagram obtained by Blob detection according to the present invention;

FIG6 is a Blob rectangle vertex coordinate diagram for comparison and supplementation of the present invention;

FIG7 is a supplementary coordinate diagram obtained by the present invention;

FIG8 is a schematic diagram of supplementary coordinates and indented coordinates of the present invention;

FIG9 is a diagram of a black image mask region obtained by a polygon circumscribed rectangle algorithm of the present invention;

FIG10 is a grayscale image mask region diagram obtained by the present invention;

FIG11 is a process diagram of obtaining a mask edge image according to the present invention;

FIG12 is a schematic diagram showing the principle of clearing a connected domain according to the present invention;

FIG13 is a schematic diagram showing the principle of calculating the cosine value of an angle according to the present invention;

FIG14 is a schematic diagram showing the principle of reordering contour vertex coordinates according to the present invention;

FIG15 is a contour information diagram of contour vertex coordinate calculation according to the present invention;

FIG16 is a schematic diagram showing the principle of calculating the indentation coordinates of the present invention;

FIG17 is a schematic diagram showing the principle of calculating a circumscribed rectangle according to the present invention;

FIG. 18 is a schematic diagram showing the principle of calculating the sum of pixels within an integral kernel according to the present invention.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

The present invention aims to solve the problem that there are few weak feature detection technologies for optical wafers and the detection accuracy of weak feature defects of optical wafers in the prior art is not high. The present invention provides a method for detecting defects of optical wafers, which can effectively identify tiny defects on the surface of optical wafers (such as slight scratches, punctures, pitting, etc.), replace manual detection to improve detection efficiency, be quickly executed on the production line, and provide highly accurate detection results.

A method for detecting defects in a light-transmitting sheet comprises the following steps:

Step 1: Obtain an original image with multiple optical pass sheets through an image acquisition device, and copy the original image into a grayscale image;

Step 2: Preprocess the grayscale image, obtain the coordinates of the contour vertices through contour detection and angle cosine value screening, and calculate the contour center coordinates, contour width, contour height, and contour inclination angle;

Step 3: Use Blob detection to obtain the center coordinates of the Blob in the grayscale image, compare the center coordinates of the Blob with the center coordinates of the contour, combine the center coordinates of the Blob with the contour information, supplement the coordinates of the contour vertices, obtain the supplementary coordinates, and use the indentation algorithm on the supplementary coordinates to obtain the indentation coordinates;

Step 4: Create a black image with the same resolution as the grayscale image, use the polygonal circumscribed rectangle algorithm for the indented coordinates of the black image to obtain the circumscribed rectangular area of the indented coordinates, i.e., the black image mask area, perform integral binarization and preprocessing on the grayscale image mask area with the same coordinates as the black image mask area, and generate a mask edge map;

Step 5: Perform defect detection on the mask edge image to obtain defect information, and then mark the defects in the original image.

Specifically, obtaining the original image: obtaining the original image with multiple optical pass sheets through an image acquisition device with a high-definition camera, and copying the original image into a grayscale image;

Get contour information: Use Gaussian blur processing on the grayscale image to reduce the noise of the grayscale image and make it smooth in preparation for feature extraction; use the Sobel operator to calculate the total gradient of the grayscale image and generate the total gradient map of the grayscale image, so that the grayscale image only displays the pixel intensity change rate, and then use the parameter threshold method (the algorithm provided in OpenCV) to binarize the total gradient map of the grayscale image, and divide the total gradient map of the grayscale image into foreground and background, that is, obtain the parameter contour map; then use the OpenCV adaptive threshold method to binarize the parameter contour map, and divide the parameter contour map into edge area and non-edge area. The contour in the parameter contour map is changed to a single pixel display, that is, the edge contour map is obtained; the area of the connected domain detected in the edge contour map is calculated and compared with the first set area, and the area in the edge contour map with an area smaller than the first set area is deleted, specifically, all pixels in the area smaller than the first set area are set to black; the total gradient of the edge contour map is calculated using the Sobel operator and binarized using the parameter threshold method to obtain the parameter edge contour map (i.e.: grayscale image preprocessing); the edge search algorithm (the algorithm provided by Opencv) is used to obtain the edge of the connected area in the parameter edge contour map, and the connected area is The edge of the domain is polygonally approximated to obtain the contour of the connected area (that is, contour detection obtains the coordinates of the vertices of the convex quadrilateral contour); the contours of the connected area other than the convex quadrilateral contour are removed from the contour of the connected area, the cosine values of the angles are calculated by the vertex coordinates of the convex quadrilateral contour, and the maximum value of the cosine value of the angle formed by the vertex coordinates of the convex quadrilateral contour is selected to determine whether the shape of the convex quadrilateral is close to a rectangle. Specifically, the maximum value of the cosine value of the angle formed by the vertex coordinates of the convex quadrilateral contour is selected and compared with a second set value. If it is less than the second set value, it is determined that the shape of the convex quadrilateral is close to a rectangle, and the convex quadrilaterals with a shape close to a rectangle are screened out. The convex quadrilateral contour is obtained by using the circumscribed tilted rectangle algorithm (the circumscribed tilted rectangle algorithm is an algorithm built into Opencv), that is, the circumscribed tilted rectangle of the convex quadrilateral contour is calculated, that is, the cosine value of the angle is filtered to obtain the contour vertex coordinates; the contour center coordinates are calculated by the contour vertex coordinates, the contour vertex coordinates are compared with the contour center coordinates, and rearranged in a clockwise direction; the contour tilt angle is obtained by performing inverse trigonometric calculations based on the angle between the first two points of the contour vertex coordinates and the X-coordinate axis, and the contour width and contour height are calculated (that is, the contour center coordinates, contour width, contour height, and contour tilt angle are obtained);

Get the Blob center coordinates and supplement the contour vertex coordinates: the Blob detector sets the detection style (i.e., sets the Blob detector) so that the Blob detector only detects quadrilateral style patches; the Blob detector sets the detection area so that the Blob detector screens out patches outside the third set area range; the configured Blob detector is used to directly detect the grayscale image to obtain the Blob center coordinates; the distance dis between the Blob center coordinates and the contour center coordinates is calculated using the Euclidean distance formula. When dis is less than or equal to the fourth set value, the Blob center coordinates If the center point of the light pass sheet is the same as the center coordinate of the contour, no supplement is required; when dis is greater than the fourth set value, it is considered that the center coordinate of the Blob is inconsistent with the center coordinate of the contour. At this time, the light pass sheet obtained by the Blob detection needs to be supplemented to the light pass sheet obtained by the contour detection to obtain supplementary coordinates (i.e., the center coordinates of the Blob detection are compared with the center coordinates of the contour detection to obtain supplementary coordinates); to prevent the light pass sheet at the edge of the image from being misdetected, the supplementary coordinates that exceed the edge of the grayscale image are removed; and the supplementary coordinates within the edge of the grayscale image are polygonally indented to obtain the indented coordinates (i.e., the supplementary coordinates are indented to obtain the indented coordinates);

Obtain the mask area and process it: create a black image with the same resolution as the grayscale image, draw a simulated light pass film on the black image with the indented coordinates as vertices, and fill the indented coordinate area of the black image with white; use the polygonal circumscribed rectangle algorithm for the indented coordinates to obtain the four vertices of the circumscribed rectangle of the indented coordinates, and the circumscribed rectangular area surrounded by the four vertices is the black image mask area (that is, obtain the mask area of the black image according to the circumscribed rectangle of the indented coordinates), traverse the pixels in the black image mask area, and obtain the ratio of the black pixel sum to the total pixel sum of the black image mask area. If the ratio of the black pixel sum to the total pixel sum is less than the fifth set value, the black image mask area is judged to be qualified; for the grayscale image mask area with the same coordinates as the black image mask area, use the pre-calculated integral map and square integral map to calculate the pixel sum, square pixel sum, pixel mean and pixel variance at any position, obtain information of multiple pixels in one calculation, and use the accumulated pixel values for calculation to complete the integral binarization to obtain the mask. Integral binary image (i.e., grayscale image mask area integral binarization); use the connected domain algorithm (Opencv's own algorithm) to obtain the area of each connected domain on the mask integral binary image, and remove the extremely small features (i.e., extremely small connected domains) whose connected domain area is smaller than the first set area, i.e., remove impurities; set the pixels in the mask integral binary image except the indented coordinate area to 0, i.e., remove interference factors; use the Sobel operator to obtain the total gradient of the mask integral binary image and generate a mask gradient image, so that the mask integral binary image only displays the pixel intensity change rate; use the parameter threshold method to binarize the mask gradient image, divide the mask gradient image into foreground and background, and obtain the mask parameter contour image; use the adaptive threshold method to binarize the mask parameter contour image, divide the mask parameter contour image into edge area and non-edge area, and obtain the mask edge contour image; use the Sobel operator to calculate the total gradient of the mask edge contour image again and binarize it by the parameter threshold method to obtain the mask edge image (i.e., mask integral binary image preprocessing);

Obtain defect information and mark it: Use the connected domain algorithm to obtain the internal connected domain information of the mask edge map. The internal connected domain information of the mask edge map includes the area, number, width and height of the internal connected domain of the mask edge map. The connected domain area, number, aspect ratio and maximum edge value of the connected domain are compared with the corresponding set parameters. If the connected domain information exceeds the corresponding set parameters, it is determined that there is a defect in the mask edge map, and the coordinates of the defective mask edge map, i.e., the defect coordinates, are recorded, and marked in the original map according to the defect coordinates.

Specifically: As shown in FIG1 , the optical wafer defect detection method comprises the following steps:

1. Get the original image:

An original image with multiple optical pass sheets as shown in FIG2 is obtained by an image acquisition device with a high-resolution camera, and the original image is copied into a grayscale image;

2. Get contour information:

Gaussian blur is used on the grayscale image to reduce the noise of the grayscale image so that the grayscale image becomes smoother and easier to extract features; the kernel of the Gaussian blur can be defined as 3 and the standard deviation as 15. The total gradient of the grayscale image is calculated using the Sobel operator to generate the total gradient map of the grayscale image as shown in Figure 3, so that the grayscale image only displays the pixel intensity change rate. The total gradient map of the grayscale image is binarized using the parameter threshold method, and the total gradient map of the grayscale image is divided into the foreground and the background, that is, the parameter contour map is obtained; the parameter contour map is then binarized using the OpenCV adaptive threshold method, and the parameter contour map is divided into the edge area and the non-edge area, so that the contour in the parameter contour map becomes a single pixel display, and the edge contour map is obtained.

The purpose of contour detection is to obtain the coordinates of the vertices of the optical pass sheet, so the non-optical pass sheet connected area is cleaned up to reduce the amount of subsequent calculations; by calculating the area of the connected domain detected in the edge contour map and comparing it with the first set area, the area in the edge contour map that is smaller than the first set area is deleted. The specific method is to set all pixels in the area in the edge contour map that is smaller than the first set area to black (pixel value is set to 0) to achieve the effect of cleaning impurities; as shown in Figure 12, the impurity area can be obtained by the connected domain algorithm to obtain the impurity connected domain attribute information, and the impurity connected domain attribute information includes the x coordinate of the upper left corner of the impurity connected domain, that is, the x coordinate of the initial pixel , the y coordinate of the upper left corner of the impurity connected domain is the y coordinate of the initial pixel , impurity connected domain width and impurity connected domain height , the x coordinate of the lower right corner of the impurity connected area can be obtained, that is, the x coordinate of the end pixel , the y coordinate of the lower right corner of the impurity connected area is the y coordinate of the end pixel ,in:

Pixels within the impurity connected domain are cleared pixel by pixel, and all connected domains whose area is smaller than the first set area are traversed. Preferably, the first set area may be 5, 6, 7, 8 or 9 pixels, but may be set according to actual needs. The traversal method is as follows:

In the formula, the matrix is the initial pixel of the i-th connected domain in the edge contour map, and the matrix is the width and height corresponding to the i- th connected domain in the edge contour graph, and the matrix It is the end pixel of the i- th connected domain in the edge contour map.

The cleaned adaptive edge contour map is once again calculated using the Sobel operator to calculate the gradient and binarized using the parameter threshold method to obtain the parameter edge contour map; the edge of the connected domain in the parameter edge contour map is obtained using the edge search algorithm, and the edge of the connected domain is polygonally approximated according to the approximation algorithm to obtain the connected domain contour, and the connected domain contour except the convex quadrilateral contour in the connected domain contour is removed, and the vertex coordinates surrounding the convex quadrilateral contour are calculated, as shown in Figure 13. By calculating the cosine value of the angle between the two vectors formed by the three adjacent points of the convex quadrilateral contour vertex and selecting the maximum value, it is determined whether the obtained convex quadrilateral contour vertex coordinates meet the requirements, and whether the shape of the convex quadrilateral is close to a rectangle. The following formula is the formula for calculating the cosine value through three adjacent points:

In the formula is the coordinate of the ith vertex of the convex quadrilateral, is the x-coordinate of the ith vertex of the convex quadrilateral, is the y coordinate of the i- th vertex of the convex quadrilateral; are the coordinates of the i +1th vertex of the convex quadrilateral, is the x- coordinate of the i +1th vertex of the convex quadrilateral, is the y coordinate of the i +1th vertex of the convex quadrilateral; , are the coordinates of the i +2th vertex of the convex quadrilateral, is the x- coordinate of the i +2th vertex of the convex quadrilateral, is the y coordinate of the i +2th vertex of the convex quadrilateral; vector For point Pointing Point vector, vector For point Pointing Point The vector of For vector and vector The angle between two vectors, is the maximum cosine value calculated, point With point for the same point;

like If the value of is less than the second set value (the second set value may be 0.3, 0.4 or 0.5, etc., which may be selected according to the implementation situation), it is considered that the convex quadrilateral is close to a rectangle, and the contour of the convex quadrilateral is the contour of the light pass sheet, which meets the requirements; the vertex coordinates of the convex quadrilateral contour that meets the requirements are selected as shown in FIG. 4.

The circumscribed inclined rectangle algorithm is used to calculate the circumscribed inclined rectangle of the convex quadrilateral contour, that is, the coordinates of the contour vertices. Since the contour vertex coordinates obtained by the circumscribed inclined rectangle algorithm are arranged counterclockwise, in order to facilitate subsequent calculations, the contour center coordinates are first calculated through the contour vertex coordinates, as shown in the following formula:

In the formula Indicates The center coordinates of the contour of the light pass film, Indicates The coordinates of the first contour vertex of the light pass sheet, For the The x coordinate of the first contour vertex of the light pass sheet, For the The y coordinate of the first contour vertex of the light pass sheet, Indicates The coordinates of the third contour vertex of the light pass sheet, For the The x coordinate of the third contour vertex of the light pass sheet, For the The y coordinate of the third contour vertex of the light channel.

Then, according to the comparison of the coordinate values of the contour vertex coordinates relative to the contour center coordinates, rearrange them clockwise as shown in Figure 14, as follows:

The right side of the equation represents the coordinates of the center of the i - th contour, represents the x- coordinate of the center of the i - th contour, represents the y coordinate of the center of the i - th contour, and All represent the vertex coordinates of the i- th contour and the j -th contour. Represents the x- coordinate of the vertex of the original j- th contour of the i- th contour, Represents the y coordinate of the vertex of the original j-th contour of the i- th contour; the left side of the equation Represents the coordinates of the first contour vertex after reordering. Represents the coordinates of the second contour vertex after reordering. Represents the coordinates of the third contour vertex after reordering. Represents the coordinates of the 4th contour vertex after reordering.

As shown in Figure 15, the inclination angle is calculated based on the coordinates of the vertices of the light pass sheet contour. The contour inclination angle theta is obtained by accumulating the inclination angles calculated from the coordinates of all contour vertices and taking the average value. The following formula is used to calculate the contour inclination angle:

In the formula, is the profile inclination angle, is the number of contours, is the coordinate of the first point of the i -th contour, is the x- coordinate of the first point of the i -th contour, is the y coordinate of the first point of the i -th contour; is the coordinate of the second point of the i -th contour, is the x- coordinate of the second point of the i - th contour, is the y coordinate of the second point of the i - th contour;

As shown in Figure 15, the contour width and contour height are calculated using the Euclidean distance formula:

In the formula, is the outline width, is the profile height, is the number of contours, is the coordinate of the first point of the i -th light channel contour, is the x- coordinate of the first point of the i -th contour, is the y coordinate of the first point of the i -th contour; is the coordinate of the second point of the i - th light channel contour, is the x- coordinate of the second point of the i - th contour, is the y coordinate of the second point of the i - th contour; is the coordinate of the third point of the i - th light channel contour, is the x- coordinate of the third point of the i - th contour, is the y coordinate of the third point of the i - th contour;

3. Get the coordinates of the Blob center and add the coordinates of the contour vertices:

The Blob detector sets the detection pattern so that the Blob detector detects only quadrilateral pattern patches; the Blob detector sets the detection area so that the Blob detector screens out patches that are not within the third set area range (the light pass sheet in the image can also be regarded as a patch), that is, screens out patches outside the third set area range; the configured Blob detector is used to detect the grayscale image to obtain the Blob center coordinates as shown in Figure 5;

Calculate the distance between the center coordinates of the contour and the center coordinates of the Blob using the Euclidean distance formula , as follows:

in For the The coordinates of the contour center, For the The x- coordinate of the center of the contour, For the The y coordinate of the center of the contour, For the Blob center coordinates, For the The x coordinate of the center of the blob, For the The y coordinate of the center of the Blob;

When dis is less than or equal to the fourth set value, the Blob center coordinates and the contour center coordinates are the center points of the same optical pass sheet, and no supplement is required. Preferably, the fourth set value can be 17, 18, 20, 21 or 22 pixels, but it can be set according to actual needs; when dis is greater than the fourth set value, it is considered that the Blob center coordinates are inconsistent with the contour center coordinates. At this time, the optical pass sheet obtained by the Blob detection needs to be supplemented to the optical pass sheet obtained by the contour detection. Specifically: the Blob center coordinates are the center of the rectangle, and the contour inclination angle is The tilt angle of the rectangle and the width of the outline is the rectangle width, and the outline height Draw a Blob rectangle for the rectangle height, obtain the Blob rectangle vertex coordinates and remove the Blob rectangle vertex coordinates beyond the edge of the grayscale image; sort the Blob rectangle vertex coordinates as shown in Figure 6 clockwise, add the obtained Blob rectangle vertex coordinates to the contour vertex coordinate points to obtain supplementary coordinates; in order to prevent the optical pass sheet at the edge of the image from being misdetected, filter the supplementary coordinates and remove the supplementary coordinates beyond the edge of the grayscale image;

As shown in FIG7 , the supplementary coordinates within the edge range of the grayscale image are used with the shrinking algorithm to remove the influence of the burrs on the final detection effect of the optical pass sheet (as shown in FIG16 ), and the shrinking coordinates after shrinking are obtained (as shown in FIG8 ); the shrinking coordinates are calculated The formula is:

in

In the formula are supplementary coordinates, is the indentation coordinate, L is the indentation distance, preferably, the indentation distance can be 3, 4, 5 or 6 pixels, but can be set according to actual needs, , is a vector consisting of a supplementary coordinate and two adjacent supplementary coordinates. for The modulus of a vector, for The magnitude of a vector, is the angle between the two sides of the supplementary coordinates, Representation vector The component in the x direction, Representation vector The component in the y direction, Representation vector The component in the x direction, Representation vector Component in the y direction.

4. Get the mask area and process it:

In order to avoid the detection area with tilt angle and the non-optical pass area affecting the detection efficiency, it is necessary to obtain the grayscale mask area without tilt angle, create a black image with the same resolution as the grayscale image, draw the simulated optical pass on the black image with the indented coordinates as the vertex, and fill the indented coordinate area of the black image with white, that is, fill the simulated optical pass with white (pixel value is 255), as shown in Figure 9, use the polygonal circumscribed rectangle algorithm for the indented coordinates of the black image to obtain the four vertices of the circumscribed rectangle of the indented coordinates. The circumscribed rectangle is a rectangle without tilt angle, as shown in the following formula:

As shown in FIG17 , the bounding rectangle is obtained by finding the maximum or minimum coordinate value in each direction of the quadrilateral. 0, 1, 2, and 3 are the four vertices of the indented coordinates. The formula for obtaining the boundary is:

Where left is the left boundary of the bounding rectangle, right is the right boundary of the bounding rectangle, top is the upper boundary of the bounding rectangle, and bottom is the lower boundary of the bounding rectangle. is the minimum x- coordinate value of all coordinates in the indented coordinate area. is the minimum y coordinate value of all coordinates in the indented coordinate area. is the maximum x- coordinate value of all coordinates in the indented coordinate area. The maximum y coordinate value among all coordinates in the indented coordinate area;

The circumscribed rectangular area is the black image mask area as shown in FIG. 9 , which is a non-tilt angle area. The pixels in the black image mask area are traversed, and the traversed black pixels are accumulated. If the ratio of the sum of the traversed black pixels to the sum of the total pixels in the black image mask area is less than the fifth set value, the mask area is judged to be qualified. Preferably, the fifth set value is 0.1, 0.2, 0.3, 0.4 or 0.5, etc., but can be set according to actual needs;

When preprocessing grayscale images, the light source will affect image areas at different angles. Images in the middle direct range will show higher brightness, while images in the edge oblique range will show lower brightness. Due to the uneven brightness, rough binarization processing can easily lead to misjudgment of image defects. Therefore, it is necessary to divide the image into different local areas, and the illumination angle of each local area is almost the same to minimize the uneven brightness problem.

As shown in FIG10 , the non-mask area of the grayscale image is the non-optical pass area. The grayscale image mask area is intercepted for detection to avoid the non-optical pass area affecting the detection efficiency. The existing integral threshold algorithm is used for the grayscale image mask area. First, the grayscale image mask area is Gaussian blurred, and the Gaussian blur standard deviation value sigma is set. The boundary extension algorithm (the algorithm provided by Opencv) is used to copy the grayscale image mask area boundary.

Calculate the pixel sum of the integral kernel in the integral image: As shown in Figure 18, the pixel sum of the integral kernel in the integral image is calculated by the relative positions of the four vertices of the grayscale image mask area to the coordinate origin; use OpenCV's anti-boundary mode to expand the image area boundary, sum and sqsum represent the integral area of the image after boundary expansion (i.e., integral image) and the integral square area of the image after boundary expansion (i.e., square integral image), respectively, and obtain the cumulative sum of the pixels in the area enclosed by the four vertices of the integral area and the background image coordinate origin, as shown in the following formula:

In the formula, and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper left corner of the integral area and the origin of the coordinates. and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper right corner of the integral area and the origin of the coordinates. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower left corner of the integration area and the coordinate origin. and All represent the sum of pixels in the area enclosed by the coordinates of the lower right corner of the integration area and the coordinate origin, i is the sum of the first i pixels in the X direction of the integration area, j is the sum of the first j pixels in the Y direction of the integration area, ker is the size of the integration kernel, is the pixel sum;

Similarly, calculate the cumulative sum of pixels in the area enclosed by the four vertices of the integral square area and the origin of the background coordinate system, as follows:

In the formula, and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper left corner of the integral square area and the coordinate origin. and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper right corner of the integral square area and the origin of the coordinate system. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower left corner of the integral square area and the origin of the coordinate system. and All represent the sum of pixels in the area enclosed by the coordinates of the lower right corner of the integral square area and the coordinate origin, i is the sum of the first i pixels in the X direction of the integral square area, j is the sum of the first j pixels in the Y direction of the integral square area, ker is the integral kernel size, is the sum of square pixels;

By integrating the pixels in the area and obtaining the pixel mean , as follows:

And get the pixel variance var as follows:

Calculate the integral binarization threshold as follows:

The grayscale image mask area is binarized pixel by pixel through the integral binarization threshold to obtain the mask integral binary image, as shown in the following formula:

in is the grayscale mask area The pixel value of the coordinate point, is the integral binarization threshold calculated for the corresponding pixel value, is the pixel value after binarization;

When performing integral binarization, the pixel sum, square pixel sum, pixel mean and pixel variance within the integral kernel are calculated. The integral map and integral square map are used to obtain information of multiple pixels in one calculation, speed up the calculation of local statistical information, improve performance, avoid repeated calculations, reduce calculation complexity, and use accumulated pixel values to enhance accuracy.

The connected domain algorithm is used to calculate the area of the connected domain in the mask integral binary image, as shown in Figure 11. The extremely small features (i.e., the extremely small connected domain) in the mask integral binary image that do not affect the quality of the optical pass film (i.e., the area of the connected domain is less than the first set area) are removed (i.e., impurities are removed), and the pixels in the mask integral binary image except the indented coordinate area are set to 0 (i.e., interference factors are removed) to avoid the burrs of the optical pass film and the non-optical pass film area in the mask area affecting the detection; the Sobel operator is used to obtain the total gradient of the mask integral binary image and generate a mask gradient image, so that the mask integral binary image only displays pixel intensity changes. rate; binarize the mask gradient map using the parameter threshold method, divide the mask gradient map into the foreground and the background, and obtain the mask parameter contour map; binarize the mask gradient map using the parameter threshold method, divide the mask gradient map into the foreground and the background, and obtain the mask parameter contour map; binarize the mask parameter contour map using the adaptive threshold method, divide the mask parameter contour map into the edge area and the non-edge area, and obtain the mask edge contour map; calculate the total gradient of the mask edge contour map using the Sobel operator again and binarize it using the parameter threshold method (i.e., preprocessing) to obtain the mask edge map;

5. Obtain defect information and mark it:

Using a connected domain algorithm to obtain the connected domain information inside the mask edge map, the connected domain information inside the mask edge map includes the area, number, width and height of the connected domain inside the mask edge map, and judging the defects of the mask edge map (i.e. judging the surface defects of the optical pass sheet according to the set conditions) by comparing the connected domain information with the set conditions (i.e. setting parameters);

Among them, larger area defects such as point damage are obtained by comparing the connected domain area obtained by mask edge map detection with the area calculated by the sixth set diameter:

In the formula, Indicates whether there is a point defect in the i - th mask edge image. =1 indicates that there is a point injury. =0 means there is no point damage; is the area of the jth connected domain in the ith mask edge graph, is the sixth set diameter, preferably, the sixth set diameter is 5, 6, 7, 8 or 9 pixels, but can be set according to actual needs, n is the number of connected domains in the i- th mask edge map;

Among them, the defects with more spots such as pits are detected by comparing the number of connected domains obtained by the mask edge map detection with the seventh set number, and the number of connected domains belonging to all defects is counted:

In the formula, Indicates that there is no pit defect in the i- th mask edge image, indicates that there is a pit defect in the i- th mask edge graph, N is the seventh set number of pits, n is the number of connected domains in the i -th mask edge graph, and preferably, the seventh set number of pits may be 2, 3, 4 or 5 pits;

Among them, the detection of relatively long and thin defects such as scratches is determined by the ratio of the connected domain width to the connected domain height and the maximum edge value of the connected domain. The first step is to determine whether it is a long and thin connected domain, based on whether the ratio of the connected domain width to the height is greater than the eighth set value or less than the ninth set value, wherein the eighth set value is greater than the ninth set value. If the ratio of the mask edge map connected domain width to the height is greater than the eighth set value or less than the ninth set value, it is a long and thin connected domain, as shown in the following formula:

In the formula, is the width of the jth connected domain of the ith mask edge graph, The height of the jth connected domain of the ith mask edge graph, n is the number of connected domains in the ith mask edge graph, is the eighth setting value, is the ninth setting value, , indicating that the connected domain does not belong to the elongated region. When , it indicates that the connected domain belongs to a slender area. Preferably, the ninth setting value is 2/17, 1/16, 1/15, 2/15, 1/14 or 2/13, and the eighth setting value is 12, 13, 14, 15, 16 or 17, but it can be set according to actual needs;

Step 2: Determine whether the maximum edge value of the connected domain is greater than the tenth set value (the larger value of the width and height of the slender defect or scratch is the maximum edge value of the connected domain):

In the formula is the maximum edge value of the connected domain, is the tenth setting value, When , it means that the connected domain does not belong to the scratch defect. When , it indicates that the connected domain belongs to a scratch defect. Preferably, the tenth setting value is 13, 14, 15, 16 or 17 pixels.

After counting all the mask edge images that are defects (spots, pits, scratches, etc.), record the center coordinates of the mask edge image, and mark the corresponding coordinate points in the original image to complete the optical film defect detection.

The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the scope of protection of the invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on these embodiments, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in this field can still combine, add, delete or make other adjustments to the features in the various embodiments of the present invention according to the circumstances without conflict, without making creative work, so as to obtain different other technical solutions that do not deviate from the concept of the present invention in essence, and these technical solutions also belong to the scope of protection of the present invention.

Claims (8)

1. A method for detecting defects in optical wafers, comprising the following steps: Step 1: Obtain the original image with the optical pass film through an image acquisition device, and copy the original image into a grayscale image; Step 2: Preprocess the grayscale image, obtain the coordinates of the contour vertices through contour detection and angle cosine value screening, and calculate the coordinates of the contour center, contour width, contour height and contour inclination angle; specifically: use Gaussian blur processing on the grayscale image to reduce the noise of the grayscale image and make the grayscale image smooth, in preparation for feature extraction; use the Sobel operator to calculate the total gradient of the grayscale image and generate the total gradient map of the grayscale image, so that the grayscale image only displays the pixel intensity change rate, and then use the parameter threshold method to binarize the total gradient map of the grayscale image, and divide the total gradient map of the grayscale image into foreground and background, that is, obtain the parameter contour map; then use the OpenCV adaptive threshold method to binarize the parameter contour map, divide the parameter contour map into edge area and non-edge area, so that the contour in the parameter contour map becomes a single pixel display, that is, obtain the edge contour map; calculate the area of the connected domain detected in the edge contour map and compare it with the first set area, and delete the area in the edge contour map where the connected domain area is smaller than the first set area; use the Sobel operator to calculate Calculate the total gradient of the edge contour map and perform binarization through the parameter threshold method to obtain the parameter edge contour map; use the edge search algorithm to obtain the edge of the connected area in the parameter edge contour map, perform polygonal approximation on the edge of the connected area, and obtain the connected area contour; remove the connected area contours except the convex quadrilateral contour in the connected area contour, calculate the cosine value of the angle between the vertex coordinates of the convex quadrilateral contour, select the maximum value of the cosine value of the angle formed by the vertex coordinates of the convex quadrilateral contour and compare it with the second set value, and if it is less than the second set value, it is judged that the shape of the convex quadrilateral is close to a rectangle, screen out the convex quadrilateral contour with a shape close to a rectangle, and use the circumscribed inclined rectangle algorithm to calculate the vertices of the circumscribed inclined rectangle of the convex quadrilateral contour, that is, the coordinates of the contour vertices; calculate the contour center coordinates through the contour vertex coordinates, compare the contour vertex coordinates with the contour center coordinates, and rearrange them clockwise; perform inverse trigonometric function calculation on the angle between the first two points sorted by the contour vertex coordinates and the X-coordinate axis to obtain the contour inclination angle, and calculate the contour width and contour height; Step 3: Use Blob detection to obtain the center coordinates of the Blob in the grayscale image, compare the center coordinates of the Blob with the center coordinates of the contour, combine the center coordinates of the Blob with the contour information, supplement the coordinates of the contour vertices, obtain the supplementary coordinates, and use the shrinkage algorithm to obtain the shrinkage coordinates on the supplementary coordinates; specifically: the Blob detector sets the detection style so that the Blob detector only detects quadrilateral style patches; the Blob detector sets the detection area so that the Blob detector screens out patches outside the third set area range; use the configured Blob detector to directly detect the grayscale image to obtain the center coordinates of the Blob; calculate the distance dis between the center coordinates of the Blob and the center coordinates of the contour using the Euclidean distance formula When dis is less than or equal to the fourth setting value, the Blob center coordinates and the contour center coordinates are the center points of the same light pass sheet, and no supplement is required; when dis is greater than the fourth setting value, it is considered that the Blob center coordinates are inconsistent with the contour center coordinates. At this time, the light pass sheet obtained by the Blob detection needs to be supplemented to the light pass sheet obtained by the contour detection to obtain supplementary coordinates; to prevent the light pass sheet at the edge of the image from being misdetected, the supplementary coordinates that exceed the edge of the grayscale image are removed; and the supplementary coordinates within the edge of the grayscale image are subjected to polygonal indentation using the indentation algorithm to obtain the indentation coordinates; Step 4: Create a black image with the same resolution as the grayscale image, use a polygonal circumscribed rectangle algorithm for the indented coordinates of the black image to obtain the circumscribed rectangular area of the indented coordinates, namely the black image mask area, perform integral binarization and preprocessing on the grayscale image mask area with the same coordinates as the black image mask area to generate a mask edge map; specifically: create a black image with the same resolution as the grayscale image, draw a simulated light pass film on the black image with the indented coordinates as vertices, and fill the indented coordinate area of the black image with white; use a polygonal circumscribed rectangle algorithm for the indented coordinates to obtain the four vertices of the circumscribed rectangle of the indented coordinates, and the circumscribed rectangular area surrounded by the four vertices is the black image mask area, traverse the pixels in the black image mask area, and if the ratio of the black pixel sum to the total pixel sum in the black image mask area is less than the fifth set value, the black image mask area is judged to be qualified; for the grayscale image mask area with the same coordinates as the black image mask area, use the calculated integral map and square integral map to calculate the pixel sum and square pixel sum of all positions , pixel mean and pixel variance, obtain information of multiple pixels in one calculation, and use the accumulated pixel values for calculation, complete the integral binarization to obtain the mask integral binary map; use the connected domain algorithm to obtain the area of each connected domain on the mask integral binary map, the connected domain area less than the first set area is the minimum connected domain and the minimum connected domain is removed, that is, remove impurities; set the pixels in the mask integral binary map except the indented coordinate area to 0, that is, remove interference factors; use the Sobel operator to obtain the total gradient of the mask integral binary map and generate a mask gradient map, so that the mask integral binary map only displays the pixel intensity change rate; use the parameter threshold method to binarize the mask gradient map, divide the mask gradient map into foreground and background, that is, obtain the mask parameter contour map; use the adaptive threshold method to binarize the mask parameter contour map, divide the mask parameter contour map into edge area and non-edge area, and obtain the mask edge contour map; use the Sobel operator to calculate the total gradient of the mask edge contour map again and binarize it by the parameter threshold method to obtain the mask edge map; Step 5: Perform defect detection on the mask edge map to obtain defect information, and then mark the defects in the original image; specifically: use the connected domain algorithm on the mask edge map to obtain the internal connected domain information of the mask edge map. If the connected domain information exceeds the corresponding set parameters, it is judged that there is a defect in the mask edge map, and the coordinates of the defective mask edge map, that is, the defect coordinates, are recorded, and marked in the original image according to the defect coordinates. 2. The optical fiber defect detection method according to claim 1, characterized in that step 2 further comprises the following steps: According to the approximation algorithm, polygonal approximation is performed on the edge of the connected domain to obtain the connected domain contour, and the connected domain contour except the convex quadrilateral contour is removed from the connected domain contour, and the coordinates of the four vertices surrounding the convex quadrilateral contour are calculated, that is, the coordinates of the four contour vertices of the convex quadrilateral are obtained; The cosine values of the angles between the vertices of the convex quadrilateral outline are calculated, and the cosine values of the angles between the two vectors formed by three adjacent points are calculated and the maximum value is selected to determine whether the shape of the convex quadrilateral is close to a rectangle. The specific formula is as follows: In the formula is the coordinate of the ith vertex of the convex quadrilateral, is the x- coordinate of the ith vertex of the convex quadrilateral, is the y coordinate of the i- th vertex of the convex quadrilateral; are the coordinates of the i +1th vertex of the convex quadrilateral, is the x- coordinate of the i+ 1th vertex of the convex quadrilateral, is the y coordinate of the i+ 1th vertex of the convex quadrilateral; , are the coordinates of the i +2th vertex of the convex quadrilateral, is the x- coordinate of the i+ 2nd vertex of the convex quadrilateral, is the y coordinate of the i+ 2nd vertex of the convex quadrilateral; vector For point Pointing Point vector, vector For point Pointing Point The vector of For vector and vector The angle between two vectors, is the maximum cosine value calculated, point With point for the same point; like If the value of is less than the second set value, it is considered that the shape of the convex quadrilateral is close to a rectangle, and the contour of the convex quadrilateral is the contour of the light pass sheet, which meets the requirements. 3. The optical fiber defect detection method according to claim 2, characterized in that: The contour inclination angle is calculated according to the contour vertex coordinates, as follows: In the formula, is the profile inclination angle, is the number of contours, is the coordinate of the first point of the i -th contour, is the x- coordinate of the first point of the i -th contour, is the y coordinate of the first point of the i -th contour; is the coordinate of the second point of the i -th contour, is the x- coordinate of the second point of the i - th contour, is the y coordinate of the second point of the i - th contour; The contour width and height are calculated according to the contour vertex coordinates using the Euclidean distance formula, as shown below: In the formula, is the outline width, is the profile height, is the number of contours, is the coordinate of the first point of the i -th light channel contour, is the x- coordinate of the first point of the i -th contour, is the y coordinate of the first point of the i -th contour; is the coordinate of the second point of the i - th light channel contour, is the x- coordinate of the second point of the i - th contour, is the y coordinate of the second point of the i - th contour; is the coordinate of the third point of the i - th light channel contour, is the x- coordinate of the third point of the i - th contour, is the y coordinate of the third point of the i - th contour. 4. The optical sheet defect detection method according to claim 2, wherein obtaining the coordinates of four contour vertices in step 2 comprises the following steps: The cosine value of the angle is calculated using three adjacent points of the detected vertices of the convex quadrilateral outline, and the maximum cosine value of the three or four corners of the convex quadrilateral outline is compared with the second set value. If the maximum cosine value of the convex quadrilateral outline is less than the second set value, it is judged that the shape of the convex quadrilateral is close to a rectangle; the inclination angle of each convex quadrilateral outline is calculated by using the inverse trigonometric function of the upper left corner vertex and the upper right corner vertex of each convex quadrilateral outline, the sum of the inclination angles of all convex quadrilateral outlines is calculated, and then the average value is calculated. 5. The optical pass defect detection method according to claim 1 is characterized in that the optical pass obtained by Blob detection is added to the optical pass obtained by contour detection in step 3, specifically: the center coordinates of the Blob are used as the center of the rectangle, and the contour inclination angle is used as the center of the rectangle. The tilt angle of the rectangle and the width of the outline is the rectangle width, and the outline height Draw a Blob rectangle for the rectangle height and get the coordinates of the Blob rectangle vertices; Sort the Blob rectangle vertex coordinates clockwise, and add the sorted Blob rectangle vertex coordinates to the contour vertex coordinate points to obtain the supplementary coordinates; The supplementary coordinates within the edge of the grayscale image are indented to obtain the indented coordinates, as shown in the following formula: in In the formula are supplementary coordinates, is the indentation coordinate, L is the indentation distance, , is a vector consisting of a supplementary coordinate and two adjacent supplementary coordinates. for The modulus of a vector, for The magnitude of a vector, is the angle between the two sides of the supplementary coordinates, Representation vector The component in the x direction, Representation vector The component in the y direction, Representation vector The component in the x direction, Representation vector Component in the y direction. 6. The optical fiber defect detection method according to claim 1, characterized in that: The polygonal circumscribed rectangle algorithm is used to obtain the four vertices of the circumscribed rectangle of the indented coordinates of the black image, as shown below: Where left is the left boundary of the bounding rectangle, right is the right boundary of the bounding rectangle, top is the upper boundary of the bounding rectangle, and bottom is the lower boundary of the bounding rectangle. is the minimum x- coordinate value of all coordinates in the indented coordinate area. is the minimum y coordinate value of all coordinates in the indented coordinate area. is the maximum x- coordinate value of all coordinates in the indented coordinate area. The maximum y coordinate value among all coordinates in the indented coordinate area; The black image mask area is a black image mask area without an inclination angle, pixels in the black image mask area are traversed, the traversed black pixels are accumulated, and a ratio of the sum of black pixels in the black image mask area to the sum of total pixels is obtained. If the ratio is less than a fifth set value, the mask area is judged to be qualified; The grayscale image mask area image processing uses an integral threshold algorithm to perform Gaussian blur processing on the grayscale image mask area, sets the Gaussian blur standard deviation value sigma, and uses the boundary extension algorithm to copy the grayscale image mask area boundary; Calculate the pixel sum of the integral kernel in the integral image: Use the anti-boundary mode to expand the grayscale mask area boundary. sum represents the integral area of the image after boundary expansion. sqsum represents the integral square area of the image after boundary expansion. Get the cumulative sum of the pixels in the area enclosed by the four vertices of the integral area and the origin of the background image coordinates, as shown in the following formula: In the formula, and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper left corner of the integral area and the origin of the coordinates . and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper right corner of the integral area and the origin of the coordinates . and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower left corner of the integration area and the coordinate origin . and All represent the sum of pixels in the area enclosed by the coordinates of the lower right corner of the integration area and the coordinate origin, i is the sum of the first i pixels in the X direction of the integration area, j is the sum of the first j pixels in the Y direction of the integration area, ker is the size of the integration kernel, is the pixel sum; Similarly, calculate the cumulative sum of pixels in the area enclosed by the four vertices of the integral square area and the origin of the background coordinate system, as follows: In the formula, and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper left corner of the integral square area and the coordinate origin. and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper right corner of the integral square area and the origin of the coordinate system. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower left corner of the integral square area and the origin of the coordinate system . and All represent the sum of pixels in the area enclosed by the coordinates of the lower right corner of the integral square area and the coordinate origin, i is the sum of the first i pixels in the X direction of the integral square area, j is the sum of the first j pixels in the Y direction of the integral square area, ker is the integral kernel size, is the sum of square pixels; By integrating the pixels in the area and obtaining the pixel mean , as follows: And get the pixel variance var as follows: Calculate the integral binarization threshold th as follows: The grayscale image mask area is binarized pixel by pixel through the integral binarization threshold to obtain the mask integral binary image, as shown in the following formula: in: is the grayscale mask area The pixel value of the coordinate point, is the integral binarization threshold calculated for the corresponding pixel value, is the pixel value after binarization; The removing of the extremely small connected domain whose area is smaller than the first set area is to set the pixels in the extremely small connected domain whose area is smaller than the first set area to 0. 7. The optical pass sheet defect detection method according to claim 6 is characterized in that the resolution of the black image is the same as the resolution of the grayscale image, that is, the grayscale image mask area has the same coordinates as the black image mask area, and an integral binarization threshold calculation is performed on the grayscale image mask area with no tilt angle having the same coordinates as the black image mask area with no tilt angle. When calculating the pixel variance, a value greater than 0 is taken for the pixel variance. If the pixel variance is less than 0, the pixel variance is taken as 0. When binarization is performed, the calculated integral binarization threshold is used to binarize the grayscale image mask area pixel by pixel. 8. The optical wafer defect detection method according to claim 1, characterized in that: performing defect detection on the mask edge image and then marking the defects in the original image comprises the following steps: 5.1. Using a connected domain algorithm to obtain the connected domain information inside the mask edge map, the connected domain information inside the mask edge map includes the area, number, width and height of the connected domain inside the mask edge map; 5.2. Point defect detection: compare the area of the connected domain obtained by the mask edge map detection with the area calculated by the sixth set diameter. When the area of the connected domain is larger than the area calculated by the sixth set diameter, it is considered that there is a point defect in the mask edge map; 5.3. Pockmark defect detection: compare the number of connected domains detected by the mask edge map with the set number. If the number of connected domains is greater than the seventh set number of pockmarks, it is considered that the mask edge map has a pockmark defect; 5.4. Slender defect detection, step 1: whether the ratio of the width to the height of the connected domain of the mask edge map is greater than the eighth set value or less than the ninth set value, wherein the eighth set value is greater than the ninth set value, if the ratio of the width to the height of the connected domain of the mask edge map is greater than the eighth set value or less than the ninth set value, it is a slender connected domain, step 2: according to the comparison between the maximum edge value of the connected domain and the tenth set value, when the maximum edge value of the connected domain is greater than the tenth set value, it is considered that the mask edge map has a slender defect; the larger value of the width and height of the connected domain is the maximum edge value of the connected domain; 5.5. After detection, store the center coordinates of all mask edge images that are judged to be defects, and mark the corresponding coordinate points in the original image.