CN113538392B - Wafer detection method, equipment and storage medium - Google Patents

Abstract

The application provides a wafer detection method, which comprises the following steps: acquiring a first image of the wafer, wherein the first image comprises an overall characteristic representing defect information of the wafer; determining a second image of the wafer based on the first image, the second image including detail features representing defect information of the wafer; fusing the overall features and the detail features to generate fused features; and detecting the fused features through the wafer defect classification model. By the method, the accuracy of wafer defect detection can be improved to a certain extent, and the labor cost is reduced.

Description

Wafer detection method, equipment and storage medium

technical field

The present application relates to the field of semiconductors, and more specifically, to a wafer defect detection method, device and storage medium.

Background technique

With the development of memory, the requirements for the degree of integration of the memory are getting higher and higher, so the feature size of the wafer is continuously reduced, and the defect detection of the wafer has become an important method to improve the yield of the wafer. The production process of memory is various and complicated. After each process, especially after the etching process and deposition process, it is necessary to detect the defects on the surface of the wafer to prevent the defective wafer from flowing to the subsequent process and affecting the electrical properties of the wafer. performance.

At present, the defect detection of wafers is mainly performed by using an Auto Defect Classification (ADC for short) system combined with a machine learning method. The ADC system mainly has two modules: a detection module and a classification module. The detection module scans the surface of the wafer to confirm the coordinates of the area where defects may exist, and then uses a scanning electron microscope (SEM) to obtain the image of the defect area; the classification module uses machine learning methods to build a classification model to achieve Classification of defects. However, the defect detection performance of current ADC systems is high, while the accuracy of defect classification is poor. Therefore, it is necessary to confirm the defect classification results through manual inspection, and professionals need to mark a large number of defect samples. Therefore, the correct rate of wafer inspection and the saving of manpower input are problems that need to be solved quickly.

Contents of the invention

Embodiments of the present application provide a wafer detection method and system that can at least partially solve the above-mentioned problems in the prior art.

According to an aspect of an embodiment of the present application, a method for inspecting a wafer is provided, the method may include: acquiring a first image of the wafer, the first image including overall features representing defect information of the wafer ; determining a second image of the wafer based on the first image, the second image including detailed features representing defect information of the wafer; fusing the overall features with the detailed features to generate the fused features; and detecting the fused features through a wafer defect classification model.

In one embodiment of the present application, the step of determining the second image of the wafer based on the first image may include: locating defect feature regions in the first image; The region is intercepted to obtain the target image as the second image.

In one embodiment of the present application, the step of determining the second image of the wafer based on the first image may include: locating the detailed region of the defect feature in the target image; and locating the located The detail area is intercepted to obtain a detail image as the second image.

In one embodiment of the present application, the step of intercepting the region of the located defect feature to obtain the target image as the second image may include: extracting the first image of the wafer based on a deep convolutional neural network. The depth feature map of the image; determine the average value of the features in each position channel on the depth feature map and the average value of the features in the overall channel of the depth feature map; confirm the position channel in the depth feature map The region whose average value of the feature is greater than the average value of the feature in the whole channel is used as the target defect area; an image of the target defect area is intercepted as the target image.

In one embodiment of the present application, after confirming that the average value of the features in the position channel in the depth feature map is greater than the average value of the features in the overall channel as the target defect area, it may include: determining the target defect area The minimum enclosing rectangle and confirm its coordinates; determine the position coordinates of the target defect area in the first image of the wafer by deconvolution; and intercept the first image of the wafer according to the position coordinates, to as the target image.

In one embodiment of the present application, the step of intercepting the located detail region to obtain the detail image as the second image may include: extracting the depth feature map of the target image based on a deep convolutional neural network; determining The average value of the features of each position channel on the depth feature map of the target image; select a sliding window to convolve the target image, and confirm at least one activation window according to the sliding window, the depth feature of the activation window The average value of the features of the map channel is greater than the average value of the features of each position channel on the depth feature map; and intercepting the target image corresponding to the at least one activation window as the detail image.

In one embodiment of the present application, selecting a sliding window to convolve the target image, and after confirming the activation window according to the sliding window, the step of intercepting the located detail region to obtain the detail image may further include: Selecting the area of the at least one active window as the detailed defect area of the wafer defect image by non-maximum value suppression; determining the position coordinates of the detailed defect area in the target image of the wafer by deconvolution; and intercepting the image of the detail defect region as the detail image.

In one embodiment of the present application, detecting the fused features through a wafer defect classification model may include: classifying defects of the wafer to determine a defect type of the wafer.

In one embodiment of the present application, after detecting the fused features through the wafer defect classification model, it may include: outputting a detection result, the detection result including the defect type of the wafer, and the The confidence level corresponding to the defect type of the wafer.

In one embodiment of the present application, the wafer defect classification model can be obtained through training. The wafer defect classification model includes a first classification model and a second classification model, and the training samples for the first classification model and the second classification model are different.

In one embodiment of the present application, the method further includes the step of separately training the wafer defect classification model, which may include: distinguishing the coarse-grained image and the fine-grained image of the wafer into pure samples and noise samples; and The pure samples are input to the first classification model, the noise samples are input to the second classification model, and the first classification model and the second classification model are trained respectively.

In one embodiment of the present application, the clean samples may include the coarse-grained image or the fine-grained image for confirming the defect type, for testing and verification of the wafer defect classification model; the noise samples may include The coarse-grained image or the fine-grained image of the defect type to be confirmed is used for training the wafer defect classification model.

In one embodiment of the present application, after training the first classification model and the second classification model, it further includes performing mixed training on the first classification model and the second classification model, which may include: The noise sample is divided into a labeled data set and an unlabeled data set; using the first classification model and the second classification model to extract the depth feature maps of the labeled data set and the unlabeled data set; The first classification model and the depth feature map of the same sample extracted by the second classification model are fused; the fused depth feature map is input to the classifier to obtain the detection result of the noise sample; and according to the noise sample. The detection result confirms the overall loss of the wafer defect classification model, and completes the training of the wafer defect classification model.

In one embodiment of the present application, the step of dividing the noise sample into a labeled data set and an unlabeled data set may include: inputting the coarse-grained image or the fine-grained image in the noise sample into the a first classification model and the second classification model; and dividing the noise samples into a labeled data set and an unlabeled data set according to the predicted defect types and confidence levels of the first classification model and the second classification model, Wherein the confidence degree of the marked data set is greater than a set value, and the confidence degree of the unmarked data set is less than a set value.

In one embodiment of the present application, before fusing the depth feature maps corresponding to the same sample extracted by the first classification model and the second classification model of the same sample, it may further include: inputting the depth feature maps into the full connection layer to obtain a one-dimensional depth feature map.

In one embodiment of the present application, confirming the overall loss of the wafer defect classification model according to the defect type and probability of the image may include: the marks obtained according to the first classification model and the second classification model The defect types and probabilities of the data sets are linearly combined as a collaborative fine-tuning loss; the defect types and probabilities of the unlabeled data sets estimated according to the first classification model and the second classification model are combined as a collaborative estimation Loss; regularization processing is performed on the first classification model and the second classification model to obtain the regularization loss of the first classification model and the second classification model; and the collaborative fine-tuning loss, the A co-estimation loss and the regularization loss are fused as the ensemble loss for a fine-grained classification model of the wafer defects.

In one embodiment of the present application, the first classification model and the second classification model classify the defect model of the image according to the depth feature map, and fuse the classification results to obtain the noise sample The detection result may include: using a fully connected layer to process the depth feature maps of the marked data set and the unlabeled data set to obtain a one-dimensional depth feature map; and after fusing the features of the one-dimensional depth feature map input to the classifier to obtain the detection result of the wafer.

In one embodiment of the present application, the detection result may include a confidence level corresponding to a defect type of the wafer.

Another aspect of the present application provides a wafer inspection system, the system may include: a memory, used to store program instructions; and a processor, used to communicate with the memory to execute the program instructions, so as to achieve the above any one of the methods described.

Another aspect of the present application provides a wafer inspection device, which may include the above-mentioned wafer inspection system.

In an embodiment of the present application, the detection device may further include: a detection device, the detection device is used to collect an image of the wafer.

In one embodiment of the present application, the detecting device may be used to collect a first image of the wafer and/or a second image of the wafer.

In one embodiment of the present application, the detection equipment may include at least one of the following: a computer, a server, a cellular phone, a smart phone, a wearable device, and a wafer processing equipment.

Another aspect of the present application provides a non-transitory computer-readable storage medium storing computer instructions, the computer instructions are used to cause the computer to execute any one of the methods described above.

According to the wafer detection method and equipment according to an embodiment of the present application, the surface defects of the wafer can be detected, and while paying attention to the overall defect characteristics of the wafer surface, attention can also be paid to the detailed defect characteristics of the wafer surface. In the process of wafer inspection, the accuracy of wafer defect type classification can be improved to a certain extent by combining and analyzing the overall defect characteristics and detailed defect characteristics of the wafer. And by training the model through the labeled data set and the unlabeled data set, the training of the wafer defect classification model can be completed with fewer labeled samples, which saves manpower investment to a certain extent and improves the detection efficiency of the wafer.

Description of drawings

Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings. in:

1 is a schematic flow diagram of a wafer detection method according to an embodiment of the present application;

FIG. 2 is a schematic flow diagram of acquiring a target image according to an embodiment of the present application;

FIG. 3 is a schematic flow diagram of acquiring a detailed image according to an embodiment of the present application;

4A is a schematic diagram of a coarse-grained image of a wafer according to an embodiment of the present application;

4B is a schematic diagram of a target image of a wafer according to an embodiment of the present application;

4C is a schematic diagram of a detailed image of a wafer according to an embodiment of the present application;

FIG. 5A is a schematic flow diagram of separately training a first classification model and a second classification model according to an embodiment of the present application;

FIG. 5B is a schematic flowchart of mixed training of the first classification model and the second classification model according to an embodiment of the present application;

6 is a schematic diagram of a detection system for a wafer according to an embodiment of the present application; and

FIG. 7 is a schematic diagram of a testing device for a wafer according to an embodiment of the present application.

Detailed ways

The defect detection process on the wafer surface is an important link in the memory production process. The memory production process is various and complicated. After each process, especially after the etching process and deposition process, it is necessary to detect the defects on the wafer surface. Detection to prevent defective wafers from flowing to subsequent processes and affecting the electrical properties of the wafers. The inventors found that in the traditional technology, the defect detection of wafers is mainly performed by using an Auto Defect Classification (ADC for short) system combined with a machine learning method, and the accuracy of defect classification is very poor. The inventor found through creative labor that in traditional technologies, machine learning methods are mainly used to extract image features of defects, and then these features are combined and input into a classifier to obtain classification results. This method is mainly based on shallow features of images , such as the size, shape, position, color, smoothness, texture complexity, contour, etc. of the image defect area. If the image background is complex, small area defects are easily overlooked or misclassified. Commonly used classifiers include support vector machines (SVM), neural networks, decision trees, etc., often requiring manual further confirmation of the classification results. The commonly used machine learning method is deep learning based on convolutional neural network. Due to the complexity of wafer surface defects and chip manufacturing process, defect detection is carried out in layers. There are many types of defects, and the scanning size of defects in electronic scanning images is different. One, and requires professionals to mark a large number of defect samples, so the correct rate of wafer inspection and saving manpower input are problems that need to be solved quickly.

Based on this, the embodiments of the present application propose a wafer inspection method, system, device, and non-transitory computer-readable storage medium storing computer instructions. Wafer defects, such as defects on the wafer surface, can be detected. In the embodiment of the present application, while paying attention to the overall defect characteristics of the wafer, it is also possible to pay attention to the detailed defect characteristics of the wafer. To a certain extent, the accuracy of wafer defect type classification is improved. And by training the model through the labeled data set and the unlabeled data set, the training of the wafer defect classification model can be completed with fewer labeled samples, which saves manpower investment to a certain extent and improves the detection efficiency of the wafer.

For a better understanding of the application, various aspects of the application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are descriptions of exemplary embodiments of the application only, and are not intended to limit the scope of the application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and/or" includes any and all combinations of one or more of the associated listed items.

In the drawings, the size, dimensions, and shapes of elements have been slightly adjusted for illustrative purposes. The drawings are examples only and are not strictly drawn to scale. As used herein, the words "approximately," "approximately," and similar words are used as words of approximation, not of degree, and are intended to describe measurements that would be recognized by those of ordinary skill in the art. Or inherent bias in calculated values. In addition, in the present application, the order of description of the processing of each step does not necessarily indicate the order in which these processes appear in actual operation, unless there is a clear other limitation or can be deduced from the context.

It should also be understood that expressions such as "comprises", "comprises", "has", "comprises" and/or "comprising" in this specification are open rather than closed expressions, which mean that there are all The stated features, elements and/or components do not exclude the presence of one or more other features, elements, components and/or combinations thereof. Furthermore, expressions such as "at least one of," when preceding a list of listed features, modify the entire list of features and do not modify just the individual elements of the list. In addition, when describing the embodiments of the present application, the use of "may" means "one or more embodiments of the present application". Also, the word "exemplary" is intended to mean an example or illustration.

Unless otherwise defined, all terms (including engineering terms and scientific and technical terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It should also be understood that unless there is an explicit statement in this application, words defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of related technologies, and should not be idealized or overly Formal meaning interpretation.

It should be noted that, in the case of no conflict, the implementations in the present application and the features in the implementations can be combined with each other. Hereinafter, the present application will be described in detail with reference to the accompanying drawings and in combination with embodiments.

FIG. 1 is a schematic flowchart of a wafer inspection method according to an embodiment of the present application. As shown in FIG. 1 , the present application provides a wafer inspection method 1000, including:

Step S110: Acquiring a first image of the wafer, the first image includes an overall feature representing defect information of the wafer;

Step S120: determining a second image of the wafer based on the first image of the wafer, the second image including detailed features representing defect information of the wafer;

Step S130: Fusing the overall feature and the detail feature to generate a fused feature;

Step S140: Detect the fused features through the wafer defect classification model.

The specific process of each step of the above-mentioned manufacturing method 1000 will be described in detail below with reference to FIG. 2 to FIG. 5 .

Step S110: Acquiring a first image of the wafer, the first image includes an overall feature representing defect information of the wafer;

Firstly, a first image of the wafer may be obtained by scanning the wafer, and the first image includes overall features representing defect information of the wafer. For example, the wafer may be scanned by a scanning electron microscope (Scanning Electron Microscope, referred to as SEM) to obtain a first image of the wafer. The first image may include a coarse-grained image of the wafer. In this embodiment, the first image is a coarse-grained image of the wafer as an example for description. The coarse-grained image of the wafer may include the overall features of the wafer, and the overall features of the wafer may include feature information such as position, shape, color, etc. of defect regions. Wafer defects may include particles (liquid, solid particles, etc.), defects such as surface scratches, irregular connections, and the like. The wafer can be a wafer after any process steps, and the image of the wafer can be obtained through an image acquisition device, wherein the image acquisition device can be a camera, a wafer inspection system (wafer inspection system, referred to as WIS) with its own shooting equipment, etc., this paper The application is not limited to this, and the coarse-grained image of the wafer can be as shown in FIG. 4A .

Step S120: Determine a second image of the wafer based on the first image of the wafer, the second image contains The detailed characteristics of the defect information;

Since the size of the defect area in the scanned image of the wafer is different, in order to determine the defect type of the wafer more accurately, it is necessary to obtain a second image of the wafer, and the second image may include detailed features representing the wafer. The second image may include a fine-grained image of the wafer. In this embodiment, description is made by taking the second image as an example of a fine-grained image of a wafer. The fine-grained image of the wafer contains detailed features used to represent the defect information of the wafer. Fine-grained image classification can also be called subcategory image classification, and its purpose can be to divide the large categories of coarse-grained images into more detailed subcategories. Firstly, the defect feature area in the coarse-grained image of the wafer is located, and the target image is obtained as a fine-grained image by intercepting according to the located defect feature area. The fine-grained image of the wafer is acquired on the coarse-grained image of the wafer based on an attention mechanism. The fine-grained image of the wafer may include a target image, and the target image may contain features for representing defect information of the wafer. Taking FIG. 4A as an example, the target image of the wafer captured by the coarse-grained image may be as shown in FIG. 4B . Fig. 2 is a schematic flow chart of obtaining a target image according to an embodiment of the present application. As shown in Fig. 2, the specific steps for obtaining a target image are as follows:

First, in step S1211, the depth feature map of the coarse-grained image of the wafer may be extracted based on a deep convolutional neural network. After the coarse-grained image of the wafer is convolved by a convolution kernel, multiple depth feature maps can be obtained. Each convolution kernel can extract specific features, and different convolution kernels can extract different features. For example, use one convolution kernel to extract the contour features of the defect area in the coarse-grained image, use another convolution kernel to extract the grayscale features of the defect area in the coarse-grained image, and use another convolution kernel to extract the position of the defect area in the coarse-grained image features etc. Those skilled in the art know that the above extracted features are illustrative, and the depth feature maps extracted in the present application are not limited thereto.

Then in step S1212, the average value of the features in each position channel on the depth feature map and the average value of the features in the overall channel of the depth feature map are calculated. After the coarse-grained image of the wafer is extracted by a deep convolutional neural network, the average value of each position channel in the depth feature map of the wafer and the average value of the overall channel of the depth feature map are calculated. For example, the depth feature map of the wafer is a feature map of x*y, then each depth feature map of the wafer contains x*y positions, where each depth feature map corresponds to a channel, and the same position in different channels is averaged The value and the average of all channels of the depth feature map.

Then, in step S1213, it is confirmed that the region in which the average value of the features in the position channel in the depth feature map is greater than the average value of the features in the overall channel is taken as the target defect area; and in step S1214, an image of the target defect area is intercepted as the target image. In one embodiment, the average value of features in each position channel in the depth profile of the wafer can be compared with the average value of the overall channel of the depth profile of the wafer, and the feature average value in the position channels in the depth profile The area whose value is greater than the average value of the features in the overall channel is used as the target defect area, and the outer rectangle is calculated for this area and the coordinates of the outer rectangle are confirmed, and the coarse-grained size of the target defect area corresponding to the outer rectangle is calculated on the wafer by deconvolution According to the position coordinates in the image, the corresponding coarse-grained image can be intercepted according to the position coordinates in the coarse-grained image, and the intercepted image can be used as the target image of the wafer. Wherein, the method of finding the enclosing rectangle for the target defect area and confirming the coordinates of the enclosing rectangle may include: calculating the minimum enclosing rectangle for the target defect area, and confirming the coordinates of the minimum enclosing rectangle. A non-maximum suppression (Non-Maximum Suppression, NMS for short) method may be used to determine the minimum enclosing rectangle of the target defect area. This is conducive to frame selection of the exact location of the target defect.

In the process of classifying wafer defects, the distinction of some defects is only subtle on the surface. In order to further improve the accuracy of the wafer defect classification model, after obtaining the target image of the wafer, the target image of the wafer can be further classified according to The attention mechanism is used to process the detailed image, and the defect type of the wafer is further confirmed through the detailed image. The fine-grained image of the wafer may also include a detail image, for example, FIG. 4C is a detail image intercepted from the target image in FIG. 4B as an example.

Fig. 3 is a schematic flow diagram of obtaining a detailed image according to an embodiment of the present application. As shown in Fig. 3, the specific steps for obtaining a detailed image are as follows:

Firstly, in step S1221, the deep feature map of the target image can be extracted based on the deep convolutional neural network. After the target image of the wafer is convolved by a convolution kernel, multiple depth feature maps can be obtained, wherein the depth feature maps of the target image can include feature information such as the position, shape, and color of the defect area of the target image.

Next in step S1222, the average value of the features of each position channel on the depth feature map of the target image is calculated. After the target image of the wafer is extracted by a deep convolutional neural network, the feature average value of each position channel in the depth feature map of the target image is calculated. The depth feature map of contains m*n positions, where each depth feature map corresponds to a channel, and the same position in different channels is averaged.

Then in step S1223, a sliding window is selected to convolve the target image, and at least one active window is confirmed according to the sliding window, and the average value of the channel features of the depth feature map of the active window is greater than the average value of the channel features. The sliding window pair is selected to slide on the target image, and the detailed area of the wafer defect area in the target image is confirmed based on the attention mechanism. Windows with different sizes and aspect ratios can be set to slide on the entire target image. For example, the size of the sliding window can be selected to be 3*3～11*11, and the number of sliding windows can be 3-7. The sliding window needs to traverse all positions of the target image, and calculate the average value of the channel features of the sliding window in the depth feature map. If the average value of the channel features of the depth feature map channel of the sliding window is greater than a certain threshold, the sliding window can be used as the activation window. . The certain threshold may include the average value of all pixels in the image. The size of the sliding window is, for example, 3*3, 6*6, and 9*9, but those skilled in the art know that the size of the window is an example, and the size of the sliding window in the present application is not limited thereto. The sliding window has a certain step size limit in the process of traversing the target image, so there may be some redundancy in the activation window, and a fixed number of window areas can be selected as the wafer defect image in the subsequent process by non-maximum suppression The detailed defect area can reduce the subsequent impact on the detection performance of the wafer defect model. Those skilled in the art can understand that the number and size of the sliding windows are only illustrative, and the number and size of the sliding windows in the present application are not limited thereto.

Then in step S1224, the target image corresponding to at least one active window is intercepted as the detail image. After confirming the position and number of active windows, the position coordinates of the detailed defect area in the target image of the wafer can be calculated by deconvolution, and at least one corresponding target image can be further intercepted as the detailed image of the wafer.

Step S130: Fusing the overall feature and the detail feature to generate a fused feature;

The coarse-grained image features of the extracted wafer are fused with the fine-grained image features of the wafer, and the fused features are output to the classifier in the wafer defect classification model for further judgment. For example, if the features representing defect information in the coarse-grained image and the fine-grained image are fused, the coarse-grained image features of the extracted wafer can be connected in series with the fine-grained image features of the wafer. As a fusion feature, the fused features can include image The size, shape, location, color, smoothness, texture complexity, and contour of the defect area.

Step S140: Detect the fused features through the wafer defect classification model.

By combining the coarse-grained image and fine-grained image of the wafer, the coarse-grained image features of the extracted wafer are fused with the fine-grained image features of the wafer, and the trained wafer defect classification model is used, Check the fused features. Therefore, not only attention is paid to the overall image of wafer defects, but also the detailed differences between defects, so that in the process of wafer inspection, by combining the analysis of the overall defect characteristics and detailed defect characteristics of the wafer, it is possible to a certain extent Improve the accuracy of wafer defect type classification. After the fused features are detected by the trained wafer defect classification model, a more accurate wafer defect classification result can be output, for example, the wafer defect classification result and the corresponding confidence level can be output.

In an embodiment of the present application, the higher the confidence level, the higher the credibility level. For example, if a defect image is classified into a certain defect category with a confidence level of 0.99, then the classification at this time is considered correct. If the obtained confidence is lower than the confidence threshold of the stuck control, the wafer defect classification model will not classify, but the coarse-grained images and/or fine-grained images with the confidence lower than the confidence threshold are placed in the unidentified In the category where the taxonomy is defined. For example, if the confidence threshold of the card control is 0.6, then for the case where the confidence is lower than 0.6, the wafer defect classification model will not classify, but the coarse-grained image and/or fine-grained image with the confidence lower than 0.6 , placed in a category that is not categorized.

Of course, the aforementioned wafer detection method does not limit the type of wafer. Both defective and non-defective wafers can use this wafer inspection method. Even if there is no defect in the wafer itself, this wafer inspection method can be used for inspection, and the inspection result and corresponding confidence level can be output. For example, when there is no defect in the wafer itself, using this wafer inspection method for inspection can output the inspection result of "no defect in the wafer" and the corresponding confidence level.

Since the training samples for training the wafer defect classification model in the above step S140 are mixed with noise samples, it is necessary to distinguish the noise samples. Based on this, the first classification model and the second classification model are introduced here to distinguish noise samples. Fig. 5A is a schematic flow diagram of separately training the first classification model and the second classification model according to an embodiment of the present application. As shown in Fig. 5A, the specific steps of separately training the first classification model and the second classification model are as follows:

Firstly, in step S1410, the coarse-grained image and the fine-grained image of the wafer are distinguished into pure samples and noise samples. Mark the defect types of the coarse-grained image and fine-grained image of the wafer, and some of the coarse-grained images and fine-grained images are calibrated by experts in this field or experienced engineers, which can be used as pure samples and can be used for subsequent detection of wafer defects Training and verification of the classification model; another part of the coarse-grained images and fine-grained images that are not marked or marked by front-line operators can be used as noise samples. Due to certain differences in the level of front-line operators, there may be wrong marks. Therefore, noise samples that are not marked or marked by front-line operators can be used for subsequent training of the wafer defect classification model. Therefore, the wafer defect classification model can be trained with a small amount of pure samples and a certain amount of noise samples to reduce manpower input, improve the accuracy of wafer defect type classification to a certain extent, and reduce production costs.

Then in step S1420, the pure samples are input into the first classification model, and the noise samples are input into the second classification model, and the first classification model and the second classification model are trained respectively. The pure samples are input to the first classification model, and the noise samples are input to the second classification model. Since the input samples of the first classification model and the second classification model are different, the trained first classification model and the second classification model contain The parameters are also different, and the same or different results may be obtained in subsequent wafer inspections. The introduction of the first classification model and the second classification model can effectively eliminate the interference of noise samples on wafer defect classification on the whole. Considering that this embodiment adopts a data-driven modeling method, and there are a large number of noise samples in the defect classification task, using this method, in the process of model training, under the condition of ensuring the classification performance of the model, Effectively reduce the workload of manual verification samples.

Fig. 5B is a schematic flow diagram of the hybrid training of the first classification model and the second classification model according to an embodiment of the present application. As shown in Fig. 5B, the specific steps of the hybrid training of the first classification model and the second classification model are as follows:

First, in step S1421, the noise samples are divided into labeled datasets and unlabeled datasets. The obtained wafer image is a noise sample, including a coarse-grained image and a fine-grained image of the wafer, and the coarse-grained image and the fine-grained image of the wafer are input into the first classification model and the second classification model, and the first classification model and The second classification model predicts the defect type of the wafer, and obtains the predicted defect type and its confidence level. Then, the Gaussian mixture model can be used to mix the sample distribution, try to simulate the real distribution of the wafer defect model, and help to distinguish between the labeled data set and the unlabeled data set. Partition labeled and unlabeled datasets based on confidence. The confidence of the labeled data set is greater than the set value, and the confidence of the unlabeled data set is less than the set value. The set value can be adjusted according to the engineer’s experience. For example, if the set value is 0.8, the confidence of the wafer’s defect type greater than 0.8 can be regarded as marked data, and the confidence of the wafer’s defect type is less than 0.8. is unlabeled data.

Then in step S1422, use the first classification model and the second classification model to extract the depth feature maps of the marked data set and the unmarked data set; in step S1423, the depth of the same sample extracted by the first classification model and the second classification model The feature maps are fused; and in step S1424, the fused depth feature maps are input to the classifier to obtain the detection results of the noise samples. The same noise sample is input into the first classification model and the second classification model, the first classification model and the second classification model extract the depth feature map for the same noise sample, and the depth feature map extracted by the first classification model and the second classification model The depth feature maps extracted by the model are fused, and the fused depth feature maps are input to the classifier, and the classifier obtains the detection results of noise samples by judging the fused depth feature maps. Among them, the depth feature map corresponding to the same sample extracted by the first classification model and the second classification model can also be input into the fully connected layer before fusion, and multiple depth feature maps can be obtained after the fully connected layer. One-dimensional depth features Atlas, simplifying subsequent data processing.

Then in step S1425, the overall loss of the wafer defect classification model is confirmed according to the detection results of the noise samples, and the hybrid training of the first classification model and the second classification model is completed. Confirm the overall loss of the wafer defect classification model based on the detection results of the noisy samples. The overall loss can be used to confirm the training effect of the wafer defect classification model. The smaller the overall loss, the higher the accuracy of the defect type predicted by the wafer defect classification model for the sample. Therefore, the overall loss can be used to confirm the training effect. Among them, the overall loss of the wafer defect classification model includes co-fine-tuning loss, co-estimation loss and regularization loss. Among them, the collaborative estimation loss is the combination of the first classification model and the second classification model on the prediction results in the unlabeled data set, and the collaborative fine-tuning loss is the combination of the first classification model and the second classification model on the actual label value and the predicted label value in the labeled data set A linear combination is performed, and the training of parameters in the first classification model and the second classification model is realized by evaluating the linear combination. The training process can also include regularization processing on the first classification model and the second classification model to obtain the regularization loss of the first classification model and the second classification model, and the regularization processing can prevent overfitting in the wafer defect type classification. combined phenomenon.

In the wafer inspection method of an embodiment of the present application, by combining the coarse-grained image and the fine-grained image of the wafer, not only the overall image of wafer defects is paid attention to, but also the detailed differences between defects; The pure samples and a certain amount of noise samples can complete the training of the wafer defect classification model, reduce manpower input, improve the accuracy of wafer defect type classification to a certain extent, and reduce production costs.

According to the embodiments of the present application, the present application also provides a wafer detection system, a wafer detection device and a readable storage medium. The wafer inspection equipment may also include a processing machine, and a wafer inspection system. The wafer inspection system is set on the processing machine for inspecting the wafer.

As shown in FIG. 6 , it is a schematic diagram of a detection system for a wafer according to an embodiment of the present application. The system is intended to represent hardware devices provided in various forms of detection equipment, such as hardware devices provided in digital computers. The detection device may represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Inspection equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smart phones, wearable devices, wafer processing equipment, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the applications described and/or claimed herein.

As shown in FIG. 6 , the wafer inspection system includes: one or more processors 601 , memory 602 , and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be mounted on a common motherboard or otherwise as desired. The processor may process instructions executed within the electronic device, including instructions stored in or on the memory, to display graphical information of a GUI on an external input/output device such as a display device coupled to an interface. In other implementations, multiple processors and/or multiple buses may be used with multiple memories and multiple memories, if desired. Likewise, multiple electronic devices may be connected, with each device providing some of the necessary operations (eg, as a server array, a set of blade servers, or a multi-processor system). In FIG. 6, a processor 601 is taken as an example.

The memory 602 is a non-transitory computer-readable storage medium provided in this application. Wherein, the memory stores instructions executable by at least one processor, so that the at least one processor executes the wafer detection method provided in the present application. The non-transitory computer-readable storage medium of the present application stores computer instructions, and the computer instructions are used to make the computer execute the inspection method for wafers provided in the present application.

As a non-transitory computer-readable storage medium, the memory 602 can be used to store non-transitory software programs, non-transitory computer-executable programs and modules. The processor 601 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions and modules stored in the memory 602, that is, implements the method for wafer inspection in the above method embodiments.

The memory 602 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created according to the use of electronic equipment for quality control, etc. . In addition, the memory 602 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 602 may include memory located remotely relative to the processor 601, and these remote memories may be connected to inspection equipment for wafers through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The inspection system for wafers may further include: an input device 603 and an output device 604 . The processor 601, the memory 602, the input device 603, and the output device 604 may be connected through a bus or in other ways. In FIG. 6, connection through a bus is taken as an example.

The input device 603 can receive input numbers or character information, and generate key signal inputs related to user settings and function control of electronic equipment for quality control, such as touch screens, keypads, mice, trackpads, touchpads, and pointers , one or more mouse buttons, trackballs, joysticks, and other input devices. The output device 604 may include a display device, an auxiliary lighting device (eg, LED), a tactile feedback device (eg, a vibration motor), and the like. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen. In an embodiment of the present application, the input device 603 may further include a detection device for collecting an image of the wafer, and the collected image may include a coarse-grained image and a fine-grained image of the wafer.

As shown in FIG. 7 , it is a schematic diagram of a detection device for a wafer according to an embodiment of the present application. The detection device 701 may include the detection system in any of the above-mentioned embodiments, and the detection device 701 may represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade Servers, mainframes, and other suitable computers. Inspection equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smart phones, wearable devices, wafer processing equipment, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the applications described and/or claimed herein.

Various implementations of the systems and techniques described herein can be implemented in digital electronic circuitry, integrated circuit systems, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor Can be special-purpose or general-purpose programmable processor, can receive data and instruction from storage system, at least one input device, and at least one output device, and transmit data and instruction to this storage system, this at least one input device, and this at least one output device an output device.

These computing programs (also referred to as programs, software, software applications, or codes) include machine instructions for a programmable processor and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine language calculation program. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or means for providing machine instructions and/or data to a programmable processor ( For example, a magnetic disk, optical disk, memory, programmable logic device (PLD)), including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with the user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user. ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Acoustic input, voice input or tactile input) to receive input from the user.

The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., as a a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN) and the Internet.

A computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server, or an intelligent cloud computing server or an intelligent cloud host with artificial intelligence technology. The server can be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server, or an intelligent cloud computing server or an intelligent cloud host with artificial intelligence technology.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present application may be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in the present application can be achieved, no limitation is imposed herein.

The purpose, technical solution and beneficial effects of the present invention are further described in detail in the specific implementation manner described above. It should be understood that the above descriptions are only specific embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (24)

1. A method for detecting a wafer is characterized by comprising the following steps:

acquiring a first image of a wafer, wherein the first image comprises an overall characteristic representing defect information of the wafer;

determining a second image of the wafer based on the first image, the second image including detail features representing defect information of the wafer;

fusing the overall features and the detail features in series to generate fused features; and

and detecting the fused features through a wafer defect classification model.

2. The method of claim 1, wherein determining a second image of the wafer based on the first image comprises:

locating a defect feature region in the first image; and

and intercepting the area of the positioned defect feature to obtain a target image as the second image.

3. The method of claim 2, wherein determining the second image of the wafer based on the first image comprises:

positioning a detail area of a defect feature in the target image; and

and intercepting the located detail area to obtain a detail image as the second image.

4. The method according to claim 2, wherein the step of intercepting the located defect feature area to obtain a target image as the second image comprises:

extracting a depth feature map of a first image of the wafer based on a depth convolution neural network;

determining an average of the features of each location channel on the depth feature map and an average of the features of the global location channels of the depth feature map;

confirming a region of the depth feature map, wherein the average value of the features of the position channels is larger than the average value of the features of the whole position channels, as a target defect region; and

and intercepting the image of the target defect area as the target image.

5. The method according to claim 4, wherein after confirming, as the target defect region, a region in the depth feature map in which an average value of the features of the position channels is larger than an average value of the features of the overall position channels, comprising:

determining the minimum outer-wrapping rectangle of the target defect area and confirming the coordinates of the minimum outer-wrapping rectangle;

determining the position coordinates of the target defect region in the first image of the wafer through deconvolution; and

and intercepting a first image of the wafer according to the position coordinate to serve as the target image.

6. The method according to claim 3, wherein the step of intercepting the located detail area to obtain a detail image as the second image comprises:

extracting a depth feature map of the target image based on a depth convolutional neural network;

determining an average value of the features of each position channel on the depth feature map of the target image;

selecting a sliding window to carry out convolution on the target image, and confirming at least one activation window according to the sliding window, wherein the average value of the characteristics of the whole position channel in the depth feature map of the activation window is larger than the average value of the characteristics of each position channel on the depth feature map; and

and intercepting the target image corresponding to the at least one activation window as the detail image.

7. The method of claim 6, wherein the step of selecting a sliding window to convolve the target image and after confirming the active window according to the sliding window, truncating the located detail region to obtain a detail image further comprises:

selecting the area of the at least one activation window as a detail defect area of the wafer defect image in a non-maximum suppression mode;

determining the position coordinates of the detail defect area in the target image of the wafer through deconvolution; and

and intercepting an image of the detail defect area as the detail image.

8. The method of claim 1, wherein detecting the fused features through a wafer defect classification model comprises:

and classifying the defects of the wafer to determine the defect type of the wafer.

9. The method of claim 1, wherein detecting the merged feature by a wafer defect classification model comprises:

and outputting a detection result, wherein the detection result comprises the defect type of the wafer and the confidence corresponding to the defect type of the wafer.

10. The method of claim 1, wherein the wafer defect classification model comprises a first classification model and a second classification model, and the first classification model and the second classification model are different in training samples.

11. The method of claim 10, further comprising the step of individually training the wafer defect classification model, comprising:

dividing the first image and the second image of the wafer into a clean sample and a noise sample; and

and inputting the pure samples into the first classification model, inputting the noise samples into the second classification model, and respectively training the first classification model and the second classification model.

12. The method of claim 11, wherein the clean sample comprises the first image or the second image identifying a defect type for testing and verification of the wafer defect classification model; the noise sample comprises the first image or the second image of the defect type to be confirmed and is used for training the wafer defect classification model.

13. The method of claim 11, wherein training the first classification model and the second classification model separately further comprises hybrid training the first classification model and the second classification model, comprising:

dividing the noise samples into labeled and unlabeled datasets;

extracting depth feature maps of the labeled data set and the unlabeled data set using the first classification model and the second classification model;

fusing the depth feature maps of the same sample extracted by the first classification model and the second classification model;

inputting the fused depth feature map into a classifier to obtain a detection result of the noise sample; and

and confirming the overall loss of the wafer defect classification model according to the detection result of the noise sample, and finishing the training of the wafer defect classification model.

14. The method of claim 13, wherein the step of partitioning the noise samples into labeled and unlabeled datasets comprises:

inputting the first image or the second image in the noise samples to the first classification model and the second classification model; and

and dividing the noise sample into a marked data set and an unmarked data set according to the predicted defect types and the confidence degrees of the first classification model and the second classification model, wherein the confidence degree of the marked data set is greater than a set value, and the confidence degree of the unmarked data set is less than the set value.

15. The method according to claim 13, wherein before fusing the depth feature maps extracted from the same sample by the first classification model and the second classification model of the same sample, the method further comprises:

and inputting the depth feature map into a full connection layer to obtain a one-dimensional depth feature map.

16. The method of claim 13, wherein determining the global penalty of the wafer defect classification model based on the defect type and probability of the image comprises:

performing linear combination according to the defect types and the probabilities of the marked data sets obtained by the first classification model and the second classification model to obtain collaborative fine tuning loss;

merging the defect types and the probabilities of the unmarked data sets estimated by the first classification model and the second classification model to be used as collaborative estimation loss;

regularizing the first classification model and the second classification model to obtain regularization losses of the first classification model and the second classification model; and

and fusing the collaborative fine tuning loss, the collaborative estimation loss and the regularization loss to serve as the overall loss of the fine-grained classification model of the wafer defects.

17. The method of claim 16, wherein the first classification model and the second classification model classify the defect models of the images according to the depth feature maps, and the fusing the classification results to obtain the detection results of the noise samples comprises:

processing the depth feature maps of the marked data set and the unmarked data set by using a full connection layer to obtain a one-dimensional depth feature map; and

and fusing the characteristics of the one-dimensional depth characteristic map and inputting the fused characteristics into a classifier to obtain the detection result of the wafer.

18. The method of claim 13, wherein the inspection result comprises a confidence level that the defect type of the wafer corresponds to.

19. An inspection system for a wafer, comprising:

a memory for storing program instructions; and

a processor in communication with the memory for executing the program instructions to implement the method of any of claims 1 to 18.

20. An apparatus for inspecting a wafer, comprising: the detection system of claim 19.

21. The detection apparatus of claim 20, further comprising: and the detection device is used for acquiring the image of the wafer.

22. The inspection apparatus of claim 21, wherein the probing device is configured to capture a first image of the wafer and/or a second image of the wafer.

23. The detection apparatus of claim 20, wherein the detection apparatus comprises at least one of: computers, servers, cell phones, smart phones, wearable devices, wafer processing devices.

24. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-18.

Images