CN108346145B - Identification method of unconventional cells in pathological section - Google Patents

Abstract

The invention discloses a method for identifying unconventional cells in pathological slices. Fine-tune the network, so that the fully convolutional network has the ability to extract the characteristics of unconventional cells, and then determine the location of unconventional cells, and more effectively classify the effective discriminant area; By combining the prediction results of multiple common classification networks, the output is more stable. Classification results. The identification method of the present invention can automatically determine the probability of the existence of unconventional cells in each 20× magnified field of view in the pathological slice, and output the unconventional cells with a probability value above 0.5 as the identification result, which greatly reduces the need for manual screening of unconventional cells in pathological slices. Cell workload, rapid and accurate screening of unconventional cells.

Description

A method for identifying unconventional cells in pathological sections

technical field

The invention belongs to the field of medical imaging, and particularly relates to a method for identifying unconventional cells in pathological slices.

Background technique

Unconventional cells (or abnormal cells) in traditional pathological sections are manually screened: under a microscope, professional pathologists scan the entire section with the naked eye through the movement of the section to find out whether there are unconventional cells in the entire section. This work is tedious and time-consuming, and as the reading time increases, so does the error rate.

With the continuous development of science and technology, the identification of unconventional cells in pathological slices can be used for preliminary screening with the help of computers.

Based on the Convolutional Neural Network (CNN) algorithm, VGGNet, ResNet, DenseNet and other networks improve the structure of computer vision with continuous updating and iteration. On natural images, the accuracy rate has exceeded the human eye. Image semantic segmentation (Semantic Segmentation) is an important research direction of computer vision, and its task is to classify each pixel of a single image through computer algorithms. In medical imaging, semantic segmentation is often used to segment organs, tissues or cells in images for subsequent classification.

Jonathan Long proposed to use convolution and deconvolution in fully convolutional neural networks (FCN) to perform semantic segmentation tasks instead of traditional fully connected semantic segmentation methods as one of the main methods of semantic segmentation models. U -Net is a typical fully convolutional network. Its main idea is to divide the network into a downsampling layer and an upsampling layer, using ordinary convolution, pooling layer, and bilinear interpolation or deconvolution for upsampling. Enlarging the feature map refers to being able to be the same size and concatenated with the shallow feature map. The final feature map is obtained by splicing the shallow feature map skip connection with the deep feature map.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a method for identifying unconventional cells in pathological slices, greatly reducing the heavy workload of manual screening of unconventional cells in pathological slices, and quickly and accurately screening unconventional cells.

The conventional cells of the present invention are normal cells of the human body, and the unconventional cells correspond to the normal cells of the human body and are abnormal morphological cells of the human body.

The working principle of the technical solution of the present invention:

The electronically scanned pathological section is preprocessed into multiple regions, and the average value of the A channel after conversion to the LAB channel is used to judge whether the region is valid, and all the valid discriminant regions in the pathological section are obtained. After processing, the effective discriminant region is input to the fully convolutional network for pre-training, and then the fully connected layer is used to replace the fully convolutional network head fine-tuning network, which enables the fully convolutional network to have the ability to extract the characteristics of unconventional cells, and then determine the location of unconventional cells. Therefore, the effective discriminant regions are classified more effectively. By combining the prediction results of multiple common classification networks, a more stable classification result is output, and the probability of unconventional cells in each 20× magnified field of view in the slice can be automatically determined.

A method for identifying unconventional cells on pathological sections, comprising:

(1) preprocessing the electronically scanned pathological slice to obtain an effective discriminating area in the pathological slice, wherein the unconventional cell pixel area in the effective discriminating area is a positive sample, and the conventional cell pixel area is a negative sample;

(2) The positive and negative samples obtained in step (1) are trained with a fully convolutional network algorithm, and the parameters of the network are adjusted according to the coincidence of the model prediction result and the label to obtain a convergent slice segmentation model;

(3) On the basis of the slice segmentation model obtained in step (2), replace its head segmenter with a classifier, and use the discriminative region containing unconventional cells as a positive example, and the discriminative region without unconventional cells as a negative example For example, fine-tune the network parameters to adapt to the classification task, and obtain a segmentation pre-trained classification model;

(4) In the effective discriminant area obtained in step (1), the discriminant area containing unconventional cells is regarded as a positive example, and the discriminative area containing no unconventional cells at all is regarded as a negative example, and k in the ordinary convolutional neural network classification method is used. Fold cross-validation to train k common classification models;

The value range of the k is an integer between 5 and 10;

(5) merging the segmentation pre-training classification model obtained in step (3) and the k common classification models obtained in step (4) by the method of model integration to construct a final classification model;

(6) Inputting the unmarked new pathological slice and the effective discriminant region obtained by processing in step (1) into the final classification model, and outputting unconventional cells with a probability value above 0.5 as the identification result.

In step (1), described preprocessing step is:

(1-1) Divide the 20× magnified pathological slice into areas with the same size as 512*512～2048*2048 pixels, and store them separately;

(1-2) For the image after each small block is converted into the LAB channel, the small block whose mean value of the A channel exceeds the threshold t is used as an effective discrimination area, and the rest are discarded;

The threshold t is 120-150.

The reason for using the mean value of channel A after conversion to LAB in step (1-2) as the basis for judging the effective area is that the effective areas such as tissue cells in pathological sections are stained purple or red. In the LAB channel, the A channel represents the effective area. The degree of redness of the pixel, so the A channel is used as the judgment basis, and if the threshold t is exceeded, it is considered that the area contains effective tissue or cells.

In step (2), the method for evaluating the degree of coincidence between the model prediction result and the label includes Dice Loss, Cross Entropy or Mean Squared Error.

In step (2), the training method of the described convergent slice segmentation model, its concrete steps are:

(2-1) Compress the input effective discriminant area into a matrix with pixels of 256*256～512*512 through the image compression algorithm; this ratio can ensure the preservation of most image features, while discarding some small features, these Features contribute less to the classification of positive and negative samples.

(2-2) Normalize and convert the above matrix to standard normal distribution by redistribution and z-score method;

The image obtained in step (2-1) (matrix with pixels ranging from 256*256 to 512*512) is RGB three-channel. In order to be better learned by the neural network, it generally needs to be redistributed and normalized. The specific operation process Yes: first divide the image pixels by 255, project to the [0,1] interval, and then use the zscore normalization method to subtract the mean and divide the variance to convert the data to a standard normal distribution. The zscore calculation method is as follows:

表示该特征的平均值，s表示该特征的标准差。where _zi represents the final output of the z-score algorithm, _xi represents the input data,

represents the mean of the feature, and s represents the standard deviation of the feature.

(2-3) Use data augmentation (Data Augmentation) technology to perform operations such as rotation, flipping, mirroring, brightness change, random offset, etc. on the standard normal distribution image converted in step (2-2), so that the network can learn to different directions and angles, while reducing the degree of overfitting of network predictions.

(2-4) Input the matrix processed in step (2-3) into a fully convolutional network to calculate Dice Loss;

In the process of training the segmentation model, we used the loss function for the segmentation task - DiceLoss for training. Dice Loss is a loss function designed for image segmentation with unbalanced pixels in positive and negative examples. It is defined as follows:

Among them, i represents the current calculation pixel, p _i , _gi represent the model prediction score of pixel i and the score corresponding to the label, N represents the total number of pixels, D represents the two-class prediction result (heat map) and The degree of overlap of the labels, the value range of D is in the [0,1] interval, the closer the value is to 1, the higher the degree of overlap; in the training process, we use 1-D as the loss function;

The label is a binary matrix with the same size as the input image, 1 represents unconventional cell pixels, 0 represents conventional cell pixels, and in (2-1), the effective discriminant area is compressed to 512*512 pixels. Similarly, segmentation The labels are also compressed to 512*512 pixels in order to match the valid discriminant regions.

(2-5) Use the Adam algorithm as an optimization method to minimize Dice Loss until the network converges, and a converged slice segmentation model is obtained.

In the fine-tuning stage, the advantage of using the fully-connected layer to fine-tune the U-Net pre-training weights is that after the U-Net training is completed, the deep layer of the U-Net model already has the ability to extract unconventional cell features, which is more useful for classification tasks. , and this method increases the interpretability of the classification model. In the prediction stage, the classification results can be output simultaneously and compared with the segmentation results in order to identify regions of unconventional cells.

Steps (2) and (3) The method of training the segmentation pre-training classification model is a two-stage method, that is, the segmentation label pre-training is used first, and then the classifier is fine-tuned. The method equivalent to the two-stage method is single-stage training. , the single-stage training method, that is, using the output of the U-Net model as an auxiliary loss function and the classification loss function as the final loss function to minimize, but the effect of the single-stage method relative to the two-stage method poor.

In step (3), the fine-tuning network method includes the following steps:

(3-1) Replace the last convolutional layer of U-Net with a fully connected layer whose output is binary classification,

(3-2) Use the discriminative region containing unconventional cells as a positive example, and the discriminative region without unconventional cells as a negative example, use the Adam algorithm to optimize the cross-entropy loss function and update the network parameters, so that the classification task model reaches convergence, Get the segmentation pretrained classification model.

In step (4), the k ordinary classification models are trained by means of k-fold cross-validation. data, the remaining k-1 copies are used as training data for training, and k ordinary classification models are obtained;

The value range of the k is an integer between 5 and 10;

The training data includes discriminative regions containing unconventional cells as positive examples and discriminative regions that do not contain unconventional cells as negative examples.

However, the classifier that only uses U-Net to assist in extracting features is still unable to classify some overall features very well. At this time, some classifiers need to be added to ensure the accuracy of the overall classification. In the present invention, we use DenseNet, which has better generalization performance in the current classification effect, for training. During the training process, we use the CrossEntropy Loss pre-training model. In order to make the model pay more attention to difficult samples, the Focal Loss fine-tuning parameters are used.

In step (5), the described model integration fusion method includes:

(1) Voting method: take the mode of the outputs of multiple models as the final result; or,

(2) Weighted mean method: assign different weights to multiple models, obtain their weighted mean, and judge the final label by the weighted mean; or,

(3) Stacking method: train a linear classifier with the output of multiple models as input as the basis for judging labels.

The model integration and fusion method of the present invention is preferably a weighted mean method, wherein the weights of the common classification models are the same; the weights of the pretrained classification models are k times the weights of the common classification models, and k is the number of common classification models.

The reason why the weighted mean method is preferred is that compared with the voting method, the weighted mean method can better balance the importance of the segmentation pre-training classification model and the common classification model, and effectively highlight the contribution of the segmentation pre-training model to the final model.

According to the segmentation pre-trained classification model and k DenseNet classification models obtained respectively according to the above steps, we use the weighted mean method to fuse k+1 models. In order to highlight the contribution of the segmentation model, we use the following formula for weighted average to obtain p( x) is the final predicted value, x represents the input image matrix:

Among them, S represents the segmentation pre-trained classification model function, d represents the DenseNet classification function, k represents the k-fold number, and λ represents the weight of the segmentation pre-trained classification model.

The method for identifying unconventional cells in pathological slices provided by the present invention uses a feature map generated by semantic segmentation combined with multiple classification networks, and achieves a good test effect. reached more than 96%. The F1 value is the harmonic mean of precision and recall, and the calculation method is as follows:

Among them, precision represents the precision, and recall represents the recall rate, which is calculated using the following formula:

Among them, tp, fp, and fn represent the number of true positives, the number of false positives and the number of false negatives, respectively.

Compared with the prior art, the present invention has the following beneficial effects:

1) The present invention can greatly reduce the heavy workload of pathologists, and play a better role in popularization for grassroots hospitals and community hospitals lacking pathologist resources.

2) The present invention can help doctors to quickly and accurately screen out unconventional cells, and the F1 value of the algorithm can reach over 96% based on the F1 value as an index of the algorithm.

Description of drawings

FIG. 1 is the overall structure of the fully convolutional network U-net in the specific implementation method of the present invention.

FIG. 2 is the overall structure of the general classification model DenseNet in the specific implementation method of the present invention.

FIG. 3 is an overall schematic diagram of the pathological slice region classification according to the specific implementation method of the present invention.

FIG. 4 is a flow chart of training a pathological slice region classification model according to a specific implementation method of the present invention.

Detailed ways

In order to further understand the present invention, a method for identifying unconventional cells in a pathological section provided by the present invention will be specifically described below in conjunction with specific implementation methods, but the present invention is not limited to this. The non-essential improvements and adjustments made still belong to the protection scope of the present invention.

A method for identifying unconventional cells on a pathological slice, the specific steps are:

1) Pathological slice preprocessing and effective area discrimination

The present invention adopts pathological slices whose input data is 20x magnification, and is divided into areas with pixel resolution of 2048*2048, which are stored separately.

Convert the above area with a pixel resolution of 2048*2048 to the LAB channel, take the area where the average value of the A channel exceeds the threshold t=132 as an effective discrimination area, and discard the rest.

2) Train a converged slice segmentation model

(2-1) Compress the effective discrimination area obtained in step 1) to an area with a pixel resolution of 512×512;

(2-2) First divide the image pixels by 255, project to the [0,1] space, and then use the zscore normalization method to subtract the mean and divide the variance to convert the data to a standard normal distribution image, zscore calculates Methods as below:

表示该特征的平均值，s表示该特征的标准差；where x _i represents the input data,

Represents the mean value of the feature, and s represents the standard deviation of the feature;

(2-3) Use data augmentation (Data Augmentation) technology to perform operations such as rotation, flipping, mirroring, brightness change, random offset, etc. on the standard normal distribution image converted in step (2-2), so that the network can learn to different directions and angles, while reducing the degree of overfitting of network predictions.

(2-4) Input the matrix obtained in step (2-3) into the fully convolutional network U-Net, whose structure is shown in Figure 1, and use the following formula to calculate Dice Loss:

where pi and _gi represent the model prediction score of pixel _i and the score corresponding to the label, respectively.

The label is a binary matrix with the same size as the input image, 1 represents unconventional cell pixels, 0 represents conventional cell pixels, and in (2-1), the effective discriminant area is compressed to 512*512 pixels. Similarly, segmentation The labels are also compressed to 512*512 pixels in order to match the valid discriminant regions.

(2-5) Use the Adam algorithm as an optimization method to minimize Dice Loss until the network converges, and a converged slice segmentation model is obtained.

3) On the basis of the slice segmentation model obtained in step 2), the last convolutional layer of U-Net is replaced with a fully connected layer whose output is binary classification, and the discriminative region containing unconventional cells is used as a positive example. The discriminative region containing unconventional cells is used as a negative example, and the Adam algorithm is used to optimize the cross entropy loss function (Cross Entropy Loss) to optimize the network parameters, so that the classification task model reaches convergence, and the segmentation pre-trained classification model is obtained.

In order to avoid the trained U-Net weights from being damaged due to the excessively large gradient of the weights of the randomly generated fully connected layer weights, we fine-tune the following steps:

a) Fixed U-Net weights and only trains the fully connected layer until convergence;

b) Reduce the learning rate of the U-Net part and train the overall network (including U-Net and fully connected layers).

4) k-fold cross-validation DenseNet training

(4-1) Divide the training set into 5 parts, 4 parts are used as the training set, and 1 part is used as the verification set;

(4-2) For each division set, use other data as the training set to train the DenseNet model. The structure of the DenseNet model is shown in Figure 2, and the model is saved when the effect on the verification set is optimal. Get 5 models.

5) Model fusion

According to the segmentation pre-trained classification model and 5 DenseNet classification models obtained by the above steps, we use the weighted average method to fuse the 6 models. In order to highlight the contribution of the segmentation model, we use the following formula for weighted average:

Among them, S represents the segmentation pre-trained classification model function, d represents the DenseNet classification function, k represents the k-fold number, and λ represents the weight of the segmentation pre-trained classification model.

After fusion, the final classification model is obtained, and the model training flow chart is shown in Figure 3.

6) Unconventional cell identification

Input the unmarked new pathological slice, the effective discriminant area obtained by step 1) into the final classification model, and judge the probability that each area in the pathological slice contains unconventional cells according to the threshold t=132, and output the probability value above 0.5. Unconventional cells were identified as a result.

Claims (7)

1. A method for identifying unconventional cells in a pathological section, comprising: (1) preprocessing the electronically scanned pathological slice to obtain an effective discriminating area in the pathological slice, wherein the unconventional cell pixel area in the effective discriminating area is a positive sample, and the conventional cell pixel area is a negative sample; (2) The positive and negative samples obtained in step (1) are trained with a fully convolutional network algorithm, and the parameters of the network are adjusted according to the coincidence of the model prediction result and the label to obtain a convergent slice segmentation model; (3) On the basis of the slice segmentation model obtained in step (2), replace its head segmenter with a classifier, and use the discriminative region containing unconventional cells as a positive example, and the discriminative region without unconventional cells as a negative example For example, fine-tuning the network parameters to adapt them to the classification task, and obtaining a segmentation pre-trained classification model; the methods of fine-tuning the network parameters include: (3-1) Replace the last convolutional layer of U-Net with a fully connected layer whose output is binary classification; (3-2) Use the discriminative region containing unconventional cells as a positive example, and the discriminative region without unconventional cells as a negative example, use the Adam algorithm to optimize the cross-entropy loss function and update the network parameters, so that the classification task model reaches convergence, Get the segmentation pre-trained classification model; (4) In the effective discriminant area obtained in step (1), the discriminant area containing unconventional cells is regarded as a positive example, and the discriminative area containing no unconventional cells at all is regarded as a negative example, and k in the ordinary convolutional neural network classification method is used. Fold cross-validation to train k common classification models; The value range of the k is an integer between 5 and 10; (5) merging the segmentation pre-training classification model obtained in step (3) and the k common classification models obtained in step (4) by the method of model integration to construct a final classification model; (6) Inputting the unmarked new pathological slice and the effective discriminant region obtained by processing in step (1) into the final classification model, and outputting unconventional cells with a probability value above 0.5 as the identification result. 2. according to the method for identifying unconventional cells in the pathological section according to claim 1, it is characterized in that, in step (1), described preprocessing, comprises: (1-1) Divide the 20× enlarged pathological slice into small blocks with the same size as 512*512～2048*2048 pixels, and store them separately; (1-2) For the image after each small block is converted into the LAB channel, the small block whose mean value of the A channel exceeds the threshold t is used as an effective discrimination area, and the rest are discarded; The threshold t is 120-150. 3. according to the identification method of unconventional cells in the pathological section according to claim 1, it is characterized in that, in step (2), the evaluation method of the coincidence degree of described model prediction result and label comprises Dice Loss, Cross Entropy or MeanSquared Error. 4. according to the identification method of the unconventional cell in the pathological slice described in claim 1 and 3, it is characterized in that, in step (2), the training method of the slice segmentation model of described convergence, comprises: (2-1) Compress the input effective discrimination area into a matrix with pixels of 256*256～512*512 through an image compression algorithm; (2-2) Normalize and convert the above matrix to standard normal distribution by redistribution and z-score method; (2-3) Use data enhancement technology to perform rotation, flip, mirror image, brightness change, and random shift operations on the standard normal distribution image converted in step (2-2); (2-4) Input the matrix processed in step (2-3) into a fully convolutional network to calculate Dice Loss; (2-5) Use the Adam algorithm as an optimization method to minimize Dice Loss until the network converges, and a converged slice segmentation model is obtained. 5. the identification method of unconventional cells in the pathological slice according to claim 1, is characterized in that, in step (4), described by the mode of k-fold cross-validation training k common classification models, its concrete steps It is: firstly divide the training data into k equal parts by stratified sampling, use one of them as the verification data each time, and use the remaining k-1 parts as the training data for training to obtain k common classification models; The value range of the k is an integer between 5 and 10; The training data includes discriminative regions containing unconventional cells as positive examples and discriminative regions that do not contain unconventional cells as negative examples. 6 . The method for identifying unconventional cells in a pathological slice according to claim 1 , wherein, in step (5), the model integration and fusion method comprises a voting method, a weighted mean method or a stacking method. 7 . 7. The method for identifying unconventional cells in pathological slices according to claim 6, wherein the weighted mean method, wherein the weights of the common classification models are the same; the weights of the pretrained classification models are divided is k times the weight of the common classification model, and k is the number of common classification models.

Images