CN105825511B - A kind of picture background clarity detection method based on deep learning - Google Patents

Abstract

The picture background clarity detection method based on deep learning that the invention discloses a kind of, this method are to carry out feature extraction using convolutional neural networks (CNN), and effectively picture is classified according to its clear background degree using the CNN features extracted；Simultaneously using the method for transfer learning, pre-training is carried out with the ImageNet pictures for possessing a large amount of known marks, solves the defect that samples pictures concentrate known background definition values picture less, to obtain preferable CNN parameters；The samples pictures for further utilizing a small amount of known background definition values, are adjusted parameter, make CNN parameter adaptations pictures to be detected；The CNN parameters being adjusted can be carried out the clear background degree detection of picture to be detected.The detection method of the present invention so that clear background degree detects the accuracy that can reach height.

Description

A method for image background sharpness detection based on deep learning

technical field

The invention relates to a method for detecting the clarity of a picture background based on deep learning, mainly relates to the application of deep learning in machine learning, and belongs to the technical field of artificial intelligence picture recognition.

Background technique

The concept of deep learning originated from the research of artificial neural networks, and the multi-layer perceptron with multiple hidden layers is a deep learning structure. Deep learning forms more abstract high-level representations (attribute categories or features) by combining low-level features to discover distributed feature representations of data. The BP algorithm is a typical algorithm for traditional multi-layer network training. In fact, this training method is not ideal for networks with only a few layers. The ubiquitous local minima in non-convex objective cost functions of deep architectures (involving multiple layers of nonlinear processing units) are the main source of training difficulties.

Most of the current learning methods such as classification and regression are shallow structure algorithms, which are limited in the ability to express complex functions under the condition of limited samples and computing units, and their generalization ability for complex classification problems is restricted to a certain extent. Deep learning can achieve complex function approximation by learning a deep nonlinear network structure, characterize the distributed representation of input data, and demonstrate a powerful ability to learn the essential characteristics of data sets from a small number of sample sets. Deep learning is a feature learning method that converts raw data into higher-level, more abstract expressions through some simple but nonlinear models. By combining enough transformations, very complex functions can also be learned. The core aspect of deep learning is that the features of the above layers are not designed by human engineering, but are learned from data using a general learning process.

The convolutional neural network (CNNs) proposed by Lecun et al. is the first real multi-layer structure learning algorithm, and the core knowledge used in the present invention is the convolutional neural network, which utilizes the spatial relative relationship to reduce the number of parameters to improve BP training performance . Convolutional neural networks have achieved good results in the field of image recognition, and have achieved good results in recognizing handwritten characters. However, its network structure has a great influence on the effect and efficiency of image recognition. In order to improve the recognition performance, a new convolutional neural network structure is designed and implemented by reusing smaller convolution kernels, which can effectively reduce the training parameters. quantity and can improve the accuracy of recognition. The comparative experiment of convolutional neural network algorithm and the algorithm that has achieved good results in the ILSVRC Challenge, which is currently the world's advanced level in the field of image recognition, verifies the effectiveness of this structure.

The training process of the convolutional neural network requires a large number of known labeled samples. If the number of labeled samples is not enough, it is easy to cause the system to overfit. Jeff Donahue and others built the Decaf framework. The idea is to first perform pre-training on a picture set containing a large number of known labeled samples, adjust the parameters of the convolutional neural network system, and use migration learning to migrate the parameters of the entire system to the pictures to be trained. Concentration, so that only a small number of known labeled samples are needed to obtain accurate classification.

At present, there are many types of image recognition using deep learning, such as handwritten characters, license plate numbers, etc., but the usage based on convolutional neural networks has not been fully developed. At present, there is no good method for artificial intelligence to recognize the visibility of the environment in the image. Visibility, that is, the degree of blurring of things in the background. At present, most of the image recognition process is to identify the objects in the picture, often ignoring the useful information in the background environment. The present invention is mainly used to solve this problem. It is very practical to identify the degree of visibility of the background of the picture. For example, using this patent to identify the level of haze according to the picture in reality has a broad application prospect.

Contents of the invention

The technical problem to be solved by the present invention is to provide a method for detecting the background clarity of a picture based on deep learning, detect the background clarity in the picture, that is, the blur degree of things in the background, extract useful information in the background environment, and provide information for picture recognition. refer to.

The present invention adopts the following technical solutions for solving the problems of the technologies described above:

A method for detecting the sharpness of a picture background based on deep learning, comprising the following steps:

Step 1, convert the known marked pictures in the image library ImageNet and the sample images that are not in the image library ImageNet but have known background sharpness values into grayscale images with pixels of 256*256;

Step 2. Pre-train the converted grayscale images in the image library ImageNet, use the convolutional neural network to extract the features of all grayscale images and classify them, calculate the loss function, and adjust the convolution parameters with the stochastic gradient descent method to make the function The loss is within the predetermined range, and the initially adjusted convolution parameters are obtained;

Step 3, for the converted grayscale image of the sample image that is not in the image library ImageNet but with known background sharpness value, based on the initially adjusted convolution parameters in step 2, use the convolutional neural network to extract features and classify them, and obtain The sharpness value is compared with the actual sharpness value, the loss function is calculated, and the convolution parameters are continuously adjusted by the stochastic gradient descent method, so that the function loss is within the predetermined range, and the final adjusted convolution parameters are obtained;

Step 4: Convert the image to be detected into a grayscale image with a pixel size of 256*256. Based on the finally adjusted convolution parameters in step 3, use the convolutional neural network to extract features and classify them to obtain the image to be detected. clarity value.

As a preferred solution of the present invention, the convolutional neural network includes an input layer, a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer, Fully connected layer, output layer, and in addition to the input layer and output layer, the first convolutional layer, the first downsampling layer, the second convolutional layer, the second downsampling layer, and the fully connected layer in the convolutional neural network The floors are 1, 2, 3, 4, and 5 respectively.

As a preferred solution of the present invention, the convolution process formula of the first convolution layer is: Wherein, l=1, x ^l represents the value of the pixel point output after the convolution of the first convolutional layer, Indicates the value of the pixel in the i-th row and j-column in the input layer, w is the convolution parameter, and b is the offset.

As a preferred solution of the present invention, the down-sampling process formula of the first down-sampling layer is: Wherein, l=2, x ^l represents the value of the pixel point output after the first down-sampling layer sampling, Indicates the value of the pixel in the i-th row and j-column in the first convolutional layer, β is the downsampling parameter, and b is the offset.

As a preferred solution of the present invention, the fully connected layer includes two fully connected processes, and the formula of the fully connected process is: Among them, l=5 during the first full connection, l=6 during the second full connection, x ^l represents the value of the pixel point output after the full connection, k represents the pixel number, k= during the first full connection 1,...,576, k=1,...,50 in the second full connection, is the weight value.

Compared with the prior art, the present invention adopts the above technical scheme and has the following technical effects:

1, the present invention is based on deep learning image background clarity detection method, in the case of insufficient sample images, using the idea of transfer learning, first pre-training is carried out in the ImageNet image set containing a large number of known markers to obtain CNN parameters, and further Adjust the CNN parameters to adapt to the picture set to be detected, so that the detection accuracy of the picture set to be detected is higher.

2. The method for detecting the background clarity of pictures based on deep learning in the present invention solves the problem of detecting the background clarity in pictures, and has a great effect on practical applications such as identifying smog levels and air quality.

Description of drawings

FIG. 1 is an overall architecture diagram of the deep learning-based image background definition detection method of the present invention.

FIG. 2 is a flow chart of the method for detecting the sharpness of a picture background based on deep learning in the present invention.

Fig. 3 is a diagram of the internal structure of the convolutional neural network in the present invention.

Detailed ways

Embodiments of the invention are described in detail below, examples of which are illustrated in the accompanying drawings. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

Since the pixels of a given picture are uncertain, and the pixel requirements of the input picture in the convolutional neural network are fixed, the picture must first be preprocessed to convert it into a picture with the same pixels, and it is known In the process of ImageNet training, all the pictures are converted into 256*256 pixel pictures for processing, so the input picture pixels are also fixed at 256*256 pixels, and the pixel size of the picture to be processed is not 256*256, then the pixel size is first processed Convert, convert it into a 256*256 pixel picture. Since the clarity of a picture has little to do with its color, all pictures should be converted to grayscale first. The present invention divides the clarity of the picture background into five grades according to the clarity value, which are excellent, good, medium, poor, and extremely poor, so as to better analyze and test.

The present invention mainly includes three processes: a pre-training process, an adjustment process and an actual detection process. The pre-training process is to use the known labeled picture set ImageNet for training, the purpose is to obtain the initial CNN parameters; the adjustment process is to use a small number of sample pictures with known background clarity values to adjust the CNN parameters, so that the CNN parameters Adapt to the set of images to be detected; after obtaining the adjusted parameters, the detection of the images to be detected can be carried out.

Convolutional Neural Network (CNN) is the core technology of the present invention. CNN is a feed-forward neural network whose artificial neurons can respond to surrounding units in a part of the coverage area, and have excellent performance for large-scale image processing. It includes alternating convolutional layers and pooling layers. The first few stages are composed of convolutional layers and downsampling layers. The units of the convolutional layer are organized in feature maps. In the feature map, each unit is connected to the upper layer by a set of weights called filters. A local block of the feature map of one layer, and then this local weighted sum is passed to a non-linear function, such as ReLU. All units in a feature map share the same filter, and feature maps of different layers use different filters. This structure is used for two reasons. First, in array data, such as image data, values near a value are often highly correlated, which can form distinctive local features that are easier to detect. Secondly, the local statistical features of different locations are not very relevant, that is to say, a certain feature that appears in one place may also appear in other places, so units in different locations can share weights and can detect the same samples. Mathematically, this filtering operation performed by a feature map is an offline convolution, which is how the convolutional neural network gets its name.

The convolutional neural network has a good feature extraction effect, and the features extracted by the convolutional neural network can classify the target object very well.

In the framework of traditional machine learning, the task of learning is to learn a classification model based on sufficient training data; and then use the learned model to classify and predict test documents. However, we see a key problem with machine learning algorithms in current Web mining research: large amounts of training data in some emerging domains are very difficult to obtain.

Traditional machine learning needs to calibrate a large amount of training data for each domain, which will consume a lot of manpower and material resources. Without a large amount of labeled data, many studies and applications related to learning cannot be carried out. Second, traditional machine learning assumes that training data and test data obey the same data distribution. However, in many cases this same distribution assumption is not satisfied. Commonly possible situations such as outdated training data. This often requires us to relabel a large amount of training data to meet our training needs, but labeling new data is very expensive and requires a lot of manpower and material resources. From another point of view, if we have a large amount of training data under different distributions, it is very wasteful to completely discard these data. How to make reasonable use of these data is the main problem to be solved by transfer learning. Transfer learning is a new machine learning method that uses existing knowledge to solve problems in different but related fields.

Our current work on transfer learning can be divided into the following three parts: instance-based transfer learning in homogeneous spaces, feature-based transfer learning in homogeneous spaces, and transfer learning in heterogeneous spaces. The transfer learning used in the present invention belongs to the second part, feature-based transfer learning in the homogeneous space. Since ImageNet and the target domain have shared parameters, it is only necessary to migrate the parameters in the CNN system.

As shown in Fig. 1 and Fig. 2, the concrete operation process of the present invention is as follows:

1. Image preprocessing: preprocess the images in ImageNet and the sample images with known marks, and convert them into grayscale images of 256*256 pixels.

2. Pre-training stage: Pre-training with the ImageNet image library, input a grayscale image of 256*256 pixels, use CNN to extract features, classify them, calculate their loss function, and use the stochastic gradient descent method to adjust the parameters in CNN.

3. Adjustment stage: use the picture whose background definition value is known as the input picture, extract features through CNN and classify, get the value of its definition, compare it with the definition value of the actual picture, calculate the loss function, and use random The gradient descent method adjusts the system parameters.

4. Actual detection stage: firstly, the picture whose clarity is not known should be converted into 256*256 pixels as input, and CNN is used to extract features and classify, and finally get the label of its clarity.

The specific process of the convolutional neural network (the number of parameters can be adjusted according to the actual situation):

The CNN in this algorithm has 5 layers in total, excluding input and output layers, and each layer contains trainable parameters (connection weights). The input image has a size of 256*256 pixels.

1. The input layer to the Convolution 1 layer is a convolution process. The method is to multiply and sum the 9*9 pixels in the input picture with four 9*9 filters, that is, each of the input pictures The weighted sum of 9*9 pixels, plus an offset, the pixels in the convolution overlap, and a filter translation of one pixel is performed after each calculation. The formula of the convolution process is as follows:

Among them, l represents the number of layers (this layer l=1), x represents the value of a certain pixel point, and i and j represent the row and column number where the pixel point is located respectively (the value of i in this layer is 1 to 9, this layer The value of j in the layer is 1 to 9), w is the convolution parameter, and b is the offset.

For details, refer to the second block diagram in Figure 3. Each block in this figure is a pixel. It can be seen that every 9*9 pixel in the input layer is converted into a pixel in the Convolution1 layer through the convolution process, and the displacement of the filter each time is one pixel. The size of the input layer is 256*256 pixels, with 1 feature map, and the size of the Convolution1 layer is 248*248, with a total of 4 feature maps.

2. Convolution 1 layer to Subsampling 2 layer is a down-sampling process, which directly sums and weights the points of 4*4 pixels in the layer, and adds an offset, the down-sampling There is no overlap in the process, and the formula for the downsampling process is as follows:

Among them, l represents the number of layers (this layer l=2), x represents the value of a certain pixel point, and i and j represent the row and column numbers where the pixel point is located respectively (the value of i in this layer is 1 to 4, this layer The value of j in the layer is 1 to 4), β is the downsampling parameter, and b is the offset.

For details, refer to the third block diagram in Figure 3. Each block in this figure is a pixel. It can be seen that every 4*4 pixels in the Convolution 1 layer are down-sampled and converted into 1 pixel in the subsampling 2 layer. The size of the Convolution 1 layer is 248*248, and there are 4 feature maps in total. The size of the Subsampling 2 layer is 62*62, and there are 4 feature maps in total.

3. The process from Subsampling layer 2 to Convolution layer 3 is the same as the first convolution process. The size of the convolution filter is also 9*9, and 16 recorders are used to perform convolution at the same time. The size of the Subsampling2 layer is 62*62, with a total of 4 feature maps, and the size of the Convolution3 layer is 54*54, with a total of 16 feature maps.

4. The process from the Convolution3 layer to the Subsampling4 layer is similar to the first downsampling process, the difference is that each 9*9 pixel in the Convolution3 layer is summed and then weighted, plus an offset, downward The sampling process did not overlap.

The values of i and j are both 1 to 9. The size of the Subsampling4 layer is 6*6, with a total of 16 feature maps.

5. 16 6*6 feature maps, a total of 16*6*6 feature points, after two layers of full connection transformation, the so-called full connection means that each output unit is obtained by the weighted sum of all input units, Subsampling4 layers There are a total of 16*6*6 units, which are transformed into 50 units after a full connection to the 5th layer, and the 50 units of the 5th layer are converted into the final 5 levels through the second full connection transformation. The formula for full connection is as follows:

Among them, l represents the number of layers (l=5 in the first full connection, l=6 in the second full connection), x represents the value of a certain pixel, and k represents the pixel number (k in the first full connection =1,...,576, k=1,...,50 in the second full connection), is the weight value.

The above embodiments are only to illustrate the technical ideas of the present invention, and can not limit the protection scope of the present invention with this. All technical ideas proposed in accordance with the present invention, any changes made on the basis of technical solutions, all fall within the protection scope of the present invention. Inside.

Claims (5)

1. a kind of picture background clarity detection method based on deep learning, which is characterized in that include the following steps：

Step 1, by the picture of known mark in picture library ImageNet and not in picture library ImageNet but known background The samples pictures of definition values are converted into the gray scale picture that pixel is 256*256；

Step 2, pre-training is carried out to transformed gray scale picture in picture library ImageNet, institute is extracted using convolutional neural networks There is the feature of gray scale picture and classify, counting loss function adjusts deconvolution parameter with stochastic gradient descent method, function is made to damage It loses within a predetermined range, obtains the deconvolution parameter after first successive step；

Step 3, to not in picture library ImageNet but the transformed gray scale picture of the samples pictures of known background definition values, Based on the deconvolution parameter after step 2 just successive step, extracts feature using convolutional neural networks and classify, obtain its clarity Value, compares, counting loss function with practical definition values, is continued to adjust deconvolution parameter with stochastic gradient descent method, makes function Loss within a predetermined range, obtains the deconvolution parameter after final adjustment；

Step 4, the picture of clarity to be detected is converted into the gray scale picture that pixel is 256*256, is based on step 3 final adjustment Deconvolution parameter afterwards extracts feature using convolutional neural networks and classifies, obtains the clarity of clarity picture to be detected Value.

2. the picture background clarity detection method based on deep learning according to claim 1, which is characterized in that the volume Product neural network include successively by be input to the input layer of output, the first convolutional layer, the first downward sample level, the second convolutional layer, Second downward sample level, full articulamentum, output layer, and in addition to input layer, output layer, the first convolutional layer, the first downward sample level, Second convolutional layer, the second downward sample level, full articulamentum number of plies where convolutional neural networks are respectively 1,2,3,4,5 layer.

3. the picture background clarity detection method based on deep learning according to claim 2, which is characterized in that described The convolution process formula of one convolutional layer is：

Wherein, l=1, x^lIndicate the value of the pixel exported after the first convolutional layer convolution,Indicate the i-th row j row in input layer The value of pixel, w are deconvolution parameter, and b is offset.

4. the picture background clarity detection method based on deep learning according to claim 2, which is characterized in that described The downward sampling process formula of one downward sample level is：

Wherein, l=2, x^lIndicate the value of the pixel exported after the first downward sample level sampling,It indicates in the first convolutional layer The value of i-th row j row pixels, β are downward sampling parameter, and b is offset.

5. the picture background clarity detection method based on deep learning according to claim 2, which is characterized in that described complete Articulamentum includes full connection procedure twice, and full connection procedure formula is：

Wherein, l=5 when first time connects entirely, l=6, x when connecting entirely for the second time^lIndicate the pixel exported after full connection It is worth, k expression pixel numbers, k=1 ... when connecting entirely for the first time, 576, for the second time k=1 ... when full connection, 50,For weight Value.