CN108416389B - Image classification method based on denoising sparse autoencoder and density space sampling - Google Patents

Abstract

The invention discloses an image classification method based on noise reduction sparse automatic encoder and density space sampling. The steps are: constructing a training set of image blocks; constructing a denoising sparse auto-encoder with a single hidden layer, inputting the training set of image blocks, and training the denoising sparse auto-encoder; The image is sampled in the density space; the denoising sparse auto-encoder is used to extract the local feature set information from the spatial region obtained by the density space sampling of each image; the feature set information is encoded by the two-layer stacked Fisher Vector to obtain each image The final Fisher vector; use the Fisher vector to train the classifier to achieve image classification. The invention can accurately acquire image information, improve the classification accuracy of images, and can be used for the construction of large-scale image classification and retrieval systems.

Description

Image classification method based on denoising sparse autoencoder and density space sampling

technical field

The invention belongs to the technical field of image classification, and particularly relates to an image classification method based on noise reduction sparse automatic encoder and density space sampling.

Background technique

With the development of multimedia technology, image classification has become a hot research topic in the field of computer vision. Image classification is to classify images into different preset categories according to certain attributes they have. How to express the image effectively is the key to improve the accuracy of image classification, among which the selection and extraction of features are the key and difficult problems in the current image classification. Traditional artificially designed feature methods such as Gabor filter, SIFT, LBP, HOG, etc., have achieved certain results in image classification, but these methods need to be carefully designed and cannot be well applied to specific problems. In recent years, deep learning has achieved great success in many application fields such as image recognition and speech recognition. Convolutional Neural Networks (CNN) in deep learning models have made breakthroughs in feature extraction, but CNNs require a large amount of manually labeled label data, while denoising sparse autoencoders can use a large amount of unlabeled data to learn features describing image content . The traditional spatial pyramid is widely used in image classification, but the grid ratio of the traditional spatial pyramid is fixed, and the division method is relatively simple, while the density space sampling method based on the sliding window can capture more image spatial information.

SUMMARY OF THE INVENTION

The invention provides an image classification method based on noise reduction sparse automatic encoder and density space sampling, aiming at solving the problem of image feature extraction and coding, overcoming the defects of the existing image classification methods, reducing the calculation cost and improving the classification accuracy.

In order to realize the above-mentioned technical purpose, the technical scheme of the present invention is:

Image classification method based on denoising sparse autoencoder and density space sampling, including the following steps:

(1) Construct a training set of image blocks;

(2) constructing a denoising sparse autoencoder with a single hidden layer, inputting the image block training set obtained in step (1), and training the denoising sparse autoencoder until the stop iteration condition is met;

(3) Perform density space sampling on each image in the training image data set M ₁ and the test image data set M ₂ , set the number of images in the training image data set M ₁ to m ₁ , and the images in the test image data set M ₂ The number is m ₂ ;

(4) Using the trained denoising sparse auto-encoder as a feature extractor, extract local feature set information from the spatial region obtained by density space sampling for each image;

(5) using two-layer stacked Fisher Vector to encode the feature set information extracted in step (4) to obtain the final Fisher vector, which is used as a feature vector for image classification;

(6) Use the Fisher vector of the training image data set M ₁ and the corresponding label to train the classifier, and input the Fisher vector of the test image data set M ₂ into the trained classifier for image classification.

Further, the concrete process of step (1) is as follows:

其中，

表示第i张图像，N为无标签图像数目；(1a) Obtain an unlabeled image set

in,

Indicates the i-th image, and N is the number of unlabeled images;

中随机抽取M个r*r大小的图像块，一共得到MN个图像块，构成图像块训练集

n＝r*r*3。(1b) from each image

Randomly extract M image blocks of size r*r, and obtain a total of MN image blocks to form the image block training set

n=r*r*3.

Further, the concrete process of step (2) is as follows:

(2a) Add noise to the image block training set X: randomly add noise to the image block data x _j in X according to the proportion q, to obtain x _j '～q(x _j '|x _j ), where x _j ' is to set a part of the value of the vector x _j to 0 according to the ratio q, j∈{1,MN};

表示编码权重，

表示解码权重，

表示编码偏置项，

表示解码偏置项；(2b) Randomly initialize the model parameters W ₁ , W ₂ , b ₁ , b ₂ , set the number of neurons in the input layer of the noise reduction sparse autoencoder to n, and the number of neurons in the hidden layer to be s, where,

represents the encoding weight,

represents the decoding weight,

represents the coding bias term,

represents the decoding bias term;

Let α _j =g(W ₁ x _j '+b ₁ ), where,

(2c) Define the overall cost function Loss of the denoising sparse autoencoder:

是所有训练样本在第t个隐藏神经元上的平均响应；where λ is the weight decay coefficient, β is the sparse penalty weight, ρ is the target sparse value,

is the average response of all training samples on the t-th hidden neuron;

(2d) Use the error back propagation algorithm to calculate the gradient of the overall cost function Loss; solve the minimization problem of the overall cost function Loss through the improved quasi-Newton method L-BFGS, so as to obtain the trained model parameters W ₁ , b ₁ , W ₂ , b ₂ .

Further, the concrete process of step (3) is as follows:

(3a) respectively acquiring each image in the training image data set M ₁ and the test image data set M ₂ ;

(3b) performing density spatial sampling on each image obtained in step (3a) from left to right and from top to bottom in an iterative manner using a sliding window with variable scale, and each image is sampled to obtain R spatial regions; The initial sliding window size is defined as [w, h], the step size of each sliding window is t, and the spatial area obtained by each sliding window is defined as a quadruple Area (m, n, w, h); among them, m and n represent the coordinate position of the upper left corner of the sliding window in the image, and w and h represent the width and height of the initial sliding window.

Further, the concrete process of step (4) is as follows:

个r*r的图像块；(4a) Randomly extract from each spatial region of each image obtained in step (3b)

r*r image blocks;

个r*r图像块的局部特征集

每幅图像获取R个局部特征集，对应R个空间区域。(4b) Using the trained denoising sparse autoencoder as a feature extractor, obtain the lth spatial region in image i

Local feature set of r*r image patches

Each image acquires R local feature sets, corresponding to R spatial regions.

Further, the concrete steps of step (5) are as follows:

(5a) Use a Gaussian mixture model containing K Gaussian components to model all the local feature sets _of the training image data set M1, and solve to obtain the parameter θ of the model;

(5b) on the basis of step (5a), utilize the Fisher vector of the first layer to encode the local feature set of the l-th spatial region of the image i to obtain the Fisher vector b _l ; after the Fisher vector encoding of the first layer , get the Fisher vector set of image i

(5c) Perform PCA dimensionality reduction on the Fisher vector sets of all images in the training image data set M ₁ and the test image data set M ₂ , and the Fisher vector set of the image i after dimensionality reduction is:

(5d) Use a Gaussian mixture model containing K' Gaussian components to model the Fisher vector set of all images of the training image data set M ₁ after dimensionality reduction, and solve to obtain the parameter θ' of the model;

(5e) On the basis of step (5d), use the Fisher vector of the second layer to encode the Fisher vector set B _i ' of the image i after dimensionality reduction, obtain the Fisher vector f _i , and obtain the training image data set M ₁ and the test image Image dataset M ₂ Final Fisher vector for each image.

其中，a_t为单个局部特征，p_i(a_t)代表第i个高斯单元，

D表示局部特征a_t的维数，π_i,μ_i,∑_i分别表示第i个高斯单元的权重、均值和协方差矩阵；通过训练图像数据集M₁的所有局部特征集来估计高斯混合模型的所有参数θ＝{π_i,μ_i,∑_i,i＝1,...K}。Further, in step (5a), the Gaussian mixture model of K Gaussian units is recorded as

where at is a single local feature, _pi (at ) represents the _{ith Gaussian} unit,

D represents the dimension of local feature a _t , π _i , μ _i , ∑ _i represent the weight, mean and covariance matrix of the ith Gaussian unit, respectively; Gaussian mixture is estimated by training all local feature sets of image dataset M ₁ All parameters of the model θ={π _i , μ _i , ∑ _i , i=1,...K}.

Further, in step (6), the classifier adopts a support vector machine.

The beneficial effects brought by the above technical solutions:

(1) The present invention adopts a noise reduction sparse auto-encoder as a feature extractor. The model is an unsupervised feature learning method, which can be trained with unlabeled samples, and can solve the current problem of requiring a large number of manually labeled samples. Furthermore, denoising sparse autoencoders are able to learn more robust feature representations compared to ordinary sparse autoencoders.

(2) The present invention adopts the density space sampling method to replace the traditional space pyramid structure. The traditional spatial pyramid segmentation of the image is relatively fixed, while the density spatial sampling method can perform more flexible segmentation of the image, so that more spatial information of the image can be captured.

(3) The present invention uses stacked Fisher vectors for feature encoding. Compared with the standard single-layer structure, the stacked structure can extract semantic information of images in a hierarchical structure, and can obtain better accuracy and performance in image classification.

Description of drawings

FIG. 1 is a flow chart of the method of the present invention.

Detailed ways

The technical solutions of the present invention will be described in detail below with reference to the accompanying drawings.

The test experiment software and hardware environment of the present embodiment is as follows:

Hardware Type:

Computer type: desktop;

CPU: Intel(R) Core(TM) i5-5200U CPU@2.20GHz

RAM: 8.00GB

System Type: 64-bit OS

Development language: Matlab

The technical solutions of the present invention will be described in detail below with reference to the accompanying drawings. The embodiment takes the STL-10 database as an example, the database contains 10 types of RGB images, and the size of each image is 96*96. Among them, the number of training samples used for supervised training is 5000, and the 5000 training samples are divided into ten folds. The number of training samples used for supervised training is 1000 each time, and the number of test samples is 8000.

The present invention provides an image classification method based on noise reduction and sparse automatic encoder and density space sampling, as shown in FIG. 1 , and the specific steps are as follows.

Step 1, build a training set of image patches:

其中，

表示第i张图像，无标签的图像数目为100000；1a, get the unlabeled image set in STL-10

in,

Indicates the i-th image, and the number of unlabeled images is 100,000;

中随机抽取10个8*8大小的图像块，从而构成图像块训练集

乘以3是因为存在R、G、B三个通道。1b, from each image

Randomly select 10 image blocks of 8*8 size from the image block to form a training set of image blocks

Multiplying by 3 is because there are three channels R, G, B.

Step 2: Construct a denoising sparse autoencoder with a single hidden layer, input the image block training set obtained in step 1b, and train the denoising sparse autoencoder until the stop iteration condition is met:

2a, add noise to the image block training set X: randomly add noise to the image block vector data x _j in X according to a certain proportion q, to obtain x _j '～q(x _j '|x _j ). Among them, x _j ' is to set a part of the value of vector x _j to 0 according to the ratio q. Here q is set to 0.5, j∈{1,1000000};

为编码权重，

为解码权重，

为编码偏置项，

为解码偏置项。假设α_j＝g(W₁x_j'+b₁)，其中

2b, randomly initialize the model parameters W ₁ , W ₂ , b ₁ , b ₂ , set the number of neurons in the input layer of the noise reduction sparse autoencoder to 192, and the number of neurons in the hidden layer to 500. in,

is the encoding weight,

is the decoding weight,

is the coding bias term,

is the decoding bias term. Suppose α _j =g(W ₁ x _j '+b ₁ ), where

2c, define the overall cost function Loss of DSAE:

是所有训练样本在第t个隐藏神经元上的平均响应，优选地，λ＝0.03，β＝3，ρ＝0.1；where λ is the weight decay coefficient, β is the sparse penalty weight, ρ is the target sparse value,

is the average response of all training samples on the t-th hidden neuron, preferably, λ=0.03, β=3, ρ=0.1;

2d, using the error back-propagation algorithm to calculate the gradient of the overall cost function Loss. The minimization problem of the cost function Loss is solved by the improved quasi-Newton method L-BFGS, so as to obtain the trained model parameters W ₁ , b ₁ , W ₂ , and b ₂ .

Step 3, perform density space sampling on each image in the training image data set M ₁ and the test image data set M ₂ , wherein the number of images in the training image data set M ₁ is 1000, and the number of images in the test image data set M ₂ is 8000 :

3a, obtain each image in the training image data set M ₁ and the test image data set M ₂ respectively;

3b, each image in step 3a is density-spatially sampled sequentially from left to right and top to bottom in an iterative manner using a variable-scale sliding window. The initial sliding window size is defined as [w, h], the step size of each sliding window is 10, and the spatial area obtained by each sliding window is defined as a quadruple Area (m, n, w, h). Among them, m and n represent the coordinate position of the upper left corner of the sliding window in the image, and w and h represent the width and height of the initial sliding window. The initial values of m and n are 0, the initial values of w and h are set to 46, and each image can be sampled to obtain 90 spatial regions.

Step 4: Using the trained denoising sparse autoencoder as a feature extractor, extract local feature set information from the spatial region obtained by density space sampling for each image:

4a, randomly extract 100 8*8 image blocks from each spatial region of each image obtained in step 3c;

每幅图像可获取90个特征集。4b, using the trained denoising sparse autoencoder as a feature extractor, obtain the local feature set of 100 8*8 image blocks in the lth spatial region of image i

90 feature sets can be obtained per image.

Step 5, use the two-layer stacked Fisher Vector to encode the feature set information of step 4b, and obtain the final Fisher vector, which is used as the feature vector of image classification:

其中，a_t为单个局部特征，p_i(a_t)代表第i个高斯单元，

D为局部特征a_t的维数，D＝192，π_i,μ_i,∑_i分别表示第i个高斯单元的权重、均值和协方差矩阵。通过训练图像数据集M₁的所有局部特征集来估计高斯混合模型的所有参数θ＝{π_i,μ_i,∑_i,i＝1,...128}；5a, use a Gaussian mixture model with 128 Gaussian components to model all the local feature sets of the training image dataset M ₁ , and solve to obtain the parameter θ of the model; the Gaussian mixture model of 128 Gaussian units is recorded as

where at is a single local feature, _pi (at ) represents the _{ith Gaussian} unit,

D is the dimension of the local feature at, D= ₁₉₂ , π _i , μ _i , Σ _i represent the weight, mean and covariance matrix of the ith Gaussian unit, respectively. Estimate all parameters of the Gaussian mixture model θ={π _i , μ _i ,∑ _i ,i=1,...128} by training all local feature sets of the image dataset M ₁ ;

5b, on the basis of step 5a, use the Fisher vector of the first layer to encode the local feature set A of the lth spatial region of the image i to obtain the Fisher vector b _l . After the Fisher vector encoding of the first layer, the Fisher vector set of image i can be obtained

5c, perform PCA dimensionality reduction processing on the Fisher vector sets of all images in the training image dataset M ₁ and the test image dataset M ₂ , and the Fisher vector set of the image i after dimensionality reduction is

5d, using a Gaussian mixture model containing 32 Gaussian components to model the Fisher vector set _of all images of the training image data set M1 after dimensionality reduction, and solving to obtain the parameter θ' of the model;

5e, on the basis of step 5d, use the Fisher vector of the second layer to encode the Fisher vector set B _i ' of the image i after dimensionality reduction to obtain the Fisher vector f _i , thereby obtaining the training image data set M ₁ and the test image The final Fisher vector f for each image of the dataset _M2 .

Step 6: Use the Fisher vector of the training image data set M ₁ in step 5e and the corresponding label to train the support vector machine, and input the Fisher vector of the test image data set M ₂ into the trained classifier for image classification. .

The embodiment is only to illustrate the technical idea of the present invention, and cannot limit the protection scope of the present invention. Any changes made on the basis of the technical solution according to the technical idea proposed by the present invention all fall within the protection scope of the present invention. .

Claims (4)

1. The image classification method based on noise reduction sparse autoencoder and density space sampling, is characterized in that, comprises the following steps: (1) Build an image block training set; the specific process of this step: (1a) Obtain an unlabeled image set

in,

Indicates the i-th image, and N is the number of unlabeled images; (1b) from each image

Randomly extract M image blocks of size r*r, and obtain a total of MN image blocks to form the image block training set

(2) constructing a denoising sparse autoencoder with a single hidden layer, inputting the image block training set obtained in step (1), and training the denoising sparse autoencoder until the stop iteration condition is met; (3) Perform density space sampling on each image in the training image data set M ₁ and the test image data set M ₂ , set the number of images in the training image data set M ₁ to m ₁ , and the images in the test image data set M ₂ The number is m ₂ ; the specific process of this step: (3a) respectively acquiring each image in the training image data set M ₁ and the test image data set M ₂ ; (3b) performing density spatial sampling on each image obtained in step (3a) from left to right and from top to bottom in an iterative manner using a sliding window with variable scale, and each image is sampled to obtain R spatial regions; The initial sliding window size is defined as [w, h], the step size of each sliding window is t, and the spatial area obtained by each sliding window is defined as a quadruple Area (m, n, w, h); among them, m and n represent the coordinate position of the upper left corner of the sliding window in the image, and w and h represent the width and height of the initial sliding window; (4) Using the trained denoising sparse autoencoder as a feature extractor, extract local feature set information from the spatial region obtained by density space sampling for each image; the specific process of this step: (4a) Randomly extract from each spatial region of each image obtained in step (3b)

r*r image blocks; (4b) Using the trained denoising sparse autoencoder as a feature extractor, obtain the lth spatial region in image i

Local feature set of r*r image patches

Each image obtains R local feature sets, corresponding to R spatial regions; (5) Encoding the feature set information extracted in step (4) by using two-layer stacked Fisher Vectors to obtain the final Fisher vector, which is used as a feature vector for image classification; the specific process of this step: (5a) Use a Gaussian mixture model containing K Gaussian components to model all the local feature sets _of the training image data set M1, and solve to obtain the parameter θ of the model; (5b) on the basis of step (5a), utilize the Fisher vector of the first layer to encode the local feature set of the l-th spatial region of the image i to obtain the Fisher vector b _l ; after the Fisher vector encoding of the first layer , get the Fisher vector set of image i

(5c) Perform PCA dimension reduction on the Fisher vector sets of all images in the training image data set M ₁ and the test image data set M ₂ , and the Fisher vector set of the image i after dimensionality reduction is:

(5d) Use a Gaussian mixture model containing K' Gaussian components to model the Fisher vector set of all images of the training image data set M ₁ after dimensionality reduction, and solve to obtain the parameter θ' of the model; (5e) On the basis of step (5d), use the Fisher vector of the second layer to encode the Fisher vector set B _i ' of the image i after dimensionality reduction, obtain the Fisher vector f _i , and obtain the training image data set M ₁ and the test image The final Fisher vector of each image in the dataset _M2 ; (6) Use the Fisher vector of the training image data set M ₁ and the corresponding label to train the classifier, and input the Fisher vector of the test image data set M ₂ into the trained classifier for image classification. 2. the image classification method based on noise reduction sparse automatic encoder and density space sampling according to claim 1, is characterized in that, the concrete process of step (2) is as follows: (2a) Add noise to the image block training set X: randomly add noise to the image block data x _j in X according to the proportion q, to obtain x _j '～q(x _j '|x _j ), where x _j ' is to set a part of the value of the vector x _j to 0 according to the ratio q, j∈{1,MN}; (2b) Randomly initialize the model parameters W ₁ , W ₂ , b ₁ , b ₂ , set the number of neurons in the input layer of the noise reduction sparse autoencoder to n, and the number of neurons in the hidden layer to be s, where,

represents the encoding weight,

represents the decoding weight,

represents the coding bias term,

represents the decoding bias term; Let α _j =g(W ₁ x _j '+b ₁ ), where,

g(z)=1/(1+exp(-z)),

(2c) Define the overall cost function Loss of the denoising sparse autoencoder:

where λ is the weight decay coefficient, β is the sparse penalty weight, ρ is the target sparse value,

is the average response of all training samples on the t-th hidden neuron; (2d) Use the error back propagation algorithm to calculate the gradient of the overall cost function Loss; solve the minimization problem of the overall cost function Loss through the improved quasi-Newton method L-BFGS, so as to obtain the trained model parameters W ₁ , b ₁ , W ₂ , b ₂ . 3. the image classification method based on noise reduction sparse automatic encoder and density space sampling according to claim 1, is characterized in that, in step (5a), the Gaussian mixture model of K Gaussian units is denoted as

where at is a single local feature, _pi (at ) represents the _{ith Gaussian} unit,

D represents the dimension of local feature a _t , π _i , μ _i , ∑ _i represent the weight, mean and covariance matrix of the ith Gaussian unit, respectively; Gaussian mixture is estimated by training all local feature sets of image dataset M ₁ All parameters of the model θ={π _i , μ _i , ∑ _i , i=1,...K}. 4 . The image classification method based on noise reduction sparse autoencoder and density space sampling according to claim 1 , wherein in step (6), the classifier adopts a support vector machine. 5 .

Images