CN112131968A - Change detection method of dual-phase remote sensing image based on DCNN - Google Patents

Abstract

The invention discloses a dual-phase remote sensing image change detection method based on DCNN. By inputting the dual-phase remote sensing image data set into a deep convolutional neural network, a dual-phase feature map is generated, and the dual-phase feature map is subjected to dual-phase processing. Linear interpolation, so that the size of the bi-temporal feature map is the same as the size of the remote sensing image in the bi-temporal remote sensing image dataset, calculate the Euclidean distance between the bi-temporal feature maps after bi-linear interpolation, and generate according to the Euclidean distance Difference image, extract the feature vector of each pixel block in the difference image, construct a feature vector space according to each feature vector, cluster the feature vector space, generate a coarse change detection map according to the clustering result, and perform morphological filtering on the coarse change detection map , to generate a change detection map, which effectively simplifies the process of change detection in the remote sensing images to be detected, and improves the detection effect and detection efficiency.

Description

Change detection method of dual-temporal remote sensing image based on DCNN

technical field

The invention relates to the technical field of image processing, in particular to a DCNN-based dual-phase remote sensing image change detection method.

Background technique

Change detection techniques are core processes for many applications that utilize remote sensing imagery. It identifies changes that occur on the Earth's surface by processing two images acquired at different times covering the same geographic area. Change detection has a wide range of uses, including land use and land cover change detection, risk assessment, environmental surveys, and more.

Based on artificial features, various algorithms have been proposed to solve the change detection problem, such as image quantification, principal component analysis, change vector analysis, expectation maximization, Markov random fields, etc. To compute these hand-designed features, parameters such as size, scale, and orientation need to be carefully and carefully chosen. In addition, the selection of features is also a difficult point in remote sensing image change detection.

Publication No. CN108830828A A method and device for detecting changes in remote sensing images, using preset filters to filter the first remote sensing image and the second remote sensing image, respectively, to obtain the first filtered image and the second filtered image; A difference image is calculated from the first filtered image and the second filtered image, and the difference image is used to identify changes in the remote sensing images of the two different phases. Although this method solves the problem of low accuracy of remote sensing image change detection due to noise in remote sensing images when the existing difference method detects changes in remote sensing images, it directly obtains the difference images from the original bi-temporal remote sensing images and generates The accuracy of the difference image is not high, which is easy to cause error accumulation, and cannot achieve a good change detection effect. The publication number CN107992891A is based on the spectral vector analysis multispectral remote sensing image change detection method. Input two multispectral remote sensing images that have been preprocessed in the same region at different times, and use the principal component analysis method to reduce the dimension of the difference space constructed by the change vector analysis method. Take the first principal component to obtain the first difference map; solve the angle information between the spectral vectors of the dual-phase remote sensing image to obtain the second difference map; solve the information entropy of the two difference maps respectively, and then obtain the fusion weight through calculation, and use the weighted The summation method is combined to obtain a better difference map; the spatial feature description is carried out; the spectral clustering method is used for cluster analysis to obtain the change detection result. Although this method can suppress the interference of the change information due to factors such as illumination and radiation to a certain extent, but because the principal component analysis method is directly used to construct the difference space, the preprocessing of the original remote sensing image has higher requirements, and more complex preprocessing is required. step. It can be seen that the traditional solutions often have technical problems of complicated detection process and poor detection effect.

SUMMARY OF THE INVENTION

In view of the above problems, the present invention proposes a DCNN-based dual-phase remote sensing image change detection method.

In order to realize the purpose of the present invention, a kind of dual-temporal remote sensing image change detection method based on DCNN is provided, comprising the following steps:

S10, constructing a bitemporal remote sensing image dataset including the remote sensing image to be detected;

S30, inputting the bi-temporal remote sensing image dataset into a pre-built deep convolutional neural network to generate bi-temporal feature maps corresponding to each remote sensing image included in the bi-temporal remote sensing image dataset;

S40, perform bilinear interpolation on the bi-temporal feature map, so that the size of the bi-temporal feature map is the same as the size of the remote sensing image in the bi-temporal remote sensing image dataset;

S50, calculating the Euclidean distance between the bi-temporal feature maps after bilinear interpolation, generating a difference image according to the Euclidean distance, extracting the feature vector of each pixel block in the difference image, and constructing a feature vector space according to each feature vector;

S60, cluster the feature vector space, and generate a rough change detection map according to the clustering result;

S70, perform morphological filtering on the coarse change detection map to generate a change detection map.

In one embodiment, before step S30, it further includes:

S20, construct a deep convolutional neural network whose main structure is based on VGG19, and load the weight parameters pre-trained on ImageNet as the weight parameters of the deep convolutional neural network to complete the construction of the deep convolutional neural network.

In one embodiment, calculating the Euclidean distance between the bi-temporal feature maps after bilinear interpolation includes:

表示双时相特征图中T1时刻特征图的像素值，

表示双时相特征图中T2时刻特征图的像素值，DI_i(x,y)表示欧氏距离。In the formula,

represents the pixel value of the feature map at time T1 in the dual-phase feature map,

Represents the pixel value of the feature map at time T2 in the dual-phase feature map, and DI _i (x, y) represents the Euclidean distance.

In one embodiment, clustering the feature vector space, and generating a rough change detection map according to the clustering result includes:

Use k-means clustering with k=2 to divide the eigenvector space into two clusters, and calculate the minimum Euclidean distance between the eigenvectors of the two clusters and the mean eigenvector, assigning each pixel to the One, generates a coarse change detection map.

In one embodiment, performing morphological filtering on the coarse change detection map to generate the change detection map comprises:

Corrosion operation is performed on the coarse change detection map, and noise pixels in the coarse change detection map are filtered out to generate a change detection map.

The above-mentioned DCNN-based bi-temporal remote sensing image change detection method constructs a bi-temporal remote sensing image dataset including the remote sensing images to be detected, and inputs the bi-temporal remote sensing image dataset into a pre-built deep convolutional neural network to generate a bi-temporal remote sensing image dataset. The bi-temporal feature maps corresponding to each remote sensing image included in the temporal remote sensing image dataset, and bilinear interpolation is performed on the bi-temporal feature maps, so that the size of the bi-temporal feature maps is the same as that of the remote sensing images in the bi-temporal remote sensing image dataset. The size of the image is the same, calculate the Euclidean distance between the bi-phase feature maps after bilinear interpolation, generate the difference image according to the Euclidean distance, extract the feature vector of each pixel block in the difference image, and construct the feature vector according to each feature vector. space, cluster the feature vector space, generate a coarse change detection map according to the clustering results, and perform morphological filtering on the coarse change detection map to generate a change detection map, realize the change detection of the remote sensing images to be detected, and effectively simplify the detection process. The change detection process of remote sensing images improves the detection effect and detection efficiency.

Description of drawings

1 is a flowchart of a method for detecting changes in dual-phase remote sensing images based on DCNN according to one embodiment;

Fig. 2 is a framework diagram of a DCNN-based dual-phase remote sensing image change detection according to an embodiment;

Figure 3 is a deep neural network framework diagram of one embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

Referring to FIG. 1, FIG. 1 is a flowchart of a method for detecting changes in dual-temporal remote sensing images based on DCNN according to one embodiment, including the following steps:

S10, constructing a bitemporal remote sensing image dataset including the remote sensing images to be detected.

The above steps can construct a bi-temporal remote sensing image dataset including the remote sensing images to be detected at two times (eg, T1 time and T2 time), and make a corresponding sample label set.

S30: Input the bi-temporal remote sensing image dataset into a pre-built deep convolutional neural network to generate bi-temporal feature maps corresponding to each remote sensing image included in the bi-temporal remote sensing image dataset.

S40, perform bilinear interpolation on the bi-temporal feature map, so that the size of the bi-temporal feature map is the same as the size of the remote sensing image in the bi-temporal remote sensing image dataset.

S50: Calculate the Euclidean distance between the bi-temporal feature maps after bilinear interpolation, generate a difference image according to the Euclidean distance, extract feature vectors of each pixel block in the difference image, and construct a feature vector space according to each feature vector.

S60, cluster the feature vector space, and generate a rough change detection map according to the clustering result.

S70, perform morphological filtering on the coarse change detection map to generate a change detection map.

The above-mentioned DCNN-based bi-temporal remote sensing image change detection method constructs a bi-temporal remote sensing image dataset including the remote sensing images to be detected, and inputs the bi-temporal remote sensing image dataset into a pre-built deep convolutional neural network to generate a bi-temporal remote sensing image dataset. The bi-temporal feature maps corresponding to each remote sensing image included in the temporal remote sensing image dataset, and bilinear interpolation is performed on the bi-temporal feature maps, so that the size of the bi-temporal feature maps is the same as that of the remote sensing images in the bi-temporal remote sensing image dataset. The size of the image is the same, calculate the Euclidean distance between the bi-phase feature maps after bilinear interpolation, generate the difference image according to the Euclidean distance, extract the feature vector of each pixel block in the difference image, and construct the feature vector according to each feature vector. space, cluster the feature vector space, generate a coarse change detection map according to the clustering results, and perform morphological filtering on the coarse change detection map to generate a change detection map, realize the change detection of the remote sensing images to be detected, and effectively simplify the detection process. The change detection process of remote sensing images improves the detection effect and detection efficiency.

In one embodiment, before step S30, it further includes:

S20, construct a deep convolutional neural network whose main structure is based on VGG19, and load the weight parameters pre-trained on ImageNet as the weight parameters of the deep convolutional neural network to complete the construction of the deep convolutional neural network.

The aforementioned ImageNet is a large visualization database containing a large number of images.

Specifically, the specific structure of the above-mentioned VGG19-based deep convolutional neural network may include:

(2.1) In the first large layer, two convolution layers with a convolution kernel size of 3*3*64, a stride of 1, and a linear rectification function are defined respectively, and a pooling layer, the pooling method Choose max pooling;

(2.2) In the second largest layer, two convolution layers with a convolution kernel size of 3*3*128, a stride of 1, and a linear rectification function are defined respectively, and a pooling layer, the pooling method Choose max pooling;

(2.3) In the third largest layer, a convolutional layer with a size of 3*3*256 convolution kernel, a stride of 1, and a linear rectification function are defined respectively, and a pooling layer, the pooling method Choose max pooling;

(2.4) In the fourth largest layer, four convolution layers with convolution kernel size of 3*3*512, stride of 1, activation function of linear rectification function, and a pooling layer are respectively defined. The pooling method Choose max pooling;

(2.5) In the fifth largest layer, four convolution layers with kernel size of 3*3*512, stride of 1, activation function of linear rectification function, and a pooling layer are defined respectively, and the pooling method Choose max pooling;

(2.6) The sixth layer is the fully connected layer;

(2.7) The seventh layer is a fully connected layer;

(2.8) The eighth layer is a fully connected layer.

In one example, the following process can be performed after the bitemporal remote sensing image dataset is fed into the above deep convolutional neural network:

Let the i-th image in the dataset be a three-channel image, which is represented as follows:

Image _i (x,y,z)＝{(x,y,z)|0≤x＜h,0≤y＜w,0≤z＜3}

Among them, h represents the height of the image, and w represents the width of the image. The image is sent to the deep convolutional neural network, and the feature map output by the fourth largest layer of the network is extracted:

feature _i (x,y,z)＝{(x,y,z)|0≤x＜h′,0≤y＜w′,0≤z＜512}

Wherein, h' represents the height of the feature map output by the fourth largest layer of the deep convolutional neural network, and w' represents the width of the feature map output by the fourth largest layer of the deep convolutional neural network.

In one embodiment, calculating the Euclidean distance between the bi-temporal feature maps after bilinear interpolation includes:

表示双时相特征图中T1时刻特征图的像素值，

表示双时相特征图中T2时刻特征图的像素值，DI_i(x,y)表示欧氏距离。In the formula,

represents the pixel value of the feature map at time T1 in the dual-phase feature map,

Represents the pixel value of the feature map at time T2 in the dual-phase feature map, and DI _i (x, y) represents the Euclidean distance.

In an example, the feature vector of each pixel block in the difference image is extracted, and the process of constructing a feature vector space according to each feature vector may include:

(6.1) The difference image DI _i can be expressed as follows:

Among them, n is 256;

(6.2) Generate h×h non-overlapping pixel blocks in the above difference image DI _i as x _d , where h is selected as a constant greater than 2:

(6.3) Expand x _d into row vector x' _d , set all x' _d to form vector set X _d , and use principal component analysis algorithm to establish feature vector space on X _d :

Among them, there are h×h elements in x′ _d ,

The shape of X _d is h ² ×h ² , and all features in X _d are centered to generate X′ _d :

Find the covariance matrix for X _d ':

Wherein, cov(f _p , f _q )=E{[f _p -E(f _p )][f _q -E(f _q )]}, using the matrix knowledge Cu=λu to calculate the covariance matrix C The eigenvalue λ and the corresponding eigenvector u are arranged in descending order of the eigenvalues, and the eigenvalue eigenvector pair before the selection constitutes the eigenvector space;

(6.4) Project the h×h pixel block around each pixel of DI _i to the feature vector space in (6.2), and establish the feature vector space FVS on the entire DI _i :

in,

In one embodiment, clustering the feature vector space, and generating a rough change detection map according to the clustering result includes:

Use k-means clustering with k=2 to divide the eigenvector space into two clusters, and calculate the minimum Euclidean distance between the eigenvectors of the two clusters and the mean eigenvector, assigning each pixel to the One, generates a coarse change detection map.

Further, performing morphological filtering on the coarse change detection map to generate the change detection map includes:

Corrosion operation is performed on the coarse change detection map, and noise pixels in the coarse change detection map are filtered out to generate a change detection map.

In one embodiment, the process of constructing a bi-temporal remote sensing image dataset including the remote sensing image to be detected may include:

Construct a remote sensing image dataset Image=[Image ₀ , Image ₁ ,...,Image _i ], and make a corresponding sample label set Lable=[Lable ₀ ,Lable ₁ ,...,Lable _i ], where i represents The constructed image dataset and the maximum number of images contained in the corresponding label set, Image _i represents the i-th image in the constructed remote sensing image dataset, and Lable _i represents the i-th label image in the produced sample label set. Each of the original images (such as remote sensing images to be detected) has a corresponding label image. The label can be a binary image, where pixels that change are shown in white, and pixels that don't change are shown in black.

In one embodiment, the process of performing bilinear interpolation on the bitemporal feature map includes:

Bilinear interpolation is performed according to the following expression:

The feature map after bilinear interpolation is:

feature _i (x,y,z)＝{(x,y,z)|0≤x＜h,0≤y＜w,0≤z＜512}

Among them, h represents the height of the original image, and w represents the width of the original image. x and y respectively represent the abscissa and ordinate of the pixel to be determined in the feature map, q ₁₁ , q ₁₂ , q ₂₁ , and q ₂₂ represent the four closest pixels to the pixel to be determined, respectively, x ₁ , x ₂ , y ₁ and y ₂ respectively represent the abscissa and ordinate of the above four pixel points. So the final calculation result is the pixel value of the pixel to be calculated.

In this embodiment, the feature map of the original dual-phase image is obtained through the deep neural network. Due to the function of the pooling layer in the deep neural network, the size of the image is reduced by half after each layer of pooling layer. In order to keep the size of the output image consistent with the input image and the detection accuracy, bilinear interpolation is performed on the obtained bi-temporal feature maps respectively, and the purpose is to enlarge the size of the feature map to the original image size.

In one embodiment, the framework of the above-mentioned DCNN-based dual-temporal remote sensing image change detection method can be referred to as shown in FIG. 2, including the following processes:

(1) Build a deep convolutional neural network whose main structure is VGG19, and select the warrant parameters pre-trained on ImageNet as the weight parameters of the deep convolutional neural network. Construct a remote sensing image dataset Image=[Image ₀ , Image ₁ ,...,Image _i ], and make a corresponding sample label set Lable=[Lable ₀ ,Lable ₁ ,...,Lable _i ], where i represents The constructed image dataset and the maximum number of images contained in the corresponding label set, Image _i represents the i-th image in the constructed remote sensing image dataset, and Lable _i represents the i-th label image in the produced sample label set. The built deep convolutional neural network is shown in Figure 2, and the parameters of each layer are set as follows:

(a) In the first large layer, two convolution layers with a convolution kernel size of 3*3*64, a stride of 1, and a linear rectification function as the activation function, and a pooling layer are defined respectively. The pooling method Choose max pooling;

(b) In the second largest layer, two convolution layers with a convolution kernel size of 3*3*128, a stride of 1, and a linear rectification function as the activation function, and a pooling layer are defined respectively. The pooling method Choose max pooling;

(c) In the third largest layer, four convolution layers with convolution kernel size of 3*3*256, stride of 1, and activation function of linear rectification function are respectively defined, and a pooling layer, the pooling method Choose max pooling;

(d) In the fourth largest layer, four convolution layers with convolution kernel size of 3*3*512, stride of 1, and activation function of linear rectification function, and a pooling layer, are respectively defined. The pooling method Choose max pooling;

(e) In the fifth largest layer, four convolution layers with kernel size of 3*3*512, stride of 1, activation function of linear rectification function, and a pooling layer are defined respectively. Pooling method Choose max pooling;

(f) The sixth layer is a fully connected layer;

(g) The seventh layer is a fully connected layer;

(h) The eighth layer is a fully connected layer.

(2) Input the constructed remote sensing image dataset into the above constructed deep neural network to generate the feature map.

(2.1) In this embodiment, the size of the remote sensing images in the dataset is 256×256, and the i-th image is a three-channel image, which is represented as follows:

Image _i (x,y,z)＝{(x,y,z)|0≤x＜h,0≤y＜w,0≤z＜3}

Among them, h represents the height of the image, w represents the width of the image, and its value is 256;

(2.2) The feature map output by the above deep convolutional neural network is:

feature _i (x,y,z)＝{(x,y,z)|0≤x＜h′,0≤y＜w′,0≤z＜512}

Wherein, h' represents the height of the feature map output by the deep convolutional neural network, and w' represents the width of the feature map output by the deep convolutional neural network. In this embodiment, the feature map output by the fourth largest layer of the above-mentioned deep convolutional neural network is extracted, so the values of h' and w' are both 16;

(2.3) Perform bilinear interpolation on the feature map output by the fourth largest layer of the deep convolutional neural network according to the following expression:

Enlarging its size to the original input image size, the interpolated feature map is:

feature _i (x,y,z)＝{(x,y,z)|0≤x＜h,0≤y＜w,0≤z＜512}

Among them, h represents the height of the original image, w represents the width of the original image, and the value is 256.

(3) Calculate the Euclidean distance on the above feature map according to the following expression, and generate a difference image:

表示双时相特征图中T1时刻特征图的像素值，

表示双时相特征图中T2时刻特征图的像素值，DI_i(x,y)表示欧氏距离。In the formula,

represents the pixel value of the feature map at time T1 in the dual-phase feature map,

Represents the pixel value of the feature map at time T2 in the dual-phase feature map, and DI _i (x, y) represents the Euclidean distance.

(4) Using the principal component analysis algorithm to construct the feature vector space on the above difference image, after generating the feature vector space, use the k-means clustering algorithm to cluster the above feature vectors to generate a coarse change map, and perform an erosion operation on the coarse change map to generate Final change detection map. By comparing the predicted values of all images in the test set with the Label values corresponding to the original remote sensing images, the detection accuracy of the entire test set can be obtained.

The present embodiment has the following beneficial effects:

(1) The features are extracted from the original remote sensing images directly through the deep convolutional neural network, which avoids the complicated steps of traditional manual feature extraction, and the extracted features have high accuracy.

(2) Constructing the difference image through the features extracted by the deep neural network can eliminate the influence of noise in the original image, and can avoid the accumulation of errors and improve the performance of change detection.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features It is considered to be the range described in this specification.

It should be noted that the term "first\second\third" involved in the embodiments of the present application is only to distinguish similar objects, and does not represent a specific ordering of objects. It is understandable that "first\second\third" "Three" may be interchanged in a particular order or sequence where permitted. It should be understood that the "first\second\third" distinctions may be interchanged under appropriate circumstances to enable the embodiments of the application described herein to be practiced in sequences other than those illustrated or described herein.

The terms "comprising" and "having" and any variations thereof in the embodiments of the present application are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product or device comprising a series of steps or modules is not limited to the listed steps or modules, but optionally also includes unlisted steps or modules, or optionally also includes Other steps or modules inherent to these processes, methods, products or devices.

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are specific and detailed, but should not be construed as a limitation on the scope of the invention patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims (5)

1. A double-time phase remote sensing image change detection method based on DCNN is characterized by comprising the following steps:

s10, constructing a double-time-phase remote sensing image data set comprising the remote sensing image to be detected;

s30, inputting the double-time-phase remote sensing image data set into a pre-constructed deep convolutional neural network to generate a double-time-phase characteristic diagram corresponding to each remote sensing image in the double-time-phase remote sensing image data set;

s40, carrying out bilinear interpolation on the double-time phase characteristic diagram to enable the size of the double-time phase characteristic diagram to be the same as that of the remote sensing image in the double-time phase remote sensing image data set;

s50, calculating Euclidean distance between the bi-temporal phase feature maps after bilinear interpolation, generating a difference image according to the Euclidean distance, extracting feature vectors of pixel blocks in the difference image, and constructing a feature vector space according to the feature vectors;

s60, clustering the feature vector space, and generating a coarse change detection graph according to the clustering result;

s70, morphologically filtering the coarse change detection map to generate a change detection map.

2. The DCNN-based two-time-phase remote sensing image change detection method according to claim 1, wherein before step S30, the method further includes:

s20, constructing a deep convolutional neural network of which the main body structure is based on VGG19, and loading weight parameters pre-trained on ImageNet as the weight parameters of the deep convolutional neural network so as to complete the construction of the deep convolutional neural network.

3. The DCNN-based dual-temporal remote sensing image change detection method according to claim 1, wherein calculating the euclidean distance between the bilinear interpolated dual-temporal feature maps comprises:

in the formula,

representing the pixel values of the feature map at time T1 in the two-time temporal feature map,

representing the pixel value, DI, of the feature map at time T2 in a two-time-phase feature map_i(x, y) represents the Euclidean distance.

4. The DCNN-based dual-temporal remote sensing image change detection method of claim 1, wherein clustering the eigenvector space and generating a coarse change detection map according to the clustering result comprises:

dividing the feature vector space into two clusters by using k-means clustering with k being 2, calculating the minimum Euclidean distance between the feature vectors of the two clusters and the mean feature vector, and allocating each pixel to one of the two clusters to generate a coarse change detection map.

5. The DCNN-based dual-temporal remote sensing image change detection method of claim 1, wherein the performing morphological filtering on the coarse change detection map to generate the change detection map comprises:

and carrying out corrosion operation on the coarse change detection graph, and filtering noise pixel points in the coarse change detection graph to generate a change detection graph.

Images