CN111126431A - A Fast Screening Method for Massive Power Defect Photos Based on Template Matching - Google Patents

Abstract

The invention discloses a rapid screening method for massive electric power defect photos based on template matching. By improving the traditional normalized cross-correlation algorithm, the complexity of calculation and the amount of calculation in the calculation process are reduced; the wavelet pyramid is used for layering The idea of matching, the resolution is from low to high, the matching is from coarse to fine, improves the image matching efficiency and improves the matching accuracy; realizes the fast and accurate matching and screening of defect images, and provides an important data basis for intelligent defect identification.

Description

A Fast Screening Method for Massive Power Defect Photos Based on Template Matching

technical field

The invention relates to the technical field of image processing, in particular to a method for quickly screening photos of massive power defects based on template matching.

Background technique

In recent years, with the rapid development of UAV technology, detection equipment such as high-resolution visible light cameras (cameras), high-precision infrared thermal imagers, and lidars have enriched the means of inspection of transmission lines. UAV remote sensing technology is used to inspect transmission lines. Inspection can effectively improve the work efficiency of power inspection, reduce the workload of manual inspection, and reduce the cost and pressure of transmission line operation and maintenance. The system analysis and management of the abundant transmission line equipment data obtained from the power inspection based on the UAV can provide more data support for the power grid disaster prevention, management and maintenance. Power grid companies have fully implemented the machine patrol operation mode. Taking China Southern Power Grid Corporation as an example, the helicopter inspection business of transmission lines has been carried out as early as 2004. In 2013, the overall promotion of machine patrol work on transmission lines began. The lean operation and maintenance of transmission lines with human patrol as a supplement.

With the continuous advancement of the machine patrol business, the data processing and analysis application of the machine patrol inspection of transmission lines will surely enter the era of "big data". The traditional machine patrol data processing mode requires a lot of manual intervention. Defect identification mainly relies on visual interpretation and manual use of drawing tools to identify the defect location. The data processing cycle is long and the workload is large. According to incomplete statistics, there are not all defects in photos. Check 2% of the photo. At the same time, the long-term manual operation can easily lead to a decrease in the accuracy of the judgment results, and the results have problems such as subjectivity, ambiguity, and incompleteness, which are not conducive to the timely detection of equipment defects and restrict the efficient use of data.

At present, the processing of machine patrol data is in the stage of automation and intelligence combined with new technologies such as artificial intelligence and deep learning. Data is the foundation and blood of artificial intelligence. At present, the defect data accumulated by power grid companies are basically all manually interpreted, and pictures of defect locations are framed by drawing tools, etc. These photos cannot be directly used for artificial intelligence learning due to the existence of identification frames. Find out the unprocessed defect photos, and provide the data basis for the data to meet the requirements for intelligent defect identification.

Currently commonly used gray-based image matching algorithms include: ABS algorithm (Absolute BalanceSearch), Sequential Similarity Detection Algorithm (Similarity Sequential Detection Algorithm, SSDA) and Normalized Cross Correlation (Normalized Cross Correlation, NCC).

The ABS algorithm is the earliest proposed matching algorithm, which uses the difference between the pixel gray value of the template image and the corresponding point pixel on the search image to represent the correlation. The calculation process can be described as: set the search image to be S, the size to be M×N, the template image to be T, and the size to be m×n, then there is (M-m+1)×(N-n+1) in the search image. matching points, and each matching point corresponds to an m×n search window. Therefore, the matching process is that the template image slides on the search image window according to a specific order, and a calculation is performed every time it slides, and whether the current search window matches the template image is judged according to the correlation value. If the difference is less than the preset threshold, the match is deemed successful, otherwise the match fails. There are generally three calculation methods for the ABS value, the mean absolute difference (MAD) method, the sum of absolute errors (SAD) method, and the sum of squared differences (SSD) method.

Although the calculation amount of the ABS algorithm is small, it only considers the distance of the pixel value. If the brightness of the image to be matched changes greatly, the matching result will be greatly affected. Moreover, different images and templates correspond to different sizes of search windows and background gray values, and the selected thresholds are also different. Therefore, the preset thresholds cannot be suitable for all situations, resulting in a high mismatch rate.

Sequence Similarity Detection Algorithm (SSDA) is an efficient image matching algorithm proposed for traditional template matching algorithm. The basic principle is as follows:

The template image T slides on the search image S at each pixel point and calculates the correlation value, and the position with the largest correlation value corresponds to the position with the best matching. In this process, the SSDA algorithm only calculates part of the matching correlation value, but does not calculate the absolute value of all pixel grayscales, thus reducing the amount of calculation and improving the operation speed. The specific algorithm is a preliminary search, and then a fine search, and the search range is gradually reduced.

The steps of the SSDA algorithm:

Step1: Define the absolute error:

in,

Step2: take a fixed threshold _Tk ;

Step3: Randomly select the pixel value in the sub-image Si, j(x, y), calculate the error between it and the corresponding point of the template image T, and accumulate this difference with the error values generated by other point pairs. Once the accumulated error exceeds T _k , the experiment chooses to stop the accumulation and record the number of accumulation r at the same time. The detection surface of the sequential similarity detection algorithm SSDA can be defined as:

Step4: The point with the largest I(i,j) value is used as the matching point, because this point can only satisfy the total error ∑ε greater than the given threshold after multiple accumulations.

The speed of the SSDA algorithm is improved, but based on the significant reduction in the number of pixel points selected, its accuracy is very low, the matching effect is not good, and it is easily disturbed by noise. Especially when the algorithm enters the information-poor area, it will lead to a significant increase in the false matching rate.

The definition of the normalized correlation algorithm is as follows:

是模板的灰度均值,

为图像中匹配区域的均值。C(x,y)的取值范围为-1～1，值越大表示相关程度越高。Among them, x and y are the size of the reference image, m and n are the size of the template, A(i, j) is the gray value of the pixel in the matching area in the real-time image, x and y are the matching points, and B(ix, iy) is the the pixel gray value in the template,

is the grayscale mean of the template,

is the mean of the matching regions in the image. The value range of C(x,y) is -1 to 1, and the larger the value, the higher the correlation degree.

The NCC algorithm not only has high matching accuracy when the grayscale change and geometric distortion are small, but also has strong anti-white noise ability. However, this algorithm needs to translate the template image one by one in the source image, and calculate the similarity operator of each position, the algorithm complexity is high, and the matching speed is slow.

SUMMARY OF THE INVENTION

Aiming at the above problems in the background art, a method for quickly screening photos of massive power defects based on template matching is provided. By constructing a fast normalized cross-correlation image template matching algorithm, it can quickly filter defect photos from massive inspection data.

A method for quickly screening photos of massive power defects based on template matching according to the present invention, comprising:

S1 obtains the image to be matched and the template image;

S2 constructs the wavelet transform pyramid of the image to be matched and the template image;

S3 performs feature point extraction on each layer of the wavelet transform pyramid, and obtains the feature point pyramid of the image to be matched and the template image;

S4 performs NCC algorithm matching on the top layer of the feature point pyramid of the image to be matched and the template image to obtain the best matching point on the top layer;

S5 uses the top-level best matching point as the center point of the bottom-level image matching, performs NCC algorithm matching, and gradually maps to the bottom-level best matching point.

The invention proposes a fast normalized cross-correlation image matching algorithm based on wavelet pyramid search strategy. On the basis of the traditional NCC algorithm, the algorithm uses the sum table method to calculate the image mean and image variance respectively to reduce the complexity of the operation and the calculation amount of the algorithm; at the same time, the search strategy is improved, the image wavelet pyramid structure is constructed, and the hierarchical matching is used to Improve the efficiency of image matching.

Wavelet transform is a transform analysis method. It inherits and develops the idea of localization of short-time Fourier transform, and at the same time overcomes the shortcomings of window size that does not change with frequency. It can provide a "time-frequency" window that changes with frequency. The ideal tool for signal time-frequency analysis and processing. Its main feature is that it can fully highlight the characteristics of some aspects of the problem through transformation, can analyze the localization of temporal (spatial) frequencies, and gradually refine the signal (function) through scaling and translation operations. Time subdivision, frequency subdivision at low frequency, can automatically adapt to the requirements of time-frequency signal analysis, so that it can focus on any details of the signal and solve the difficult problem of Fourier transform.

Feature point extraction is widely used in target matching, target tracking, 3D reconstruction and other applications. During target modeling, target features are extracted from images. Commonly used features are color, corner, feature point, contour, texture and so on. Harris corner detection is the basis of feature point detection, and the concept of applying the grayscale difference of adjacent pixels is proposed to judge whether it is a corner, edge or smooth area. The Harris corner detection principle is to use the moving window to calculate the grayscale change value in the image. The key process includes converting to grayscale image, calculating difference image, Gaussian smoothing, calculating local extreme value, and confirming corner points.

Specifically, the to-be-matched image is an original image captured by electric power inspection; the template image is a manually marked contaminated defect image.

Further, the step of constructing the wavelet transform pyramid of the image to be matched and the template image comprises: using Haar wavelet transform to decompose the image in two layers, the sub-image of the minimum resolution is used as the top layer of the pyramid, and the original image is used as the bottom layer of the pyramid, forming Three-level pyramid image.

Further, the step of forming a three-layer pyramid image includes: using L to represent a low-pass filter and H to represent a high-pass filter, respectively convolving the rows and columns of the image, and performing 2-to-1 subsampling to decompose the original image. into 4 sub-bands LL ₁ , LH ₁ , HL ₁ and HH ₁ ; LL ₁ reflects the low-frequency components of the original image, which are sub-bands obtained by low-pass filters in both horizontal and vertical directions; LH ₁ reflects the low-frequency components of the original image. The horizontal edge detail is the subband obtained by the horizontal low-pass filter and the vertical high-pass filter; HL ₁ is the subband obtained by the horizontal high frequency and the vertical low frequency; HH ₁ is the subband obtained by the horizontal high frequency and vertical low frequency The sub-band obtained at high frequency in the direction; set the resolution to 1/2 of the original image, and further decompose the sub-band LL ₁ to obtain a total of 4 sub-bands of LL ₂ , LH ₂ , HL ₂ and HH ₂ ; Set as 1/4 of the original image, and further decompose the subband LL ₂ to obtain a total of 4 subbands of LL ₃ , LH ₃ , HL ₃ and HH ₃ .

The high and low frequency of the image is a measure of the intensity change between the various positions of the image. The low-frequency components are mainly a comprehensive measure of the intensity of the entire image. High-frequency components are mainly measures of image edges and contours. If the intensity of each position of an image is equal, the image only has low-frequency components. From the spectrogram of the image, there is only one main peak, which is located at the position of zero frequency. If the intensity of each position of an image changes sharply, there are not only low-frequency components, but also a variety of high-frequency components in the image. From the spectrum of the image, there is not only one main peak, but also multiple side peaks.

Further, the feature point extraction is based on the constructed wavelet transform pyramid, and the Harris feature point detection algorithm with the same parameter settings is used to extract each layer of pyramid images to obtain a feature point pyramid.

Further, perform NCC algorithm matching on the top layer of the feature point pyramid of the image to be matched and the template image, that is, start matching from the low-frequency component image of subband LL ₁ ; the NCC algorithm is:

是模板的灰度均值,

为图像中匹配区域的均值；σ_B是模板图像的方差；By constructing a summation table, the mean value S ₁ (x, y) of the image to be matched and the variance S ₂ (x, y) of the image to be matched are calculated to reduce the amount of calculation; where x and y are the size of the reference image, and m and n are Template size, A(i, j) is the gray value of the pixel in the matching area in the real-time image, (x, y) is the matching point, B(ix, iy) is the gray value of the pixel in the template,

is the grayscale mean of the template,

is the mean value of the matching area in the image; σ _B is the variance of the template image;

The point with the highest matching degree C(x, y) is obtained from the mean and variance of the template image, the accumulated sum of the images to be matched, and the squared accumulated sum as the top-level best matching point.

Further, using the top-level best matching point as the center point of the lower-level image matching, performing NCC algorithm matching, and gradually mapping to the bottom-level best matching point, the steps include: using the best matching point of the upper layer as the lower layer. The center point of image matching is searched again in the neighborhood of the center point of the image to be matched and the template image, and the NCC matching calculation is performed to obtain the best matching point of the lower layer; it is gradually matched to the bottom layer, and as the resolution increases, the cross-correlation The search range for matching is gradually limited, and the accuracy of matching points is gradually improved; finally, the best matching point on the image to be matched is obtained.

In order to understand the present invention more clearly, the specific embodiments of the present invention will be described below with reference to the accompanying drawings.

Description of drawings

Fig. 1 is the algorithm frame diagram of the fast screening method of massive electric power defect photos based on template matching;

2 is a schematic diagram of three-level decomposition of wavelet transform according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a wavelet pyramid search according to an embodiment of the present invention.

Detailed ways

Please refer to FIG. 1 , which is an algorithm framework diagram of a method for quickly screening photos of massive power defects based on template matching according to an embodiment of the present invention.

The original image captured by the power inspection is used as the image to be matched, and the manually marked contaminated defect image is used as the template image. After obtaining the image to be matched and the template image, the image to be matched and the template image are first decomposed by Haar wavelet transform. The sub-image of the resolution is used as the top layer of the pyramid, and the original image is used as the bottom layer of the pyramid, forming a three-layer pyramid image.

As shown in Figure 2, L represents the low-pass filter, H represents the high-pass filter, convolves the rows and columns of the image respectively, and performs sub-sampling of 2 out of 1, and decomposes the original image into 4 sub-bands LL ₁ , LH _1. HL ₁ and HH ₁ ; LL ₁ reflects the low-frequency components of the original image, which are sub-bands obtained by low-pass filters in both horizontal and vertical directions; LH ₁ reflects the horizontal edge details of the original image, which is determined by the horizontal direction. The subband obtained by the low-pass filter and the high-pass filter in the vertical direction; HL ₁ is the subband obtained by the high frequency in the horizontal direction and the low frequency in the vertical direction; HH ₁ is the subband obtained by the high frequency in the horizontal direction and the high frequency in the vertical direction; Set the resolution to 1/2 of the original image, and further decompose the sub-band LL ₁ to obtain 4 sub-bands of LL ₂ , LH ₂ , HL ₂ and HH ₂ in total; set the resolution to 1/4 of the original image , and further decompose the subband LL ₂ to obtain a total of 4 subbands of LL ₃ , LH ₃ , HL ₃ and HH ₃ . So far, the three-level decomposition of the wavelet transform in this embodiment is completed.

Then, based on the constructed image pyramid, the Harris feature point detection algorithm with the same parameter settings is used to extract feature points from each layer of the pyramid image to obtain the feature point pyramid of the image to be matched and the template image;

The top layer of the feature point pyramid of the image to be matched and the template image is matched with the NCC algorithm, that is, the matching starts from the low-frequency component image of the subband LL ₁ ; the NCC algorithm is:

是模板的灰度均值,

为图像中匹配区域的均值；σ_B是模板图像的方差；The algorithm is a further improvement based on the NCC algorithm in the prior art, and the calculation amount is reduced by constructing a summation table to calculate the mean value S ₁ (x, y) of the images to be matched and the variance S ₂ (x, y) of the images to be matched; Among them, x and y are the size of the reference image, m and n are the size of the template, A(i,j) is the gray value of the pixel in the matching area in the real-time image, (x,y) is the matching point, B(ix,iy) ) is the pixel gray value in the template,

is the grayscale mean of the template,

is the mean value of the matching area in the image; σ _B is the variance of the template image;

The point with the highest matching degree C(x, y) is obtained from the mean and variance of the template image, the accumulated sum of the images to be matched, and the squared accumulated sum as the top-level best matching point.

After obtaining the top-level best matching point, take the top-level best matching point as the center point of the lower layer image matching, re-search in the neighborhood of the center point of the image to be matched and the template image, and perform NCC algorithm matching to obtain the best matching point of the lower layer. Matching points; gradually match to the bottom layer. With the increase of resolution, the search range of cross-correlation matching is gradually limited, and the accuracy of matching points is gradually improved; it is gradually mapped to the best matching point at the bottom layer. The above matching process is shown in Figure 3.

In FIG. 3 , the labeled original image is the image at the bottom of the pyramid, which has the highest resolution, and is the defect image that has been manually labeled and polluted as described in the present application. The top-level image marked is the top-level image of the pyramid, and its resolution is the lowest, which is the original image captured by the power inspection described in this application. The best matching point of the top layer described above), and then take this point as the center, match the middle-level matching position in the corresponding high-level matching position of the middle-level image (that is, the best matching point of the next layer described in this application), and then use this point As the center, the best matching position is matched in the corresponding middle-level position of the bottom-level image, and the effect of gradually mapping the top-level pyramid to the best matching point of the bottom-level pyramid is realized. Since the bottom-level image has the highest resolution, this process The accuracy of the matching point pairs is also gradually improving. Finally, the best matching position is output, and the images to be matched are processed accordingly to complete the matching process. In electric power inspection, the present invention can realize the matching of large-resolution, manually marked defect photos in the original images captured by mass inspections to corresponding line segments with small resolution, so as to realize fast and accurate defect images. Matching screening provides an important data basis for intelligent defect identification.

With respect to the prior art, on the basis of the normalized cross-correlation algorithm, the present application adopts the sum table method to calculate the image mean value and the image variance respectively to reduce the complexity of the operation and reduce the calculation amount of the algorithm;

When choosing the feature point matching search strategy, the image wavelet pyramid structure is constructed, and the layered matching is used to improve the efficiency of image matching;

The normalized cross-correlation template matching algorithm is introduced into the cleaning of machine patrol data, and by optimizing the similarity operator and improving the search strategy, a fast normalized cross-correlation image template matching algorithm is constructed to realize the rapid detection of defective images. , accurate matching and screening, providing an important data basis for intelligent defect identification.

The present application can not only reduce the complexity of calculation, reduce the amount of calculation in the operation process, improve the matching speed, but also improve the matching accuracy.

The image matching algorithm based on grayscale correlation uses the grayscale information of the image to establish a similarity measure between the source image and the template image, and then uses a certain search method to find the position of the optimal similarity measure as the matching position. In this application, the manually marked contaminated defect image is used as the matching template, the original inspection image is the template to be matched, and an image matching algorithm based on grayscale correlation is selected to realize rapid screening of the original image.

Aiming at the problems of the traditional normalized cross-correlation algorithm (NCC) with large amount of calculation, slow operation speed and low accuracy rate, on the one hand, the present application improves the traditional normalized cross-correlation algorithm to reduce the computational complexity and reduce the On the other hand, the search strategy is improved, and the idea of layered matching of wavelet pyramid is used, the resolution is from low to high, and the matching is from coarse to fine, which improves the efficiency of image matching and improves the matching accuracy.

The present invention is not limited to the above-mentioned embodiments. If various changes or deformations of the present invention do not depart from the spirit and scope of the present invention, and if these changes and deformations belong to the claims of the present invention and the equivalent technical scope, then the present invention is also These changes and variants are included together.

Claims (7)

1. A method for rapidly screening massive electric power defect photos based on template matching comprises the following steps:

acquiring an image to be matched and a template image;

constructing a wavelet transformation pyramid of the image to be matched and the template image;

extracting feature points of each layer of the wavelet transformation pyramid to obtain a feature point pyramid of the image to be matched and the template image;

carrying out NCC algorithm matching on the top layers of the golden towers of the characteristic points of the image to be matched and the template image to obtain the top layer optimal matching point;

and taking the top layer optimal matching point as a central point of lower layer image matching, performing NCC algorithm matching, and gradually mapping to the bottom layer optimal matching point.

2. The method for rapidly screening the massive electric power defect photos based on the template matching as claimed in claim 1, characterized in that: the image to be matched is an original image shot by power inspection; the template image is a defect image which is artificially marked and polluted.

3. The method for rapidly screening the massive electric power defect photos based on the template matching as claimed in claim 1, wherein the step of constructing the wavelet transformation pyramid of the image to be matched and the template image comprises: and (3) carrying out two-layer decomposition on the image by using Haar wavelet transform, wherein the subgraph with the minimum resolution is used as the top layer of the pyramid, and the original image is used as the bottom layer of the pyramid to form a three-layer pyramid image.

4. The method for rapidly screening massive electric power defect photos based on template matching according to claim 3, wherein the step of forming a three-layer pyramid image comprises: l and H denote low-pass filters, respectively convolving the rows and columns of the image, performing sub-sampling with 1 out of 2, and decomposing the original image into 4 sub-bands LL₁、LH₁、HL₁And HH₁(ii) a Wherein LL is₁Reflecting the low-frequency components of the original image, wherein the low-frequency components are sub-bands obtained by low-pass filters in the horizontal direction and the vertical direction; LH₁Reflecting the horizontal edge details of the original image, wherein the horizontal edge details are sub-bands obtained by a horizontal low-pass filter and a vertical high-pass filter; HL (HL)₁Subbands obtained for high frequency in the horizontal direction and low frequency in the vertical direction; HH (Hilbert-Huang) with high hydrogen storage capacity₁Are subbands obtained by horizontal direction high frequency and vertical direction high frequency; 1/2 with resolution set as the original image, and subband LL₁Further decomposing to obtain LL₂、LH₂、HL₂And HH₂4 total sub-bands; 1/4 with resolution set as the original image, and subband LL₂Further decomposing to obtain LL₃、LH₃、HL₃And HH₃For a total of 4 sub-bands.

5. The method for rapidly screening the massive electric power defect photos based on the template matching as claimed in claim 1, characterized in that: the feature point extraction is based on the constructed wavelet transform pyramid, and a Harris feature point detection algorithm with the same parameter setting is adopted to extract images of each layer of pyramid, so that a feature point pyramid is obtained.

6. The method for rapidly screening the massive electric power defect photos based on the template matching as claimed in claim 4, wherein the method comprises the following steps: carrying out NCC algorithm matching on the top layer of the pyramid of the characteristic points of the image to be matched and the template image, namely, carrying out sub-band LL₁The low-frequency component images of (a) start to match; the NCC algorithm is as follows:

calculating the mean value S of the image to be matched by constructing a summation table₁(x, y), variance S of image to be matched₂(x, y) to reduce the amount of computation; wherein x and y are the size of the reference image, m and n are the size of the template, A (i, j) is the gray value of the pixel in the matching area in the real-time image, (x and y) is the matching point, B (i-x, i-y) is the gray value of the pixel in the template,

is the average of the gray levels of the templates,

the mean value of the matching area in the image is obtained; sigma_BIs the variance of the template image;

and (3) obtaining the point with the highest matching degree C (x, y) by using the mean value and the variance of the template image, the accumulated sum and the square accumulated sum of the images to be matched as the top-layer optimal matching point.

7. The method for rapidly screening massive electric power defect photos based on template matching according to claim 1, wherein the top-layer best matching point is used as a central point of lower-layer image matching, NCC algorithm matching is performed, and the top-layer best matching point is gradually mapped to the bottom-layer best matching point, and the method comprises the following steps: taking the optimal matching point of the upper layer as the central point of the lower layer image matching, searching again in the neighborhood of the central points of the image to be matched and the template image, and performing NCC matching calculation to obtain the optimal matching point of the lower layer; gradually matching to the bottom layer, gradually limiting the search range of cross-correlation matching along with the improvement of resolution, and gradually improving the precision of matching points; and finally obtaining the best matching point on the image to be matched.

Images