CN116843909B - Power line extraction method and device, storage medium and computer equipment - Google Patents

Abstract

This application discloses a power line extraction method and device, storage medium, and computer equipment. The method includes: predicting the coordinates of the electric tower based on the coordinate prediction model on the standardized image of the transmission line corresponding to the transmission line to be detected, and obtaining the predicted coordinates of the electric tower; Perform edge detection on the standardized image of the transmission line, obtain the edge detection result image and calculate the offset angle of the electrical tower; determine the electrical tower positioning frame in the standardized image of the transmission line based on the predicted coordinates of the electrical tower, and determine it based on the electrical tower positioning frame and the offset angle of the electrical tower. Effective power line extraction area; perform straight line detection on the effective power line extraction area, and determine the straight line perpendicular to the offset angle of the power tower among the detected straight lines as the candidate power line, vote on the candidate power line according to the preset power line template, and determine based on the voting results Power lines of transmission lines to be inspected. By combining unsupervised algorithms and supervised algorithms, the accuracy of power line extraction is improved.

Description

Power line extraction method and device, storage medium, and computer equipment

Technical Field

The present application relates to the technical field of power line inspection, and in particular to a power line extraction method and device, a storage medium, and a computer device.

Background Art

With the continuous expansion of the scale of the country's high-voltage transmission lines, the environmental scope covered by the transmission network is becoming more and more complex, and it is becoming more and more difficult to ensure the safe and stable operation of the transmission lines. Traditional transmission line inspections mainly rely on manual ground inspections and manned helicopter inspections. However, high-voltage transmission lines are mostly distributed in sparsely populated mountainous areas with different environments and inconvenient transportation. The timeliness and accuracy of manual inspections are not high, and there are safety risks, while helicopter inspections are less economical. Compared with traditional transmission line inspection methods, the use of unmanned machines such as drones for inspections has the advantages of low cost and easy operation. Unmanned machines such as drones can be used to carry high-resolution aerial cameras or other shooting machines to take aerial images. The method of inspecting high-voltage transmission lines through aerial images is low-cost and has a low safety risk factor.

Based on the relevant features of aerial images, the methods used by domestic and foreign researchers for power line detection in aerial images are mainly divided into two categories: unsupervised methods and supervised methods. The unsupervised method is based on the linear features of power lines, identifying power lines as continuous straight lines, and realizing power line detection through classic straight line detection methods, such as Radon transform, Hough transform, LSD straight line detection, and FLD linear discrimination method. In complex backgrounds, it is easily interfered by objects with linear features such as roads and trees, and there is noise interference in drone aerial images. Direct use of unsupervised methods will lead to false detection and missed detection of power lines.

Supervised methods use deep learning models, usually convolutional neural networks, to train models through a large number of manually labeled data sets. Power line, insulator, and transmission tower detection can all be achieved through supervised learning. For example, the target-based MaskR-CNN and Fast R-CNN detection methods have high detection accuracy, but high detection time cost. Another type is regression-based target detection methods, such as Yolo and SSD algorithms. The above supervised algorithms require a certain number of labeled data sets, and the algorithm performance depends largely on the quality of the data set.

Summary of the invention

In view of this, the present application provides a power line extraction method and apparatus, a storage medium, and a computer device, which improves the accuracy of power line extraction by combining an unsupervised algorithm (straight line detection) and a supervised algorithm (coordinate prediction model).

According to one aspect of the present application, a power line extraction method is provided, the method comprising:

Obtain a transmission line standardized image corresponding to the transmission line to be detected, and perform tower coordinate prediction on the transmission line standardized image according to a coordinate prediction model to obtain tower predicted coordinates;

Performing edge detection on the standardized image of the power transmission line to obtain an edge detection result image, and calculating the tower offset angle based on the edge detection result image;

According to the predicted coordinates of the electric tower, a tower positioning frame is determined in the standardized image of the power transmission line, and according to the tower positioning frame and the tower offset angle, an effective power line extraction area is determined;

The effective power line extraction area is subjected to straight line detection, and the straight lines perpendicular to the tower offset angle are determined as candidate power lines among the detected straight lines. The candidate power lines are voted according to a preset power line template, and the power lines of the transmission line to be detected are determined according to the voting results.

According to another aspect of the present application, a power line extraction device is provided, the device comprising:

The power tower coordinate prediction module is used to obtain the transmission line standardized image corresponding to the transmission line to be detected, and predict the tower coordinates of the transmission line standardized image according to the coordinate prediction model to obtain the tower predicted coordinates;

An electric tower angle calculation module is used to perform edge detection on the standardized image of the power transmission line to obtain an edge detection result image, and calculate the electric tower offset angle based on the edge detection result image;

An effective area determination module is used to determine an electric tower positioning frame in the standardized image of the power transmission line according to the predicted coordinates of the electric tower, and determine an effective power line extraction area according to the electric tower positioning frame and the electric tower offset angle;

The power line extraction module is used to perform straight line detection on the effective power line extraction area, and determine the straight lines that are perpendicular to the tower offset angle among the detected straight lines as candidate power lines, vote on the candidate power lines according to the preset power line template, and determine the power lines of the transmission line to be detected according to the voting results.

According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned power line extraction method is implemented.

According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned power line extraction method when executing the program.

By means of the above technical scheme, the present application provides a power line extraction method and device, storage medium, and computer equipment, which predict the tower coordinates of the transmission line standardized image of the transmission line to be detected according to the coordinate prediction model to obtain the tower prediction coordinates; perform edge detection on the transmission line standardized image and calculate the tower offset angle; determine the tower positioning frame according to the tower prediction coordinates, and determine the effective power line extraction area according to the tower positioning frame and the tower offset angle; perform straight line detection on the effective power line extraction area, and determine the straight line perpendicular to the tower offset angle among the detected straight lines as candidate power lines, vote for the candidate power lines according to the preset power line template, and determine the power line of the transmission line to be detected according to the voting results. Based on the features of aerial images, the predicted coordinates of the tower are obtained according to a supervised algorithm (coordinate prediction model). Then, edge detection is performed on the standardized image of the transmission line to detect accurate and clear edges. The offset angle of the tower is calculated based on the edge detection result image. Combined with the prior knowledge that the power line must be within the range of the power tower and the power line is perpendicular to the power tower, a straight line detection algorithm is used to detect straight lines in the effective power line extraction area (unsupervised algorithm). Finally, a straight line that is approximately perpendicular to the tower offset angle is selected as a candidate power line, and the power line is determined according to the preset power line template, thereby improving the accuracy of power line detection.

The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

FIG1 is a schematic diagram showing a flow chart of a power line extraction method provided in an embodiment of the present application;

FIG2 is a schematic diagram showing a flow chart of another power line extraction method provided in an embodiment of the present application;

FIG3 is a schematic diagram showing a flow chart of another power line extraction method provided in an embodiment of the present application;

FIG4 shows a schematic structural diagram of a power line extraction device provided in an embodiment of the present application;

FIG5 shows a schematic structural diagram of another power line extraction device provided in an embodiment of the present application.

DETAILED DESCRIPTION

The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

In this embodiment, a method for extracting power lines is provided. As shown in FIG1 , the method includes:

Step 101, obtaining a transmission line standardized image corresponding to the transmission line to be detected, and predicting the tower coordinates of the transmission line standardized image according to a coordinate prediction model to obtain tower predicted coordinates.

With the development of unmanned driving technology, drone technology is applied to all walks of life. In the field of power line inspection technology, the use of unmanned machines such as drones for inspection has the advantages of low cost and easy operation. Unmanned machines such as drones can be used to carry high-resolution aerial cameras or other shooting machines to take aerial images. The method of inspecting high-voltage transmission lines through aerial images is low-cost and has a low safety risk factor. Aerial images have the following characteristics: power lines are linear objects and are approximately parallel; the background of power lines is natural scenery; power lines are made of special materials and have uniform brightness; power lines appear as thick or thin lines in aerial images, with opposite pixels on the left and right and partially parallel; power lines have different widths and appearance colors.

In the above-mentioned embodiment of the present application, the power lines are extracted based on the features of the aerial image, so as to detect the transmission lines according to the extracted power lines. Specifically, the basic image of the transmission line (aerial image) corresponding to the transmission line to be detected is obtained, and the basic image of the power line is standardized to obtain a standardized image of the transmission line. For example, an image of a preset size (1280×1280 or 640×640) is captured with the center of the aerial image as a reference point, and finally normalized to a 640×640 size image. The tower coordinates are predicted for the image of the standardized part of the transmission line according to the coordinate prediction model to obtain the tower prediction coordinates. It can be known from the prior knowledge of the tower positioning power lines that the power lines must be distributed within the range of the tower. After the tower position is determined by the coordinate prediction model, the power lines can be extracted according to the tower position, thereby improving the efficiency of power line extraction.

Step 102: Perform edge detection on the standardized image of the power transmission line to obtain an edge detection result image, and calculate the tower offset angle based on the edge detection result image.

Step 103, determining a tower positioning frame in the standardized image of the power transmission line according to the predicted tower coordinates, and determining an effective power line extraction area according to the tower positioning frame and the tower offset angle.

Next, edge detection is performed on the standardized image of the power transmission line to detect accurate and clear edges, which is conducive to further power line extraction. After edge detection, an edge detection result image is obtained, and the tower offset angle is calculated based on the edge detection result image to prepare for subsequent power line extraction.

Next, according to the predicted coordinates of the tower, the tower positioning frame is determined in the standardized image of the transmission line. In particular, according to the prior knowledge of tower positioning power lines, after the position of the tower is determined, the power lines must be distributed within the range of the tower. Therefore, according to the tower positioning frame and the tower offset angle, the effective power line extraction area can be further determined to eliminate the interference of the straight line group outside the power tower range on the power line extraction process.

Step 104, performing straight line detection on the effective power line extraction area, and determining the straight lines perpendicular to the tower offset angle among the detected straight lines as candidate power lines, voting on the candidate power lines according to a preset power line template, and determining the power lines of the transmission line to be detected according to the voting results.

Next, the effective power line extraction area is detected, and the straight lines detected that are perpendicular to the tower offset angle are determined as candidate power lines. The candidate power lines are voted according to the preset power line template, and the power lines of the transmission line to be detected are determined according to the voting results. Combined with the prior knowledge that the power line must be within the power tower range and the power line is perpendicular to the power tower, the tower offset angle is obtained based on the image after edge detection (edge detection result image), and the power line range (effective power line extraction area) is obtained through the tower positioning frame and the tower offset angle, and then the environmental interference outside the power line range is filtered out, and then the straight line detection algorithm is used to detect the straight line, and the straight line that is approximately perpendicular to the tower offset angle is selected as the candidate power line. Finally, the power line is extracted by combining the preset power line template, which improves the accuracy of power line detection.

By applying the technical solution of the present embodiment, the tower coordinates are predicted for the transmission line standardized portion image of the transmission line to be detected according to the coordinate prediction model to obtain the tower predicted coordinates; the edge detection is performed on the transmission line standardized image and the tower offset angle is calculated; the tower positioning frame is determined according to the tower predicted coordinates, and the effective power line extraction area is determined according to the tower positioning frame and the tower offset angle; straight line detection is performed on the effective power line extraction area, and the straight line perpendicular to the tower offset angle among the detected straight lines is determined as the candidate power line, the candidate power lines are voted according to the preset power line template, and the power line of the transmission line to be detected is determined according to the voting result. Based on the features of aerial images, the predicted coordinates of the tower are obtained according to a supervised algorithm (coordinate prediction model). Then, edge detection is performed on the standardized image of the transmission line to detect accurate and clear edges. The offset angle of the tower is calculated based on the edge detection result image. Combined with the prior knowledge that the power line must be within the range of the power tower and the power line is perpendicular to the power tower, a straight line detection algorithm is used to detect straight lines in the effective power line extraction area (unsupervised algorithm). Finally, a straight line that is approximately perpendicular to the tower offset angle is selected as a candidate power line, and the power line is determined according to the preset power line template, thereby improving the accuracy of power line detection.

Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another power line extraction method is provided, as shown in FIG2 , the method includes:

Step 201, obtaining a transmission line standardized image corresponding to the transmission line to be detected, and predicting the tower coordinates of the transmission line standardized image according to a coordinate prediction model to obtain tower predicted coordinates.

In the above embodiments of the present application, a drone can be used to photograph the transmission line to be inspected to obtain an aerial image (a basic image of the transmission line), and the basic image of the transmission line can be standardized, such as cropping and normalization, to obtain a standardized image of the transmission line, which can effectively improve the quality of the aerial power line image and improve the accuracy of power line extraction. The tower coordinates are predicted for the transmission line standardized image according to the coordinate prediction model to obtain the tower predicted coordinates.

Step 202, grayscale adjustment is performed on pixels in the standardized image of the power transmission line to obtain a grayscale image of the power transmission line, the grayscale image of the power transmission line is processed by histogram equalization to obtain a balanced image of the power transmission line, and denoising is performed on the balanced image of the power transmission line according to a Gaussian filter method to obtain a preprocessed image of the power transmission line.

Next, the standardized image of the power transmission line is preprocessed. Specifically, the grayscale of the pixels in the standardized image of the power transmission line is adjusted to obtain a grayscale image of the power transmission line, and the grayscale image of the power transmission line is processed by histogram equalization to obtain a balanced image of the power transmission line. The balanced image of the power transmission line is denoised according to the Gaussian filter method to obtain a preprocessed image of the power transmission line. Among them, the image contrast is adjusted by using image preprocessing techniques such as grayscale and histogram equalization, and the preprocessed image of the power transmission line is obtained after normalization operation, which reduces the amount of calculation for the later power line extraction and also provides convenience for the subsequent further edge detection.

Optionally, step 202 includes:

Step 202-1, adjusting the gray value of each pixel in the standardized image of the power transmission line according to a pixel gray adjustment formula to obtain a gray image of the power transmission line, wherein the pixel gray adjustment formula is:

Gray(i,j)＝0.29*R+0.578*G+0.114*B

Gray(i,j) represents the gray value of the i-th row and j-th column in the pixel matrix corresponding to the standardized image of the power transmission line, and R, G, and B represent the red channel matrix, green channel matrix, and blue channel matrix, respectively.

Step 202-2, equalizing the grayscale distribution in the grayscale image of the power transmission line according to a histogram equalization formula to obtain a transmission line equalized image, wherein the histogram equalization formula is:

n represents the total number of pixels in the grayscale image of the transmission line, k represents the k-level grayscale, _nk represents the number of pixels with grayscale level _rk , L represents the total number of grayscale levels in the grayscale image of the transmission line, and p _r (r) represents the grayscale probability density of the grayscale image of the transmission line.

In the above embodiment of the present application, the power tower and the power line are usually silvery white, and the grayscale image of the blue channel is more easily distinguished from the environment, so the blue channel is used as the grayscale image. For the standardized image of the transmission line, the weighted average method is used for grayscale processing, that is, the grayscale value of each pixel in the standardized image of the transmission line is adjusted according to the pixel grayscale adjustment formula to obtain the grayscale image of the transmission line. The pixel grayscale adjustment formula is:

Gray(i,j)＝0.29*R+0.578*G+0.114

Where Gray(i,j) represents the gray value of the i-th row and j-th column in the pixel matrix corresponding to the standardized image of the power transmission line, and R, G, and B represent the red channel matrix, green channel matrix, and blue channel matrix, respectively.

Since the power lines of the power line standardized image are blurred after graying, the contrast of the power line graying image is adjusted by using histogram equalization, which can make the power line details more obvious. Specifically, the gray distribution in the power line graying image is equalized according to the histogram equalization formula to obtain the power line equalization image, where the histogram equalization formula is:

Where n represents the total number of pixels in the image, k represents a certain gray level, _nk is the number of pixels with gray level _rk , L is the total number of gray levels in the image, and _pr (r) represents the gray level probability density of the image.

Step 203, after adding a first diagonal gradient template and a second diagonal gradient template on the basis of the horizontal gradient template and the vertical gradient template included in the Canny edge detection algorithm, the gradient value of the transmission line preprocessed image is calculated according to the four directional gradient templates.

Step 204, according to the iterative algorithm, generate a maximum threshold and a minimum threshold in the gray value of each pixel of the transmission line preprocessing image, retain the edge pixels corresponding to the gradient values between the maximum threshold and the minimum threshold, and obtain an edge detection result image.

Next, edge detection is performed on the filtered image (power transmission line preprocessing image). The traditional Canny edge detection algorithm uses a 2×2 template to calculate the gradient amplitude and size. Due to the large noise interference, the edge details of the image cannot be detected well. Therefore, the Canny edge detection algorithm is improved, that is, on the basis of the original horizontal gradient template and vertical gradient template, the first diagonal gradient template and the second diagonal gradient template are added, and the gradient value of the power transmission line preprocessing image is calculated according to the four directional gradient templates. Then, according to the iterative algorithm, the highest threshold and the lowest threshold are generated in the gray value of each pixel of the power transmission line preprocessing image, and the edge pixels corresponding to the gradient values between the highest threshold and the lowest threshold are retained to obtain the edge detection result image.

Specifically, the traditional Canny edge detection algorithm is improved by adding two gradient direction templates (the first diagonal direction gradient template and the second diagonal direction gradient template). A total of four gradient templates for horizontal, vertical and diagonal directions are as follows:

The filtered image (pre-processed image of the transmission line) is convolved using the above four templates, where:

The calculation formula of 45° direction gradient is:

The calculation formula for the 135° direction gradient is:

The horizontal gradient is calculated as:

The vertical gradient calculation formula is:

The calculated gradients at 45° and 135° are projected to the horizontal and vertical directions, and then summed to obtain the new gradient values of the x-axis and y-axis, as follows:

Then calculate the gradient size M and direction D of the current pixel gray value. The calculation formula is:

By adding the gradient calculation method in two directions, the gradient values in four directions of the eight pixels around the pixel can be fully considered, so that more image edge information can be obtained and the edge positioning can be made more accurate, thereby greatly reducing the false detection rate and missed detection rate.

Furthermore, due to the limitation of the Canny edge detection algorithm in selecting high and low thresholds through manual experience, it is difficult to obtain the optimal threshold, which affects the power line detection result. The embodiment of the present application uses an iterative algorithm to obtain the optimal high and low thresholds.

Specifically, first set the initial threshold, the formula is:

T{T _k |K＝0}

Among them, T is the initial threshold of the image, K is the number of iterations of the algorithm, Z _max is the maximum grayscale value, and Z _min is the minimum grayscale value. After calculating the initial threshold, the transmission line preprocessing image is divided into two parts: H ₀ above the initial threshold and H ₁ below the initial threshold. The formulas for H ₀ and H ₁ are:

H ₀ = {f(x,y)|f(x,y)>T}

H ₁ = {f(x,y)|f(x,y)<T}

Calculate the grayscale average values _TH and _TL of the two parts respectively, where the formulas of _TH and _TL are:

Here f(x,y) represents the gray value of point (i,j) in the image, where the formulas for N ₀ (i,j) and N ₁ (i,j) are:

Then calculate the new threshold TN, the formula is:

When the final iteration threshold is equal to the initial threshold or meets the set reasonable error range, the iteration stops, otherwise the iteration continues to run, and finally the best _TH and _TL are obtained, so that the interference of noise on the threshold selection is greatly reduced.

By performing edge detection on the Gaussian filtered image (preprocessed image of the transmission line), an image with less interference is obtained, and the Canny edge detection algorithm is improved by introducing 45° and 135° gradient templates, so that the gradient values of the four directions of the eight pixels around the pixel are fully considered, so that more image edge information can be obtained, making edge positioning more accurate, greatly reducing the false detection rate and missed detection rate. In view of the numerical relationship between the high and low thresholds in the traditional Canny edge detection algorithm, the optimal dual threshold is obtained through an iterative method. Compared with the traditional Canny edge detection algorithm, the interference of noise on the threshold selection is reduced, and the detection accuracy and robustness are greatly improved.

Step 205, calculating the potential lines of force and the potential lines of force slope corresponding to the edge detection result image according to the Hough transform algorithm, and determining a group of potential lines of force that are parallel to each other according to the potential lines of force slope.

Step 206, clustering the slopes of the potential power line groups according to the K-means++ clustering algorithm, determining a target class with the largest number of slopes among the multiple classes, obtaining the cluster center slope of the target class as the angle θ ₁ , and calculating the tower offset angle θ ₂ according to the angle θ ₁ and the tower offset angle calculation formula, wherein the tower offset angle calculation formula is:

Next, the potential power lines and the potential power line slopes corresponding to the edge detection result image are calculated according to the Hough transform algorithm. According to the potential power line slopes, the potential power line groups parallel to each other are determined. According to the K-means++ clustering algorithm, the slopes of the potential power line groups are clustered to determine the target class with the largest number of slopes among multiple classes. The cluster center slope of the target class is obtained as the angle θ ₁ , and the tower offset angle θ ₂ is calculated according to the angle θ ₁ and the tower offset angle calculation formula.

Specifically, the Hough transform algorithm is a type of image feature extraction technology. Based on the duality of points and lines, the shape extraction problem is converted into a peak value calculation problem in parameter space. The equation for converting a straight line into parameter space can be expressed as:

_rθ ＝ _xθcosθ + _yθsinθ

In the formula, r _θ represents the distance from the origin to the straight line in the rectangular coordinate system, θ represents the angle between the straight line and the x-axis, and for each pair (r _θ ,θ) represents the straight line passing through (x _θ ,y _θ ).

Next, the local maximum value calculated by the parameter space accumulator corresponds to the specific shape, and the specific shape in the original image is obtained. Specifically, a two-dimensional array (r, θ) of the parameter space accumulator is established, and the accumulator array is obtained according to the straight line polar coordinates of all the pixels to be tested in the image, and the accumulator array is added by 1 to obtain the (r′, θ′) corresponding to the local maximum value in the accumulator, and the corresponding straight line segment is extracted.

Since power lines run through the entire image in an aerial image, the Hough transform algorithm has accurate detection results for long lines, and the detection threshold can be increased to filter out some short straight lines to reduce interference. In order to obtain potential power line groups, the embodiment of the present application applies the Hough transform algorithm to the edge detection result image after detection by the improved Canny edge detection algorithm to obtain the line group parameters (r, θ).

For the straight lines detected by the Hough transform algorithm, not all straight lines are necessarily the required power lines, so the straight line results obtained need to be filtered. According to prior knowledge, in aerial images, the power lines are parallel to each other, so the slopes of the straight lines are filtered and the parallel pairs are saved. At the same time, since the power lines run through the entire image, increasing the length of the lowest line segment detected by the Hough transform algorithm is beneficial to classification in the clustering algorithm.

K-Means clustering algorithm is one of the data mining algorithms. Since the influence of K random initial cluster centers on the convergence of clustering results is uncertain, in order to improve the convergence speed, K-means++ is selected in the embodiment of the present application. Specifically, a random point is selected from the input data set X as the first cluster center, and the distance D(x) between each point x in the data set X and the selected cluster center is determined. Then, the p value of the possibility that the sample point x will be selected as the cluster center is determined. The formula for calculating the p value is:

Select sample points with high probability p values until the necessary k cluster centers are selected as new cluster centers. Then, the K-means algorithm is performed using the selected k initial cluster centers.

The embodiment of the present application uses the K-means++ algorithm to cluster the slopes of the parallel line group, and obtains θ ₁ with the most count results. The tower angle θ ₂ is calculated by θ ₁ , and the formula is:

To this end, K-Means clustering is used to obtain the tower tilt angle θ ₂ .

Step 207, determining a tower positioning frame in the standardized image of the power transmission line according to the predicted tower coordinates, and determining an effective power line extraction area according to the tower positioning frame and the tower offset angle.

Step 208, perform straight line detection on the effective power line extraction area, and determine the straight lines that are perpendicular to the tower offset angle among the detected straight lines as candidate power lines, vote on the candidate power lines according to the preset power line template, and determine the power lines of the transmission line to be detected according to the voting results.

Next, according to the predicted coordinates of the power tower, the power tower positioning frame is determined in the standardized image of the power transmission line, and according to the power tower positioning frame and the power tower offset angle, the effective power line extraction area is determined, for example, the power tower offset frame is determined according to the power tower positioning frame and the power tower offset angle, and the upper left corner point and the lower right corner point (or the upper right corner and the lower left foot) of the power tower offset frame are extended parallel to the power line _θ1 to obtain the effective power line extraction area, so as to eliminate the interference of the straight line outside the power tower range on the power line extraction. The effective power line extraction area is detected by straight lines, and the straight lines perpendicular to the tower offset angle among the detected straight lines are determined as candidate power lines, and the candidate power lines are voted according to the preset power line template, and the power lines of the power transmission line to be detected are determined according to the voting results.

At present, the implementation method of unsupervised power line detection algorithm is to regard power lines as straight lines, remove noise through Gaussian filtering method, use Canny operator for edge detection, extract object edges, and then combine Hough transform algorithm to extract straight lines in the image. Finally, connect the power lines through parallel constraints and straight line grouping methods. Among them, the Canny edge detection algorithm has the characteristics of few calculation steps and easy implementation, but due to environmental interference such as noise, roads and trees, there will be problems of pseudo edges and missing edges, and double edges are prone to appear in power line detection, affecting the accuracy of edge positioning. The Hough transform algorithm has good anti-interference characteristics, but it has high time and space complexity, only detects the direction of the straight line, loses the information of the length of the straight line, and in complex backgrounds, it is interfered by trees and ridges, and the performance is significantly affected. The power lines cannot be fully detected, and irrelevant line segments will be detected. The embodiment of the present application improves the Canny edge detection algorithm based on the image features of aerial images, introduces 45° and 135° gradient templates, and at the same time, based on the numerical relationship between high and low thresholds in the traditional Canny edge detection algorithm, obtains the optimal dual threshold through an iterative method, thereby improving the accuracy and robustness of edge detection, adopts the Hough transform algorithm to extract power lines, and uses the Kmeans++ clustering method to obtain the tower offset angle, thereby improving the accuracy of power line extraction.

By applying the technical solution of this embodiment, the predicted coordinates of the tower are obtained according to the coordinate prediction model. The standardized image of the transmission line is preprocessed to obtain the preprocessed image of the transmission line. The Canny edge detection algorithm is improved, and the first diagonal gradient template and the second diagonal gradient template (for example, 45° and 135° gradient templates) are introduced to calculate the gradient value of the preprocessed image of the transmission line, and the highest threshold and the lowest threshold are determined according to the iterative algorithm, and the edge pixels corresponding to the gradient values between the highest threshold and the lowest threshold are retained to obtain the edge detection result image. The potential power line group is determined according to the Hough transform algorithm, and the tower offset angle is calculated according to the K-means++ clustering algorithm and the potential power line group, and the effective power line extraction area is determined according to the predicted coordinates of the tower. Straight line detection is performed on the effective power line extraction area, and the straight line perpendicular to the tower offset angle among the detected straight lines is determined as the candidate power line, and finally the power line of the transmission line to be detected is determined by voting. Based on the image features of aerial images, the Canny edge detection algorithm is improved by introducing 45° and 135° gradient templates, so that the gradient values of the four directions of the eight pixels around the pixel are fully considered. According to the numerical relationship between the high and low thresholds in the traditional Canny edge detection algorithm, the optimal dual threshold is obtained through an iterative method, which improves the precision and robustness of edge detection and the accuracy of power line extraction.

Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another power line extraction method is provided, as shown in FIG3, the method includes:

Step 301, obtain a transmission line standardized image corresponding to the transmission line to be detected, and predict the tower and insulator coordinates of the transmission line standardized image according to the coordinate prediction model to obtain the tower predicted coordinates, insulator predicted coordinates and insulator predicted center point.

Step 302: Perform edge detection on the standardized image of the power transmission line to obtain an edge detection result image, and calculate the tower offset angle based on the edge detection result image.

In the above embodiment of the present application, a transmission line standardized image corresponding to the transmission line to be detected is obtained, and the tower and insulator coordinates are predicted for the transmission line standardized image according to the coordinate prediction model, so as to obtain the tower prediction coordinates, the insulator prediction coordinates and the insulator prediction center point. Edge detection is performed on the transmission line standardized image to obtain an edge detection result image, and the tower offset angle is calculated based on the edge detection result image to prepare for subsequent power line extraction.

Step 303: determine a tower positioning frame in the standardized image of the transmission line according to the predicted coordinates of the tower, rotate the tower positioning frame according to the tower direction and the tower offset angle _θ2 , and magnify the rotated tower positioning frame according to a preset magnification factor with the center point of the rotated tower positioning frame as the center to obtain a tower offset frame, wherein the tower positioning frame is a rectangle.

Step 304: determine a first extension point and a second extension point according to the diagonal corners of the tower offset frame, and determine an effective power line extraction area according to the first extension point and the second extension point.

Specifically, the tower positioning frame is determined in the standardized image of the transmission line according to the predicted coordinates of the tower, and the power line range (effective power line extraction area) is obtained by rotating the tower positioning frame and adding a certain offset. Specifically, the tower positioning frame is rotated according to the tower offset angle _θ2. In particular, when rotating, it is necessary to distinguish the direction of the tower in the image so that the tower as a whole can be displayed in the rotated tower positioning frame to avoid missed detection. The four points of the tower positioning frame rectangle are added with a preset offset, or the rotated tower positioning frame is enlarged according to the preset magnification factor with the center point of the rotated tower positioning frame as the center to obtain the tower offset frame. The preset offset is specifically set according to different detection lines. According to the diagonal of the tower offset frame, the first extension point and the second extension point (the upper left corner point and the lower right corner point, or the upper right corner and the lower left foot) are determined so as to determine the effective power line extraction area according to the first extension point and the second extension point.

After determining the effective power line extraction area, verification can be performed. Specifically, for a point in space, it can be determined whether it is on the left or right side of the line. Assuming that the line has two points A (x ₁ , y ₁ ) and B (x ₂ , y ₂ ) on it, and the direction of the line points from A to B, then the line can be expressed as:

αx+βy+γ＝0

Among them, α is y ₂ -y ₁ , β is x ₂ -x ₁ , and γ is x ₂ *y ₁ -x ₁ *y ₂ .

C＝αx _d +βy _p +γ

The inspector can calculate C to determine which side of the line a point D (x _d , y _d ) in space is on.

Step 305 , performing line detection on the effective power line extraction area according to the LSD line detection algorithm to obtain candidate power line segments to be fitted, and putting the candidate power line segments to be fitted into a line segment pool.

Step 306 : according to the distances between the candidate power line segments to be fitted in the line segment pool, the candidate power line segments to be fitted whose distances are less than a preset distance threshold are put into a candidate power line segment group to be fitted.

Step 307, fitting all the candidate power line segments in the candidate power line segment group to be fitted according to the least square method to obtain fitting straight lines, and determining the fitting straight lines perpendicular to the tower offset angle among the fitting straight lines as candidate power lines.

In the effective power line extraction area, the LSD line detection algorithm is used for line detection. LSD is a local extraction algorithm for lines. Its advantages are that it is faster than the Hough transform algorithm and is designed to be parameter-free on digital images. However, due to the self-increase nature of the local detection algorithm, occlusion and local blur, long line segments are often cut into multiple straight lines, which does not exist in the Hough transform algorithm. The LSD line detection algorithm is selected because it can adapt to various complex environments without parameter adjustment, and line detection is better than the Hough transform algorithm.

According to the LSD straight line detection algorithm, the effective power line extraction area is detected to obtain the candidate power line segments to be fitted, and the candidate power line segments to be fitted are filtered and merged. Specifically, the candidate power line segments to be fitted are placed in a line segment pool. In the line segment pool, some close line segments are merged by setting a preset distance threshold, and the line segments with a distance less than the distance threshold are divided into a group (according to the distance between each candidate power line segment to be fitted in the line segment pool, the candidate power line segments to be fitted with a distance less than the preset distance threshold are placed in the candidate power line segment group to be fitted), the preset distance threshold can be set to 7 pixel values, for example, and the distance between each candidate power line segment to be fitted can be calculated using the following formula:

Where k is the slope of the current line segment, I is the intercept, and then the line segment group of each line is obtained. Then, the straight line is fitted by the least squares method, and the straight line that is approximately perpendicular to the tower angle obtained by Kmeans++ clustering is selected as the candidate power line.

Step 308, according to the predicted coordinates of the insulator, obtain a standard image of the tower in the standard template library of the tower corresponding to the transmission line to be detected, map the standard image of the tower to the standardized image of the transmission line according to the preset power line template, vote for the candidate power lines according to the distance between the predicted center point of the insulator and the candidate power lines, and determine the power lines of the transmission line to be detected according to the voting results, wherein the preset power line template is obtained by calculating the homography matrix of mapping the standardized image of the transmission line to the standard image of the tower.

In computer vision, homography is essentially a property that exists between image transformations. According to the different transformation properties, it can be divided into rigid transformation, affine transformation and projective transformation. Among them, rigid transformation refers to a transformation form that only contains translation and rotation, which only changes the position and direction of the original image without changing the shape. Compared with rigid transformation, affine transformation further changes the shape of the image, but maintains the parallel relationship of the original image during the transformation process.

The transformation property between planes is essentially the change in the position of a point, which can be represented by a matrix. Assuming that point Q in the original plane becomes Q′ in another plane after projection transformation, the correspondence between the point in the two planes can be expressed as follows through the homography constraint:

Q′＝AQ

Among them, A is called the homography matrix, which is generally 3*3 in size. After expansion, the transformation between two points can be expressed as:

Depending on the specific transformation process, the values of the parameters in the homography matrix are also different. In general, we need to pay attention to how to obtain the matrix A, so that for any point in the original plane, the position of the point on another plane can be calculated through the homography matrix.

Expand Q′＝AQ to get the following three equations:

x′＝a ₁₁ x+a ₁₂ y+a ₁₃

y′＝a ₂₁ x+a ₂₂ y+a ₂₃

1＝a ₃₁ x+a ₃₂ y+1

After multiplication, we get:

xa ₁₁ +ya ₁₂ +a ₁₃ -xx′a ₃₁ -yx′a ₃₂ =x′

xa ₂₁ +ya ₂₂ +a ₂₃ -xy′a ₃₁ -yy′a ₃₂ =y′

The final unified expression is in the matrix form Ba=b, that is:

There are 8 unknowns in the formula. Two equations can be obtained from a pair of matching points. According to polynomial theory, at least four pairs of matching points are required to solve the homography matrix. Multiply both sides by B ^T at the same time, and then use the properties of the inverse matrix to move the terms, and the solution formula for a is:

B ^T Ba = B ^T b

a＝(B ^T B) ^-1 B ^T b

For multiple multi-view aerial images of the same location, the premise of homography is that two plane spaces under different viewpoints belong to the same plane in the actual space. To solve the homography matrix, it is first necessary to find a common plane under different viewpoints. For aerial power line images (standardized images of transmission lines), the planes of the parts where the towers and insulators are located meet this condition. Therefore, the embodiment of the present application calculates the homography matrix under different viewpoints based on the position of the center of the calibrated insulator detection frame (the tower standard image obtained from the tower standard template library corresponding to the transmission line to be detected) and the position of the center point of the predicted coordinates of the insulator predicted by the coordinate prediction model, and selects matching point pairs in two power line images (the tower standard image and the transmission line standardized image corresponding to the same tower). In order to make the homography matrix calculation more accurate and make full use of the detection information, six pairs of matching points are selected for the homography matrix calculation.

In order to verify the accuracy of the obtained homography matrix, the embodiment of the present application maps the center point of the actual frame (standard image of the electric tower) in the standard template to the standardized image of the transmission line through the homography matrix between two viewing angles, and compares the coordinates of the center point of the predicted insulator coordinates with the mapped standard insulator center point coordinates, showing that the error between the mapped standard insulator center point coordinates and the insulator predicted coordinate center point coordinates is small. Therefore, the insulator coordinates in the standard template are mapped to the image of the power line to be detected through homography transformation, and the power line candidate group is voted according to the distance between the insulator center point and the power line to obtain the corresponding power line.

By applying the technical solution of this embodiment, first, perform image standardization operations, and crop and normalize the acquired drone aerial images. Secondly, preprocess the image, use the coordinate prediction model to obtain the power tower position (power tower prediction coordinates), perform grayscale and histogram equalization image preprocessing on the standardized image of the transmission line to obtain an image with less interference, and combine the tower position information to use Gaussian filtering to remove background information outside the power tower range (transmission line preprocessed image). Thirdly, perform edge detection on the transmission line preprocessed image, and use the improved Canny edge detection algorithm to obtain accurate and clear image edges. Finally, the power lines are extracted based on prior knowledge. The image after edge detection uses K-means clustering to obtain the angle of the power tower (tower offset angle), and the power line range (effective power line extraction area) is obtained by rotating the tower positioning frame and adding a certain offset. The environmental interference outside the power line range is filtered out, and the LSD line detection algorithm is used to detect the power lines. Finally, the lines are merged, and the lines that are approximately perpendicular to the tower offset angle are selected as candidate power lines in the aerial image. The power line template and the homography matrix are combined to vote to determine the line where the power line is located, thereby improving the accuracy of power line extraction.

In an embodiment of the present application, optionally, the coordinate prediction model is an improved model of the YOLOv7 model, and the coordinate prediction model is obtained by replacing the SiLU activation function in the YOLOv7 model with the Mish activation function, and replacing the CIou loss function in the YOLOv7 model with the SIoU loss function.

In an embodiment of the present application, optionally, a method for training a coordinate prediction model is to obtain a plurality of basic images of a data set including electric towers, perform standardization processing on the basic images of the data set to obtain a standardized image of the data set, and use a preset data annotation tool to annotate the electric towers and insulators contained in the standardized image of the data set to generate an electric tower positioning data set; divide the electric tower positioning data set into a tower positioning training data set of a preset first ratio, a tower positioning test data set of a preset second ratio, and a tower positioning verification data set of a preset third ratio without intersection by uniform random sampling; input the tower positioning training data set, the tower positioning test data set, and the tower positioning verification data set into a coordinate prediction model for training to obtain a trained coordinate prediction model.

The supervised learning power line detection method is implemented by establishing a convolutional neural network model, constructing a manually annotated power line dataset, extracting and classifying the preprocessed aerial power line dataset images, filtering the feature map through structured information, and obtaining power line information.

In supervised learning methods, Mask R-CNN and Fast R-CNN are divided into two steps: generating candidate boxes, CNN extracting features, generating classifications, and bounding box regression. The R-CNN series can continuously improve accuracy by optimizing the network, but the two detection steps reduce the detection speed and have a high computational load. The YOLO series divides the input image into S*S grids, sets K predicted bounding boxes and C target classification scores for the grid, and then sorts the K predicted box confidences and C target bounding boxes by the final score, and uses the grid with the highest score as the final prediction result. Experimental results show that the YOLO network runs 42 times faster than the Fast R-CNN network and twice as fast as the Faster R-CNN network. However, since each grid only outputs one result, misjudgment and missed detection may occur when the grid is too large or the power line target in the aerial image is too small. When using supervised learning methods for power line detection, high requirements are placed on the quality of the data set. Most power line data sets are non-public data sets, and there are too few images of broken power lines, hanging foreign objects, fireworks, etc. in the public data sets. This makes it difficult to use supervised learning methods for training to meet the detection requirements for safe and stable operation of the power system.

In the above embodiment of the present application, a data set is produced for the high-voltage transmission line images obtained by aerial photography. The center of the aerial image (including multiple data set basic images of the power tower) is used as a reference point to capture 1280×1280 and 640×640 images, which are normalized to 640×640. Labelme is used to annotate the power towers in the image to generate a data set standardized image (data set). Since the aerial images contain multiple backgrounds, all aerial images are from drones along the line. Specific data sets may include, for example: grassland background (400 images), forest background (500 images), farmland background (350 images), water background (50 images), and city background (100 images).

The dataset is augmented by rotating the image at intervals of 45° to expand it to 16 times its original size. In subsequent training, the image weight coefficient is used to select images according to the number of label boxes during training. The more label boxes there are, the greater the weight, and the probability of being sampled will increase accordingly. For label allocation, consistent with the YOLOv7 model, the SimOTA strategy is adopted, and k positive samples are adaptively and dynamically allocated to each label box during training, which significantly reduces the training time compared to OTA.

Secondly, the tower positioning data set is divided into a tower positioning training data set of a preset first ratio, a tower positioning test data set of a preset second ratio, and a tower positioning verification data set of a preset third ratio without intersection. For example, the ratio of the training set (tower positioning training data set), the test set (tower positioning test data set), and the verification set (tower positioning verification data set) can be set to 6:2:2. Finally, the 640×640×3 image in the processed data set is input into the YOLOv7 network.

The YOLOv7 network is divided into two parts: Backbone and Head. Backbone consists of Conv convolution module, ELAN module, and MP-1 module. Conv in YOLOv7 is not an ordinary convolution layer. It is composed of ordinary two-dimensional convolution, BN layer, and SiLU activation function. Head consists of Conv convolution module, SPPCSPC module, MP-2 module, and Detect module.

The SiLU activation function is a Sigmoid weighted linear combination function commonly used in machine learning.

SiLU(x)＝x*Sigmoid(x)

It is obtained by linearly combining the input x and the Sigmoid function. It has the excellent characteristics of no upper bound, lower bound, smoothness, and non-monotonicity. No upper bound avoids the saturation caused by the upper bound and effectively reduces the disappearance of the gradient. The lower bound can reduce overfitting and has a certain regularization effect. Non-monotonicity ensures that negative input produces negative output, thereby improving the network's expression ability.

A large number of experiments have shown that the Mish activation function

Mish(x)＝x*tanh(ln(1+e ^x ))

The invention has almost the same characteristics as the SiLU function and has better accuracy. The present invention is based on the YOLOv7 model and replaces the SiLU activation function in Conv with the Mish activation function. The data set in the present application improves the mAP value by 1% by replacing the activation function.

The target detection loss function mainly consists of two parts: classification loss and regression positioning loss. As an indispensable part of the target detection loss function, the loss function used by YOLOv7 is CIoU, which considers three geometric parameters: overlapping area, center point distance and aspect ratio. On the basis of DIoU function, the length and width losses are added. The CIoU loss function formula is:

Among them, α is the weight parameter, ρ is the Euclidean distance between two points, v is the aspect ratio consistency of the prediction box and the target box, and the calculation formula of v is:

Where w _gt , h _gt , w, h represent the width and height of the real box and the predicted box respectively.

The CIou function does not consider the angle problem, which makes the convergence slow and inefficient. Therefore, the SIoU function is introduced to introduce the matching direction of the predicted box and the real box to speed up the convergence of the model. The SIoU function consists of four cost functions. The angle cost function is defined as follows:

in, Represent the center coordinates of the real box and the predicted box respectively,

Due to the introduction of angle loss, the distance cost function is modified and the formula is as follows:

Here ρ _x , ρ _y and γ are:

γ＝2-Λ

Among them, _cw and _ch are the width and height of the minimum bounding rectangle of the real box prediction box, respectively, and the shape cost function is defined as:

Here w _w and w _h are:

Finally, the IOU cost function is defined as:

Among them, B and B ^GT represent the candidate box and the standard box respectively, and the final SiOU loss function formula is:

To this end, the improved YOLOv7 model is used to train the data set to obtain the center point, width, and height of the prediction box of the power tower and insulator. The acquired drone aerial images are cropped and normalized (image standardization operation is performed), and then image preprocessing is performed. The improved YOLOv7 network is used to train the data set, and the Mish activation function is used to replace the SiLU activation function, which effectively reduces the disappearance of the gradient and the overfitting phenomenon. It has a certain regularization effect and allows negative output, improves the expression ability of the network, and accelerates the convergence and accuracy of the model. The SIoU loss function is used to replace the CIou loss function, and the angle parameter is considered. The training speed and the accuracy of reasoning are effectively improved, so that the Map value of the tower detection is improved by 2% (YOLOv7 0.864, this application embodiment 0.881).

By applying the technical solution of this embodiment, the YOLOv7 network model, Canny edge detection algorithm, Hough transform algorithm and power line extraction method are improved. The YOLO model is a classic single-stage detection network with the characteristics of fast running speed and small memory usage, but the detection accuracy is not ideal. By combining the YOLOv7 network for improvement, the Mish activation function is used to replace the original activation function, SIoU is used as the loss function, and the angle term is added to the previous loss function to accelerate the model convergence and improve the model's detection performance for power towers. At the same time, LSD line detection is used for straight line detection , and perform straight line screening and least square connection to obtain candidate power lines. Based on the detected straight line feature information, such as length, width, and direction information, the structural information of the power line and the YOLOv7 power tower detection results are used to determine the power line range (effective power line extraction area) by rotating the tower positioning frame and increasing the offset, and filter out the messy straight lines outside the power line range. Finally, the center point, width, and height information of the predicted frame of the power tower and insulator detected by YOLOv7 are combined with the standard template (preset power line template) for projection transformation, and the candidate power lines and each insulator are voted according to the distance to obtain the final power line.

Further, as a specific implementation of the method of FIG. 1 , an embodiment of the present application provides a power line extraction device, as shown in FIG. 4 , the device includes:

The power tower coordinate prediction module 401 is used to obtain a transmission line standardized image corresponding to the transmission line to be detected, and perform power tower coordinate prediction on the transmission line standardized image according to a coordinate prediction model to obtain power tower predicted coordinates;

The tower angle calculation module 402 is used to perform edge detection on the standardized image of the power transmission line to obtain an edge detection result image, and calculate the tower offset angle based on the edge detection result image;

An effective area determination module 403 is used to determine an electric tower positioning frame in the standardized image of the power transmission line according to the predicted coordinates of the electric tower, and determine an effective power line extraction area according to the electric tower positioning frame and the electric tower offset angle;

The power line extraction module 404 is used to perform straight line detection on the effective power line extraction area, and determine the straight lines that are perpendicular to the tower offset angle among the detected straight lines as candidate power lines, vote on the candidate power lines according to the preset power line template, and determine the power lines of the transmission line to be detected according to the voting results.

Optionally, the tower angle calculation module 402 is further used for:

Calculate the potential lines of force and the slopes of the potential lines of force corresponding to the edge detection result image according to the Hough transform algorithm, and determine a group of potential lines of force that are parallel to each other according to the slopes of the potential lines of force;

The slopes of the potential power line groups are clustered according to the K-means++ clustering algorithm, a target class with the largest number of slopes among the multiple classes is determined, the cluster center slope of the target class is obtained as the angle θ ₁ , and the tower offset angle θ ₂ is calculated according to the angle θ ₁ and the tower offset angle calculation formula, wherein the tower offset angle calculation formula is:

Optionally, the effective area determination module 403 is further configured to:

The tower positioning frame is rotated according to the tower direction and the tower offset angle _θ2 , and the rotated tower positioning frame is enlarged according to a preset magnification factor with the center point of the rotated tower positioning frame as the center to obtain a tower offset frame;

A first extension point and a second extension point are determined according to the diagonal corners of the tower offset frame, and an effective power line extraction area is determined according to the first extension point and the second extension point.

Optionally, the power line extraction module 404 is further configured to:

Perform straight line detection on the effective power line extraction area according to the LSD straight line detection algorithm to obtain candidate power line segments to be fitted, and put the candidate power line segments to be fitted into a line segment pool;

According to the distances between the candidate power line segments to be fitted in the line segment pool, the candidate power line segments to be fitted whose distances are less than a preset distance threshold are put into a candidate power line segment group to be fitted;

All the candidate power line segments to be fitted in the candidate power line segment group to be fitted are fitted according to the least square method to obtain a fitting straight line, and a fitting straight line perpendicular to the tower offset angle among the fitting straight lines is determined as a candidate power line.

Optionally, the tower coordinate prediction module 401 is further used for:

The insulator coordinates are predicted for the transmission line standardization image according to the coordinate prediction model to obtain the insulator prediction coordinates and the insulator prediction center point.

Optionally, the power line extraction module 404 is further configured to:

According to the preset power line template, after mapping the standard image of the power tower to the standardized image of the transmission line, the candidate power lines are voted according to the distance between the predicted center point of the insulator and the candidate power lines, and the power lines of the transmission line to be detected are determined according to the voting results, wherein the preset power line template is obtained by calculating the homography matrix of mapping the standardized image of the transmission line to the standard image of the power tower.

Furthermore, an embodiment of the present application provides another power line extraction device, as shown in FIG5 , the device includes:

The tower coordinate prediction module 501 is used to obtain a transmission line standardized image corresponding to the transmission line to be detected, and perform tower coordinate prediction on the transmission line standardized image according to a coordinate prediction model to obtain tower predicted coordinates;

The tower angle calculation module 502 is used to perform edge detection on the standardized image of the power transmission line to obtain an edge detection result image, and calculate the tower offset angle based on the edge detection result image;

An effective area determination module 503 is used to determine a tower positioning frame in the standardized image of the power transmission line according to the predicted coordinates of the tower, and determine an effective power line extraction area according to the tower positioning frame and the tower offset angle;

The power line extraction module 504 is used to perform straight line detection on the effective power line extraction area, and determine the straight lines that are perpendicular to the tower offset angle among the detected straight lines as candidate power lines, vote on the candidate power lines according to the preset power line template, and determine the power lines of the transmission line to be detected according to the voting results.

The line image processing module 505 is used to adjust the grayscale of pixels in the standardized image of the power transmission line to obtain a grayscale image of the power transmission line, perform histogram equalization on the grayscale image of the power transmission line to obtain a balanced image of the power transmission line, and perform denoising on the balanced image of the power transmission line according to the Gaussian filtering method to obtain a preprocessed image of the power transmission line; based on the horizontal gradient template and the vertical gradient template included in the Canny edge detection algorithm, add a first diagonal gradient template and a second diagonal gradient template, and calculate the gradient value of the preprocessed image of the power transmission line according to the four directional gradient templates; according to the iterative algorithm, generate a maximum threshold and a minimum threshold in the grayscale value of each pixel of the preprocessed image of the power transmission line, retain the edge pixels corresponding to the gradient values between the maximum threshold and the minimum threshold, and obtain an edge detection result image.

The prediction model improvement module 506 is used to replace the SiLU activation function in the YOLOv7 model with the Mish activation function, and replace the CIou loss function in the YOLOv7 model with the SIoU loss function, wherein the coordinate prediction model is an improved model of the YOLOv7 model.

The prediction model training module 507 is used to obtain multiple basic images of data sets containing power towers, standardize the basic images of the data sets to obtain standardized images of the data sets, and use a preset data annotation tool to annotate the power towers and insulators contained in the standardized images of the data sets to generate a power tower positioning data set; divide the power tower positioning data set into a tower positioning training data set of a preset first ratio, a tower positioning test data set of a preset second ratio, and a tower positioning verification data set of a preset third ratio without intersection by uniform random sampling; input the tower positioning training data set, the tower positioning test data set, and the tower positioning verification data set into a coordinate prediction model for training to obtain a trained coordinate prediction model.

Optionally, the line image processing module 505 is further used to:

The grayscale value of each pixel in the standardized image of the power transmission line is adjusted according to a pixel grayscale adjustment formula to obtain a grayscale image of the power transmission line, wherein the pixel grayscale adjustment formula is:

Gray(i,j)＝0.29*R+0.578*G+0.114*B

Gray(i,j) represents the gray value of the i-th row and j-th column in the pixel matrix corresponding to the standardized image of the power transmission line, and R, G, and B represent the red channel matrix, green channel matrix, and blue channel matrix, respectively.

Optionally, the line image processing module 505 is further used to:

The grayscale distribution in the grayscale image of the power transmission line is equalized according to a histogram equalization formula to obtain a transmission line equalized image, wherein the histogram equalization formula is:

n represents the total number of pixels in the grayscale image of the transmission line, k represents the k-level grayscale, _nk represents the number of pixels with grayscale level _rk , L represents the total number of grayscale levels in the grayscale image of the transmission line, and p _r (r) represents the grayscale probability density of the grayscale image of the transmission line.

Optionally, the tower angle calculation module 502 is further used for:

Calculate the potential lines of force and the slopes of the potential lines of force corresponding to the edge detection result image according to the Hough transform algorithm, and determine a group of potential lines of force that are parallel to each other according to the slopes of the potential lines of force;

The slopes of the potential power line groups are clustered according to the K-means++ clustering algorithm, a target class with the largest number of slopes among the multiple classes is determined, the cluster center slope of the target class is obtained as the angle θ ₁ , and the tower offset angle θ ₂ is calculated according to the angle θ ₁ and the tower offset angle calculation formula, wherein the tower offset angle calculation formula is:

Optionally, the effective area determination module 503 is further configured to:

The tower positioning frame is rotated according to the tower direction and the tower offset angle _θ2 , and the rotated tower positioning frame is enlarged according to a preset magnification factor with the center point of the rotated tower positioning frame as the center to obtain a tower offset frame;

A first extension point and a second extension point are determined according to the diagonal corners of the tower offset frame, and an effective power line extraction area is determined according to the first extension point and the second extension point.

Optionally, the power line extraction module 504 is further configured to:

Perform straight line detection on the effective power line extraction area according to the LSD straight line detection algorithm to obtain candidate power line segments to be fitted, and put the candidate power line segments to be fitted into a line segment pool;

According to the distances between the candidate power line segments to be fitted in the line segment pool, the candidate power line segments to be fitted whose distances are less than a preset distance threshold are put into a candidate power line segment group to be fitted;

All the candidate power line segments to be fitted in the candidate power line segment group to be fitted are fitted according to the least square method to obtain a fitting straight line, and a fitting straight line perpendicular to the tower offset angle among the fitting straight lines is determined as a candidate power line.

Optionally, the tower coordinate prediction module 501 is further used for:

The insulator coordinates are predicted for the transmission line standardization image according to the coordinate prediction model to obtain the insulator prediction coordinates and the insulator prediction center point.

Optionally, the power line extraction module 504 is further configured to:

According to the preset power line template, after mapping the standard image of the power tower to the standardized image of the transmission line, the candidate power lines are voted according to the distance between the predicted center point of the insulator and the candidate power lines, and the power lines of the transmission line to be detected are determined according to the voting results, wherein the preset power line template is obtained by calculating the homography matrix of mapping the standardized image of the transmission line to the standard image of the power tower.

It should be noted that, for other corresponding descriptions of the functional units involved in the power line extraction device provided in the embodiment of the present application, reference may be made to the corresponding descriptions in the methods of FIG. 1 to FIG. 3 , which will not be repeated here.

Based on the above method as shown in Figures 1 to 3, accordingly, an embodiment of the present application also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the power line extraction method as shown in Figures 1 to 2 is implemented.

Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

Based on the above-mentioned method as shown in Figures 1 to 3, and the virtual device embodiments shown in Figures 4 and 5, in order to achieve the above-mentioned purpose, the embodiment of the present application also provides a computer device, which can be specifically a personal computer, a server, a network device, etc. The computer device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to implement the above-mentioned power line extraction method as shown in Figures 1 to 3.

Optionally, the computer device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.

Those skilled in the art will appreciate that the computer device structure provided in this embodiment does not constitute a limitation on the computer device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

The storage medium may also include an operating system and a network communication module. The operating system is a program that manages and saves the hardware and software resources of the computer device, and supports the operation of information processing programs and other software and/or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the physical device.

Through the description of the above implementation methods, technical personnel in this field can clearly understand that the present application can be implemented by means of software plus necessary general hardware platforms, and can also be implemented by hardware, and the tower coordinates are predicted for the transmission line standardized image of the transmission line to be detected according to the coordinate prediction model to obtain the tower predicted coordinates; edge detection is performed on the transmission line standardized image and the tower offset angle is calculated; the tower positioning frame is determined according to the tower predicted coordinates, and the effective power line extraction area is determined according to the tower positioning frame and the tower offset angle; straight line detection is performed on the effective power line extraction area, and the straight line perpendicular to the tower offset angle among the detected straight lines is determined as the candidate power line, and the candidate power lines are voted according to the preset power line template, and the power lines of the transmission line to be detected are determined according to the voting results. Based on the features of aerial images, the predicted coordinates of the tower are obtained according to a supervised algorithm (coordinate prediction model). Then, edge detection is performed on the standardized image of the transmission line to detect accurate and clear edges. The offset angle of the tower is calculated based on the edge detection result image. Combined with the prior knowledge that the power line must be within the range of the power tower and the power line is perpendicular to the power tower, a straight line detection algorithm is used to detect straight lines in the effective power line extraction area (unsupervised algorithm). Finally, a straight line that is approximately perpendicular to the tower offset angle is selected as a candidate power line, and the power line is determined according to the preset power line template, thereby improving the accuracy of power line detection.

Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.

Claims (11)

1. A power line extraction method, characterized in that the method includes: Obtain the standardized image of the transmission line corresponding to the transmission line to be detected, predict the electric tower coordinates of the standardized image of the transmission line according to the coordinate prediction model, and obtain the predicted coordinates of the electric tower; Wherein, the training method of the coordinate prediction model includes: Obtain multiple basic images of the data set including electric towers, standardize the basic images of the data set, and obtain a standardized image of the data set, and use a preset data annotation tool to identify the electric towers and electric towers in the standardized images of the data set. The included insulators are labeled and a power tower positioning data set is generated; The electric tower positioning data set is divided into a preset first proportion of electric tower positioning training data set and a preset second proportion of electric tower positioning data set by uniform random sampling. Test data set and preset third-ratio power tower positioning verification data set; Input the electric tower positioning training data set, the electric tower positioning test data set and the electric tower positioning verification data set into a coordinate prediction model for training, and obtain a trained coordinate prediction model; According to the predicted coordinates of the electric tower, determine the electric tower positioning frame in the standardized image of the transmission line; Perform edge detection on the standardized image of the transmission line to obtain an edge detection result image, and calculate the offset angle of the electric tower based on the edge detection result image; Determine the effective power line extraction area according to the electric tower positioning frame and the electric tower offset angle; Perform straight line detection on the effective power line extraction area, and determine the straight line perpendicular to the offset angle of the electric tower among the detected straight lines as the candidate power line; In the electric tower standard template library corresponding to the transmission line to be detected, search for the electric tower standard image according to the predicted coordinates of the electric tower, where the electric tower standard image and the transmission line standardized image are images of the same electric tower from different perspectives; The insulators in the standard image of the electric tower are mapped to the standardized image of the transmission line through homography transformation, and the candidate power line is voted on based on the distance between the mapped insulator and the candidate power line, and the location of the power line of the transmission line to be detected is determined based on the voting results. Straight line, wherein the homography transformation is implemented using a homography matrix. The homography matrix is calculated based on six pairs of insulator matching points. The six pairs of insulator matching points are based on the standards of electrical towers corresponding to the same electrical tower with different viewing angles. Select from images and standardized images of transmission lines. 2. The method according to claim 1, characterized in that, performing edge detection on the standardized image of the transmission line to obtain an edge detection result image includes: The pixels in the standardized image of the transmission line are grayscale adjusted to obtain a grayscale image of the transmission line. The grayscale image of the transmission line is processed by histogram equalization to obtain an equalized image of the transmission line. All the pixels in the standardized image of the transmission line are obtained. The above-mentioned equalized image of the transmission line is denoised to obtain a pre-processed image of the transmission line; On the basis of the horizontal gradient template and vertical gradient template included in the Canny edge detection algorithm, after adding the first diagonal gradient template and the second diagonal gradient template, the transmission line preprocessing is calculated based on the four direction gradient templates The gradient value of the image; According to the iterative algorithm, the highest threshold and the lowest threshold are generated from the grayscale values of each pixel of the transmission line preprocessed image, and the edge pixels corresponding to the gradient values between the highest threshold and the lowest threshold are retained to obtain the edge. Detection result image. 3. The method according to claim 2, characterized in that, performing grayscale adjustment on pixels in the standardized image of the transmission line to obtain a grayscale image of the transmission line includes: The grayscale value of each pixel in the standardized image of the transmission line is adjusted according to the pixel grayscale adjustment formula to obtain a grayscale image of the transmission line, wherein the pixel grayscale adjustment formula is: Gray(i,j)＝0.29*R+0.578*G+0.114*B Gray(i,j) represents the gray value of row i and column j in the pixel matrix corresponding to the standardized image of the transmission line. R, G and B represent the red channel matrix, green channel matrix and blue channel matrix respectively. 4. The method according to claim 2, characterized in that the histogram equalization processes the grayscale image of the transmission line to obtain an equalized image of the transmission line, including: The grayscale distribution in the grayscale image of the transmission line is equalized according to the histogram equalization formula to obtain the transmission line equalization image, where the histogram equalization formula is: n represents the total number of pixels in the grayscale image of the transmission line, k represents the k-level grayscale, n _k represents the number of pixels with grayscale level r _k , L represents the total number of grayscale levels in the grayscale image of the transmission line, p _r (r) represents the gray level probability density of the grayscale image of the transmission line. 5. The method according to claim 1, characterized in that the calculation of the electric tower offset angle based on the edge detection result image includes: Calculate potential power lines and potential power line slopes corresponding to the edge detection result image according to the Hough transform algorithm, and determine mutually parallel potential power line groups based on the potential power line slopes; Cluster the slopes of the potential power line group according to the K-means++ clustering algorithm, determine the target class containing the largest number of slopes among the multiple classes, and obtain the cluster center slope of the target class as the angle θ ₁ , according to the The angle θ ₁ and the calculation formula for the offset angle of the electrical tower are used to calculate the offset angle θ ₂ of the electrical tower, where the calculation formula for the offset angle of the electrical tower is: 6. The method according to claim 5, characterized in that the electric tower positioning frame is a rectangle; and determining the effective power line extraction area according to the electric tower positioning frame and the electric tower offset angle includes: The electric tower positioning frame is rotated according to the electric tower direction and the electric tower offset angle θ ₂ , and with the center point of the rotated electric tower positioning frame as the center, the rotated electric tower positioning frame is adjusted according to the preset amplification factor. Zoom in on the tower positioning frame to obtain the tower offset frame; The first extension point and the second extension point are determined according to the diagonal angles of the electric tower offset frame, and the effective power line extraction area is determined according to the first extension point and the second extension point. 7. The method according to claim 6, characterized in that the effective power line extraction area is subjected to straight line detection, and a straight line perpendicular to the offset angle of the electric tower among the detected straight lines is determined as a candidate power line. ,include: Perform straight line detection on the effective power line extraction area according to the LSD straight line detection algorithm to obtain candidate power line segments to be fitted, and put the candidate power line segments to be fitted into a line segment pool; According to the distance between the candidate power line segments to be fitted in the line segment pool, the candidate power line segments to be fitted that are smaller than the preset distance threshold are placed into the candidate power line segment to be fitted group; Fit all the candidate power line segments to be fitted in the candidate power line segment group to be fitted according to the least square method to obtain a fitting straight line, and the fitting straight line perpendicular to the offset angle of the electric tower in the fitting straight line is , determined as a candidate power line. 8. The method according to any one of claims 1 to 7, characterized in that, The coordinate prediction model is an improved model of the YOLOv7 model. The coordinate prediction model is obtained by replacing the SiLU activation function in the YOLOv7 model with the Mish activation function, and replacing the CIou loss function in the YOLO v7 model with the SIoU loss function. 9. A power line extraction device, characterized in that the device includes: The electric tower coordinate prediction module is used to obtain the standardized image of the transmission line corresponding to the transmission line to be detected, predict the electric tower coordinates of the standardized image of the transmission line according to the coordinate prediction model, and obtain the electric tower predicted coordinates, wherein the coordinate prediction The training method of the model includes obtaining multiple basic images of a data set including electric towers, standardizing the basic images of the data set, obtaining a standardized image of the data set, and using a preset data annotation tool to perform standardization of the data set images. The electric towers and the insulators contained in the electric towers are marked to generate an electric tower positioning data set. The electric tower positioning data set is divided into preset positions without intersection through uniform random sampling. A first-proportion electric tower positioning training data set, a preset second-proportion electric tower positioning test data set, and a preset third-proportion electric tower positioning verification data set. Combine the electric tower positioning training data set and the electric tower positioning data set. The positioning test data set and the electric tower positioning verification data set are input into the coordinate prediction model for training, and the trained coordinate prediction model is obtained; An effective area determination module, configured to determine the electric tower positioning frame in the standardized image of the transmission line according to the predicted coordinates of the electric tower; An electric tower angle calculation module is used to perform edge detection on the standardized image of the transmission line, obtain an edge detection result image, and calculate the electric tower offset angle based on the edge detection result image; The effective area determination module is also used to determine the effective power line extraction area according to the electric tower positioning frame and the electric tower offset angle; A power line extraction module, configured to detect straight lines in the effective power line extraction area, and determine the straight line perpendicular to the offset angle of the electric tower among the detected straight lines as a candidate power line; The power line extraction module is also used to search for a standard image of an electric tower according to the predicted coordinates of the electric tower in the electric tower standard template library corresponding to the transmission line to be detected, wherein the standard image of the electric tower interacts with the standardized image of the transmission line. Images from different perspectives of the same power tower; The power line extraction module is also used to map the insulators in the standard image of the electric tower to the standardized image of the transmission line through homography transformation, and vote on the candidate power line based on the distance between the mapped insulator and the candidate power line. As a result, the straight line of the power line of the transmission line to be detected is determined. The homography transformation is implemented using a homography matrix. The homography matrix is calculated based on six pairs of insulator matching points. The six pairs of insulator matching points are based on the same electric tower. Choose from standard images of electric towers and standardized images of transmission lines with different viewing angles. 10. A storage medium with a computer program stored thereon, characterized in that when the computer program is executed by a processor, the method of power line extraction according to any one of claims 1 to 8 is implemented. 11. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that when the processor executes the computer program, claims 1 to 8 are implemented The method of power line extraction according to any one of the above.