CN117994222B - Straight line segment detection method and system based on prediction correction mechanism - Google Patents

Abstract

The invention provides a straight line segment detection method and a system based on a prediction correction mechanism, and relates to the technical field of line segment detection; the method comprises the steps of predicting the edge of a line segment in an image based on a multi-scale self-adaptive Canny edge detector designed in advance, finally executing non-maximum suppression on the edge, performing line segment fitting by using a least square method to obtain an equation expression of the line where the line segment is located and the end point coordinates of the line segment, obtaining a predicted line segment, correcting the predicted line segment by adopting a directional routing method, verifying the corrected line segment, and outputting a final line segment. The invention can improve the integrity of the line segment, improve the line segment detection efficiency, improve the accuracy of the direction and the position of the final line segment, and effectively detect the line segment in the low-contrast area of the image.

Description

A straight line segment detection method and system based on prediction and correction mechanism

Technical Field

The present invention relates to the technical field of line segment detection, and in particular to a straight line segment detection method and system based on a prediction correction mechanism.

Background Art

Straight line segments (line segments for short) convey a wealth of geometric and topological information in real-world scenes, making them widely used in various computer vision tasks such as 3D reconstruction, simultaneous localization and mapping, pose estimation, vanishing point detection, and power line extraction in drone imagery.

The existing edge point-based straight line segment detection technology mainly includes the following steps: (1) Gaussian smoothing of the image to suppress noise; (2) calculation of gradient magnitude and gradient direction; (3) calculation of anchor points, i.e., local maximum values of gradient magnitude; (4) edge drawing, using routing methods to link edge points for line segment fitting; (5) verification of line segments. The general process of this type of method is shown in Figure 1.

The above steps (1) and (2) are easy to understand and consistent with most image processing methods. Step (3) is to first apply a threshold to the gradient amplitude of the image to filter some pixels, and then perform non-maximum suppression, that is, to determine whether the gradient amplitude of each pixel is the maximum along the gradient direction of each pixel. If so, the pixel is an anchor point. Step (4) is to use the gradient direction of the anchor point to find other anchor points that meet certain conditions in its neighborhood, and then proceed in sequence to form a pixel chain with these anchor points. If the pixel chain meets a certain scale, the least squares method is used to fit the pixel chain to obtain the parametric equation of the straight line where the line segment is located and the coordinates of the endpoints of the line segment. Step (5) is to use the gradient information of the image for further verification to control false positives.

In summary, the existing line segment detection methods have the following disadvantages:

Disadvantage 1: In the step of calculating the image gradient, the existing method applies a preset threshold to the image gradient amplitude to suppress false positives, but this threshold also filters out some pixels used to generate line segments. Therefore, the integrity of the line segments detected by the existing method is not high.

Disadvantage 2: The gradient amplitude applied by the existing methods in the step of calculating the image gradient is fixed, and the gradient amplitude of pixels in the low-contrast area of the image is usually lower than this threshold. Therefore, the existing methods often fail to detect line segments in low-contrast areas.

Disadvantage 3: In the anchor point selection step, the existing methods usually directly perform non-maximum suppression on the gradient amplitude, but this usually counts noise points as anchor points. Therefore, the detection efficiency of the existing methods is not high.

Disadvantage 4: Based on the disadvantages described in the steps of calculating image gradients and selecting anchor points, the existing methods remove pixels that are useful for line segments and introduce noise effects. Therefore, the existing methods cannot fully extract all the details of the line segments in the image.

Disadvantage 5: In the edge drawing step, the existing methods usually select an anchor point as the starting point, and then search for the next anchor point that meets certain conditions in the neighborhood according to the gradient direction of the anchor point. This search method is heavily dependent on the calculated gradient direction. Therefore, the pixel chain extracted by the existing methods for fitting the line segment through this linking method is usually of low quality, resulting in low accuracy in the direction and position of the detected line segment.

Summary of the invention

To this end, an embodiment of the present invention provides a straight line segment detection method and system based on a prediction and correction mechanism, which is used to solve the problems in the prior art that the integrity of the line segments detected by the existing methods is not high, the clues of low contrast areas cannot be detected, the detection efficiency is low, all the details about the line segments in the image cannot be extracted, and the direction and position of the detected line segments are not accurate.

In order to solve the above problem, an embodiment of the present invention provides a straight line segment detection method based on a prediction correction mechanism, the method comprising:

Step S1: Calculate the gradient magnitude and gradient direction of the input image;

Step S2: predicting line segments in the image based on a pre-designed multi-scale adaptive Canny edge detector, specifically comprising: firstly, performing edge detection on the input image based on the multi-scale adaptive Canny edge detector by setting three scale parameters; then fusing the edges from the three different scale information into the final edge; finally, performing non-maximum suppression on the edge and using the least squares method to perform line segment fitting, obtaining the equation expression of the straight line where the line segment is located and the coordinates of the endpoints of the line segment, that is, obtaining the predicted line segment;

Step S3: Correcting the predicted line segment using a directional routing method;

Step S4: Verify the corrected line segment and output the final line segment.

Preferably, in step S1, calculating the gradient magnitude and gradient direction of the input image specifically includes:

First, the input color image is converted into a grayscale image I(x,y); secondly, the grayscale image I(x,y) is Gaussian smoothed to reduce the influence of noise; then, the Sobel operator is used to calculate the gradient of the grayscale image after Gaussian smoothing. Where _gx and _gy are the derivatives with respect to x and y respectively, and the calculation formula for the gradient direction _Gθ is:

_Gθ =arctan( _gy / _gx );

The calculation formula of the gradient amplitude _Gm is:

G _m = |g _x |+|g _y |

In the formula, |·| represents the _L1 norm.

Preferably, the detection process of the adaptive Canny edge detector at a single scale includes the following steps:

Step 1: Set the scale parameter α = α ₁ , α ₂ , α ₃ , and divide the input image into α ² sub-images based on α. The size of each sub-image is Where h and w represent the length and width of the original image respectively;

Step 2: Based on the Helmholtz principle, the minimum gradient amplitude that is meaningful for line segment detection corresponding to each sub-image I _i is calculated. The calculation formula is as follows:

e _i ,m _i =CannyL(I _i )

Where, CannyL(·) represents the edge detection method of CannyLines; e _i represents the adaptive edge detection result of the sub-image I _i ; _mi represents the minimum gradient amplitude meaningful for detecting line segments in sub-image I _i ;

Step 3: Concatenate the edge results e _i of each sub-image I _i and calculate the mean of each sub-image _mi . Then, the edge detection result E _α of image I at scale α is obtained:

And the minimum gradient magnitude M _α that is meaningful for detecting line segments:

In the formula, Concat(·) represents the concatenation function; Avg(·) represents the mean function;

Step 4: Output the edge detection result E _α under scale α and the minimum gradient amplitude M _α that is meaningful for detecting line segments.

Preferably, the multi-scale adaptive Canny edge detection result is expressed as:

Where E(x,y) represents the multi-scale adaptive Canny edge detection result; represents the edge detection result of image I at scale α=α _i ; Avg(·) represents the mean function; M represents the minimum gradient amplitude in image I that is meaningful for detecting line segments; It represents the minimum gradient magnitude meaningful for detecting line segments in image I at scale α= _αi .

Preferably, in step S3, a directional routing method is used to correct the predicted line segment, specifically including:

Find pixel extension segments with the following properties near the endpoints of the predicted line segment: (a) having a local maximum gradient amplitude and G _m ≥ M, where G _m represents the gradient amplitude and M represents the minimum gradient amplitude meaningful for line segment detection; (b) being close enough to the line where the line segment is located, that is, the distance from the point to the line is less than a certain threshold; (c) being consistent with the general direction of the line segment, that is, the binary gradient direction of the pixel is consistent with the line segment type to which the line segment belongs, and the line segment types include horizontal line segments and vertical line segments;

Use the depth-first strategy to extend the line segment as much as possible, that is, add as many pixels as possible along the current direction; when encountering different directions, create a sibling branch in the search tree, that is, use the currently searched pixel as the starting point and the different directions encountered as the initial direction to extend the new line segment.

Preferably, each time a line segment is extended, the line segment needs to be fitted again.

Preferably, in step S4, the corrected line segment is verified and the final line segment is output, which specifically includes:

A post-verification strategy is adopted for the corrected line segment: for each pixel (x, y) used to fit the line segment, it is first projected onto the line segment, and then the gradient direction of the projected point is calculated using bilinear interpolation and compared with the normal direction of the line segment to obtain the angle error θ _E ;

Based on the angle error θ _E , the confidence S of the corrected line segment is calculated, and unreliable line segments are removed according to the size of the confidence S, that is, only line segments with S>0.5 are output as the final line segments.

Preferably, the confidence S of the corrected line segment is calculated as follows:

Where L represents the pixel set used to fit the line segment; I _{·} represents the indicator function; τ _θ represents the threshold value, which is set to 0.15 based on experience.

The embodiment of the present invention further provides a straight line segment detection system based on a prediction and correction mechanism, which is used to implement the above-mentioned straight line segment detection method based on the prediction and correction mechanism, and specifically includes:

A calculation module, used for calculating the gradient magnitude and gradient direction of the input image;

The prediction module is used to predict the line segments in the image based on the pre-designed multi-scale adaptive Canny edge detector, specifically including: firstly, based on the multi-scale adaptive Canny edge detector, edge detection is performed on the input image by setting three scale parameters; then, the edges from the three different scale information are fused into the final edge; finally, non-maximum suppression is performed on the edge and the least squares method is used to perform line segment fitting to obtain the equation expression of the straight line where the line segment is located and the endpoint coordinates of the line segment, that is, the predicted line segment is obtained;

A correction module, used for correcting the predicted line segments using a directional routing method;

The verification module is used to verify the corrected line segments and output the final line segments.

An embodiment of the present invention further provides a computer storage medium storing a computer software product. The computer software product includes several instructions for enabling a computer device to execute the above-mentioned straight line segment detection method based on the prediction and correction mechanism.

It can be seen from the above technical solutions that the present invention has the following beneficial effects:

(1) To address the problem of low integrity of line segments detected by existing methods, the present invention does not directly apply a threshold in the step of calculating image gradients to control false positives, but instead designs a multi-scale adaptive Canny edge detector that can extract all edge features related to line segments in the image and take into account all pixels that are beneficial to detecting line segments.

(2) To address the problem that existing methods cannot detect clues in low-contrast areas, the present invention designs a prediction and correction method. First, based on the edge features and adaptive gradient threshold provided by the multi-scale adaptive Canny edge detector, the line segments contained in any area of the image are effectively predicted. Then, the predicted line segments are continuously corrected using a directed routing method. This two-step approach can effectively detect line segments in low-contrast areas.

(3) In order to address the problem of low detection efficiency of existing methods, the present invention directly performs non-maximum suppression on the edges extracted by the multi-scale adaptive Canny edge detector, which greatly reduces the amount of calculation and further excludes noise factors. It can extract high-quality anchor points for fitting predicted line segments, which not only improves efficiency but also provides a solid foundation for subsequent detection steps.

(4) In view of the problem that existing methods cannot extract all details of line segments in an image, the multi-scale adaptive Canny edge detector designed in the present invention captures and integrates the details of line segments at multiple different image scales, while considering the overall and local line segment details of the image. Therefore, it is possible to capture all details of line segments in an image.

(5) In view of the problem that the direction and position of the existing detected line segments are not accurate, the present invention designs a directional routing method, which can use the direction and position of the line segment to find the next anchor point that meets the conditions, reducing the dependence on the gradient direction. In addition, the method will add pixels that are beneficial to the detected line segment to the corresponding linked list for line segment extension. Each extension will refit the line segment to correct the direction and position of the line segment, continuously improving the accuracy of its direction and position.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the implementation cases of the present invention or the technical solutions in the prior art, the following is a brief description of the drawings required for use in the embodiments. By referring to the drawings, the features and advantages of the present invention will be more clearly understood. The drawings are schematic and should not be understood as limiting the present invention in any way. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

FIG1 is a flowchart of an existing edge point based straight line segment detection method in the background art;

FIG2 is a flow chart of a straight line segment detection method based on a prediction correction mechanism provided in an embodiment;

FIG3 is a simplified flow chart of a straight line segment detection method based on a prediction and correction mechanism provided in an embodiment;

FIG4 is a detection flow chart of an adaptive Canny edge detector at a single scale in an embodiment;

FIG5 is a subjective comparison diagram of different edge detection methods in the embodiment;

FIG6 is a subjective comparison diagram of different methods in the embodiment;

FIG. 7 is a block diagram of a straight line segment detection system based on a prediction and correction mechanism provided in an embodiment.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

Embodiment 1

As shown in FIG. 2 and FIG. 3 , an embodiment of the present invention proposes a straight line segment detection method based on a prediction correction mechanism, the method comprising:

Step S1: Calculate the gradient magnitude and gradient direction of the input image;

Step S2: predicting line segments in the image based on a pre-designed multi-scale adaptive Canny edge detector, specifically comprising: firstly, performing edge detection on the input image based on the multi-scale adaptive Canny edge detector by setting three scale parameters; then fusing the edges from the three different scale information into the final edge; finally, performing non-maximum suppression on the edge and using the least squares method to perform line segment fitting, obtaining the equation expression of the straight line where the line segment is located and the coordinates of the endpoints of the line segment, that is, obtaining the predicted line segment;

Step S3: Correcting the predicted line segment using a directional routing method;

Step S4: Verify the corrected line segment and output the final line segment.

It can be seen from the above technical scheme that the present invention proposes a straight line segment detection method based on a prediction and correction mechanism, which first calculates the gradient amplitude and gradient direction of the input image; then, based on the pre-designed multi-scale adaptive Canny edge detector, the line segments in the image are predicted, and the designed multi-scale adaptive Canny edge detector can capture all the details of the relevant line segments in the image. At the same time, anchor point extraction is performed based on the edges extracted by the multi-scale adaptive Canny edge detector designed by the present invention, which can significantly improve the efficiency of line segment detection and further reduce noise interference; secondly, the present invention can improve the integrity of the line segment by using a directional routing method, and the proposed prediction and correction mechanism for line segment detection can continuously correct the direction and position of the line segment, thereby improving the accuracy of the direction and position of the final line segment; finally, the line segment correction verification strategy designed by the present invention can remove low-confidence line segments, thereby effectively controlling false positives.

In this embodiment, in step S1, the gradient magnitude and gradient direction of the input image are calculated, which specifically includes:

First, the input color image is converted into a grayscale image I(x,y); secondly, the grayscale image I(x,y) is Gaussian smoothed to reduce the influence of noise; then, the Sobel operator is used to calculate the gradient of the grayscale image after Gaussian smoothing. Where _gx and _gy are the derivatives with respect to x and y respectively, and the calculation formula for the gradient direction _Gθ is:

_Gθ =arctan( _gy / _gx );

The calculation formula of the gradient amplitude _Gm is:

G _m = |g _x |+|g _y |

Here, |·| represents the _L1 norm. The _L1 norm is used here to calculate the gradient magnitude because of its lower computational cost. To facilitate the subsequent directional routing step, the gradient direction _Gθ is binarized as follows: if | _gx |≥| _gy | of a pixel, the pixel is vertically oriented; otherwise, it is horizontally oriented. In simple terms, it is to determine in advance whether the pixel belongs to a vertical line segment or a horizontal line segment.

In this embodiment, in step S2, line segments in the image are predicted based on a pre-designed multi-scale adaptive Canny edge detector, specifically including: first, based on the multi-scale adaptive Canny edge detector, edge detection is performed on the input image by setting three scale parameters; then, the edges from the three different scale information are fused into the final edge; finally, non-maximum suppression is performed on the edge and the least squares method is used to perform line segment fitting to obtain the equation expression of the straight line where the line segment is located and the endpoint coordinates of the line segment, that is, the predicted line segment is obtained.

Specifically, CannyLines proposed a dual threshold method for finding Canny edges based on the Helmholtz principle. This method is adaptive and can provide the minimum gradient amplitude that is meaningful for detecting line segments for different images. In addition, CannyLines optimizes the edges so that the detected edges can conform to the line segment characteristics, that is, the gradient directions of local edge pixels are almost consistent. However, CannyLines only considers a single scale of the image and loses some details. In order to obtain more robust and accurate edge information, edge detection is performed on the image at three different scales (using the first adaptive Canny edge detector, the second adaptive Canny edge detector, and the third adaptive Canny edge detector).

Further, as shown in FIG4 , the detection process of the adaptive Canny edge detector at a single scale includes the following steps:

Step 1: Set the scale parameter α = α ₁ , α ₂ , α ₃ , and divide the input image into α ² sub-images based on α. The size of each sub-image is Where h and w represent the length and width of the original image respectively;

Step 2: Based on the Helmholtz principle, the minimum gradient amplitude that is meaningful for line segment detection corresponding to each sub-image I _i is calculated. The calculation formula is as follows:

e _i ,m _i =CannyL(I _i )

Where, CannyL(·) represents the edge detection method of CannyLines; e _i represents the adaptive edge detection result of the sub-image I _i ; _mi represents the minimum gradient amplitude meaningful for detecting line segments in sub-image I _i ;

Step 3: Concatenate the edge results e _i of each sub-image I _i and calculate the mean of each sub-image _mi . Then, the edge detection result E _α of image I at scale α is obtained:

And the minimum gradient magnitude M _α that is meaningful for detecting line segments:

In the formula, Concat(·) represents the concatenation function; Avg(·) represents the mean function;

Step 4: Output the edge detection result E _α under scale α and the minimum gradient amplitude M _α that is meaningful for detecting line segments.

Furthermore, the multi-scale adaptive Canny edge detection result is expressed as:

Where E(x,y) represents the multi-scale adaptive Canny edge detection result; represents the edge detection result of image I at scale α=α _i ; Avg(·) represents the mean function; M represents the minimum gradient amplitude in image I that is meaningful for detecting line segments; It represents the minimum gradient magnitude meaningful for detecting line segments in image I at scale α= _αi .

Based on these edges, it is easy to predict line segments: define an edge segment as e, and for each pixel (x, y)∈e, if its gradient magnitude is a local maximum, it is an anchor point. The set of anchor points belonging to the edge is a line segment prediction, and then these anchor point sets are fitted based on the least squares method, which is the line segment prediction (including the equation expression of the line where the line segment is located and the coordinates of the endpoints of the line segment).

The present invention uses the adaptive Canny edge detection method to perform edge detection on the global and local (i.e., multi-scale) features of the image, and fuses this information to obtain more robust and accurate edge details that conform to line segment features. Experiments have shown that the edge detector can effectively and comprehensively extract and integrate image edges that conform to line segment features. Based on these edges, all meaningful and high-quality line segment predictions in the image can be given, thereby improving the overall line segment detection performance.

In this embodiment, in step S3, a directional routing method is used to correct the predicted line segment.

Specifically, since the predicted line segment only gives the approximate position and method of the line segment in the image, there may be a large error and the integrity of the line segment is not high. Therefore, we designed a directional routing method for correcting the line segment. Follow the steps below to implement it:

Find pixel extension segments with the following properties near the endpoints of the predicted line segment: (a) having a local maximum gradient amplitude and G _m ≥ M, where G _m represents the gradient amplitude and M represents the minimum gradient amplitude meaningful for line segment detection; (b) being close enough to the line where the line segment is located, that is, the distance from the point to the line is less than a certain threshold; (c) being consistent with the general direction of the line segment, that is, the binary gradient direction of the pixel is consistent with the line segment type to which the line segment belongs, and the line segment types include horizontal line segments and vertical line segments;

Use the depth-first strategy to extend the line segment as much as possible, that is, add as many pixels as possible along the current direction; when encountering different directions, create a sibling branch in the search tree, that is, use the currently searched pixel as the starting point and the different directions encountered as the initial direction to extend the new line segment.

It should be noted that each time a line segment is extended, it needs to be refitted to improve the accuracy of the direction and position of the line segment. In addition, new line segments may be found during the extension process, which will be recorded by the directional routing method for subsequent line segment detection.

This paper applies the prediction correction method to line segment detection for the first time, using a multi-scale adaptive Canny edge detector to capture all edge details that meet the line segment characteristics in the image for line segment prediction, and then uses a directional routing method to correct these predicted line segments to improve their integrity and the accuracy of their position and direction. Experiments have shown that this method can effectively detect line segments in low-contrast areas, which is impossible with existing methods. This will provide better underlying technical support for further image tasks, such as 3D reconstruction.

In this embodiment, in step S4, the corrected line segment is verified and the final line segment is output.

Specifically, the line segments corrected by the directional routing method may have many false positives, which mainly occur in areas with high edge pixel density, such as leaves, grass, etc. In order to ensure the confidence of the final output line segments, a post-verification strategy is adopted for the corrected line segments: for each pixel (x, y) used to fit the line segment, it is first projected onto the line segment, and then the gradient direction of the projected point is calculated using bilinear interpolation, and compared with the normal direction of the line segment to obtain the angle error θ _E .

Furthermore, based on the angle error θ _E , the confidence S of the corrected line segment is calculated, and unreliable line segments are removed according to the size of the confidence S, that is, only line segments with S>0.5 are output as the final line segments. Experiments have shown that this strategy can remove low-confidence line segments, thereby effectively controlling false positives.

The confidence S of the corrected line segment is calculated as follows:

Where L represents the pixel set used to fit the line segment; I _{·} represents the indicator function; τ _θ represents the threshold value, which is set to 0.15 based on experience.

Table 1 shows the results of the objective evaluation indicators of the present invention and the existing line segment detection methods. The comparison methods include the latest classical methods and deep learning methods. The objective evaluation indicators are: the arithmetic mean of precision and recall (APR), the harmonic mean of precision and recall (F-score) and the intersection over union (IoU). _{Con 1} and Con ₂ represent loose and strict evaluation environments, respectively. It can be seen that the present invention achieves the best results in most cases.

Table 1

FIG5 is a comparison of the edge detection results of the multi-scale adaptive Canny edge detector designed by the present invention, the CannyLines edge detection method, and the latest existing deep edge detector. It can be seen that the detection effect of the present design is better and the edge details that conform to the line segment features can be fully captured.

Figure 6 shows the subjective evaluation of the present invention and the existing line segment detection method. It can be seen that the present invention can effectively detect line segments in low-contrast areas, and the detected line segments are more complete, and the angle and position errors between the actual line segments are smaller. From the objective and subjective comparison, it can be concluded that our method can effectively solve the shortcomings of the existing methods, and thus achieve the best detection performance.

Embodiment 2

As shown in FIG. 7 , the present invention provides a straight line segment detection system based on a prediction correction mechanism, which is used to implement the straight line segment detection method based on a prediction correction mechanism of the first embodiment, and specifically includes:

A calculation module 10, used to calculate the gradient magnitude and gradient direction of the input image;

The prediction module 20 is used to predict the line segments in the image based on the pre-designed multi-scale adaptive Canny edge detector, specifically comprising: firstly, based on the multi-scale adaptive Canny edge detector, edge detection is performed on the input image by setting three scale parameters; then, the edges from the three different scale information are merged into the final edge; finally, non-maximum suppression is performed on the edge and the line segment is fitted using the least squares method to obtain the equation expression of the straight line where the line segment is located and the endpoint coordinates of the line segment, that is, the predicted line segment is obtained;

A correction module 30, for correcting the predicted line segment using a directional routing method;

The verification module 40 is used to verify the corrected line segment and output the final line segment.

A straight line segment detection system based on a prediction and correction mechanism in this embodiment is used to implement the aforementioned straight line segment detection method based on a prediction and correction mechanism. Therefore, the specific implementation method of the straight line segment detection system based on the prediction and correction mechanism can be seen in the embodiment part of the straight line segment detection method based on the prediction and correction mechanism in the previous text. For example, the calculation module 10, the prediction module 20, the correction module 30, and the verification module 40 are respectively used to implement steps S1, S2, S3, and S4 in the above-mentioned straight line segment detection method based on the prediction and correction mechanism. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part. In order to avoid redundancy, it will not be repeated here.

Embodiment 3

An embodiment of the present invention further provides a computer storage medium storing a computer software product. The computer software product includes several instructions for enabling a computer device to execute the above-mentioned straight line segment detection method based on the prediction and correction mechanism.

Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

The present application is described with reference to the flowchart and/or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and/or box in the flowchart and/or block diagram, and the combination of the process and/or box in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and/or one box or multiple boxes in the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes of the flowchart and/or one or more blocks of the block diagram. These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operating steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes of the flowchart and/or one or more blocks of the block diagram.

Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims (8)

1. The straight line segment detection method based on the prediction correction mechanism is characterized by comprising the following steps of:

Step S1, calculating the gradient amplitude and gradient direction of an input image;

The method comprises the steps of firstly carrying out edge detection on an input image based on a multi-scale self-adaptive Canny edge detector by setting three scale parameters, then fusing edges from three different scale information into a final edge, finally carrying out non-maximum suppression on the edge and carrying out line segment fitting by using a least square method to obtain an equation expression of a straight line where the line segment is located and end point coordinates of the line segment, and obtaining a predicted line segment, wherein the detection flow of the self-adaptive Canny edge detector under a single scale comprises the following steps:

Setting a scale parameter alpha=alpha ₁,α₂,α₃, dividing an input image into alpha ² sub-images based on alpha equal division, wherein the size of each sub-image is as follows Wherein h and w respectively represent the length and the width of the original image;

And secondly, calculating the minimum gradient amplitude which is corresponding to each subgraph I _i and has significance to the detection of the line segment based on the Helmholtz principle, wherein the calculation formula is as follows:

e_i,m_i＝CannyL(I_i)

Wherein CannyL (·) represents an edge detection method of CANNYLINES, e _i represents an adaptive edge detection result of the sub-graph I _i, and m _i represents a minimum gradient amplitude value in the sub-graph I _i, which is significant for detecting a line segment;

Step three, splicing the edge results E _i of each sub-graph I _i, and calculating the average value of m _i of each sub-graph, so as to obtain an edge detection result E _α of the image I under the scale alpha:

and a minimum gradient magnitude M _α that is significant to the detected line segment:

wherein Concat (DEG) represents a splicing function, and Avg (DEG) represents a mean function;

outputting an edge detection result E _α under the scale alpha and a minimum gradient amplitude M _α which is significant to a detection line segment;

And S3, correcting the predicted line segment by adopting a directional routing method, wherein the method comprises the following steps of:

Searching a pixel extension line segment near the end point of the predicted line segment, wherein the pixel extension line segment has the following properties that (a) the pixel extension line segment has local maximum gradient amplitude, G _m is more than or equal to M, G _m represents gradient amplitude, M represents minimum gradient amplitude which is significant for detecting the line segment, (b) the distance from a point to a straight line is smaller than a certain threshold value, (c) the binarization gradient direction of the pixel is consistent with the line segment type of the line segment, and the line segment type comprises a horizontal line segment and a vertical line segment;

when encountering different directions, creating a same-level branch in the search tree, namely taking the currently searched pixel as a starting point, and carrying out new line segment extension by taking the encountered different directions as initial directions;

And S4, verifying the corrected line segment and outputting a final line segment.

2. The straight line segment detection method based on the prediction correction mechanism according to claim 1, wherein in step S1, the gradient magnitude and the gradient direction of the input image are calculated, specifically comprising:

Firstly, converting the input color image into a gray image I (x, y), secondly, performing Gaussian smoothing on the gray image I (x, y) to reduce noise influence, and then, calculating the gradient of the Gaussian-smoothed gray image by using a Sobel operator Where G _x and G _y are derivatives with respect to x and y, respectively, the calculation formula for the gradient direction G _θ is:

G_θ＝arctan(g_y/g_x);

The calculation formula of the gradient amplitude G _m is:

G_m＝|g_x|+|g_y|

where |·| represents the L ₁ norm.

3. The straight line segment detection method based on a prediction correction mechanism according to claim 1, wherein the multi-scale adaptive Canny edge detection result is expressed as:

wherein E (x, y) represents a multi-scale adaptive Canny edge detection result; Representing the edge detection result of the image I at the scale α=α _i, avg (·) representing the mean function, M representing the minimum gradient amplitude in the image I that is significant for the detected line segment; representing the minimum gradient magnitude that is significant to the detected line segment in image I at the scale α=α _i.

4. The method for detecting straight line segments based on a prediction correction mechanism according to claim 1, wherein the line segments need to be fitted again each time the line segments are extended.

5. The method for detecting straight line segments based on a prediction correction mechanism according to claim 1, wherein in step S4, the corrected line segments are verified, and final line segments are output, and the method specifically comprises:

For each pixel (x, y) used for fitting the line segment, firstly projecting the line segment onto the line segment, then calculating the gradient direction of the projection point by bilinear interpolation, and comparing the gradient direction with the normal direction of the line segment to obtain an angle error theta _E;

Based on the angle error θ _E, the confidence S of the corrected line segment is calculated, and the unreliable line segment is removed according to the magnitude of the confidence S, that is, only the line segment with S >0.5 is output as the final line segment.

6. The method for detecting straight line segments based on a prediction correction mechanism according to claim 5, wherein the confidence S of the corrected line segment is calculated by the formula:

Where L represents the set of pixels used to fit the line segment, I _{·} represents the indicator function, τ _θ represents the threshold, empirically set to 0.15.

7. A straight line segment detection system based on a prediction correction mechanism, wherein the system is configured to implement the straight line segment detection method based on a prediction correction mechanism as set forth in any one of claims 1 to 6, and specifically includes:

The computing module is used for computing the gradient amplitude and the gradient direction of the input image;

The prediction module is used for predicting line segments in an image based on a multi-scale self-adaptive Canny edge detector designed in advance, and concretely comprises the steps of firstly carrying out edge detection on an input image based on the multi-scale self-adaptive Canny edge detector by setting three scale parameters, then fusing edges from three different scale information into a final edge, finally carrying out non-maximum suppression on the edge and carrying out line segment fitting by using a least square method to obtain an equation expression of a straight line where the line segments are located and end point coordinates of the line segments, and obtaining predicted line segments;

the correction module is used for correcting the predicted line segment by adopting a directional routing method;

And the verification module is used for verifying the corrected line segment and outputting a final line segment.

8. A computer storage medium storing a computer software product comprising instructions for causing a computer device to perform the prediction correction mechanism-based straight line segment detection method of any one of claims 1 to 6.