CN109034151A - A kind of localization method for the identification of multiple pointer instruments - Google Patents

Abstract

The invention discloses a kind of localization methods for the identification of multiple pointer instruments, it is slided in LARK characteristic pattern including the input picture using the LARK characteristic pattern of the template image after dimensionality reduction as sliding window at multiple scales, obtains the cosine similarity figure under different scale；Cosine similarity figure under different scale is transformed to multiple RM similarity graphs, constitute the similar diagram of the input picture under each scale, instrument detection is carried out under each scale, if the maximum RM similarity in similar diagram is greater than threshold value, then using the highest preceding M% of RM similarity, corresponding region excludes extra overlapping region and obtains final pointer instrument candidate region as preliminary pointer instrument candidate region in the input image；The characteristic point for extracting final pointer instrument candidate region image and template image is matched, and the accurate positioning in pointer gauge field domain in input picture is obtained.Adaptability of the present invention is good, strong real-time, and pointer instrument locating accuracy is high.

Description

A positioning method for identifying multiple pointer instruments

technical field

The invention belongs to the technical field of image processing and pattern recognition, and more specifically relates to a positioning method for identifying multiple pointer instruments.

Background technique

With the development of industrial automation, image processing and pattern recognition technology, for various types of pointer instruments widely used in electric power, petroleum, chemical and other industries, robot intelligent inspection is required to replace manual inspection and improve inspection efficiency. Reduce the risk and cost of manual inspections.

For the positioning of pointer instruments, there are generally two existing methods: (1) assume that the instrument is a circular instrument, and use the Hough algorithm to detect the circle to locate the position of the pointer instrument in the image to be recognized; (2) detect The feature points of the pointer instrument and the feature points of the template image are matched to locate the position of the pointer instrument.

The positioning method of the pointer meter (1) requires the pointer meter to be a circular meter, and there must be no other round objects in the surrounding environment of the pointer meter; the camera for shooting the meter is required to be fixed, which will be limited in practical applications, and The Hough transform circular detection method has high time complexity and poor real-time performance. When there are multiple pointer instruments in the image, the algorithm takes a long time; in addition, it is also required that the angle of view deflection of the pointer instruments cannot be too large, otherwise it cannot be detected arrive. Method (2) requires that there are no other external objects in the surrounding environment of the pointer meter, and the meter must occupy the main position in the input image, otherwise it cannot be detected; at the same time, the meter must also have stable feature points, otherwise it cannot be detected from the meter area. Feature points are extracted, resulting in positioning failure.

Therefore, the existing pointer instrument positioning method has the technical problems of poor adaptability, poor real-time performance, and low positioning accuracy of the pointer instrument.

Contents of the invention

In view of the above defects or improvement needs of the prior art, the present invention provides a positioning method for identifying multiple pointer instruments, thereby solving the problem of poor adaptability and poor real-time performance of the existing pointer instrument positioning methods , the technical problem that the positioning accuracy of the pointer instrument is not high.

In order to achieve the above purpose, the present invention provides a positioning method for identifying multiple pointer instruments, including:

(1) Obtain the LARK feature map of the input image and the LARK feature map of the template image, scale the LARK feature map of the input image after dimensionality reduction, and obtain the LARK feature map of the input image at multiple scales;

(2) The LARK feature map of the template image after dimensionality reduction is used as a sliding window, and the sliding window is used to slide the LARK feature map of the input image at multiple scales, and the sliding window and the input image in the sliding window are calculated after each sliding The cosine similarity of the LARK feature map of the input image at each scale is obtained after multiple slidings; the cosine similarity of the LARK feature map of the input image;

(3) Multiple cosine similarities are transformed into multiple RM similarities, and multiple RM similarities are used to form a similarity map of the input image under each scale, and instrumentation detection is performed on the similarity map of the input image under each scale, If the maximum RM similarity in the similarity graph is greater than the threshold, select the region corresponding to the top M% of the highest RM similarity in the similarity graph in the input image as the preliminary candidate region of the pointer instrument, and then use the non-maximum value suppression algorithm to exclude Get the final pointer instrument candidate area by removing the redundant overlapping areas in the preliminary pointer instrument candidate area;

(4) Extract the feature points of the final candidate area image of the pointer instrument and the template image, and perform matching to obtain the precise positioning of the pointer instrument area in the input image.

Further, step (1) includes:

(1-1) Obtain the LARK feature of the input image to form the LARK feature map of the input image, obtain the LARK feature of the template image to form the LARK feature map of the template image, and use the PCA algorithm to reduce the LARK feature map of the input image and the LARK feature map of the template image dimension;

(1-2) Scale the LARK feature map of the input image after dimensionality reduction to obtain the LARK feature map of the input image at multiple scales.

Further, the dimensions of the LARK feature map of the input image after dimension reduction are consistent with the dimensions of the LARK feature map of the template image after dimension reduction.

Further, the dimension is reduced by 1-4 dimensions during dimensionality reduction.

Further, the value range of M is 0.1-2.

Further, the threshold is 0.4-0.6.

Further, step (2) also includes:

Use the Fourier transform to accelerate the convolution operation in computing the cosine similarity between a sliding window and the LARK feature map of the input image within the sliding window.

Further, step (4) includes:

(4-1) Use the AKAZE algorithm to extract the feature points of the final pointer instrument candidate area image and the template image, perform matching, and obtain multiple matching points, and then use the method based on grid motion statistics to eliminate errors in multiple matching points Matching points to get the remaining matching points;

(4-2) Use the remaining matching points to complete image registration, and obtain the precise positioning of the pointer instrument area in the input image.

Further, step (4-2) includes:

The RANSAC method is used to process the remaining matching points to obtain the homography matrix; the image registration is completed through the homography matrix, and the precise positioning of the pointer instrument area in the input image is obtained.

Generally speaking, compared with the prior art, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

(1) The method of the present invention can realize the accurate positioning of the pointer instrument in the intelligent inspection system of industries such as electric power, chemical industry, petroleum, and can be used in the situation that there are a plurality of pointer instruments in the input image, the method adaptability is good, real-time With strong performance and high accuracy, it provides high-precision positioning for the subsequent identification of pointer instrument readings.

(2) The present invention aims at the positioning problem of the pointer instrument, uses the LARK feature that is stable under different illumination and noise conditions, uses the PCA algorithm, highlights the edge features of the instrument, and improves the accuracy of instrument positioning. At the same time, by performing multi-scale detection on the LARK feature maps of the input image at different scales, it can effectively prevent missed detection, and it is also applicable to the situation where there are multiple pointer instruments.

(3) The present invention adopts the sliding window method to detect the input image of the pointer instrument, which can be used when there are multiple pointer instruments in the image, and for the problem of slow matching of the sliding window, the Fourier transform is used to accelerate the convolution Operation, improve the instrument positioning speed.

(4) The present invention can effectively solve the positioning error problem caused by the deflection of the angle of view of the input image of the pointer instrument through image registration, adopt the grid motion statistical method to eliminate the wrong matching points, and use the RANSAC method to process the remaining matching points, which improves the The registration accuracy of the pointer instrument area positioning provides high-precision positioning for the subsequent pointer instrument reading recognition.

Description of drawings

Fig. 1 is a flow chart of a positioning method for identifying multiple pointer instruments provided by an embodiment of the present invention;

Fig. 2 is the input image of the pointer instrument to be identified and positioned provided by Embodiment 1 of the present invention;

Fig. 3 is a template image of the pointer meter provided by Embodiment 1 of the present invention;

Fig. 4 (a) is the LARK feature map after using the PCA algorithm to reduce the dimensionality of the input image of the pointer instrument to be recognized provided by Embodiment 1 of the present invention;

Fig. 4(b) is the LARK feature map of the template image provided by Embodiment 1 of the present invention after using the PCA algorithm for dimensionality reduction;

Figure 5(a) is a schematic diagram of the cosine similarity provided by Embodiment 1 of the present invention;

Figure 5(b) is a schematic diagram of the RM similarity provided by Embodiment 1 of the present invention;

Fig. 6 is a schematic diagram provided by Embodiment 1 of the present invention without the non-maximum value suppression algorithm;

FIG. 7 is a schematic diagram of the non-maximum value suppression algorithm provided by Embodiment 1 of the present invention;

Fig. 8 is a schematic diagram of error matching points not excluded before registration provided by Embodiment 1 of the present invention;

Fig. 9 is a schematic diagram of excluding wrong matching points after registration provided by Embodiment 1 of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other.

As shown in Figure 1, the present invention provides a positioning method for identifying multiple pointer instruments, including:

(1) Obtain the LARK (Locally Adaptive Regression Kernels) features of the input image to form the LARK feature map of the input image, obtain the LARK features of the template image to form the LARK feature map of the template image, and use PCA (Principal Component Analysis, the main Component analysis) algorithm reduces the dimensions of the LARK feature map of the input image and the LARK feature map of the template image; scales the LARK feature map of the input image after dimensionality reduction, and obtains the LARK feature map of the input image at multiple scales. The dimensions of the LARK feature map of the input image after dimension reduction are consistent with the dimensions of the LARK feature map of the template image after dimension reduction.

(2) The LARK feature map of the template image after dimensionality reduction is used as a sliding window, and the sliding window is used to slide the LARK feature map of the input image at multiple scales, and the sliding window and the input image in the sliding window are calculated after each sliding The cosine similarity of the LARK feature map, after sliding multiple times, multiple cosine similarities of the LARK feature map of the input image at each scale are obtained; use the Fourier transform to accelerate the calculation of the LARK of the sliding window and the input image within the sliding window Convolution operations in cosine similarity of feature maps.

(3) Multiple cosine similarities are transformed into multiple RM similarities, and multiple RM similarities are used to form a similarity map of the input image under each scale, and instrumentation detection is performed on the similarity map of the input image under each scale, If the maximum RM similarity in the similarity map is greater than the threshold, select the area corresponding to the top M% of the RM (Resemblance Map) similarity in the similarity map in the input image, as the preliminary pointer instrument candidate area; in the preliminary pointer Using the Non Maximum Suppression (NMS) algorithm to detect multiple pointer meters in the candidate area of the pointer meter, the redundant overlapping areas in the candidate region of the pointer meter are eliminated, and the maximum similarity value of each pointer meter is obtained. area, which is the final candidate area of the pointer instrument.

(4) Use the AKAZE algorithm to extract the feature points of the final pointer instrument candidate area image and the template image, and perform matching to obtain multiple matching points, and then use the method based on grid motion statistics to eliminate the wrong matching points among the multiple matching points , to obtain the remaining matching points; use the RANSAC (RandomSample Consensus, random sampling consistency) method to process the remaining matching points to obtain a homography matrix; complete image registration through the homography matrix, and realize the precise positioning of the pointer instrument area in the input image.

Further, the dimension is reduced by 1-4 dimensions during dimensionality reduction, preferably, the dimensionality is reduced by 2 dimensions during dimensionality reduction, at this time, the edge features of the instrument can be better highlighted.

Further, the value range of M is 0.1-2. Preferably, M is 1. If M is set too low, there may be missed detection, and if it is set too high, false detection may occur. When M is 1, the detection effect is the best.

Further, the threshold value is 0.4-0.6, preferably, the threshold value is 0.5, so that the subsequently obtained pointer instrument candidate areas are more accurate.

Example 1

For the input image of the pointer instrument to be identified and positioned as shown in Figure 2 and the template image of the pointer instrument shown in Figure 3, the precise positioning of multiple pointer instruments is achieved through the following steps:

Step (1): Extract the LARK feature of the input image and the template image, obtain the LARK feature map of the input image and the template image, and use the PCA algorithm to reduce the dimension of the LARK feature map, and obtain the input image shown in Figure 4(a) using PCA The LARK feature map after dimensionality reduction by the algorithm and the template image shown in Figure 4(b) use the LARK feature map after dimensionality reduction by the PCA algorithm; the edge features of the instrument are highlighted. Then the dimensionally reduced LARK feature map of the input image is scaled, the scaling range is 0.6-1.5, and the interval is 0.1, thus obtaining 10 dimensionally reduced LARK feature maps of the input image at different scales.

Step (2): Using the sliding window method, the LARK feature image after dimensionality reduction of the template image is used as a sliding window to slide in the 10 dimensionality-reduced LARK feature maps of different scales of the input image, and the sliding window is calculated after each sliding With the cosine similarity of the LARK feature map of the input image in the sliding window, multiple cosine similarities of the LARK feature map of the input image at each scale are obtained after multiple slidings, as shown in Figure 5(a); using Fu The Lie transform accelerates the convolution operation in calculating the cosine similarity between the sliding window and the LARK feature map of the input image within the sliding window.

Step (3): Transform the cosine similarity ρ _i , as shown in formula (1):

Afterwards, 10 RM similarity images of the input image at different scales can be obtained. Compared with the cosine similarity ρ _i , using f(ρ _i ) as the RM similarity can distinguish the background and the foreground more effectively, as shown in Figure 5(b ), it can be clearly seen that using the RM similarity, the extreme value is more obvious, so that the distinction between the foreground and the background is very different.

Each RM similarity map is judged. If there is at least one similarity map with a maximum similarity value greater than the set threshold of 0.5, it is confirmed that there is a pointer meter area in the input image. For the similarity graphs with the maximum similarity value greater than the set threshold, the regions in the original input image corresponding to the top 1% of the similarity graphs with the highest similarity value are selected as the preliminary candidate regions of pointer instruments. In this example 1, generally 20 to 30 preliminary pointer instrument candidate areas will be obtained.

Since there may be multiple pointer instrument areas in the input image, and after each pointer instrument area passes through the previous steps, there will be multiple overlapping candidate areas, and then use the Non Maximum Suppression (NMS) algorithm , to detect multiple pointer instruments, search for local maxima, suppress non-maximum elements, and then eliminate redundant overlapping areas in the pointer instrument candidate area. After that, the area with the largest similarity value of each pointer instrument can be obtained, that is, the final candidate area of the pointer instrument.

The dense multiple candidate boxes in Figure 6 are candidate boxes that have not passed the non-maximum value suppression algorithm, and the box area in Figure 7 is the area with the largest similarity value after the non-maximum value suppression algorithm, that is, the final pointer The candidate area of the type instrument is also the positioning area of the pointer type instrument before registration.

Step (4): AKAZE algorithm is used to extract the feature points of the final pointer instrument candidate area image and the template image, and perform matching to obtain multiple matching points. Generally, there will be wrong matching points in the matching result, as shown in FIG. 8 . Then use the method based on grid motion statistics to exclude wrong matching points, as shown in Figure 9.

Then, use the RANSAC method to process the remaining matching points to obtain the homography matrix. The image is transformed through the homography matrix, the image registration is completed, and the precise positioning of the pointer meter area is realized, thereby providing high-precision positioning for the subsequent pointer meter reading recognition.

It is easy for those skilled in the art to understand that the above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, All should be included within the protection scope of the present invention.

Claims (9)

1. A positioning method for multiple pointer instrument identification, characterized in that, comprising: (1) Obtain the LARK feature map of the input image and the LARK feature map of the template image, scale the LARK feature map of the input image after dimensionality reduction, and obtain the LARK feature map of the input image at multiple scales; (2) The LARK feature map of the template image after dimensionality reduction is used as a sliding window, and the sliding window is used to slide the LARK feature map of the input image at multiple scales, and the sliding window and the input image in the sliding window are calculated after each sliding The cosine similarity of the LARK feature map of the input image at each scale is obtained after multiple slidings; the cosine similarity of the LARK feature map of the input image; (3) Multiple cosine similarities are transformed into multiple RM similarities, and multiple RM similarities are used to form a similarity map of the input image under each scale, and instrumentation detection is performed on the similarity map of the input image under each scale, If the maximum RM similarity in the similarity graph is greater than the threshold, select the region corresponding to the top M% of the highest RM similarity in the similarity graph in the input image as the preliminary candidate region of the pointer instrument, and then use the non-maximum value suppression algorithm to exclude Get the final pointer instrument candidate area by removing the redundant overlapping areas in the preliminary pointer instrument candidate area; (4) Extract the feature points of the final candidate area image of the pointer instrument and the template image, and perform matching to obtain the precise positioning of the pointer instrument area in the input image. 2. A kind of positioning method for a plurality of pointer instrument identifications as claimed in claim 1, is characterized in that, described step (1) comprises: (1-1) Obtain the LARK feature of the input image to form the LARK feature map of the input image, obtain the LARK feature of the template image to form the LARK feature map of the template image, and use the PCA algorithm to reduce the LARK feature map of the input image and the LARK feature map of the template image dimension; (1-2) Scale the LARK feature map of the input image after dimensionality reduction to obtain the LARK feature map of the input image at multiple scales. 3. A kind of location method that is used for a plurality of pointer instrument identifications as claimed in claim 1 or 2, is characterized in that, the LARK feature map of the input image after the described dimension reduction and the LARK of the template image after the dimension reduction The dimensions of the feature maps are consistent. 4. A positioning method for identifying a plurality of pointer instruments according to claim 1 or 2, characterized in that, the dimension reduction time dimension is reduced by 1-4 dimensions. 5. A positioning method for identifying multiple pointer instruments as claimed in claim 1 or 2, characterized in that the value range of M is 0.1-2. 6. A positioning method for identifying multiple pointer instruments according to claim 1 or 2, wherein the threshold is 0.4-0.6. 7. A kind of positioning method for multiple pointer instrument identification as claimed in claim 1 or 2, is characterized in that, described step (2) also comprises: Use the Fourier transform to accelerate the convolution operation in computing the cosine similarity between a sliding window and the LARK feature map of the input image within the sliding window. 8. A kind of positioning method for multiple pointer instrument identification as claimed in claim 1 or 2, is characterized in that, described step (4) comprises: (4-1) Use the AKAZE algorithm to extract the feature points of the final pointer instrument candidate area image and the template image, perform matching, and obtain multiple matching points, and then use the method based on grid motion statistics to eliminate errors in multiple matching points Matching points to get the remaining matching points; (4-2) Use the remaining matching points to complete image registration, and obtain the precise positioning of the pointer instrument area in the input image. 9. A kind of positioning method for a plurality of pointer instrument identifications as claimed in claim 8, is characterized in that, described step (4-2) comprises: The RANSAC method is used to process the remaining matching points to obtain the homography matrix; the image registration is completed through the homography matrix, and the precise positioning of the pointer instrument area in the input image is obtained.