CN117746025A - Indication recognition method and system for pointer instrument of substation equipment - Google Patents
Indication recognition method and system for pointer instrument of substation equipment Download PDFInfo
- Publication number
- CN117746025A CN117746025A CN202311791940.2A CN202311791940A CN117746025A CN 117746025 A CN117746025 A CN 117746025A CN 202311791940 A CN202311791940 A CN 202311791940A CN 117746025 A CN117746025 A CN 117746025A
- Authority
- CN
- China
- Prior art keywords
- instrument
- dial
- pointer
- image
- substation equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a method and a system for indicating and identifying pointer type instrument of substation equipment, wherein the method comprises the following steps: acquiring a pointer instrument image of substation equipment shot by a substation inspection robot; extracting an instrument dial image based on the SE-YOLO target detection model; based on the improved key point detection model, position coordinates of four peaks of an instrument dial and the starting point and the end point of an indicator in the dial in an instrument dial image are obtained, and then perspective transformation is combined to carry out image correction on the instrument dial image; based on a SegFormer model, performing image segmentation on the corrected meter dial image to obtain a meter pointer, a long scale line and a meter digital region; and identifying the instrument number by adopting a template matching method, assigning values to the long scale marks according to the identified instrument number, and calculating the indication of the instrument by a distance-based method. The invention can realize accurate identification and reading of various pointer instrument readings of the substation equipment.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to an indication recognition method and system for pointer type meters of substation equipment.
Background
In a transformer substation scene, a large number of meters are needed to monitor the working state of each transformer substation device, because of factors such as complex electromagnetic environment, the meters in the transformer substation usually adopt a pointer type meter reading mode, the pointer type meters need manual reading, so that a large amount of labor is needed to be consumed, the reading precision is influenced by factors such as subjective factors and environmental illumination, and the like, so that the automatic identification of the pointer type meters and the automatic identification and reading of meter readings are researched to have important application values.
The realization of automatic identification and reading of pointer instrument readings usually takes a substation inspection robot as a platform, a visible light camera and the like are carried on the platform, corresponding substation equipment instrument images are acquired according to a planned path and inspection shooting points, then automatic identification and reading of instrument dials and dial readings are carried out on the shot images, and a data foundation is laid for subsequent monitoring and data analysis of substation equipment. Therefore, the accuracy of indicating number identification of the pointer instrument of the substation equipment is important to intelligent analysis and processing of the substation.
However, in the actual automatic identification process of the inspection robot, because the inspection robot has different shooting angles, shooting parameters, camera imaging mechanisms and the like on each device, the shot meter dial images often have the conditions of inclination, rotation, scaling and the like, and the accuracy of reading the readings in the subsequent meter images is affected; in addition, the existing pointer type meters of the substation equipment are more in types, meter dials are irregular, small target pointers and the like exist, and the existing pointer type meters based on the image recognition technology and the indication recognition method thereof are difficult to realize accurate recognition and reading of indication of the pointer type meters of the substation equipment of multiple types.
Disclosure of Invention
In order to solve the defects in the prior art, the invention provides a method and a system for identifying the indication of a pointer type instrument of substation equipment, which adopt an SE-YOLO target detection algorithm to extract an instrument dial image, utilize an improved key point detection algorithm and perspective transformation processing image, extract important information such as an instrument pointer, a scale, a number and the like in the image through a SegFormer algorithm, further calculate and obtain accurate indication of the instrument, improve the feasibility and the accuracy of reading the indication of the instrument, and realize the accurate identification and reading of the indication of various types of pointer type instruments of substation equipment.
In a first aspect, the invention provides a method for identifying the indication of a pointer instrument of substation equipment.
An indication recognition method of a pointer instrument of substation equipment comprises the following steps:
acquiring a pointer instrument image of substation equipment shot by a substation inspection robot;
based on the SE-YOLO target detection model, dial detection is carried out on pointer type instrument images of substation equipment, and instrument dial images are extracted;
based on the improved key point detection model, position coordinates of four peaks of an instrument dial and the starting point and the end point of an indicator in the dial in an instrument dial image are obtained, and then perspective transformation is combined to carry out image correction on the instrument dial image;
based on a SegFormer model, performing image segmentation on the corrected meter dial image to obtain a meter pointer, a long scale line and a meter digital region;
based on the areas of the instrument pointer, the long scale line and the instrument number, the instrument number is identified by adopting a template matching method, the long scale line is assigned according to the identified instrument number, and the indication of the instrument is calculated by a distance-based method.
In a second aspect, the invention provides an indication recognition system of a pointer instrument of substation equipment.
An indication recognition system of a pointer instrument of substation equipment, comprising:
the instrument image acquisition module is used for acquiring pointer instrument images of the substation equipment shot by the substation inspection robot;
the instrument dial image extraction module is used for carrying out dial detection on pointer instrument images of substation equipment based on the SE-YOLO target detection model and extracting instrument dial images;
the instrument dial image correction module is used for acquiring the position coordinates of the four peaks of the instrument dial in the instrument dial image and the starting point and the end point of the pointer in the dial based on the improved key point detection model, and carrying out image correction on the instrument dial image by combining perspective transformation;
the instrument dial image segmentation module is used for carrying out image segmentation on the corrected instrument dial image based on the SegFormer model to obtain an instrument pointer, a long scale mark and an instrument digital region;
the instrument indication recognition module is used for recognizing instrument numbers by adopting a template matching method based on the areas of instrument pointers, long scale marks and instrument numbers, assigning values to the long scale marks according to the recognized instrument numbers, and calculating the indication of the instrument by a distance-based method.
In a third aspect, the invention also provides an electronic device comprising a memory and a processor and computer instructions stored on the memory and running on the processor, which when executed by the processor, perform the steps of the method of the first aspect.
In a fourth aspect, the present invention also provides a computer readable storage medium storing computer instructions which, when executed by a processor, perform the steps of the method of the first aspect.
The one or more of the above technical solutions have the following beneficial effects:
1. the invention provides a method and a system for identifying the indication of pointer type meters of substation equipment, which extract meter dial images by adopting an improved YOLO algorithm, namely an SE-YOLO target detection algorithm, so as to realize the accurate extraction of meter dial images of different types of pointer type meters and different rules and lay a foundation for the accurate identification and reading of the indication of the meter dial of a subsequent meter.
2. The indicating number recognition method and system for the pointer type instrument of the substation equipment, provided by the invention, utilize the improved key point detection algorithm and perspective transformation processing image to solve the problem that indicating number reading accuracy in subsequent instrument images is affected due to the fact that the instrument dial plate image shot by the inspection robot is inclined, rotated and the like, and improve the indicating number recognition and reading accuracy of the instrument.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.
Fig. 1 is an overall flowchart of an indication recognition method of a pointer instrument of substation equipment according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a network structure of a SE-YOLO target detection model in an embodiment of the invention;
FIG. 3 is a schematic diagram of a Focus unit according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a compressed activated attention module according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a Neck module according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a network structure of an improved keypoint detection model in accordance with an embodiment of the present invention;
FIG. 7 is a schematic diagram of perspective transformation in an embodiment of the invention;
FIG. 8 is a schematic diagram of the segmentation result of the images of the dial of the instrument in the embodiment of the invention;
FIG. 9 is a schematic diagram of a scale of an arcuate dial plate tiled as a horizontal scale in an embodiment of the present invention;
fig. 10 is a schematic diagram of recognition results of the indication number of the meter dial in the embodiment of the invention.
Detailed Description
It should be noted that the following detailed description is exemplary only for the purpose of describing particular embodiments and is intended to provide further explanation of the invention and is not intended to limit exemplary embodiments according to the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Furthermore, it will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, steps, operations, devices, components, and/or groups thereof.
Example 1
In order to solve the problem that the existing image recognition method cannot realize automatic accurate recognition and reading of pointer instrument readings of multi-type substation equipment, the embodiment provides an indication recognition method of pointer instruments of the substation equipment, firstly, instrument dial detection is carried out by utilizing an SE-YOLO target detection model, instrument detection and instrument dial image extraction aiming at different types and different shape characteristics are realized, and the method has the advantages of high detection speed, high robustness and the like; secondly, due to reasons of shooting angles, equipment parameters and the like, certain inclination and rotation problems possibly exist in the meter dial image, and the follow-up work is difficult, so that an improved method for combining a key point detection method with perspective transformation is adopted to correct the extracted meter dial image; then, dividing long scale marks, instrument numbers and an instrument pointer area in an instrument dial by a SegFormer image dividing model; and finally, identifying the instrument number in the graph by adopting a template matching method, assigning a value to the long scale mark according to the shortest distance between the corresponding number and the long scale, and calculating the instrument reading by adopting a distance-based method. The method realizes accurate identification and reading of various pointer instrument readings of the substation equipment.
The indication recognition method of the pointer instrument of the substation equipment provided by the embodiment, as shown in fig. 1, comprises the following steps:
step S1, acquiring a pointer instrument image of substation equipment shot by a substation inspection robot;
step S2, dial detection is carried out on pointer type instrument images of substation equipment based on an SE-YOLO target detection model, and instrument dial images are extracted;
step S3, based on an improved key point detection model, acquiring position coordinates of four peaks of an instrument dial in an instrument dial image and a starting point and an ending point of a pointer in the dial, and carrying out image correction on the instrument dial image by combining perspective transformation;
s4, based on a SegFormer model, performing image segmentation on the corrected meter dial image to obtain a meter pointer, a long scale line and a meter digital region;
and S5, identifying the instrument number by adopting a template matching method based on the instrument pointer, the long scale line and the area of the instrument number, assigning the long scale line according to the identified instrument number, and calculating the indication of the instrument by a distance-based method.
The following describes each step and model in the indicating number identifying method of the pointer instrument of the substation equipment in this embodiment in more detail.
In step S2, dial detection is carried out on pointer type instrument images of substation equipment based on the SE-YOLO target detection model, and instrument dial images are extracted.
At present, in the target detection based on deep learning, a common deep learning network comprises a one-stage YOLO series algorithm, an SSD algorithm, a two-stage Fast-RCNN network, a Fast-RCNN network and the like, and the detection performance is further improved by adding an SE module to improve the YOLO algorithm in consideration of the fact that the YOLO series algorithm is Fast in detection speed, simple in network structure and high in robustness and accuracy, and therefore based on the YOLO 5 algorithm, the SE-YOLO target detection algorithm is provided.
The network structure of the SE-YOLO target detection model proposed in this embodiment is shown in fig. 2, and includes five parts, i.e., an Input module (Input), a BackBone network (BackBone), a compressed excitation attention module (Squeeze and Excitation, SE), a Neck network (Neck), and an output module (Head).
The Input module (Input) comprises a Mosaic (Mosaic) data enhancement unit, an adaptive anchor frame calculation unit and an adaptive picture scaling unit. In fact, the input image is preprocessed through the input module, wherein the mosaics data enhancement unit is used for randomly selecting a plurality of pictures from the data set to splice so as to increase small target data, thereby enriching and expanding the data set and enhancing the robustness. The potential candidate set of the target boundary frame is established by utilizing a self-adaptive anchor frame calculation method, so that network convergence can be accelerated and accuracy can be improved when a model is trained; and the size of the picture is adjusted by utilizing a self-adaptive picture scaling method, the picture size in the data set is unified, and the network model is convenient to optimize.
The BackBone network (BackBone) described above includes Focus units, conv units, bottleeckcsp units, and SPP units. As shown in fig. 3, the Focus unit is a special convolution structure, and is critical to slicing operation, so that a value is selected from every other pixel of a picture input into the backbone network to obtain four groups of pictures, and the dimension of a channel is enlarged to four times. Based on the Focus unit, the image before processing is an RGB three-channel image, the image is changed into twelve channels after being spliced, and the final double downsampled feature image is obtained through the convolution operation of the Conv unit. In the process of extracting the picture features based on the Focus unit, the information can be prevented from being lost to the greatest extent. The extracted feature map can be subjected to multi-scale aggregation to extract features through a Bottleneck CSP unit, so that the detection precision is further improved.
The bottlebeckcsp unit contains two parts, the bottleneck layer (bottlebeck) and CSP. Wherein, the bottleneck is a classical residual structure, and adopts a 1x1 convolution layer to reduce the calculated amount; the CSP divides the input into two parts, one part is firstly subjected to n times of bottleneck operation, then is subjected to convolution operation, and the other part is directly subjected to convolution operation, wherein the two convolution operations aim to halve the number of channels, and then the two parts are spliced by the concat and then output. Through the Bottleneck CSP unit, the calculation amount is reduced, and the learning ability of the model can be improved. The Bottleneck CSP unit has strong feature expression capability, high-efficiency calculation performance and effective information transmission capability: the Bottleneck CSP unit adopts the combination of depth-separable convolution (depth-wise separable convolution) and cross-stage part connection (cross stage partial connection), the depth-separable convolution can reduce the quantity of parameters and calculation, and meanwhile, stronger characteristic expression capacity is maintained, and the cross-stage part connection can promote information transmission and fusion, so that a model can learn richer characteristic expression; the Bottleneck CSP unit reduces the parameter quantity and the calculated quantity of the model through the use of the depth separable convolution, which is very important for running the target detection algorithm on the resource-limited equipment in real time, and the Bottleneck CSP unit can accelerate the reasoning process of the model and improve the detection speed by reducing the calculation complexity; the cross-stage partial connection allows direct information transfer between feature graphs of different stages, and the connection mode is beneficial to solving gradient disappearance and information bottleneck problems in a deep network, so that a model can better utilize information of low-level and high-level features, and the accuracy and the robustness of a target detection algorithm can be improved through effective information transfer by a Bottleneck CSP unit.
SPP (Spatial Pyramid Pooling) is a pyramid pooling structure consisting of three parts, namely a convolution layer conv, a maximum pooling layer maxpooling and a connection layer concat. Specifically, features are extracted and output by a convolution layer conv, the extracted features are downsampled by a maximum pooling layer maxpooling of three different kernel_size, respective output results are spliced and fused, the fused features are added with initial features, and finally the output is restored to be consistent with the initial input by the convolution layer conv.
The compressed Excitation attention module (namely SE module) introduces a spatial attention mechanism into a target detection framework through an explicit modeling channel interdependence relation, the basic structure of the module is shown in figure 4, the module comprises a compression (squeize) unit, an Excitation unit and a scaling (Scale) unit, a feature map extracted through a backbone network is input into the SE module, the extracted feature map is pooled through global average in the compression unit, two-dimensional features (H.times.W) of each channel of the feature map are compressed into 1 real number, and the feature map is converted into [1, c ] from [ H, W, c ], so that global features of channel level (namely channel level) are obtained; in the excitation unit, a weight value is generated for each characteristic channel, in the embodiment, the correlation among channels is constructed through two full-connection layers, so that the number of the output weight values is the same as the number of the channels for inputting the characteristic map, the characteristic map is converted from [1, c ] to [1, c ], and the relation among channels is learned in the process, so that the weights of different channels are obtained; in the scaling unit, the normalized weights learned in the previous step are weighted onto the features of each channel, and in this embodiment, the weighting is implemented by multiplying the weighting coefficients by channels, that is, [ h, w, c ] [1, c ], so as to convert the normalized weights into [ h, w, c ], and at this time, the processing of the feature map is completed.
The compression excitation attention module (Squeeze and Excitation, SE) performs an attention or a gating operation in a channel dimension, and the attention mechanism can make the model pay more attention to channel characteristics with the largest information quantity and restrain unimportant channel characteristics, so that the model can pay attention to more useful and more effective characteristics, redundant interference information is removed, and the gain of target detection accuracy is realized.
Specifically, a compression (Squeeze) operation is used to compress image features in the spatial dimension. The compression unit converts the two-dimensional characteristics of each channel into a real number, so that the characteristic information of each channel is fused together, the real number has a global receptive field to a certain extent, and the output dimension is matched with the input characteristic channel number. This real number represents the global distribution of responses over the characteristic channels and enables layers close to the input to also obtain a global receptive field. This compression operation is achieved by global averaging pooling, with the dimension of the output feature map being 1x1xC, where C is the number of feature channels.
After outputting the 1x1xC global feature by the compression operation, an Excitation operation is performed. Excitation refers to the prediction of the importance of each channel through the fully connected layers. Excitation operations can be seen as a mechanism similar to gates in recurrent neural networks, by learning parameters to generate weights for each characteristic channel, which are used to explicitly model the correlation between characteristic channels. In order to reduce the number of channels to reduce the amount of computation, a scaling parameter (sera) is introduced. In this embodiment, the excitation unit includes two full-connection layers, where the first full-connection layer has c×servio neurons, inputs a feature map of 1×1×c, outputs a feature map of 1×1×c×servio, and plays a role in dimension reduction; the second fully connected layer has C neurons, inputs a 1×1×c×seratio feature map, and outputs a 1×1×c feature map.
Scaling (Scale) operates to treat the weights output by the excitation operation as the importance of each feature channel after feature selection, and then apply the weights to the previous features by channel-by-channel multiplication, thereby recalibrating the original features in the channel dimension. The scaling operation is actually a channel-level reorganization of the original features.
The foregoing negk module is used for feature fusion and upsampling operations, and adopts a fpn+ PAN (Path Aggregation Network) structure, as shown in fig. 5, where the FPN layer conveys strong semantic features from top to bottom and the PAN conveys positioning features from bottom to top.
In the process of target detection, the resolution of the final detection output feature map is lower due to multiple downsampling operations, so that the detection accuracy is limited, and for this purpose, the Neck module upsamples the low-resolution feature map to the same size as the high-resolution feature map through upsampling operations (such as bilinear interpolation or transposed convolution), so that the detection accuracy and spatial resolution are improved. Meanwhile, the Neck module acquires multi-scale semantic information by fusing feature graphs from different levels, and the feature fusion is beneficial to improving the detection capability of the model on objects with different scales. By introducing operations such as feature cascade and feature superposition into the Neck module, low-level and high-level features can be effectively combined, and the receptive field size and feature expression capability of the model are improved.
The Neck module realizes multi-scale detection by processing feature graphs from different levels. By carrying out feature fusion and up-sampling on different levels, the model can have the perceptibility of targets with different scales, so that the model for simultaneously detecting targets with different sizes is more flexible, the adaptability of the model is improved, and more accurate and robust target detection results are realized.
The Head layer is configured to output a prediction frame, and includes three prediction indexes: rectangular box, confidence, classification probability. The rectangular frame represents the size and the accurate position of the target, the confidence coefficient represents the credibility of the prediction frame, the range is 0-1, and the classification probability represents the class of the target.
In this embodiment, the rectangular frame is used to label the meter dial areas in the pointer meter images of different types of substation equipment, a training set is constructed by using the pointer meter images of the substation equipment of the labeled meter dial, and the images in the training set are input into the built SE-YOLO target detection model to train the model. The loss function of the target detection model consists of the three prediction indexes, and the loss function is as follows:
Loss＝a×loss_obj+b×loss_rect+c×loss_clc
in the above formula, loss is a Loss function; loss_obj is rectangular frame Loss and is calculated by adopting a CIOU_loss method; the loss_rect and the loss_clc are confidence loss and classification loss respectively, and are calculated by adopting a BCE_loss method; a. b and c represent weight coefficients, and the preference can be adjusted manually.
In step S3, based on the improved key point detection model, position coordinates of four vertices of the meter dial and start points and end points of the pointer in the dial in the image of the meter dial are obtained, and then perspective transformation is combined to perform image correction on the image of the meter dial.
In practical application, because the shooting angles, shooting parameters, imaging mechanisms and the like of the inspection robot on different devices are different, the shot and extracted meter dial images often have conditions of inclination, rotation, scaling and the like, which affects the accuracy of subsequent readings. In order to eliminate the position difference between the images, the embodiment adopts an improved key point detection algorithm and perspective transformation to process the extracted meter dial images.
Firstly, an improved key point detection algorithm is adopted to obtain the position coordinates of four peaks of an instrument dial and the starting point and the ending point of a pointer in the dial in an instrument dial image. The embodiment adopts a key point detection method based on thermodynamic diagram, maps the image characteristics into the thermodynamic diagram (one of the characteristic diagrams is a visualized characteristic diagram), and converts the key points into probability distribution. The improved key point detection model adopts an improved FCN structure, namely a structure combining HourGlass and FCN, and particularly as shown in figure 6, the model adopts jump connection of channel-by-channel learning parameters to control the data flow of each channel in a module, learning is used for controlling information from the previous stage to be transmitted to the next stage in each channel, and each module is encouraged to learn more complex functions; meanwhile, a hybrid network structure HourGlass is introduced, the HourGlass and the FCN framework are combined, external convolution branches are introduced for jump connection based on the FCN decoding and encoding structure, the framework minimizes the number of identity connections in a network, the performance is improved under the same parameter budget, and the characteristic diagram of an input image can be accurately and rapidly obtained by using the model.
Finally outputting 6 feature graphs through the model, and predicting a coordinate on each feature graph to obtain the position coordinates (i.e. p 1 ,p 2 ,p 3 ,p 4 ) And the position coordinates (p s ,p e ) The calculation formula is as follows:
wherein,ω denotes the network weight, f denotes the feature map output by the network model, c is the number of channels of the last layer network, and c=6. Further, the optimal position p is obtained by a non-maximum suppression (NMS) method i I ε {1,2,3,4, s, e }, the formula is:
wherein,the calculated optimal position coordinates of the four vertexes of the instrument dial, the optimal position coordinates of the pointer starting point and the optimal position coordinates of the pointer ending point are respectively represented.
And secondly, carrying out image correction on the meter dial image based on the position coordinates of the four peaks of the meter dial and the starting point and the ending point of the pointer in the dial in the meter dial image by combining perspective transformation.
The principle of perspective transformation, i.e. projective transformation, is that a two-dimensional plane is projected onto a three-dimensional plane through a perspective matrix, and then converted back into the two-dimensional plane, as shown in fig. 7.
In the present embodiment, the position coordinates p 1 The point is exemplified by the two-dimensional plane coordinates before transformation are (x, y) T Projected to three-dimensional plane coordinates of [ X Y Z ]] T The three-dimensional matrix P is a perspective transformation matrix, then:
further, a transformation relationship between the three-dimensional plane coordinates and the two-dimensional plane coordinates can be obtained:
X＝a 11 ×x+a 12 ×y+a 13
Y＝a 21 ×x+a 22 ×y+a 23
Z＝a 31 ×x+a 32 ×y+a 33
the two-dimensional coordinates (x ', y') after perspective are:
the above perspective transformation is a transformation of two-dimensional coordinates, and the parameter Z in the perspective transformation does not play a role in the final transformation and may be set to 1. Based on the calculation mode, a transmission matrix P is obtained according to the corresponding coordinates of the four points before and after conversion, transmission conversion is carried out on the whole instrument dial image, the P matrix is directly utilized for solving and calculating, the instrument dial image which is rotated and distorted is corrected, and the corrected instrument dial image is obtained.
In step S4, based on the SegFormer model, the corrected meter dial image is subjected to image segmentation, and the areas of the meter pointer, the long scale line and the meter number are obtained by segmentation.
In order to obtain three key target areas, namely, an instrument pointer, a long scale line and an instrument number, image segmentation is required to be carried out on an instrument dial image, in this embodiment, an existing SegFormer model is adopted to carry out image segmentation operation on the key target areas of the instrument dial image, and the segmentation results are shown in fig. 8, so that the instrument number shown in fig. 8 (a), the long scale line shown in fig. 8 (b) and the area of the instrument pointer shown in fig. 8 (c) are respectively segmented.
In step S5, based on the meter pointer, the long scale line and the area of the meter number, the meter number is identified by using a template matching method, the long scale line is assigned according to the identified meter number, and the meter indication is calculated by a distance-based method.
First, meter numbers in the meter number area are identified. Namely, a single character area is divided in the instrument number area by using a binarization method; identifying a single number in a single character area by adopting a template matching method; all individual numbers identified are combined in order.
Specifically, because the gray values between the industrial numbers and the dial are greatly different, a binarization method is adopted to carry out connected domain analysis, and single characters are segmented from the instrument number area in the dial according to the minimum bounding rectangle; calculating the similarity between the single character image and the template image by adopting a template matching method, and obtaining the numerical value of the single character image according to the optimal matching result; since the dial numbers of the substation are always printed from left to right, the recognized numbers are combined in order of small to large according to the abscissa of the rectangular frame. In addition, for the identification of the decimal point in the number, the decimal point is identified by the connected domain area being significantly smaller than the connected domain area of the number.
And then, assigning the long graduation line according to the recognized instrument number. Specifically, the distance between the number and each long scale line is calculated, and accurate instrument number assignment is performed on the long scale line according to the principle that the instrument calibration value is closest to the distance of the corresponding scale line. By this method, the value corresponding to each long scale is obtained.
Finally, the gauge reading is calculated by a distance-based method. Specifically, the scales of the arc dial are mapped and tiled into a horizontal scale by mathematical transformation, as shown in fig. 9, and then the long scale values closest to the left and right sides of the meter pointer are respectively set as x 1 、x 2 The distance between the instrument pointer and the left and right long scale marks is d 1 、d 2 The reading of the meter is calculated, and the formula is:
by the above steps of the present embodiment, an accurate final reading of the meter can be obtained, as shown in fig. 10, where (a) in fig. 10 indicates that the identification meter reading is 0.409, (b) in fig. 10 indicates that the identification meter reading is 2.041, and (c) in fig. 10 indicates that the identification meter reading is 2.939.
Example two
The embodiment provides an indication recognition system of pointer type instrument of substation equipment, which comprises:
the instrument image acquisition module is used for acquiring pointer instrument images of the substation equipment shot by the substation inspection robot;
the instrument dial image extraction module is used for carrying out dial detection on pointer instrument images of substation equipment based on the SE-YOLO target detection model and extracting instrument dial images;
the instrument dial image correction module is used for acquiring the position coordinates of the four peaks of the instrument dial in the instrument dial image and the starting point and the end point of the pointer in the dial based on the improved key point detection model, and carrying out image correction on the instrument dial image by combining perspective transformation;
the instrument dial image segmentation module is used for carrying out image segmentation on the corrected instrument dial image based on the SegFormer model to obtain an instrument pointer, a long scale mark and an instrument digital region;
the instrument indication recognition module is used for recognizing instrument numbers by adopting a template matching method based on the areas of instrument pointers, long scale marks and instrument numbers, assigning values to the long scale marks according to the recognized instrument numbers, and calculating the indication of the instrument by a distance-based method.
Example III
The embodiment provides an electronic device, which comprises a memory, a processor and computer instructions stored on the memory and running on the processor, wherein the computer instructions complete the steps in the indicating number identification method of the pointer instrument of the substation equipment when the computer instructions are run by the processor.
Example IV
The present embodiment also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, perform the steps in the indication recognition method of the substation equipment pointer meter as described above.
The steps involved in the second to fourth embodiments correspond to the first embodiment of the method, and the detailed description of the second embodiment refers to the relevant description of the first embodiment. The term "computer-readable storage medium" should be taken to include a single medium or multiple media including one or more sets of instructions; it should also be understood to include any medium capable of storing, encoding or carrying a set of instructions for execution by a processor and that cause the processor to perform any one of the methods of the present invention.
It will be appreciated by those skilled in the art that the modules or steps of the invention described above may be implemented by general-purpose computer means, alternatively they may be implemented by program code executable by computing means, whereby they may be stored in storage means for execution by computing means, or they may be made into individual integrated circuit modules separately, or a plurality of modules or steps in them may be made into a single integrated circuit module. The present invention is not limited to any specific combination of hardware and software.
While the present invention has been described in connection with the preferred embodiments, it should be understood that the present invention is not limited to the specific embodiments, but is set forth in the following claims.
Claims (10)
1. The indicating number recognition method of the pointer instrument of the substation equipment is characterized by comprising the following steps of:
acquiring a pointer instrument image of substation equipment shot by a substation inspection robot;
based on the SE-YOLO target detection model, dial detection is carried out on pointer type instrument images of substation equipment, and instrument dial images are extracted;
based on the improved key point detection model, position coordinates of four peaks of an instrument dial and the starting point and the end point of an indicator in the dial in an instrument dial image are obtained, and then perspective transformation is combined to carry out image correction on the instrument dial image;
based on a SegFormer model, performing image segmentation on the corrected meter dial image to obtain a meter pointer, a long scale line and a meter digital region;
based on the areas of the instrument pointer, the long scale line and the instrument number, the instrument number is identified by adopting a template matching method, the long scale line is assigned according to the identified instrument number, and the indication of the instrument is calculated by a distance-based method.
2. The indicating number recognition method of the pointer instrument of the substation equipment according to claim 1, wherein the SE-YOLO target detection model comprises an input module, a main network, a compression excitation attention module, a neck network and an output module which are connected in sequence;
constructing a training data set by using a plurality of pointer instrument images of substation equipment marked with an instrument dial, inputting images in the training data set into a SE-YOLO target detection model, preprocessing the input images, and training the model by using the preprocessed images until a loss function is minimum, thereby completing training of the model; the loss function of the SE-YOLO target detection model consists of three prediction indexes including a rectangular frame, confidence level and classification probability.
3. The indicating number recognition method of the pointer instrument of the substation equipment according to claim 2, wherein the input module is used for preprocessing an image and comprises a mosaic data enhancement unit, an adaptive anchor frame calculation unit and an adaptive picture scaling unit; randomly selecting a plurality of pictures from the data set through a mosaic data enhancement unit to splice, and expanding the data set; establishing a potential candidate set of the target boundary rectangular frame in the picture through the self-adaptive anchor frame calculation unit; and adjusting the size of the picture by a self-adaptive picture scaling method, and unifying the sizes of the pictures in the data set.
4. The method for identifying the indication of the pointer instrument of the substation equipment according to claim 2, wherein the compressed excitation attention module is used for modeling the interdependence relationship among channels and introducing a spatial attention mechanism into a target detection framework;
the compressed excitation attention module comprises a compression unit, an excitation unit and a scaling unit, wherein the feature map extracted by the main network is input into the compressed excitation attention module, and the feature map is processed, and the compressed excitation attention module comprises:
in a compression unit, the input feature map is pooled through global average, and the two-dimensional features of each channel of the feature map are compressed into a real number, so that the global features of the channel level are obtained;
in the excitation unit, constructing global feature correlation among channels through two full-connection layers, enabling the number of the output weight values to be the same as the number of the channels of the input feature map, and generating a weight value for each feature channel through self-learning;
and normalizing the obtained weights in a scaling unit, and weighting the normalized weights to the characteristics of each channel to finish channel level reorganization of the original input characteristic map.
5. The indicating number recognition method of the pointer instrument of the substation equipment according to claim 1, wherein the obtaining of the position coordinates of the four peaks of the meter dial in the meter dial image and the start point and the end point of the pointer in the dial based on the improved key point detection model comprises:
outputting 6 feature images of the dial plate image of the instrument through an improved key point detection model, wherein each feature image predicts a position coordinate; the improved key point detection model adopts a framework combining a hybrid network and a full convolution neural network;
based on the 6 obtained feature images, the position coordinates of the four peaks of the instrument dial and the starting point and the ending point of the pointer in the dial in the instrument dial image are obtained by calculation, and the calculation formula is as follows:
wherein ω represents the network weight of the improved key point detection model, f represents the feature map output by the improved key point detection model, c is the number of channels of the last layer of network, and c=6;
and obtaining the optimal position coordinates of four peaks of the instrument dial and the starting point and the end point of the pointer in the dial in the instrument dial image through a non-maximum value inhibition method.
6. The indicating number recognition method for pointer instrument of transformer substation equipment according to claim 1, wherein recognizing the instrument number by using a template matching method comprises:
dividing a single character area in the instrument digital area by using a binarization method;
identifying a single number in a single character area by adopting a template matching method;
all individual numbers identified are combined in order.
7. The method for identifying the indication of the pointer instrument of the substation equipment according to claim 1, wherein the assigning the long scale line according to the identified instrument number comprises the following steps: and calculating the distance between all the recognized instrument numbers and each long scale line, and carrying out instrument number assignment on the long scale lines according to the principle that the instrument calibration value is closest to the distance of the corresponding scale line.
8. An indication recognition system of a pointer instrument of substation equipment is characterized by comprising:
the instrument image acquisition module is used for acquiring pointer instrument images of the substation equipment shot by the substation inspection robot;
the instrument dial image extraction module is used for carrying out dial detection on pointer instrument images of substation equipment based on the SE-YOLO target detection model and extracting instrument dial images;
the instrument dial image correction module is used for acquiring the position coordinates of the four peaks of the instrument dial in the instrument dial image and the starting point and the end point of the pointer in the dial based on the improved key point detection model, and carrying out image correction on the instrument dial image by combining perspective transformation;
the instrument dial image segmentation module is used for carrying out image segmentation on the corrected instrument dial image based on the SegFormer model to obtain an instrument pointer, a long scale mark and an instrument digital region;
the instrument indication recognition module is used for recognizing instrument numbers by adopting a template matching method based on the areas of instrument pointers, long scale marks and instrument numbers, assigning values to the long scale marks according to the recognized instrument numbers, and calculating the indication of the instrument by a distance-based method.
9. An electronic device comprising a memory and a processor and computer instructions stored on the memory and running on the processor, which when executed by the processor, perform the steps of a method for identifying an indication of a pointer instrument of a substation equipment according to any one of claims 1-7.
10. A computer readable storage medium storing computer instructions which, when executed by a processor, perform the steps of a method of indicating identification of a substation equipment pointer instrument according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311791940.2A CN117746025A (en) | 2023-12-22 | 2023-12-22 | Indication recognition method and system for pointer instrument of substation equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311791940.2A CN117746025A (en) | 2023-12-22 | 2023-12-22 | Indication recognition method and system for pointer instrument of substation equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117746025A true CN117746025A (en) | 2024-03-22 |
Family
ID=90257327
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311791940.2A Pending CN117746025A (en) | 2023-12-22 | 2023-12-22 | Indication recognition method and system for pointer instrument of substation equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117746025A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118334351A (en) * | 2024-06-12 | 2024-07-12 | 广东工业大学 | A pointer instrument reading recognition method based on deep learning |
| CN119516446A (en) * | 2025-01-16 | 2025-02-25 | 四川农业大学 | A real-time animal tracking and positioning method and computer device in a multi-target scene |
| CN119516532A (en) * | 2025-01-16 | 2025-02-25 | 金仪计量科技(苏州)有限公司 | Pointer pressure gauge reading recognition method and system based on multi-task interactive network |
| CN119919922A (en) * | 2025-04-03 | 2025-05-02 | 华雁智能科技(集团)股份有限公司 | Substation meter low-level defect identification method, system, equipment and storage medium |
-
2023
- 2023-12-22 CN CN202311791940.2A patent/CN117746025A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118334351A (en) * | 2024-06-12 | 2024-07-12 | 广东工业大学 | A pointer instrument reading recognition method based on deep learning |
| CN119516446A (en) * | 2025-01-16 | 2025-02-25 | 四川农业大学 | A real-time animal tracking and positioning method and computer device in a multi-target scene |
| CN119516532A (en) * | 2025-01-16 | 2025-02-25 | 金仪计量科技(苏州)有限公司 | Pointer pressure gauge reading recognition method and system based on multi-task interactive network |
| CN119919922A (en) * | 2025-04-03 | 2025-05-02 | 华雁智能科技(集团)股份有限公司 | Substation meter low-level defect identification method, system, equipment and storage medium |
| CN119919922B (en) * | 2025-04-03 | 2025-06-24 | 华雁智能科技(集团)股份有限公司 | Substation meter low-level defect identification method, system, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112766244B (en) | Target object detection method and device, computer equipment and storage medium | |
| CN117746025A (en) | Indication recognition method and system for pointer instrument of substation equipment | |
| CN111950453B (en) | Random shape text recognition method based on selective attention mechanism | |
| US20240070972A1 (en) | Rendering new images of scenes using geometry-aware neural networks conditioned on latent variables | |
| CN109712071B (en) | UAV image stitching and positioning method based on track constraints | |
| US20230145048A1 (en) | Real-time body pose estimation system and method in unconstrained video | |
| CN113642582A (en) | Ammeter reading identification method and device, electronic equipment and storage medium | |
| CN114842389B (en) | Real-time robust two-stage attitude estimation method | |
| CN117036891A (en) | Cross-modal feature fusion-based image recognition method and system | |
| CN117727046A (en) | Novel mountain torrent front-end instrument and meter reading automatic identification method and system | |
| CN116935474A (en) | Micro-expression recognition method and device, electronic equipment and storage medium | |
| CN110969104B (en) | Method, system and storage medium for detecting drivable area based on binary network | |
| CN110298233B (en) | Palmprint recognition method, palmprint recognition device, computer equipment and storage medium | |
| CN117474932B (en) | Object segmentation method and device, electronic equipment and storage medium | |
| CN119516576A (en) | A lightweight human posture estimation method based on Lite GS-HRNet | |
| CN118864841A (en) | Image segmentation method, storage medium and device taking into account boundary information | |
| CN118657716A (en) | Insulator fault detection method based on multi-scale expansion and perceptual codec | |
| CN117115432A (en) | Defect detection method and device for distribution line, electronic equipment and medium | |
| CN112818965B (en) | Multi-scale image target detection method and system, electronic equipment and storage medium | |
| CN116051422A (en) | Portrait beauty method, system and storage medium based on CNN | |
| CN116503737A (en) | Ship detection method and device based on space optical image | |
| CN115457499A (en) | Deep learning network training method, device, equipment and storage medium | |
| CN117557933A (en) | Electric vehicle violation identification method based on deep learning | |
| CN119360394B (en) | An industrial pointer instrument reading recognition system based on deep learning | |
| CN120047450B (en) | Remote sensing image change detection method based on double-current time-phase characteristic adapter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |