CN118799813A - Automatic identification method of airport support nodes based on relative position learning - Google Patents

Abstract

The present invention discloses an automatic identification method for airport support nodes based on relative position learning, which relates to the field of apron support technology, including: S1, according to the relative position relationship between the support vehicle and the aircraft, a data set is produced to mark the relative position data of the support vehicle and the aircraft; S2, the data set is trained by deep learning to obtain a prediction model; S3, the image data transmitted in real time from the airport is detected by an improved YOLOV7 model to obtain the key point position data of the aircraft; S4, the key point position data is input into the prediction model to complete the automatic identification of the airport support nodes. The present invention makes up for the shortcomings of the support node identification method that completely relies on image detection, and improves the detection efficiency of the support node.

Description

Automatic identification method of airport support nodes based on relative position learning

Technical Field

The invention relates to the technical field of apron support, and in particular to an automatic identification method of airport support nodes based on relative position learning.

Background Art

The airport security node system is a vital part of modern airport operations. It covers a series of technologies and facilities to ensure the efficiency, safety and reliability of various airport operations. In traditional airport operation management, the collection of security node information usually relies on manual observation and manual reporting, which has some problems. Relying solely on manual observation and manual reporting will face the following problems: increasing the workload of personnel, which may affect the quality of work; the real-time performance of data is poor, which may lead to untimely information; manual reporting is prone to omissions, false reports and other problems, which affects the accuracy and reference of the data. The airport security node system can solve the above problems, and through the integration of multiple subsystems and equipment, real-time detection and feedback coordination can be carried out to ensure the order of the airport.

At present, image processing technology plays a very important role in flight support. Through intelligent sensing technology, sensor networks and surveillance cameras are used to monitor the situation in various areas of the airport in real time. The collected image data can be used to detect the location and movement of aircraft, vehicles, luggage and personnel through image recognition technology, as well as environmental factors such as weather, temperature and visibility. Establish a reliable communication and network infrastructure to ensure information exchange and interconnection between various devices. This includes technologies such as local area networks, wireless networks, and satellite communications to ensure the stable operation of the system in complex environments. Integrate with the systems of air traffic control departments and airlines to ensure flight safety and normal operation.

The existing automatic entry of support nodes based on image recognition requires identifying the type of support vehicle based on the video and being able to track the vehicle during the entire video process. However, this technology may have serious errors when the support vehicle is obscured or exposed to rain, fog and light, which greatly reduces the recognition effect of the entire system.

Summary of the invention

In view of the above-mentioned deficiencies in the prior art, the airport support node automatic identification method based on relative position learning provided by the present invention does not directly identify the support vehicle, but first identifies the position and posture of the aircraft, and predicts the relative position of the support vehicle and the aircraft based on the position and posture of the aircraft. As long as a vehicle enters the identification area, it can be considered as the appearance of the corresponding support vehicle. Since the method only needs to identify several key parts of the aircraft based on image recognition technology, such as the nose and the tail, the recognizability of these two key parts is very high, and they can be well identified even in rain and fog or at night, and there is almost no problem of being blocked. It solves the problem that the existing support node automatic entry method based on image recognition cannot perform support node detection when the image recognition effect is poor.

In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: an automatic identification method of airport support nodes based on relative position learning, comprising:

S1. Create a data set with the relative position data of the support vehicle and the aircraft according to the relative position relationship between the support vehicle and the aircraft;

S2. Use deep learning to train the data set to obtain a prediction model;

S3. Use the improved YOLOV7 model to detect the real-time image data transmitted from the airport to obtain the key point position data of the aircraft;

S4. Input the key point location data into the prediction model to complete the automatic identification of airport support nodes.

Further: S1 includes:

S11, randomly transforming the 3D image of the aircraft at different shooting angles and shooting distances, and marking key points of the data to obtain aircraft posture image data;

S12. Obtain relative position features based on the aircraft posture image data and the corresponding support vehicle position, and use the relative position features as a data set for annotating the relative position data of the support vehicle and the aircraft.

Further: in S12, the relative position features include the nose x-coordinate, the nose y-coordinate, the tail x-coordinate, the tail y-coordinate, the aircraft orientation, the upper left corner coordinate of the support vehicle detection frame, and the lower right corner coordinate of the support vehicle detection frame.

Further: in S2, the prediction model includes an input layer, a first hidden layer, a second hidden layer and an output layer connected in sequence;

The first hidden layer includes 64 neurons, and the second hidden layer includes 32 neurons.

Further: In S3, the method for improving the YOLOV7 model to process the image data transmitted in real time from the airport includes:

S31. Input the real-time image data from the airport into the improved YOLOV7 model, and extract feature maps of three different sizes by improving the backbone network and neck network of the YOLOV7 model;

S32. Use the decoupling head to predict the feature maps of three different sizes respectively, and obtain the Cls feature for classification, the Reg feature for positioning, and the Obj feature for the confidence task;

S33, the Cls feature used for classification, the Reg feature used for positioning, and the Obj feature used for the confidence task are combined into a final feature map through a stacking operation, and the final feature map is used as the key point position data of the aircraft.

Further: in said S32, the decoupling head includes a classification branch and a regression branch;

The classification branch includes two 3×3 convolutional layers and one 1×1 convolutional layer connected in sequence;

The classification branch outputs a Cls feature for classification;

The regression branch includes two 3×3 convolutional layers connected in sequence and a 1×1 convolutional layer in parallel;

The regression branch outputs Reg features for positioning and Obj features for confidence tasks.

Further: S4 includes:

S41, inputting the key point position data into the prediction model, and outputting the detection frame of the security vehicle through the prediction model;

S42, tracking the movement trajectory of the support vehicle segment in the detection frame of the support vehicle, obtaining the start time node and the end time node of the airport support according to the movement trajectory of the support vehicle segment, and completing the automatic identification of the airport support node.

The beneficial effects of the present invention are:

The airport support node automatic identification method based on relative position learning provided by the present invention does not directly identify the support vehicle, but first identifies the posture of the aircraft, and predicts the relative position of the support vehicle and the aircraft based on the posture of the aircraft. As long as a vehicle enters the identification area, it can be considered that the corresponding support vehicle appears.

Since this method only needs to identify several key parts of the aircraft based on image recognition technology, such as the nose and the tail, these two key parts have high identifiability and can be well identified in rain, fog or at night, and there is almost no problem of being blocked. It solves the problem that the existing automatic entry method of support nodes based on image recognition cannot perform support node detection when the image recognition effect is poor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of an automatic identification method for airport support nodes based on relative position learning.

Figure 2 is a schematic diagram of key point annotation of the 3D model of an aircraft at different distances.

FIG3 is a schematic diagram of key point annotation of the aircraft 3D model at different top-down angles.

Figure 4 is a schematic diagram of the front and back sides of the aircraft relative positions.

FIG5 is a schematic diagram of the prediction model structure.

FIG6 is a schematic diagram showing the principle of an automatic identification method for airport support nodes based on relative position learning.

DETAILED DESCRIPTION

The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

As shown in FIG1 , in one embodiment of the present invention, a method for automatically identifying airport support nodes based on relative position learning is provided, comprising:

S1. Create a data set with the relative position data of the support vehicle and the aircraft according to the relative position relationship between the support vehicle and the aircraft;

S2. Use deep learning to train the data set to obtain a prediction model;

S3. Use the improved YOLOV7 model to detect the real-time image data transmitted from the airport to obtain the key point position data of the aircraft;

S4. Input the key point location data into the prediction model to complete the automatic identification of airport support nodes.

The S1 includes:

S11, randomly transforming the 3D image of the aircraft at different shooting angles and shooting distances, and marking key points of the data to obtain aircraft posture image data;

S12. Obtain relative position features based on the aircraft posture image data and the corresponding support vehicle position, and use the relative position features as a data set for annotating the support vehicle and aircraft relative position data.

In one embodiment of the present invention, as shown in FIG. 2 and FIG. 3 , in the S11, FIG. 2 shows a schematic diagram of key point annotation of a 3D model of an aircraft at different distances, FIG. 2 (a) represents a shooting distance of a short distance, FIG. 2 (b) represents a shooting distance of a medium distance, and FIG. 2 (c) represents a shooting distance of a long distance;

Figure 3 shows screenshots of the 3D model at three top-view angles of 0°, 30°, and 60°. Figure 3(a) represents a top-view angle of 0°, Figure 3(b) represents a top-view angle of 30°, and Figure 3(c) represents a top-view angle of 60°.

Corresponding annotation data is generated for each different combination of angle and distance, marking the position of the aircraft nose and tail as well as the relative positions of different support vehicles in the current aircraft posture. These annotated position data are used as training data for support vehicle position prediction.

In one embodiment of the present invention, the relative position feature selection process in S12 is a delicate and critical task, which includes the screening of the relative positions of the aircraft and the support vehicles, and usually involves different types of relative position features such as the nose position, the tail position, and the passenger elevator position. When performing feature selection, it is first necessary to observe and analyze the image or video to determine the position and type of each feature. Then, according to the shape, size, position and other characteristics of the target, different calculation formulas are used to determine its specific annotation position and attributes.

In this embodiment, the selected relative position features include the nose x-coordinate, nose y-coordinate, tail x-coordinate, tail y-coordinate, aircraft orientation, upper left corner coordinate of the support vehicle detection frame, and lower right corner coordinate of the support vehicle detection frame;

Since the origin of the image is in the upper left corner of the image, we use the distance from the center point of the nose annotation box to the x coordinate of the upper left corner of the image as the x coordinate of the nose, the distance from the center point of the nose annotation box to the y coordinate of the upper left corner of the image as the y coordinate of the nose, the distance from the center point of the tail annotation box to the x coordinate of the upper left corner of the image as the x coordinate of the tail, and the distance from the center point of the tail annotation box to the y coordinate of the upper left corner of the image as the y coordinate of the tail;

Therefore, the x-coordinate of the nose is the x-coordinate of the center point of the nose annotation box; the y-coordinate of the nose is the y-coordinate of the center point of the nose annotation box; the x-coordinate of the tail is the x-coordinate of the center point of the tail annotation box, and the y-coordinate of the tail is the y-coordinate of the center point of the tail annotation box;

As shown in Figure 4, in the feature selection of the aircraft orientation, Figure 4 (a) marks the nose of the aircraft facing the video as the positive nose, and the nose of the aircraft facing the video as the positive tail. Figure 4 (b) marks the nose of the aircraft facing the video as the reverse nose, and the tail of the aircraft facing the video as the reverse tail, so as to enhance the recognition accuracy of the passenger elevator.

Since the relative position of the support vehicle and the aircraft needs to be predicted as the detection frame position of the support vehicle, and the detection frame only needs to determine the coordinates of the upper left corner and the lower right corner of the detection frame, it is worth pointing out that it is not easy to predict the absolute position of the support vehicle in the picture, but it is easier to predict the relative position of the support vehicle and the aircraft. Therefore, the coordinates of the upper left corner and the lower right corner of the detection frame are the relative positions of the detection frame and the nose of the aircraft;

The detection of the support vehicle is done by determining the x-coordinate and y-coordinate of the support vehicle. The detection frame is determined by the x-, y-coordinate and width and height of the support vehicle. The center position of the support vehicle is: X _{support vehicle} = X _{support vehicle x-coordinate} - X _head ;

Where _{Xsupportvehicle} is the x-coordinate of the support vehicle relative to the nose, _{Xsupportvehiclexcoordinate} is the absolute x-coordinate of the support vehicle, and _Xnose is the actual x-coordinate of the nose;

The coordinates of the upper left corner of the security vehicle detection frame = (X _{security vehicle} - W _{security vehicle} / 2, y - H _{security vehicle} / 2), and the coordinates of the lower right corner of the security vehicle detection frame = (X _{security vehicle} + W _{security vehicle} / 2, y + H _{security vehicle} / 2), where W _{security vehicle} is the width of the security vehicle detection frame and H _{security vehicle} is the height of the security vehicle detection frame. In this way, the coordinates of the upper left corner and the lower right corner can be obtained, thereby determining the position of the entire security vehicle detection frame.

Through the annotations of Figure 4(a) and Figure 4(b), we obtained the relative position training data as shown in Table 1:

Table 1 Relative position data of aircraft and its support vehicles based on 3D model

Serial number Head x coordinate Head y coordinate Tail x coordinate Tail y coordinate Aircraft direction Ensure the coordinates of the upper left corner of the vehicle detection frame Ensure the coordinates of the lower right corner of the vehicle detection frame Protected vehicle type Figure 4(a) 0.493229 0.492969 0.497396 0.538281 Positive 0.614323 0.561719 Passenger elevator Figure 4(b) 0.271094 0.508984 0.599740 0.513672 Reverse 0.380208 0.564844 Passenger elevator

As shown in FIG. 5 , in S2 , the prediction model includes an input layer, a first hidden layer, a second hidden layer and an output layer connected in sequence; the first hidden layer includes 64 neurons, and the second hidden layer includes 32 neurons.

Using the prepared feature annotation dataset, the preprocessed data is input into the prediction model for training, and the weights are updated through the back propagation algorithm to minimize the loss function. The model updates the weights through the back propagation algorithm to minimize the loss function.

As shown in Figure 6, the upper part describes how to identify the key point data of an aircraft based on the YOLOV7 model. At the input end, the batched image data is processed after feature processing and a series of data enhancement techniques. The backbone network is the core part of the YOLOv7 model, responsible for feature extraction and representation of the input image. It usually consists of a series of convolutional layers, pooling layers, and other basic operations to convert the input image into a feature map with higher-level semantic information. The main function of the backbone network is to extract features from the original image so that the model can understand the content and structure in the image.

The neck network is located behind the backbone network and is responsible for further processing and fusing information from feature maps at different levels to improve the model's detection performance of the target. The design of the neck network uses operations such as feature fusion, scale transformation, and attention mechanism to enhance the model's perception and target modeling capabilities.

Since the field of target detection includes two tasks, classification and prediction, the classification task mainly focuses on the similarity between the features extracted by the feature layer and the real category features, and the prediction task mainly focuses on the position coordinates of the prediction box and the real box to correct the parameters of the prediction bounding box. Since the goal is to collect the data in Table 1, in order to make the data in Table 1 as accurate as possible, it is necessary to make the parameter correction of the prediction bounding box as accurate as possible. The YOLO series of algorithms have always used the same feature map for classification and positioning tasks, and different tasks use the same shared parameters, so that the two tasks cannot focus on their respective targets, and their detection effect will deteriorate. In order to separate the classification and prediction tasks, the present invention replaces the prediction head at the network output end in YOLOv7 with a decoupled head structure.

In S3, the method for improving the YOLOV7 model to process the image data transmitted in real time from the airport includes:

S31. Input the real-time image data from the airport into the improved YOLOV7 model, and extract feature maps of three different sizes by improving the backbone network and neck network of the YOLOV7 model;

S32. Use the decoupling head to predict the feature maps of three different sizes respectively, and obtain the Cls feature for classification, the Reg feature for positioning, and the Obj feature for the confidence task;

S33, the Cls feature used for classification, the Reg feature used for positioning, and the Obj feature used for the confidence task are combined into a final feature map through a stacking operation, and the final feature map is used as the key point position data of the aircraft.

In the S32, the decoupling head includes a classification branch and a regression branch;

The classification branch includes two 3×3 convolutional layers and one 1×1 convolutional layer connected in sequence;

The classification branch outputs a Cls feature for classification;

The regression branch includes two 3×3 convolutional layers connected in sequence and a 1×1 convolutional layer in parallel;

The regression branch outputs Reg features for positioning and Obj features for confidence tasks.

The S4 includes:

S41, inputting the key point position data into the prediction model, and outputting the detection frame of the security vehicle through the prediction model;

S42, tracking the movement trajectory of the support vehicle segment in the detection frame of the support vehicle, obtaining the start time node and the end time node of the airport support according to the movement trajectory of the support vehicle segment, and completing the automatic identification of the airport support node.

Once a vehicle is detected at the predicted relative position, it means that the corresponding support vehicle has entered the work area, and its movement trajectory can be tracked. This can be achieved by matching the bounding box of the vehicle between consecutive frames. When a vehicle is detected, the timestamp of its appearance is recorded. At the same time, when the vehicle disappears from the image or video, the corresponding timestamp is also recorded. By analyzing these recorded timestamp data, the time when the vehicle appears and disappears at the predicted position can be obtained. The appearance time of each detected vehicle, that is, after various vehicles are in place, the start time of the support node can be obtained, and it is executed according to the normal procedure. After the start of the support node, each vehicle enters in an orderly manner. After boarding is completed, each vehicle evacuates in an orderly manner. After the disappearance time of each vehicle is obtained, the end time of the support node can be obtained when the aircraft takes off.

The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims (7)

1. The airport guarantee node automatic identification method based on relative position learning is characterized by comprising the following steps:

S1, manufacturing a data set for marking the relative position data of a guarantee vehicle and an airplane according to the relative position relation between the guarantee vehicle and the airplane;

s2, training the data set by deep learning to obtain a prediction model;

s3, detecting image data transmitted in real time to an airport by using an improved YOLOV model to obtain key point position data of the airplane;

and S4, inputting the position data of the key points into a prediction model to finish automatic identification of the airport guarantee nodes.

2. The method for automatically identifying airport security nodes based on relative position learning of claim 1, wherein S1 comprises:

s11, carrying out random transformation on the 3D images of the aircraft at different shooting angles and shooting distances, and carrying out data key point labeling to obtain pose image data of the aircraft;

s12, acquiring relative position features according to the pose image data of the aircraft and the corresponding position of the guarantee vehicle, and taking the relative position features as a data set for labeling the relative position data of the guarantee vehicle and the aircraft.

3. The method for automatically identifying airport security nodes based on relative position learning of claim 1, wherein in S12, the relative position features comprise a nose x coordinate, a nose y coordinate, a tail x coordinate, a tail y coordinate, an airplane orientation, an upper left corner coordinate of a security vehicle detection frame, and a lower right corner coordinate of a security vehicle detection frame.

4. The method for automatically identifying airport security nodes based on relative position learning according to claim 1, wherein in S2, the prediction model comprises an input layer, a first hidden layer, a second hidden layer and an output layer which are sequentially connected;

The first hidden layer includes 64 neurons and the second hidden layer includes 32 neurons.

5. The method for automatically identifying airport security nodes based on relative position learning according to claim 1, wherein in S3, the method for processing image data of an airport incoming in real time by using the improved YOLOV model comprises the following steps:

S31, inputting image data transmitted in real time to an airport into an improved YOLOV model, and extracting three feature images with different sizes by improving a main network and a neck network of the YOLOV model;

s32, respectively predicting three feature graphs with different sizes by using a decoupling head to obtain Cls features for classification, reg features for positioning and Obj features for confidence tasks;

S33, combining the Cls features for classification, the Reg features for positioning and the Obj features for confidence tasks into a final feature map through stacking operation, and taking the final feature map as the key point position data of the airplane.

6. The method for automatically identifying airport security nodes based on relative position learning of claim 5, wherein in S32, the decoupling head comprises a classification branch and a regression branch;

the classifying branch comprises two layers of 3 multiplied by 3 convolution layers and one layer of 1 multiplied by 1 convolution layer which are sequentially connected;

The classification branch outputs Cls characteristics for classification;

the regression branch comprises two layers of 3 multiplied by 3 convolution layers and a parallel layer of 1 multiplied by 1 convolution layer which are connected in sequence;

the regression branches output Reg features for localization and Obj features for confidence tasks.

7. The method for automatically identifying airport security nodes based on relative position learning of claim 1, wherein S4 comprises:

s41, inputting the position data of the key points into a prediction model, and outputting a detection frame for guaranteeing the vehicle through the prediction model;

S42, tracking a motion track of a guarantee vehicle section in a detection frame of the guarantee vehicle, obtaining a start time node and an end time node of airport guarantee according to the motion track of the guarantee vehicle section, and completing automatic identification of the airport guarantee node.