CN110412564A - A train wagon recognition and ranging method based on multi-sensor fusion - Google Patents

Abstract

The invention discloses a train wagon recognition and ranging method based on multi-sensor fusion, comprising the following steps: S1 dual sensors collect and process target area data to obtain respective standard data; S2 construct data according to respective standard data Integrate the platform and obtain the effective target area; S3 identifies and detects the target according to the effective target area; S4 extracts the distance information of the target wagon in the effective target area, and transmits the obtained information of whether the target wagon is the target wagon and the corresponding distance information to the front end. The invention can automatically measure the distance between the rear of the train car and the wagon to be towed, and provide the distance parameter for the locomotive driver in real time, so that the locomotive can adjust the train speed accurately and in real time, and ultimately achieve reliable and safe hooking of the towed train.

Description

A train wagon recognition and ranging method based on multi-sensor fusion

technical field

The invention belongs to the technical field of traffic control systems, and in particular relates to a train wagon recognition and ranging method based on multi-sensor fusion.

Background technique

At present, freight trains are attached to wagons in the freight yard by manual methods. This method uses human eyes to measure the distance between the train and the wagons to be attached. The error is relatively large. The locomotive driver’s adjustment of the speed by using this distance parameter is not accurate enough, which affects the wagons. Work quality of the hook job. In addition, due to the influence of factors such as seasons, weather, occlusion and light, the manual visual distance will be further affected, which further reduces the operating efficiency.

After receiving the order, the driver and guide of the train observe and distinguish with the naked eye, and then cooperate with the relevant procedures. The wagons to be attached are distributed on different forks. The train driver is responsible for driving the train into the forks where the wagons are to be attached and judges with the naked eye. The coupling is done at a controlled speed, which is widely used on most of the freight railways in the country. However, manual work has the following disadvantages. First, the scheduling strategy is complex and the traffic efficiency is low. Second, the controllability of manual work is low, and misjudgment and negligence are prone to occur, resulting in operational accidents (when the vehicle speed is too fast (greater than 5km/h), the wagon collision occurs, causing the wagon body to deform; the vehicle speed is too slow (less than 5km/h). h), reasoning is insufficient, causing wagons to be articulated). Third, it is seriously affected by weather, such as midnight, and the ability of manual scheduling affected by lighting conditions will be greatly reduced.

At present, the advantages of mainstream ranging sensor technology such as infrared ranging are cheap price and safe working process, but poor anti-interference, short distance (effective distance is usually within ten meters), and poor directionality; the advantages of laser radar ranging are accurate, The disadvantage is that human safety needs to be paid attention to during the working process, and the production is difficult and costly, and the optical system needs to be kept clean, otherwise it will affect the measurement; the millimeter-wave radar has a long distance (the effective distance is usually within 100 meters) and real-time performance Good but the false alarm rate is high, and the visual sensor has strong recognition ability. The disadvantage is that the ranging strategy is complex and the accuracy is low. Due to the influence of environmental factors, such as the cold environment (about ten degrees below zero) caused by seasonal changes, the low-visibility environment such as foggy and rainy nights, and the complex environment where non-target objects at the work site (cargo factory) cause occlusion or interference, although the researchers In the field of sensor research, by improving system hardware performance and optimizing algorithms, or proposing new system solutions, the performance of a single sensor can be greatly improved. However, the precise perception ability of a single sensor in the above-mentioned complex traffic conditions still needs to be improved. For this research People have gradually begun to pay attention to the recognition method of multi-sensor information fusion, and in recent years, the progress of software and hardware technology and the rapid development of mathematical algorithms have provided technical and theoretical support for multi-sensor fusion. Through various algorithms and mathematical models, the distance to the target in front can be automatically detected. This control mode with high controllability and high degree of automation has gradually become a research hotspot and technical research field at home and abroad. Therefore, in order to meet the needs of the working process, the sensor selection includes millimeter-wave radar and monocular RGB vision sensor fusion sensors, which complement each other, can reduce single dependence and improve performance to achieve the goal.

Contents of the invention

The invention discloses a train wagon recognition and ranging method based on multi-sensor fusion. The method adopts the fusion of millimeter-wave radar and visual sensor to automatically measure the distance between the rear of the train wagon and the wagon to be towed, and provide real-time The locomotive driver provides this distance parameter, which is convenient for the locomotive to adjust the train speed in real time and accurately, and finally achieves reliable and safe hooking up of the waiting train.

The present invention is realized through the following technical solutions: a multi-sensor fusion-based train wagon recognition and ranging method, the method comprising the following steps:

S1 dual sensors collect and process the data of the target area to obtain their respective standard data;

S2 builds a data fusion platform based on their respective standard data and obtains an effective target area;

S3 identifies and detects the target according to the effective target area;

S4 extracts the distance information of the target wagon in the effective target area, and transmits the obtained information about whether the target wagon is the target wagon and the corresponding distance information to the remote terminal controller in real time.

Further, step S1 includes:

S11 The first sensor collects the image data of the front area in real time, and outputs the original image data; at the same time, the second sensor collects the radar data of the front area in real time, and outputs the original radar data;

S12 standardizes the image raw data and the radar raw data to obtain image standard data and radar standard data respectively, and executes step S2.

Further, step S12 includes:

S121 performing distortion correction on the original image data;

S122 encodes the distortion-corrected original image data;

S123 performs pixel processing on the encoded original image data, and then generates image standard data, and executes step S2.

Further, step S12 also includes:

S124 performs threshold filtering on the radar raw data;

S125 performs Kalman state estimation on the threshold-filtered raw radar data, and then generates radar standard data, and executes step S2.

Further, it includes: a first sensor installation calibration module, a second sensor installation calibration module, a joint calibration module and a time synchronization module, wherein,

The first sensor is equipped with a calibration module, which is used to install and calibrate the first sensor;

The second sensor is equipped with a calibration module, which is used to install and calibrate the second sensor;

The joint calibration module is used for joint calibration of the first sensor and the second sensor;

The time synchronization module is configured to perform time synchronization on the first sensor and the second sensor,

Include in step S2:

S21 installing and calibrating the first sensor;

S22 installing and calibrating the second sensor;

S23 performs joint calibration and time synchronization on the first sensor and the second sensor to obtain an effective target area.

Further, the first sensor is a camera sensor.

Further, the second sensor is a radar sensor.

Further, the radar sensor is a millimeter wave radar.

Further, step S3 includes:

S31 training to obtain a deep neural network target detector;

S32: The effective target area is detected by the deep neural network target detector to detect the railroad track in the front area, and the effective measurement area containing the target wagon is obtained by clipping.

Further, in step S4, specifically, steps S1-S3 are packaged and integrated, and the distance information of the target wagon is extracted from the effective measurement area obtained by image recognition, and the obtained information on whether the target wagon is obtained and the corresponding distance The information is transmitted to the remote terminal controller in real time.

The beneficial effects of the present invention are:

First, through the fusion of the millimeter wave mine sensor and the image sensor, it can greatly improve the defect that the single sensor can not obtain enough information in the road environment;

Second, in severe weather conditions, human judgment will be severely disturbed by weather factors such as light and low temperature, while millimeter-wave radar is relatively strong in environmental adaptability, which greatly improves the potential of on-board environmental perception to adapt to all-weather work, including winter. (when the temperature is lower than 30°C), summer, daytime, nighttime, sunny day, foggy day, rainy day, smog day, etc. Compared with only relying on manual operation, it greatly improves the reliability, accuracy and adaptability of the environmental perception ability in the working process;

Third, through the use of sensors, the distance of the target wagon can be calculated scientifically and accurately instead of the fuzzy judgment obtained by the operator based on experience and naked eye observation, thus greatly eliminating the defect of low efficiency and instability of manual control;

Description of drawings

Fig. 1 is a method flow chart of a train wagon recognition and ranging method based on multi-sensor fusion of the present invention;

Fig. 2 is a module execution diagram of a train wagon recognition and ranging method based on multi-sensor fusion in the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

With reference to shown in Fig. 1, the present invention is realized through the following technical solutions: the present invention provides a kind of embodiment based on the train wagon identification and ranging method of multi-sensor fusion, and described method comprises the following steps:

S1 dual sensors collect and process the data of the target area to obtain their respective standard data;

S2 builds a data fusion platform based on their respective standard data and obtains an effective target area;

S3 identifies and detects the target according to the effective target area;

S4 extracts the distance information of the target wagon in the effective target area, and transmits the obtained information about whether the target wagon is the target wagon and the corresponding distance information to the remote terminal controller in real time.

Referring to Fig. 1, in the preferred embodiment of this part, step S1 includes:

S11 The first sensor collects the image data of the front area in real time, and outputs the original image data; at the same time, the second sensor collects the radar data of the front area in real time, and outputs the original radar data;

S12 standardizes the image raw data and the radar raw data to obtain image standard data and radar standard data respectively, and executes step S2.

Specifically, the radar is a millimeter-wave radar, which uses a special radar technology of short-wave electromagnetic waves, and a special millimeter-wave radar technology of frequency modulated continuous wave (FMCW). By capturing the reflected FMCW signal, the radar system collects the front target in real time The distance, angle and relative speed and other data. The camera (camera sensor) selects a 1024×728 resolution RGB color camera to collect color images in real time. Step S12 is to preprocess the collected images, filter and correct the radar data.

Referring to Fig. 1, in the preferred embodiment of this part, step S12 includes:

S121 performing distortion correction on the original image data;

S122 encodes the distortion-corrected original image data;

S123 performs pixel processing on the encoded original image data, and then generates standard data, and executes step S2.

Referring to Fig. 1, in the preferred embodiment of this part, step S12 also includes:

S124 performs threshold filtering on the radar raw data;

S125 performs Kalman state estimation on the threshold-filtered raw radar data, and then generates radar standard data, and executes step S2.

Referring to Figures 1-2, in the preferred embodiment of this part, it includes: a first sensor installation calibration module, a second sensor installation calibration module, a joint calibration module and a time synchronization module, wherein,

The first sensor is equipped with a calibration module, which is used to install and calibrate the first sensor;

The second sensor is equipped with a calibration module, which is used to install and calibrate the second sensor;

The joint calibration module is used for joint calibration of the first sensor and the second sensor;

The time synchronization module is configured to perform time synchronization on the first sensor and the second sensor,

Include in step S2:

S21 installing and calibrating the first sensor;

S22 installing and calibrating the second sensor;

S23 performs joint calibration and time synchronization on the first sensor and the second sensor to obtain an effective target area.

Referring to Fig. 2, in the preferred embodiment of this part, the first sensor is a camera sensor.

Referring to Fig. 2, in the preferred embodiment of this part, the second sensor is a radar sensor.

Referring to Fig. 2, in the preferred embodiment of this part, the radar sensor is a millimeter-wave radar.

Referring to Fig. 1, in the preferred embodiment of this part, step S3 includes:

S31 training to obtain a deep neural network target detector;

S32: The effective target area is detected by the deep neural network target detector to detect the railroad track in the front area, and the effective measurement area containing the target wagon is obtained by clipping.

Referring to Fig. 1, in the preferred embodiment of this part, in step S4, specifically, the steps S1-S3 are packaged and integrated, and the distance information of the target wagon is extracted from the effective measurement area obtained by image recognition, and the The obtained information of whether the target wagon is available and the corresponding distance information are transmitted to the front end in real time.

Specifically, the above steps are packaged and integrated, and the distance information of the target wagon is extracted from the effective area obtained by image recognition by using the obtained radar information. The information of whether the target wagon is the target and the corresponding distance information are transmitted to the front end in real time to assist the staff to complete the wagon hooking work through the obtained data. Preferably, the millimeter wave radar adopts ARS408-217-77GHz millimeter wave radar.

The specific work process of this embodiment:

When the train is attached to the wagon, the driver is responsible for the train turning from the straight track to the branch road to be attached to the wagon and starting the related equipment involved in the method of the present invention. The color image sensor (camera sensor) collects the RGB image of the front area and sends it to the processor. After image preprocessing Afterwards, a deep neural network framework is built to identify rail tracks and narrow the radar monitoring range. Judging whether there is a target car in the front area or an obstacle in the viewing angle, if there is a target car, the millimeter-wave radar sensor collects the distance, speed and angle information of the rear of the target car in front and sends it to the processor, and then uses the information fusion platform to perform radar calibration, camera installation calibration, Joint calibration and time synchronization. After the final processing, the real-time distance information of the target wagon in front is output to the front end; if there is an obstacle in the viewing angle, the information is returned and the real-time distance between the current and the obstacle in front is calculated. To assist the operator to control the speed of the vehicle to realize the wagon coupling of rail transit.

Claims (10)

1. it is a kind of based on Multi-sensor Fusion train railway carriage identification and distance measuring method, which is characterized in that the method includes with Lower step:

S1 dual sensor is acquired and handles to target area data, obtains respective normal data；

S2 builds data fusion platform according to respective normal data, and obtains effective target region；

S3 is identified and is detected to target according to effective target region；

S4 extracts the range information of target railway carriage in effective target region, by obtain whether the information and correspondence of target railway carriage Range information real-time delivery to remote-terminal controller.

2. a kind of identification of train railway carriage and distance measuring method based on Multi-sensor Fusion according to claim 1, feature It is, includes: in step S1

S11 first sensor acquires front region image data in real time, and exports original image data, meanwhile, second sensor Acquisition front region radar data in real time, and export radar initial data；

S12 is standardized described image initial data and the radar initial data, respectively obtains graphics standard number According to radar normal data, execute step S2.

3. a kind of identification of train railway carriage and distance measuring method based on Multi-sensor Fusion according to claim 2, feature It is, includes: in step S12

S121 carries out distortion correction to original image data；

S122 carries out coded treatment to the original image data after distortion correction；

S123 carries out processes pixel to the original image data after carrying out coded treatment, then generates graphics standard data, holds Row step S2.

4. a kind of identification of train railway carriage and distance measuring method based on Multi-sensor Fusion according to claim 2, feature It is, in step S12 further include:

S124 carries out threshold filter to radar initial data；

S125 carries out Kalman estimator to by the radar initial data of threshold filter, then generates radar normal data, Execute step S2.

5. a kind of identification of train railway carriage and distance measuring method based on Multi-sensor Fusion according to claim 2, feature It is, comprising: it is same that first sensor installs demarcating module, second sensor installation demarcating module, combined calibrating module and time Walk module, wherein

The first sensor installs demarcating module, for the first sensor to be installed and demarcated；

The second sensor installs demarcating module, for the second sensor to be installed and demarcated；

The combined calibrating module, for carrying out combined calibrating to the first sensor and second sensor；

The time synchronization module, for carrying out time synchronization to the first sensor and second sensor,

Include: in step S2

S21 is installed and is demarcated to first sensor；

S22 is installed and is demarcated to second sensor；

S23 carries out combined calibrating and time synchronization to first sensor and second sensor, obtains effective target region.

6. a kind of identification of train railway carriage and distance measuring method based on Multi-sensor Fusion according to claim 5, feature It is, the first sensor is camera sensor.

7. a kind of identification of train railway carriage and distance measuring method based on Multi-sensor Fusion according to claim 5, feature It is, the second sensor is radar sensor.

8. a kind of identification of train railway carriage and distance measuring method based on Multi-sensor Fusion according to claim 7, feature It is, the radar sensor is millimetre-wave radar.

9. a kind of identification of train railway carriage and distance measuring method based on Multi-sensor Fusion according to claim 1, feature It is, includes: in step S3

S31 training obtains deep neural network object detector；

Front region rail is detected by deep neural network object detector in effective target region by S32, and reduction is contained The effective measuring area of target railway carriage.

10. a kind of identification of train railway carriage and distance measuring method based on Multi-sensor Fusion according to claim 1, feature It is, in step S4, specifically, step S1-S3 is packaged and is integrated, in the obtained effective measuring area of image recognition It is middle to extract the range information for completing target railway carriage, by obtain whether the information of target railway carriage is passed with corresponding range information in real time It is handed to remote-terminal controller.