CN106980855A - Traffic sign quickly recognizes alignment system and method - Google Patents

Abstract

The invention relates to a system and method for quickly identifying and locating traffic signs, comprising: a video stream acquisition module, the video stream acquisition module at least includes a high-definition camera and a positioning module, which are used to shoot videos along the road and locate the captured videos; traffic signs The recognition module, the traffic sign recognition module is used to recognize the traffic signs in the video along the road taken by the video stream acquisition module and record the positioning information; and the traffic sign map positioning module, the traffic sign map positioning module is used to use the traffic sign recognition module The recognized traffic signs are localized on the map. It also includes a data post-processing module. The data post-processing module is used to count the number of traffic signs along the road and their corresponding positions. The map output includes video stream key frame interception and an overview map of the position of traffic signs on the electronic map. The invention can quickly and accurately locate the traffic signs along the road, and provides a tool for traffic accident investigation.

Description

System and method for rapid identification and positioning of traffic signs

technical field

The invention relates to a traffic sign rapid identification and positioning system and method, belonging to the technical field of vehicle auxiliary driving.

Background technique

Road traffic accidents occur frequently in our country, and the fatality rate of road traffic accidents remains high. Data show that in 2014 alone, my country received a total of 6.76 million road traffic accidents and 196,812 road traffic accidents involving casualties, resulting in 58,523 deaths and direct property losses of 1.08 billion yuan. Among them, accidents on curves, ramps, and curved slopes combined road sections accounted for more than 20%, and the number of deaths accounted for about 30%. Traffic sign facilities on curved road sections are particularly important, which can remind drivers of road conditions and effectively reduce the number of accidents.

However, there are still some road traffic signs missing in some curved slope sections, and serious traffic accidents have been caused repeatedly. The investigation of the types and quantities of traffic signs along the accident road section is an important investigation content in the investigation of serious and extraordinarily serious traffic accidents. In recent years, relevant researchers have fully realized the importance of road traffic signs in road traffic safety, and have carried out a series of analyzes on the identification and positioning of traffic signs by means of image processing, remote sensing images, global positioning, and pattern recognition. with research.

Chinese patent application CN 102609702 A discloses a method and system for fast positioning of road guide signs. The system includes an acquisition unit, a segmentation unit, an area acquisition unit and a positioning unit. The method is to first obtain the road image, and then divide the obtained road image into upper and lower two parts of the image, and then use the blue detection model based on the RGB color model to obtain effective candidate areas for the upper part of the image, and finally to acquire The effective candidate area is used to detect horizontal long lines, and then locate the guide signs. Since this patent application uses straight line feature detection instead of traditional rectangular shape feature detection, the time for the process of locating guide signs can be greatly shortened, and the robustness is greatly improved. This patent application is widely used in the field of traffic sign recognition as a fast positioning method and system for road guide signs. However, this patent application is aimed at a single image rather than a video stream file. Manual operations are required to take pictures, and the type of traffic sign cannot be identified, only it can be identified as a traffic sign. In addition, the positioning accuracy of the application of image processing algorithms is not high, which is difficult to meet the actual needs.

Chinese patent application CN 105718860 A provides a positioning method and system based on a driving safety map and binocular traffic sign recognition, which uses a positioning system in a high-precision map to perform primary positioning on a driving vehicle; at the same time collects images in front of the vehicle, Detect and recognize the traffic signs in the image; recognize the coordinates of the traffic signs on the high-precision map, measure the distance between the vehicle and the sign, and calculate the position of the vehicle by comparing the coordinates of the traffic signs to realize vehicle positioning. On the basis of traditional navigation data, this patent application adds the collection of road traffic signs, uses road signs to assist the positioning of vehicles, and uses the coordinates and sizes of signs recognized by the left and right eye cameras to measure the distance between vehicles and traffic signs Calculate, and calculate the position of the vehicle according to the spatial position coordinates of traffic signs in the high-precision map, so as to provide sub-meter coordinate positioning and establish a topological network that can be based on lanes. The premise of this patent application is that the positioning coordinates of the traffic signs need to be known to confirm the coordinates of the vehicle, which is not the purpose of realizing the positioning of the traffic signs.

Chinese patent application CN 104361350 A provides a traffic sign recognition system, which is characterized in that: the recognition system includes installing a high-dynamic camera at the position of the rearview mirror of the vehicle, collecting traffic sign information on the road ahead of the road, and respectively analyzing the traffic sign information from the road environment Recognize traffic markings, traffic lights, and traffic signs in the image and establish a corresponding spatiotemporal correlation model; due to the above-mentioned structure and method, this patent application combines temporal and spatial relationships to establish a spatiotemporal correlation criterion for traffic sign recognition results, in the same image Recognize multiple traffic signs internally, integrate the recognition results of various traffic signs, obtain credible output results, and reduce the impact of traffic sign recognition errors on smart cars. The traffic sign recognition system is mainly used to build the space-related information of traffic signs and to direct the driving of smart cars. And the traffic sign can only identify its seven frames, but not the specific type.

The above three patent applications all involve traffic sign recognition, and patent application CN 102609702 A and patent application CN105718860 A also involve positioning. At present, camera-based traffic sign recognition is only suitable for single still photos, not for traffic sign recognition in dynamic video streaming files; in addition, current road traffic signs can only recognize the outline features of traffic signs, but cannot Accurately distinguish the types of traffic signs; finally, the positioning of traffic signs is not accurate. At this stage, further research is needed on the rapid identification and precise positioning of traffic signs to meet the needs of road traffic sign identification and positioning related to traffic accidents in my country.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies in the prior art and provide a system and method for quickly identifying and locating traffic signs, which can quickly and accurately identify traffic signs along the road and position them on the electronic map, with good Maneuverability and precision.

According to the technical solution provided by the present invention, the rapid identification and positioning system for traffic signs is characterized in that it includes:

A video stream acquisition module, the video stream acquisition module at least includes a high-definition camera and a positioning module, used to shoot video along the road and locate the captured video;

The traffic sign recognition module is used to recognize the traffic signs in the video along the road captured by the video stream acquisition module and record the positioning information;

And, the traffic sign map positioning module, the traffic sign map positioning module is used to position the traffic signs recognized by the traffic sign recognition module on the map.

Further, the data post-processing module is also included, and the data post-processing module is used for counting the number of traffic signs on the road section and exporting the location distribution map of the traffic signs.

Further, the video stream acquisition module adopts an electronic mobile device with a high-definition camera and a positioning module.

Further, the working process of the traffic sign recognition module includes traffic sign shape recognition in the motion camera, traffic sign content recognition, and classifying the recognized traffic signs.

Further, the process of traffic sign shape recognition in the moving camera is as follows: firstly, a sample image set is established according to the existing traffic signs, and samples of traffic signs in the sample images are extracted, a sample feature set is established, and traffic sign images are obtained through machine learning. Then decompose the video sequence image from the video stream, establish a set of scan window images, obtain the features of traffic signs from the video image, and obtain the feature vector in the image; then match the feature vector with the classification model for similarity, The detection result is obtained by judgment; at the same time, after the confirmed traffic sign feature vector is input into the classification model, the classification model feeds back the feature vector to the sample feature set, and continues to learn and optimize the classification model;

Further, the working process of the traffic sign map positioning module is: use the image frames of the recognized traffic signs captured at different positions in the video stream to reconstruct a measurable three-dimensional model of the traffic sign in the video, and calculate the The coordinates from the center position of the three-dimensional traffic sign to the camera coordinate system where each frame of image is located, combined with the positioning information of the above-mentioned captured image frame, can obtain the positioning information of the same traffic sign in each frame of image, based on the least squares algorithm, The positioning information is used to determine the final positioning information of the traffic sign.

The method for quickly identifying and locating traffic signs is characterized in that it comprises the following steps:

Step S1: taking a video along the road and locating the taken video;

Step S2: Decompose the video sequence obtained in step S1 into single frame images, and stack them, and record the positioning information corresponding to each stack image; take out the image at the top of the stack, extract the feature vector in the frame image, and compare it with the classification model Analyze the sample feature set in the frame to detect whether there is a traffic sign in the frame image, if not, discard the frame image, and take a frame image from the top of the stack to repeat the analysis; if there is, perform a new traffic sign together with the frame image stack, which is used to store images, and continue to take out new images from the top of the image stack for analysis until all frames containing the traffic sign are detected; you can obtain n frames of traffic sign images with local location information At the same time, classify the identified traffic signs according to the standards to provide data for subsequent data statistics; after completing the identification and classification of a traffic sign, the stack ID number used to store the detected traffic signs is increased by 1, and re-enters the next detection process;

Step S3: After completing the identification and classification of traffic signs, a three-dimensional model of the traffic sign can be constructed based on all n frames of images stored in the stack of the same traffic sign, and the location information of the n frames of images can be combined to obtain the traffic sign Location information; at this time, the traffic sign should have n pieces of location information, and the best location information of the traffic sign is obtained by using the least square method described above.

Furthermore, by combining the electronic map and the classified traffic sign categories, an electronic map containing the distribution of traffic signs can be output.

The advantage of the present invention is that: the present invention can quickly and accurately locate the traffic signs along the road, and locate them on the electronic map. It has good flexibility and operability. The invention comprehensively applies technical methods such as image recognition, feature detection, navigation positioning, multi-view three-dimensional positioning, etc., has good operability and precision, and can provide an effective tool for the investigation of road traffic sign facilities for major traffic accidents.

Description of drawings

Fig. 1 is a structural block diagram of the traffic sign rapid identification and positioning system of the present invention.

Fig. 2 is a working diagram of the traffic sign recognition module.

Fig. 3 is a working diagram of the traffic sign map positioning module.

Fig. 4 is a schematic diagram of an outputtable traffic sign distribution map.

Fig. 5 is a working flow chart of the system for quickly identifying and locating traffic signs according to the present invention.

detailed description

The present invention will be further described below in conjunction with specific drawings.

As shown in FIG. 1 , the traffic sign rapid identification and positioning system of the present invention includes a video stream acquisition module, a traffic sign recognition module, a traffic sign map positioning module and a data post-processing module.

The video stream collection module is based on the electronic mobile device, and the APP application used in conjunction with the electronic mobile device is installed, the APP application is started, the camera carried by the electronic mobile device is called on the APP interface, and the electronic mobile device is fixed on the vehicle. It can shoot the video stream along the road with the car, and record the location information of the machine at the time of recording each frame while shooting the video. The electronic mobile device can continuously and smoothly shoot video for at least 30 minutes, and the video quality is not affected by environmental factors such as heating and shaking of the electronic mobile device, so as to ensure that the video stream file is suitable for later analysis and processing.

The working process of the traffic sign recognition module includes traffic sign shape recognition in the moving camera, traffic sign content recognition, and classifying the recognized traffic signs. Since the captured video stream is a high-definition video, after the traffic signs in the mobile video are recognized, the content on the traffic signs in the image can be continuously identified, and the recognized traffic signs can be classified. For traffic signs whose content cannot be recognized, Coarse classification can be performed based on the outline shape of the recognized traffic signs. The classification basis of road traffic signs refers to "Road Traffic Signs and Markings Part 2: Road Traffic Signs (GB5768.2-2009)".

The traffic sign recognition module integrates an image processing algorithm, which can recognize the characteristic target in the moving camera, and the identified traffic signs are based on "Road Traffic Signs and Markings Part 2: Road Traffic Signs (GB5768.2-2009) "sort. The traffic sign recognition module adopts a moving object detection algorithm based on feature classification. The original moving object detection algorithm based on feature classification requires a fixed camera and a moving target to be detected. In the present invention, due to the known speed and trajectory of the camera, Therefore, the establishment of the camera coordinate system can be transformed into the establishment of the relative motion relationship between the camera and the traffic sign. In this coordinate system, the camera is relatively stationary, while the road and roadside features are relatively moving, so the moving object detection algorithm based on feature classification can be used. As shown in Figure 2, the moving target detection based on feature classification includes two processing processes, namely, the learning process and the decision-making process. The basic idea of the learning process is to select or construct an image feature that is beneficial to the target description of the type of attention, and map a set of marked image samples to the feature space to form a feature sample set through a feature extraction algorithm; then use the sample set as input, supervised training is performed on the corresponding pattern recognition classifier, and finally a trained detection classifier is obtained. The basic idea of the decision-making process is to first list all regions in the current image that may contain objects of the type of interest, then use the trained detection classifier to quantify the possibility of objects in these regions, and finally use the decision strategy to evaluate the output of the classifier, Realize the detection of the target. The two core points of moving object detection based on feature classification are image features and classification model, and the construction of classification model is closely related to the dimension of feature vector. In the present invention, traffic signs belong to image features with small dimensions, mainly including color histogram, color moment, HOG, LBP and other features, so the decision-making method of distance measurement can be used, that is, the intra-class distance calculation using the training target sample features The optimal linear decision threshold, and then by comparing the distance between the image feature to be decided and the average feature of the target sample, the detection of the target in the scene is realized.

After the traffic sign recognition is completed, the location information of the traffic sign needs to be determined. Objects in a two-dimensional image do not have positioning information, but in the present invention, when the camera shoots a video stream file, it records the local positioning information of the camera when each frame is shot. As shown in Figure 3, in the video stream, the same traffic sign is photographed from different positions (position1, position2, ..., positionn), and is recognized by the traffic sign recognition module of the present invention. At the same time, the n frames of images are all recorded The location information of this machine. Therefore, a three-dimensional reconstruction method based on multi-view can be adopted, based on the above n frames of images, a three-dimensional model of the traffic sign in the image is constructed, and based on position1, position2, ..., positionn and the three-dimensional model to position1, position2, ..., The distance of position n determines the positioning information of the traffic sign, and the least square method is used to determine the most accurate positioning information of the traffic sign from the n positioning information. The positioning formula is shown in formula 1.

In formula (1), P _sign is the positioning information of the traffic sign, and P _k is the corresponding positioning information of the traffic sign in the single frame image.

After the identification and positioning of traffic signs are completed, the next step of data analysis and processing can be carried out. One of the functions of the traffic sign rapid identification and positioning system of the present invention is to draw an electronic map of traffic sign distribution, so as to add the distribution information of traffic signs to the original electronic map. As shown in Figure 4, the base map of the traffic sign distribution electronic map is the electronic map of a certain highway, in conjunction with the traffic sign recognition function and the positioning function of the traffic sign rapid identification and positioning system of the present invention, this traffic sign is identified as "rain and fog" The weather slows down" warning sign is located on the K1386+700 section of the expressway. The traffic sign rapid identification and positioning system of the present invention can complete the distribution and display information of all traffic signs in the road section.

The working process of the traffic sign rapid identification and positioning system of the present invention is shown in Figure 5. Before starting work, the electronic mobile device needs to be fixed somewhere in the vehicle to ensure that the camera of the electronic mobile device is not blocked and can take clear and coherent pictures. For video along the road, check the battery life and memory of the electronic mobile device, and ensure that it can shoot and store more than 30 minutes of high-definition video. After the preparatory work is completed, start the electronic mobile device, open the special APP installed on the electronic mobile device, start the vehicle, and open the video sequence file for shooting along the road. After the video shooting is completed, the video file is exported to the workstation, which is integrated with image processing algorithms. First, the video sequence can be decomposed into single-frame images, and stacked, and the local positioning information corresponding to each stack of images can be recorded. Take out the image at the top of the stack, extract the feature vector in the frame image, and analyze it with the sample feature set in the classification model to detect whether there is a traffic sign in the frame image, if not, discard the frame image, and take it out from the stack top again Repeat the analysis of a frame image; if there is, make a new stack of the traffic sign together with the frame image, the stack is used to store the image, and continue to take out new images from the top of the image stack for analysis until all frames containing the traffic sign were detected. You can get n frames of traffic sign images with local positioning information, and classify the recognized traffic signs according to the standard to provide data for subsequent data statistics. After the identification and classification of a traffic sign is completed, the stack ID number used to store the detected traffic sign is increased by 1, and the next detection process is re-entered. After completing the recognition and classification of traffic signs, a three-dimensional model of the traffic sign can be constructed based on all n frames of images stored in the stack of the same traffic sign, and the positioning information of the traffic sign can be obtained by combining the positioning information of the n frames of images. At this time, the traffic sign should have n pieces of positioning information, and the best positioning information of the traffic sign is obtained by using the least square method described above. Combining the electronic map and the classified traffic sign categories, the electronic map containing the distribution of traffic signs can be output. At the same time, traffic signs can also be counted to analyze whether they comply with relevant road standards and specifications.

Claims (8)

1. a kind of traffic sign quickly recognizes alignment system, it is characterized in that, including：

Video flowing acquisition module, video flowing acquisition module at least includes high-definition camera and locating module, for shooting road edge Line video is simultaneously positioned to captured video；

Traffic Sign Recognition module, Traffic Sign Recognition module is used in roadside video captured by video flowing acquisition module Traffic Sign Recognition come out and record location information；

And, traffic sign map location module, traffic sign map location module is used to be known Traffic Sign Recognition module Other traffic sign is navigated on map.

2. traffic sign as claimed in claim 1 quickly recognizes alignment system, it is characterized in that：Also include Data Post mould Block, Data Post module is used to count road section traffic volume mark quantity, export traffic sign location map.

3. traffic sign as claimed in claim 1 or 2 quickly recognizes alignment system, it is characterized in that：The video flowing gathers mould Block uses the electronic mobile device with high-definition camera and locating module.

4. traffic sign as claimed in claim 1 or 2 quickly recognizes alignment system, it is characterized in that：The Traffic Sign Recognition The course of work of module includes the identification of traffic sign profile, traffic sign content recognition and the friendship to identification in moving camera Logical mark is classified.

5. traffic sign as claimed in claim 4 quickly recognizes alignment system, it is characterized in that：Traffic mark in the moving camera Will profile identification process be：Sample image set is set up according to existing traffic sign first, and extracted in sample image The sample of traffic sign, sets up sample feature set, and the disaggregated model of traffic sign is obtained by machine learning；Then from video Video sequence image is decomposed in stream, scanning window image collection is set up, the feature of traffic sign is obtained from video image, and obtain Characteristic vector into image；Characteristic vector and disaggregated model are then subjected to similarity mode, judgement draws testing result；Together When, the traffic sign characteristic vector having confirmed that is inputted after disaggregated model, and this feature vector feedback is given sample special by disaggregated model Collection is closed, and proceeds to learn Optimum Classification model.

6. traffic sign as claimed in claim 1 or 2 quickly recognizes alignment system, it is characterized in that：The traffic sign map The course of work of locating module is：The picture frame of the traffic sign identified photographed using diverse location in video flowing, Reconstruct the measurable threedimensional model of traffic sign in video, and calculate the center of the three-dimensional traffic mark to each frame figure As the coordinate under the camera coordinate system of place, with reference to the location information of above-mentioned photographing image frame, you can obtain each two field picture In same traffic sign location information, based on least-squares algorithm, determine that the traffic sign is final using these location informations Location information.

7. a kind of quick recognition positioning method of traffic sign, it is characterized in that, comprise the following steps：

Step S1：Shoot the video of roadside and captured video is positioned；

Step S2：The step S1 video sequences obtained are resolved into single-frame images, and carry out storehouse, each stack image pair is recorded The location information answered；Take out the image of stack top, extract the characteristic vector in the two field picture, and with the sample characteristics in disaggregated model Set is analyzed, and is detected in the two field picture whether there is traffic sign, if nothing, is abandoned the two field picture, takes out one from stack top again Two field picture replicate analysis；If so, the traffic sign to be carried out to new storehouse together with the two field picture, the stack is used for depositing image, and Continue to be analyzed from the new image of image stack top taking-up, until all frames containing the traffic sign are detected；It can obtain Obtain n frames and carry the traffic indication map of the machine location information, while will identify that the traffic sign establishing criteria come is classified, be Follow-up data statistics provides data；After the identification and classification of a traffic sign is completed, for depositing detection traffic sign The increase of stack ID 1, reenter next testing process；

Step S3：, can be based on n frames all in the stack for depositing same traffic sign after the identification and classification that complete traffic sign Image, builds the threedimensional model of the traffic sign, with reference to the location information of this n two field picture, can obtain the positioning of traffic sign Information；Now, the traffic sign should have n location information, using the least square method of foregoing description, obtain the traffic sign Best orientation information.

8. the quick recognition positioning method of traffic sign as claimed in claim 7, it is characterized in that：With reference to electronic map and divide Traffic sign classification, you can export containing traffic sign be distributed electronic map.