CN107491756B - Lane turning information recognition method based on traffic signs and ground signs - Google Patents

Abstract

The invention relates to a lane turning information recognition method based on traffic signs and ground signs, comprising: step 1, using a mobile survey vehicle to obtain image sequence data of the actual road; step 2, detecting and identifying traffic signs for each image, and obtaining the lane The turning information of the traffic signs in the lane; step 3, detect and identify the ground signs for each image, and obtain the turning information of the ground signs in the lane; step 4, perform image sequence analysis on the turning information recognition results of steps 2 and 3, Calculate the reliability of the turning information of traffic signs and ground signs respectively, and take the highest reliability as the respective recognition results; step 5, compare and verify the traffic sign turning recognition results and ground sign recognition results in the lane, and obtain the lane section Turn to information. The invention realizes the high-accuracy identification of lane turning information by analyzing the characteristics of the distribution of ground signs and traffic signs in actual roads and adopting a mutual confirmation mechanism.

Description

Lane turning information recognition method based on traffic signs and ground signs

technical field

The invention mainly relates to the field of intelligent transportation, especially for extracting steering information of lanes, by detecting traffic signs and ground turning signs in lanes, and verifying the two to improve the accuracy of information.

Background technique

The rapid development of automobiles provides convenience for people's daily travel, but it also brings various problems, such as traffic congestion and frequent traffic accidents. Therefore, the traffic sign recognition system has received extensive attention. The system can accurately transmit road information to the driver in real time, effectively helping the driver to predict possible dangers, so as to achieve safe driving. Due to the large variety of traffic signs, the large amount of information to be processed, and the existence of a large number of interference factors, such as lighting conditions, complex road conditions, etc., the image of the traffic sign to be recognized often has noise interference, so the traffic sign recognition algorithm is required. high accuracy.

At present, there are some studies on improving the accuracy of traffic sign recognition at home and abroad, but most of the research is only on traffic sign recognition, and the algorithm accuracy is continuously improved, and a small part of the research is on the recognition of ground signs. Jobson et al. used the Retinex algorithm to eliminate the influence of illumination, but the algorithm was very complex and the experimental results were mediocre; Teague proposed a method of using continuous orthogonal distances to construct invariant distances for feature extraction. There are two main types, Zernike distance and Legendre distance, but There are discrete errors in the calculation of such continuous invariant distances, which will lead to a large change in the value of invariant distances and a decrease in orthogonality performance, resulting in a decrease in recognition accuracy. Chen Fang proposed a road sign recognition method based on Hu invariant moments and contour projection. Jie Xuanhui designed a crosswalk detection algorithm based on bipolarity and feature fusion, and proposed a feature point search algorithm based on convolution filtering for stop line detection. Using this method has a low false detection rate and fast processing speed, but the design algorithm for specific signs lacks versatility.

However, as far as the existing algorithms are concerned, there is no mechanism for verifying and confirming the recognition results. According to the actual situation, roads with multiple lanes in the same direction are generally designed with separate driving lanes, each lane has its steering information, and there will be lane driving direction signs in the sky, that is, there are multiple signs on the actual road indicating the same information Case. However, existing studies have not made good use of multiple markers containing the same information.

Contents of the invention

In order to significantly improve the correct rate of identification and classification of traffic signs and make full use of multiple signs containing the same information, the invention discloses a lane turning information recognition method based on mutual verification of traffic signs and ground signs.

The technical solution of the present invention is a lane turning information recognition method based on mutual verification of traffic signs and ground signs, comprising the following steps,

Step 1, obtain the image sequence data of the actual road;

Step 2, the detection and recognition of traffic signs for each image, including the following sub-steps,

Step 2.1, perform HSV color space threshold segmentation on each image, and extract areas that match the color of the traffic sign;

Step 2.2, in the extracted region that conforms to the color of the traffic sign, perform shape detection, and extract the region that conforms to the shape of the traffic sign;

Step 2.3, perform hough straight line detection in the obtained traffic sign area, divide the obtained straight lines into different lanes, each lane corresponds to a turn sign, and extract the feature vector V1 of each turn sign;

Step 2.4, input the feature vector V1 into the support vector machine for recognition, and obtain the steering information of the traffic signs in the lane;

Step 3, perform detection and recognition of ground signs on each image, including the following sub-steps,

Step 3.1, performing gray scale, histogram equalization and scene reconstruction processing on each image;

Step 3.2, extract the lane lines in the image, obtain the area range of the lane, and keep the ground marks between the lanes;

Step 3.3, within the scope of each lane obtained in step 3.2, extract the feature vector V2 of the turn sign in each lane;

Step 3.4, input the feature vector V2 to the support vector machine for identification, and obtain the steering information of the ground signs in the lane;

Step 4: Carry out image sequence analysis on the recognition results of turning information in steps 2 and 3, respectively calculate the reliability of the turning information of traffic signs and ground signs, if the obtained reliability is less than the specified value, mark it as unreliable The information is left for re-inspection. If the obtained reliability meets the specified value, then the recognition result with the maximum reliability will be correspondingly output;

Step 5, compare and verify the recognition results of traffic signs in the lane and the recognition results of ground signs, and obtain the steering information of the lane section. The implementation method is as follows,

First judge whether the same lane contains the recognition results of traffic signs and ground signs at the same time. If so, then according to the prior knowledge, determine the weights of the recognition results of traffic signs and ground signs respectively, and then execute (1) and (2), if not , then execute (3);

(1) If the traffic sign recognition result is the same as the ground sign recognition result, use any one of the recognition results as the turning information of the lane;

(2) If there is a deviation between the traffic sign recognition result and the ground sign recognition result, the recognition result with a larger weight is used as the steering information of the lane;

(3) If there are only traffic sign recognition results or ground sign recognition results in the same lane, the recognition results are used as the steering information of the lane.

Further, in the step 1, the image sequence data of the actual road is obtained by moving the measuring vehicle.

Further, the shape detection in the step 2.2 is a rectangle detection, and the implementation of extracting the area conforming to the shape of the traffic sign is as follows,

For a graphic, the circumscribed rectangle can be found. L and W represent the length and width of the circumscribed rectangle, Tag represents the degree of rectangle Tag=S/(L*W), and S represents the number of pixels of a certain graph after segmentation, that is, the graph The real size of ; when the range of the rectangularity Tag is within [0.8,1.4], the figure is determined to be a rectangle.

Further, in the step 2.3, the radial Tchebichef invariant moment is used to extract the feature vector V1 of each turn sign.

Further, in the step 3.3, the normalized Fourier descriptor is used to extract the feature vector V2 of each turn sign.

Further, the prescribed value in step 4 is 80%.

By analyzing the characteristics of the distribution of ground signs and traffic signs in actual roads, and adopting a mutual confirmation mechanism, the present invention realizes high-accuracy identification of lane steering information, provides support for safe driving, and can serve for the production of high-precision maps, making no Human driving becomes possible.

Description of drawings

Fig. 1 is a flowchart of an embodiment of the present invention.

Fig. 2 is a flow chart of mutual checking in the embodiment of the present invention.

Detailed ways

The technical scheme of the present invention will be analyzed and described in detail below in conjunction with the accompanying drawings and the embodiments of the present invention. The implementation process of the embodiment can be summarized as the following steps:

Step 1: first obtain the image data of the actual road, use the mobile measurement vehicle for data collection, and obtain the corresponding image sequence data.

Here, a mobile survey vehicle is used to continuously collect image information including traffic signs and ground signs on urban roads. The car is equipped with a GPS system, so it is possible to obtain the track information of the measuring car. The road can be divided into different road sections according to the road intersection, and the images can be classified according to the road section according to the shooting time information. Because the car is equipped with GPS, there will be a reminder at the intersection, so you can use it at this time The image is marked, and the image taken on a section of the road is used as a lane segment before the next intersection appears. Turning traffic signs are mostly located near intersections, so image sequences that are more likely to appear turning traffic signs can be obtained based on this, for subsequent detection and identification. The steering information is based on the lane, and the existing lane line detection method can be used to obtain the number of lanes and the lane boundary.

Step 2: Carry out detection and recognition of traffic signs for each image, including the following sub-steps,

Step 2.1, color segmentation, based on its color (usually red, blue, yellow, etc.) characteristics, the part that may be a traffic sign is extracted from the entire image; the HSV color system is closer to people's experience and The perception of color is more suitable for color segmentation of color images in natural scenes. In the embodiment of the present invention, color threshold segmentation is performed after converting the RGB color space to the HSV color space, and the conversion formula is as follows,

where MAX=MAX(R,G,B),MIN=MIN(R,G,B)

Table 1 HSV color segmentation threshold table

During specific implementation, those skilled in the art can set the HSV color segmentation threshold by themselves. After image segmentation, multiple connected regions will be generated, each region has its own color, and the colors of traffic signs are usually red, blue and yellow, so as to extract the parts that match the color of traffic signs.

Step 2.2, shape detection, based on the fact that traffic signs usually have a specific shape (circle, triangle, rectangle) and size, in the extracted areas that match the color, filter out those areas that do not match the shape (that is, except for circles, triangles) , other shapes other than rectangles) to improve recognition efficiency; in this embodiment, rectangle detection is mainly carried out, and the area of the connected domain is calculated according to the number of pixels in the detected connected domain, and then its rectangularity is obtained, if in Within the appropriate threshold, it is judged as a rectangle.

For a graphic, the circumscribed rectangle can be found. L, W represent the length and width of its circumscribed rectangle, Tag represents the degree of rectangle Tag=S/(L*W), where S represents the number of pixels of a certain graph after segmentation, That is, the actual size of the graphic, those skilled in the art can set the threshold of rectangle degree by themselves to extract the rectangular traffic sign.

Step 2.3, feature extraction, in the range obtained in step 2.2, divide it into different regions according to the dotted lines of the signs, extract feature vectors for each region, and use them for classification and recognition of traffic signs;

Perform hough straight line detection in the obtained rectangular range, and divide different turning signs according to the obtained straight lines. For example, if four straight lines are found in the rectangular range, then divide the graph into three regions according to the four straight lines (each region is is a lane), every two adjacent straight lines define an area, and there is a turn sign in each area. When extracting the feature vectors of each sign, the embodiment of the present invention will use the radial Tchebichef invariant moments proposed by Mukundan et al. For N*N images:

In the formula, n is the maximum number of pixels on the circumference, m=N/2, k=0,1,2...n-1. Thus, the radial Tchebichef invariant moments are as follows:

In the formula, are the real and imaginary parts of _spq , respectively.

Four radial Tchebichef invariant moments of η ₁₁ , η ₂₂ , η ₄₂ , and η ₄₄ are used to extract the feature vectors of traffic sign images and use them for classification and recognition of traffic signs.

Step 2.4, classification recognition, the embodiment of the present invention will use a support vector machine (SVM) for recognition, input samples in the early stage for training, obtain a classifier with better performance, input the feature vector in step 2.3, and output the category to which it belongs. After the samples have been trained, put the steering information in each lane into the trained classifier for testing, and the corresponding steering information will be output. According to the dotted line division results and lane information, the recognition results of each area are assigned to lanes.

Step 3: Perform detection and recognition of ground signs on each image, specifically including the following sub-steps,

Step 3.1, preprocessing, including graying to reduce the amount of computation, filtering to eliminate noise, histogram equalization to further enhance the contrast between the sign and the background, and scene reconstruction to restore the shape features of the ground sign;

In order to strengthen the difference between the logo and the background, the image is grayscaled and then histogram equalization is used to enhance the contrast. At the same time, due to the unknown and unstable camera parameters and the influence of the camera's own speed and pitch angle during the shooting process, there are large distortions in the imaging process, the most important of which is perspective distortion. The distortion leads to changes in the shape characteristics of road traffic signs, which cause changes in the characteristics of the target image and affect the accuracy of ground sign recognition. Therefore, scene reconstruction of the original image is essential. The inverse perspective transformation is essentially to transform the road image in the image coordinate system into the plane in the car body coordinate system. After a simple coordinate system transformation, it can be obtained. The road image coordinate value in the image coordinate system is converted into the car body coordinates The calculation formula of the actual physical distance in the vehicle body coordinate system can be calculated by the conversion formula, and the coordinate value (actual width and height distance) of the corresponding point on the plane of the vehicle body coordinate system can be obtained.

The model of inverse perspective transformation is as follows,

Among them, c ₁ =cosα, c ₂ =cosβ, s ₁ =sinα, s ₂ =sinβ, α is the pitch angle, β is the yaw angle, (u,v) is the coordinate in the image coordinate system, (X,Y , Z) are the coordinates in the WGS-84 coordinate system, (f _u , f _v ) are the lengths of the horizontal and vertical focal lengths, and (c _u , c _v ) are the coordinates of the principal point of the image. h is the height from the center of the camera to the ground.

In practical application, the internal and external parameters of the camera can be determined by the checkerboard calibration method, and then the coordinate conversion can be completed. The original image can be reversed and converted into a top view.

Step 3.2, lane line extraction and road area division, including segmenting the lane line from the road image to remove non-target objects and interference information in the road image, and obtain the area range of the designated lane.

Segment the lane line from the road image, the embodiment of the present invention adopts the Otsu maximum inter-class variance method to carry out binarization, and remove the small connected domains after binarization (those with only 1 to 20 pixel values can be regarded as small connected domains). area). Then use the position constraints and area constraints that the ground turning signs should be located between the lanes (the turning signs all have a fixed size), remove other objects in the binary image, and keep the ground signs.

Step 3.3, feature extraction, including dividing the range obtained in step 3.2 into different areas according to the lane lines, extracting feature vectors for each area, and using them for classification and recognition of ground signs;

The range obtained in step 3.2 is divided into different regions according to the lane lines, and feature vectors are extracted for each region. In the embodiment of the present invention, Fourier descriptors are selected for feature extraction. The basic idea of the Fourier descriptor: Assume that the image contour to be processed is a closed curve, and each point P(t) on the curve has a corresponding coordinate area (x(t), y(t)), t=0, 1,2,…N-1, where N is the total number of points on the contour. The coordinate sequence is expressed as a complex number form x+yi, which is regarded as a periodic function whose period is the total length of the curve, and a series of coefficients are obtained through discrete Fourier transform, which is the Fourier descriptor, in which the high-frequency coefficients reflect the details of the contour , the low-frequency coefficients reflect the overall shape of the profile. The calculation formula of the Fourier descriptor is as follows, where:

According to the nature of the Fourier transform, the Fourier descriptor is related to the scale, direction and starting point of the curve, that is, the scale of the graph is different, the starting position of the contour is different, and the obtained Fourier descriptor is also different. To ensure the scale, rotation and translation invariance of the method, a normalization method is adopted. The normalized Fourier descriptor expression is:

By comparing the value with the first-level modulus, the normalized Fourier descriptor eliminates the influence of the starting point and ensures invariance. The calculation is simple, no control parameters need to be set, and the stability of the feature is high. Most importantly, Fourier descriptors within a certain frequency range can be selected as shape features for shape recognition according to requirements.

Step 3.4, classification recognition, the embodiment of the present invention also uses SVM for recognition here, after classification, the ground sign recognition result can be obtained, combined with the obtained specific lanes, the recognition information can be assigned to specific lanes.

Step 4: Carry out sequence analysis on the identification results of Step 2 and Step 3, respectively calculate the reliability of the turning information of ground signs and traffic signs, if the obtained reliability is less than the specified value (usually 80% or higher ), it will be marked as unreliable information and will be reserved for re-inspection. If the obtained reliability meets the specified value, the recognition result with the maximum reliability will be correspondingly output.

The extracted image sequence contains repeated turning information. For example, for the same right-turn lane, there may actually be three ground right-turn signs distributed at a certain interval on the road, and the same sign here may be captured by five consecutive frames of images. In this way, a total of 3*5=15 frames of images contain the same ground marker turning information. However, in actual recognition, the recognition results of these 15 frames may not be completely consistent due to factors such as noise interference or poor shooting conditions. In order to make the best use of the information in these 15 frames of images, sequence analysis is required. Here, the strategy of accumulating reliability and selecting the one with the highest reliability as the result is adopted. That is, the reliability is proportional to the frequency of occurrence. For example, in the recognition results of these 15 frames, 12 frames are turning right, 2 frames are going straight, and 1 frame is turning left, and the right turn with a reliability of 12/15 is selected as the result. At this time, the ground turning sign is considered to be a right turn. . Similarly, if the same turn sign on each image appears once on the traffic sign and is captured by 5 consecutive frames of images, a total of 1*5=5 frames of images contain the turn information of the traffic sign, of which 3 turn right and once Turn left and go straight once, then 3/5 right turns are the recognition results of traffic signs. Since the reliability of the ground sign recognition result is 12/15=80% meeting the specified value, and the traffic sign recognition result is 3/5=60% less than the specified value, only the ground sign recognition result is output.

Step 5: Compare and verify the recognition results of the traffic signs and ground signs, and obtain the turning information of the lane segment. It includes first judging whether the recognition results of traffic signs and ground signs are included in the same lane at the same time. If so, then according to the prior knowledge, the weights of the recognition results of traffic signs and ground signs are respectively determined, and then (1) and (2) are executed. If No, then perform (3); in the present embodiment, if think that ground sign recognition result is more accurate, then give the weight of ground sign recognition result to be 0.7, so the weight of traffic sign recognition result is 0.3,

(1) If the traffic sign recognition result is the same as the ground sign recognition result (that is, the same turning information), then use any one of the recognition results as the turning information of the lane; for example, the traffic sign recognition result and the ground sign recognition result are both turn right , and the reliability meets the specified value, then the steering information of this lane segment is right turn.

(2) If there is a discrepancy between the traffic sign recognition result and the ground sign recognition result, the recognition result with a larger weight is used as the steering information of the lane; for example, the ground sign recognition result is right turn (0.7), and the traffic sign recognition result is left Turn (0.3), then the steering information of this lane segment is right turn.

(3) If there are only traffic sign recognition results or ground sign recognition results in the same lane segment, this recognition result is used as the turning information of the lane; for example, the ground sign recognition result is right turn, and the traffic sign recognition result is empty, then the The steering information of the lane is right turn.

The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which the present invention belongs can make various modifications or supplements to the described specific embodiments or adopt similar methods to replace them, but they will not deviate from the spirit of the present invention or go beyond the definition of the appended claims range.

Claims (6)

1. the lane direction information recognition methods based on traffic sign and surface mark, which comprises the following steps:

Step 1, the image sequence data of real road are obtained；

Step 2, the detection for carrying out traffic sign to each image identifies, including following sub-step,

Step 2.1, the segmentation of hsv color capacity-threshold is carried out to each image, extracts the region for meeting traffic sign color；

Step 2.2, in the region for meeting traffic sign color extracted, SHAPE DETECTION is carried out, traffic mark will be met The extracted region of board shape comes out；

Step 2.3, hough straight-line detection is carried out in the traffic sign regional scope of acquisition, is divided into according to obtained straight line Different lanes are corresponding with a turn marking in each lane, extract the feature vector V1 of each turn marking；

Step 2.4, feature vector V1 is input to support vector machines to identify, obtains the steering letter of traffic sign in lane Breath；

Step 3, the detection for carrying out surface mark to each image identifies, including following sub-step,

Step 3.1, the processing of gray processing, histogram equalization and scene rebuilding is carried out to each image；

Step 3.2, the lane line in image is extracted, obtains the regional scope in lane, and retain the surface mark between lane；

Step 3.3, within the scope of each lane that step 3.2 obtains, the feature vector V2 of the turn marking in each lane is extracted；

Step 3.4, feature vector V2 is input to support vector machines to identify, obtains the steering letter of surface mark in lane Breath；

Step 4, image sequence analysis is carried out to the direction information recognition result of step 2 and step 3, calculates separately out traffic sign It is labeled as unreliable information, is remained if obtained confidence level is respectively less than specified value with the confidence level of surface mark direction information Reinspection, if obtained confidence level meets specified value, corresponding output has the recognition result of maximum confidence；

Step 5, traffic sign recognition result and surface mark recognition result in lane are veritified in comparison, obtain the steering of the roadway segment Information, implementation is as follows,

First determine whether in the same lane whether and meanwhile include traffic sign and surface mark recognition result, if so, according to Priori knowledge determines the weight of traffic sign and surface mark recognition result respectively, then (1) and (2) is executed, if it is not, then holding Row (3)；

(1) if traffic sign recognition result is identical with surface mark recognition result, using any recognition result as the vehicle The direction information in road；

(2) if there are deviations for traffic sign recognition result and surface mark recognition result, using the identification with greater weight As a result the direction information as the lane；

(3) if only existing traffic sign recognition result or surface mark recognition result in the same lane, by this recognition result Direction information as the lane.

2. the lane direction information recognition methods based on traffic sign and surface mark, feature exist as described in claim 1 In: the image sequence data of real road are obtained in the step 1 by traverse measurement vehicle.

3. the lane direction information recognition methods based on traffic sign and surface mark, feature exist as claimed in claim 2 In: SHAPE DETECTION is hough transform in the step 2.2, the implementation that the extracted region for meeting traffic sign shape is come out It is as follows,

The length and width of its boundary rectangle can be indicated a figure in the hope of boundary rectangle, L, W, and Tag indicates rectangular degree Tag=S/ (L*W), after S indicates segmentation, the number of the pixel of some figure, the i.e. actual size of figure；When the range of rectangular degree Tag exists In [0.8,1.4], determine the figure for rectangle.

4. the lane direction information recognition methods based on traffic sign and surface mark, feature exist as claimed in claim 3 In: using radial direction Tchebichef, bending moment does not extract the feature vector V1 of each turn marking in the step 2.3.

5. the lane direction information recognition methods based on traffic sign and surface mark, feature exist as claimed in claim 4 In: the feature vector V2 of the turn marking in each lane is extracted in the step 3.3 using normalized Fourier descriptor.

6. the lane direction information recognition methods based on traffic sign and surface mark, feature exist as described in claim 1 In: the specified value in the step 4 is 80%.