CN101872416B

CN101872416B - Vehicle license plate recognition method and system of road image

Info

Publication number: CN101872416B
Application number: CN 201010166981
Authority: CN
Inventors: 金城; 王琰滨; 冯瑞; 薛向阳
Original assignee: Fudan University
Current assignee: Fudan University
Priority date: 2010-05-06
Filing date: 2010-05-06
Publication date: 2013-05-01
Anticipated expiration: 2030-05-06
Also published as: CN101872416A

Abstract

A method and system for license plate recognition are described. The method can provide support for intelligent traffic systems, and can be widely used in parking lots, toll checkpoints, criminal arrests, and the like. The method mainly includes three stages of positioning, character segmentation and character recognition. It has a high recognition rate, can cope with more complex environments and can meet the requirements of real-time processing. The system encapsulates the above methods and provides different parameters for flexible use. Easy to distribute and process data in batches.

Description

Road image is carried out the method and system of car plate identification

Technical field

The invention belongs to Digital Image Processing and mode identification technology, be specifically related to a kind of method and system that road image is carried out car plate identification.

Background technology

In recent years, develop rapidly along with computing machine and Internet technology, the rapid growth of various vehicles numbers, various information comprises that the information relevant with traffic presents the situation of explosive growth, in order to manage safer, efficiently these information, intelligent transportation system (Intelligence Transportation System) is arisen at the historic moment.Intelligent transportation system can be at the charge bayonet socket, the parking lot, and the aspects such as criminal tracking show powerful effect, and are its most ingredients of core to the detection and Identification of car plate.

At present, although a lot of relatively ripe car plates identification products have been arranged now, higher accuracy rate, still less consuming time still attracting people to go constantly to study.In fact, along with the new development of association area, for example invention of new low-level image feature, the proposition of better sorting algorithm etc., all the improvement for Recognition Algorithm of License Plate provides new chance.In addition, commercial Vehicle License Plate Recognition System is mostly only to there being reasonable effect under the specific condition now, for example specific illumination, distance, angle, which system is car plate standard (comprising color, form, literal), and the accuracy rate of system may reduce even lose efficacy greatly under the environment that has changed also do not have to accomplish healthy and strong stable (identify the ability of car plate also there is a big difference with the people) under various condition.Third part at this paper can be discussed specially to one piece of article about this respect.Have, for picture and video that natural conditions are taken, for example hand-held or vehicle mounted camera shooting gets off again, and carrying out the car plate detection also is a direction that is worth research with identification.

In some license plate recognition technologies that exist at present, mostly whole identification is divided into three processes, as shown in Figure 1, is b car plate location, c License Plate Character Segmentation, d Recognition of License Plate Characters.

Wherein for b, the technology of use is of a great variety, although can be divided into several classes, the boundary between the class is not clearly.The following method is roughly arranged: binaryzation after the A rim detection, this is maximum a kind of method of using, it just can access reasonable just result after Mathematical Morphology Method is combined.An edge detection operator that is in daily use is vertical sobel operator.Its calculating formula is

And longitudinal edge detects compared to laterally, and its advantage, and longitudinal edge detects compared to laterally, its advantage are that the transverse edge of car in the image that contains car plate is more, and the longitudinal edge of character is more on the car plate.This method is calculated fast, effect is better, but very large shortcoming is exactly to be difficult to process complicated image, how effectively to remove incoherent marginal information, (vehicle intake mouth for example, the light zone, trees on every side, coarse ground etc.), be a very crucial problem .B mathematical morphology operation, mainly be that burn into expands opening and closing operation etc.C connected component analysis (CCA), most typical is four connected sums, eight connection methods and various clustering method, identical with the connected region purpose, tells some candidate regions.The D block analysis.Image is divided into some, calculates respectively the features such as its average, variance, marginal information.Moving window.Similar with section thinking, but be the node-by-node algorithm feature.The E Color Image Processing is according to the RGB colouring information.The various sorters of F comprise Ada-Boost, SVM, ANN, GP, GA etc.

For c, present technology is divided into following a few class.A binary Images Processing projecting method.Be maximum a kind of technology of using in the current various document, common way is to carry out first transverse projection, cuts off up and down zone; Then carry out longitudinal projection, be syncopated as each character.B local auto-adaptive binaryzation.Local auto-adaptive binaryzation or similar method in a lot of articles, it calculates average in a certain zone by piecemeal, pointwise or minute character, and then the features such as contrast carry out respectively binaryzation.Go out the C sloped correcting method.Kind is many, have to utilize HT positioning licence plate frame, also have to use colouring information, also has histogram analysis.The D level is cut apart and merging, splitting method.The E mathematical morphology, the burn into dilation operation.This step of License Plate Character Segmentation in fact difficulty be larger, because if cut apart the error or the two-value effect bad, the character recognition of back probably will be lost efficacy.And different illumination conditions, approximate color or shape around the car plate, different car plate standards all can restrict the method for License Plate Character Segmentation, so that be difficult at present produce extremely healthy and strong cutting techniques.Majority method all also just shows good performance under specific circumstances.

For d, OCR (optical character identification) is an important branch of area of pattern recognition, and its target is that the various literal with image format are identified as under the textual form.Recognition of License Plate Characters is a kind of special shape of OCR, and the process of Recognition of License Plate Characters can be reduced to feature extraction and characteristic matching.Method for the feature extraction of character picture is varied, has by pixel characteristic extraction method, framework characteristic extraction method, vertical orientation data statistical nature extraction method, a lot of based on the feature extraction of grid, radian Gradient Features extraction method etc.Do not carry out in addition feature extraction, in other words with the black and white values of character picture directly as feature, rely on the powerful classification capacity of sorter to identify, also be a kind of mode that can adopt.And the method for characteristic matching mainly is divided into following three kinds of A. based on statistics/mixing/with different levels sorter B.ANN.C. template matching method. wherein ANN is the most common and performance is more excellent.

Summary of the invention

The object of the present invention is to provide a kind of method and system that still image is carried out car plate identification, be intended to solve the key problem in the intelligent transportation system (ATI), obtain the number-plate number in the monitoring image, for the deeper application in back is prepared.

The method of described car plate identification provided by the invention comprises the car plate location, three steps of License Plate Character Segmentation and Recognition of License Plate Characters;

Described car plate location is to the Image Segmentation Using of input, obtains one group of car plate candidate regions.It is input as a pictures, then size carries out pre-service arbitrarily, based on two steps of cluster of DBSCAN, exports one group of car plate candidate regions;

The task of described preprocessing process is that original color image is processed, and generates a bianry image that comprises marginal information.It comprises image gray processing, and sobel longitudinal edge detection+car plate color strengthens, three steps of image binaryzation.

Described sobel longitudinal edge detection+car plate color strengthens, full figure to be carried out sobel vertically detect, in testing process, for marginal information p (x, y), if＞K, x*x neighborhood around it is scanned, to every delegation wherein, occur if any the car plate color, then strengthen p (x, y) 10%; K=30 among the embodiment, x=7; The color set that the car plate color here only may occur in the various car plates.

Described cluster based on DBSCAN is to use DBSCAN Density Clustering method that the two-value picture is carried out cluster, all points are divided into several high density areas, the zone that surpasses certain threshold value T is called candidate regions, and threshold value T determines by debugging according to the scene size of reality.Here critical radius is divided into RH and RW, the radius on the expression length and width direction, general RW=3*RH; Get RH=10 among the embodiment, RW=30, T=300; Then calculate the attribute of regional, comprise length and width, length breadth ratio etc. are deleted the very few point in regional both sides.At last the attribute of candidate regions is judged, got rid of the zone that does not conform to shape, export possible candidate regions.

Described License Plate Character Segmentation is that the car plate candidate regions is carried out cutting apart of character respectively, and the deletion error candidate regions obtains one group of character block to each other candidate.It comprises the secondary pre-service and based on two steps of cutting apart of projection;

Described secondary preprocessing process is first regional picture to be carried out gray processing, then utilizes the average threshold value with its binaryzation, and at last by horizontal projection, low ebb is regional up and down in the searching perspective view, thinks that it is useless region, deletes it.Obtain two-value car plate topography.

Described based on the cutting apart of projection, it be input as two-value car plate topography, output is one group of image character block, is generally 7 (number of characters on the car plate).At first carry out vertical projection, think that the trough district is the dead sector, other is character area, marks off some zones with this.Then analyze judgement for the zone, to get rid of non-license plate area.Blank up and down to each character picture excision more at last.Generate one group of character zone.

Described Recognition of License Plate Characters is that every group of character block identified, and the debug candidate regions generates the number-plate number, car plate color and the license plate area coordinate that identify.It is input as single two-value zone, input is the character that identifies, and comprises Chinese character, numeral and alphabetical.If the ratio of width to height＞4 in zone at first, dot density is greater than a certain threshold value in its zone, and then this character of Direct Recognition is 1.Then with whole bianry image sequence as feature, bring separately sorter into and classify Output rusults.If the character of identification is not 7, think that then it is non-license plate area.The final number-plate number, car plate color and the license plate area coordinate that identifies that generate.

Described sorter is characterized in that, respectively the two-value template is set up in numeral, character and Chinese character in advance, adopts the mode of template matches to identify during classification, the output recognition result.

The present invention also provides a kind of system of car plate identification, comprises three modules that realize three steps of described licence plate recognition method: car plate locating module, License Plate Character Segmentation module and Recognition of License Plate Characters module.Input is a pictures, and input is that some subregion pictures and recognition result are described document xml file.Can there be many kinds of parameters to select, function with dynamic-configuration parameter, function with Dynamic Definition output, can export the local picture of car plate, the local picture of position of driver, the local picture of car mark, the local picture of vehicle body, thumbnail etc., can conveniently replace licence plate recognition method, be easy to carry out batch processing.

Described many kinds of parameters is selected to refer to add parameter behind program name, plate, and other, all can only identify respectively car plate output number and position, and only calculating is exported other local pictures (by the car plate location estimating) and is all processed.Realized separating of critical process (car plate identification) and other processes.

The local picture of described calculating output position of driver according to car plate position and vehicle size, is estimated position of driver, and is compared with original image, the local picture of output position of driver after proofreading and correct.

The local picture of described calculating output car mark according to the car plate position, scans the car plate upper area, seeks the car mark, the local picture in output car plate position after cutting apart.

The local picture of described calculating output vehicle body according to car plate position and vehicle size, is estimated the vehicle body position, and is compared with original image, the local picture of output vehicle body after proofreading and correct.

Description of drawings

Fig. 1 is car plate identification process figure.

Fig. 2 is car plate locating module process flow diagram.

Fig. 3 is License Plate Character Segmentation module process flow diagram.

Fig. 4 is the enforcement illustration that a pictures is identified.

Fig. 5 is the process flow diagram of Vehicle License Plate Recognition System.

Embodiment

In order to make purpose of the present invention, technical scheme and advantage clearer, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein only in order to explain the present invention, is not intended to limit the present invention.

Fig. 1 shows the main flow process of licence plate recognition method.Comprise the car plate location, License Plate Character Segmentation and Recognition of License Plate Characters three phases.The value of intermediate transfer is 1 or a plurality of license plate candidate area, and in the middle of b, c, d operational process successively, the candidate regions of mistake is progressively abandoned.The final output number-plate number.

Fig. 2 shows car plate locating module flow process.For an input original image, for example shown in Fig. 4 upper left corner.At first it is carried out gray processing, the formula of gray processing is gray=0.299R+0.587G+0.114B, and wherein gray represents gray-scale value, and R, G, B are respectively three components of image.Next carries out vertical sobel rim detection, in the middle of vertical testing process, carries out the enhancing to the car plate color, concrete rule is (for example blue), for marginal information p (x, y), if＞30,7*7 neighborhood around it is scanned, to every delegation wherein, occur if any blue dot, then strengthen p (x, y) 10%, here blueness refers to (G＜150, R＜150, B＞G, B＞R).Again carry out binaryzation, obtain a width of cloth black and white point diagram, shown in the upper right corner among Fig. 4, be not difficult to find out, near the point the car plate is than comparatively dense, and this is the basis of next step cluster.

Adopt afterwards DBSCAN Density Clustering algorithm to obtain one group of candidate regions, part shown in the dotted line is the concrete steps of clustering algorithm among Fig. 2.Its general idea is exactly for the higher zone of density ratio, thinks that they belong to the same area.In this embodiment, lateral radius RH value is 10, and vertically radius R W value is 30, and density threshold gets 250.Carry out afterwards aftertreatment, delete white space twice, get rid of the zone that does not conform to shape, in this embodiment, think that the license plate area shape should satisfy: 2.5＜length breadth ratio＜7,80＜length＜220,20＜wide＜60.So far, positioning stage finishes.Among Fig. 4, a right side central figure on the upper side can find out, has been divided into out 5 zones, and these 5 zones are candidate regions.Middle part picture on the lower side shows the topography of that candidate regions of car plate.

Fig. 3 shows License Plate Character Segmentation module flow process.Each candidate regions for input carries out first a series of pre-service, comprises gray processing, binaryzation and horizontal projection.That bottom central authorities is the result of binaryzation among Fig. 4, and its left side corresponding horizontal projection that is it, is it above it through the result after the cutting between the dead sector up and down.Then carry out specifically comprising vertical projection, telling character area and clear area based on the cutting apart of projection; Regional analysis, superseded false candidates; Single character is frittered minute three processes.The result has been syncopated as each character by several vertical white lines shown in the little figure on the lower side of Fig. 4 central authorities.Bring afterwards the identification engine into and carry out the identification of literal, the output number-plate number BPG016 of Soviet Union.Shown in the little figure of Fig. 4 left side central portion.

Fig. 5 shows the cardinal principle flow process of Vehicle License Plate Recognition System.It supports three kinds of different use-patterns, is respectively parameter and is all, printenv or parameter other but without the situation of follow-up coordinate; The situation of parameter p late; Parameter other back adds the situation of two groups of coordinates.When parameter is all, this system will carry out all operations were, locate first and identify car plate, then calculate various regional area pictures by it, at last output.Parameter p late represents only to identify car plate and output, and parameter other represents only to calculate various regional area pictures and output.The regional area here comprises, driver zone, car mark zone and vehicle body zone.These regional area pictures will provide certain help for other subsequent applications of intelligent transportation system.Also comprise an xml document in the last output, describe the result of identification.

Claims

1. A method for carrying out license plate recognition to road image, is characterized in that, comprises license plate location, license plate character segmentation and license plate character recognition three steps;

The license plate location is to segment the input image to obtain a group of license plate candidate areas;

The character segmentation of the license plate is to segment the characters of the license plate candidate area respectively, delete the wrong candidate area, and obtain a group of character blocks for each other candidate;

The license plate character recognition is to identify each group of character blocks, eliminate the wrong candidate area, and generate the recognized license plate number, license plate color and license plate area coordinates;

(1) in the described license plate location step, input is a picture, size is arbitrary, then carry out preprocessing, two steps of clustering based on DBSCAN;

The task of the preprocessing process is to process the original color image to generate a binary image containing edge information; the preprocessing process includes image grayscale, sobel longitudinal edge detection+license plate color enhancement, image binarization three step;

The task of the described clustering based on DBSCAN is to carry out cluster processing to the binary image to generate a group of license plate candidate areas; its steps are: use the DBSCAN density clustering method to cluster the binary image, and divide all points into Several high-density areas, areas exceeding a certain threshold T are called candidate areas; here, the critical radius is divided into RH and RW, which represent the radius in the length and width direction, RW=3*RH; then calculate the attributes of each area, including length , width, aspect ratio, and delete points that are too few on both sides of the area; finally, judge the attributes of the candidate area, exclude areas that do not fit the shape, and output possible candidate areas;

Described sobel vertical edge detection+license plate color enhancement, its steps are: carry out sobel vertical detection to whole picture, in detection process, for edge information p (x, y), if greater than K, its surrounding x*x adjacent Domain is scanned, for each row wherein, if license plate color appears, then strengthen p (x, y) 10%, the license plate color here refers to the color set that may appear in various license plates;

(2) the license plate character segmentation includes two steps of secondary preprocessing and segmentation based on projection;

The secondary preprocessing process is to grayscale each candidate area, perform local binarization, and perform horizontal projection operations to generate a binary license plate image;

The projection-based segmentation is to vertically project the binary image, then segment it character by character, and finally output a group of image character blocks;

The steps of the secondary preprocessing are as follows: first grayscale the region picture, then use the mean value threshold to binarize it, and finally find the upper and lower trough regions in the projection map through horizontal projection, and consider it as a useless region, Delete it to get the partial image of the binary license plate;

Described segmentation based on projection, its steps are: first carry out vertical projection, think that valley area is useless area, other is character area, divides several areas with this; Then analyze and judge for area, get rid of non-license plate area; Finally Cut off the upper and lower blanks for each character image; generate a set of character regions;

(3) described license plate character recognition, its step is: input is a single binary area, output is the character that comes out, comprises Chinese character, numeral and letter; First if the aspect ratio of area＞4, point density in its area If it is greater than a certain threshold, the character is directly recognized as 1; then the entire binary image sequence is used as a feature, brought into the respective classifier for classification, and the result is output; if the recognized character is not 7, it is considered as a non-license plate area ;Finally generate the recognized license plate number, license plate color and license plate area coordinates;

Using the classifier to classify is to establish binary templates for numbers, characters and Chinese characters in advance, and use template matching for recognition during classification, and output the recognition results.

2. a system of license plate recognition, is characterized in that, comprises: license plate location module, license plate character segmentation module, license plate character recognition module three parts, is based on the license plate recognition method described in claim 1; Input is a picture , the output is a number of sub-region pictures and recognition result description document xml files;

There are a variety of parameter options, with the function of dynamically configuring parameters and dynamically defining the output, outputting partial pictures of the license plate, partial pictures of the driver's position, partial pictures of the car logo, partial pictures of the car body, and thumbnails; it is convenient to replace the license plate recognition method, easy perform batch processing;

The multiple parameter options refer to adding the parameter plate or other or all after the program name to only recognize the output number and position of the license plate, and only calculate and output other partial pictures and perform all processing, so as to realize the separation of key processes and other processes.

3. The license plate recognition system according to claim 2, wherein the calculation and output of the partial picture of the driver's position is to estimate the driver's position based on the position of the license plate and the size of the vehicle model, compare it with the original image, and output it after correction The partial picture of the driver's position; the calculation and output of the partial picture of the vehicle logo is based on the position of the license plate, scanning the area above the license plate to find the vehicle logo, and outputting the partial picture of the license plate position after segmentation; the calculation and output of the partial picture of the car body is based on The position of the license plate and the size of the vehicle model, the position of the body is estimated, and compared with the original image, and the partial image of the body is output after correction.