Home

Recognition of vehicle number plates

1. IS OUMACLKA 1589 99 11 ABC Figure 2 2 a Original image b Horizontal rank filter c Vertical rank filter d Sobel edge detection e Horizontal edge detection f Vertical edge detection 2 2 Horizontal and vertical image projection After the series of convolution operations we can detect an area of the number plate according to a statistics of the snapshot There are various methods of statistical analysis One of them is a horizontal and vertical projection of an image into the axes x and y The vertical projection of the image is a graph which represents an overall magnitude of the image according to the axis y see figure 2 3 If we compute the vertical projection of the image after the application of the vertical edge detection filter the magnitude of certain point represents the occurrence of vertical edges at that point Then the vertical projection of so transformed image can be used for a vertical localization of the number plate The horizontal projection represents an overall magnitude of the image mapped to the axis x 14434 INFONMACIKAY 93829579 J Figure 2 3 Vertical projection of image toa y axis Let an input image be defined by a discrete function f x y Then a vertical projection p of the function f ata point y is a summary of all pixel magnitudes in the y row of the input image Similarly a horizontal projection at a point x of that function 15
2. 69 023 jpg width 354 px see 308 px_ Class difficult environment This is a typical a LARSKE PRACE snapshot with a difficult surrounding environment The table in the background contains more characters in one line than a number plate in the foreground This fact causes a bigger amount of horizontal edges in an area of the table The three detected peaks in the graph of the horizontal projection correspond to three rows in the table Although the number plate candidate is wrong the further analysis did not refuse it because the number of characters 10 is within the allowed range Recognized plate OCNCKEP 70 044 jpg width L px EL DK pee skewed plates The part of graph corresponding to the peak 2 in vertical graph has a wider distribution due to the improper vertical projection caused by a skew of the plate Because of this the bottom of the last character E has been cut off improperly Point of failure vertical projection band clipping 1 W TY VW VM Recognized plate RKS92AF 71 067 jpg width 402 px height 298 px Class extremely small characters This snapshot contains extremely small characters which are not distinguishable by a machine The number plate has been properly detected and characters have been segmented as well but it is very hard to distinguish between the B and 8 on the first position and
3. Let F be a hypothetic function that assign each element from the set A to an element from the set B F A B x F x where xe A is a description vector pattern which describes the structure of classified character and xe B is a classifier which represents the semantics of such character The function F is the probably best theoretical classificator but its construction 15 impossible since we cannot deal with each combination of descriptors In praxis we construct pattern classifier by using only a limited subset of the A B mappings This subset is known as a training set such as A C A and B CB Our goal is to construct an approximation F x w of the hypothetic function F where w is a parameter that affects the quality of the approximation where xe A CA Xe B CB Formally we can say that F w is a restriction of the projection F overa set A CA We assume that for each x A we know the desired value x B Xo gt Xo X X X gt XX gt X 42 Figure 5 1 The projection between sets A and B The F 15 a hypothetic function that maps every possible combination of input pattern XE A to a corresponding class E B This projection is approximated by a function F w which maps input patterns from training set A into the corresponding classes from the set B The problem is to find an optimal value or values of a parameter w The w is typically a vector or matrix of syntactical weights in a
4. A A 2220 AM DSE After substitution of concrete points and concrete number of gray levels J 256 H i H H H 228 RR g i 128 mnm 4255 mn TT g s ia p PVE 16 s i A I 1 S 2 5 S gt I 0 er H iig brightness before transformation Figure 4 1 We use the Lagrange interpolating polynomial as a transformation function to normalize the brightness and contrast of characters The Lagrange interpolating polynomial as a transformation function is a costly solution It is like harvesting one potato by a tractor In praxis there is more useful to construct the transformation using a simple linear function that spreads the interval H H into the unified interval 0 7 1 26 7 jm sli The normalization of image 15 proceeded by the transformation function in the following way 3 3 4 1 2 Global Thresholding The global thresholding 15 an operation when a continuous gray scale of an image 15 reduced into monochrome black amp white colors according to the global threshold value Let 0 1 be a gray scale of such image If a value of a certain pixel is above the threshold the new value of the pixel will be zero Otherwise the new value will be one for pixels with values above the threshold t Let v be an original value of the pixel such as ve 0 1 The new value v is computed as 0 if v
5. It also has one output connection called axon which can be several meters long The data flow in the biological neural network is represented by electrical signal which propagates along the axon When the signal reaches a synaptic connection between the axon and a consecutive dendrite it relieves molecules of chemical agent called mediators or neuro transmitters into such dendrite This action causes a local change of polarity of a dendrite transmission membrane The difference in the polarity of the transmission membrane activates a dendrite somatic potential wave which advances in a system of branched dendrites into the body of neuron 43 Cell body Neuron dendritic tree axon Figure 5 2 The biological neuron The biological neural network contains two types of synaptic connections The first 15 an excitive connection which amplifies the passing signal The second inhibitive connection suppresses the signal The behavior of the connection is represented by its weight The neural network contains mechanism which is able to alter the weights of connections Because of this the system of synaptic weights is a realization of human memory As the weights are continually altered the old information is being forgotten little by little axon terminal button of axon dendrite terminal buttons dendrite A B Figure 5 3 a Schematic illustration of the neural cell b The synaptic connection betw
6. but machines do not understand this definition Because of this there is a need to find an alternative definition of the number plate based on descriptors which will be comprehensible for machines This is a fundamental problem of machine vision and of this chapter Chapter three describes principles of the character segmentation In most cases characters are segmented using the horizontal projection of a pre processed number plate but sometimes these principles can fail especially if detected number plates are too warped or skewed Then more sophisticated segmentation algorithms must be used Chapter four deals with various methods normalization and detection of characters At first character dimensions and brightness must be normalized to ensure invariance towards a size and light conditions Then a feature extraction algorithm must be applied on a character to filter irrelevant data It 15 necessary to extract features which will be invariant towards character deformations used font style etc Chapter five studies pattern classifiers and neural networks and deals with their usage in recognition of characters Characters can be classified and recognized by the simple nearest neighbor algorithm 1NN applied to a vector of extracted features or there is also possibility to use one of the more sophisticated classification methods such as feed forward or Hopfield neural networks This chapter also presents additional heuristic analyses wh
7. me I I 1 7 dw 52 5 5 Heuristic analysis of characters The segmentation algotithm described in chapter three can sometimes detect redundant elements which do not correspond to proper characters The shape of these elements after normalization is often similar to the shape of characters Because of this these elements are not reliably separable by traditional OCR methods although they vary in size as well as in contrast brightness or hue Since the feature extraction methods described in chapter four do not consider these properties there is aneed to use additional heuristic analyses to filter non character elements The analysis expects all elements to have similar properties Elements with considerably different properties are treated as invalid and excluded from the recognition process The analysis consists of two phases The first phase deals with statistics of brighness and contrast of segmented characters Characters are then normalized and processed by the piece extraction algorithm Since the piece extraction and normalization of brightness disturbs statistical properties of segmented characters it is necessary to proceed the first phase of analysis before the application of the piece extraction algorithm In addition the heights of detected segments are same for all characters Because of this there 15 aneed to proceed the analysis of dimensions after application of the piece extraction algorithm The piec
8. 0 5 1 0 0 1 0 0 0 1 0 0 1 where S and S are shear factors The S is always zero because we shear the plate only in a direction of the Y axis Let P be a vector representing the certain point such as P x y 1 where x and y are coordinates of that point The new coordinates P x y 1 of that point after the shearing can be computed as P P A S where A is a corresponding transformation matrix Let the deskewed number plate be defined by a function f The function f can be computed in the following way f y lx y1 A 1 0 0 ev A 0 107 After the substitution of the transformation matrix A 1 tan 8 0 11 1 tan 0 0 f x y f x y 1 0 1 0 0 x y 1 0 1 0 I 0 0 1 0 0 0 1 0 18 Figure 2 13 a Original number plate Number plate after deskewing 19 Chapter 3 Principles of plate segmentation The next step after the detection of the number plate area is a segmentation of the plate The segmentation is one of the most important processes in the automatic number plate recognition because all further steps rely on it If the segmentation fails a character can be improperly divided into two pieces or two characters can be improperly merged together We can use a horizontal projection of a number plate for the segmentation or one of the more sophisticated methods such as segmentation using the neural networks If we assume only one row plates t
9. 1 is scalable but it affects the recognition abilities of a neural network at a whole Too few neurons in the hidden layer causes that the neural network would not be able to learn new patterns Too many neurons cause network to be overlearned so it will not be able to generalize unknown patterns as well The information in a feed forward neural network is propagated from lower layers to upper layers by one way connections There are connections only between adjacent layers thus feed forward neural network does not contain feedback connections or connections between arbitrary two layers In addition there exist no connections between neurons from the same layer 46 Output layer 2 Hidden layer 1 Data flow Input layer 0 Figure 5 5 Architecture of the three layer feed forward neural network 5 4 Adaptation mechanism of feed forward neural network There has been proven that a multilayered neural network composed of perceptons with a sigmoid saturation function can solve an arbitrary non linear problem Mathematically for each function 1 R there exists a multilayered feed forward neural network that is able to realize this function The proof is based on the Kolmogorov s theorem which tells that every continuously growing function f defined on interval 0 1 can be written as Fx wale da Sa x where are properly chosen continuous functions with one parameter The proble
10. Humans define a number plate in a natural language as a small plastic or metal plate attached to a vehicle for official identification purposes but machines do not understand this definition as well as they do not understand what vehicle road or whatever else is Because of this there is a need to find an alternative definition of a number plate based on descriptors that will be comprehensible for machines Let us define the number plate as a rectangular area with increased occurrence of horizontal and vertical edges The high density of horizontal and vertical edges on a small area is in many cases caused by contrast characters of a number plate but not in every case This process can sometimes detect a wrong area that does not correspond to a number plate Because of this we often detect several candidates for the plate by this algorithm and then we choose the best one by a further heuristic analysis Let an input snapshot be defined by a function f x y where x and y are spatial coordinates and f is an intensity of light at that point This function is always discrete on digital computers such as xe NJ A ye Ny where N denotes the set of natural numbers including zero We define operations such as edge detection or rank filtering as mathematical transformations of function f The detection of a number plate area consists of a series of convolve operations Modified snapshot is then projected into axes x and y
11. The thinning is a morphological operation which preserves end pixels and does not break connectivity Assume that pixels of the piece are black value of zero and background pixels are white value of one The thinning is an iterative process of two successive steps applied to boundary pixels of a piece With reference to the eight pixel neighborhood notation in figure 4 9 the first step flags a boundary pixel p for deletion if each of the following conditions 15 satisfied e At least one of the top right and bottom neighbor of the pixel p must be white the pixel p is white just when it does not belong to the piece P plePvp ePvp eP e At least one of the left right and bottom neighbor of pixel p must be white pePvp ePvp eP e The pixel p must have at least two and at most six black neighbors from the piece P This condition prevents the algorithm from erasing end points and from breaking the connectivity 36 2 lt r lt 6 e The number of white to black transitions in the ordered sequence Ds p p p p p p p p must be egual to one v p eP ap eP v p ePap eP v p ePap eP v p ePAp eP v p ePA p eP v p ePA pleP v p ePap eP v p ePAp eP 1 2 1 if The first step flags pixel p for deletion if its neighborhood meets the conditions above However the pixel 1s not deleted until all other pixels have been processed If at least one of the conditions is not satisfied th
12. This 15 most frequent reason why the global thresholding fail The adaptive thresholding solves several disadvantages of the global thresholding because it computes threshold value for each pixel separately using its local neighborhood Chow and Kaneko approach There are two approaches to finding the threshold The first is the Chow and Kaneko approach and the second is a local thresholding The both methods assumes that smaller rectangular regions are more likely to have approximately uniform illumination more suitable for thresholding The image is divided into uniform rectangular areas with size of mxn pixels The local histogram is computed for each such area and a local threshold is determined The threshold of concrete point is then computed by interpolating the results of the subimages o Te Figure 4 3 The number plate from figure 4 2 processed by the Chow and Kaneko approach of the adaptive thresholding The number plate is divided into the several areas each with own histogram and threshold value The threshold value of a concrete pixel denoted by is computed by interpolating the results of the subimages represented by pixels 1 6 Local thresholding The second way of finding the local threshold of pixel is a statistical examination of neighboring pixels Let x y be a pixel for which we compute the local threshold t For 28 simplicity we condider asquare neighborhood with width 2 r 1
13. where x r y r x r y r i x r y r and x r y r are corners of such sguare There are severals approaches of computing the value of threshold 9 Mean of the neighborhood t x y mean IF i j x r lt i lt x r y rsjsytr e Median of the neighborhood t x y median if i j X rSISX r y rsjsytr 9 Mean of the minimum and maximum value of the heighborhood 1 i oy 5 min 16 6 max 1 7 y rsjsytr y rsjsytr The new value f x y of pixel x y is then computes as 0 jf 53 5 0 53 f x x 1 f x x e 0 1 x y 4 2 Normalization of dimensions and resampling Before extracting feature descriptors from a bitmap representation of a character it is necessary to normalize it into unified dimensions We understand under the term resampling the process of changing dimensions of the character As original dimensions of unnormalized characters are usually higher than the normalized ones the characters are in most cases downsampled When we downsample we reduce information contained in the processed image There are several methods of resampling such as the pixel resize bilinear interpolation or the weighted average resampling We cannot determine which method is the best in general because the successfulness of particular method depends on many factors For example usage of the weighed average downsampling in combination with a detection of character edges is not a good so
14. 3 73 76 List of Figures 1 1 1 1 b 2 1 22 2 3 2 4 2 3 2 6 2 2 8 2 9 2 10 2 11 22 2 13 pA 3 2 3 3 3 3 b 4 1 4 2 4 3 4 4 4 5 4 6 4 7 4 8 4 9 4 10 4 11 4 12 Illuminated number plate Snapshot degraded by the significant motion blur Convolution matrix Various rank and edge detection filters Vertical projection of an image into a y axis Double phase plate clipping Vertical projection of the snapshot after convolution with a rank vector Band detected by an analysis of a vertical projection Horizontal projection of the band and its derivative Wider area of the number plate after deskewing Principle of a number plate positivity determination using the color histogram Difference between the rotated and sheared number plate Illustration of Hough transformation Example of the Hough transformation Example of a number plate before and after deskewing Example of a number plate after application of the adaptive thresholding Piece extraction algorithm Segmentation phase input Segmentation phase output Histogram normalization by the Lagrange interpolating polynomial Partially shadowed number plate Chow and Kaneko approach of adaptive thresholding Principle of the downsampling Nearest neighbor and weighted average downsampling The pixel matrix feature extraction method Region layouts in character bitmap Possible types of 2x2 edges in character bitmap The four pixel and eight pixel neighborho
15. A B Figure 5 4 a The summation X and gain saturation g function of the percepton with a threshold implemented as a dedicated input b The sigmoid saturation function 5 3 Feed forward neural network Formally the neural network is defined as an oriented graph G N E where N 15 a non empty set of neurons and E 15 a set of oriented connections between neurons The connection e n n e E is a binary relation between two neurons n and The set of all neurons N is composed of disjunctive sets N N N where N is a set of all neurons from the i layer N N UN UN The weight of a i neuron in a k layer is denoted as and the threshold of i neuron in a k layer 15 denoted as v Numbers of neurons for the input 0 hidden 1 and output 2 and o N The number of neurons in the input layer m is equal to a length of an input pattern x in order that each value of the pattern is dedicated to one neuron Neurons in the input layer do not perform any computation function but they only distribute values of an input pattern to neurons in the hidden layer Because of this the input layer neuron has only one input directly mapped layer are denoted as m n suchas m No into multiple outputs Because of this the threshold value V of the input layer neuron 1s egual 0 to zero and the weights of inputs w are equal to one The number of neurons in the hidden layer
16. Brno Czech Republic Kvasnicka V Benuskova L Pospichal J Farkas 1 Tino P Kral A Introduction to Neural Networks Technical University Kosice Slovak Republic Minsky M Papert S Perceptons An Introduction to Computational Geometry MIT Press Cambridge Massachusetts 1969 Shapiro V Dimov D Bonchev S Velichkov V Gluhchev G Adaptive License Plate Image Extraction International Conference Computer Systems and Technologies Rousse Bulgaria 2004 Smagt P Comparative study of neural network algorithms applied to optical character recognition International conference on Industrial and engineering applications of artificial intelligence and expert systems Charleston South Carolina USA 1990 Srivastava R Transformations and distortions tolerant recognition of numerals using neural networks ACM Annual Computer Science Conference San Antonio Texas USA 1991 Wang J Jean J Segmentation of merged characters by neural networks and shortest path Symposium on Applied Computing Indianapolis Indiana USA 1993 Zboril F Neural Networks Department of Intelligent Systems Faculty of Information Technology BUT Brno Czech Republic Zhang Y Zhang C New Algorithm for Character Segmentation of License Plate Intelligent Vehicles Symposium IEEE 2003 ANPR tutorial com Quercus technologies 2006 76
17. These projections are used to determine an area of a number plate 2 1 Edge detection and rank filtering We can use a periodical convolution of the function f with specific types of matrices m to detect various types of edges in an image 7 xy f x y m x y SY f x 7 m mod x i mod y J i 0 j 0 Il where w and h are dimensions of the image represented by the function f Note The expression m x y represents the element in x column and y row of matrix m p Y rep y 2 1 1 Convolution matrices Each image operation or filter is defined by a convolution matrix The convolution matrix defines how the specific pixel is affected by neighboring pixels in the process of convolution Individual cells in the matrix represent the neighbors related to the pixel situated in the centre of the matrix The pixel represented by the cell y in the destination image fig 2 1 is affected by the pixels x X according to the formula Y X X My X XM X X My x XM X X My 35 XM X X Mg XM Xg X Mg Source image Destination image Convolution matrix m M Ms Affecting pixels Affected pixel Figure 2 1 The pixel is affected by its neighbors according to the convolution matrix Horizontal and vertical edge detection To detect horizontal and vertical edges we convolve source
18. a summary of all magnitudes in the x column We can mathematically define the horizontal and vertical projection as h 1 1 p x gt f x j gt 2 7 3 iy j 0 i ll where w and h are dimensions of the image 2 3 Double phase statistical image analysis The statistical image analysis consists of two phases The first phase covers the detection of a wider area of the number plate This area is then deskewed and processed in the second phase of analysis The output of double phase analysis 15 an exact area of the number plate These two phases are based on the same principle but there are differences in coefficients which are used to determine boundaries of clipped areas The detection of the number plate area consists of a band clipping and a plate clipping The band clipping is an operation which is used to detect and clip the vertical area of the number plate so called band by analysis of the vertical projection of the snapshot The plate clipping is a consequent operation which is used to detect and clip the plate from the band not from the whole snapshot by a horizontal analysis of such band Snapshot Assume the snapshot is represented by a function f x y where x Sx lt x and y lt lt The ET Yo represents the upper left corner of the snapshot and represents the bottom right corner If w and h are dimensions of the snapshot then x 0 y 0 x w l and y
19. active phase of feed forward neural network The notation is the same as in figure 5 5 48 procedure activePhase input o W vector of thresholds and weights X input pattern to be classified output Z vector of activities of neurons in hidden layer y vector of activities of neurons in output layer neural network response begin first step evaluate activities of neurons in the hidden layer for each neuron in hidden layer with index ie 0 n 1 do begin lee wa for each input with index jes 0 m 1 do DE let z 8 end second step evaluate activities of neurons in the output layer for each neuron in oa a e a e aa EO begin let E w 4 for each input with index nen do 2 let w w Z Ise a s 6 end end 5 4 2 Partial derivatives and gradient of error function The goal of the training phase 1s to find optimal values of thresholds and weights to minimize the error function E Adaptation phase is an iterative process in which a response y to an input pattern x is compared with the desired response x The difference between the obtained and desired response is used for a correction of weights The weights are iteratively altered until the value of the error function E is negligible Gradient of error function related to a single pattern We compute a gradient g of an error function related to a single pattern x with desired and obtained responses y and x
20. between the 6 and 8 on the third position of plate Point of failure character recognition Segmentation Number of detected characters 10 Recognized plate 8Y849A4 72 Appendix B Demo recognition software User s manual JavaANPR is an ANPR recognition software that demonstrates principles described in this thesis It is written in the Java programming language If you want to run it you will need the Java 1 5 0 SE runtime environment or higher After downloading the distribution package please unpack it into a chosen directory The distribution package contains compiled program classes jar archive source codes and additional program resources such as bitmaps neural networks etc build Compiled classes dist Distribution directory contains JAR file and additional resources lib Compile time libraries nbproject Project metadata and build configuration resources Resources configuration file bitmaps neural networks src Source files 1 Cleaning compiling and building the project optional Normally you do not have to compile the project because distribution package already contains precompiled binaries If you want to recompile it again you can do it using the Apache Ant utility At first change a working directory to the 888 and type the following command to clean the previous build of the JavaANPR Issuing this command will delete whole content of the build and dist
21. in the local coordinate system of the segment The segment usually contains several pieces One of them represents the character and others represent redundant elements which should be eliminated The goal of the heuristic analysis is to find a piece which represents character Let us place the piece P into an imaginary rectangle x9 Vss y where X Yo is an upper left corner and x y is a bottom right corner of the piece 23 X minfx x le P Yo min y x yle P x max x x yle P y max y x y e P The dimensions and area of the imaginary rectangle are defined as w xy gt h yo and S w h Cardinality of the set P represents the number of black pixels n The number of white pixels n can be then computed asn S n w h P 1 6 overall magnitude M ofa piece is a ratio between the number of black pixels n and the area S of an imaginary rectangle M n S In praxis we use the number of white pixels n as a heuristics Pieces with a higher value of n will be preferred The piece chosen by the heuristics is then converted to a monochrome bitmap image Each such image corresponds to one horizontal segment These images are considered as an output of the segmentation phase of the ANPR process see figure 3 3 Figure 3 3 The input a and output b example of the segmentation phase of the ANPR B recognition process 24 Chapter 4 Feature extraction and normali
22. license plates much easier A B Figure 1 1 a Illumination makes detection of reflexive image plates easier b Long camera shutter and a movement of the vehicle can cause an undesired motion blur effect 1 4 Notations and mathematical symbols Logic symbols pq PA PY p Exclusive logical disjunction p xor q Logical conjunction p and q Logical disjunction p or q Exclusion not p Mathematical definition of image f x y x and y are spatial coordinates of an image and f is an intensity of light at that point This function is always discrete on digital computers xe Aye Ny where N denotes the set of natural numbers including Zero f p The intensity of light at point p f p f xy where p x v Pixel neighborhoods PINy p gt Pixel p is in a four pixel neighborhood of pixel p and vice versa PN p Pixel p is in an eight pixel neighborhood of pixel p and vice versa Convolutions a x b x Discrete convolution of signals a x and b x a x b x Discrete periodical convolution of signals a x and b x Vectors and sets mfx y max A min A mean A median A A X X l i Intervals a lt x lt b XE a b Quantificators aX Aly Fx sx Vx Rounding x x i The element in x column and y row of matrix m The maximum value contained in the set A The scope of elements can be specified by additional conditions The min
23. networks in construction of an automatic number plate recognition system ANPR This problematic includes mathematical principles and algorithms which ensure a process of number plate detection processes of proper characters segmentation normalization and recognition Work comparatively deals with methods achieving invariance of systems towards image skew translations and various light conditions during the capture Work also contains an implementation of a demonstration model which 1s able to proceed these functions over a set of snapshots Key Words Machine vision artificial intelligence neural networks optical character recognition ANPR Bibliographical citation Ondrej Martinsk Recognition of vehicle number plates bakal ska pr ce Brno FIT VUT v Brn 2007 VI Declaration I declare that I wrote this thesis myself with the help of no more than the mentioned literature and auxiliary means Ondrej Martinsky Acknowledgement The author is indebted to the supervisor of this thesis doc Ing Franti ek Zbo il CSc for his great help Copyright 2007 Ondrej Martinsky THIS WORK IS A PART OF THE RESEARCH PLAN SECURITY ORIENTED RESEARCH IN INFORMATION TECHNOLOGY MSM _ 0021630528 AT BRNO UNIVERSITY OF TECHNOLOGY Licensed under the terms of Creative Commons License Attribution NonCommercial NoDerivs 2 5 You are free to copy distribute and transmit this work under the following conditions You must attrib
24. piece P 15 defined 85 a 72 PPL SPA The following algorithm can be used to detect the number of junctions and number line ends in a skeletonized piece P 38 let Nn 0 Le 73 0 for each pixel p in piece P do begin let neighbors 0 EDC DAC Os p Mooc 0 oh do peP jee n let neighbors neighbors 1 if neighbors 1 then let n n 1 else if neighbors23 then let n n 1 end 8 Figure 4 11 a b The junction is a pixel which as at least three neighbors in eight pixel neighborhood The line end 15 a pixel which has only one neighbor in eight pixel neighborhood 0 The loop is a group of pixels which encloses the continuous white space Loops It is not easy to determine the number of loops n in the skeletonized character The algorithm is based on the following principle At first we must negate the bitmap of the skeletonized character Black pixels will be considered as background and white pixels as foreground The number of loops in the image is equal to a number of lakes which are surrounded by these loops Since the lake is a continuous group of white pixels in the positive image we apply the piece extraction algorithm on the negative image to determine the number of black pieces Then the number of loops is equal to the number of black pieces minus one because one piece represents the background of the original image negated to the fore
25. this there is need to describe a character in another way The description of the character should be invariant towards the used font type or deformations caused by a skew In addition all instances of the same character should have a similar description A description of the character 15 a vector of numeral values so called descriptors or patterns e E Generally the description of an image region is based on its internal and external representation The internal representation of an image is based on its regional properties such as color or texture The external representation is chosen when the primary focus is on shape characteristics The description of normalized characters is based on its external characteristics because we deal only with properties such as character shape Then the vector of descriptors includes characteristics such as number of lines bays lakes the amount of horizontal vertical and diagonal or diagonal edges and etc The feature extraction is a process of transformation of data from a bitmap representation into a form of descriptors which are more suitable for computers If we associate similar instances of the same character into the classes then the descriptors of characters from the same class should be geometrically closed to each other in the vector space This is a basic assumption for successfulness of the pattern recognition process This section deals with various methods of feature extraction
26. 2 2 w Y 2 OO WO ew WW gt 1 5 1 1 50 lt AN ANG Figure 5 7 The numeric approach of finding the global minimum in the error landscape The vector of optimal thresholds and weights w is represented by a global minimum in the error landscape Since we cannot compute this minimum analytically we have to use a numeric approach There are various numeric optimization algorithms such as Newton s method or the gradient descent We use the gradient descent algorithm to find the global minimum in the error landscape The single step of the iterative algorithm can looks like follows k gD _ Kg S Eg m 0 k all 02 SR ORG JE A 0 k wi Wij Wi j k D M l Wij Wij th th where W 15 a weight of the connection between the i neuron in I layer and j neuron in l 1 layer tet in ak step of iterative process The speed of convergence is represented by a parameter Too small value of the parameter A causes excessively slow convergence Too big value of the 4 breaks the monotony of convergence The U 15 a momentum value which prevents the algorithm of getting stuck in local minimums Note The notation ok of the vector or gradient The means the component i 92 is a partial derivative of error function E by the threshold value Similarly the 92 2 is a partial deriva
27. 2 is ooo P while continue true Note The pixel p belongs to the piece P when it is black pe PS f p 0 A B c Figure 4 10 a The character bitmap before skeletonization b The thinning algorithm iteratively deletes boundary pixels Pixels deleted in the first iteration are marked by a light gray color Pixels deleted in the second and third iteration are marked by dark gray c The result of the thinning algorithm is a skeleton or a medial axis Structural analysis of skeletonized character The structural analysis is a feature extraction method that considers more complex structures than pixels The basic idea is that the substantial difference between two compared characters cannot be evaluated by the statistical analysis Because of this the structural analysis extracts features which describe not pixels or edges but the more complex structures such as junctions line ends and loops Junction The junction is a point which has at least three black neighbors in the eight pixel neighborhood We consider only two types of junctions the junction of three and four lines The number of junctions in the skeletonized piece P is mathematically defined as nj or CPA pp n 72 P P GPA Pp Line end The line end is a point which has exactly one neighbor in the eight pixel neighborhood The number of line ends in 8 skeletonized
28. 2357 PO69 ADOPORO09 EFKPOTXY469 ap BENE BE Table 4 1 Structural constraints of characters output structural constraints Z input output output output structural constraints input Figure 4 13 a b Structural constraints can be applied before and after the recognition by C the neural network c Example of the skeletonized alphabet Feature extraction In case we know the position of structural elements we can form a vector of descriptors directly from this information Assume that there are several line ends loops and junctions in the 40 image The position of loop is defined by its centre To form the vector we must convert rectangular coordinates of the element into polar coordinates see figure 4 14 r xy 9 9 7 ss KO ya 7 X w where x and y are normalized rectangular coordinates The length and the structure of resulting vector vary according to a number and type of structural elements contained in the character Since the structural constraints divide characters into the several classes there are several possible types of description vector Each type of vector corresponds to one class of character For example consider character with two line ends and one junction This constraint determines the following class of possible characters G I J L M N S U V W Z 1 2 3 5 7 We defin
29. OE O Sela on Ask 0 Man nase ovals ANA KE ele 1 P ae ce ale FRED VS SE olo begin for each matrix h where js 0 1 1 do begin f xy 2 53 1 then f xytl f x Ly 1 begin let E let X X tl 11 1 Sme Sie end 34 4 3 3 Skeletonization and structural analysis The feature extraction techniques discussed in the previous two chapters are based on the Statistical image processing These methods do not consider structural aspects of analyzed images The small difference in bitmaps sometimes means a big difference in the structure of contained characters For example digits 6 and 8 have very similar bitmaps but there 15 a substantial difference in their structures The structural analysis is based on higher concepts than the edge detection method It does not deal with terms such as pixels or edges but it considers more complex structures like junctions line ends or loops To analyze these structures we must involve the thinning algorithm to get a skeleton of the character This chapter deals with the principle of skeletonization as well as with the principle of structural analysis of skeletonized image The concept of skeletonization The skeletonization is a reduction of the structural shape into a graph This reduction 15 accomplished by obtaining a skeleton of the region via the skeletonization algorithm The skeleton of a shape is mathematically defined as a medial axis transform
30. P Two plate numbers are equal if all characters on corresponding positions are equal p pe where p is the i character of plate number P 7 2 2 Weighted score If P is a plate number recognized by a machine and P is the correct one then weighted score s of plate P is given as p p pi n r i P p gt where m is the number of correctly recognized characters and n is the number of all characters in plate For example if the plate KE123AB has been recognized as KE128AB the weighted correctness score for this plate is 0 85 but the binary score is 0 7 3 Results The table 7 1 shows recognition rates which has been achieved while testing on various set of number plates According to the results this system gives good responses only to clear plates because skewed plates and plates with difficult surrounding environment causes significant degradation of recognition abilities Total number Total number Weighted score of plates of characters Clear plates 68 470 87 Blurred plates 46 87 Skewed plates 51 64 Average plates 1254 73 02 Table 7 1 Recognition rates of the ANPR system 61 Summary The objective of this thesis was to study and resolve algorithmic and mathematical aspects of the automatic number plate recognition systems such as problematic of machine vision pattern recognition OCR and neural networks The problematic has been divided into sever
31. The constant c is used to determine foots of peak x The optimal value of c is 0 7 The constant c determines the minimum height of the peak related to the maximum value of the projection v If the height of the peak is below this minimum the peak will not be considered as a space between characters It is important to choose a value of constant c carefully An inadequate small value causes that too many peaks will be treated as spaces and characters will be improperly divided A big value of c causes that not all regular peaks will be treated as spaces and characters will be improperly merged together The optimal value of 15 0 86 To ensure a proper behavior of the algorithm constants and c should meet the following condition V JE Pic Vie Py 4 AP X where is a set of all detected peaks x with corresponding foots x and x 21 3 2 Extraction of characters from horizontal segments The segment of plate contains besides the character also redundant space and other undesirable elements We understand under the term segment the part of a number plate determined by a horizontal segmentation algorithm Since the segment has been processed by an adaptive thresholding filter it contains only black and white pixels The neighboring pixels are grouped together into larger pieces and one of them is a character Our goal is to divide the segment into the several pieces and keep only one piece representing the
32. The gradient g is computed in direction from upper layers to lower layers as follows At first we compute components of the gradient related to thresholds ee in the output layer as 9 33 2 AF Uy 49 Then we compute components of the gradient related to thresholds 2 in the hidden layer These components are computed using the components from the previous step as OE 992 follows OE lt OE fj 1 SYG 1 wi aa 2 987 H Similarly we compute components of the gradient related to weights wi and L in the following way E dE OE dw i 40 o dwt a ra X The gradient g is a vector of components is given as follows J 3E dE OE dE dE OE JE 38 921 98 95 Iw TUNE TERE Overall gradient The overall gradient is defined as a summary of gradients related to individual patterns of the training set A Let 8 4 be a gradient related to a training pair x x The overall gradient is A computed as 2 8 pas x X 5 4 3 Adaptation phase The adaptation phase 15 an iterative process of finding optimal values of weight and thresholds for which a value of the error function E 15 in a local minimum The figure 5 7 schematically illustrates a graph of the function E so called error landscape Generally the error landscape 15 w 1 dimensional where w is a cardinality of the vector of thresholds and weights such as 1940 1 79 2 1 1
33. VYSOK U EN TECHNICK V BRN BRNO UNIVERSITY OF TECHNOLOGY FAKULTA INFORMA N CH TECHNOLOGI STAV INTELIGENTN CH SYST M FACULTY OF INFORMATION TECHNOLOGY DEPARTMENT OF INTELLIGENT SYSTEMS RECOGNITION OF VEHICLE NUMBER PLATES BAKALARSKA PRACE BACHELOR S THESIS AUTOR PR CE ONDREJ MARTINSK AUTHOR BRNO 2007 1111111111 6 o o NY wo FAKULTA INFORMA N CH TECHNOLOGI UD stav INTELIGENTNICH systemo FACULTY OF INFORMATION TECHNOLOGY DEPARTMENT OF INTELLIGENT SYSTEMS fae 9 K S O ROZPOZN V N EVIDEN N CH SEL VOZIDEL RECOGNITION OF VEHICLE NUMBER PLATES BAKAL SK PR CE BACHELOR 5 THESIS AUTOR PR CE ONDREJ MARTINSK AUTHOR VEDOUC PR CE DOC ING FRANTI EK ZBO IL CSC SUPERVISOR BRNO 2007 Note for binding place for a copy of the thesis submission Zad n bakal sk pr ce 302 2006 xmarti35 Vysok u en technick v Brn Fakulta informa n ch technologi stav inteligentn ch syst m Akademick rok 2006 2007 Zad n bakal sk pr ce e itel Martinsk Ondrej Obor Informa n technologie T ma Rozpozn v n eviden n ch sel vozidel Kategorie Um l inteligence Pokyny 1 Prostudujte teori neuronov ch s t problematiku statistick ho rozpozn v n obraz a mo nosti optick ho rozpozn v n znak OCR 2 Navrhn te zp sob pro automatickou detekci oblasti eviden n h
34. ack pixels X ix vf x y 1 e Let A be an auxiliary set of pixels Principle of the algorithm is illustrated by the following pseudo code se see S SE SSE 06 x y f x y 14 0 0 lt x y lt w 4 While set X a do begin Met seo b Wet set A U Dll lone Piz ron py aA while set A is not empty do begin let x y be a certain pixel from A pull pixel x y from a set A ae f x y 1a x AA 0 0 lt x y lt w A then begin pull pixel x y fronecet MAA ast me BO sets le x ly x Ly x y 1 x y 1 into set A end end Soc JB Box s m end Note I The operation pull one pixel from a set is non deterministic because a set is an unordered group of elements In real implementation a set will be implemented as an ordered list and the operation pull one pixel from a set will be implemented as pull the first pixel from a list Note 2 The mathematical conclusion Ea Ymin lt x y lt ET Ymax means The pixel E y lies in a rectangle defined by pixels Ymin and Xmax Ymax More formally x 7112 y xRx yRy 5 where R is a one of the binary relations lt gt lt 2 and 3 2 2 Heuristic analysis of pieces The piece is a set of pixels
35. al chapters according to a logical sequence of the individual recognition steps Even though there is a strong succession of algorithms applied during the recognition process chapters can be studied independently This work also contains demonstration ANPR software which comparatively demonstrates all described algorithms I had more choices of programming environment to choose from Mathematical principles and algorithms should not be studied and developed in a compiled programming language I have considered usage of the Matlab and the Java Finally I implemented ANPR in Java rather than in Matlab because Java is a compromise programming environment between the Matlab and compiled programming language such as C Otherwise 1 would have to develop algorithms in Matlab and then rewrite them into a compiled language as a final platform for their usage in the real environment ANPR solution has been tested on static snapshots of vehicles which has been divided into several sets according to difficultness Sets of blurry and skewed snapshots give worse recognition rates than a set of snapshots which has been captured clearly The objective of the tests was not to find a one hundred percent recognizable set of snapshots but to test the invariance of the algorithms on random snapshots systematically classified to the sets according to their properties 62 Appendix A Case study 134 jpg width 488 px height 366 px O T
36. amp white representation of image is more appropriate for analysis because it defines clear boundaries of contained characters 4 1 1 Histogram normalization The histogram normalization is a method used to re distribute intensities on the histogram of the character segments The areas of lower contrast will gain a higher contrast without affecting the global characteristic of image Consider a grayscale image defined by a discrete function f x y Let I be a total number of gray levels in the image for example J 256 We use a histogram to determine the number of occurrences of each gray level i ie 0 I 1 H i ffx y Josx lt wadsy lt ha f x y i The minimum maximum and average value contained in the histogram is defined as 1 w l h 1 1 Hnm mani E aa edt HY Os y lt h Os y lt h x 0 y 0 25 where the values Hm and Hag are in the following relation max O lt Hin S Ha SH SI The goal of the histogram normalization is to obtain an image with normalized statistical characteristics such as Hmn 0 H I 1 Ha z To meet this goal we construct a m transformation function gli as a Lagrange polynomial with interpolation points I X H min 0 Haws and 3 3 H max 1 3 NE 1X s D X j 1 k Xj TAk kaj This transformation function can be explicitly written as 2 3 1 3 2 g i y lt y
37. and explains which method is the most suitable for a specific type of character bitmap For example the edge detection method should not be used in combination with a blurred bitmap 4 3 1 Pixel matrix The simplest way to extract descriptors from a bitmap image is to assign a brightness of each pixel with a corresponding value in the vector of descriptors Then the length of such vector is equal to a square w h of the transformed bitmap 31 where ie 0 w h l Bigger bitmaps produce extremely long vector of descriptors which 15 not suitable for recognition Because of this size of such processed bitmap 15 very limited In addition this method does not consider geometrical closeness of pixels as well as its neighboring relations Two slightly biased instances of the same character in many cases produce very different description vectors Even though this method is suitable if the character bitmaps are too blurry or too small for edge detection 251 181 068 041 032 071 197 196 014 132 213 187 043 041 174 O11 200 254 254 232 164 202 014 012 128 242 255 255 253 212 089 005 064 196 253 255 255 251 196 030 009 165 127 162 251 254 197 009 105 062 005 100 144 097 006 170 207 083 032 051 053 134 250 Figure 4 6 The pixel matrix feature extraction method 4 3 2 Detection of character edges In contrast with the previous method the detection of c
38. ate numbers Number plates are not unified so each country has own type Because of this number plate recognition system should be able to recognize a type of the number plate and automatically assign the correct syntactical pattern to it The assignation of the right syntactical pattern is a fundamental problem in syntactical analysis Syntactical pattern is aset of rules defining characters which can be used on a certain position in a plate number If the plate number P 15 a sequence of n alphanumerical characters P p pe then the syntactical pattern P is a n tuple of sets P p p and p is a set of all allowed characters for the i position in a plate For example czech number plates can contain digit on a first position followed by a character denoting the region where the plate has been registered and five other digits for a registration number of a car Formally the syntactical pattern P for czech number plates can looks like this 0 l 2 3 4 M 6 7 8 9 5 C B K H L T N E P A S U J Z 2 P 10 1 2 3 4 5 6 7 8 91 40 1 2 3 4 5 6 7 8 91 40 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 6 1 3 Choosing the right pattern If there are n syntactical patterns P P we have to choose the most suitable one for the evaluated plate number P For this purpose we define a metrics or a cost for a computation of a similarity between the evaluated plate number and t
39. ation To define the medial axis transformation and skeletonization algorithm we must introduce some elementary prerequisite terms Let N bea binary relation between two pixels x y and Ee y such as aNb means a is a neighbor of b This relation is defined as x y N xy x x 1v ly 7 foreight pixel neighbourhood 2 1 12 gt x x 18 y y 1 for four pixel neighbourhood The border B of character 1s a set of boundary pixels The pixel x y is a boundary pixel if it is black and if it has at least one white neighbor in the eight pixel neighborhood x yle Be f x y 0A7 x y f x y 1A x N xy The inner region of character is a set of black pixels which are not boundary pixels x ve IS f x y 0a x yl B Four pixel neighbourhood Boundary points Inner points Eight pixel neighbourhood A B Figure 4 9 a Illustration of the four pixel and eight pixel neighborhood b The set of boundary and inner pixels of character The piece P 15 then a union of all boundary and inner pixels P BUT Since there 15 only one continuous group of black pixels all black pixels belong to the piece P The principle and the related terminology of the skeletonization are similar to the piece extraction algorithm discussed in section 3 2 1 35 Medial axis transformation The medial axis transformation of the piece P defined as follows For each inner pixel pel we find th
40. d to evaluate the cost of candidates and to sort them according to this cost There are several independent heuristics which can be used to evaluate the cost The heuristics can be used separately or they can be combined together to compute an overall cost of candidate by a weighted sum a 0 15 a 0 25 a 0 4 0 04 Illustration a Yeo Yo The height of band in pixels Bands with a lower height will be preferred The p y is a maximum value of peak of vertical projection of snapshot which corresponds to the processed band Bands with a higher amount of vertical edges will be preferred This heuristics is similar to the previous one but it considers not only the value of the greatest peak but a value of area under the graph between points y and y These points define a vertical position of the evaluated band 13 The proportions of the one row number plates are similar in the most countries If we assume that width height ratio of the plate is about five we can compare the measured ratio with the estimated one to evaluate the cost of the number plate 2 4 2 Deeper analysis The deeper analysis determines the validity of a candidate for the number plate Number plate candidates must be segmented into the individual characters to extract substantial features The list of candidates 15 iteratively processed until the first valid number plate is found The candidate is considered as a
41. directories javaanpr ant clean Then issue the ant compile and ant jar commands The compile target will compile all source files in the src directory The jar target will create the dist directory with a jar archive and additional run time resources javaanpr ant compile javaanpr ant jar 2 Running the viewer You can run the interactive ANPR viewer using the ant by typing the following command javaanpr ant run If you do not have installed the ANT utility you can run viewer manually by the following commands ZEM OG Se ydet dist 1 1 1 11 1 jal Another way to run the viewer 15 a double click to a javaanpr jar archive in the MS Explorer 73 4 JavaANPR 1 1 xj Image Help 011 jpg 012 jpg 013 jpg 014 jpg 015 jpg 016 jpg 017 jpg 018 jpg 019 jpg 021 jpg 022 jpg 024 jpg 025 jpg 026 jpg 027 jpg 028 jpg 029 jpg 030 jpg 031 jpg 032 jpg 033 jpg 035 jpg 036 jpg 037 jpg zi recognize plate BB 51BH Copyright c 2006 Ondrej Martinsky Figure B 1 Graphical user interface of the JavaANPR viewer Important By default the program expects the configuration file config xml and other resources in the working directory Because of this please do not run the jar archive from other directories Otherwise the program will not be able to start 3 Using command line arguments Besides the graphical user interface program also contai
42. e 0 2 1 if ve t 1 The threshold value t can be obtained by using a heuristic approach based on a visual inspection of the histogram We use the following algorithm to determine the value of t automatically Select an initial estimate for threshold t for example t 0 5 2 The threshold r divides the pixels into the two different sets S iL y f x y lt i and S 2 vf xy gt r 3 Compute the average gray level values 4 and 44 for the pixels in sets S and S as l 1 s X fay ei M F Sa x yles S x yles 1 4 Compute a new threshold value t p U 5 Repeat steps 2 3 4 until the difference in successive iterations is smaller than predefined precision t Since the threshold is global for a whole image the global thresholding can sometimes fail Figure 4 2 8 shows a partially shadowed number plate If we compute the threshold using the algorithm above all pixels in a shadowed part will be below this threshold and all other pixels will be above this threshold This causes an undesired result illustrated in figure 4 2 b 27 T S a 5 z O b Brightness B Figure 4 2 a The partially shadowed number plate b The number plate after A thresholding The threshold value determined by an analysis of the histogram 4 1 3 Adaptive thresholding The number plate can be sometimes partially shadowed or nonuniformly illuminated
43. e a vector of descriptors to distinguish between these characters as follows x 1 8 n 0 n 0 u l line end line end junction 2 Figure 4 14 a The skeleton of the character contains several structural elements such as Junctions loops and line ends b c Each element can be positioned in the rectangular or polar coordinate system 41 Chapter 5 Recognition of characters The previous chapter deals with various methods of feature extraction The goal of these methods 15 to obtain a vector of descriptors so called pattern which comprehensively describes the character contained in a processed bitmap The goal of this chapter is to introduce pattern recognition techniques such as neural networks which are able to classify the patterns into the appropriate classes 5 1 General classification problem The general classification problem is formulated using the mapping between elements in two sets Let A be a set of all possible combinations of descriptors and B be a set of all classes The classification means the projection of group of similar element from the set A into a common class represented by one element in the set B Thus one element in the set B corresponds to one class Usually the group of distinguishable instances of the same character corresponds to the one class but sometimes one class represents two mutually indistinguishable characters such as 0 and
44. e closest boundary pixel p e B If a pixel p has more than one such neighbor it is said to belong to the medial axis or skeleton of the P The concept of the closest boundary pixel depends on the definition of the Euclidean distance between two pixels in the orthogonal coordinate system Mathematically the medial axis or skeleton S 15 a subset of the P defined as pe 5 amp 27 7 pe BA p BAd p p d p p mintd p p j The pixel p belongs to the medial axis S if there exists at least two pixels p and p such as Euclidean distance between pixels p and p is equal to the distance between pixels p and p and these pixels are closest boundary pixels to pixel p The Euclidean distance between two pixels p x y and p x y is defined as 2 2 d p pi xx 4 2 Skeletonization algorithm Direct implementation of the mathematical definition of the medial axis transformation 15 computationally expensive because it involves calculating the distance from every inner pixel from the set J to every pixel on the boundary B The medial axis transformation is intuitively defined by a so called fire front concept Consider that a fire is lit along the border All fire fronts will advance into the inner of character at the same speed The skeleton of a character is then a set of pixels reached by more than one fire front at the same time The skeletonization or thinning algorithm 15 based on the fire front concept
45. e extraction algorithm strips off white padding which surrounds the character Respecting the constraints above the sequence of steps can be assembled as follows 1 Segment the plate result is in figure 5 8 a 2 Analyse the brightness and contrast of segments and exclude faulty ones 3 Apply the piece extraction algorithm on segments result is in figure 5 8 b 4 Analyse the dimensions of segments and exclude faulty ones A Figure 5 8 Character segments before a and after b application of the piece extraction algorithm This algorithm disturbs statistical properties of brightness and contrast If we assume that there are not big differences in brightness and contrast of segments we can exclude the segments which considerably differs from the mean Let i segment of plate be defined by a discrete function f x y where w and h are dimensions of the element We define the following statistical properties of an element The global brightness of such segment is defined as a mean of brightnesses of individual pixels w hj DDM x 0 0 53 The global contrast of the i segment is defined as a standard deviation of brightnesses of individual pixels The function f x y represents only an intensity of grayscale images but the additional heuristic analysis of colors can be involved to improve the recognition process This a
46. e regions can be disjunctive as well as they can overlap each other Edge types in region Let us define an edge of the character as a 2x2 white to black transition in a bitmap According to this definition the bitmap image can contain fourteen different edge types illustrated in figure AE n Figure 4 8 The processed bitmap can contain different types of 2x2 edges 11 12 13 The statistics of occurrence of each edge type causes uselessly long vector of descriptors Because of this the similar types of edges are considered as the same The following lists shows how the edges can be grouped together 1 vertical edges 3 horizontal edges 6 9 7 type diagonal edges 5 7 8 V type diagonal edges 10 bottom right corner 11 bottom left corner 12 top right corner 13 top left corner O 2 4 a ee S For simplicity assume that edge types are not grouped together Let 77 be a number of different edge types where h is a 2x2 matrix that corresponds to the specific type of edge 1 0 0 1 1 1 0 0 1 0 0 1 1 0 h h h wh h h 1 0 0 1 0 0 1 1 0 1 1 0 0 0 11 TO Oa FOOT 40 116 o 0 gt 1 Oe 18 i ge 411 10 1 og 33 Let p be a number of rectangular regions in the character bitmap where x min Ymm max i Y max types for each of p reg
47. e value of pixel p is not changed After step one has been applied to all boundary pixels the flagged pixels are definitely deleted in the second step Every iteration of these two steps thins the processed character This iterative process is applied until no further pixels are marked for deletion The result of thinning algorithm is a skeleton or medial axis of the processed character e Let the piece P be a set of all black pixels contained in skeletonized character e Let B bea set of all boundary pixels The following pseudo code demonstrates the thinning algorithm more formally This algorithm proceeds the medial axis transformation over a piece P do iterative thinning process Lec let B O for each pixel p in piece P do create a set of boundary pixels 1 so pt PA PN p then the pixel p has at least one white neighbor insert pixel p into set B butkeepit also P for each pixel p in set B do 1 step of the iteration begin if at least one condition is violated skip this pixel meg p Pv p Pv peP then continue p Pv p Pv peP then continue fs 2 lt 7 v ape then continue if v ptePap eP v p ePap eP v p ePAp eP v p ePAp eP p eP v p EPA p P v p ePap P v p e Pa p P l hen begin CONE NUS si all tests passed be jean ji Ile 1 let continue true 37 end for each pixel p in set B do 2 step of the iteration bb
48. ection of lines The Hough transform is widely used for miscellaneous purposes in the problematic of machine vision but I have used it to detect the skew of captured plate and also to compute an angle of skew It is important to know that Hough transform does not distinguish between the concepts such as rotation and shear The Hough transform can be used only to compute an approximate angle of image in a two dimensional domain The mathematical representation of line in the orthogonal coordinate system is an equation y a x b where a is a slope and b 15 a y axis section of so defined line Then the line is a set of all points x y for which this equation is valid We know that the line contains an infinite number of points as well as there are an infinite number of different lines which can cross a certain point The relation between these two assertions is a basic idea of the Hough transform The equation y a x b can be also written as b x a y where x and y are parameters Then the equation defines a set of all lines a b which can cross the point x y For each point in the XY coordinate system there is a line in an AB coordinate system so called Hough space gt Figure 2 11 The XY and AB Hough space coordinate systems Each point Ee Yo in the XY coordinate system corresponds to one line in the Hough space red color The are several points ma
49. een a dendrite and terminal button of the axon Since the problematic of the biological neuron is very difficult the scientists proposed several mathematical models such as McCulloch Pitts binary threshold neuron or the percepton 5 2 1 McCulloch Pitts binary threshold neuron The McCulloch Pitts binary threshold neuron was the first model proposed by McCulloch and Pitts in 1943 The neuron has only two possible output values 0 or 1 and only two types of the synaptic weights the fully excitative and the fully inhibitive The excitative weight 1 does not affect the input but the inhibitive one negates it 1 44 The weighted inputs are counted together and processed by a neuron as follows Ja Da X j 0 0 if E lt 0 TOBI E20 This type of neuron can perform logical functions such as AND OR or NOT In addition McCulloch and Pitts proved that synchronous array of such neurons is able to realize arbitrary computational function similarly as a Turing machine Since the biological neurons have not binary response but continuous this model of neuron 15 not suitable for its approximation 5 2 2 Percepton Another model of neuron is a percepton It has been proved that McCulloch Pitts networks with modified synaptic connections can be trained for the recognition and classification The training is based on a modification of a neuron weights according to the reaction of such neuron as follows If the
50. ent lanes for a better congestion control in busy urban communications during the rush hours 1 2 Mathematical aspects of number plate recognition systems In most cases vehicles are identified by their number plates which are easily readable for humans but not for machines For machine a number plate is only a grey picture defined as a two dimensional function f x y where x and y are spatial coordinates and f is a light intensity at that point Because of this it 15 necessary to design robust mathematical machinery which will be able to extract semantics from spatial domain of the captured image These functions are implemented in so called ANPR systems where the acronym ANPR stands for an Automatic Number Plate Recognition ANPR system means transformation of data between the real environment and information systems The design of ANPR systems is a field of research in artificial intelligence machine vision pattern recognition and neural networks Because of this the main goal of this thesis is to study algorithmic and mathematical principles of automatic number plate recognition systems Chapter two deals with problematic of number plate area detection This problematic includes algorithms which are able to detect a rectangular area of the number plate in original image Humans define the number plate in a natural language as a small plastic or metal plate attached to a vehicle for official identification purposes
51. g max p x Xo 31 The x and x are coordinates of the plate which can be then detected as Xpo Max 1 17 x Sc 2260 Xo SXSXyy min Se Xpm P 2 ERE Um where c is a constant which is used to determine the foot of peak x The constant is calibrated to c 0 86 for the first phase of detection Second phase In the second phase of detection the horizontal position of a number plate is detected in another way Due to the skew correction between the first and second phase of analysis the wider plate area must be duplicated into a new bitmap Let x y be a corresponding function of such bitmap This picture has a new coordinate system such as 0 0 represents the upper left corner and w 1 h 1 the bottom right where w and h are dimensions of the area The wider area of the number plate after deskewing is illustrated in figure 2 8 In contrast with the first phase of detection the source plate has not been processed by the vertical detection filter If we assume that plate is white with black borders we can detect that borders as black to white and white to black transitions in the plate The horizontal projection P x of the image is illustrated in the figure 2 7 a To detect the black to white and white to black transitions there is a need to compute a derivative p x of the projection p x Since the projection is not continuous the derivation step cannot be an i
52. ground Another way 15 to use a Series of morphological erosions A BIC D Figure 4 12 a We determine the number of lakes in skeleton by applying the piece extraction algorithm on negative image The negative image b contains three pieces Since the piece 3 is a background only two pieces are considered as lakes c d The similar skeletons of the same character can differ in the number of junctions 39 Since we do not know the number of edges of the skeleton we cannot use the standard cyclomatic eguation know from the graph theory In addition two similar skeletons of the same character can sometimes differ in a number of junctions see figure 4 12 Because of this it 15 not recommended to use constraints based on the number of junctions Structural constraints To improve the recognition process we can assume structural constraints in the table 4 1 The syntactical analysis can be combined by other methods described in previous chapters such as edge detection method or pixel matrix The simplest way is to use one global neural network that returns several candidates and then select the best candidate that meets the structural constraints figure 4 13 a More sophisticated solution is to use the structural constraints for adaptive selection of local neural networks figure 4 13 b Lineends 90 005 08 800008 CEFGHIJKLMNSTUVWXYZ123457 CDGILMNOSUVWZ01
53. h l Band The band b in the snapshot f is an arbitrary rectangle b X9 Ypo Xp1 gt Yp gt Such as X50 Xmin A A1 Xmax A Ymi S Yoo lt Yor A Ymax Plate Similarly the plate p in the band b is an arbitrary rectangle p a Y p0 gt Xp1 gt Yp Such as X50 lt Xpo SX SX A Y po y lAl Yo The band can be also defined as a vertical selection of the snapshot and the plate as a horizontal selection of the band The figure 2 4 schematically demonstrates this concept Figure 2 4 The double phase plate clipping Black color represents the first phase of plate clipping and red color represents the second one Bands are represented by dashed lines and plates by solid lines 2 3 1 Vertical detection band clipping The first and second phase of band clipping is based on the same principle The band clipping is a vertical selection of the snapshot according to the analysis of a graph of vertical projection If h is the height of the analyzed image the corresponding vertical projection P y contains h values such as ye 0 h 1 The graph of projection may be sometimes too ragged for analysis due to a big statistical dispersion of values P y There are two approaches how to solve this problem We can blur the source snapshot costly solution or we can decrease the statistical dispersion of the ragged projection P by convolving its projection with a rank vector p y p
54. haracter edges does not consider absolute positioning of each pixel but only a number of occurrences of individual edge types in a specific region of the character bitmap Because of this the resulting vector is invariant towards the intra regional displacement of the edges and towards small deformations of characters Bitmap regions Let the bitmap be described by a discrete function f x y where w and h are dimensions such as OS x lt w and 0 lt y lt h We divide it into six equal regions organized to three rows and two columns in the following way Let Eee A and oa be an upper left and bottom right point of a rectangle which determinates the region 7 such as e Region 7 x 0 p 0 x0 2 l 1 y 45 min min max 2 max min min max e Region n x 45 02 45 2 3 h W 2 h Resin a Wy Sl 4 slz s 2 min ymin 3 max 2 Y maz 3 e Region n 50 12 yQ 2 a w 1 eE 32 e Region y a m0 2 A There are several ways how to distribute regions in the character bitmap The regions can be disjunctive as well as they can overlap each other The figure 4 7 shows the several possible layouts of regions 1 7 1 p 1 2 1 7 gm E E E E A 3 4 m 5 3 m M 3 3 mm sE mm Ae Figure 4 7 Layouts of regions in the character bitmap Th
55. he corresponding syntactical pattern zj x i i s X 1 samp i Xq y 0 lt lt Yi where p p g pt is a number of characters which do not match to corresponding positions in the syntactical pattern P Let y be an output vector for the i recognized character in a plate The greatest component of that vector max 9 then indicates how Os j lt z successfully the plate has been recognized Then the reciprocal value of 013817 is a cost Os j lt z of the character Another way of the cost evaluation is a usage of the Smith Waterman algorithm to compute the difference between the recognized plate number and the syntactical pattern For example assume that plate number OB01234 has been recognized as 0801234 and the recognition pattern does not allow digit at the second position of a plate If the character 8 has been recognized with similarity ratio of 0 90 and other characters with the ratio of 0 95 the metrics for this pattern is determined as follows 10 10 107 10 10 10 10 0 95 0 90 0 95 0 95 0 95 0 95 0 95 1 1 07426 57 If there 15 a pattern that exactly matches to the evaluated plate number we can say that number has been correctly recognized and no further corrections are needed In addition it is not possible to detect a faulty number plate if it does not break rules of a syntactical pattern Other
56. he plate has been successfully recognized no further comment needed TNA ita Recognized plate 959 63 149 jpg width 526 px height 350px O The plate has been successfully recognized no further comment needed 44 Segmentation Number of detected characters 10 Recognized plate RK9S9AD 64 034 jpg width 547 px height 410px 65 The plate has been successfully recognized no further comment needed 049 jpg width 410 px height 360 pp The plate has been successfully recognized no further comment needed Recognized plate RK878AC 66 040 jpg width 576 px height 432 px E a The plate has been successfully recognized no further comment needed Recognized plate BA738DE 67 The plate has been successfully recognized no further comment needed Recognized plate 1B19839 68 60 jpg width Bus height 336 px _Class blurred characters A significant blur caused improper detection of the band in a graph of vertical projection In addition further heuristic analyses did not detect this fault Because of this the incorrect candidate to number plate has been deskewed and then segmented even though it does not have any semantics Point of failure vertical projection and heuristic analysis of band Segmentation Number of detected characters 6 Recognized plate
57. he segmentation is a process of finding horizontal boundaries between characters Section 3 2 deals with this problematic The second phase of the segmentation is an enhancement of segments The segment of a plate contains besides the character also undesirable elements such as dots and stretches as well as redundant space on the sides of character There is a need to eliminate these elements and extract only the character Section 3 3 deals with these problems 3 1 Segmentation of plate using a horizontal projection Since the segmented plate is deskewed we can segment it by detecting spaces in its horizontal projection We often apply the adaptive thresholding filter to enhance an area of the plate before segmentation The adaptive thresholding is used to separate dark foreground from light background with non uniform illumination You can see the number plate area after the thresholding in figure 3 1 a After the thresholding we compute a horizontal projection p x of the plate f x y We use this projection to determine horizontal boundaries between segmented characters These boundaries correspond to peaks in the graph of the horizontal projection figure 3 1 b A Figure 3 1 a Number plate after application of the adaptive thresholding b Horizontal projection of plate with detected peaks Detected peaks are denoted by dotted vertical lines 20 The goal of the segmentation algorithm 15 to find peaks which correspo
58. hich the function E has a minimum see figure 5 6 E gt 7 0 Figure 5 6 Dependency of error functions and E on the number of neurons in input layer 1 and the number of iteration steps k For simplicity we will assume only a feed forward neural network with one layer of hidden neurons defined in section 5 3 All neurons in adjacent layers are connected by oriented connections There are no feedback connections or connections between neurons within a single layer The activities of hidden and output neurons are defined as m 1 n 1 1 d 2 2 476 gt 2 8 0 j 0 activities of neurons in the hidden layer activities of neurons in the output layer where g is a sigmoid saturation function see figure 5 4 b 5 4 1 Active phase Evaluation of the activities of hidden and output neurons 15 performed in so called active phase The active phase consists of two steps in three layer neural networks The first step is an evaluation of activities z in the hidden layer and the second step is an evaluation of activities y Since the evaluation of activities is performed from bottom layers to top ones the term feed forward refers to this principle The active phase corresponds to an approximation F x w of function F x and it is performed every time when there is a need to classify the input pattern x The following pseudo code demonstrates the
59. ich are used for elimination of non character elements from the plate Sometimes the recognition process may fail and the detected plate can contain errors Some of these errors can be detected by a syntactical analysis of the recognized plate If we have a regular expression or a rule how to evaluate a country specific license plate we can reconstruct defective plates using this rule For example a number zero 0 can be automatically repaired to a character O on positions where numbers are not allowed Chapter six deals with this problematic 1 3 Physical aspects of number plate recognition systems Automatic number plate recognition system is a special set of hardware and software components that proceeds an input graphical signal like static pictures or video sequences and recognizes license plate characters from it A hardware part of the ANPR system typically consists of a camera image processor camera trigger communication and storage unit The hardware trigger physically controls a sensor directly installed in a lane Whenever the sensor detects a vehicle in a proper distance of camera it activates a recognition mechanism Alternative to this solution is a software detection of an incoming vehicle or continual processing of the sampled video signal Software detection or continual video processing may consume more system resources but it does not need additional hardware equipment like the hardware trigger Image proces
60. image with matrices m and m The convolution matrices are usually much smaller than the actual image Also we can use bigger matrices to detect rougher edges Sk 1 0 1 m 0 0 0 gt 1 0 1 1 1 1 0 1 Sobel edge detector The Sobel edge detector uses a pair of 3x3 convolution matrices The first is dedicated for evaluation of vertical edges and the second for evaluation of horizontal edges 1 2 I 1 0 1 G 0 0 0 3G 2 0 2 1 2 1 1 0 1 The magnitude of the affected pixel is then calculated using the formula G G In G a praxis it is faster to calculate only an approximate magnitude as G G Horizontal and vertical rank filtering Horizontally and vertically oriented rank filters are often used to detect clusters of high density of bright edges in the area of the number plate The width of the horizontally oriented rank filter matrix is much larger than the height of the matrix w A and vice versa for the vertical rank filter w amp h To preserve the global intensity of an image it is necessary to each pixel be replaced with an average pixel intensity in the area covered by the rank filter matrix In general the convolution matrix should meet the following condition w l h 1 gt gt m i j 1 0 i 0 j 0 where w and h are dimensions of the matrix The following pictures show the results of application of the rank and edge detection filters W
61. imum value contained in the set A The mean value of the elements contained in the set A The median value of the elements contained in the set A The cardinality of the set A Number of elements contained in the set Vectors or any other ordered sequences of numbers are printed bold The elements of vectors are denoted as x where i is a Sequence number starting with zero such as ie 0 n 1 where n x 15 a cardinality of the vector number of elements The element a of the vector x For example the vector x can contain elements a b c d suchas x a b c d If there 15 more than one vector denoted as x they are distinguished by their indexes i The upper index i does not mean the i element of vector x lies in the interval between a and b This notation is used when x is the spatial coordinate in image discrete as well as continuous This notation has the same meaning as the above one but it is used when x is a discrete sequence number There exists at least one x There exists exactly one x There exists exactly n x There does not exist x For every x Number x rounded down to the nearest integer Number x rounded up to the nearest integer Chapter 2 Principles of number plate area detection The first step in a process of automatic number plate recognition is a detection of a number plate area This problematic includes algorithms that are able to detect a rectangular area of the number plate in an original image
62. ing significantly reduces information contained in the original image by ignoring a big amount of pixels it preserves sharp edges and the strong bipolarity of black and white pixels Because of this the nearest neighbor downsamping 15 suitable in combination with the edge detection feature extraction method described in section 4 3 2 4 2 2 Weighed average downsampling In contrast with the nearest neighbor method the weighted average downsamping considers all pixels from a corresponding group of pixels in the original image Let r and r be a horizontal and vertical aspect ratio of the resampled image The value of the pixel 12 y in the destination image is computed as a mean of source pixels in the range cae gt Ymin 0 Er gt Ymax Xmax Ymax C C Xni 3 ad 2 min J Ymin 7 where 30 The weighted average method of downsampling does not preserve sharp edges of the image in contrast with the previous method You can see the visual comparison of these two methods in Figure 4 5 A B Figure 4 5 a Nearest neighbor resampling significantly reduces information contained in the original image but it preserves sharp edges b Weighted average resampling gives a better visual result but the edges of the result are not sharp 4 3 Feature extraction Information contained in a bitmap representation of an image is not suitable for processing by computers Because of
63. ions the length of the resulting vector x is computed as 77 P and are boundaries of the region r ie 0 9 1 If the statistics consider 77 different edge X Xos Xis s Xyp Feature extraction algorithm At first we have to embed the character bitmap f x y into a bigger bitmap with white padding to ensure a proper behavior of the feature extraction algorithm Let the padding be one pixel wide Then dimensions of the embedding bitmap will be w 2 and h 2 The embedding bitmap f x y is then defined as JM if x 0vy 0vx w lvy h 1 f wy f x Ly 1 if x 0vy 0Ovx w lvy h 1 where w and h are dimensions of character bitmap before embedding Color of the padding is white value of 1 The coordinates of pixels are shifted one pixel towards the original position The structure of vector of output descriptors is illustrated by the pattern below The notation h r means number occurrences of an edge represented by the matrix h in the region r x hy 1 h 7 h 1 hy Or hy 7 hy Or jh r h ry ee FF region region 7 region 72 1 We compute the position k of the 7 in the vector x as k i 7 j where 77 is the number of different edge types and also the number of corresponding matrices The following algorithm demonstrates the computation of the vector of descriptors x zeroize vector X Tor cachi region hy where is0 p 1 do begin fe
64. istogram normalization 4 1 2 Global thresholding 4 1 3 Adaptive thresholding Normalization of dimensions and resampling 4 2 1 Nearest neighbor downsampling 4 2 2 Weighted average downsampling Feature extraction 4 3 1 Pixel matrix 4 3 2 Detection of character edges 4 3 3 Skeletonization and structural analysis 5 Recognition of characters Sal 32 General classification problem Biological neuron and its mathematical models Vill U NRF F oO H UaU 01 13 15 14 LS 16 18 20 20 22 22 23 Zo PAS 2 21 29 2 30 k eal 52 29 42 42 43 53 5 4 5 5 5 2 1 McCulloch Pitts binary threshold neuron 5 2 2 Percepton Feed forward neural network Adaptation mechanism of feed forward neural network 5 4 1 Active phase 5 4 2 Partial derivatives and gradient of error function 5 4 3 Adaptation phase Heuristic analysis of characters Syntactical analysis of a recognized plate 6 1 Principle and algorithms 6 1 1 Recognized character and its cost 6 1 2 Syntactical patterns 6 1 3 Choosing the right pattern Tests and final considerations 7 1 Choosing the representative set of snapshots 7 2 Evaluation of a plate number correctness 7 2 1 Binary score 7 2 2 Weighted score 7 3 Results Summary Appendix A Case study Appendix B Demo recognition software user s manual Bibliography 1X 44 45 46 47 48 49 50 23 56 56 56 S Sag 59 59 60 60 61 61 62 6
65. j t k jeho ztrat pri b n manipulaci Vedouc Zbo il Franti ek doc Ing CSc UITS FIT VUT Datum zad n 1 listopadu 2006 Datum odevzd n 15 kv tna 2007 VYSOK u Fakulta l u V BANE Ustav intaia Chnologi a 612 J St m lan doc Dr Ing Petr Han ek vedouc stavu 11 Note for binding place for a copy of the license page 1 1V Note for binding place for a copy of the license page 2 Abstrakt Pr ce se zab v mo nostmi vyu it teoretick ch poznatk z oblasti um l inteligence strojov ho vid n a neuronov ch s t pri konstrukci syst m pro automatick rozpozn v n eviden n ch sel vozidel Do t to problematiky spadaj matematick principy a algoritmy kter zabezpe detekci oblasti eviden n ho sla vozidla segmentaci normalizaci a samotn rozpozn n znak Pr ce komparativn m zp sobem pojedn v o mo nostech zabezpe en invariance syst m z pohledu sv teln ch podm nek nebo deformace obrazu z pohledu kamery kterou je obraz sn m n Sou st pr ce je tak implementace demonstracn ho modelu kter je schopn tyto funkce realizovat nad sadou statick ch sn mk Kl ov slova Strojov vid n um l inteligence neur nov s t optick rozpozn van znak ANPR Abstract This work deals with problematic from field of artificial intelligence machine vision and neural
66. lution because this type of downsampling does not preserve sharp edges discussed later Because of this the problematic of character resampling is closely associated with the problematic of feature extraction We will assume that mxn are dimensions of the original image and m xn are dimensions of the image after resampling The horizontal and vertical aspect ratio is defined as r m m and r n n respectively 4 2 1 Nearest neighbor downsampling The principle of the nearest neighbor downsamping is a picking the nearest pixel in the original image that corresponds to a processed pixel in the image after resampling Let f xy be a discrete function defining the original image such as OSx lt m and O lt y lt n Then the function f x of the image after resampling is defined as 29 where OS x lt m and 0S y lt n If the aspect ratio is lower than one then each pixel in the resampled destination image corresponds to a group of pixels in the original image but only one value from the group of source pixels affects the value of the pixel in the resampled image This fact causes a significant reduction of information contained in original image see figure 4 5 original image Image after downsampling Figure 4 4 One pixel in the resampled image corresponds to a group of pixels in the original image Although the nearest neighbor downsamp
67. m is how to construct the neural network corresponding to a given non linear function At first we choose a proper topology of the network The number of neurons in the input and output layer is given by lengths of the input and output patterns while the number of neurons in the hidden layer is scalable An adaptation of the neural network means finding the optimal parameter w of the approximation function F x w discussed in section 5 1 Let us define two error functions to evaluate a worthiness of the parameter w A X 1 son Ae 1 P kD E Y F x w x E F x w x i i where subscript t means train and x means test The E is an error function defined for patterns from the training set and E for patterns from the test set The response of the neural network to an input pattern x is given as y F x w The error function E goes down as a number of neurons in the hidden layer grows This relation is valid also between the function E and a number of iterative steps of the adaptation process These relations can be mathematically described as follows 47 lim E 0 im E 0 n k where n is the number of neurons in the input layer and k is the number of iteration steps of the adaptation process The error function E does not have a limit at zero as n and k goes to infinity Because of this there exists an optimal number of neurons and optimal number of iteration steps in w
68. mation systems Because a standalone information system without any data 85 no sense there was also a need 10 transform information about vehicles between the reality and information systems This can be achieved by a human agent or by special intelligent eguipment which 1s be able to recognize vehicles by their number plates in a real environment and reflect it into conceptual resources Because of this various recognition techniques have been developed and number plate recognition systems are today used in various traffic and security applications such as parking access and border control or tracking of stolen cars In parking number plates are used to calculate duration of the parking When a vehicle enters an input gate number plate is automatically recognized and stored in database When a vehicle later exits the parking area through an output gate number plate is recognized again and paired with the first one stored in the database The difference in time is used to calculate the parking fee Automatic number plate recognition systems can be used in access control For example this technology is used in many companies to grant access only to vehicles of authorized personnel In some countries ANPR systems installed on country borders automatically detect and monitor border crossings Each vehicle can be registered in a central database and compared to a black list of stolen vehicles In traffic control vehicles can be directed to differ
69. n uncommon width height ratio and elements 1 and 4 due to a small height 54 6 om vom 084 i i i 059 5 6 6 093 6 i 833 sfo TEJ s 004 0012 0003 0061 019 oes 9 017 006 0002 0063 0189 oss oos 0016 0007 0056 0189 050 0025 oon 005 008 ona 083 00 9 005 0012 004 ona 000 0019 068 0009 068 014 053 0 Table 5 1 Properties of segments in figure 5 8 The meaning of abbreviations 15 as follows BRI brightness CON contrast HUE hue SAT saturation HEI height WHR width height ratio 5 7 BEE E IES 12 B M 15 BIs Bl Die E 1 1 55 Chapter 6 Syntactical analysis of recognized plate 6 1 Principle and algorithms In some situations when the recognition mechanism fails there is a possibility to detect a failure by a syntactical analysis of the recognized plate If we have country specific rules for the plate we can evaluate the validity of that plate towards these rules Automatic syntax based correction of plate numbers can increase recognition abilities of the whole ANPR system For example if the recognition software is confused between characters 8 and B the final decision can be made according to the syntactical pattern If the pattern allows only digits for that position the character 8 will be used rather
70. nalysis separates character and non character elements on color basis If the captured snapshot 15 represented by a HSV color model we can directly compute the global hue and saturation of the segments as a mean of hue and saturation of individual pixels w h p D Yala p Es x 0 y 0 x 0 y 0 where A x y and s x y is a hue and saturation of the certain pixel in the HSV color model If the captured snapshot is represented by a RGB color model there is need to transform it to the HSV model first To determine the validity of the element we compute an average value of a chosen property n 1 over all elements For example the average brightness is computed as p gt p where n 15 i 0 does a number of elements The element 7 is considered as valid if its global brightness p not differ more than 16 from the average brightness p The threshold values of individual properties have been calibrated as follows G ie brightness BRI To ie lt 0 16 Contrast CON Bee lt 0 1 Pp Pe Pi Pr p p hue HUE lt 0 145 Saturation SAT fs lt 0 24 Pp Ps h width height ratio W Height HEI lt 0 2 WHR 0 1 lt h lt 0 92 If the segment violates at least one of the constraints above it is considered as invalid and excluded from the recognition process The table 5 1 contains properties of elements from figure 5 8 According to this table elements 0 and 10 have been refused due to a
71. nd to the spaces between characters At first there 15 a need to define several important values in a graph of the horizontal projection p x e v The maximum value contained in the horizontal projection Pp x such as Vv max P x Where w 15 a width of the plate in pixels 0 lt x lt w a LT 9 v The average value of horizontal projection p x Such as v gt Py x W 20 e v This value is used as a base for evaluation of peak height The base value is always calculated as v 2 v v The v must lie on vertical axis between the values v and v The algorithm of segmentation iteratively finds the maximum peak in the graph of vertical projection The peak is treated as a space between characters if it meets some additional conditions such as height of peak The algorithm then zeroizes the peak and iteratively repeats this process until no further space is found This principle can be illustrated by the following Steps 1 Determine the index of the maximum value of horizontal projection X arg max p 2 Detect the left and right foot of the peak as X max 1 17 x SC DP x OSXSx p x Sc Py n x min ix Xn SX lt W 3 Zeroize the horizontal projection p x on interval x x 4 If p x lt c v go to step 7 5 Divide the plate horizontally in the point x 6 Goto step 1 7 End Two different constants have been used in the algorithm above
72. neural network According to this parameter the values of the function F x w should be as closest as possible to the values of F x for input patterns x from the training set A We define an error function to evaluate worthiness of the parameter w 23 17 i where m is a number of patterns x x in the training set A Let w to be an optimal value of the parameter w such as w argmin E w Then the approximation F x w of weW the function F x is considered as adapted The adapted approximation F x w simulates original function F x for patterns x from the training set A In addition this approximation is able to predict the output classifier x for unknown pattern x from the test set A A A A The function with such prediction ability partially substitutes the hypothetic classificator F x Since the function F x w is only a model we use a feed forward neural network for its implementation 5 2 Biological neuron and its mathematical models For a better understanding of artificial neural network architecture there is a need to explain the structure and functionality of a biological neuron The human brain is a neural network of about ten billions interconnected neurons Each neuron 15 a cell that uses a biochemical reaction to process and transmit information The neural cell has a body of size about several micrometers and thousands of input connections called dendrites
73. neuron is not active and it should be we increase the weights If the neuron 15 active and it should not be we decrease them This principle was been used in a first model of the neural classifier called ADALINE adaptive linear neuron The major problem of such networks is that they are not able to solve linearly nonseparable problems This problem has been solved when the scientists Rumelhart Hilton and Williams proposed the error back propagation method of learning for multilayered percepton networks The simple McCulloch Pitts binary threshold neurons have been replaced by neurons with continuous saturation input output function Percepton has multiple analogous inputs and one analogous output Let x x be inputs with corresponding weights w o w j The weighted inputs are counted thresholded and saturated together in the following way 1 VER Wij AG TV j 0 1 2 2 ees where g is a sigmoid saturation function see figure 5 4 b and amp is a threshold value Sometimes the threshold 15 implemented as a dedicated input with a constant weight of 1 see figure 5 4 a Then the function of a neuron can be simplified to y g x w where x 15 a vector of inputs including the threshold value and w is a vector of weights including the constant weight 1 45 1 PERE o 25 5 55 Re thee ee eee ee gt Xo X 2 y o Tm jp
74. nfinitely small number 11 h limx If we derive a discrete function the derivation step h must be an integral number x0 for example h 4 Let the derivative of p x be defined as _ P x p x h h p x Where h 4 Pp x 100 0 A Figure 2 7 a The horizontal projection p x of the plate in figure 2 8 b The derivative of p x Arrows denote the BW and WB transitions which are used to determine the boundaries of the plate Figure 2 8 The wider area of the number plate after deskewing The left and right boundary of the plate can be determined by an analysis of the projection p x The left corner X 0 is represented by the black to white transition positive peak in figure 2 7 b and right corner x by the white to black transition negative peak in figure 2 7 b px x 2 eq max p x O lt x lt w Xpo ger fx 0 lt x lt 2 p x lt c min fp O lt x lt w X max fx P w lt x lt w 2 where c 15 a constant used to determine the most left negative and the most right positive peak The left and right corners must lie on the opposite halves of the detected plate according to the i Ww w constraints OS x lt for x a and lt x lt w for x 2 Aj 2 Ki 12 In this phase of the recognition process it is not possible to select a best candidate for a number plate This can be done by a heuristic analysis of charac
75. ns additional functions which are accessible using the command line arguments For more information about it please run the jar file with a help command Automatic number plate recognition system 111 1 1110 ENE nSk OOC E 007 Licensed under the Educational Community License Usage Sieve a VA APE FOR s Where options include help Displays this help ORDER Run GUI viewer default choice recoognize 110 1 Recognize single snapshot SCO MO eu c Recognize single snapshot and save report html into specified directory Neweoniig O lt file gt Generate default configuration file newnetwork o lt file gt Train neural network according to specified feature extraction method and learning parameters in config file and saves it into output file NENS 11 1 JO else ents gt Normalize ali images in SS RE ie sand Save to detdir 3 1 Command line recognition If you do not want to use the GUI viewer you can recognize snapshot by issuing the following command The recognized plate will be written to standard output 74 OS 1 1 1 111 1 1 0 1112 LOT 3 2 Recognition report Sometimes it is good to see inside the recognition process of concrete image Because of this JavaANPR supports a generation of HTML reports The recognition report contains images and verbose debugging information about each step of the recognition pr
76. ny ways due to the positioning of vehicle towards the camera Since the skew significantly degrades the recognition abilities it is important to implement additional mechanisms which are able to detect and correct skewed plates The fundamental problem of this mechanism is to determine an angle under which the plate is skewed Then deskewing of so evaluated plate can be realized by a trivial affine transformation It is important to understand the difference between the sheared and rotated rectangular plate The number plate is an object in three dimensional space which 15 projected into the two dimensional snapshot during the capture The positioning of the object can sometimes cause the skew of angles and proportions If the vertical line of plate v is not identical to the vertical line of camera objective v the plate may be sheared If the vertical lines v and v are identical but the axis a of plate is not parallel to the axis of camera a the plate may be rotated see figure 2 10 15 a a Up wee B je g eg ear Vy in hem ene Yeo a fa v v c oe a lla AV EY Figure 2 10 a Number plate captured under the right angle b rotated plate c Sheared plate 2 5 1 Detection of skew Hough transform is a special operation which is used to extract features of a specific shape within a picture The classical Hough transform is used for the det
77. o sla vozidla zp soby segmentace normalizace a rozpozn van znak V e en zohledn te mo nosti invariance syst mu vzhledem ke sv teln m podm nk m rozmanitosti eviden n ch sel a p padn i k ikmosti zna ky z pohledu kamery kterou je obraz sn man 3 Navrhn te a ve zvolen m programov m prost ed C Java Matlab implementujte model kter bude rozpozn van realizovat 4 Otestujte syst m na vybran sad sn mk 5 Zhodno te dosa en v sledky a rozeberte p padn nedostatky a jejich pr iny 6 Posudte mo nosti praktick ho vyu it syst mu Literatura 9 Podle doporu en vedouc ho pr ce P i obhajob semestr ln sti projektu je po adov no e Spln n prvn ch dvou bod zad n DP Podrobn z vazn pokyny pro vypracov n bakal sk pr ce naleznete na adrese http www fit vutbr cz info szz Technick zpr va bakal sk pr ce mus obsahovat formulaci c le charakteristiku sou asn ho stavu teoretick a odborn v chodiska e en ch probl m a specifikaci etap 20 a 30 celkov ho rozsahu technick zpr vy Student odevzd v jednom v tisku technickou zpr vu a v elektronick podob zdrojov text technick zpr vy plnou programovou dokumentaci a zdrojov texty program Informace v elektronick podob budou ulo eny na standardn m pam ov m m diu disketa CD ROM kter bude vlo eno do p semn zpr vy tak aby nemohlo do
78. ocess HTML report can be used to determine a point in which the recognition process failed The following command will recognize the image specified by its name and save the report into a specified destination directory OSE AA a AAA A i Mane cor EMAS 50 AS 3 3 Creating the default configuration file Configuration file contains settings and parameters which are needed during the recognition process If configuration file does not exist program will not be able to start Because of this JavaANPR 15 able to generate a default configuration file with recommended configuration settings by the following command ASP A0 A A Me SN SOLO STS 75 Bibliography 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Fajmon B Numeric Math and Probability scripts Faculty of Electrical Engineering and Communication Brno Czech Republic 2005 Fraser N Introduction to Neural Networks http www virtualventures ca neil neural neuron html Fukunaga K Introduction to statistical pattern recognition Academic Press San Diego USA 1990 Gonzalez R Woods R Digital Image Processing Prentice Hall Upper Saddle River New Jersey 2002 Kovar M Discreet Math scripts Faculty of Electrical Engineering and Communication Brno Czech Republic 2003 Kuba M Neural Networks scripts Faculty of Informatics Masaryk University
79. od Skeletonization algorithm Types of structural elements in character bitmap Different types of junctions in two instances of the same character 4 13 ab Combination of structural constraints and neural networks 4 13 c Example of the 9x13 upper case alphabet 4 14 5 1 5 2 5 3 a 5 3 b 5 4 8 5 4 b 5 5 5 6 5 7 5 8 7 1 Rectangular and polar coordinate systems in the character bitmap General classification problem Biological neuron Parts of the biological neuron Synaptic connections between dendrites and terminal buttons Summation and gain function of the percepton Sigmoid saturation function Three layer feed forward neural network Dependency of an error function on a number of neurons Finding a global minimum in the error landscape Character segments after application of the piece extraction algorithm Different types of car snapshots o HQ DW U LO 12 12 To 16 16 17 P 20 22 24 24 26 ZO 28 30 oak 52 33 33 5 38 39 39 40 40 41 43 44 44 44 46 46 47 48 sdi ie 60 List of Tables 4 1 5 1 7 1 Structural constraints of characters Table of segment properties related to the figure 6 8 Recognition rates of the ANPR system XI 40 23 61 Chapter 1 Introduction 1 1 ANPR systems as a practical application of artificial intelligence Massive integration of information technologies into all aspects of modern life caused demand for processing vehicles as conceptual resources in infor
80. pixels with a relatively high value are marked by a red color Each such pixel corresponds to a long white line in the figure 13 a If we assume that the angle of such lines determines the overall angle we can find the longest line as ab il a b To compute the angle of such a line there is a need to transform it back to the original coordinate system 1 b l b w he 2 where w and h are dimensions of the evaluated image Then the overall angle of image can be computed as 0 arctan a 17 The more sophisticated solution is to determine the angle from a horizontal projection of the Hough transform 4 This approach is much better because it covers all parallel lines together not only the longest one 1 lt lt 1 arctan w a arg max a j where p a is a horizontal projection of the Hough space such as 1 Py a F a b db s 2 5 2 Correction of skew The second step of a deskewing mechanism 15 a geometric operation over an image f x y AS the skew detection based on Hough transform does not distinguish between the shear and rotation it is important to choose the proper deskewing operation In praxis plates are sheared in more cases than rotated To correct the plate sheared by the angle 0 we use the affine transformation to shear it by the negative angle For this transformation we define a transformation matrix A 1 S 0 1 tan 8
81. regular character This concept 15 illustrated in figure 3 2 SE OOM Piece te Nee ew tt XCM segment Piece 3 Figure 3 2 Horizontal segment of the number plate contains several groups pieces of neighboring pixels 3 2 1 Piece extraction Let the segment be defined by a discrete function f x y in the relative coordinate system such as 0 0 is an upper left corner of the segment and w 1 h 1 is a bottom right corner where w and h are dimensions of the segment The value of f xy is 1 for the black pixels and 0 for the white space The piece P is a set of all neighboring pixels x y Which represents a continuous element The pixel x y belongs to the piece P 1f there is at least one pixel k y from the P such as x y and x y are neighbors 2 alx yle P x y N xy 66 The notation aN b means a binary relation a is a neighbor of b in a four pixel neighborhood 23 2 x x 18ly y Algorithm The goal of the piece extraction algorithm is to find and extract pieces from a segment of the plate This algorithm is based on a similar principle as a commonly known seed fill algorithm 22 e Let piece P be a set of neighboring pixels x y 9 Let S bea set of all pieces P from a processed segment defined by the function f xy 9 Let X bea set of all bl
82. rked ask L m in the Hough space that correspond to the lines in the XY coordinate system which can cross the point Yo 16 Let f x y be a continuous function For each point a b in Hough space there is a line in the XY coordinate system We compute a magnitude of point a b as a summary of all points in the XY space which lie on the line a x b Assume that f x y is a discrete function which represents the snapshot with definite dimensions wX h To compute the Hough transform of the function like this it is necessary to normalize it into a unified coordinate system in the following way 2X DY x l y 1 w h Although the space defined by a unified coordinate system 15 always discrete floating point on digital computers we will assume that it is continuous Generally we can define the Hough transform h a b of a continuous function f x y in the unified coordinate system as 1 h a b Fax o ad b ZEME Figure 2 12 a Number plate in the unified XY coordinate system after application of the horizontal edge detection filter b Hough transform of the number plate in the coordinate system Colored Hough transform in the AB coordinate system We use the Hough transform of certain image to evaluate its skew angle You can see the colored Hough transform on the figure 2 12 c The
83. ronment and S is a subset of plates with little characters 59 DL RSK PR CE gt memmy m A B Figure 7 1 Typical snapshot from the set of a clear plates b plates with little or blurred CD characters c skewed plates d plates with difficult surrounding environment 7 2 Evaluation of a plate number correctness Plate numbers recognized by a machine can sometimes differ from the correct ones Because of this there is a need to define formulas and rules which will be used to evaluate a degree of plate correctness Let P be a plate number and S SB BS be a set of all tested plate numbers Then recognition rate R S of the ANPR system tested on set S 15 calculated as 1 R S s P i 0 where n is a cardinality of the set S and s P is a correctness score of the plate P The correctness score is a value which express how successfully the plate has been recognized Now the question is how to define the correctness score of individual plates There are two different approaches how to evaluate it The first is a binary score and the second is a weighted score 7 21 Binary score Let us say that plate number P is a sequence of n alphanumerical characters P Dn p If P is the plate number recognized by a machine and P is the correct one then binary score s of plate P is evaluated as follows 60 ey Oa Ppo s P i r po if P
84. s well as plates degraded by the significant motion blur or skew Then a recognition ability of a machine is represented by a ratio between the number of plates which have been recognized by the machine and the number of plates recognized by a human Practically it is impossible to build a machine with the same recognition abilities as a human has Because of this the test like this 15 extremely difficult and useless In praxis it 15 more useful to find a representative set of number plates which can be captured by an ANPR camera The position of the camera has a significantly affects the quality of captured images and a successfulness of the whole recognition process The suitable position of the camera towards the lane can lead to a better set of all possible snapshots In some situations we can avoid of getting skewed snapshots by a suitable positioning of the camera Sometimes this is cleverer than a development of the robust de skewing mechanisms Let S be a representative set of all snapshots which can be captured by a concrete instance of the ANPR camera Some of the snapshots in this set can be blurred some of them can be too small too big too skewed or too deformed Because of this I have divided the whole set into a following subsets S S IS IS US US where S is a subset of clear plates S is a subset of blurred plates 5 is a subset of skewed plates S is a subset of plates which has a difficult surrounding envi
85. sor recognizes static snapshots captured by the camera and returns a text representation of the detected license plate ANPR units can have own dedicated image processors all in one solution or they can send captured data to a central processing unit for further processing generic ANPR The image processor is running on special recognition software which is a key part of whole ANPR system Because one of the fields of application is a usage on road lanes it is necessary to use a special camera with the extremely short shutter Otherwise quality of captured snapshots will be degraded by an undesired motion blur effect caused by a movement of the vehicle For example usage of the standard camera with shutter of 7 100 sec to capture a vehicle with speed of SO km h will cause a motion skew in amount of 0 22 m This skew means the significant degradation of recognition abilities There is also a need to ensure system invariance towards the light conditions Normal camera should not be used for capturing snapshots in darkness or night because it operates in a visible light spectrum Automatic number plate recognition systems are often based on cameras operating in an infrared band of the light spectrum Usage of the infrared camera in combination with an infrared illumination is better to achieve this goal Under the illumination plates that are made from reflexive material are much more highlighted than rest of the image This fact makes detection of
86. ters after the segmentation 2 4 Heuristic analysis and priority selection of number plate candidates In general the captured snapshot can contain several number plate candidates Because of this the detection algorithm always clips several bands and several plates from each band There is a predefined value of maximum number of candidates which are detected by analysis of projections By default this value is equals to nine There are several heuristics which are used to determine the cost of selected candidates according to their properties These heuristics have been chosen ad hoc during the practical experimentations The recognition logic sorts candidates according to their cost from the most suitable to the least suitable Then the most suitable candidate is examined by a deeper heuristic analysis The deeper analysis definitely accepts or rejects the candidate As there 15 a need to analyze individual characters this type of analysis consumes big amount of processor time The basic concept of analysis can be illustrated by the following steps Detect possible number plate candidates Sort them according to their cost determined by a basic heuristics Cut the first plate from the list with the best cost Segment and analyze it by a deeper analysis time consuming If the deeper analysis refuses the plate return to the step 3 Se Od 2 4 1 Priority selection and basic heuristic analysis of bands The basic analysis is use
87. than the character B Another good example 15 a decision between the digit 0 and the character The very small difference between these characters makes their recognition extremely difficult in many cases impossible 6 1 1 Recognized character and its cost In most cases characters are recognized by neural networks Each neuron in an output layer of a neural network typically represents one character Let y Va Vo Vaca z be a vector of output activities If there are 36 characters in the alphabet the vector y will be also 36 dimensional Let y be an i component of the vector y Then y means how much does the input character corresponds to the i character in the alphabet which is represented by this component The recognized character is represented by the greatest component of the vector y Y chr max y O lt i lt z where chr y is the character which is represented by the i component of vector y Let y be a vector y descendingly sorted according to the values of components Then the recognized character 1s represented by the first component of so sorted vector S Z chr When the recognition process fails the first component of y can contain invalid character which does not match the syntax pattern Then it is necessary to use the next valid character with a worse cost 56 6 1 2 Syntactical patterns In praxis ANPR systems must deal with many different types of pl
88. tive of function E by the value of weight ws 7 dw The whole adaptation algorithm of feed forward neural network can be illustrated by the following pseudo code 51 procedure adaptation input A training set of patterns begin initialize weights and thresholds in W to random values let let ZA speed of convergence fi momentum value oe r precision of adaptation process output W maximum number of iterations 1 vector of optimal weights and thresholds WM we haven t 8 previous value of W at the beginning oe E while AE gt do begin compute overall gradient ZDOLA S L aa of A do begin compute gradient for training pair x X vio Oval 1 ap activePhase W X Z y compute activities Z Yy OE A LOr each Es Model wy do 3 7 y 1 y y for each threshold DAM do 1 ws z DME wm 83 8 90 2 00 lt 57 f h weight WW do g gE g 2 or each weig Wi do m 1 1 x X dwi x X 92 J OE OE NN S kopon eo ae X 57 x X 1 x X D J dw 2 Ler sto alter values of thresholds and weights according to the gradient g for each threshold DAM in eko tet 2 1 7 l for each weight ws kolo
89. ute the work in the manner specified by the author or licensor but not in any way that suggests that they endorse you or your use of the work You may not use this work for commercial purposes For further information please read the full legal code at http creativecommons org licenses by nc nd 2 5 legalcode Vil Contents Introduction 1 1 1 2 1 3 1 4 ANPR systems as a practical application of artificial intelligence Mathematical aspects of number plate recognition systems Physical aspects of number plate recognition systems Notations and mathematical symbols Principles of number plate area detection 2 1 Led 2 3 2 4 2 Edge detection and rank filtering 2 1 1 Convolution matrices Horizontal and vertical image projection Double phase statistical image analysis 2 3 1 Vertical detection band clipping 2 3 2 Horizontal detection plate clipping Heuristic analysis and priority selection of number plate candidates 2 4 1 Priority selection and basic heuristic analysis of bands 2 4 2 Deeper analysis Deskewing mechanism 2 5 1 Detection of skew 2 5 2 Correction of skew 3 Principles of plate segmentation 3 1 32 Segmentation of plate using a horizontal projection Extraction of characters from horizontal segments 3 2 1 Piece extraction 3 2 2 Heuristic analysis of pieces 4 Feature extraction and normalization of characters 4 1 4 2 4 3 Normalization of brightness and contrast 4 1 1 H
90. valid number plate if it meets the requirements for validity Assume that plate p is segmented into several characters Pp P Where n is a number of characters Let w be a width of i character see figure 2 9 a Since all segmented characters have roughly uniform width we can use a standard deviation of these values as a heuristics n 1 where w is an arithmetic average of character widths w gt vw i i 0 If we assume that the number plate consists of dark characters on a light background we can use a brightness histogram to determine if the candidate meets this condition Because some country specific plates are negative we can use the histogram to deal with this type of plates see figure 2 9 b Let H b be a brightness histogram where b is a certain brightness value Let b in and bax be a value of a darkest and lightest point Then H b 15 a count of pixels whose values are equal to b The plate is negative when the heuristics is negative Dax Uhin where b is a middle point in the histogram such as b 14 P2 P3 Da Ps De JM Pg Po P n Po sIaquinu 5 11 Brightness Figure 2 9 a The number plate must be segmented into individual characters for deeper heuristic analysis b Brightness histogram of the number plate 15 used to determine the positivity of the number plate 2 5 Deskewing mechanism The captured rectangular plate can be rotated and skewed in ma
91. wise it is necessary to correct detected plate using the pattern with lowest cost 0 P arg min 5 P O lt i lt n The correction of a plate means the replacement of each invalid character by another one If the character p at the i position of the plate P does not match the selected pattern Po it will be replaced by the first valid one from y y is a sorted vector of output activities denoting how much the recognized character is similar to an individual character from the alphabet Heuristic analysis of a segmented plate can sometimes incorrectly evaluate non character elements as characters Acceptance of the non character elements causes that the recognized plate will contain redundant characters Redundant characters occur usually on sides of the plate but rarely in the middle If the recognized plate number is longer than the longest syntax pattern we can select the nearest pattern and drop the redundant characters according to it 58 Chapter Tests and final considerations 7 1 Choosing the representative set of snapshots I have captured many of static snapshots of vehicles for the test purposes Random moving and standing vehicles with Slovak and Czech number plates have been included At first my objective was to find a representative set of number plates which are recognizable by humans Of course the set like this contains extremely wide spectrum of plates such as clear and easy recognizable a
92. y m y where m 15 the rank vector analogous to the horizontal rank matrix in section 2 1 1 The width of the vector m is nine in default configuration After convolution with the rank vector the vertical projection of the snapshot in figure 2 3 can look like this 0591 o ver Yo M Figure 2 5 The vertical projection of the snapshot 2 3 after convolution with a rank vector The figure contains three detected candidates Each highlighted area corresponds to one detected band The fundamental problem of analysis is to compute peaks in the graph of vertical projection The peaks correspond to the bands with possible candidates for number plates The maximum value of p y corresponding to the axle of band can be computed as Yom arg max p y YoSySy The y and y are coordinates of band which can be detected as lt c Ypo y P Y Sc Py Yom Yp min 1y P Se Pym Yom EVEN c 1s a constant which is used to determine the foot of peak y In praxis the constant is calibrated to c 0 55 for the first phase of detection and c 0 42 for the second phase n Figure 2 6 The band detected by the analysis of vertical projection This principle is applied iteratively to detect several possible bands The y and coordinates are computed in each step of iterative process After the detection values of projection p in interval ee are zeroized This idea is illustrated b
93. y the following pseudo code leccos vot detected scandudares po 26 number of bands to be detected do begin detect Yo and yy by analysis of projection SAV o any oo L Z ea end The list L of coordinates y and y will be sorted according to value of peak y The band clipping is followed by an operation which detects plates in a band 10 2 3 2 Horizontal detection plate clipping In contrast with the band clipping there 15 a difference between the first and second phase of plate clipping First phase There is a strong analogy in a principle between the band and plate clipping The plate clipping is based on a horizontal projection of band At first the band must be processed by a vertical detection filter If w is a width of the band or a width of the analyzed image the corresponding horizontal projection p x contains w values Yp1 p x X f x j J Ypo0 Please notice that p x is a projection of the band not of the whole image This can be achieved by a summation in interval igs Vis which represents the vertical boundaries of the band Since the horizontal projection p x may have a big statistical dispersion we decrease it by convolving with a rank vector p x x m x The width of the rank vector is usually equal to a half of an estimated width of the number plate Then the maximum value corresponding to the plate can be computed as Xpm ar
94. zation of characters To recognize a character from a bitmap representation there 15 8 need to extract feature descriptors of such bitmap As an extraction method significantly affects the guality of whole OCR process it is very important to extract features which will be invariant towards the various light conditions used font type and deformations of characters caused by a skew of the image The first step 15 a normalization of a brightness and contrast of processed image segments The characters contained in the image segments must be then resized to uniform dimensions second step After that the feature extraction algorithm extracts appropriate descriptors from the normalized characters third step This chapter deals with various methods used in the process of normalization 4 1 Normalization of brightness and contrast The brightness and contrast characteristics of segmented characters are varying due to different light conditions during the capture Because of this it is necessary to normalize them There are many different ways but this section describes the three most used histogram normalization global and adaptive thresholding Through the histogram normalization the intensities of character segments are re distributed on the histogram to obtain the normalized statistics Techniques of the global and adaptive thresholding are used to obtain monochrome representations of processed character segments The monochrome or black

Recognition of vehicle number plates

Contents

Download Pdf Manuals

Related Search

Related Contents