License Plate Recognition Ai | AI Final Year Project Report

Complete Technical Paper Content — License Plate Recognition Ai

1. Executive Abstract, Problem Formulation & Background

Automatic number-plate recognition (ANPR; see also other names below) is a technology that uses optical character recognition on images to read vehicle registration plates to create vehicle location data. It can use existing closed-circuit television, road-rule enforcement cameras, or cameras specifically designed for the task. ANPR is used by police forces around the world for law enforcement purposes, including checking if a vehicle is registered or licensed. It is also used for electronic toll collection on pay-per-use roads and as a method of cataloguing the movements of traffic, for example by highways agencies. Automatic number-plate recognition can be used to store the images captured by the cameras as well as the text from the license plate, with some configurable to store a photograph of the driver. Systems commonly use infrared lighting to allow the camera to take the picture at any time of day or night. ANPR technology must take into account plate variations from place to place. Privacy issues have caused concerns about ANPR, such as government tracking citizens' movements, misidentification, high error rates, and increased government spending. Critics have described it as a form of mass surveillance.

Automatic (or automated) license-plate recognition (ALPR) Automatic (or automated) license-plate reader (ALPR) Automatic vehicle identification (AVI) Danish: Automatisk nummerpladegenkendelse, lit. 'Automatic number plate recognition' (ANPG) Car-plate recognition (CPR) License-plate recognition (LPR) French: Lecture automatique de plaques d'immatriculation, lit. 'Automatic reading of registration plates' (LAPI) Mobile license-plate reader (MLPR) Vehicle license-plate recognition (VLPR) Vehicle recognition identification (VRI)

ANPR uses optical character recognition (OCR) on images taken by cameras. When Dutch vehicle registration plates switched to a different style in 2002, one of the changes made was to the font, introducing small gaps in some letters (such as P and R) to make them more distinct and therefore more legible to such systems. Some license plate arrangements use variations in font sizes and positioning – ANPR systems must be able to cope with such differences to be truly effective. More complicated systems can cope with international variants, though many programs are individually tailored to each country. ANPR camera's are deployed by law enforcement agencies, on police vehicles, portable units and via mobile devices. They are mounted on community entry and egress points, key traffic areas and critical infrastructure in order to improve overall community safety. Portable ANPR equipment have been especially useful in special law enforcement operations such as retail theft, public gatherings or other similar events. The cameras used can be existing road-rule enforcement or closed-circuit television cameras, as well as mobile units, which are usually attached to vehicles. Some systems use infrared cameras to take a clearer image of the plates.

During the 1990s, significant advances in technology took automatic number-plate recognition (ANPR) systems from limited expensive, hard-to-set-up, fixed-based applications to simple "point and shoot" mobile ones. This was made possible by the creation of software that ran on cheaper PC-based, non-specialist hardware that also no longer needed to be given the pre-defined angles, direction, size and speed in which the plates would be passing the camera's field of view. Further scaled-down components at lower price points led to a record number of deployments by law enforcement agencies globally. Smaller cameras with the ability to read license plates at higher speeds, along with smaller, more durable processors that fit in the trunks of police vehicles, allowed law enforcement officers to patrol daily with the benefit of license plate reading in real time, when they can interdict immediately. Despite their effectiveness, there are noteworthy challenges related with mobile ANPRs. One of the biggest is that the processor and the cameras must work fast enough to accommodate relative speeds of more than 160 km/h (100 mph), a likely scenario in the case of oncoming traffic. This equipment must also be very efficient since the power source is the vehicle electrical system, and equipment must have minimal space requirements. Relative speed is only one issue that affects the camera's ability to read a license plate. Algorithms must be able to compensate for all the variables that can affect the ANPR's ability to produce an accurate read, such as time of day, weather and angles between the cameras and the license plates. A system's illumination wavelengths can also have a direct impact on the resolution and accuracy of a read in these conditions.

Installing ANPR cameras on law enforcement vehicles requires careful consideration of the juxtaposition of the cameras to the license plates they are to read. Using the right number of cameras and positioning them accurately for optimal results can prove challenging, given the various missions and environments at hand. Highway patrol requires forward-looking cameras that span multiple lanes and are able to read license plates at high speeds. City patrol needs shorter range, lower focal length cameras for capturing plates on parked cars. Parking lots with perpendicularly parked cars often require a specialized camera with a very short focal length. Most technically advanced systems are flexible and can be configured with a number of cameras ranging from one to four which can easily be repositioned as needed. States with rear-only license plates have an additional challenge since a forward-looking camera is ineffective with oncoming traffic. In this case one camera may be turned backwards.

2. Related Work, Literature Survey & Algorithmic Foundations

There are seven primary algorithms that the software requires for identifying a license plate:

Plate localization – responsible for finding and isolating the plate on the picture Plate orientation and sizing – compensates for the skew of the plate and adjusts the dimensions to the required size Normalization – adjusts the brightness and contrast of the image Character segmentation – finds the individual characters on the plates Optical character recognition Syntactical/Geometrical analysis – check characters and positions against country-specific rules The averaging of the recognised value over multiple fields/images to produce a more reliable or confident result, especially given that any single image may contain a reflected light flare, be partially obscured, or possess other obfuscating effects. The complexity of each of these subsections of the program determines the accuracy of the system. During the third phase (normalization), some systems use edge detection techniques to increase the picture difference between the letters and the plate backing. A median filter may also be used to reduce the visual noise on the image. Contemporary ANPR systems use multiple data sources and analytical techniques that go beyond simple number plate recognition. Weigh-in-Motion uses ANPR cameras and AI analytical techniques to calculate the weight of a vehicle and to alert if the weight is too high for the vehicle and conditions.

There are a number of possible difficulties that the software must be able to cope with. These include:

Poor file resolution, usually because the plate is too far away but sometimes resulting from the use of a low-quality camera Blurry images, particularly motion blur Poor lighting and low contrast due to overexposure, reflection or shadows An object obscuring (part of) the plate, quite often a tow bar, or dirt on the plate Read license plates that are different at the front and the back because of towed trailers, campers, etc. Vehicle lane change in the camera's angle of view during license plate reading A different font, popular for vanity plates (some countries do not allow such plates, eliminating the problem) Circumvention techniques Lack of coordination between countries or states. Two cars from different countries or states can have the same number but different design of the plate. While some of these problems can be corrected within the software, it is primarily left to the hardware side of the system to work out solutions to these difficulties. Increasing the height of the camera may avoid problems with objects (such as other vehicles) obscuring the plate but introduces and increases other problems, such as adjusting for the increased skew of the plate. On some cars, tow bars may obscure one or two characters of the license plate. Bikes on bike racks can also obscure the number plate, though in some countries and jurisdictions, such as Victoria, Australia, "bike plates" are supposed to be fitted. Some small-scale systems allow for some errors in the license plate. When used for giving specific vehicles access to a barricaded area, the decision may be made to have an acceptable error rate of one character. This is because the likelihood of an unauthorized car having such a similar license plate is seen as quite small. However, this level of inaccuracy would not be acceptable in most applications of an ANPR system.

Many countries now use license plates that are retroreflective. This returns the light back to the source and thus improves the contrast of the image. In some countries, the characters on the plate are not reflective, giving a high level of contrast with the reflective background in any lighting conditions. A camera that makes use of active infrared imaging (with a normal colour filter over the lens and an infrared illuminator next to it) benefits greatly from this as the infrared waves are reflected back from the plate. This is only possible on dedicated ANPR cameras, however, and so cameras used for other purposes must rely more heavily on the software capabilities. Further, when a full-colour image is required as well as use of the ANPR-retrieved details, it is necessary to have one infrared-enabled camera and one normal (colour) camera working together. To avoid blurring it is ideal to have the shutter speed of a dedicated camera set to 1⁄1000 of a second. It is also important that the camera use a global shutter, as opposed to rolling shutter, to assure that the taken images are distortion-free. Because the car is moving, slower shutter speeds could result in an image which is too blurred to read using the OCR software, especially if the camera is much higher up than the vehicle. In slow-moving traffic, or when the camera is at a lower level and the vehicle is at an angle approaching the camera, the shutter speed does not need to be so fast. Shutter speeds of 1⁄500 of a second can cope with traffic moving up to 65 km/h (40 mph) and 1⁄250 of a second up to 8 km/h (5 mph). License plate capture cameras can produce usable images from vehicles traveling at 190 km/h (120 mph). To maximize the chances of effective license plate capture, installers should carefully consider the positioning of the camera relative to the target capture area. Exceeding threshold angles of incidence between camera lens and license plate will greatly reduce the probability of obtaining usable images due to distortion. Manufacturers have developed tools to help eliminate errors from the physical installation of license plate capture cameras.

3. Dataset Ingestion, Preprocessing & Stochastic Augmentations

In 2012 a state consortium was formed among the Hungarian Ministry of Interior, the National Police Headquarters and the Central Commission of Public Administration and Electronic Services with the aim to install and operate a unified intelligent transportation system (ITS) with nationwide coverage by the end of 2015. Within the system, 160 portable traffic enforcement and data-gathering units and 365 permanent gantry installations were brought online with ANPR, speed detection, imaging and statistical capabilities. Since all the data points are connected to a centrally located ITS, each member of the consortium is able to separately utilize its range of administrative and enforcement activities, such as remote vehicle registration and insurance verification, speed, lane and traffic light enforcement and wanted or stolen vehicle interception among others. Several Hungarian auxiliary police units also use a system called Matrix Police in cooperation with the police. It consists of a portable computer equipped with a web camera that scans the stolen car database using automatic number-plate recognition. The system is installed on the dashboard of selected patrol vehicles (PDA-based hand-held versions also exist) and is mainly used to control the license plate of parking cars. As the Auxiliary Police do not have the authority to order moving vehicles to stop, if a stolen car is found, the formal police is informed.

The Home Office states the purpose of automatic number-plate recognition in the United Kingdom is to help detect, deter and disrupt criminality including tackling organised crime groups and terrorists. Vehicle movements are recorded through a network of nearly 13,000 cameras that capture approximately 55 million ANPR 'read' records daily. These records are stored for up to two years in the National ANPR Data Centre, which can be accessed, analysed and used as evidence as part of investigations by UK law enforcement agencies. In 2012, the UK Parliament enacted the Protection of Freedoms Act which includes several provisions related to controlling and restricting the collection, storage, retention, and use of information about individuals. Under this Act, the Home Office published a code of practice in 2013 for the use of surveillance cameras, including ANPR, by government and law enforcement agencies. The aim of the code is to help ensure their use is "characterised as surveillance by consent, and such consent on the part of the community must be informed consent and not assumed by a system operator. Surveillance by consent should be regarded as analogous to policing by consent." In addition, a set of standards was introduced in 2014 for data, infrastructure, and data access and management.

In the United States, ANPR systems are more commonly referred to as ALPR (Automatic License Plate Reader/Recognition) technology, due to differences in language (i.e., "number plates" are referred to as "license plates" in American English) Since 2019, private companies like Flock Safety have grown rapidly, promoting stationary ALPR cameras to private individuals as well as neighbourhood associations and law enforcement. By April 2022, 1500 cities across the United States had implemented Flock cameras, despite criticism from the ACLU and other civil rights organisations and concerns about whether the system actually reduces crime. In 2025, ALPR cameras are being used by ICE and ERO to search for "immigration" related activities, potentially aiding in deportation detainment. Currently, there is no need for a warrant nor restriction on how law enforcement can use ALPR data.

Mobile ANPR use is widespread among US law enforcement agencies at the city, county, state and federal level. According to a 2012 report by the Police Executive Research Forum, approximately 71% of all US police departments use some form of ANPR. Mobile ANPR is becoming a significant component of municipal predictive policing strategies and intelligence gathering, as well as for recovery of stolen vehicles, identification of wanted felons, and revenue collection from individuals who are delinquent on city or state taxes or fines, or monitoring for Amber Alerts. With the widespread implementation of this technology, many U.S. states now issue misdemeanor citations of up to $500 when a license plate is identified as expired or on the incorrect vehicle. Successfully recognized plates may be matched against databases including "wanted person", "protection order", missing person, gang member, known and suspected terrorist, supervised release, immigration violator, and National Sex Offender lists. In addition to the real-time processing of license plate numbers, ANPR systems in the US collect (and can indefinitely store) data from each license plate capture. Images, dates, times and GPS coordinates can be stockpiled and can help place a suspect at a scene, aid in witness identification, pattern recognition or the tracking of individuals. In 2014, The Department of Homeland Security proposed a federal database to combine all monitoring systems, which was cancelled after privacy complaints. In 1998, a Washington, D.C. police lieutenant pleaded guilty to extortion after blackmailing the owners of vehicles parked near a gay bar. In 2015, the Los Angeles Police Department proposed sending letters to the home addresses of all vehicles that enter areas of high prostitution. Early private sector mobile ANPR applications have been for vehicle repossession and recovery, although the application of ANPR by private companies to collect information from privately owned vehicles or collected from private property (for example, driveways) has become an issue of sensitivity and public debate. Other ANPR uses include parking enforcement, and revenue collection from individuals who are delinquent on city or state taxes or fines. The technology is often featured in the reality TV show Parking Wars featured on A&E Network. In the show, tow truck drivers and booting teams use the ANPR to find delinquent vehicles with high amounts of unpaid parking fines.

ANPR is used for speed limit enforcement in Australia, Austria, Belgium, Dubai (UAE), France, Ireland, Italy, The Netherlands, Spain, South Africa, the UK, and Kuwait. This works by tracking vehicles' travel time between two fixed points, and calculating the average speed. These cameras are claimed to have an advantage over traditional speed cameras in maintaining steady legal speeds over extended distances, rather than encouraging heavy braking on approach to specific camera locations and subsequent acceleration back to illegal speeds.

4. Neural Network Backbone, Layer Specifications & Loss Formulations

One of the most notable stretches of average speed cameras in the UK is found on the A77 road in Scotland, with 32 miles (51 km) being monitored between Kilmarnock and Girvan. In 2006 it was confirmed that speeding tickets could potentially be avoided from the 'SPECS' cameras by changing lanes and the RAC Foundation feared that people may play "Russian Roulette" changing from one lane to another to lessen their odds of being caught; however, in 2007 the system was upgraded for multi-lane use and in 2008 the manufacturer described the "myth" as "categorically untrue". There exists evidence that implementation of systems such as SPECS has a considerable effect on the volume of drivers travelling at excessive speeds; on the stretch of road mentioned above (A77 Between Glasgow and Ayr) there has been noted a "huge drop" in speeding violations since the introduction of a SPECS system.

Many cities and districts have developed traffic control systems to help monitor the movement and flow of vehicles around the road network. This had typically involved looking at historical data, estimates, observations and statistics, such as:

Car park usage Pedestrian crossing usage Number of vehicles along a road Areas of low and high congestion Frequency, location and cause of road works CCTV cameras can be used to help traffic control centres by giving them live data, allowing for traffic management decisions to be made in real-time. By using ANPR on this footage it is possible to monitor the travel of individual vehicles, automatically providing information about the speed and flow of various routes. These details can highlight problem areas as and when they occur and help the centre to make informed incident management decisions. Some counties of the United Kingdom have worked with Siemens Traffic to develop traffic monitoring systems for their own control centres and for the public. Projects such as Hampshire County Council's ROMANSE provide an interactive and real-time website showing details about traffic in the city. The site shows information about car parks, ongoing road works, special events and footage taken from CCTV cameras. ANPR systems can be used to provide average point-to-point journey times along particular routes, which can be displayed on a variable-message sign (VMS) giving drivers the ability to plan their route. ROMANSE also allows travellers to see the current situation using a mobile device with an Internet connection (such as WAP, GPRS or 3G), allowing them to view mobile device CCTV images within the Hampshire road network. The UK company Trafficmaster has used ANPR since 1998 to estimate average traffic speeds on non-motorway roads without the results being skewed by local fluctuations caused by traffic lights and similar. The company now operates a network of over 4000 ANPR cameras, but claims that only the four most central digits are identified, and no numberplate data is retained.

Ontario's 407 ETR highway uses a combination of ANPR and radio transponders to toll vehicles entering and exiting the road. Radio antennas are located at each junction and detect the transponders, logging the unique identity of each vehicle in much the same way as the ANPR system does. Without ANPR as a second system it would not be possible to monitor all the traffic. Drivers who opt to rent a transponder for CA$2.55 (US$1.92) per month are not charged the "Video Toll Charge" of CA$3.6 (US$2.71) for using the road, with heavy vehicles (those with a gross weight of over 5,000 kg or 5.5 short tons) being required to use one. Using either system, users of the highway are notified of the usage charges by post. There are numerous other electronic toll collection networks which use this combination of Radio frequency identification and ANPR. These include:

The Golden Gate Bridge in San Francisco, California, which began using an all-electronic tolling system combining Fastrak and ANPR on 27 March 2013 NC Quick Pass for the Interstate 540 (North Carolina) Triangle Expressway in Wake County, North Carolina Bridge Pass for the Saint John Harbour Bridge in Saint John, New Brunswick Quickpass at the Golden Ears Bridge, crossing the Fraser River between Langley and Maple Ridge e-TAG, all Australian toll roads FasTrak in California, United States Highway 6 in Israel Tunnels in Hong Kong Autopista Central in Santiago, Chile E-ZPass in New York, New Jersey, Pennsylvania, Massachusetts (as Fast Lane until 2012), Virginia (formerly Smart Tag), and other states. Maryland Route 200 uses a combination of E-ZPass and ANPR. TollTag in North Texas and EZ-Tag in Houston, Texas I-Pass in Illinois Pikepass in Oklahoma Peach Pass I-85 Atlanta, Georgia (Gwinnett County). OGS (Otomatik Geçiş Sistemi) used at Bosphorus Bridge, Fatih Sultan Mehmet Bridge, and Trans-European Motorway entry points in Istanbul, Turkey M50 Westlink Toll in Dublin, Ireland Hi-pass in South Korea Northern Gateway, SH 1, Auckland, New Zealand Evergreen Point Floating Bridge, Seattle, and Washington State Route 167 HOT-lanes in western Washington ETC in Taiwan SunPass In Florida

5. Training Protocol, Convergence Behavior & Experimental Evaluation

Portuguese roads have old highways with toll stations where drivers can pay with cards and also lanes where there are electronic collection systems. However most new highways only have the option of electronic toll collection system. The electronic toll collection system comprises three different structures:

ANPR which works with infrared cameras and reads license plates from every vehicle Lasers for volumetric measurement of the vehicle to confirm whether it is a regular car or an SUV or truck, as charges differ according to the type of vehicle RFID-like to read on-board smart tags. When the smart tag is installed in the vehicle, the car is quickly identified and owner's bank account is automatically deducted. This process is realized at any speed up to over 250 km/h (160 mph). If the car does not have the smart tag, the driver is required to go to a pay station to pay the tolls between 3rd and 5th day after with a surplus charge. If he fails to do so, the owner is sent a letter home with a heavy fine. If this is not paid, it increases five-fold and after that, the car is inserted into a police database for vehicle impounding. This system is also used in some limited access areas of main cities to allow only entry from pre-registered residents. It is planned to be implemented both in more roads and in city entrance toll collection/access restriction. The efficacy of the system is considered to be so high that it is almost impossible for the driver to complain.

The London congestion charge is an example of a system that charges motorists entering a payment area. Transport for London (TfL) uses ANPR systems and charges motorists a daily fee of £11.50 if they enter, leave or move around within the congestion charge zone between 7 a.m. and 6:00 p.m., Monday to Friday. A reduced fee of £10.50 is paid by vehicle owners who sign up for the automatic deduction scheme. Fines for traveling within the zone without paying the charge are £65 per infraction if paid before the deadline, doubling to £130 per infraction thereafter. There are currently 1,500 cameras which use automatic number plate recognition (ANPR) technology. There are also a number of mobile camera units which may be deployed anywhere in the zone. It is estimated that around 98% of vehicles moving within the zone are caught on camera. The video streams are transmitted to a data centre located in central London where the ANPR software deduces the registration plate of the vehicle. A second data centre provides a backup location for image data. Both front and back number plates are being captured, on vehicles going both in and out – this gives up to four chances to capture the number plates of a vehicle entering and exiting the zone. This list is then compared with a list of cars whose owners/operators have paid to enter the zone – those that have not paid are fined. The registered owner of such a vehicle is looked up in a database provided by the DVLA.

Section control, to measure average vehicle speed over longer distances Border crossings Automobile repossessions Petrol stations to log when a motorist drives away without paying for their fuel A marketing tool to log patterns of use Targeted advertising, a-la "Minority Report"-style billboards Traffic management systems, which determine traffic flow using the time it takes vehicles to pass two ANPR sites Analyses of travel behaviour (route choice, origin-destination etc.) for transport planning purposes Drive-through customer recognition, to automatically recognize customers based on their license plate and offer them the items they ordered the last time they used the service To assist visitor management systems in recognizing guest vehicles Police and auxiliary police Car parking companies To raise or lower automatic bollards Hotels Enforcing Move over laws for emergency vehicles Automated emissions testing

Vehicle owners have used a variety of techniques in an attempt to evade ANPR systems and road-rule enforcement cameras in general. One method increases the reflective properties of the lettering and makes it more likely that the system will be unable to locate the plate or produce a high enough level of contrast to be able to read it. This is typically done by using a plate cover or a spray, though claims regarding the effectiveness of the latter are disputed. In most jurisdictions, the covers are illegal and covered under existing laws, while in most countries there is no law to disallow the use of the sprays. Other users have attempted to smear their license plate with dirt or utilize covers to mask the plate. Novelty frames around Texas license plates were made illegal in Texas on 1 September 2003 by Texas Senate Bill 439 because they caused problems with ANPR devices. That law made it a Class C misdemeanor (punishable by a fine of up to US$200), or Class B (punishable by a fine of up to US$2,000 and 180 days in jail) if it can be proven that the owner did it to deliberately obscure their plates. The law was later clarified in 2007 to allow novelty frames. If an ANPR system cannot read the plate, it can flag the image for attention, with the human operators looking to see if they are able to identify the alphanumerics. In 2013 researchers at Sunflex Zone Ltd created a privacy license plate frame that uses near infrared light to make the license plate unreadable to license plate recognition systems.