2006-14 - Minnesota Department of Transportation
Contents
1. BarciendTie Dam A 13 External Data File The user can also designate Lock Station file and Flag Station File If such files are not needed leave the input blank See Figure A 14 Rim Extract Fu Bxwxme Gap en T Octo emng Extract Duss di Loc Simion Fis Balance Data Extreci Bin Dieta H Fr leis B Sear to Emon E Fie Drs Mi Bases Dala Eaarenginrea eternal Lahat da farra aiu Habari Darka sakini Lj ipa t H dana Ula A E Deia anres Lock aion F ile E Gets Graptics Bl Hie ipic oneri Figure A 14 External Data Files Setup Lock Station File By lock it is meant that the data in this file is considered to be error free hence in the subsequent balancing process these data will be left untouched while other non locked station will be modified whenever necessary by the algorithm to achieve balanced results The data in the Lock Station File could be results from other freeway balancing For example the locked station could be one of the boundary stations that are shared by two freeways If the data of such station have already been balanced for one freeway then the user may wish to lock this station when proceed to balance the other Lock Station Is useful especially when the user intends to balance loop detector station data on a network wide basis Lock Station File should be strictly in the following format A 14 Data Owverwritting Start Time 14 00 Data Owve2. 3 l l 4 l 3 3 D 1361 zT 2 zi 1i i I7 b 15 5 10 iT 1365 11 11 5 amp B B B i 1 2 20050603 3 20050603 20050603 20050603 20050603 20050603 20050603 20050603 20050603 20050603 20050603 20050603 14 20050603 15 20050603 16 20050603 17 20050603 18 20050603 18 20050603 FIX DATA After data have been successfully extracted click Next button to the Fix Data general information page This page provides general information about the Scan for Data Errors and Fix Errors functionality Click Next or select Scan for Data Errors to proceed Re Extract x Bisre Geohies Help 3 Deis Feces sing Ed Emmi baa di Losi Sapa Fin Fix Data a Exec 3in Da i L cca for Err rs F li Fu Dern Bzzniar Das Ermon MI Balance Dala Game Cais E Oats Gapics 3I Inia Campis A Hala Bl Heb innar F Fix Eno Sce ha dete for tte lich anorz cesehnlas minc da cto cal treeeholdsasimhons nd empra nues Lene Adusien Factor dl bs emp med whenanrg edimmi caidos fid sure reris ki FE the chaka iba sw Teen dele F les cen na hs lead Ir CONSE MERON OFS printed wih axcipscer detectors betes dein nom naimaca dees wal Ea ingi Figure A 6 Fix Data General Information Page Scan for Data Errors TradaX scans the loaded data for errors The user needs to specify volume and occupancy thresholds and the scanning log option in this page Then click Scan Data button to start scanni
3. PMR If the PMR of an aggregated volume or occupancy is non zero TradaX will flag such data as missing and report related information in the data scanning log Figure 3 3 As Figure 3 3 shows Detector 1351 which belongs to Station 426 at 19 00 have 3 volume missing meaning from 18 55 to 19 00 three of the ten 30 sec raw volume data are missing N Scanning Log x Ea Screen Detector Data Im Detector D ZStationI D Time VolMlissing Oechissing ThresholdVialation Locked i 1350 426 05 45 1350 426 06 00 13517426 01 30 3 3 13517425 D4 15 zs 3 13517426 18 30 3 3 13517426 19 00 3 3 1352 1352 01 30 3 3 1352 1352 14 15 bz bz 1352 1352 15 45 1352 1352 06 00 1352 1352 07 00 135271352 08 45 3 3 1352 1352 11 45 3 3 1352 1352 12 45 3 3 1352 1352 18 45 3 3 1353727427 17 30 1353272427 17 45 4 k Figure 3 3 Scanning Log Showing Partial Missing Rate for Detectors Stations Apart from the hardware reported data missing TradaX also applies the following heuristic rules to supplement the determination of missing data Note if any of the following rules are found to hold the related measurement will be flagged as 10040 missing and reported in the scanning log accordingly 1 Zero Volume with non zero occupancy 100 volume missing 2 Zero Occupancy with non zero volume 100 occupancy missing 3 Non HOV detectors report zero volume and zero occupancy for 2 consecutive aggregated time intervals 100
4. and thru movements on ramp intersection legs These percentages are applied to the balanced freeway ramp data to get the new volumes which are then locked when balancing the remaining arterial volumes Data Matching Example Assume that the collected arterial data includes 35 left turn and 87 right turn vehicles for a ramp leg Figure 5 1 a This gives the percentage of 87 _ 35 87 35 87 71 4 However the freeway ramp volume is reported to be 135 vehicles as a result of freeway balancing Therefore the matched new arterial volumes for this leg are 135x28 6 39 left turn vehicles and 135x714 96 right turn vehicles These movements are then locked while balancing the remaining arterial data Figure 5 1 b left turn vehicles as 2 28 6 and right turn vehicles as 20 EMMY Freewa s r d 135 vehicles tt w Fmm Freeway 47 M ls Figure 5 1 a Example illustrating how to match arterial intersection data to balanced freeway data before matching Epy Freewa iievehiicles wen LI mamase i a Figure 5 1 b Example illustrating how to match arterial intersection data to balanced freeway data after matching 21 5 3 DATA BALANCING In a nutshell balancing arterial intersection turning movement data means adjusting the relevant turning movement volumes so that the total traffic entering a link equals the total tr
5. those at M2 33 CHECKING METHODOLOGY Partial Missing Rate As introduced earlier raw volume and occupancy are collected on a 30 second basis These raw data are stored as 8 bit volume data or 16 bit binary numbers occupancy data If during a 30 second interval the hardware controller receives no data from the detector binary number OxFF 8 bit binary representation of 1 or OxF FFF 16 bit binary representation of 1 will be used to indicate data holes for volume or occupancy respectively It is required in many practical projects that raw 30 sec data volume or occupancy or both be aggregated into longer time interval such as 5 minutes or 15 minutes This leads to the concept of Partial Missing Rate PMR Specifically PMR 1s defined as the time percentage of the entire aggregation interval during which the measurement is missing The next example best illustrates this concept Assuming that raw 30 sec volume data are being aggregated into a 5 minute volume 1 e 10 successive 30 sec volumes are added up to yield a 5 minute volume If four out of the ten 30 sec volume data are missing 1 e they are flagged as 1 by the controller the Partial Missing Rate for the aggregated 5 minute volume will be 40 4 divided by 10 Note that TradaX computes the Partial Missing Rate for any aggregated time interval greater than 30 sec Checking Missing Data TradaX filters missing data by first checking the Partial Missing Rate
6. 5l 53 0 k 4 obs D Or LS s 118r 5p 0y 34 ue boe Ss ds Sre deb vw Hye 3g scr d que ds D 228 Mu 3f DV D UU s Du x 85 0 PO Ee gs 55y Sp Q bU 3 8 DS Du dB Jc du m d Des BB SIS e 4 20 3 55 0 U Lala 70 04 Ig 2 210 Das gs Bg he oy ul 4 Hu oe Uu W Leg y B dal d 0 08 307 40 102 1 0 43 2 43 0 Q 94 75 0 1 1 1 O OSES BL DO 2 2 10 Aas U 220 0 0 83 Gle Ur da de ly D Figure B 2 A Sample of Intersection Data File Output y Output Directory Designate output directory for the balancing results NOTE e Balancing results will be stored in the designated output directory as csv files that can be opened with Excel e One intersection will have two output files associated with it i e intersection name csv which contains integer valued balanced results and intersection name float csv which contains floating point valued balanced results Here intersection name represents the name of the intersection in question Run y Balance Data this command will execute the data balancing subroutine on the intersections NOTE Run Balance Data command is disabled unless the intersections pass geometry check to perform geometry check click Check Draw button B 4 EXAMPLE The following example illustrates in detail how to use ArtBaT program Assuming we are to balance the intersections as shown in Fig 3 We have four intersections in this figure D C B A The four in
7. Adjustment Factor for different lanes default value is 1 00 where LO is the rightmost lane L1 is the second rightmost lane etc Lane Adjustment Factor will be used when fixing missing data with adjacent detectors For example if LO volume is missing L1 volume available and LO 1 10 and L1 1 25 L1 has a volume of 100 veh 5min then LO is reconstructed by 100 x1 25 1 10 114 veh 5 min Hi Edi fir Bas aha Hak F Duss Process rg Band Darts J Leader F la Fix Data aF Eua Ein oa JB To Diels Bj sni P ram EP ae mmm Lii docet Fii a pu LIU i311 ml iz if lili m ir i ni mc B Cie LHE A Lua krash mi Eie apd 1 3 E Feke Kaman ba Pinar e De A ni am Figure A 9 Fix Data Page Reference Dates for Missing Data The two reference dates are for correcting missing data 1 e when the missing data can not be corrected by adjacent detector value or using conservation computation historical data from these two dates whichever is available multiplied by the Daily Adjustment Factor default value is 1 00 will be used as substitute for the missing one Default Value When the missing data can not be fixed by adjacent detector conservation computation or historical data the default volume value will be used to fill the data holes Advanced Options By default TradaX will fix 24 hour raw data for all the detectors listed in the station file However the user may not desire a 24 hour
8. C Bickel P Chen C Kwon J Rice J Zwet E and Varaiya P 2004 Measuring Traffic http www stat berkeley edu users rice 664 pdf accessed at Oct 15 2005 FHWA 2004 Traffic Analysis Tool Box Volume III Guidelines for Applying Traffic Microsimulation Modeling Software Report No FHW A HRT 04 040 23 Appendix A Freeway Traffic Data Filtering and Balancing Tool TradaX Version 2 0 User Manual SOFTWARE OVERVIEW INTRODUCTION TradaX Traffic Data Analysis is a Windows software package designed and developed at the University of Minnesota for filtering correcting and balancing freeway traffic data It can automatically identify freeway geometry and retrieve traffic data in UTDFF format filter and correct data faults and reconcile spatial discrepancies of traffic volumes It also provides rich graphical functionalities and flexible data output options to facilitate data examination and outputs TradaX integrates empirical procedures in conjunction with a validated ARIMA Auto Regressive Integrated Moving Average model to filter and correct common temporal data errors while a non linear optimization algorithm has been developed and implemented in TradaX to balance the identified gaps of traffic counts on a system scale INSTALLATION System Recommendations CPU Memory gt 1GHz Operating Systems 256MB 512 MB Preferable Resolution Windows ME NT 2000 XP 1280 x 960 Preferable Installation Directi
9. Leg Data Input A Left Turn Movement Thru Movement Fight Turn Movement Durectiorr Data Source Col Number E Leck Right Turn Movement Figure B 7 Configuring First Intersection Step 7 ConFigure B Intersection Legs Take SB Leg of the first intersection as an example SB Leg Left turn movement y Select Data Source File from the file list in this case select CSAH 73 at No Frontage Rd txt Figure B 11 y Input 1 in Col Number as the first data column of CSAH 73 at No Frontage Rd txt data file corresponds to SB left turn movements Figure B 11 y Check Lock Left Turn Movement if SB LT is supposed to be locked during the balancing process y Do the same for other movements for SB leg y Do the same for other legs and other intersections NOTE e The intersection configuration can be saved by SaveToFile button and existing intersection file can be loaded by LoadFromFile Button B 9 e For a turning movement that doesn t actually exist the user should designate its Data Source File as empty and Col Number as 0 e When it is desired that some turning movements counts should not changed during the balancing the user must designate such movements as LOCKED by checking the respective Lock XXX turn movement checkbox Figure B 8 x Intersection Information Basie Info Intersection Mame CSAH73 at North Frontage Rd Intersection Type Intersection Data Optiorr Cl
10. Scan tor Errors H Fix Data Rl Balance Data ar Balance Data Date Graphics E Data Graphics z Mj Help Help Contents Max Figure A 4 a Browse an Existing Station File A 4 P Dai Procaszing El Extract Linim Ig Lear Sion Fle 4 Exkart Ein Date kd F p Daa Eg San tor Enor lbi Fix Date a Balance Dats EJ Dein Graphics TE Cata Graphics Hi Hep al Help Contents Figure A 4 b Select an Existing Station File Look im O Staton Fie v mer En m a TH34ER ITN tipo The contents of the selected station file is shown in TradaX To proceed the user needs to click Load for TradaX to establish data then Click Next button to next page Dsi Processing Estrect Debs P Edad Gm Dals fel Fix Dena H Scantor Emors B Es Dieta E Daae Goin A Eoln Ons Deis Grephes EJ Com Graphics a Help ll Figln Cankanis Figure A 4 c Contents of Selected Station File A 5 Load and Extract Binary Data Browse and Load Binary Traffic File Once the station file has been loaded 7TradaX needs to know about the location of the binary traffic data file Use Location button to navigate and select the required binary file Extraction Options Time Interval After select appropriate binary file the data aggregation interval should be designated the default value is 15 minutes 1 e Tradax will aggregates 30 sec raw data into 15 min interval data Station Occupancy TradaX extracts data In
11. fixing and may wish to fix the data for a 2 hour period with only a few detectors This can be accomplished via Advanced Options To setup desired time period for fixing and specify detectors to be fixed click Advanced button See Figure A 10 M Advanced phi lar Firing Daka 5 elected Lint Figure A 10 Advanced Options for Fixing Data Start Fixing Click Fix button to start fixing the detected data errors A Fixing Recommendations Dialogue will pop up See Figure A 11 The user needs to approve these recommendations for TradaX to implement them Click Select All button to approve all at once or check each individual recommendation one by one Click Accept to approve the checked recommendations 28 The following list gives recommendations far fixing data Check to confirm Actions to take Detector 1352 Stall 1352 at 18 45 3 vol missing fixed using rescaling Horn 15 ta 15 Detector 1353 StalD 427 at 17 30 acc outliers replaced with ARIMA fitted value from 17 to Detector 1353 5talD 427 at 17 45 acc outliers replaced with ARIMA fitted value from 28 ta 10 Detector 1353 StalD 427 at 18 00 occ outliers replaced with ARIMA fitted value from 30 to 17 Detector 1353 StalD 427 at 18 15 acc autliers replaced with ARIMA fitted value Horn to 22 Detector 1353 5talD 427 at 18 30 acc outliers replaced with ARIMA fitted value from to 16 Detector 1353 5talD 427 at 21 15 3 vol missing fired using rescali
12. ied mt etex tor Stop F iltering j Missing Dala Figure 3 4 Filtering Missing Data Flow Chart 10 Filtering Temporal Outliers In order to filter temporal outliers the ARIMA p d q model was selected in this study and fitted using field data to determine the best parameters Specifically the ARIMA p d q model can be represented as X 0 X X Fun t6 X 0 05 O T E 1 where 1S a constant X represents the observation at time t 0 is the autoregressive parameter n 1 2 p 0 is the moving average parameter n 1 2 q e is White noise random error at time f ww 0 07 p is the number of autoregressive terms q is the number of moving average terms d is the order of differencing needed to make the time series stationary It should be noted that the ARIMA model only applies to stationary time series 1 e for time series x the mean value E x is constant and the covariance E X Xn is only dependent on the time lag h In this study the ARIMA model represented by equation 1 was fitted to historical traffic counts and occupancy time series to determine the appropriate p d and q values The historical data were measured at 10 randomly selected detectors from the Twin Cities freeway network Note that traffic counts and occupancy measurements are non stationary as they clearly have time dependent tendencies In order to produce stationary time serie
13. sensors are corrected in the balancing stage to be detailed next Correcting Temporal Outliers Detected temporal outliers are corrected using the predicted value generated by the ARIMA model as discussed in preceding sections Start Correcting NO v Multiply sibling s vol by a predefined adjustment factor Use the adjusted vol as sensor j s volume Missing Data Set J 0 Is Sensor j flagged as have missing data partial Missing rate 100 YES Is sensor j s sibling flagged as Vol Missing San Sensor j be fixed by Vol Missing YES NO NO YES conservation v Fix Sensor j using conservation principle predefined defaul v T Fix Sensor j usin t values Fix Sensor j using historical data NO partial Missing m 100 YES Is sensor s sibling flagge as Occ Missing NO v Use siblings occ asa substitute for sensor j s OCC A lt j j 1 v Fix Sensor j by recaling NO CO Is sensor j the las sensor Stop Filtering Missing Data Figure 3 6 Flow Chart for Correcting Missing Data 14 4 RECONCILING METHODOLOGY 4 1 IDENTIFICATION OF SPATIAL DISCREPANCIES Once the temporal errors have been processed throughout the dataset ju
14. srapha rs indak porem ders comect anc habnrces loop deiecipr dra Sele mn dem nthe bi p perkarrm an actin Bl Help Contents arral Buskillaka Capia 1E RE Veran Hu ED 13 335 Copan Lrives diy of Winnie at Ten Danes Poss rare e Frei Herc H H HHB EEUU Told Mor MEK Tate From Perec lente EK Fess Figure A 2 Tree View of Actions List TradaX organizes its functionalities via a tree view list located in the left part of the interface The tree view has three father nodes Data Processing Data Graphics and Help corresponding to three major functionalities provided by TradaX A 2 Data Processing includes e Data Extraction e Data Fix e Data Balance Data Graphics can generate graphics for Raw Data Fixed Data and Balanced Data while Help provides on line help for the user TIPS The user can use either the Tree View List or the Previous Next button to navigate through user interfaces EXTRACT BINARY TRAFFIC DATA F Extat Fu Baawce Grapes Help Extract Binary Traffic Data a Exact Bin Dota Je Fu kain mt Scan for Errors Bj Fi Dein eg Load Stminn Fila Il Baan Das E Balance Desa Staton File dere he D and geometinc lout of detector simiong B Daia Graphics Selaci nn easing Sahan Fila ar came rara nna wah SaF ie Editor E Dais Graphies Bj Halp Heip Comente Fact na Tre Data imeta dein im In LITE hinan tomal Select this karika mirer ihabinare usc cela Figure A 3 Extract Binary Traffic Data Gen
15. terms of individual detector as well as stations Station occupancy is derived either from averaging detector occupancies or taking highest detector occupancy Extract Button Click Extract button to start extract data Save Options The extracted raw data can be saved as csv file either in terms of detectors or stations The data can be in row format or column format Data Pracassing El Exact Dern al Loci Staion File Load Binary Traffic Data L2 Emai and Load Eris Taie Fe Enia Coin e dran Dai Tiree rien Faria E r m laraphics JAS UH Linda Pies T E Help a Fg Locis Ei Cirip TEE MAn Hur aei Lice 4 5 Highiedi Datictor E 7 8 99 ird i2 I3 14 15 16 17 1E 1B 3 2 2 2 A 28 25 zm 3 23 xl Gaee Dae SOHO Pise Eak un he cere to check tee rem puse dae Figure A 5 Load and Extract Binary Traffic Data An example of saved data in Row Format mum B C D E F G H I J T etertonD 015 cas 1 00 1 15 1 30 1 45 2 00 215 230 z 1 1351 eB a ni n 51 21 23 D 47 47 j 1381 el 3 ad ao 41 26 2B dd 20 13 4 1332 2 2 1 d 2 3 1323 LE eo as 2e GT 24 Zh ze 41 1354 3T zb 31 35 24 20 12 19 IB I3 fr 1352 B 10 3 x 3 g B z B 13585 BB 31 amp 3 i J4 a6 6B 42 4j 1357 3T ES a 2 M 21 24 23 2l 12 I 1238 ri zT l E l3 l1 41 IT 13 l4 I 1338 BH ea ae ad 17 dn a ak dn zT 12 1360 32 a0 2 21 Z3 ig 23 25 15 n 1d 1348 14 1358 zg 35 25 I3 3 E a0 34 B l 15 1117 B
16. volume missing and 100 occupancy missing Note that HOV detectors are not included in Rule 3 as it is highly probable for HOV detectors to have zero measurements during multiple data collecting periods Further it should be stressed that Rule 3 is very likely to generate a false alarm during non peak periods e g from midnight to 4 00 a m 1n the morning In this case the user is strongly suggested to examine the Data Scanning Log and visually check verify the reported missing data during non peak hours using the Data Graphics function Figure 3 4 gives the flow chart for filtering missing data Filtering Locked on Sensors and Threshold Violations TradaX considers the data as locked on if both volume and occupancy have non zero values and remain identical unchanged for two consecutive data aggregation intervals Furthermore if the data exceed user defined threshold values TradaX will flag the data as violating thresholds Siar Filtering Mitsing D ata STRIS Te T arial E ip Siraq rata o ea a tm I l Vo D actos HO T EB M cd WAL and mizaiug a rt 1 a E8 s aa cuti va data celiscdian terse a e ob j and N D und U se GEC UT 1 VOL 07 7 D Flep o piacir we 100 OL Missing 100 occ HMissamg Flagged ag Flag a3 Flag as ZA M h TEG Miang 10900 VOL M is sing ToS OCC Missing M Q aE Tt
17. 2006 14 Final Report otreamlining of the Traffic Modeling Process for Implementation in the Twin Cities Freeway Network Phase ll WANESOz YAN xh Or TRE Q 2 b Wt b P U2J8e9oso Technical Report Documentation Page Report No 25 3 Recipients Accession No MN RC 2006 14 4 Title and Subtitle 5 Report Date Streamlining of the Traffic Modeling Process for May 2006 Implementation in the Twin Cities Freeway Network Phase II i 7 Author s 8 Performing Organization Report No Wuping Xin John Hourdos Panos Michalopoulos 9 Performing Organization Name and Address 10 Project Task Work Unit No University of Minnesota Department of Civil Engineering 11 Contract C or Grant G No Minneapolis MN 55455 0116 12 Sponsoring Organization Name and Address 13 Type of Report and Period Covered Minnesota Department of Transportation 395 John Ireland Boulevard Mail Stop 330 14 Sponsoring Agency Code St Paul Minnesota 55155 15 Supplementary Notes http www lrrb org PDF 200614 pdf 16 Abstract Limit 200 words Comprehensive methodologies are proposed for improving the quality of both freeway and arterial intersections traffic volumes for the purpose of enabling and improving traffic simulations Specifically established and enhanced procedures for checking and correcting freeway temporal errors are integrated with an optimization based algorithm for reconciling spatial inconsi
18. A TradaX 2 0 User Manual APPENDIX B ArtBaT 1 0 User Manual List of Figures Figure 3 1 Traffic Volumes Temporal Outliers 7 Iipure 3 2 Spatial Discrepancies uu yu y EN ER uay q Suk tus EN 7 Figure 3 3 Scanning Log Showing Partial Missing Rate for Detectors Stations 9 Figure 3 4 Filtering Missing Data Flow Chart 10 Figure 3 5 a Traffic Volume Outliers 12 Figure 3 5 b Z Statistics for Traffic Volumes 12 Figure 3 6 Flow Chart for Correcting Missing Data 14 Figure 4 1 Numerical examples illustrating the data balancing algorithm 17 Figure 5 1 a Example illustrating how to match arterial intersection data to balanced freeway data before matchinzg 21 Figure 5 1 b Example illustrating how to match arterial intersection data to balanced freeway data after matchine 21 Executive Summary In recent years micro simulation has become an increasingly indispensable tool in many demanding ITS and planning applications In order to build reliable and realistic simulation models high quality input data are required including roadway geometry vehicle driver characteristics traffic volumes composition and others Among these d
19. XT file format can be edited using notepad exe 2 Line 1 to line 10 inherited format from JAMAR file format carrying no specific meaning to ArtBaT program Line 11 to the last line must be comma separated containing 1 time stamp column i e the 1st column and 16 data columns i e 2nd to 17 columns For those turning movements related to freeway entering exiting ramps ArtBaT assumes the user has already conducted necessary data matching splitting See Section 2 2 i e the data file to be loaded by ArtBaT are the already matched data 5 The user is suggested in the strongest term to examine the JAMAR data file prior to inputting required column numbers 6 Total number of data lines that ArtBaT can handle is 96 this is based on the assumption that the data are on 15 min basis and 24 hours contain ninety four 15 min intervals B 3 ZACHARY LM HOPKINS CROSSROAD fo Southbound LOOP FROM i RAMP TO I 394 noO0O00000 5 23 2001 06 00 I 384 AT ZACHARY LAME PCSAH F 3 E RAMPS REF PT 001 946 TAMARZ 531 532 RE DR TURN MOVEMENT COUNT Movement 1 AUTO amp TRUCK Key X 1 2 3 4 5 6 7 amp 9 10 11 12 13 14 15 16 D5 00 18 11 1 0 8 3 16 0 O 14 12 O0 1 2 1 0 Up 302223 Spe xpo due dus D 221 ele ds a he Hos Horan s ZSB cBU 30 Lu 6 DoD S Me DS 1 Js 1 0 D62545 54 45 20 104 2B do aay UU Ua Bd 48 D 249 u 0 OP EO i eee SLE 3 Ud nk o Gi D Qu
20. affic exiting the link This is conducted by proportioning the extra link volume i e the volume difference between the total inflow and total outflow of a link among any movements that can be changed Specifically the formula used in this process 1s v the balanced volume of movement j in direction where v the raw volume data for movement j in direction Av the volume differential between turning count data sets for a specific link 22 REFERENCES 1 10 11 12 13 14 Jacobson L N Nihan N L and ender J D 1990 Detecting Erroneous Loop Detector Data in a Freeway Traffic Management System Transportation Research Record 1287 pp151 166 Chen L and May A D 1987 Traffic Detector Errors and Diagnostics Transportation Research Record 1132 pp82 93 Payne H J and Thompson S 1997 The I 880 Database Malfunction Detection and Data Repair for Induction Loop Sensors Annual Meeting Transportation Research Board Washington D C 1997 Rick Schmoyer Patricia S Hu and Richard T Goeltz 2001 Statistical Data Filtering and Aggregation to Hour Totals of Intelligent Transportation System 30s and 5 min Vehicle Counts Transportation Research Record 1789 pp79 86 Srinvas Peeta and loannis Anastassopoulos 2002 Automated Real Time Detecting and Correction of Erroneous Detector Data Using Fourier Transforms for On line Traffic Control Architectures Annual Meeting Transport
21. als or simply averaging over different days at each measurement station without checking conservation Discrepancies may also arise from the inconsistent projection of the base year demands into future years As mentioned earlier spatial discrepancies of traffic counts have not been well addressed in previous studies 3 CHECKING AND CORRECTING METHODOLOG Y This section presents the checking and correcting methodology implemented in TradaX 3 1 UNIFIED TRAFFIC DATA FORMAT The Minnesota Department of Transportation Mn DOT has one of the largest traffic data collection systems in the United States This system consists of more than 4000 loop detectors collecting traffic volume number of vehicles and occupancy percentage of time a detector is occupied by a passing vehicle data in real time every 30 seconds As the benefits of storing this large amount of data in a traditional database are far offset by the accompanied complications Mn DOT finally turned to a Unified Traffic Data File Format UTDFF In this format raw traffic data 1s stored as 8 bit volume data or 16 bit binary numbers occupancy data This simple and compact format greatly facilitates everyday data storage retrieval as well as development of data analysis tools such as TradaX In UTDFF data format each detector has two data files associated with it 1 e volume file v30 and occupancy file c30 The volume file v30 is a flat binary file with 2880 byt
22. ata volumes are of major importance yet they are hard to obtain even when collected with advanced surveillance systems they are still susceptible to problems such as miscounting gaps in time and space and other inaccuracies More importantly even when the data appear to be accurate frequently they do not balance out 1 e they are inconsistent in terms of maintaining conservation throughout the system Inevitably these problems could lead to anomalies or considerable errors during the simulation seriously tainting the reliability and accuracy of its outputs while weakening the credibility of the resulting conclusions In this study systematic methodologies have been proposed to check correct common temporal errors for freeway volumes These methodologies also include rigorous methods for reconciling both freeway volumes and arterial turning movement counts Specifically the freeway data processing methodology can handle common temporal data errors including missing data locked on data threshold violations temporal outliers as well as spatial discrepancies Established procedures in conjunction with a fitted ARIMA model are used to check and correct common temporal freeway data errors while an optimization based approach is proposed to identify and reconcile spatial gaps in freeway traffic counts on a system wide scale In addition to this an empirical methodology is also integrated to balance arterial intersection traffic counts The
23. ation Research Board Washington D C 2002 Sherif Ishak 2003 Quantifying Uncertainties of Freeway Detector Observations Using Fuzzy Clustering Approach Annual Meeting Transportation Research Board Washington D C 2003 Cleghorn D Hall F L and Garbuio D 1991 Improved Data Screening Technique for Freeway Traffic Management Systems Transportation Research Record 1320 pp17 13 Turochy R E and Smith B L 2000 A New Procedure for Detector Data Screening in Traffic Management Systems Paper No 00 0842 Annual Meeting Transportation Research Board Washington D C 2000 Nguyen L N Scherer W T 2003 Imputation Techniques to Account for Missing Data in Support of Intelligent Transportation Systems Applications Research Report No UVACTS 13 0 78 May 2003 University of Virginia Conklin J H Scherer W T 2003 Data Imputation Strategies for Transportation Management Systems Research Report No UVACTS 13 0 80 May 2003 University of Virginia Turner S M Eisele W L Gajewski B J Albert L P and Benz R J 1999 TTS Data Archiving Case Study Analysis of San Antonio TransGuide Data Report No FHWA PL 99 024 Aug 1999 Texas Transportation Institute Chen C Kwon J Rice Jl Skabardonis A and Varaiya P 2002 Detecting Errors and Imputing Missing Data for Single Loop Surveillance Systems paper presented at 82 Annual Meeting Transportation Research Board Jan 2003 Washington D
24. ced with ARIMA fitted value fram 41 to 21 Detector 1356 StalD 428 at 18 15 acc outliers replaced with ARIMA fitted value from 8 to 32 Detector 1357 5talD 428 at 01 30 32 vol missing fed using rescaling Horn 21 ta 21 Detector 1357 StalD 428 at 04 15 B vol missing fired using rescaling fram 15 ta 17 Detector 1357 5talD 428 at 09 15 3 val missing feed using rescaling From 222 to 228 Detector 1357 StalD 428 at 17 30 acc outliers replaced with ARIMA fitted value from 40 to 8 v Save As Log Select All Accept Cancel Figure A 11 Fixing Recommendations A 12 BALANCE DATA Balance Data General Information Page rdg in Fe Edat Fic Bina Cabo Hop Duis Poza req El cist 0m ill Load io Fie Balance Dala F Extract Ei Ootn il rr ris H Sosa ter Emon E Deecnwic Emlsryce the rai dss en finc heim culi mes ere cone rient hicughcut WE ts Grephire Whe stem ii arrsa k of mainisinin CG Su remcion i elo il Hali cones Figure A 12 Balance Data General Information Page Balance Data ConFigure A uration Balance Interval The user can designate the balancing interval e g the user may wish to balance data from 3 00pm to 6 00pm See Figure A 13 F Don rocening p od Fin Balance Data P Exec Bin Deis u inm Emon Balancing Corka Lund Balarced Liata cand HalascregIniereal Estona Dua Fla waringin ait pen Tra dare Pea Tad M Han Wl Help Comers Diciro Shad Tra E 5
25. d to the outlier detection problem mostly in other engineering domains In the context of traffic a Fourier based framework has been recently developed to detect outliers in real time traffic measurements 5 for the purpose of on line control In addition a fuzzy clustering approach was also applied in identifying and correcting outliers in archived traffic data 6 Once faulty data are identified they are excluded from the data set or corrected either empirically or using statistical methods For example Payne and Thompson 3 used historical traffic information and the data from adjacent detectors to replace data holes while Turner et al 11 used regression and time series models Chen et al 12 used lane to lane and location to location correlations to impute missing data In addition if the faulty measurement was determined to be a temporal outlier its model predicted 5 or smoothed value 6 was used as a replacement Apart from the aforementioned errors frequently traffic volumes do not balance out i e they are inconsistent in terms of maintaining traffic conservation throughout the system In such cases large discrepancies exist between traffic counts upstream and downstream of specific locations These spatial discrepancies may result from any of the aforementioned errors storage or discharge of queuing vehicles on the freeway traffic sinks or sources not accounted for using data collected on different days or time interv
26. decisions and erroneous adjustments of model parameters during calibration Inevitably these problems could lead to anomalies or considerable errors during the simulation seriously tainting the reliability and accuracy of its outputs while weakening the credibility of resulting conclusions Specifically traffic volume data required in simulation include freeway volume data and arterial intersection turning movements counts For freeway volume data practical procedures and guidelines have been proposed in the literature for reviewing and correcting common temporal errors 1 8 however most of them are not intended for simulation purposes while a unified systematic methodology is still lacking for checking and correcting temporal data errors in conjunction with reconciling spatial gaps in traffic counts Likewise frequently arterial intersection turning movement counts are susceptible to errors of spatial gaps yet there seems no well established methodology existing in the literatures as to reconciling such gaps for simulation purpose In this study systematic methodologies have been proposed to check correct common temporal errors for freeway volumes These methodologies also include rigorous methods for reconciling both freeway volumes and arterial turning movement counts The proposed methodologies have been successfully automated and incorporated into two computer programs called Freeway Traffic Data Analysis Tool TradaX and Arte
27. design and performance of roadway facilities as well as for implementing innovative ITS technologies engineers are now increasingly relying on sophisticated high fidelity microscopic traffic simulators rather than conventional approaches Recognizing simulation s effectiveness and improved realism the Federal Highway Administration FHWA recently introduced a mandate requiring comprehensive simulation analysis to be conducted for all new Interstate Access Requests IAR prior to actual implementation In this regard high quality input data are essential for building realistic and accurate simulation models such data include the roadway geometry vehicle driver characteristics traffic volumes composition and others Among these data traffic volumes are of major importance as they are the starting point for generating traffic demands and developing traffic control scenarios and serve as essential inputs for model calibration However reliable traffic volume data more often than not are hard to obtain even when collected with advanced surveillance systems they are still susceptible to problems such as miscounting gaps in time and space and other inaccuracies caused by malfunctioning sensors More importantly even though the data may appear to be accurate frequently they do not balance out i e they are inconsistent in terms of maintaining traffic conservation throughout the system resulting in inaccurate traffic demands unrealistic control
28. eral Information Page Prior to fixing and balancing traffic data must be loaded extracted into TradaX memory Figure A 3 shows the general information page of Extract Data functionality The user can click the link on the page the action node in the tree view or Next button to proceed Load Station File Before starting to extract data a station file needs to be loaded Station files are ASCII files with sta extension It provides a convenient way describing the spatial layout of detector stations that are deployed on the freeway A station file can be easily created edited with Station File Editor or any text editor such as notepad exe as long as it complies with the prescribed data format An example of a station file 426 M 2 1350 1351 1352 X 1 1352 427 M 2 1353 1354 1919 H 1 1919 Each line in station file describes name and type of the detector station mainline entrance ramp exit ramp HOV ramp the number of detectors in the station and IDs of each composing detector For instance 426 M 2 1350 1351 describes detector Station 426 which is a mainline station with 2 detectors The IDs of the two detectors are 1350 and 1351 respectively Click Browse to browse and select an existing station file See Figure A 4 Fie Extract Fix Balance Graphics Help F Date Processing Bl Extract Data f fLoad Station File Load Station File Extract Bin Data JH Fix Data Stetion Fie ht
29. erval field measurements for counting Stationl 7 are as follows Figure 4 1 y 20 veh y 20 veh y3 170 veh 17 A storagel 10 veh y4 730 veh ys 95 veh A storage2 15 veh yg 40 veh y 40 veh A storage3 10 veh Furthermore assuming Station 3 and Station 7 are flagged as threshold violations and temporal outlier respectively Thus we have the following optimization problem min x 20 x 20 x3 170 x 30 xs 95 x 40 x 40 Subject to X1 X5 X3 10 Conservation Constraints 4 x4 x xs 15 ka he X19 Non negative Constraints x 20 for i212 7 D f y t Precision Constraints jd m it lt 3 fori z3 iz7 Ji Relaxed Constraints x lt x lt Solving the above optimization problem yields the optimized solutions Puatai Xa ky 22 22 54 28 97 44 03 18 5 ARTERIAL BALANCING METHODOLOGY This section describes the underlying methodology employed in the ArtBaT program Specifically this methodology involves three stages data normalization data matching and data balancing 5 1 DATA NORMALIZAION Intersection volume data sets are collections of 15 minute volume for individual turning movement e g 50 left turn vehicles 100 right turn vehicles 276 through vehicles etc collected over a 3 hour time period As the number of intersections requiring turning movement counts usually exceeds the available ma
30. es In length Each byte is an 8 bit signed integer corresponding to a 30 second vehicle count Within each volume file the integer value of the first byte represents the first 30 second vehicle counts of the day 1ie from midnight to 12 00 30 am while the last value corresponds to the final 30 second vehicle counts of that day e from 11 59 30 to midnight A negative value OxFF indicates missing data this OxFF is generated by hardware controller if it receives no data from detector The occupancy file 030 is akin to volume files except that each value is a 16 bit signed integer This means each file 1s 5760 bytes in length As with the volume file a negative value OxFFFF indicates missing data In the end all the detectors volume and occupancy files are zipped into one single file on a daily basis This single file is conventionally named with an 8 digit date followed by an extension of traffic For example 20031105 traffic would contain all the detectors volume and occupancy data of Nov 5th 2003 3 2 CLASSIFICATION OF DATA ERRORS TradaX identifies and corrects data errors from two categories common temporal errors and spatial discrepancies 1 Common temporal errors refer to missing data locked on sensors contradictory data threshold violations and temporal outliers Such errors are called temporal because they are associated with the time series of the measurements at each individual detector 2 M
31. formulation is to determine the most plausible actual values x when the field measurements y are given without materially affecting the actual trends in the traffic patterns The objective is to minimize the total measurement errors while the conservation constraints ensure that the final optimizer x i 1 2 N complies with the conservation concept This means that sta any discrepancies in traffic counts are implicitly resolved during this optimization process The relaxed constraint x e holds for station i flagged as problematic in the previous stage of checking and correcting temporal errors or for stations with adjusted projected counts This Is necessary otherwise the errors associated with a specific counting station will be distributed over other healthy stations As stations diagnosed as healthy could still have inherent small random errors the constraint t y I x E iW p is added to reflect this where p represents the desired Yi measurement precision of the related detectors stated otherwise it is also an indicator reflecting the analyst s desired confidence level based on professional judgments Numerical Example Aen nacen zm z Surat Storage i Storage H LN F i I d 1 am i u m sh E Jj I m 5 E I i Figure 4 1 Numerical example illustrating the data balancing algorithm Assuming during a certain time int
32. ge Ad txt Browse Data File i AIL si e SL it SI EE Peery E MM 2 T m gt JUN Ji Direction M orth Sou th Figure B 5 Intersection Data Files Loaded Step 4 Click North South radio to tell the program that the balancing is for north south direction iL Step 5 Click 4 times dH icon and we have Figure B 6 The intersections are lined from top to bottom B 7 File Data Qutput Run Help About Draw Function rection North South Check Draw C East w est SL 7 i ni E i 7 S T M Figure B 6 Four Intersections in the Drawing Area Step 6 Click the top 1 intersection we have the configuration window as shown in Figure B 7 y Input Intersection Name An intersection MUST be associated with an UNIQUE name B 8 W File Data Output Helo About al Intersection Information f bo Basic Info x Intersection Mame NONE Intersection Type TA Intersection Data Option al Click LoadFromFile to load an existing lt data configuration file or configure the 7 E intersection data now by designate the LoadFromF ile correspondence between tuming movement j and data source file column number X d zc Click Saved oF ile to save the configuration Savel oF ile 20 that pou can load it nest time SB Leg Data VB Leg Data MB Leg Data EB Leg Data il i EB
33. ick LoadFromFile to load an existing data configuration file or configure the intersection data now by designate the comespondence between turning movement and data source file column number Click SaveT oFile to save the configuration Save oF ile so that you can load it next time SB Leg Data VB Leg Data MB Leg Data EB Leg Data SB Leg Data Input Left Turn Movement Thru Mavement Right Turn Movement Data Source ADoecuments and Settingsswuping zEBRASBesktaps5 fo at Ma Frontage Ad tst Col Number C Documents and Settinge swuping zZEBRASD esktop CSAH 73 at 394 Ho Ramp t t C Documents and Settingssvwvuping zEBRA SDesktepsC5AH 73 at 384 So Ramp tet C Documents and Settingssvwuping zEBRASDeskrtapsCS5AH 73 at So Frontage Ad tet Figure B 8 ConFigure B SB Intersection Leg B 10 SB Leg Data WE Leg Data MB Leg Data EB Leg Data SB Leg Data Inpu Left Turn Movement Thru Movement Right Turn Movement Data Source C Documents and Settings wuping EBRA Desktop CS 4H 7 3 at Ma Frontage Ad tet Lol Number Lock Left Turn Movement Figure B 9 ConFigure B SB Left Turn Movement Step 8 After all intersections are configured Fig 13 click Check Draw to check the geometry x Arterial Traffic Balancing Tool File Data Output Run Help About ah CS 4H 73 at North Frontage Ad CSAH FS at 1394 North Ramp Lit A CS4H FS at 1394 South Ramp JiLs S CSAH 3 at South Fro
34. igure A 17 Zone ID D Differenc 10 difference 2 425 427 425 427 425 427 425 427 425 427 425 427 425 427 4257427 4257327 4257327 4257327 425 427 426 427 425 427 425 427 425 427 426 427 425 427 Mir 2477 3 Figure A 17 Balancing Log A 17 ZoneID The ZoneID is a combination of upstream and downstream station IDs for the freeway section has these two stations as endpoints I O difference 1 This 1s the difference of total input volume and output volume before the balancing I O difference 2 This is the difference after balancing TradaX always balances I O difference 2 to zero A warning message will be put to NOTE if I O difference 1 exceeds the user defined threshold Balance Outputs The user can specify the output format for the balanced data See Figure A 18 Note Output Time Interval is the same as the Balance Interval which can only be modified through Balancing Configuration page Re Esiet Fe Een maka Hep F Doo Pr oun El Ezirsst Cents lli Lcad Sesion Fik Balance Data d purs Cari Laks Je Fix Dato 5 r Chil Ealarcad Cis y Scen kr Eron Ealerz reg riwni Dupul lma rima zii T inis 9 Dats Graphics End To Aman Bl Hop Contents p Dan Fama r Coin Forsa a Foy Fond vea d Figure A 18 Balance Outputs Setup A 18 Example of Column Format including Balanced Fixed Raw Volumes Exa
35. in the data set Threshold violation tests refer to the comparison of traffic measurements with prescribed upper and lower bounds For example the lower upper bounds for traffic volume used in Maryland are based on historical data while in Minnesota a lower bound of 20 veh 5 min per lane and an upper bound of 250 veh 5 min per lane are used 2 Lock on sensors refer to situations where traffic measurements remain identical for several consecutive data collection intervals When this occurs the values are 1n most cases Zero but runs of values other than zero could also occur Schmoye et al 4 provided an empirical method for determining the feasible run length 1 e number of collection intervals of identical values If the number of time intervals with identical values became greater than a pre determined run length the data were considered to be invalid Another type of data error occurs when traffic volume and occupancy measurements contradict each other e g zero volume with non zero occupancy or zero occupancy with non zero volume Payne and Thompson 3 identified this as inconsistent data and proposed a test for its detection Temporal outliers refer to suspicious observations in the time series of traffic measurements Usually they are in the form of abrupt increases or decreases inconsistent with the general pattern Neural Network models Genetic Algorithms and Autoregressive Integrated Moving Average models ARIMA have been applie
36. issing Data refers to situation where a detector fails to report any data within some time duration This is sometimes called data holes in the data set 3 Lock on Sensors means traffic measurements both volume and occupancy remain identical values for several consecutive data collection intervals When this occurs the values were most often zero but cases of values other than zero also are also frequent When the identical value is zero TradaX considers such situation the same as data missing 4 Contradictory Data refers to the situation where traffic volume and occupancy measurements contradict each other e g zero volume with non zero occupancy or zero occupancy with non zero volume TradaX considers contradictory data the same as missing data volume or occupancy missing 5 Threshold Violation means the collected traffic measurements exceed reasonable upper and lower bounds For example 300 veh 5 minutes lane is an out of bound value as it entails an average flow rate of 1 veh sec for the entire 5 minute period Different thresholds can be prescribed in practice to test the data In Maryland the lower upper bounds for traffic volume are based on historical data while in Minnesota a lower bound of 20 veh 5 min per lane and an upper bound of 250 veh 5 min per lane are used 6 Temporal outliers refer to suspicious observations in the time series of traffic measurements Usually they are in the form of abrupt increases or decrea
37. me stamps for notational purposes line 5 Data line which must be comma separated the first item must be the station ID other iteme are the volumes The file may have more data lines depending on the number of flagged stations Comment lines are not parsed by the program The Data Line MUST be comma separated Kz K m LN B 5 w where Courts Ic uh E A uc F patty Tom er ecce 3 Kosa DE m E BD Uus wu ge wc Oe Wes EUG 1080 sm 190 4E T27D EIE Te PF31825 HSH se 225 MP PB Pk y Figure A 15 An unusual data error that can not be properly filtered A 16 Warning Setup The user can active the Warning Message Setup for balancing This means a warning message will be generated if the zone balance 1 e the difference between total input volume and output volume of a zone exceed the user defined threshold See Figure A 16 Fie Extract Fe Byee Gree Help Ces Processing El Extract Cats ill Load Station Fla Balance Data JP Evae Elin Cate SJ Pre Inis A Santor Emirs Bis Fie nts Bil Bainne ntn Balancing aswel Estera Diis File Warring Sep Exncing Coige Dipi Bade DIA s jw ur ERT ll Dais Garita B Dats Grephics Wiring Vazrage Lordi Et z PU Hup ah i UM Hele Cann Zone balsis amp obras Figure A 16 Balancing Warning Setup Start Balance Click Balance button to start balancing A Balance Log will pop up when the program completes balancing See F
38. mple of Row Format includes Balanced Volumes only PES m kn Lo aa sdi LA E ES m EE s E k a Si sss E mE E 4 EF m yu d 3 4 B d ii H W mW Hh WH um Q SR s J om T g sa c SSS aos s DEM m A 19 DATA GRAPHICS Data Graphics General Information Page Fk Extract Fh Balea Grapes Hap F Dens Processing Ee Bact Das dl Lood Sinion Fie Data Graphics aS Extacd Ba l simi Fix Dein Mt Scan for Erma te Fe Oeste li Bolonce Dai inim Graphics Bininnpa inia f TTE Linpicts heretic dete geephacsily induding rm dats ined dels and E Deas Graphics balancoj data These gnsphice can hel identity raic panier probe Wine meunc amp omed duluciors and damonsirstc the chock oltha fang and P balancing Figure A 19 Data Graphics General Information Page Data Graphics Plot Page Select Station ID or Detector ID from the Tree View List specify Time Interval then click Plot button Tradit Fle Extrxt Fw Balance Gapo Help 3 Des Processing Gl Extract Data dip Losd Staten File Data Graphics d Extract Gin Dais M Fie Cate Cats hisp Be Sean few Errore B Fre esis B Balance Dats ar Balance Daia Ij Os Gephice E Date Graphics Y Help B Help Contents Figure A 20 Data Graphics Plot Page A 20 Data Plot The data pl
39. mps or drops of traffic counts at all nearby adjacent mainline counting stations are checked in sequential order 1 e from upstream to downstream Specifically for every freeway section between two successive mainline counting stations the difference between total input e total traffic counts at the upstream counting station and entrance ramps and output volumes 1 e total counts at the downstream counting station and exit ramps is computed for every data collection interval If the discrepancy denoted as AC exceeds a reasonable threshold then this discrepancy is considered large enough and needs to be reconciled In order to determine the allowable discrepancy threshold AC the concept of traffic conservation 1s considered t At t At t At o AL faxu Dd XE la x D4t faxa dt Y atx t dt t ee E t t exe X t Xd X 4 k x t At dx k x t dx 2 X X u u where Ar is the data collection interval e g 5 min or 15 min E is the index set for counting stations at entrance ramps X is the index set for counting stations at exit ramps q x 1 q xq D q x D q x4 t are the flow rate at upstream downstream entrance ramp and exit ramp counting stations respectively k x t 1s traffic density of the freeway section delimited by upstream and downstream counting stations The left hand side of Equation 2 1s equivalent to the difference between total 1nput and output during interval Az which equals AC
40. ng Fe Extat Ay Balance Graphics Hep Date Procasaing Exact Duis ai Lond Simon File af ExtrmctBin Des JA Fi Coin Mf Scmnidor Errore ES Fe Onis BI Balance Daia ir Balance Deja Dein Graphics E Dem Grephics Scan for Data Errors haare Satu H ra aae X Sean i5 mnn 5 RT 15am W Hap 8 Help Contents SranLag pore F Heng Dade E Lociarion loista fe Thesthok viaiia F Tempor Cure Figure A 7 Scan for Data Errors Page A 8 Threshold Setup The default volume and occupancy thresholds are 900 veh 15min and 99 15min respectively The user can setup different threshold values through the edit box Outlier Detection Model In current version of 7TradaX a fitted ARIMA model is implemented to detect temporal outliers Scan Log Options When data scanning is completed a log that lists detected errors will pop up See Figure A 8 The user can customize the log output by specifying which type of error he wishes to scan for via Scan Log Options The scan log can be saved as csv file A csv file can be opened with Microsoft Excel or any text editor such as notepad exe for later reference 3 8 Den Pipes E EI Exkrart nin x Sransinglng Y orania Ems EZ Fie Data B Balance Dein amp r Balance Dei Ki Dises Drap his BB Oss Gieptics Help al Hele Canzntz Figure A 8 Scanning Log Listing Detected Errors Fix Data Page Lane Adjustment Factor Input Lane
41. ng from 151 to 155 Detector 1354 5talD 427 at 01 30 325 vol missing Fixed using rescaling Horn 20 to 2 Detector 1354 StalD 427 at 04 15 B vol missing fixed using rescaling fram 15 to 15 Detector 1354 StalD 3427 at 06 45 vol outliers replaced with ARIMA fitted value fram 301 ta 185 Detector 1354 StalD 427 at 08 45 32 val missing feed using rescaling fram 273 to 281 Detector 1354 StalD 427 at 17 30 acc outliers replaced with ARIMA fitted value from 15 to 7 Detector 1354 5talD 427 at 17 45 acc outliers replaced with amp RIMA fitted value from 23 to 10 Detector 1354 StalD 3427 at 18 15 aec autliers replaced with ARIMA fitted value from ta 18 Detector 1354 5talD 427 at 13 45 3 vol missing feed using rescaling from 117 ta 120 Detector 1354 5talD 427 at 21 15 3 val missing fed using rescaling Horn 118 to 121 Detector 1355 StalD 1355 at 01 30 3 vol missing fixed using rescaling from 4 ta 4 Detector 1355 StalD 1355 at 08 45 3 vol missing fixed using rescaling fram 62 to 63 Detector 1355 StalD 1355 at 15 30 acc outliers replaced with ARIMA fitted value from 10 ta 5 Detector 1355 5talD 1355 at 17 30 vol autliers replaced with ARIMA fitted value from 88 ta 153 Detector 1355 5talD 428 at 01 30 325 vol missing feed using rescaling from 53 ta 60 Detector 1355 StalD 328 at 17 30 acc outliers replaced with ARIMA fitted value fram 40 ta 3 Detector 1355 5talD 428 at 17 45 acc outliers repla
42. npower turning movements are seldom collected on the same day or even the same month Therefore turning movements should be first normalized adjusted to match the month and day of the balanced freeway data should monthly daily adjustment factors be available To normalize arterial turning movement data the following formula is used Vol _ Vol xM x D M x D 7 where Vol Raw arterial volume Vol Normalized arterial volume M Monthly adjustment factor for the month when the arterial data was collected M Monthly adjustment factor for the month of the balanced freeway data D Daily adjustment factor for the day when the arterial data was collected D Daily adjustment factor for the day of the balanced freeway data Data Normalization Example Assume that on a Wednesday in May 100 thru vehicles were counted at a certain intersection approach while the balanced freeway data are given for a Tuesday in September In addition the adjustment factors are given as Monthly adjustment factors May 0 95 September 0 85 Daily adjustment factors Wednesday 1 10 Tuesday 1 15 In order to match the date of the freeway data the normalized arterial data are computed as ES 100x 0 95 x1 10 2107 0 85 x1 15 5 22 DATA MATCHING The balanced arterial data must match the balanced freeway ramp volume This 1s achieved by using the raw turning movement volume to determine the percentage of left turn right turn
43. ntage Ad AE I R DS Silt ba CN 1 i Draw Function gt Direction e North South Check Draw CO Eastwest Figure B 10 Configuration of all intersections done B 11 Step 9 Because we have not designate output directory path the geometry check fails See Figure B 11 w Check Geometry ERROR Output directory path empty NOTE The program doesn t check the correspandence between data source and column number The user should ensure all the correspondence is legal before run banancing command i eometry Check Failed Figure B 11 Check Draw Information Dialogue Step 10 Input Output Directory Path through Output gt Output Directory command Figure B 12 and check geometry again If passed Run Balnace Data command will be enabled Step 11 Click Run Balnace Data and the balanced results from North to South and from South to North balancing will be output to the designated directory Step 12 The user can save the balancing scenario through Filet Save Scenario Saved scenario includes the following information 5 Number of intersections Intersection Type for each intersection Intersection data files as loaded from Datat Intersection Data File Files in the selected directory J RA UMN Linda Project Arterial Balancing ArtBaT Selected Directory JAA UMA Linda Project 4rternal Balancing 4tb aT Figure B 12 Designate Output Directory Path B 13
44. ons The installation package includes user manuals samples and executable program Two types of installation are available Beta Version 1 Select a working directory and unzip TradaX zip file Windows Registry will not be modified 2 Double click TradaX exe to start the program 3 To uninstall delete all the files in the working directory Release Version 1 Before installing it is suggested to close any running Windows programs 2 To install new TradaxX click setup exe and follow the screen instructions 3 To uninstall from START menu under TradaX directory click UNINSTALL the uninstall program will automatically remove the software from the computer USER INTERFACE TRADAX INTRODUCTION PAGE Tandag Fh Pura Pe Bare Daphni Dub z E Deis Grephics t a Hele Trafic Data Analysis Toci Tradaz pinn Wan caesos and balances ip danariti dac SHerlar iem mi Hee bef da parer ar achat Bui nad Basie hats Ephesi AS daran n 202 16206 nmnrigni Lire mi rr HE Cuni T Accuses Pres dbrxs SPP Toth EHE Trad Furr Laney CLK aa c fud Figure A 1 Tradax Introduction Page When starting TradaX exe TradaX introductory page will first show up This page provides general information of the program including built date and system resources information TREE VIEW OF ACTION LISTS File Extract Fe Esen 3 EERO e El Fstmri ints c JU Fix Dade Traffic Data Analysis Tool BB Pin nre Dais M Cots
45. ot includes plots for Raw Data Fixed Data and Balanced Data See Figure A 21 PsJ Data bragi nr SEstinn 4258 1 mines isterral f Doarea s oe kichun 2 pumice _ B B 8 8 Ea x Mw hree M Fico Cupseno y Figure A 21 TradaX Data Plot ON LINE HELP The user can start on line help See Figure A 22 by click Start Help Engine on the Help Content Page EIL LIE ELS Esai in Tempoa Errors Presi Spatial Emos Pricey Cope Freeway Traffic Data Filtering aad Balancing Tool Tria Version 8 ET d Copyigho 2004 2005 Figure A 22 TradaX On Line Help A 21 Appendix B Arterial Intersection Data Balancing Tool ArtBaT 1 0 User Manual OVERVIEW Figure B 1 gives a snapshot of the main interface of ArtBaT As is shown in this figure ArtBaT interface has four major components 1 Geometry Type Library 2 Drawing Area 3 Draw Function Keys 4 Balancing Direction Selection Radio Button It is through these components that the user interacts with ArtBaT to balance arterial intersection data Geometry Type Library Geometry Type Library contains all the possible arterial intersection types that could be encountered in practice By clicking the geometry type icons the user can build a certain layout of several intersections So far ArtBaT program can handle up to 17 types of intersections Drawing Area Drawing Area 1s where the user builds and configures intersection geometries Up
46. proposed methodologies have been successfully automated and incorporated into two standard Windows programs called Freeway Traffic Data Analysis Tool TradaX and Arterial Data Balance Tool ArtBaT In a nutshell TradaX provides a user friendly interface comprehensive data processing and graphical functionalities These functionalities automate the data pre processing tasks required in practical simulation projects These tasks used to be tedious and error prone In addition the arterial intersection traffic balancing methodology is automated and implemented in the ArtBaT program ArtBaT can build user defined intersection layouts and is capable of handling up to 17 types of intersections More importantly ArtBaT 1s designed to be capable of interfacing with 7radaX and the JAMAR intersection data collector JAMAR collector is commonly used by traffic engineering professionals to collect arterial intersection data The flexible interface of ArtBaT greatly facilitates data preparation for Interstate Access Request IAR projects which require both balanced freeway as well as adjacent arterial intersection data Initial evaluations of these tools suggest that they have the potential of reducing total modeling time by 25 30 while resulting in improved calibration of simulation models more reliable analysis and better use of staff resources for meeting project deadlines 1 INTRODUCTION In response to the growing need for improving the
47. rial Data Balance Tool ArtBaT Initial evaluations of these tools suggest that they have the potential of reducing total modeling time by 25 30 while resulting in improved calibration of simulation models more reliable analysis and better use of staff resources for meeting project deadlines 2 BACKGROUND The methodologies proposed in this study concentrate on improving the quality of traffic volume data with the objective of enabling and improving traffic simulation This includes the integration of established procedures dealing with common temporal errors with a new optimization based algorithm reconciling spatial discrepancies To be sure checking and correcting temporal volume errors e g missing data threshold violations or suspicious outliers are not new and have been widely studied in previous research e g 9 10 11 12 13 However spatial discrepancies of traffic volumes do not seem to have been adequately addressed there 1s no guidance in the literature on precisely what constitute a large discrepancy nor does an effective methodology seem to exist for checking and reconciling system wide discrepancies 14 Existing procedures for checking temporal volume errors include tests for missing data threshold violation locked on sensors data inconsistency and temporal outliers Missing data refers to a situation where a detector fails to report any data within some time duration This is sometimes called data holes
48. rwritting End Time 15 00 Number of Locked Stations stalbp 14 900 14 15 14 30 14 45 15 00 355 300 350 325 340 332 where Line 1 Comment line for the start time of the locked data Line 2 Start Time of the locked data Line 3 Comment line for the end time of the locked data Line 4 End Time of the locked data Line 5 Comment line for the number of locked stations Line 6 Number of locked stations Line 7 Comment line thie line includes the time stampe for notational purposes Line amp Data line which must be comma separated the first item must be the station ID other items are the volumes NOTE 1 The file may have more data lines depending on the number of locked stations 2 Comment lines are not parsed by the program 3 The Data Line MUST be comma separated NO TAB are allowed in the file Be cautious The program is VERY sensitive when parsing the Load Station File 4 It is stressed in the strongest term that ONLY Boundary stations i e upstream freeway entrance station and ramp entrance exit stations can be lock stations The program can still generate outputs when non boundary stations e g mainline stations are locked but the user should be warned that the results are not reliable 5 It is VERY important that the number of data items in the data line match the number of time stamps determined by start time and end time For example start time 14 00 and end time 15 00 with 15 minutes time inter
49. s from these measurements the square root was calculated followed by differencing until stationary time series were obtained After some experimentation the ARIMA J 1 1 model was found to be most appropriate for modeling the square root of the traffic count and occupancy time series for all the detectors The fitted ARIMA model produces a prediction of the likely actual measurements at each time interval based on the preceding measurements At any time interval the difference between the predicted value and actual observation produces a residual A measurement is labeled as outlier if the standardized residual exceeds a specified confidence level For instance any standardized residual value greater than 1 96 would indicate a highly probable temporal outlier with 9596 confidence level Figure 3 5 illustrates an example of traffic sensor volume measurements loop detector No 982 on I 94 August 3 2000 In this figure three suspicious outliers are indicated by the arrows Figure 3 4 also includes the z statistics of the residuals after the fitted ARIMA model has been applied It is clearly depicted in the figure that the 11 three suspicious outliers correspond to high z statistics exceeding the critical residual value Volume veh 5 mirn a 160 A alle da s yoo Lg OR L kala NENNEN kk LU a dw 0 oq fN C 4 I 04 N 4 5 W TN T VN O 00 3 00 6 00 4 00 12 00 16 00 18 00 21 00 Time Figure 3 5 a Traffic Volume Ou
50. ses inconsistent with the general pattern See Figure 3 1 7 Spatial Discrepancies refers to the situation where traffic volumes do not balance out ie they are inconsistent in terms of maintaining traffic conservation throughout the system See Figure 3 2 In such cases large gaps exist between traffic counts upstream and downstream of specific locations These spatial discrepancies may result from any of the aforementioned errors storage or discharge of queuing vehicles on the freeway traffic sinks or sources not accounted for using data collected on different days or time intervals or simply averaging over different days at each measurement station without checking conservation Discrepancies may also arise from the inconsistent projection of the base year demands into future years Volume vehi5 mir ma 160 Md td vo Le PI I es a a Xv Dhu a C FER o y x N 47 Im i lc J4 TE 0 00 3 00 600 900 1200 16 00 1800 21 20 IL Figure 3 1 Traffic Volume Temporal Outliers Three temporal outliers are indicated by arrows Figure 3 2 Spatial Discrepancies For spatially correlated traffic detector stations total input volume should approximate total output volume during a given time period As is shown In Figure 3 2 M1 M2 M3 are three mainline detector stations E1 and E2 are two entrance detector stations If traffic volumes are balanced vehicle counts at M1 E1 should approximately equal to
51. stations under study including stations on mainline entrance and exit ramps as N At counting interval f for the counting station i let y r denote raw field measurement of traffic counts x r denote the true measurement of traffic counts ande t the measurement error Then y r can be expressed as y t x amp j 0 i21 2 Ny 5 The measurement errors e r in Equation 5 may be in relation to errors discussed in preceding sections inconsistent adjustments between different days of or small random miscounting errors due to unknown reasons Note that counting stations are deployed in certain spatial arrangement this triggers conservation constraints i e for spatially related counting stations on a given freeway section the difference between input and output equals the change of vehicle storage during the specified counting interval In order to reconcile large discrepancies in traffic counts while at the same time minimize the measurement errors the following optimization formulation is used minimize Y 0 Y y 0 x 01 Vt 6 S t Conservation Constraints as In Equation 2 x t 20 i 1 2 N a 16 pu a f p for station i diagnosed as healthy in the data Yi filtering stage where p is prescribed parameter x oo for station i flagged as erroneous in the data filtering stage or has been adjusted projected using historical data The objective of the above
52. stencies in freeway traffic counts In addition to this an empirical methodology is further integrated to balance arterial intersection traffic counts The proposed methodologies have been successfully automated and implemented as two computer programs i e TradaX for processing freeway volume and ArtBaT for arterial intersection traffic counts Initial evaluations of these tools suggest that they have the potential of reducing total modeling time by 25 30 while resulting in improved calibration of simulation models more reliable analysis and better use of staff resources for meeting project deadlines 17 Document Analysis Descriptors 18 Availability Statement Traffic simulation microscopic No restrictions Document available from simulation freeway model National Technical Information Services traffic measurements detector Springfield Virginia 22161 counts detector malfunction 19 Security Class this report 20 Security Class this page 2 No of Pages 22 Price Unclassified Unclassified 67 Streamlining of the Traffic Modeling Process for Implementation in the Twin Cities Freeway Network Phase ll Final Report Prepared by Wuping Xin John Hourdos Panos Michalopoulos University of Minnesota Department of Civil Engineering May 2006 Published by Minnesota Department of Transportation Hesearch Services Section 395 John Ireland Boulevard MS 330 St Paul Minnesota 55155 This report represents the resul
53. tersection data files involved are 1 CSAH73 at No Frontage Rd txt corresponding to Intersection D 2 CSAH73 at I394 No Ramp txt corresponding to Intersection C 3 CSAH73 at I394 So Ramp txt corresponding to Intersection B 4 CSAH73 at So Frontage Rd txt corresponding to Intersection A O 394 EB south Frontage Road Figure B 3 Example Intersections Step 1 Start ArtBaT exe Step 2 Designate Data Files through Data gt Load Intersection Data File See Fig 4 Step 3 Load the four intersection data files See Figure B 5 and click Save 1 CSAH73 at No Frontage Rd txt corresponding to Intersection D 2 CSAH73 at I394 No Ramp txt corresponding to Intersection C 3 CSAH73 at I394 So Ramp txt corresponding to Intersection B 4 CSAH73 at So Frontage Rd txt corresponding to Intersection A B 5 x Load Intersection Data File x List of Intersectian Data Files Browse Data File Function Check Draw Figure B 4 Load Intersection Data Files Dialogue B 6 File Data Output Run Help About c Load Intersection Data File List af Intersection Data Files Ji CADocuments and Settingerwuping E BAA DesktopsCSAH 73 at Ma Frontage Ad tst C Documents and Settingsswuping zEBRASDeskteps L5 AH 73 at 1394 Mao Ramp tet C Documents and Settingsswuping zEBRASDesktepsCSAH 73 at 1394 So Ramp tet C Documents and Settingswiuping zZEBRA SDesktepsC5AH 73 at So Fronta
54. the right hand side of Equation 2 can be approximated from k x t At kx 0 xq x k x t At k x t Ax 2a where Ax is the distance between upstream and downstream counting stations Using the relationship between occupancy measurement and traffic density O 3 L C where O is occupancy measurement K is density L is average vehicle length C is the detector length Equation 4 can be simplified to AC K t At K t x Ax 4 15 where K t and K t Ar are traffic densities approximated from Equation 3 Equation 4 gives the allowable upper bound for the discrepancies of traffic counts i e if the computed discrepancy exceeds the bounds defined by l KG At K t x Ax then such a discrepancy is considered to violate traffic conservation therefore needs to be reconciled However it 1s important to clarify that if the counting interval is sufficiently long or the analyst is only interested 1n homogeneous state of traffic where traffic density is assumed to be uniform 1 e time and space invariant then AC 0 meaning that the total input should always equal to the total output of the freeway section NOTE In the current version of 7TradaX AC is hard coded as 0 meaning TradaX always balance traffic volume to zero 42 RECONCILING SPATIAL DISCREPANCIES The identified large discrepancies in traffic counts are reconciled in this step First denote the number of the counting
55. tliers z tatizti acta 1L bg Nn di LUE AE LE me P ral eu IR Lu PESE s ci Mie Let Cpu PEP SIRE ERA SER RH LULA qe H i n Om Time Interval S5 minmn from Midnight Figure 3 5 b Z Statistics for Traffic Volumes 34 CORRECTING METHODOLOGY Correcting Missing Data The flow chart for correcting missing data is depicted in Figure 3 6 As is shown in the flow chart detectors that have been flagged with a specific Partial Volume Missing Rate smaller than 100 will be fixed by rescaling This means that true measurement values will be reconstructed by rescaling raw data according to the partial missing rate For example if a measurement of 50 veh 5 min is flagged as uz 83 veh 5 min cs 40 volume missing the reconstructed volume will be I Further detectors that are flagged as 100 VOL Missing will be fixed using adjusted sibling adjacent detectors in the same station detector s volume or conservation computation if siblings are not available or applying historical data if neither of the above works Similarly 100 OCC missing will be fixed using sibling measurement or historical data Note that in case historical data is also missing predefined default value will be used to fill the data holes Correcting Locked on Sensors and Threshold Violations Threshold violations are corrected using prescribed upper or lower bounds for the related measurements Locked on
56. to SIX intersections of different types can be balanced using ArtBaT program Draw Function Keys e Clear Draw Clear the entire drawing area e Undo Last Delete the last intersection e Refresh Refresh the drawing area Check Draw Check the geometry configuration before run balancing data command B 1 Balancing Direction Selection Radio Button e Select relevant radio button to designate the balancing direction sles Fal Fol rs hod d bad oe cr aan eis ss Arterial Traffic Balancing Tool File Data Output fun Hep About Geomery Type Library Drawing Area Dae Funclices Clear Draw Drawing Func Keys Felresh Oat South _ Balancing Direction Selection Radio Button Se Figure B 1 Snapshot of ArtBaT User Interface B 2 MENU ITEMS ArtBaT has six menu items File Data Output Run Help and About File y Load Scenario Load existing balancing scenario y Save Scenario Save current configurations as scenario sco y Exit Exit the ArtBaT program Data NOTE A scenario file consists of the following information e Number of intersections to be balanced e Intersection types for each intersection e Intersection Data Files y Load Intersection Data File This command loads intersection data file NOTE 3 4 Intersection Data File must comply with the following format which is the same format as JAMAR file See Figure B 2 1 T
57. ts of research conducted by the authors and does not necessarily represent the views or policies of the Minnesota Department of Transportation and or the Center for Transportation Studies This report does not contain a standard or specified technique The authors and the Minnesota Department of Transportation do not endorse products or manufacturers Trade or manufacturers names appear herein solely because they are considered essential to this report Table of Contents INTRODUC TION Hr 1 2 BACKGROUND rutru uquti utqa CIVI DIMExF TEE POERI es 3 3 CHECKING AND CORRECTING METHODOLOGY 5 3 1 Unified Traffic Data Format 5 3 2 Classification of Data Errors 6 3 3 Checking MethodolOoby e a ua assaka NEA UEPE aans 8 3 4 Correcting Methodolosy 13 4 RECONCILING METHODOLOCGY 15 4 1 Identification of Spatial Discrepancies 15 4 2 Reconciling Spatial Discrepancies 16 5 ARTERIAL BALANCING METHODOLOCGY 19 5 1 Data Normalizationm u y erre rete EROR E ERI ERE FE E ayasa 19 5 2 Data NIQDIChRIB6012uuk u aa akukuy bai eia RUE EN ERES IM OO 20 5 5 Data Balancing uuu u vinee eoe su uyu REN E OPER AU 22 REFERENC E Xu au utu uy mu EOD EN 2 MR 23 APPENDIX
58. vals would give 5 time stamps 1 e 14 00 14 15 14 30 14 45 15 00 If the data line includes 6 data items e g 300 350 325 340 332 360 then this data line 1s inconsistent with the time stamps This situation should be strictly avoided Flag Station File This function is added because during the data fixing stage certain data errors can not be properly filtered and corrected for example the kind of errors illustrated in the following Figure A 15 This kind of errors is unusual and can only be discovered through visual checking However errors like this have to be corrected in order to ensure the credibility of balancing results The data in the Flag Station File will be used to replace the recommended fixing proposed by the program when the user determines the latter is not reliable Note the TradaX program will modify the Flag Station data whenever necessary to achieve balanced results Flag Station File must comply with the following format A 15 Flag Data Overwriting Start Time 14 00 Flag Data Overwriting End Time 15 00 Number of Flagged Stations L 2 2 55 14 45 25 00 where Line 1 Comment line for the start time of the flagged data Line 2 Start Time of the flagged data Line 3 Comment line for the end time of the flageed data Line 4 End Time of the flagged data Line 3 Comment line for the number of flagged stations Line 5 Number of flagged stations Line 7 Comment line thie ine includes the ti
Download Pdf Manuals
Related Search