Peak Converter User Manual - University of Minnesota Duluth
Contents
1. ko koi n koi ki koi L L ko L ko ko koi WE GAG KL KA HA HE Be PP a G EE OD GL GGG GGG GGG Se C C LA C C C HA CS C oS m n m 18 n m ko n kl 18 ko aek w ko n 8 wu wa 18 n 8 n D D C C C C HA C D D C Ou koni lt m ko koni koni n koni koni koni ci koul koni koni ko kon koni lt koni 3 C C C C C C KA C pp Oo a aa e a aae ea Kha a a aae e e e a e e PTT TTT Se C C AA C C C HA CS C HAA Ti N O O DO C O CHA C C HA C C HA C O D O O O O C C Yd LA KAJ PAJ Gn MA E KA J L pie CLA C Ko C C C MN LA C Ca 3 DO O DO C3 C3 C C3 oOo 2 O O C C C C C C K LC Kom LC ZO L L L Ko C L D C3 O DO C3 C3 C C3 C C3 Ko C Figure 5 Classification file example with lane by lane True format 3 Peak converter taps and buttons The functions are implemented using four taps as shown in Figure 6 The functions are described based on each tap Eg Peak Binary Converter V1 6 updated 9 28 2008 Seles Destination Folder Crtesth sciD ata Browse folder Figure 6 A screen shot of Peak Binary Converter Conv File Tap This tap is designed for examining individual binary files and contains two processing buttons First the user must specify the Destination Folder before using the two functions in the tap Please do not use the CSV archive folder since running individual files is likely a simple test The files produced by each button are stored in the Destination Folder Convert Peak Bi2. BH Rawesy DO000S 2 000028 000051 2 000053 2002 2003 T L 2004 T L 2005 ES 2006 T L 2007 T L 2008 ias DO E E E Figure 3 Structure of Rawcsv archive 2 CSV Files All CSV files conform the following filename convention yyyymmdd 000 ext yyyy year mm month dd day 000 three digit numeric site ID Three zeros are for future expansion ext three extensions used are vol for volume cls for classification and spd for speed bin data It is important to note that Peak binary files may contain a number of possible combinations of data for example some files contain vol and spd data and some files contain vol spd and cls data In the CSV format each file contains only one type The data type selection is done automatically by the software based on the priority of the following type order 1 cls 2 spd and 3 vol For example if the binary data contains vol and spd data only spd data are present in the CSV data Volume data 1s still not lost since it can be computed by adding the spd bin data If all three types of data are present in the Peak binary data only cls data is saved since the cls data has the highest priority Again volume data can be easily derived from the cls data Each CSV file consists of two header lines followed by data The first header line describes the site and data information It consists of five fields sitelD numOfLanes dataType
3. Raw folder which contains site folders The site folders have a name that consists of 12 digit numeric numbers and houses all binary data for that site The first six digits of the site folder name represent the site number but the actual site number used by Mn DIT is the three digits used in the middle Under each site folder data is further separated into month year folders In this documentation the path to the Raw folder is referred to as the Peak binary archive folder The Rawcsv folder is the folder structure created as the output of this utility using the binary data in the Raw archive The basic structure is shown in Figure 3 In this case the raw csv archive name is Rawcsv and consists of site folders The site folder names are now reduced to six digits This change was made to avoid the long 12 digit names that are often arduous to type in and easy to make a mistake Under each site folder the CSV data are separated into year folders The year folder contains all CSV files of that year of the site C3 Class T L Raw T L Rawosy Figure 1 Folder structure of the Peak binary data archive at UMD data center Raw 171 DoDODSODDDOD 00001 7000000 L oooozsonooon 2 000051000000 O o00052000000 0o00053000000 T L DL 2002 L 1 2003 T L 1 2004 T L 1 2005 T L 1 2006 gt 1 2007 1 2008 Hp oF 2002 T LT 2 2003 T L 2 2004 DI E 80 Figure 2 Structure of binary raw data archive
4. 08 August data is processed Note that Convert only the selected month automatically includes the year restriction Conversion of the whole archive In order to convert the whole archive fist the source folder must be changed to the Peak binary archive folder 1 e Source Folder ZO Class A an Destination csy Archive fcsy root folder Z Class A awicey start from Year 2008 Month Convert only the selected month Convert only the selected year In the above example Z Class Raw is the Peak binary archive folder and Z Class Rawcsv is the CSV archive Next simply click the Convert the Whole Archive Overwrite button Convert the Whole Archive Uvenarite In the above example the binary data in every site for year 2008 Aug files are converted and saved to the Rawcsv folder Again the CSV site folders are automatically created if they do not exit so there is no need to manually create CSV site folders Caution Converting the whole archive can take a long time since the archive size 1s very large It presently contains about 160 000 binary files in 5 355 folders Therefore it is strongly recommended that Convert only the selected year is always checked 4 Log Files There are two functions for log files and it is important to read the log file after each processing Cleanup Log File vlew Log File It is important to clean up the log file often since its file size can grow s
5. Peak Binary Converter User Manual Version 1 6 Peak Binary Converter V1 6 updated 9 78 7008 Conver a Sroigfe Peak Binary Fife to ASCH CSV Format Destination Folder Ctest4eciD ata Browse folder Convert Peak Binary Convert Peak Binary ta ta ASCIL Inel Headers ASCIL CSV Format By Prof Take M Kwon Transportation Data Research Laboratory TDRL University of Minnesota Duluth Peak Binary Converter User Manual By Prof Taek Kwon This utility program translates Peak binary files into easy to read ASCII files with a Coma Separated Value CSV format The acceptable binary formats are Peak volume speed and classification files However this utility cannot convert the Peak vehicle record PVR files PVR files can be converted using the BullConverter utility provided to the WIM group 1 Prerequisite In order to correctly use this utility program it is important to understand the basic file structure of the Mn DOT raw data archive The basic structure is illustrated using the directory structure implemented at the UMD data center The classification data which Mn DOT calls Piezo data and UMD data center calls Class data consists of the Raw and Rawcsv folder as shown in Figure 1 All peak binary data are stored in a single folder called Raw and has the structure shown in Figure 2 Notice that the root folder of the Peak binary data in this case is the
6. date and lane by lane each separated by a comma The format is described using an example shown in Figure 4 S1telb 054 num fLanes 2 dataType spd date 20070114 lane by lane False Time 0 40 45 50 55 60 65 70 75 80 865 100 111 00 00 0 01 00 02 00 Fe D J C C C FR e 6 66 6 a 666 a 66 La 6 6 6 6 6 6 6 6 6 5 KA J C C C C C C C Ko C m H E H H E m wm m E i a wm m E wm m E wm m E l E C3 C3 C3 C3 C3 C3 C3 Ko C3 C3 H E m H E m H E m E 5 C5 E m E E m E wm m E l O C3 C C KO C C Ko Ko C A H E m H E m H E m i ci wm m E wm m wm m E i C C HA C C3 C C C KO Ko C m H E m H E m H E H E 5 C5 wm m E E m E E m E l E C3 C3 C3 C3 C3 C3 C3 Ko C3 C3 5 wm m H E m H E m E i ci wm m wm m a wm m E l L E C3 C3 C3 C3 C3 C3 C3 C3 C3 C3 LC LD LC C C3 Ko C C C C GO G C L C C3 C3 K C C C C3 C C C A KA KA C KA UJ PA W KA KA C C KA C C C C C C C C KA C EP KO C3 C3 C3 C3 Ko Ko Ko Ko K s C A KA MC Ln EKA LM E C C KA KA C C C O C O C C C C C C AKA A Ln KA E GO UJ C KA AJ KA KA C KA C C3 C C orm C3 C C C KA HA C E E mono EC KAJ CAJ CA KA C EA ee E T ia 8 RAAT ee C3 C3 C C C C C C CAJ E Aa C3 C3 C3 KO C3 C3 C3 C3 C3 C3 Ko poo C3 C3 C3 KO C3 C3 C3 C3 C3 C3 Ko K 5 C3 C3 C3 KO C3 C3 C3 C3 C3 C3 C3 poo C3 C3 C3 KO C3 C3 C3 C3 C3 C3 Ko K 5 C3 C3 C3 C3 C3 C3 C3 C3 C3 C3 C3 poo Oo Figure 4 CSV data format The si
7. e Unly New Files The difference between theses two buttons is whether to overwrite the existing converted files or not The Convert Specified Site Folder Overwrite button overwrites existing converted files while the Convert Specified Site Folder Only New Files button does not overwrite if a converted file already exists For using these functions it is important to set the Source Folder to a site directory as shown in Figure 7 Source Folder Class Raw 0001 28000000 Destination csy Archive csy root folder Z Class A awcsy Note that the destination folder should always be the CSV archive folder The software automatically creates the site folders if they do not exit Which month or which year to process depends on the selection and year entry Start from Year 2008 Honth Convert only the selected month l Convert only the selected year The Start from Year text box specifies the year the user want to start processing If both check marks are unchecked it will process all of the files starting from the year specified In the example above it will only process February 2008 of site 188 The following examples illustrate more choices the user can make Start from Year 2008 Honth Convert only the selected month Convert only the selected year All of the 2008 data is processed Start from Year 2008 Honth Convert only the selected month Convert only the selected year Only 20
8. has two lanes each time period consists of two lines each for the corresponding lane Since the first line contains date if data and time information is needed the date information in the first line along with the time in the first column should be combined in the data processing Again the time 1s the beginning time of each one hour period C 20080808 000054 cls Notepad SiteID G54 numoFlanes 2 dat aType cls dat e 2Q080808 ane by lane Tr ue Time Lane Typel Typee Types Typed Types Typed Type Types Types Typelo Typell Typele Typels Typel4 Typeli 00 00 1 0 1 0 0 0 0 0 0 1 0 0 0 0 0 L LC L C3 PJ LC PJ gt C Ko L LC L C3 C3 C3 Ko C3 C3 L O L C3 d KA KJ KA PA KA KA KA KAJ KA PA KA KA KA KA KA KJ KA KA KA KJ KA KA A LS S L L O C3 C O O C PA C C C C C KA AJ KA A A C C C C KA C C C C C C C C C C C C C C O C KA C O D O DO C C O C KG C C3 C C Ko C L C C C LC O C C O O C LA C C O O C CG C3 C3 KOW KON C3 C KO Ko Ko Ko C3 C C C C C C3 KO C C C C C O C3 C3 C C3 C3 C KO C C3 C3 C3 O C3 C3 C C3 C3 C C3 C C3 C3 C3 KG C C3 C C C3 C L C C C LC 08 U Ji UJ LM PJ Ln J fe KA Ln l UMA KA PAA PAJ KA KA C G KA KA hon CET A ao Mm en See Se Ua TOA Ca u u u a u a a a W 6 R 8 a 8 8 B B E 8 8 8 EE N TOE H E as R R LE a a Ma Mama Ma M wam Ma 8 Cee FAR EY AA Re ae rae a C3 D C C DO Kok C3 C C3 C KO Ko Kom L L L ko koi n L L L n m ci kol a ko n koi kol ko ko L L L L L ko
9. nary to ASCII Incl Headers button Pressing this button triggers a file dialog from which user selects a peak binary file Selecting a binary file and clicking the Open button produces a file that contains all header information of the original followed by the binary data The output file is also opened on a notepad for immediate viewing This function is useful to examine the details of a binary file for a debugging purpose Convert Peak Binary to ASCII CSV Format button Pressing this function produces a CSV output file 1 e it removes the detailed header information and organizes the data into the CSV data defined Section 2 Conv Directory tap i Peak Binary Converter V1 6 updated 9 78 2008 Cony File li Cony Directory Log File Binary Debug Source Folder Z AClass Faw 0001 88000000 Destination c y Archive cs root folder ZAClass4A ANCSY Start from Year 2008 Convert only the selected month Convert only the selected year Convert Specitied Site Folder Convert Specified Site Flolder Convert the Whole Archive Uvenrwrite Only New Files Uvenwrite Figure 7 Conv Directory tap This tap contains the conversion utilities that can convert a bulk of files There are two ways that you can use these utilities The first approach is converting the data for each site using the left side two buttons 1 Convert Specitied Site Folder Convert Specified Site Flalder Overwrit
10. nverting Converting Converting Converting Converting OO04 60000000 gt gt gt Converting Dir gt gt gt Converting Dir Error Data may be aa earch Counter did not change 11 72 72 007 Dir Dir Dir DIr DIr Dir Dir Dir 305 files conv Figure 7 Log file example Z MC lass Raw ODOd4 0000000 LO 2007 zZiMCl assMRaw p004 000000011 2007 ZINI lassxRaws OOOY4 6ODOGOGOSI2 2007 Z MC lass Raw oo00460000000 1_ 2007 S Class Raw 0004600000002 2007 Dw Class RAW 0004 60000000 3_ 2007 P Class Raws 0004 600000004 6_ 2007 E lass Rawi 0004600000004 7_200 WE lasskRawk 0004600000008 2007 S iiss a kk kk ki mai erte B N KI 10 11
11. te information heading is in the first line given by the following SiteID 054 numOfLanes 2 dataType spd date 20070114 lane by lane False This line indicates that the site number is 054 the site has two lanes data type is speed the date is Jan 14 2007 and data is not lane by lane data but combined one The column heading is in the second line 1 e Fame 0 409 45 007 59700 65 FO 157007 65 100 111 In this case it is a speed bin data and consists of time and speed bin column headings The meaning of each column is summarized in Table 1 Table 1 Column headings of speed CSV file Column Column name Description number l Time hh mm It is the beginning time of the time slot one hour 2 00 means data between 2 00 3 00 am The 24 hour military system 1s used 2 O 0 40 mph bin 40 45 mph bin 45 50 mph bin OSS 55 00 mph bin I 60 60 65 mphbin Bo OS 65 70 mph bin _ Oo 10 75 mpbn u 80 80 85 mphbin If lane by lane True is present in the site information heading the data is recorded by lane by lane This means that the number of lines for each time period is equal to the number lanes It is illustrated using a file generated from site 54 shown in Figure 5 In this case the data type is classification and the data bins are arranged from types 1 though 15 from left to right The second column is the lane number in the order present in the binary data Since this site
12. ubstantially and can reduce the processing speed The Cleanup Log File button does not actually delete the log file but it copies to a different file name The log file is plain text files and located in C Program Files UMD Peak_Conv Log Here is an example of files in the log directory 2008 2 text 2008 3 tet Z 2008 tect In this case 2008 txt is the current log file and the rest are backup log files that are created when you click on the Cleanup Log File button If these files are not needed the user should delete the backup log files You can also delete the current log file In that case when you process the software will recreate the current log file The View Log File opens the current log file on a notepad Inside the log file the line starting with Error contains important information In the example shown in Figure 7 site 460 11 3 2007 had an error There are two possibilities First the binary data is corrupted which would be the most case Second the decoder was not able to recognize the format In either case the error message is present in the csv file If the data line does not start with time it 1s a sign of the error and should not use the CSV file Some cases the data is good up to the line indicating Error in that case the good data can be still used for further processing gt gt site oo ke mee oe oo at oo otk oF ee FA Converting Converting Converting Co
Download Pdf Manuals
Related Search