Home

User's Guide to the Model Validation Kit

1. e Asa convective cell passes over the plant area the plume is rapidly entrained upward into the call A cumulus cloud then forms in the cell e The cloud formation is likely enhanced by added moisture and particulate matter from the plume July 25 1980 0825 1700 CST Plume being entrained into propagating convective cells with subsequent cumulus development 63 Kincaid28 ulyCumulus Plume entrained into convective cells cumulus formed 64 e At 1400 bifurcated plume e Note Clock time is wrong July 28 1980 0820 1530 Plume producing large cumulus clouds The plume is being entrained into propagating convective cells Note Clock time is wrong ASTM Standard Guide D6589 Premise of ASTM methodology Average observations within regimes Near centreline concentrations Datasets Kincaid Indianapolis Prairie Grass 11 Notes on the ASTM package As noted in chapter 2 Key to the Model Validation Kit there is a concern that direct comparison of model predictions against observations could cause misleading results Therefore an alternative approach has been proposed by John Irwin and has resulted in ASTM Standard Guide D6589 This procedure has also been incorporated in the latest version of the BOOT software as an option However there exists also a separate package software and data sets specifically devised as an implementation of the ASTM procedure here referred to as the ASTM package
2. 272 272 272 272 272 272 272 272 272 272 273 272 U1 OO 00 OO OO CO OO OO OO OO Ul UT U1 U1 UT UT UT UI iS iS UT OO CO SWD 02 999 19 999 CEILNWS DPNWS WDNWS WSNWS PNWS TNWS NNWS PRECNWS oooooooo 1 1 1 1 f 1 1 1 1 1 1 74 21 21 213 21 08 44 24 8 3 8 5 6 7 8 10 0 15 4 20 2 3 9 By oooooooo 2 117 9 12 9 14 2 12 0 6 0 4 1 8 i 10 1 10 3 10 4 8 3 8 m o NNUNNNNNN 80 ARCMAX Q QUAL m w WOWDANAR WWRERRRW WWNHWWWNHWNWH ooooooooooo ooooooooooo ZI 2076 2092 2104 893 1032 1175 1355 1539 1545 1300 1743 1840 ZI 154 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 100 00 00 oooooooooo DTHDZ 9 9999 9 9999 9 9999 0 0022 0 0022 0 0022 0 0022 0 0022 0 0022 0 0002 0 0002 0 0002 DTHDZ 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 29 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 9 9999 SWD50 999 00 999 00 N N NPPNNERNNN SWD30 999 999 00 00 L TURNER Q 8 6 4 10 11 2 4 8 9 9 4 8 3 9 2 11 4 8 1 11 10 4 3 11 6 3 3 11 84 7 3 11 20 9 4 10 2345 3 11 SWD10 SIGW 999 00 9 99 999 00 9 99 999 00 9 99 70 49 0
3. National Environmental Research Institute Ministry of the Environment Denmark User s Guide to the Model Validation Kit Research Notes from NERI No 226 2 DVT Kincaid Developement Data Experiment Options Help TV 30 View allt lz 172480 z Meteo Data SF6 Data Temp difference Temperature T10 Tom side National Environmental Research Institute Ministry of the Environment Denmark User s Guide to the Model Validation Kit Research Notes from NERI No 226 2005 Helge Rerdam Olesen Data sheet Title Author s Department s Serial title and no Publisher URL Date of publication Editing complete Referee Financial support Please cite as Abstract Keywords Layout ISSN electronic Number of pages Internet version For sale at User s Guide to the Model Validation Kit Helge Rerdam Olesen Department of Atmospheric Environment Research Notes from NERI No 226 National Environmental Research Institute Ministry of the Environment Denmark http www dmu dk December 2005 December 2005 Matthias Ketzel Department of Atmospheric Environment NERI No external funding Olesen H R 2005 User s Guide to the Model Validation Kit National Environmental Research Institute Denmark 72pp Research Notes from NERI no 226 http research notes dmu dk Reproduction is permitted provided the source is explicitly
4. for file type names INP Input for BOOTW and RESIDUAL BOO Output from BOOTW DAT Data file that can be plotted using SIGPLOT Some of these DAT files can be generated by RESIDUAL QUA Data file suitable for generating quantile quantile plots generated by RESIDUAL INQ Template file for SIGPLOT TEK Picture in Tektronix format can be shown on the screen using Tekplot PIC Identical to TEK format picture in Tektronix format Some required input files for generating the plots described on the subsequent pages are included in the Too1s folder Thus there are 18 template files used by SIGPLOT extension INQ They determine the layout of plots Table 18 on page 52 provides an overview of the files 6 2 Kincaid In order to obtain results that are comparable to those of other groups you should perform an analysis where you adhere to the following protocol e analyze the behaviour of normalized concentrations ARCMAX Q using the same units as in SF6_KIN DAT e conduct an analysis for data with the quality indicator QUAL equal to 3 the most reliable data since this is what other modellers have done the fact that c is never zero for data of quality 3 should be kept in mind this leads to a bias You can also perform an analysis where you include data of quality 2 e use your own preprocessor if possible e however use the observed mixing height The details are explained in the following File with results
5. 84 27 52 0 90 32 17 0 77 65 40 0 71 40 79 0 80 29 04 0 62 16 56 0 74 7459 0 53 12 03 0 45 4 33 0 22 20 20 0 13 28 54 0 09 21 19 0 06 8 01 0 19 4 86 0 37 Oooooooonbouonbenbon UNIWRWNNBNN TQ 416 416 416 432 432 432 432 432 432 441 oooooooooo VSQ FLAG 14 6 0 14 6 0 15 0 0 29 6 0 29 2 0 29 6 0 29 9 0 30 0 2 30 3 0 27 9 0 25 Table 5 Format of RAWIN DAT radiosonde data Format of RAWIN DAT DATA SERTES ZNPTCATOR 5600 8A 620l Pive digit Staben ID lt 4942 Pesria TL YY MM DDHH Cure Tit mo of pressure levels in the Sund ing fetal no of pressure evel Uxtracted from Xf Sounding REDER gt 5600 14842 80 224 0 64 16 993 7 200 273 1 270 4 963 0 451 271 2 300 8 C DATA 901 0 977 270 1 293 12 900 0 986 270 1 293 12 REDRPS 826 0 1668 270 7 286 14 800 0 1921 268 8 285 14 752 0 2407 261 4 284 16 750 0 2428 267 4 284 16 HEADER gt 5600 14842 80 22412 53 997 4 200 272 0 300 4 989 0 268 272 2 299 5 pata 900 0 1012 266 0 309 10 893 0 1073 265 5 311 11 RECRDS 950 0 1464 270 5 318 14 800 0 1943 268 1 308 14 x Viu ds url 18 COR rere wind Spread c mfc Bee ient FORO D ck kkk cda 6200 format 3x 6201 6200 format 3x 5600 write irawin 6210 1 i 1 istop 6210 format 4 3x f6 1 f5 0 26 f5 1 7 I O statement used to write the header records write irawin 6200 stnid year
6. 9 16 2 000 337 85 9 16 3 000 254 85 9 16 4 000 200 85 9 16 6 000 141 85 9 16 8 000 109 85 9 16 10 000 90 85 9 16 12 000 76 Please note that the units for ARCMAX Q should be s me10 corresponding to the units used in the file with SF data SF6 IND DAT Merging modelled values and observations Cross wind integrated concentrations are not included in the present version of the Indianapolis data set as they are difficult to assess with confidence You can use the utility Combine indi exe to prepare an input file for BOOT as well as a file appropriate for a scatter plot First you must edit the file Combine Indi ini This is a plain text file which tells the program where it can find the model results and where it should place the files that it generates Run Combine indiexe by double clicking its icon The program allows selection according to quality level There are 479 observations of quality 3 You can then run utilities BOOT RESIDUAL SIGPLOT etc in a similar way as for Kincaid When running RESIDUAL you may use the same filter value 15 as for Kincaid The required files are included in the Tools folder Master ind INQ files for SIGPLOT etc Table 18 shows the names of the INO files SIGPLOT templates 6 6 Hints on automatizing the process It is possible to achieve a high degree of automatition for the programs if you work in a command line environment This is not the place for a full
7. 999 00 272 4 999 00 272 4 999 00 271 9 999 00 271 5 999 00 291 5 999 00 271 5 999 00 271 5 999 00 271 1 999 00 271 31 I 999 00 271 1 999 00 271 1 999 00 271 1 999 00 271 5 999 00 271 5 999 00 271 5 999 00 271 5 99900 271 5 999 00 271 5 999 00 272 4 S30 WS10 WD100 5 8 2 7 281 6 3 2 9 302 6 3 2 5 308 2 2 2 0 51 2 3 2 1 52 3 23 3 0 53 2 3 2 1 29 9 0 999 0 42 2 5 2 3 31 4 0 3 5 15 3 9 3 4 21 3 5 3 0 33 3 4 2 8 47 2 2 1 4 67 1 9 0 9 133 1 7 1 0 323 2 0 1 7 331 2 8 2 5 340 WS100 WD100 SIGV S o o C ND IO IO 9 IO IO OY UL UT T100 999 999 999 283 284 284 285 286 286 289 290 290 Ui oNUOOOoooo0o T100 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 OOOOOoOooo0o000000000000000000 WD50 999 VO 00 O i WWW QJ CO CO 28 30 T50 1 2 999 999 999 283 284 285 286 286 287 290 290 291 T50 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 999 Oc tto0n P5 ooooo OOOOOooo0o0000000000000000000 9 99 9 99 9 99 ROWRROR N w T10 298 298 298 284 285 286 286 287 287 290 291 291 DAWOIWHANNNAAK T10 272 272 273 274 274 273 273 2735 273 213 273 273 273 272
8. DY HR 85 9 16 11 85 9 16 11 85 9 16 11 85 9 16 11 85 9 16 11 85 9 16 11 85 9 16 11 85 9 16 11 85 9 16 12 85 9 16 12 85 9 16 12 85 9 16 12 85 9 16 12 85091611 DAT 0 129 0 042 0 170 0 228 0 383 0 417 0 387 0 310 0 101 0 039 0 410 38 FROOCODRWNEKROOCO ooooooooooo E S 280 470 440 330 240 120 320 450 470 720 520 TOTAL AMPLE N TOT NONZERO SAMPLE NONZERO DAODRWIKRUBRUWWAP Dist oOoooooooooo 310 470 470 400 450 430 500 550 480 720 660 ARC MAX ARC MAX Emission ARCMAX QUALITY AZIMUTH CY SIGY PPT nG M 3 rate g s Q INDEX TO MAX uG M 2 M ARCMAXPP ARCMAXNG Q C_OBS QUAL AZMAX CY SIGY 7 0 42 6 4 94 8 6 0 65 3 999 0 999 0 155 0 942 4 4 94 190 8 3 84 9 309 5 101 5 84 0 510 7 4 94 103 4 1 93 1 999 0 999 0 206 0 1252 5 4 94 253 5 1 105 3 999 0 999 0 444 0 2699 5 4 94 546 5 2 91 8 1595 8 391 9 55 0 334 4 4 94 67 7 1 79 6 465 6 557 0 467 0 2839 3 4 94 574 8 2 91 5 2692 3 418 6 258 0 1568 6 4 94 317 5 1 94 5 1398 2 655 7 200 0 1216 0 4 94 246 2 3 99 0 3005 9 1121 8 110 0 666 8 4 94 135 0 1s 107 9 999 0 999 0 337 0 2042 8 4 94 413 5 2 84 9 833 6 147 1 458 0 2776 3 4 94 562 0 2 111 9 1209 1 237 1 327 0 1982 2 4 94 401 3 2 105 3 1362 3 296 3 150 0 909 3 4 94 184 1 2 83 9 1068 1 340 6 Structure of this chapter Outcome from the tools Avoid spaces in file names 8 character limit for files used by SIGPLOT 6 Step by step instruc
9. Effectiveness MOE as well as several others FB and MOE are in fact closely related 42 1 File with statistics Note that a useful extension to FB and MG has been implemented in the current BOOT package FB is of limited value because overpredictions and underpredictions compensate each other However FB and MG can be separated into overpredicting and underpredicting components Thus FB false negative only considers underpredictions while EB only considers overpredictions Bootstrap resampling is used to estimate the confidence limits of a performance measure hence the name BOOT of the package Consult the BOOT User s Guide for a detailed discussion of the interpretation of results from BOOT The User s Guide Chang and Hanna 2005 is part of the Model Validation Kit Proceed as follows in order to generate a file containing statistics for data of quality 3 with the use of BOOT As noted above we assume that the tools reside in C MvK Tools BOOT can be launched in various ways E g you can use Explorer to show C MvK Tools and then double click on the icon for BOOT Alternatively assuming some familiarity with DOS BOOT can also be launched by these steps e Open a Command Prompt window explained in Section 6 2 5 Table 12 Dialogue when using the BOOT programme Name of input file Name of output file Select one from the following options 1 straight Co and Cp comparison 4 consider l1n
10. It was prepared by John Irwin and is available on the Internet www harmo org astm This is not part of the Model Validation Kit but it can be used as a supplement or an alternative to the Model Validation Kit In order to place the Model Validation Kit in a perspective we will here outline the main principles of the ASTM methodology and further explain some features that distinguish the two packages The ASTM guide is a guide on statistical evaluation of dispersion model performance and is general in the sense that it is not confined to the problem of dispersion from an isolated point source which is the focus of the Model Validation Kit However as an example of application of the principles in the Guide the particular problem of point source dispersion is addressed in an Appendix to the Guide as well as in the ASTM package The fundamental premise of the ASTM procedure is that observations and model predictions should not be compared directly and that observations should be properly averaged before comparison The comparison takes place within regimes which for example can be defined according to atmospheric stability and distance to the source The ASTM procedure then calculates performance measures based on regime averages i e averaging over all experiments within a regime rather than the values for individual experiments In the specific implementation of the ASTM methodology found in the ASTM package the observat
11. Meteorological observations were taken from a 94 m height at the top of a building in the middle of the urban area and from three 10 m 36 towers in urban suburban and rural locations at the urban site the measuring height for some variables was 11 m however Standard National Weather Source NWS observations were available from the local airport In addition vertical profiles were taken by minisondes and acoustic sounders 5 4 3 Tracer data Concentrations were observed on a network of about 160 ground level monitors on arcs at distances ranging from 0 25 to 12 0 km from the stack Vertical cross sections of the plume were made by a lidar a few hundred meters downwind of the stack The design of the field experiment was similar to earlier EPRI field experiments at the Kincaid and Bull Run power plants As in the case of Kincaid a quality indicator was assigned to the arc wise maximum concentrations A subjective numerical ranking of the quality of the data on that monitoring arc was undertaken by Steve Hanna It has been determined by studying the ground level SF observation patterns during each hour and assigning a quality index ranging from 0 to 3 to each monitoring arc Monitoring arcs are recommended to be used in the model evaluation exercises only if their quality indicator is 2 or 3 5 4 4 Data files Concentration data are summarized in the SF6 IND DAT data file where a separate line of data is given for each monitor
12. S Irwin and Associates Raleigh NC USA E mail jsirwinetal nc rr com Information related to this document can be found on the World Wide Web Look at http www harmo org kit 71 72 Danish Summary Dansk resum Det s kaldte Model Validation Kit er en vaerktejskasse best ende af data fra spredningseksperimenter i atmosf ren samt programmel og dokumentation der tilsammen udg r en referenceramme for evaluering af atmosf riske spredningsmodeller Det er m ntet p modeller der beskriver spredning fra punktkilder lokalt omkring kilden Model Validation Kit har vundet vid international udbredelse siden den f rste version forel i 1993 Adskillige hundrede forskningsgrupper har rekvireret materialet I tiden siden 1992 har det s kaldte Initiative on Harmonisation within Atmospheric Dispersion Modelling for Regulatory purposes www harmo org afholdt en serie workshops og konferencer om spredningsmodellering til administrative formal Et fokusomr de har veeret evaluering af modeller og Model Validation Kit er blevet hyppigt benyttet i forbindelse med disse konferencer Den foreliggende rapport udger en brugervejledning til materialet Den give et overblik over hele vaerktejskassen og indeholder ydermere detaljerede anvisninger pa brugen af materialet Ud over dataset og programmel til modelevaluering omfatter pakken ogs supplerende materiale s som v rkt j til data visualisering og videoklip fra sprednin
13. an offset so that zero values are clearly displayed The layout can be changed by changing the template file Kiscat inq see the SIGPLOT User s guide and its addendum for details Table 14 displays the conversation described above An alternative not recommended is HPGL format HP Graphics Language An eps file can be converted to HPGL by the command EPS2HP Kiscat3 don t indicate the extension eps You may receive an error message even if the conversion is successful However HPGL is a format which is typically not recognised by modern applications A standard installation of MS Word does not recognise it anyhow it is possible to download an unsupported import filter for Word at the Microsoft web site look for hpgl32 exe Table 14 Commands required to produce a scatter plot as described C MvK Tools Run1 gt sigplot Type sigplot nolabels instead if you want to disable labelling Name of template file Name of input data file Name of tektronix picture file Plotted frame 1 C WMvKNTools WRun1 ESKpIoE Kiscat3 tek C MvK Tools Runi gt ps kiscat3 TekPS version 2 0 c 1988 Arlindo da Silva version 3 0 revised by J Chang SRC June 1993 to make the output comply with the encapsulated PostScript file format kiscat3 tek gt kiscat3 eps working done 3 Q Q plot Running Residual The procedures can be automatized 6 2 7 Creating O O plots and box plots In or
14. concentrations are larger than predictions e A plume may not be properly captured by an arc of monitors As a consequence you may obtain misleading values for observed arc wise maxima and or crosswind integrated concentrations This problem is attempted solved in the Model Validation Kit by means of quality indicator for arc wise maxima for Kincaid and Indianapolis Crosswind integrated concentrations from these two data sets are not included among the data because no proper quality assurance has been undertaken In the case of Copenhagen and Lillestrom data the coverage by monitoring arcs has been considered good enough for both arc wise maxima and crosswind integrated concentration to be determined e Pay attention to averaging times In the context of the Lillestr m experiment sampling took place during 15 minute periods Such a plume should be expected to be narrower than a plume sampled over an entire hour so if your model predicts one hour averages a comparison of arc wise maxima may be well be misleading A comparison of crosswind integrated concentrations will be more reasonable as the effect of plume meandering is irrelevant in such a comparison Kincaid Copenhagen Lillestrem Indianapolis Other examples are these examples are not relevant in the case of the Model Validation Kit deposition may occur a problem relevant for the Prairie Gras experiment a comparison of observed near centreline concentrati
15. ensure that the system can find the utilities of the Model Validation Kit if you once and for all adjust the Path variable in your system environment In Windows 2000 or XP follow the subsequent steps Select Start Settings Control Panel System choose the Advanced tab select Environment variables in the panel with System variables choose Path Edit In the field Variable value you should edit the contents so it ends with e g C NMMvKNTools namely a semicolon followed by the location of the tools After you have finished the Path may look like this SSystemRoot system32 SystemRoot C MvK Tools the path may be longer depending on your system You need to know a few things in order to work in a command line environment DOS First you enter this environment by selecting Start gt Run write cmd and then press the Enter key You will need to use the command cd Change Directory and possibly dir show Directory Write cd CD followed by two dots to move up in the tree of folders The command cd can also be used to move down to a branch Thus if your current folder is the root of C the command cd mok will change the current folder to C MvK 6 2 6 Using SIGPLOT We assume that you have used the utility Combine_kin exe to produce KISCAT3 DAT as explained in section 6 2 2 Next create a scatter plot showing c following steps versus C by following the mod obs Use Start gt
16. follow concerning e The SIGPLOT software e The Dispersion Visualisation Tool e Tools for Grapher e Video clips from Kincaid e Notes on the ASTM package e Changes since the previous version of the Model Validation Kit Details on the BOOT software are not included here as there is a separate User s Guide in the Boot folder of the CD The User s Guide also contains a general discussion on model evaluation 13 Pitfalls FAQ Should model predictions really fit observations 14 3 Pitfalls and FAQ Please browse through this chapter It gives an overview of pitfalls that you may run into when working with the Model Validation Kit Although nearly all of these are mentioned elsewhere in the material it may save you time and trouble to become acquainted with them as soon as you begin your work Further the chapter provides answers to some commonly asked questions 3 1 Pitfalls It is a basic assumption that for a good model you expect model predictions to fit observed results This assumption may not always be warranted It is important to consider this question when interpreting results from model evaluation Don t throw your results blindly into a statistical blackbox Some examples follow e Quantile quantile plots should be interpreted with care because of the stochastic nature of atmospheric dispersion A model typically predicts ensemble averages so it must be expected that the very highest observed
17. meteorological parameters u w L and h be used with care or replaced o and o are suspected to be unreliable According to Steve Hanna there were many problems with Gill c data and use of them may severely degenerate modelling results Also according to results for one specific model shown at the workshop in Manno the effect of choosing observed values of c as opposed to computed values resulted in predictions of the Should pred 21 22 maximum concentration for the entire data set which were factor of three larger than otherwise e tis recommended to use data with a quality indicator of 2 or 3 when analyzing model behaviour One point is important to be aware of observations with QUAL 3 are biased in the sense that they are never zero 5 1 6 Additional information The data distributed constitute only a small fraction of the wide variety of variables collected during the campaign A large number of reports concerning the Kincaid experiment have been published by EPRI Therefore if you wish to analyze certain questions in further detail you may want to request some of these reports from EPRI see the list of addresses at the end of the chapter with references Also please note that a piece of recommended reading concerning the Kincaid experiment is the paper Hybrid Plume Dispersion Model HPDM Development and Evaluation by Hanna and Paine 1989 It gives a brief description of the layout of the Ki
18. of model calculations Modelled kin Utility to combine model results with observations Combine Kin 6 2 1 Instructions on modelling Perform model calculations for the 12 distances represented in the Kincaid data set 0 5 1 2 3 5 7 10 15 20 30 40 50 km Dump your output in a file Here we will call the file Modelled kin you may use another name The format of Modelled kin should be the following e There is a one line heading e There should be six columns or more separated by blanks the values are read using free format input e There must be a line for all 2052 arc hours 171 hours times 12 distances The distances must be those indicated above An example of the first thirteen lines of a Modelled kin file is shown below The date and time is indicated then the distance and finally the normalised arcwise maximum concentration ARCMAX Q YR MO DY HR DIST ARCMAX Q 0 0 80 4 20 14 0 5 80 4 20 14 1 0 50 4 80 4 20 14 2 0 134 0 80 4 20 14 3 0 107 1 80 4 20 14 5 0 57 7 80 4 20 14 7 0 36 9 80 4 2014 10 0 23 8 80 4 2014 15 0 15 9 80 4 2014 20 0 12 5 80 4 20 14 30 0 9 0 80 4 20 14 40 0 Ted 80 4 2014 50 0 5 9 Please note that the units for ARCMAX Q should be s me10 corresponding to the units used in the file with SF data SF6 KIN DAT Cross wind integrated concentrations are not included in the present version of the Kincaid data set as they are difficult to assess with confidence it is n
19. s Guide found in the same folder Note that the format of certain template files INQ files has changed so the current version of SIGPLOT will not always work with old template files The SIGPLOT software was developed by Joe Chang of George Mason University VA USA 59 60 8 Dispersion Visualisation Tool The Dispersion Visualisation Tool DVT was created by Alexandar Markoski Bitola University the Former Yugoslav Republic of Macedonia The present version is preliminary For updates see the web site of the Model Validation Kit www harmo org kit Data from the Kincaid experiment are divided into two parts The Development data and the Evaluation data Only the Development data are presently included here but the software has been prepared to use either of the data sets When using the DVT you must first choose the appropriate data set from the menu Experiment i e you must select Kincaid Development After this choice choose date and time Note that you can step through hours by clicking in the hour field and subsequently use the arrow keys With a mouse you can change the viewing angle E DVT Kincaid Developement Data Oo x Experiment Options Help Zoom View View Alldata z C Use filter gt 3D View n gt BasUp I Bars Down Pesem ema co raswa Figure 8 Example of observed SF concentrations at Kincaid displayed with t
20. to or multiple of the time lapse interval of 30 seconds Meteorology Class B wind speed 5 s July 17 1980 1130 1820 CDT Plume behaviour influenced by strong wind shear e North wind aloft initially carries entire plume southward meanwhile the surface wind is from the south e During the afternoon the south winds take over through an increasingly deeper layer e By mid afternoon wind shear effect is very apparent Most of plume continues southward but chunks are occasionally broken off early in the plume rise by the south wind and are carried northward e At 1530 CDT the bulk of plume is affected alternately by north and south winds A T shaped plume results i e two plumes visible at one time going in opposite directions e After 1600 CDT shear is no longer present through plume height South wind has taken over and entire plume now is carried northward Note clouds aloft still move southward July 23 1980 1500 CDT Bifurcated plume often observed at the plant July 24 1980 0810 1415 CDT Classical morning transition from night stable plume aloft through inversion break up fumigation to the convectively unstable looping plume e Initially a stable compact plume aloft No touchdown within 50 km e At 1000 CDT sudden fumigation apparent between plume height and ground e By late morning 1150 CDT strong convective eddies have formed and are propagating through the area in parade fashion
21. to the Model Validation Kit The chapter Notes on the ASTM package of the present Compendium outlines the main principles of the ASTM methodology Further it explains some features that distinguish the two packages and lists certain issues of concern 28 Forum for compilation of experiences a Wiki A Wiki is a website that allows users to easily create web pages and edit pages others have created Wiki s are excellent for collaboration A Wiki on atmospheric dispersion modelling has recently been created and this is a potential forum for reporting and retrieving experiences on use of the Model Validation Kit The address of the Wiki is http atmosphericdispersion wikicities com There is also a link to the Wiki from the web site of the kit www harmo org kit 29 Structure of the User s Guide In order to become acquainted with the Model Validation Kit the two subsequent chapters are recommended reading They concern respectively Pitfalls and FAQ and Package contents Then follows a long chapter on Field data yielding an overview of the four field experiments and of the data included in the kit Chapter 6 Step by step instructions explains in detail how the tools of the kit can be used You may choose not to use all of the tools as some of them especially those related to the SIGPLOT package may seem unfamiliar to today s computer users After Chapter 6 several short chapters with optional information
22. video is from time lapse sequences where a picture was taken every 30 seconds while the movie was played back at a rate of 9 pictures per second This corresponds to a rate of 1 real hour being played in 13 3 seconds The video clips are in MPEG format A field observer from TRC wrote a set of notes on the content of the movie which is reproduced below July 12 1980 1930 CDT Close up of Kincaid power plant stack amp dirty plume This is not a time lapse sequence Some data about the power plant e 600 ft stack 2 650 MW boilers No scrubber e 4 5 sulphur coal 2 4 mil tons burned per year e Baseload plant Generates 4 3 mil MWH per year e SO emissions 200 000 T yr 23 T hr 5 700 g s e NO emissions 1 3 SO emissions e Particulate emissions 800 T yr July 14 1980 0910 1600 CST Study of plume looping periodicity The period of the large eddies affecting the plume appears to be several times the time lapse interval of 30 seconds Meteorology Class B wind speed 5 m s July 15 1980 1045 1720 CST Kincaid15 ulyLooping Study of plume looping periodicity Kincaid17JulyShear Strong wind shear 1 43 Kincaid23JulyBifurcated Bifurcated plume Kincaid24JulyClassical Classical morning transition Kincaid25JulyEntrainment Plume entrained into convective cells Study of plume looping periodicity Plume appears to stop or back up Strobe effect implies that large eddies have a period equal
23. with SIGPLOT C MvK Tools Run1 gt Sigplot Type sigplot nolabels instead if you want to disable labelling Name of template file Name of input data file Name of tektronix picture file Plotted frame 1 Plotted frame 2 Plotted frame 3 Plotted frame 4 C MvKNToolsNRuni 4a Box plot of ratios 4b Box plot analysing c 4c Box plot analysing c mod 50 Next you should run SIGPLOT with an appropriate template file for a quantile quantile plot In our example all templates reside in a folder one level closer to the root than the current Therefore in order to create the quantile quantile plot indicate that KIQUA INQ is the template file while KI3 QUA is the input file In order to create a box plot of ratios you have already established the required data set KIBRATIO DAT through your previous dialogue with RESIDUAL Now run SIGPLOT once more with KIRATIO INQ as template file and KIBRATIO DAT as input In order to generate item 4b you will have to run RESIDUAL once more This time again use KI3 INP as input but KISOBS DAT as output file Indicate the numbers 1 0 in order to analyze c You need not impose thresholds as this is a plain analysis of concentration data You will not be asked whether to use extended capabilities when the ratio as here has 1 in the denominator In this case the boxes will automatically represent percentiles 2 4 16 50 84 and 97 6 Run SIGPLOT to
24. 01 12 2945 1000 0 440 0 102 113 774 7588 2 8 27451 36 87 01 17 1000 1015 0 150 0 102 41 637 6245 6 9 67647 36 87 01 17 1000 1015 0 470 0 102 98 484 4745 323 32353 36 87 01 17 1000 1015 0 900 0 102 134 2732 26784 7 6 74510 36 87 01 17 1015 1030 0 150 0 102 69 256 2510 P7 16667 36 87 01 17 1015 1030 0 490 0 102 79 1327 13010 6 2 60784 36 87 01 17 1015 1030 0 900 0 102 131 1138 11157 5 8 56863 36 87 02 09 1000 1015 0 190 0 102 55 2342 22961 29 6 290196 36 87 02 09 1000 1015 0 410 0 102 95 3411 33441 9 95098 36 87 02 09 1015 1030 0 190 0 102 59 4770 46765 45 8 449020 36 87 02 09 1015 1030 0 430 0 102 98 5328 52235 20 0 196078 36 DIST_L DAT YR MO DY HRE NUARC D1 D2 D3 87 01 10 945 3 0 160 0 490 0 810 87 01 10 1000 3 0 140 0 440 0 820 87 01 12 945 3 0 150 0 300 0 460 87 01 12 1000 3 0 160 0 300 0 440 87 01 17 1015 3 0 150 0 470 0 900 87 01 17 1030 3 0 150 0 490 0 900 87 02 09 1015 2 0 190 0 410 87 02 09 1030 2 0 190 0 430 Release from an 84 m stack 170 one hour samples Hoosier Dome and other buildings 34 5 4 Indianapolis 5 4 41 Experimental set up The EPRI Indianapolis field study involved SF tracer releases from the 83 8 m stack with diameter 4 72 m at the Perry K power plant in Indianapolis Indiana USA The geographic coordinates of this stack are UTM N 4401 59 km 39 8 latitude and UTM E 571 40 km 86 2 longitude The elevation of the plant is 214 m 170 hours of tracer data are ava
25. 2 3 meters above the ground o J GERSBORG SKOVS HOVE 96 o og GENTOFTE ie Ty 2 o o 40 o o o GLADSAXE 90 2 o o o o eu fo Arc MV g v d STERBRO N RREBRO e Arc 37 Figure 3 The experimental site in Copenhagen A thick line indicates the coast of the straight of Oresund Tracer sampling unit positions available for the experiments are indicated by circles Not all positions were used for an experiment Typically 20 sampling units were used in each arc and the units were deployed according to meteorological conditions 27 28 5 2 2 Meteorological data The meteorological measurements performed during the experiments included three dimensional wind velocity fluctuations at the height of release Much emphasis was put into the measurements at the release height Gryning and Thomson 1979 These measurements comprised u o and o It is recommended to make use of these measurements cf Gryning and Tassone 1994 The temperature and wind speed profile along the mast was taken from routine measurements The mixing height was determined from the daily radio sounding at Copenhagen which was carried out around the time of tracer sampling In the revised data set the values of u L and heat flux have been determined using a standard meteorological preprocessor in an implementation described by Sozzi and Fraternali 1994 Two levels of temperature and one level of wind
26. 23167 3436 87 10 945 0 470 22755 3449 87 10 945 0 490 21924 3464 87 10 945 0 810 11659 3045 87 10 945 0 820 11448 3027 87 10 945 0 900 9941 2885 Please note that the units for ARCMAX Q should be s m e10 and for CY Q s m e10 corresponding to the units used in the file with SF data SF6 LIL DAT You can use the utility Combine CphLilexe to prepare an input file for BOOT as well as a file appropriate for a scatter plot First you must edit the file Combine CphLil ini This is a plain text file which tells the program where it can find the model results and where it should place the files that it generates Run Combine CphLilexe by double clicking its icon The program creates 4 files LLINP LIY INP LISCAT DAT and LIYSCAT DAT The letter Y refers to crosswind integrated concentrations 55 Procedure for modelling 56 The two INP files are input files for BOOT for arcwise maximum and cross wind integrated concentration respectively and the two DAT files can be used to create scatter plot You can then run BOOT in a similar way as for Kincaid There are so few data that there is not much sense in creating box plots Therefore the RESIDUAL programme will not be of very much use in the analysis of the Lillestr m data except that RESIDUAL can be used to sort data so that quantile quantile plots can be produced Template files for SIGPLOT INO files have been prepared so you can use SIGPLO
27. 25 etc 100 00 T T nm 10 00 4 am o iu i 1 00 Ht a l o i o 0 10 7 v N A E 2 AP ka e 0 01 L 0 10 20 30 40 Distance km Box percentiles 5 25 etc 100 00 T m 10 00 4 m S l S amd ul o sees I o 0 10 r 4 D N o O0 R APY 0 01 7 i S 300 200 100 0 100 zi L C MOD C OBS C MOD C OBS 100 00 10 00 1 00 0 10 0 01 100 00 10 00 1 00 0 10 Ratio of concentrations Box percentiles 5 25 etc T T T 10 10 05 DoS S HS A i 9 pre 0 0 0 2 0 4 0 6 0 8 1 0 u m s Box percentiles 5 25 etc T Ha LF A X HP GS P YP Zw 7 1 v S 0 1000 2000 3000 4000 zi m Figure 7 Sample plot produced with SIGPLOT using the template file KIRATIO INQ 48 Table 15 Commands entered in a dialogue with the RESIDUAL utility in order to prepare data files for both a quantile quantile plot and a box plot of ratios C MvK Tools Run1 gt residual Name of input file ki3 inp Name of output file ki3ratio dat Following models are available choose two models i and j where the ratio of Model i Model j will be analyzed against the independent variables 1 C OBS 2 C_MOD 0 All 1 s Enter the two numbers in order between 0 and 2 Implement a lower threshold on the ratio Y n Enter the lower threshold on the ratio e g 0 01 Implement a upper threshold on the ratio Y n Enter the upper thres
28. 4 q Results file Example modelled dat File with model results Indicate only the file name not the path Master pat D Example C Master This is the path where the program expects to find the files MASTER KIN and KIN BTT supplied with the Model Validation Kit The utility produces as well a file that can be used as input for BOOT as a file that can be used for scatter plots with the SIGPLOT The steps for running the utility are as follows Prepare a file with modelled data e g MODELLED KIN e Edit Combine Kin ini to reflect the correct path and file names Double click the icon for Combine Kin exe e Follow the instructions you are asked for a title The output consists of two files a file of type INP e g KI3 inp designed as input to BOOT and RESIDUAL a file of type DAT e g KISCAT3 DAT designed for SIGPLOT scatter plot The file names depend on the user prescribed quality level Thus KIZ inp contains data of quality 2 and 3 while KI3 inp contains data exclusively of quality 3 The procedure for generating statistical analyses and plots is explained in the following sections 6 23 Analysing data with BOOT Capabilities of BOOT The BOOT package is capable of computing performance measures such as the Fractional Bias FB the Normalised Mean Square Error NMSE the Geometric Mean Bias MG the Geometric Variance VG the fraction within a factor of 2 FAC2 the Measure of
29. 6_ALL DAT The format of this file is a bit awkward but in the folder Grapher tools there is software capable of extracting information from it See Chapter 9 An alternative is to use data in the ASTM package see Section 3 2 with FAQ Table 1 The file QUAL TXT contains the following explanation of quality indicator QUAL for Kincaid and Indianapolis The indicator variable has values from 0 to 3 indicating the following 0 This value should clearly be disregarded examples the plume obviously missed the monitors the arc is only a continuation of a neighbouring arc 1 This value is most probably not the maximum value examples there are gaps in the monitoring arc the observed maximum is isolated there is no smooth variation from one arc to the next the maximum is on the edge of the arc 2 A maximum is identified but the true value may well be different examples the concentration pattern is irregular there are only 2 or 3 monitors impacted the plume is near the edge of the arc Note Also arcs where the observed maximum is essentially zero but where there is evidence that a plume is present aloft have been categorized in this group 3 A relatively well defined maximum is observed which is continuous in space is away from the edge of the monitoring arc and is not irregular or isolated It is recommended that you use data with a quality indicator of 2 or 3 in your analyses Note that observations with QUAL 3 are bi
30. 85 280 292 293 289 cy 2074 739 1722 944 2624 1990 1376 2682 2150 1869 1590 1228 688 567 1608 780 535 1248 606 456 1511 1026 855 CONFUADH IY cy Q T4 0 284 287 284 284 284 278 290 290 288 ULORALCCHAUN T12 283 286 283 283 284 277 289 289 287 0 UOUANWODAK FRENOCOKSO ARCMAX ARCMAX Q 648 231 538 295 820 622 430 1166 672 584 497 396 222 183 670 325 223 416 202 152 458 311 259 SCONDONDONOKRPNENWWENUOCWOW 360 685 152 906 226 544 203 613 875 tod 520 306 045 539 275 629 276 928 792 294 812 878 653 1050 214 985 283 1633 795 375 1570 1210 724 475 743 337 173 947 262 T5 976 264 98 852 266 197 0OPPOOOObHnUD 0PbPU 0OnpUOb0b0onPnmnBombo SIGW ooomPooonpo x Bp HS 115 115 115 115 115 115 115 115 115 115 115 115 115 115 115 115 115 115 115 115 115 115 115 ZI 1980 1920 1120 390 820 1300 1850 810 2090 NCLD OU o U mUuo CEIL 7500 6000 480 3600 750 750 2400 6000 600 36 T3 38 38 45 05 64 69 75 TURNER RWWA PS CO gs iS CO Release from a 36 m mast 15 minute sampling Norwegian winter Snow covered ground low sun 5 3 Lillestr m data 5 3 1 Experimental set up The tracer experiments of concern here took place in the town of Lillestrem ne
31. 9 9999 9999 T 9x 9 Y 3 9 24 8 0 T 2 T 9 9 5 9 9 9 9 9 v 9 94 9s 9 9z 9 9 9 9 94 9 9 9 9 9 9 9 9 ZI 2076 2288 9 2092 2313 9 2104 2333 9 893 1156 0 1032 1332 0 1175 1575 0 1355 1753 0 1539 1768 0 1545 1774 0 1300 1932 0 PRES NE 1000 0 99 999 7 99 999 7 99 995 4 50 995 0 43 994 7 43 994 0 36 993 5 25 993 2 9 994 5 33 994 3 15 994 2 6 PRES NE 988 1 99 984 8 99 982 7 99 981 4 99 980 0 99 979 3 99 978 7 99 977 3 99 976 0 99 975 3 99 975 3 99 976 0 99 976 6 99 978 0 99 978 0 99 980 0 99 980 7 99 981 4 99 982 7 99 983 4 99 984 1 99 984 1 99 984 8 99 984 8 99 984 8 99 984 8 99 WS100 ws5 5 8 999 5 8 999 63 999 2 9 2 2 3 2 343 3 2 4 2 999 0 999 2 8 2 3 9 4 4 0 4 3 8 34 4 0 Bis 3 2 v 2 4 2 4 9 25 4 6 2 3 0 2a OWONNENOPDORWANOOO 0022 0022 0022 0022 0022 0022 0002 Ui JUU JO 0o00000 OOOOOooo0o0o000000000000000000 99 T10 WS10 298 4 2 7 298 4 25 9 298 4 2 5 284 2 2 0 285 2 254 286 2 3 0 286 6 2 1 287 3 541 287 7 2 3 290 8 3 5 TOT DP100 999 00 279 5 999 00 278 8 999 00 278 8 773 84 271 6 662 27 272 0 685 54 272 3 572 16 272 6 415 56 272 5 214 55 272 6 545 19 278 6 280 01 278 2 179 78 278 1 TOT DP100 999 00 271 5 999 00 269 1 999 00 270 8 999 00 273 5 999 00 272 8 999 00 272 4 999 00 272 4
32. Co and l1n Cp Use ASTM procedure y N Print out original data y N Use E or F format for mean sigma and bias F e Calculate partial correlation y N That is the influence from a certain model is removed Do the bootstrap resampling Y n Print out detailed information on confidence limits y N Create files containing FB with its 95 confidence limits and NMSE that can later be plotted Y n Reading data Calculating performance measures Start resampling procedure Computing c l of performance measures for each model Make another run y N 43 Change to the folder where BOOT and the associated data files reside using the CD command e Type BOOT to launch the program The latter option has the advantage that it is possible to automatize the procedure see Section 6 5 Table 12 shows an example of a dialogue with the BOOT programme which results in a file with statistics Ki3 boo A central part of the resulting file is shown in Table 13 Table 13 Table with statistics which is a central part of the output file produced by BOOT Nominal median results MODEL MEAN SIGMA C OBS 54 40 25 FBfn 0 000 FBfp 0 000 MOEfn 1 000 MOEfp 1 000 FB FBfn FBfp C MOD 47 45 48 FBfn 0 428 FBfp 0 292 MOEfn 0 600 MOEfp 0 687 FB FBfn FBfp No of regimes 1 BIAS NMSE CORR FA2 FB HIGH 2nd HIGH PCOR 0 00 0 00 1 000 1 000 0 000 319 225 n
33. EIL CY Q ARCMAX Q The parameter P Pasquill class has been omitted and instead a Turner Stability class has been included The derived values of UST L and HF have been determined using a standard preprocessor whereas they were formerly determined by a manual method 29 Table 7 The full contents of files from Copenhagen MET CPH DAT Revised data set for Copenhagen Version from May 94 based on 6 10 min values w10 30 MO DY HRS 1317 1140 1213 1320 1330 1302 1245 1250 1215 HRS 1317 1317 1140 1140 1213 1213 1213 1320 1330 1330 1330 1302 1302 1302 1245 1245 1245 1250 1250 1250 1215 1215 1215 HRE 1417 1240 1313 1420 1430 1402 1345 1350 1318 HRE 1417 1240 1313 1420 1430 1402 1345 1350 1315 HRE 1417 1417 1240 1240 1313 1313 1313 1420 1430 1430 1430 1342 1342 1342 1345 1345 1345 1350 1350 1350 1318 1318 1318 NUARC WWWWWRWNHN UR S RUN NO GR ZO PNEPNPUWOR is A a d OY IOUL UO D UL iS IO UL S B2 Ov i IO I UT C9 ID iS B9 C9 NPNNNARBNE 90 W6 999 999 999 999 999 11 Tey 10 9 WWWWWWNHNNWWWWWWNWWWWWWw WWWOCORBBRERENNNWNNNNNNN w e w ROR f IB 0 Oo ooooo w1 3 10 5 4 6 13 7 9 10 SIGY 254 444 329 438 184 283 404 301 185 279 376 999 999 999 290 595 786 190 402 580 236 460 623 D3 Uu ouuuo 15 Um otN ooommr T2 285 288 285 284 2
34. Run write cmd to open a command line environment 45 46 Change directory with cd so your current folder is C MvK Tools Run1 Write SIGPLOT in order to run SIGPLOT You are asked for the name of a template file Such a file is part of the Model Validation Kit It is called Kiscat inq and it is located together with the other tools in C MvK Tools In response to the question you can write kiscat ing where serves to indicate that in the tree of folders the file is one level closer to the root The input data file is Kiscat3 dat this file was created by Combine_kin exe You are asked for the name of tektronix picture file which is the resulting output file Enter kiscat3 tek You can view the plot on the screen using the command Tekplot Write Tekplot kiscat3 tek You may have to enter the command twice sometimes there is no reaction the first time The Tektronix file format is typically not supported by modern Windows applications Therefore if you wish to produce a file that can be included in a report you should convert the file to another format We recommend using the EPS format Encapsulated PostScript HPGL is a possible alternative which is not recommended A file in EPS format can be produced by writing ps kiscat3 tek PS is a utility in the Tools folder The result is kiscat3 eps This file can be included in e g a Word document The scatter plot produced has linear axes and there is
35. T to produce Simple scatter plots Cpa vs c and CY poa vs Cy mod Quantile quantile plots cumulative distribution for arc wise maxima and cross wind integrated concentrations In order to carry out an analysis of the variation of model results with parameters it is recommended to use a utility like Microsoft Excel The starting point can be the INP files which can be edited slightly 6 5 Indianapolis The procedure is completely parallel to that for Kincaid Perform model calculations for the 12 distances represented in the Indianapolis data set 0 25 0 5 0 7 1 0 1 5 3 4 6 8 10 12 km Dump your output in a file Here we will call the file Modelled ind you may use another name The format of Modelled ind should be the following e There is a one line heading e There should be six columns or more separated by blanks the values are read using free format input e Typically there is a line for all 2040 arc hours 170 hours times 12 distances However it is acceptable that only some of these 2040 arc hours are represented in the file The distances must be those indicated above An example of the first thirteen lines of a Modelled ind file is shown below The date and time is indicated then the distance and finally the normalised arcwise maximum concentration ARCMAX Q YR MO DY HR DIST ARCMAX 85 9 16 0 250 33 85 9 16 0 500 479 85 9 16 0 750 643 85 9 16 1 000 587 85 9 16 1 500 403 85
36. a 6 88 1 24 0 146 0 547 0 135 256 248 n a Exploratory data analysis is a must SIGPLOT capabilities and limitations Alternatives to SIGPLOT Reference information on SIGPLOT Overview of products from BOOT and SIGPLOT 44 6 2 4 SIGPLOT A tool for graphical analyses When performing model evaluation it is not sufficient to consider just statistical evaluation that produces some performance metrics Rather it is recommended that exploratory data analysis also be performed using graphical techniques The Model Validation Kit includes some tools for such graphical analyses in the form of the SIGPLOT graphical package The SIGPLOT package is offered as an option that is specifically tailored for model performance evaluation The package can produce residual plots where model residuals are depicted as a function of independent variables such as the downwind distance and time of day However SIGPLOT is an old software package Thus in order to use SIGPLOT you will have to work in a DOS environment as explained in the following More modern and interactive tools than the SIGPLOT package can certainly be used to achieve the same goals For example a potential alternative is to use Microsoft Excel for data handling and graphical analyses Excel offers some very powerful tools for interactive data analysis Nevertheless Excel does not offer the specialised plots that SIGPLOT produces The advantages of using SIGPLOT
37. a copy of the BOOT program itself is also included in the Tools folder e Sigplot software package The complete package is in folder SIGPLOT while a copy of the SIGPLOT program itself is also included in the Tools folder e Tools Various software and template files designed to be helpful for model evaluation Thoroughly explained in Chapter 6 Folder Tools e Samples The folder Samples contain some samples of files referring to Kincaid data They illustrate the results of using BOOT RESIDUAL and SIGPLOT as described in Chapter 6 e The Dispersion Visualisation Tool which is a utility for displaying observed concentration data Described in Chapter 8 Folder Visualisation e Tools for preparing concentration data from Kincaid and Indianapolis so they can be plotted in a map like fashion with the commercial plotting software Grapher Described in Chapter 9 Folder Grapher tools e Video films from Kincaid as described in Chapter 10 Folder Kincaid video E mm 111MB Model Validation Kit mm 92 707 kb Kincaid video Ec 6 888 kb Field data O 4 710 kb Sigplot 0 4 331 kb Tools CoC 1 828kb Visualisation co 1 781 kb Boot c a 793 kb Files og 395 kb Samples co 17 kb Grapher tools Figure 1 Space used by the various folders of the Model Validation Kit 17 Terrain Source 18 5 Field data Please pay attention to the information given in the sections Points to be noted for each data set In these secti
38. acknowledged The so called Model Validation Kit is a compilation of field data sets software and documentation that provides a framework for evaluation of atmospheric dispersion models The kit has been used extensively by a large number of research groups since it was first introduced in 1993 In particular it has been used for a series of workshops and conferences on Harmonisation within Atmospheric Dispersion Modelling for Regulatory purposes see www harmo org The present report is a User s Guide to the kit and provides an overview of the entire material Besides data sets and software for model evaluation the package also in cludes supplementary material such as a data visualisation tool and video film from experiments The Model Validation Kit has undergone a major revision to version 2 0 in autumn 2005 The package can be downloaded from the Internet at www harmo org kit Model Validation Kit model evaluation atmospheric dispersion harmonisation Kincaid Indianapolis Lillestrom Copenhagen Gladsaxe BOOT SIGPLOT Helge R rdam Olesen 1399 9346 72 The report is available only as a PDF file from NERI s homepage http www2 dmu dk 1 viden 2 Publikationer 3 arbrapporter rapporter AR226 pdf Ministry of the Environment Frontlinien Rentemestervej 8 DK 2400 Copenhagen NV Denmark Tel 45 70 12 02 11 frontlinien frontlinien dk Contents Summary 5 1 Introduction 7 2 Key to the Model Valida
39. ailable on the web http www dmu dk atmosphericenvironment Harmoni Co nferences Belgirate BelgiratePapers asp Sozzi R and Fraternali D 1994 PBL MET Library for advanced meteorological and air quality data processing In Cuvelier 1994 TRC 1986 Urban power plant plume studies EPRI Report EA 5468 EPRI 3412 Hillview Ave Palo Alto Ca 94304 Turner D B 1964 A diffusion model for an urban area J Appl Met 3 83 9 A large number of reports concerning the Kincaid experimental campaign were published by the Electric Power Research Institute 3412 Hillview Ave Palo Alto Ca 94304 USA www epri com 14 1 Addresses The main contact for additional information is Helge R rdam Olesen However in the text some other persons with an intimate knowledge of various parts of the kit have been mentioned This list of addresses can be used if there is a need to contact them directly Sven Erik Gryning Rise National Laboratory Rise DK 4000 Roskilde Denmark Tel 45 46 775005 E mail gryning risoe dk Helge Rordam Olesen National Environmental Research Institute P O Box 358 DK 4000 Roskilde Denmark Tel 45 46 301200 Fax 45 46301214 E mail hro dmu dk Steve Hanna Hanna Consultants 7 Crescent Ave Kennebunkport ME 04046 USA E mail hannaconsult adelphia net Joseph Chang School of Computational Sciences George Mason University Fairfax Virginia USA E mail joseph chang alum mit edu John Irwin John
40. alues are missing i e 99 9 T10 Temperature at 10m deg K DT Temperature difference 36 10 m deg K SIGV Sigma of cross wind speed at 10 m sonic anemometer m s SIGW Sigma of vertical velocity at 10 m sonic anemometer m s ZI Mixing height missing i e 999 NCLD Fractional cloud cover EIGHTS CEIL Height to lowest cloud m 1 unlimited UST Friction velocity at 10 m sonic anemometer m s L Monin Obukhov length sonic anemometer m Derived meteorological parameters TURNER Turner stability class according to Turner 1964 Tracer release parameters DIST Distance km to arc of monitors Q Emission rate g s SIGY Sigma y m at DIST cy Cross wind integ conc ug m2 at DIST cy Q CY normalized by emission times 10 6 s m2 10 6 ARCMAX Max conc ug m3 at DIST ARCMAX Q Conc normalized by emission times 10 9 s m3 10 9 HS Tracer release height m NUARC Number of arcs for an experiment D1 D3 Distance km to arcs for an experiment Notes on parameters Turner stability class is computed according to the original paper by Turner J Appl Met 1964 3 83 thus there is no provision for the snow cover on the ground Changes compared to the data set distributed for the Manno workshop The following parameters have been included NCLD CEIL CY Q ARCMAX Q DT The parameter P Pasquill class has been omitted and instead a Turner stability class has been included Four values of W36 are no
41. ar Oslo Norway in 1987 They were performed by the Norwegian Institute of Air Research NILU which has put the data at our disposal A detailed description is given by Haugsbakk and T nnesen 1989 available on request A shorter description is found in the paper by Gronskei 1990 The experiments were carried out in a flat residential area with 6 10 m high buildings and trees A tracer system was used in which SF was released from a mast 36 m above the ground Each experiment consisted of two sequential 15 min periods Thus the sampling period is shorter than for the other experiments considered at the workshop The meteorological measurements were carried out along the 36 m high mast Sonic anemometer measurements were processed to give 10 min average values for wind speed and wind directions at the 10 m level Further covariances were determined between velocity components and between velocity components and temperature fluctuations The temperature during the tracer experiments was low 20 Celsius and the ground was snow covered The sun was above the horizon but at a very low angle The surface roughness was about 0 5 m Generally the vertical temperature profiles in the lowest 100 m showed an inversion Haugsbakk and T nnesen 1989 For all runs during the experimental campaign the crosswind profiles of tracer concentrations were well determined thus making a relatively accurate estimate of crosswind integra
42. arch NILU OR 41 89 Gryning S E and Lyck E 2002 The Copenhagen Tracer Experiments Reporting of Measurements Rise R 1054 rev 1 EN Rise National Laboratory Roskilde Denmark ISBN 87 550 3101 3 75 pp Available at http www risoe dk rispubl VEA ris r 1054 rev1 htm Gryning S E and E Lyck 1984 Atmospheric dispersion from elev ated sources in an urban area Comparison between tracer experiments and model calculations J Cl Appl Meteor 23 651 660 69 70 Gryning S E and Thomson 1979 A tall tower instrument system for mean and fluctuating velocity fluctuating temperature and sensible heat flux measurements J Appl Meteor 18 1674 1678 Gryning S E 1981 Elevated source SF6 tracer dispersion experiments in the Copenhagen area Riso R 446 Rise National Laboratory 187 pp Available from the author see list of addresses Gryning S E and Tassone C 1994 The Copenhagen tracer experi ment short description and some model results In Cuvelier 1994 Gronskei K E 1990 Variation in dispersion conditions with height over urban areas results of dual tracer experiments 9th AMS Symposium on Turbulence and Diffusion 1990 Olesen H R 1994 Summary of model evaluation discussions in Manno In Cuvelier 1994 Olesen H R 1995a The model validation exercise at Mol Overview of results Workshop on Operational Short range Atmospheric Dispersion Models for Environmental Impact Assessmen
43. are that you will be able to produce residual and other types of specialised plots in a relatively standardised format that the required utilities are already prepared and that the procedures are described in detail here There is reference information on SIGPLOT to be found in the SIGPLOT handbook and its addendum see Chapter 7 for an introduction In order to get started with SIGPLOT you do not need that information but can just follow the instructions given here As noted previously the following products are a typical outcome when the results of a model run are processed Preparing to work in a command line environment Redefine the Path environment variable Use CMD to open a command line environment DOS The Change Directory command 2 Scatter plot 1 A file with statistics FB NMSE etc 2 A scatter plot c 3 Aquantile quantile plot cumulative distribution VS C mod obs 4 Plots for diagnosing model behaviour 4a Box plots analyzing the ratio c parameters ma Cos in terms of physical 4b Box plots analyzing the behaviour of c 4c Box plots analyzing the behaviour of cea BOOT was used to create the first product Here we will here explain how to create the remaining products 6 2 5 Preparations to work with SIGPLOT First make a few preparations so that SIGPLOT can be used in a convenient manner You will have to work in a command line environment DOS environment You can
44. arn you against using measured values of o from the Kincaid study According to Steve Hanna there were many many problems with the Gill o data and he cautions anyone about using them Also there are indications that observed o values are unreli able this statement is based on experience with their use as discussed during the Manno workshop Most meteorological measurements the 100 m and 10 m meteorological towers solar and terrestrial radiation equipment were taken from a Central Site located around 650 m east of the Kincaid plant This site was situated in fallow fields away from major obstacles The NWS data are from the National Weather Service station in Springfield 30 6 km northwest of the source The radiosonde data supplied on diskette are routine data from the station Peoria 120 km north of the source 199 m above mean sea level 5 1 3 Tracer data There were approximately 350 hours of tracer experiments during the experimental campaign When used by Hanna and Paine 1989 the data were divided by day into two parts a developmental data base and an evaluation data base The distinction between these two subsets of data has been maintained and the data distributed here belong to the development data base There is a total of 171 hours in the development data base distributed For each hour data are avail able from several crosswind arcs of monitors Screening of data has led to the conclusion that a few obs
45. ased in the sense that they are never zero 5 1 4 Data files Hour indicates end of the hour for time averaged parameters Thus 10 means an average over the period between 9 and 10 Central Standard Time CST is equivalent to GMT 6 The following files are supplied in the Kincaid folder DISTM_K DAT Distances to arcs with values of max conc EMISSION DAT Emission data for all hours not just sampling hours GEO KIN TXT Info on geographical coordinates MET KI L DAT Meteorological data long continuous period MET K1 DAT Meteorological data tracer hours MET K2 L DAT More met parameters MET K2 DAT MET K3 L DAT MET K3 DAT MISC KIN DAT Miscellaneous data OUTLIERS TXT Information on outliers changes to original data PAR_KIN TXT Overview of parameters and missing data QUAL TXT Explanation of quality indicator RAWIN DAT Routine radiosonde data SF6_ALL DAT SF data all monitors SF6_KIN DAT SF data arc wise maxima There are corresponding pairs of files such as MET_K1 DAT and MET_K1 L DAT The T files are long and include meteorological information for the hours between tracer experiments They have been included to permit modellers to run met preprocessor requiring continuous periods of data The tables on the next pages yield an overview of the variables contained in the data sets 5 1 5 Points to be noted A summary of some potential pitfalls when using data is given below e The derived
46. can be summarised as follows e Only four experimental data sets are considered e The emphasis is on operational short range models e The problem of interest is relatively simple namely a point source emitting a non reactive gas over flat terrain due to the fact that 11 Quantile quantile plots cannot be expected to show one to one correspondence A separate ASTM package exists 12 this is the scenario represented by the four field experiments On the other hand much of the software included in the Kit is general and applicable to many different release scenarios e Further the emphasis is primarily on a arc wise maximum concentrations and to some extent b cross wind integrated concen trations e The Kit does not explicitly account for the stochastic nature of dispersion problems The tools in the Kit can be used to diagnose strengths and weaknesses of the models but as a consequence of the above limitations you should be careful in interpreting the results To further elaborate the last bullet in the above list atmospheric dispersion processes are stochastic whereas models in general predict only ensemble averages not individual realisations This means that there is a basic conceptual problem with the procedure of directly comparing model predictions to observations as they cannot be expected to have the same statistical distribution One consequence is that if the monitoring network is sufficiently den
47. ced Practical Short Range Atmos pheric Dispersion Models August 30 September 3 1993 Manno Switzerland CSCS Centro Svizzero di Calcolo Scientifico Joint Research Centre European Commission EUR 15603 EN Available from C Cuvelier JRC Ispra TP 690 21020 Ispra Italy Hanna S R and RJ Paine 1989 Hybrid Plume Dispersion Model HPDM development and evaluation J Appl Meteorol 28 206 224 Hanna S R and J C Chang 1992 Boundary Layer parameterizations for applied dispersion modelling over urban areas Boundary Layer Meteorology 58 229 259 Hanna S R and R J Paine 1989 Hybrid Plume Dispersion Model HPDM development and evaluation J Appl Meteorol 28 206 224 Hanna S R Strimaitis D G and J C Chang 1991 Hazard response modeling uncertainty a quantitative method Vol I User s guide for software for evaluating hazardous gas dispersion models Sigma Research Corporation Westford Ma Hanna S R and J C Chang 1991 Modification of HPDM for Urban Conditions and Its Evaluation using the Indianapolis Data Set Final Report Prepared for EPRIby EARTH TECH 196 Baker Ave Concord MA 10742 Hanna S R and J C Chang 1993 Hybrid Plume Dispersion Model HPDM improvements and testing at three field sites Atmos Environ 27A 1491 1508 Haugsbakk I and Tennesen D A 1989 Atmospheric Dispersion Experiments at Lillestrom 1986 1987 Data Report Lillestrom Norwegian Institute for Air Rese
48. d data sets as well as software for model evaluation The Kit is a practical tool intended to serve as a common frame of reference for model performance evaluation It is however limited in scope as described in subsequent discussions The Kit has been used for the series of Harmonisation workshops and conferences A preliminary version of the Kit was used for the workshop in 1993 while a subsequent version was used essentially unchanged throughout the period 1994 2005 in 1997 a supplement was added It has been distributed in hardcopy diskette CD and paper to more than 250 research groups during that period The package was updated to Version 2 0 in October 2005 The new version allows the same studies to be carried out as the previous version but has been revised in several respects New software and computing environments have made it necessary to update the package Furthermore the documentation is significantly improved and brought up to date The package can be downloaded from the Internet at www harmo org kit Elements of the package The package contains the following main elements e Field data sets from Kincaid Indianapolis Copenhagen and Lillestrom e The BOOT statistical model evaluation software package e Tools for exploratory data analysis useful for diagnostic model evaluation e A recommended procedure protocol for model performance evaluation The approach is explained in the Chapter Step by step
49. der to create items 3 and 4 Q Q plots and various box plots you will have to run the RESIDUAL utility several times to generate the necessary data files and then create the plots with SIGPLOT As before we assume that you are in a command line environment with C MvK Tools Run1 as your current folder Write RESIDUAL on the command line in order to run the utility Table 15 shows a set of commands pertinent to running Residual with Kincaid data Certain thresholds are indicated in order to assure that the ratio Crroa Core lies in the interval 0 01 100 Furthermore to make the values of the ratio less noisy a filter value has been set to 15 This is suitable for Kincaid data and many other modellers have used this value If you have to repeat such runs several times with slight variations the task of running Residual becomes tedious and you will probably feel a desire for some degree of automatition You will find various hints on this in Section 6 5 47 Kincaid scatter 400 pmr Cmod Er te He He 50 el 50 0 50 100 150 20 Cobs 0 2 Qual 3 til 50 300 10 10 05 350 400 MOD C Kincaid Quantile quantile plot 10 10 05 50 O 50 100 150 200 250 300 350 400 C OBS Figure 6 Sample plots produced with SIGPLOT using the template files KISCAT INQ a simple scatter plot and KIQUA INO a quantile quantile plot Box percentiles Kincaid 5
50. description of DOS tricks but a few hints can be given If you run e g BOOT from a command line environment you can enter the command BOOT boot commands txt and then avoid manually entering the responses to the dialogue with BOOT You just have to prepare the file boot commands txt so it contains all of the responses to the questions asked by BOOT A similar approach can be taken with Residual and Sigplot There are samples in the files e Boot commands txt input for BOOT e Residual ratio commands txt input for Residual when preparing files for box plots of ratio e Sigplot commands txt input for Sigplot when producing box plots of ratios These samples can be found in the Toois folder However they should be copied to a different folder The sample files will work directly if the current folder is the one with model results for Kincaid such as C MvK Tools Run1 of the example For other use the sample files should be adjusted Furthermore if you are familiar with DOS you can prepare BAT files and execute several programmes in a sequence 6 7 Hints on software problems e Sometimes when you use TEKPLOT to display a file nothing is displayed Just try once more 57 58 If a file cannot be found even though it exists then check the length of the filename Some of the old utilities e g Tekplot don t recognise filenames of more than 8 characters 3 characters for file type The BOOT utility and the C
51. detail the procedure for processing of Kincaid data 6 1 File naming conventions conventions for the example Avoid using file names with spaces in their name as some of the utilities will not work Furthermore the SIGPLOT utility that can optionally be used for graphical analyses will not recognise names of files or folders that are longer than 8 characters This is because SIGPLOT is quite old However when you work with SIGPLOT you will need only relative path names so there is no problem in locating all of your work in a folder with a long complicated name as long as you stick to short relative names in your conversation with SIGPLOT a detailed example follows 39 Format changes since previous version of the Model Validation Kit Assumption about C MvK Tools Conventions for file types 40 If you have used a previous version of the Model Validation Kit please note there that have been a few changes in the format of the various files Thus some files which previously were without header now have one e g Modelled xxx Details on changes can be found in Chapter 12 In the following for the example we will assume that you have copied the Tools folder of the Model Validation Kit to the folder C MvK Tools on your computer Further we assume that you keep your model results in a subfolder of that folder namely C MvK Tools Run1 In the subsequent explanation the following conventions will be used
52. documentation is significantly improved and brought up to date The package can be downloaded from the Internet at www harmo org kit 1 Introduction The present report is a companion to a set of software and datasets for evaluation of atmospheric dispersion models The entire material is known as the Model Validation Kit and it can be found through the web page of the initiative on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes ww w harmo org Also future updates to the material can be found can be found here more specifically at www harmo org kit The material has been compiled by Helge R rdam Olesen of the National Environmental Research Institute in Denmark but it is based on joint efforts by many persons see Chapter 13 for Acknowledgements The recommended way to use the report is as follows Read the chapter Key to the Model Validation Kit in order to understand what the Model Validation Kit is and where you should look for the various types of information Further browse through the chapter Pitfalls and FAQ as it may save you work and trouble Finally go on reading the remaining chapters according to your needs Note that the chapter Package contents gives an overview of the available material The kit has been used at the Harmonisation conferences Exploratory data analysis is important A common frame of reference History 2 Key to the Model Validation Kit The so call
53. dy the sensitivity of the evaluation results to the definition of regimes i e how data are stratified There is always only a limited number of regimes e g 20 to 40 that can be defined The performance measures are always determined by this limited number of regime averages It is necessary to carefully examine the implication of accounting for only the variance in regime averages rather than the full variance in the complete dataset In the current implementation of the procedure with near centreline concentrations NCCS it is problematic that observed NCCs are compared to model predictions in the exact centerline which by definition are higher than near centerline values The basic assumption that averaged model results should fit averaged observations may be unwarranted if the quality of observed data is not properly assured This is especially a concern when experimental data are fed into a statistical blackbox where these data are processed and averaged before a result is inspected Problems with the observed data or the way they are interpreted may easily pass unnoticed Use of a quality indicator could alleviate such problems 12 Changes since previous version The previous version of the Model Validation Kit was distributed between 1993 and October 2005 There have been no changes in values in data values compared to that version However the following changes are notable Some extra material has been added to th
54. e Model Validation Kit notably the Dispersion Visualisation Tool and the video films The BOOT software is version 2 0 which has many additional features compared to the earlier version There have been many changes in the utilities in order to allow for unproblematic use of the kit in a Windows environment The entire documentation is considerably updated Some information carrying files have been renamed so they now have extension TXT instead of DAT The format for input to BOOT has changed so there is an additional column of data values compared to previously See Table 19 Table 19 Format of an INP file in the previous version left and the current version right 338 2 1 4 338 2 1 4 338 338 C DBS C MOD DIST u h L zmix YR MO DY HR QUAL C OBS C MOD DIST u h L zmix YR MO DY HR QUAL Block of all data Block of all data 735 93 0 3 0 0 300 241 4 2076 80 4 20 14 3 1 73 3 107 0 3 00 0 300 241 4 2076 80 42014 3 53 9 85 0 3 0 0 310 186 8 2092 80 4 2015 3 1 53 9 111 0 3 00 0 310 186 8 2092 80 4 2015 3 30 4 58 0 5 0 0 310 186 8 2092 80 4 2015 3 1 30 4 63 0 5 00 0 310 186 8 2092 80 4 2015 3 17 2 42 0 7 0 0 310 186 8 2092 80 4 2015 3 1 17 2 43 0 7 00 0 310 186 8 2092 80 4 2015 3 The format for input to SIGPLOT and for SIGPLOT template files has also changed slightly see the Addendum to the SIGPLOT User s Guide 67 68 13 Acknowledgements The material has been compiled through the jo
55. e experiments u was recorded as zero for the experiment with the highest concentrations In the data set stability category has been computed based upon the original method by Turner 1964 This is consistent with the method used for the other data sets but it does not very well take account of Norwegian winter conditions with snow covered ground Pay attention to the time format which is a four digit format like that of Copenhagen e g 1030 for 10 30 Pay attention to the following problems when using data from Indianapolis 15 16 e There is a mixing height of 0 m for several night time hours September 21 28 and 29 Rawinsonde showed a ground based inversion on the hours in question 3 2 Frequently Asked Questions In the Kincaid data set focus is on arc wise concentrations What should I do if I am interested in the entire data set If you wish to inspect the concentrations visually then the Dispersion Visualisation Tool is an excellent option see Chapter 8 There is also an alternative which requires the commercial software package Grapher as described in Chapter 9 The data are provided in the data set SF6_ALL DAT Note that all concentrations are included in SF6_ALL DAT also those considered outliers details in the file OUTLIERS TXT Further note that the unit for concentrations in this file is ppt contrary to other concentration data in the package As an alternative you can find a v
56. ed Model Validation Kit has been used for a series of workshops and conferences on Harmonisation within Atmospheric Dispersion Modelling for Regulatory purposes see www harmo org During the series of Harmonisation conferences many papers have used the Kit which was introduced in 1993 The present Guide describes the material after a revision in autumn 2005 This chapter serves as a key to the entire material Its purpose is to give you a background so you can assess how well the kit fulfils your needs and give you a qualified background to decide which parts of the Kit you will work with 2 4 Some basic recommendations It is recommended that any model evaluation exercise start with clear definitions of the evaluation goal and the variables to be considered followed by exploratory data analysis as explained in Section 2 5 and then statistical performance evaluation The implications of this are discussed more closely in the User s Guide to BOOT which is part of the material at hand Chang and Hanna 2005 Thus statistical model performance evaluation should not be a stand alone exercise It is highly recommended to be coupled with exploratory data analysis which can reveal model errors and errors and inconsistencies in data The Model Validation Kit offers tools for this 2 2 The Model Validation Kit The Model Validation Kit is intended to be used for evaluation of atmospheric dispersion models It is a collection of four fiel
57. ed concentrations are considered reliable e Data from an experiment in Lillestrom Norway 1987 with tracer releases from a non buoyant source at 36 m in stable winter conditions Sampling took place during 8 15 minute periods not during an entire hour Therefore when comparing observations with models yielding one hour averages crosswind integrated Quality indicator BOOT is a general tool Performance measures considered in BOOT 10 concentrations can be compared without problems whereas it is not straightforward to compare arc wise maxima e The Indianapolis experiment 1985 with tracer releases from an 84 m power plant stack in the city of Indianapolis USA There are 170 hours of tracer data from monitoring arcs at distances from 0 25 to 12 km The emphasis is on arc wise maxima One experience from the past work an experience that has been repeatedly confirmed is the usefulness of assigning a quality indicator to experimental data indicating how reliable a particular set of observations is Such a quality indicator can be assigned by subjective methods e g inspection of graphs or assigned by a computer code according to certain objective criteria The use of a quality indicator is valuable because subsets of data can be selected in a well defined manner This can be utilised to discard data that would have been misleading if they were blindly included in an analysis For two of the experiments Kincaid and I
58. enhagen 53 6 4 Lillestr m 54 6 5 Indianapolis 56 6 6 Hints on automatizing the process 57 6 7 Hints on software problems 57 SIGPLOT software 59 Dispersion Visualisation Tool 60 Tools for Grapher plots 61 Video clips from Kincaid 62 Notes on the ASTM package 65 Changes since previous version 67 Acknowledgements 68 References 69 14 1 Addresses 71 Danish Summary Dansk resum 72 Summary The so called Model Validation Kit is a compilation of field data sets software and documentation that provides a framework for evaluation of atmospheric dispersion models The kit has been used extensively by a large number of research groups since it was first introduced in 1993 In particular it has been used for a series of workshops and conferences on Harmonisation within Atmospheric Dispersion Modelling for Regulatory purposes see www harmo org The present report is a User s Guide to the kit and provides an overview of the entire material Besides data sets and software for model evaluation the package also includes supplementary material such as a data visualisation tool and video film from experiments In autumn 2005 the Model Validation Kit has undergone a major revision resulting in version 2 0 The new version allows the same studies to be carried out as the previous version but it has been revised in several respects New software and computing environments have made it necessary to update the package Furthermore the
59. er by Turner J Appl Met 1964 3 p 83 this is the case also for Copenhagen and Lillestrom data Be careful about units when using NWS data they differ from the European units The frequency distribution of the quality index for tracer arc hours is as follows QUAL Frequency 0 442 1 256 2 248 3 338 1 1284 24 Table 4 Samples of the most important data files from Kincaid MISC KIN DAT 80 4 20 14 80 4 20 15 80 4 20 16 80 4 25 12 80 4 25 13 80 4 25 14 80 4 25 15 80 4 25 16 80 4 25 17 80 5 1 16 MET K1 DAT YR MO DY HR 80 4 20 14 80 4 20 15 80 4 20 16 80 4 25 12 80 4 25 13 80 4 25 14 80 4 25 15 80 4 25 16 80 4 25 17 80 5 1 16 80 5 1 17 80 5 1 18 80 414 1 80 414 2 80 414 3 80 414 4 80 414 5 80 414 6 80 414 7 80 414 8 80 414 9 80 4 14 10 80 4 14 11 80 4 14 12 80 4 14 13 80 4 14 14 80 4 14 15 80 4 14 16 80 4 14 17 80 4 14 18 80 4 14 19 80 4 14 20 80 4 14 21 80 4 14 22 80 4 14 23 80 4 14 24 80 415 1 80 415 2 MET K2 DAT 80 4 20 14 80 4 20 15 80 4 20 16 80 4 25 12 80 4 25 13 80 4 25 14 80 4 25 15 80 4 25 16 80 4 25 17 80 5 1 16 80 5 1 17 80 5 1 18 80 5 1 19 80 5 1 20 80 5 121 80 5 4 7 80 5 4 8 80 5 4 9 MET_K3 DAT YR MO DY HR 80 4 20 14 80 4 20 15 80 4 20 16 80 4 25 12 80 4 25 13 80 4 25 14 80 4 25 15 80 4 25 16 80 4 25 17 80 5 1 16 80 5 1 17 80 4 20 14 80 4 20 14 80 4 20 14 80 4 20 14 80 4 20 14 80 4 20 14 80 4 20 15 80 4 20 15 ZIPRE DTHDZ 999
60. ersion of the Kincaid data set in the ASTM package see Chapter 11 prepared by John Irwin of the US EPA NOAA The package contains a file KINReanArcs DAT where data have been organised in arcs Note that in this version a few outliers are marked as negative concentration values consult the documentation in the package for further details If you are interested in the entire data set from Indianapolis you will find 170 files with data in the folder Field dataNMIndNXY conc standard As for Kincaid you can find an alternative file in the ASTM package How can I create a Command Line environment run DOS on a PC with Windows XP Windows 2000 See Section 6 2 5 What has happened to outliers in the data sets For Kincaid five values from the raw data set have been discarded See the file OUTLIERS TXT in the folder Field dataWKin for details 4 Package contents When the Model Validation Kit is distributed on CD the material is organised in folders as described below The material can also be downloaded from the Web in a number of packages self extracting zipped files The CD with the complete Model Validation Kit contains the following elements e This compendium where most of the documentation related to the kit is compiled Resides in the root folder of the CD e Field data from Kincaid Indianapolis Copenhagen and Lillestr m Folder Field data e Boot software package The complete package is in folder Boot while
61. erved values were unreliable 5 cases and they have been removed details in data set OUTLIERS TXT This has resulted in a total of 1284 arc hours in the data set 19 Warning irregular concentration patterns 20 It is important to note that the concentration pattern is often irregular for the Kincaid experiment high and low concentrations may occur intermittently along an arc Figure 2 shows an example Therefore it is often difficult to determine a representative maximum concentration along a crosswind arc of monitors Further there may be gaps in the monitoring arcs Therefore a variable has been assigned to each monitoring arc indicating how reliable the arc wise maximum should be considered This quality indicator has been assigned by Earth Tech on the basis of manual inspection of the geographical patterns of concentration distribution The criteria for assigning the indicator are shown in Table 1 104 i 0023 ap o 110 119 o i e o o o 64 38 E i eem s 4 o 000 0009 7km 2 see o ve o d woo 10k SE o 13 A A 8 03 N 3km ei 2km o 0 Source X 1km I 6 4 2 o 2 4 6 East West km Figure 2 Geographical distribution of measured concentrations at Kincaid 22 May 1981 10 11 hours Values are in ppt and the arcwise maxima are enclosed in circles The complete set of tracer measurements at all monitors is distributed in the file SF
62. gseksperimenter Model Validation Kit har gennemg et en omfattende revision i efter ret 2005 Den ny version version 2 0 muligg r de samme analyser som den tidligere men er blevet opdateret p forskellige omr der is r hvad ang r software Endvidere er dokumentationen blevet betragteligt forbedret og opdateret Pakken med Model Validation Kit kan downloades fra Internettet p adressen www harmo org kit
63. hat it generates Run Combine CphLilexe by double clicking its icon The program creates 4 files CP INP CPY INP CPSCAT DAT CPYSCAT DAT The letter Y refers to crosswind integrated concentrations The two INP files are input files for BOOT for arcwise maximum and cross wind integrated concentration respectively and the two DAT files can be used to create scatter plot You can then run BOOT in a similar way as for Kincaid There are so few data that there is not much sense in creating box plots Therefore the RESIDUAL programme will not be of very much use in the analysis of the Copenhagen data except that RESIDUAL can be used to sort data so that quantile quantile plots can be produced Template files for SIGPLOT INO files have been prepared so you can use SIGPLOT to produce z Simple scatter plots c vs c and CY poa vs Cy mod Quantile quantile plots cumulative distribution for arc wise maxima and cross wind integrated concentrations In order to carry out an analysis of the variation of model results with parameters it is recommended to use a utility like Microsoft Excel The starting point can be the INP files which can be edited slightly It is characteristic for the Copenhagen data set that the monitoring arcs never catch the maximum ground level concentration The concentrations decrease with distance indicating that the maximum is closer to the source than the nearest arc Therefore it is
64. he Dispersion Visualisation Tool 9 Tools for Grapher plots The Dispersion Visualisation Tool provides a very intuitive way of displaying concentration data For some purposes however it is to be preferred to display data on 2D maps with colour coding like the one shown in Figure 9 There are utilities for this in the folder Grapher tools and the procedure is explained in the document XY Explanation txt You must have installed the commercial software Grapher for Windows if you wish to apply the utilities iu ete 850916 Hour 11 122 Figure 9 Geographical distribution of concentrations for a specific hour during the Indianapolis experiment A plot file has been prepared with the utilities and subsequently displayed with Grapher 61 Introduction Kincaid12JulyPlant Close up of plant Shows stack and dirty plume 0 18 Kincaid14JulyLooping Study of plume looping periodicity 62 10 Video clips from Kincaid The Kincaid experiment in 1980 was sponsored by the Electric Power Research Institute EPRI in the USA EPRI through Dr Charles Hakkarinen has kindly given permission so a set of video clips here can be freely distributed The video clips were originally on a movie put together by TRC Environmental Consultants who were contracted to conduct field experiments The clips have been edited and put in digital format by H R Olesen of the National Environmental Research Institute Denmark Most of the
65. hold on the ratio e g 100 Do you wish to use extended capabilities of RESIDUW Y n Do you wish to impose a filter so that the ratio C MOD C OBS is considered unity when both C MOD and C OBS are small This will remove noise thus making it easier to diagnose tendencies in data Y n y What is the threshold for this filter e g 15 15 Normally on box plots produced by RESIDUAL the following percentiles are indicated 1 2 4 16 50 84 and 9746 For plots of ratios this is not always pertinent Often the boxes stretch over many decades so that no tendencies can be seen in such plots Therefore you can choose to use an alternative set of percentiles 2 5 25 50 75 95 Enter 1 or 2 according to your desire gt gt gt gt Do you want to create a file suitable fora quantile quantile plot Y n Name of additional output file suggested type QUA ki3 qua Reading data Summary of choices made A lower threshold of 9 99999978E 03 has been imposed on the ratio An upper threshold of 100 000000 has been imposed on the ratio A filter has been imposed on small values of variables The filter value is 15 00 The ratio has been changed to unity in 23 of 338 cases The alternative set of percentiles 5 25 50 75 95 is used The file ki3ratio dat has been created The file ki3 qua has been created as basis for q q plots C MvK Tools Run1 gt 49 Table 16 Creating four box plots
66. ilable from September and October 1985 and represent all stability classes and most wind speed ranges Data were taken in 8 or 9 hour blocks There are a total of 19 such blocks in the Indianapolis dataset During a test day trace gas emission and stack measurements began one to two hours before the field sampling began The 83 8 m stack at the Perry K plant is located in a typical industrial commercial urban complex with many buildings within one or two kilometres of the stack For example the Hoosier Dome sports stadium is a few hundred meters to the east Concerning potential influence by this building Steve Hanna writes Our analyses of the data Hanna and Chang 1991 1993 and an independent wind tunnel study have suggested that the Hoosier Dome and the other buildings do not influence the plume which tended to rise a hundred meters or more above the stack top most of the time As a result our modeling exercises have ignored the effects of nearby individual buildings We specify a surface roughness length of 1 m in order to parameterize the overall effect of the buildings on the boundary layer We also specify a minimum Monin Obukhov length L of 50 m during stable conditions for the urban area in order to account for the fact that the urban boundary layer does not stabilize significantly due to the mechanical mixing generated by the buildings and due to the anthropogenic heating in the urban area Hanna and Chang 1991 If other mode
67. ing arc for each hour The hour notation refers to the one hour period ending at that time The file INDI description doc provides a description of data which is more detailed than the one presented above The following files are supplied in the Field_data Ind folder Information carrying files README TXT Gives an overview of the material INDI description doc A brief description of the available data sets for the Indianapolis study PAR INDLTXT Can be regarded as an Appendix to the description in INDI_description doc Contains details on missing parameters etc XY DOC TXT Describes format of files with detailed concentrations for each hour There are 170 such files located in the subfolder XY conc SOND_DOC TXT Describes the format of upper air data file INDY30 DAT Data files INDI DAT and its equivalent INDI XLS INDI DAT is in ASCII format It contains mainly meteorological variables but also includes information on emission This file contains all information needed to model dispersion It has very long lines the record length is 542 INDI XLS contains the same information as INDI DAT but in EXCEL format SF6 IND DAT Contains SF6 measurements Modellers need normally not use this file If they want to compare model results to measurements 37 it is easiest to use the program COMBINE INDI as described in Chapter 6 85101123 DAT etc 170 files in this format are provided with detailed concentration measurements o
68. instructions This procedure is relatively simple and thus has some limitations For the Kincaid experiment there is also supporting material that can be useful video clips and a Dispersion Visualisation Tool see Chapters 8 and 10 Note that although the emphasis of the Model Validation Kit is on the protocol some tools included in the Kit in particular the BOOT software are general and can be applied for problems beyond the scope of the protocol When the Model Validation Kit is distributed on CD the material is organised in folders as described in Chapter 4 on Package contents Here in the documentation we use the folder names of the CD The material can also be downloaded from the Web in a number of packages self extracting zipped files 2 3 Datasets The Model Validation Kit addresses the classic problem of a single stack emitting a non reactive gas The Kit comprises data from the following four field experiments e The Kincaid experiment 1980 81 with tracer releases from a 187 m stack There are 171 hours of tracer data from monitoring arcs at distances from 0 5 to 50 km In the Model Validation Kit the emphasis is on arc wise maximum concentrations e Data from an experiment in Copenhagen Denmark in 1978 79 with releases from a non buoyant elevated source 115 m in neutral and unstable conditions Nine hours of tracer data are available on arcs from 2 to 6 km Both arc wise maxima and crosswind integrat
69. int efforts of many persons Steve Hanna has provided the data sets from Kincaid and Indianapolis Joe Chang has created much software Alexandar Markovski has created the Dispersion Visualisation Tool and Sven Erik Gryning Per Lefstrom Chuck Hakkarinen Russ Lee and Dag T nnesen have contributed in various ways The EPRI Air Quality Data Center was operated by Earth Tech in the early nineties when the data were provided The Kincaid and Indianapolis data were put at disposal on behalf of EPRI by Earth Tech Steve Hanna and Joe Chang Joe Chang has developed the major part of the software including BOOT SIGPLOT while Helge Rordam Olesen has provided supplementary utilities EPRI through Dr Charles Hakkarinen has kindly given permission so a set of video clips from Kincaid can be freely distributed John Irwin has assembled the ASTM package which is described in Chapter 11 but is not part of the Model Validation Kit 14 References Bowne N E and Londergan R J 1983 Overview Overview Results and Conclusions for the EPRI Plume Model Validation and development Project Plains Site EPRI report EA 3074 Chang J C and Hanna S R 2004 Air quality model performance evaluation Meteorol Atmos Phys 87 167 196 Chang J C and Hanna S R 2005 Technical Descriptions and User s Guide for the BOOT Statistical Model Evaluation Software Cuvelier C editor 1994 Proceedings of the workshop Intercomparison of Advan
70. ions of interest are near centreline concentrations NCCs NCCs bear some relation to arc wise maxima but for a given experiment and arc there may be several NCCs as opposed to only one arc wise maximum NCCs are selected among those observations that lie within a distance of 0 67 c from the cloud centre where 6 is the cloud width The software of the ASTM package is capable of retrieving NCCs from observations and process them in accordance with the prescribed methodology Note however that selecting NCCs is not straightforward as it depends on regime definitions and there are questions as to which arcs can be considered having enough data for NCCs to be defined The ASTM package includes software documentation and three datasets Kincaid Indianapolis and Prairie Grass The data in the package have not been quality flagged but the software performs certain checks when retrieving NCCs 65 66 The ASTM procedure implemented in the BOOT software in the Model Validation Kit assumes that NCCs have been retrieved separately The ASTM procedure represents a framework and is not a fixed protocol For example regimes can be defined in many different ways and this may lead to differing results of a performance evaluation depending on regime definitions Altogether the ASTM procedure represents a promising approach but still with some issues that are not fully resolved Some issues deserving attention are There is a need to stu
71. lers would like to directly model the influence of the buildings or other aspects of the urban area they can find the buildings locations in the TRC 1986 report The EPRI report Urban Power Plant Plume Studies EPRI EA 5468 contains the following description of the urban meteorological site D on the map The urban site was located at the northwest corner of Ohio and Senate Streets adjacent to a State employees parking lot This area was surrounded by large buildings and received a heavy volume of traffic The site was located approximately 1 5 km northeast of the Perry K plant ES mere L ACH als bee T IA Te pul Bg MO Ai m d DE E AE an i s fal a zx Teale I Ng 1n Elea arising pi 100 ve ee PEERS R B he aT SCALE meters Figure 4 Map showing the relationship of the Perry K Station A the Hoosier Dome Sport Stadium B and the central Indianapolis business district C The downtown surface meteorological site is located at D and the bank tower site was on the top of the building at E The horizontal and vertical scales are equal from TRC 1986 35 Figure 5 Location of meteorological sensors The filled circle is the Perry K power station The filled triangles are measurements of surface temperature The asterisks are the primary meteorological sites The filled square is the rawinsonde launch site from TRC 1986 5 4 2 Meteorological data
72. month day hour nlev istop 5x a5 5x 4i2 5x i12 t69 12 or 5x a5 5x 412 5x 12 t69 12 I O statement used to write the data records pres i height i temp i wd i ws i i3 13 5 2 Copenhagen 5 2 1 Experimental set up The experiments in question took place in the Northern part of Copenhagen in 1978 79 A full description can be found in Gryning 1981 available on request A shorter description is found in a paper by Gryning and Lyck 1984 available from Helge Rerdam Olesen on request A comprehensive data report is available electronically Gryning and Lyck 2002 The dispersion experiments were carried out under neutral and unstable conditions The tracer SF was released without buoyancy from a tower at a height of 115 m and collected at ground level positions in up to three crosswind series of tracer sampling units positioned 2 6 km from the point of release The site was mainly residential having a roughness length of 0 6 m For all runs during the experiment the crosswind profiles of tracer concentrations were well determined thus making a relatively accu rate estimate of crosswind integrated concentration possible The maximum concentrations given in the data set is the highest observed concentration along each arc The release took place in the suburb of Gladsaxe latitude 55 735 N longitude 12 494 E the height of terrain is 49 m a m s l The tracer sampling units were mounted at lampposts at a height of
73. nal cloud cover EIGHTS CEIL Height to lowest cloud m Derived meteorological parameters UST Friction velocity from profile m s L Monin Obukhov length m HF Heat flux W m2 TURNER Turner stability class according to Turner 1964 Tracer release parameters DIST Distance km to arc of monitors Q Emission rate g s SIGY Sigma y m at DIST CY Cross wind integ conc ug m2 at DIST CY Q CY normalized by emission times 10 6 s m2 10 6 ARCMAX Max conc ug m3 at DIST ARCMAX Q Conc normalized by emission times 10 9 s m3 10 9 HS Tracer release height m NUARC Number of arcs for an experiment D1 D3 Distance km to arcs for an experiment Notes on parameters Measured values of SIGV and SIGW are considered more reliable than those deduced from profile measurements The values of T2 W10 etc have been formed as the average of 6 10 minute averages HF UST and L have been determined using measurements from a met tower two levels of temperature one level of wind speed T2 T40 and W10 A roughness length of 0 6 m has been assumed For the processing the subroutine PBL 1 of the software library PBL MET described by Sozzi and Fraternali 1994 has been used Turner stability class is computed according to the original paper by Turner J Appl Met 1964 3 83 Changes compared to the data set distributed for the Manno workshop The following parameters have been included W60 T120 NCLD C
74. ncaid experimental campaign See the list of references and the list of addresses in the back for details Table 2 Contents of the file PAR KIN TXT Parameters supplied in the distributed files from Kincaid 3 files with each 2040 obs 4 files with each 171 obs 1 file with 1284 obs Basic parameters YR Year MO Month DY Day HR Hour end of hour GMT 6 Observed meteorological parameters PRES Pressure mb NET Net radiation W m2 TOT Total radiation W m2 DP100 Dew point temperature at 100 m K T100 Temperature at 100 m K T50 Temperature at 50 m K T10 Temperature at 10 m K ZI Mixing height observed m DTHDZ Pot temp grad between 100 50 m K m WS100 Wind speed at 100 m m s ws50 Wind speed at 50 m m s WS30 Wind speed at 30 m m s WS10 Wind speed at 10 m m s WD100 Wind direction at 100 m deg WD50 Wind direction at 50 m deg WD30 Wind direction at 30 m deg WD10 Wind direction at 10m deg SWD100 Sigma WD100 deg SWD50 Sigma WD50 deg SWD30 Sigma WD30 deg SWD10 Sigma WD10 deg SIGW Sigma of vertical velocity at 100 m m s SIGV Sigma of cross wind speed at 100 m m s FLAG 2 CEILNWS Ceiling 100 s of feet 1 unlimited DPNWS Dew point temp F WDNWS Wind direction deg WSNWS Wind speed knots PNWS Pressure inch Hg TNWS Temperature F NNWS Cloud cover 1 10 PRECNWS Precipitation mm Derived parameters ZIPRE Predicted mixing height m UST Fricti
75. ndianapolis the tracer data have been flagged by a manually assigned quality indicator assessing the quality of arc wise maximum concentrations The quality index has values of 0 1 2 and 3 with 2 and 3 representing the most reliable data Comparison studies of observed data with model results should in general be conducted with a quality indicator of 2 or 3 The data sets are described in the chapter Field data 2 4 The BOOT software The main tool for statistical performance evaluation is the BOOT software package The BOOT program has been improved and is now available in version 2 0 with a comprehensive rewritten User s Guide Chang and Hanna 2005 Besides detailed technical description of performance measures and the use of the software the User s Guide also provides a discussion of model evaluation objectives and exploratory data analysis The BOOT package is flexible and general in nature Although it has been primarily used to evaluate the performance of air dispersion models the same procedures and approaches implemented in BOOT also apply to other types of models Compared to the previous version of BOOT the program now includes some additional performance measures and an implementation of the ASTM statistical model evaluation procedure see later The BOOT package is capable of computing performance measures such as the Fractional Bias FB the Normalised Mean Square Error NMSE the Geometric Mean Bias MG the Geome
76. ne for each hour These data files are located in the subfolder XY conc They are not necessary for normal use of the Model Validation Kit INDY30 dat Radiosonde data from an urban and a rural radiosonde 5 4 5 Additional information A full description of the Indianapolis field study is given by TRC 1986 and some results of analysis are given by Hanna and Chang 1991 1993 The data included here represents a subset of the full data set which includes many magnetic tapes full of lidar data and fast response turbulence data The data have been supplied by the EPRI Atmospheric Science Data Center operated by Earth Tech USA The persons involved were Steve Hanna and Joe Chang Further preparation of the data was performed by H R Olesen National Environmental Research Institute Denmark 5 4 6 Points to be noted There are missing values for a number of variables Details are listed in the file PAR INDI DAT To most modellers the missing values will not be any problem except possibly for the case of September 29 where winds from the 94 m level are missing Further note that there is a mixing height of 0 m for several night time hours September 21 28 and 29 Rawinsonde showed a ground based inversion on the hours in question Table 10 Sample from SF6_IND DAT and one of the files with detailed concentrations 85091611 DAT The file INDI DAT or INDI XLS has too long records to display in a table SF6 IND DAT YR MO
77. of interest also to investigate how a model behave closer to the source than the monitoring arcs In a modelling exercise for a workshop in 1994 it was prescribed that modellers should compute maximum ground level concentrations and cross wind integrated concentrations at the following distances km 0 2 0 3 0 4 0 5 0 6 0 7 0 9 1 0 1 3 1 6 1 9 2 2 3 0 4 0 You may encounter some results based on such data and you may wish to perform similar calculations with your model 6 4 Lillestr m Note concerning selection of data The experimental period on 87 02 09 1015 1030 is interesting because it has the highest concentrations However it cannot be reproduced by all models requiring L and u as input because L is undetermined and u indicated as zero Therefore for a baseline comparison you should omit the data from the last experiment The procedure for processing Lillestr m data is similar to that of Copenhagen Thus Procedure for modelling Merging modelled values and observations 1 There are only 21 data points and therefore it does not make sense to produce box plots like those for Kincaid Scatter plots of various kinds are more informative You can use SIGPLOT to produce some of these but in this case SIGPLOT has no particular advantages over other plotting packages 2 Itis pertinent to consider crosswind integrated concentration values On the other hand one should exert care if model results for arc
78. ombine xxx utilities are intended to be executed in a Windows environment If you use them from a command line they will also work but they behave awkwardly in case of an error Seemingly the program just freezes Actually a separate window has appeared which may be hidden behind other windows The window tells you that there is an error and you have to click an OK button in order to continue working in the command line environment If you work from a command line and want to avoid such behaviour you can find an alternative version of BOOT called BOOT NO PAUSE EXE in the Boot folder The required files for input and template files for SIGPLOT has changed in certain respects since the version distributed before 2005 In case of problems consult the Addendum to the SIGPLOT User s Guide found in folder SIGPLOT 7 SIGPLOT software The SIGPLOT software is offered as an option but it must be recognized that the software is old and the documentation not complete The easiest way to get started with Sigplot is to follow the step by step explanation in Chapter 6 SIGPLOT requires that you work in a command line environment DOS environment The details are given in Section 6 2 5 The SIGPLOT software is described in the User s Guide from 1991 which is available as a scanned pdf file SIGPLOT User s Guide 1991 pdf in folder stcpLot The software has been enhanced over the years so you must also consider the Addendum to the User
79. on velocity m s WST Convective velocity m s L Monin Obukhov length m TURNER Turner stability class based on NWS data Tracer parameters FILE NAMES MET_K1 L DAT MET_K2 L DAT MET_K3 L DAT MISC KIN DAT MET_K1 DAT MET K2 DAT MET K3 DAT 1 tee tt 44 4 Q Emission rate g s TQ Gas temp K VsQ Gas exit velocity m s ARCMAX Max conc ug m3 at DIST 3 DIST Distance km to arc of monitors AZMAX Direction deg to ARCMAX ARCMAX Q Conc normalized by emission times 10 9 s m3 10 9 QUAL Quality indicator for ARCMAX 1 5 6 observations are substituted with converted NWS observations 2 Value of flag describes which of the parameters T10 WS10 or WD100 are substituted For each of the substitutions FLAG is added the value 1 3 Converted from ppt by multiplying with 1 758 p T 2 resp 4 no subs The file DISTM_K DAT is meant as a help for deciding at which distances computations 12 different distance values appear in should be performed where SF6 was measured the data set Parametre Units YR Year MO Month DY Day HR Hour GMT 6 NUARCM Number of arcs with values of max conc DM1 Distances km to arcs with DM2 values of ARCMAX DM12 ARCMAX DISTM_K 444 FLAG 0 SF6 KIN DAT tee tt 23 Table 3 Contents of the file PAR KIN TXT continued N
80. ons some potential pitfalls are pointed out Before using the data also carefully inspect the files PAR KIN TXT PAR CPH TXT PAR LIL TXT and PAR INDLTXT which contain important notes One basic detail The von Karman constant x has been assumed to have a value of 0 40 in the data presented 5 1 Kincaid The Kincaid related files are located in the folder Field dataNKin Note that there is video from the Kincaid experiment in the folder Kincaid video see Chapter 10 and that the Dispersion Visualisation Tool described in Chapter 8 can be used to visualise observed concentrations 5 1 1 Experimental set up The Kincaid field experiment was performed as part of the EPRI Plume Model Validation and Development Project A very compre hensive experimental campaign was conducted in 1980 and 1981 A large number of reports concerning the Kincaid experiment have been published by EPRI including Overview Results and Conclusions for the EPRI Plume Model Validation and development Project Plains Site Bowne and Londergan 1983 which gives a good overall description of the Kincaid experimental campaign The Kincaid power plant is situated in Illinois USA 39 59 N 89 49 W and is surrounded by flat farmland with some lakes The UTM coordinates are 285 66 Easting and 4385 10 Northing The terrain is at an elevation of approximately 180 m a m s l The roughness length is approximately 10 cm There is further information on geog
81. ons with predicted centerline concentrations will not be fair a problem related to an implementation of the ASTM procedure Pay attention to the following problems when using data from Kincaid The derived meteorological parameters u w L and h should be used with care or replaced c and o are suspected to be unreliable It is recommended to use data with a quality indicator of 2 or 3 when analyzing model behaviour One point is important to be aware of observations with QUAL 3 are biased in the sense that they are never zero pred Pay attention to the following problems when using data from Copenhagen The tracer monitoring arcs were in general placed at distances where the concentration was decreasing i e the maximum was closer to the source than any of the arcs It is observed Gryning and Tassone 1994 that measured values of c are smaller than predictions by theory The computed heat flux values may not be representative for a greater area When using the enclosed tools pay attention to the format used for time E g 1417 means 14 17 whereas Kincaid and Indianapolis data are given for integer values of hour Pay attention to the following problems when using data from Lillestrom The averaging period is only 15 minutes for the tracer data Concentration averages taken over longer time will tend to be smaller than those registered due to meandering There was generally very light wind during th
82. ot and box plots as for S SE Kincaid E S E INRATIO INQ AERE o 7 5 INOBSN INMOD INQ 52 Procedure for modelling Merging modelled values and observations 6 3 Copenhagen In order to process Copenhagen data you must go through a procedure somewhat similar to the one outlined above However some changes are appropriate 1 There are only 23 data points and therefore it does not make sense to produce box plots like those for Kincaid Scatter plots of various kinds are more informative You can use SIGPLOT to produce some of these but in this case SIGPLOT has no particular advantages over other plotting packages Some template files for SIGPLOT are included see below 2 Besides arc wise maxima it is pertinent also to consider crosswind integrated concentration values These are included among the observed data and can be considered relatively reliable 3 Note that the format for time is different from that for Kincaid E g 1417 means 14 17 whereas Kincaid and Indianapolis data are given for integer values of hour Perform model calculations for the 13 distances represented in the Copenhagen data set 1 9 2 0 2 1 3 6 3 7 4 0 4 1 4 2 5 3 5 4 5 9 6 0 6 1 km Dump your output in a file Here we will call the file Modelled cph you may use another name The format of Modelled cph should be the following e There is a one line heading e There should be seven columns o
83. ot impossible to do but requires considerable work with quality control the subject is discussed by Olesen 2001 6 2 2 Matching model results with observed data The Model Validation Kit contains a utility for combining model results and observed data resulting in a file that can be directly used as input to the statistical model evaluation program BOOT The utility Combine kin exe allows selection according to quality level so that either data with a quality indicator of 2 or 3 are considered 586 observations or exclusively data with a quality indicator of 3 338 observations Before using the tool you must edit the file Combine Kin ini This is a plain text file which tells the program where it can find the model results and where it should place the files that it generates The file is reproduced in Table 11 41 Table 11 The file Combine Kin ini The user should adjust the three names indicated in yellow to suit his needs In the example here the user has placed the utilities in C MvK Tools while his model results are in C MvK Tools Run1 Comments General This file contains options for the programme Combine kin exe Capitalisation Keywords always have the first letter capitalised Files Results poth MENNENEENENENEENI Example C xxx This is the path where the program expects to find the file with model results The same folder is used to store the output files resulting from the program
84. plot the KISOBS DAT file A template file NKKIOBS INQ has been prepared for this purpose The last item 4c can be produced exactly corresponding to 4b Run RESIDUAL where you indicate the numbers 2 0 in order to analyze Caa Then run SIGPLOT with template file KIMOD INQ mod 6 2 8 Recapitulation Run your model Run Combine kin exe after having prepared Combine kin ini Run BOOT Run RESIDUAL and SIGPLOT a number of times It is not necessary to use Residual to generate a simple scatter plot but it must be used if you wish to generate a quantile quantile plot or plots with boxes e Run TEKPLOT to see plots and PS to produce EPS versions of the files Figure 6 and Figure 7 show samples of plots produced by the procedures outlined here Table 17 gives an overview of utilities in the Tools folder Table 18 lists all of the predefined template files for SIGPLOT Note that the procedure can be automatized see Section 6 6 Table 17 Overview of utilities Tool What it does COMBINE xxx Produces a file of type INP that can be directly used as input for BOOT and a file xxx is kin indi or CphLil suitable for scatter plots BOOT Produces files with statistics RESIDUAL Used repeatedly to prepare input files to SIGPLOT not needed for scatter plots but for all other plots SIGPLOT Produces plot files in Tektronix format e Scatter plot e Quantile quantile plot e Box plots analysing ratios s
85. r more separated by blanks the values are read using free format input e There must be a line for all arc hours 9 hours times 13 distances altogether 117 lines e The distances must be those indicated above An example of the first fourteen lines of a Modelled cph file is shown below The date and time is indicated then the distance then the normalised arcwise maximum concentration ARCMAX Q and finally the normalised crosswind integrated concentration CY Q YR MO DY HRE DIST ARCMAX Q cy Q 78 9 20 1417 1 900 332 374 78 9 20 1417 2 000 301 355 78 9 20 1417 2 100 245 336 78 9 20 1417 3 600 101 196 78 9 20 1417 3 700 92 191 78 9 20 1417 4 000 84 178 78 9 20 1417 4 100 81 174 78 9 20 1417 4 200 78 170 78 9 20 1417 5 300 555 146 78 9 20 1417 5 400 54 145 78 9 20 1417 5 900 49 141 78 9 20 1417 6 000 48 140 78 9 20 1417 6 100 47 140 Please note that the units for ARCMAX Q should be s m e10 and for CY Q s m e10 corresponding to the units used in the file with SE data SF6_CPH DAT You can use the utility Combine_CphLil exe to prepare an input file for BOOT as well as a file appropriate for a scatter plot 53 How does a model behave close to the source One experiment has a zero value of u 54 First you must edit the file Combine CphLil ini This is a plain text file which tells the program where it can find the model results and where it should place the files t
86. raphical coordinates in the file geo kin txt The power plant has a 187 m stack with a diameter of 9 m During the experiment SF was released from the stack The tracer releases started some hours before the sampling There is a nearby building with a height of approximately 75 meter It is rectangular 25 m by 95 m with the long side oriented east west The stack is 152 m south of the centre of the southern edge of the building and 182 m south of the tallest part of the building which has a maximum significant elevation of 74 4 m Selection of data 5 1 2 Meteorological data The data that you receive have been supplied by the EPRI Air Quality Data Center operated by Earth Tech formerly Sigma Research Cor poration The meteorological parameters u w L and h were derived by Earth Tech using pre processing methods described in Hanna and Paine 1989 Steve Hanna who was affiliated to Earth Tech when the data were prepared warns that these parameters should be used with caution because his suggested boundary layer formulas have been slightly modified since 1989 cf the paper by Hanna and Chang 1992 He recommends that modellers use their own pre processing methods Thus the presence of these parameters in the data does not indicate a recommendation of their use Observed mixing heights were determined manually by inter pretation of radiosonde data there were on site radio soundings several times a day We wish to w
87. residual plots where model residuals are depicted as a function of independent variables such as the downwind distance and time of day Examples are shown in Figure 7 in Chapter 6 It is recognised that the somewhat archaic SIGPLOT package is only one of the many ways of performing exploratory data analysis More modern and interactive tools than the SIGPLOT package can certainly be used to achieve the same goals For example a potential alternative is to use Microsoft Excel for data handling and graphical analyses Excel offers some very powerful tools for interactive data analysis In particular its Autofilter feature is useful for investigation of model behaviour Nevertheless Excel does not offer the specialised plots that SIGPLOT produces The advantages of using SIGPLOT are that you will be able to produce residual and other types of specialised plots with data in a relatively standardised format which has been used by others Furthermore the required utilities are already prepared and the procedures for using the software are described in detail The drawback is that you will have to work in a DOS environment Section 6 2 5 provides some hints on this More details on Sigplot can be found in the chapter Step by step instructions as well as in the chapter SIGPLOT software 2 6 Limitations It must be recognised that model evaluation studies performed on the basis of the Model Validation Kit are limited in scope These limitations
88. se and if the data represent a sufficient number of scenarios then a perfect model is likely to underpredict the highest observed concentrations this issue is elaborated by Olesen 1997 Note further that the so called quantile quantile plots from an entire experimental database should not stand alone as the result from a model evaluation A very useful supplement is residual plots which provide more insight into model behaviour Despite its limitations the Model Validation Kit has the advantage of being straightforward to apply and practically oriented It also provides a common framework where the results of different studies can be intercompared 24 An alternative The ASTM methodology As noted there is a concern that direct comparison of model predictions against observations could cause misleading results Therefore an alternative approach has been proposed by John Irwin and has resulted in ASTM Standard Guide D6589 This procedure has also been incorporated in the latest version of the BOOT software as an option The procedure is not treated in depth in the present compendium However there exists also a separate package software and data sets specifically devised as an implementation of the ASTM procedure here referred to as the ASTM package It was prepared by John Irwin and is available on the Internet www harmo org astm This is not part of the Model Validation Kit but it can be used as a supplement or an alternative
89. speed were used not research grade data The computed heat fluxes etc may be influenced by very local effects in one case in particular the heat flux was much larger than expected 5 2 3 Points to be noted The tracer monitoring arcs were in general placed at distances where the concentration was decreasing i e the maximum was closer to the source than any of the arcs It is observed Gryning and Tassone 1994 that measured values of c are smaller than predictions by theory The computed heat flux values may not be representative for a greater area 5 2 4 Additional information It is possible upon request to obtain supplementary meteorological data for the time before the measuring periods Please note that a comprehensive data report on the Copenhagen data set is available through the Web Gryning and Lyck 2002 Table 6 Contents of the file PAR CPH TXT Parameters supplied in the distributed files from Copenhagen Basic parameters YR Year MO Month DY Day HRS Time start of period GMT 1 Example 1317 means 13 17 HRE Time end of period GMT 1 Observed meteorological parameters W10 Wind speed at 10 m m s W60 Wind speed at 60 m m s W115 Wind speed at 115 m m s T2 Temperature at 2 m deg K T40 Temperature at 40 m deg K T120 Temperature at 120 m deg K SIGV Sigma of cross wind speed at 115 m m s SIGW Sigma of vertical velocity at 115 m m s ZI Mixing height observed m NCLD Fractio
90. t in Europe Mol Belgium Nov 1994 Int J Environment and Pollution Vol 5 Nos 4 6 pp 761 784 Olesen H R 1995b Toward the establishment of a common framework for model evaluation Paper presented at the 21 International Meeting on Air Pollution Modeling and its Applications in Baltimore Nov 6 10 1995 Olesen H R 1997 Tools for model evaluation In Air Pollution Modeling and Its Application XII pp 519 528 Edited by S E Gryning and N Chaumerliac Plenum Press New York Olesen H R 1997 Pilot study Extension of the Model Validation Kit Paper presented at the 4th workshop on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes Oostende Belgium 6 9 May 1996 Int J Environment and Pollution Vol 8 Nos 3 6 pp 378 387 Gives an introduction to data sets from Indianapolis and Bull Run the Indianapolis data set was later added to the Model Validation Kit Also conveys general experiences on the preparation and use of data sets for model evaluation Olesen H R 1997 Tools for model evaluation In Air Pollution Modeling and Its Application XII pp 519 528 Edited by S E Gryning and N Chaumerliac Plenum Press New York Contains additions to previous overviews of model evaluation activities Also contains a detailed discussion of one particular problem with the methodology of the Model Validation Kit observed arcwise maxima are compared directly to modelled concentrations This t
91. ted concentration possible The maximum concentrations given in the data set is the highest observed concentration along each arc Geographical coordinates for the position of release are latitude 59 889 N longitude 11 051 E the height of terrain is 110 m a m s l 5 3 2 Points to be noted Note that for Lillestrom data the averaging period is only 15 minutes for the tracer data Concentration averages taken over longer time will tend to be smaller than those registered due to meandering There was generally very light wind during the experiments u was recorded as zero for the experiment with the highest concen trations 31 In the data set stability category has been computed based upon the original method by Turner 1964 This is consistent with the method used for the other data sets but it does not very well take account of Norwegian winter conditions with snow covered ground Table 8 Contents of the file PAR LIL TXT Parameters supplied in the distributed files from Lillestrom Note the last observation in the data set was performed in conditions with VERY low wind speed 0 2 m s at a height of 36 m L was not determined Basic parameters YR Year MO Month DY Day HRS Time start of period GMT 1 Example 1317 means 13 17 HRE Time end of period GMT 1 Observed meteorological parameters WS10 Wind speed at 10 m sonic anemometer m s WS36 Wind speed at 36 m cup anemometer m s four v
92. tion Kit 8 2 1 Some basic recommendations 8 22 The Model Validation Kit 8 23 Datasets 9 2 4 The BOOT software 10 25 Tools for exploratory data analysis 11 2 6 Limitations 11 27 An alternative The ASTM methodology 12 2 8 Forum for compilation of experiences a Wiki 13 29 Structure of the User s Guide 13 3 Pitfallsand FAO 14 3 1 Pitfalls 14 3 2 Frequently Asked Questions 16 4 Package contents 17 5 Field data 18 51 Kincaid 18 5 1 1 Experimental set up 18 5 1 2 Meteorological data 19 5 1 3 Tracer data 19 5 1 4 Data files 21 5 1 5 Points tobe noted 21 5 1 6 Additional information 22 5 2 Copenhagen 27 5 2 1 Experimental set up 27 5 2 2 Meteorological data 28 5 2 9 Points to be noted 28 5 2 4 Additional information 28 5 3 Lillestr m data 31 5 3 1 Experimental set up 31 5 3 2 Points tobe noted 31 5 4 Indianapolis 34 5 4 1 Experimental set up 34 5 4 2 Meteorological data 36 5 4 3 Tracer data 37 5 4 4 Data files 37 5 4 5 Additional information 38 5 4 6 Points to be noted 38 10 11 12 13 14 Step by step instructions 39 6 1 File naming conventions conventions for the example 39 6 2 Kincaid 40 6 2 1 Instructions on modelling 41 6 2 2 Matching model results with observed data 41 6 2 3 Analysing data with BOOT 42 6 2 4 SIGPLOT A tool for graphical analyses 44 6 2 5 Preparations to work with SIGPLOT 45 6 2 6 Using SIGPLOT 45 6 2 7 Creating Q Q plots and box plots 47 6 2 8 Recapitulation 50 6 3 Cop
93. tions After a few general comments a step by step explanation on the processing of the Kincaid data set follows in Section 6 2 which is quite long Processing of the other three data sets follow the same principles but with some deviations Such deviations are explained in the subsequent three sections one for each of the three remaining data sets Indianapolis data are treated similarly to Kincaid data while Copenhagen and Lillestrom data should be processed a bit differently Typically when the results of a model run are processed with the enclosed tools the outcome is the following products 1 A file with statistics FB NMSE etc 2 A scatter plot c 3 A quantile quantile plot cumulative distribution vs C mod obs 4 Plots for diagnosing model behaviour 4a Box plots analyzing the ratio c c parameters in terms of physical obs 4b Box plots analyzing the behaviour of c 4c Box plots analyzing the behaviour of cea The file with statistics is generated using the BOOT program while the plots can be generated using RESIDUAL and SIGPLOT All of the required programs and files are assembled in the Tools folder on the CD Thus for convenience the Tools folder includes a copy of the BOOT program as well as of the SIGPLOT program These two utilities can also be found in the folders BooT and SIGPLOT where they are accompanied by user manuals and sample files The subsequent sections describe in
94. tratified by physical parameters e Box plots analysing observations e Box plots analysing model results Tekplot Displays plot file on screen PS Converts plot file to EPS format 51 Table 18 Overview of template files for SIGPLOT Template file Input file Resulting plot supplied on created by one of diskette the Combine utilities or by the RESIDUAL i rogram KISCAT INQ KISCAT3 DAT Scatter plot KIQUA INO KI3 QUA Quantile quantile plot ul EIE 8 E Ks z KIRATIO INQ KI3RATIO DAT Ratio 4 plots on a page log ratio axis lt 3 x KIOBS INQ KISOBS DAT Observations 4 plots on a page lin conc axis GS 2 KIMOD INO KI3MOD DAT Model results 4 plots on a page lin conc axis 2 CPSCAT INQ CPSCAT DAT Two pages of plots presenting a scatter plot and a quantile quantile plot x 5 CPQUA INQ e 7 D 50 S S S S aS 5 5 S CPYSCAT INQ CPYSCAT DAT Two pages of plots as above but for crosswind S E integrated concentration t a 9 CPYQUA INQ H OB o di UN E LISCAT DAT Two pages of plots centerline concentrations as z LISCAT INO for Copenhagen zB g 2 LIQUAINQ S E 9 o D ds LIYSCAT INO LIYSCAT DAT Two pages of plots crosswind integrated g g pag P 8 2S8 concentrations layout as above a bb y o E LIYQUAINQ INSCAT IN 2 5 Q Five pages of plots for Indianapolis Scatter plot Fi u INQUA IN a quantile quantile pl
95. tric Variance VG the fraction within a factor of 2 FAC2 the Measure of Effectiveness MOE as well as several others FB and MOE are in fact closely related With the new software version FB and MG can be separated into overpredicting and underpredicting components Bootstrap resampling is used to estimate the confidence limits of a performance measure hence the name BOOT of the package Files related to BOOT The SIGPLOT graphical package features and drawbacks On the distribution CD the Boot folder contains the BOOT program a comprehensive User s Guide and various sample files The Tools folder contains additional utilities for use in the present context as described in Chapter 6 on Step by step instructions 2 5 Tools for exploratory data analysis When performing model evaluation it is not sufficient to consider just statistical evaluation that produces some performance metrics Rather it is recommended that exploratory data analysis also be performed using graphical techniques The Model Validation Kit includes some tools for such graphical analyses in the form of the SIGPLOT graphical package and the RESIDUAL utility The SIGPLOT package is offered as an option that is specifically tailored for model performance evaluation It must be mentioned that the SIGPLOT program as well as a number of associated utility programs included in the Model Validation Kit only function in a DOS environment The package can produce
96. umber of missing parameters and their dummy values in the distributed files from Kincaid FILE NAMES Para Dummy MISC KIN DAT MET K1 DAT MET K1 L DAT MET K2 DAT MET K2 L DAT MET K3 DAT MET K3 L DAT SF6 KIN DAT meters value YR 0 0 0 0 0 0 0 0 MO E 0 0 0 0 0 0 0 0 DY 0 0 0 0 0 0 0 0 HR 0 0 0 0 0 0 0 0 PRES 999 5 62 NET 999 8 202 TOT 999 8 379 DP100 999 52 370 T100 999 8 205 T50 999 8 205 T10 999 0 5 37 ZI 999 6 6 906 DTHDZ 9 9999 8 8 239 WS100 999 6 6 112 WS50 999 9 206 WS30 999 6 40 WS10 999 0 6 42 WD100 999 0 5 44 WD50 999 8 207 WD30 999 8 208 WD10 999 5 40 SWD100 999 6 190 SWD50 999 6 189 SWD30 999 6 189 SWD10 999 6 189 SIGW 9 99 18 18 461 SIGV 9 99 9 9 885 FLAG CEILNWS i 0 0 DPNWS 0 0 WDNWS E 0 0 WSNWS 0 0 PNWS 0 0 TNWS 0 0 NNWS 0 0 PRECNWS 0 0 ZIPRE 0 UST 0 WST 0 L 0 TURNER 0 Q 0 0 TO 0 VSQ E 0 DIST 0 ARCMAX 0 AZMAX 999 355 ARCMAX Q 0 QUAL 0 Notes on parameters The derived meteorological parameters have been included for reference They are computed using one of many possible methods and their inclusion in the data set does not indicate a recommendation of their use It is recommended that modellers use their own processing methods Measured values of SIGW and SIGV are not to be considered reliable The Turner stability class has been computed based on NWS data It is included for reference it is computed according to the original pap
97. w considered missing 32 Table 9 The full contents of data files from Lillestr m MET LIL DAT YR MO DY HRS HRE W10 W36 T10 DT SIGV SIGW ZI NCLD CEIL UST L TURNER 87 01 10 930 945 2 1 4 4 247 7 0 1 0 62 0 47 999 0 1 0 374 235 4 87 01 10 945 1000 1 7 3 6 247 7 0 2 0 54 0 42 999 0 1 0 283 130 3 87 0112 930 945 1 7 3 0 252 7 2 3 0 32 0 22 999 0 1 0 173 27 3 87 01 12 945 1000 1 6 3 1 252 7 1 3 0 32 0 22 999 0 1 0 173 41 3 87 01 17 1000 1015 0 9 99 9 252 0 1 5 0 24 0 10 999 0 ai 0 224 1601 3 87 01 17 1015 1030 0 5 99 9 252 0 1 3 0 17 0 10 999 0 1 0 100 188 3 87 02 09 1000 1015 0 5 99 9 260 4 0 2 0 17 0 17 999 4 3050 0 100 8 1 3 87 02 09 1015 1030 0 4 99 9 260 4 0 5 0 22 0 14 999 4 3050 0 000 9999 3 SF6 LIL DAT YR MO DY HRS HRE DIST Q SIGY cy cy Q ARCMAX ARCMAX Q HS 87 01 10 930 945 0 160 0 102 65 1082 10608 7 6 74510 36 87 01 10 930 945 0 490 0 102 129 1029 10088 4 8 47059 36 87 01 10 930 945 0 810 0 102 144 1049 10284 3 4 36275 36 87 01 10 2945 1000 0 140 0 102 54 1161 11382 8 3 81373 36 87 01 10 2945 1000 0 440 0 102 132 1337 13108 5 2 50980 36 87 01 10 945 1000 0 820 0 102 237 1486 14569 3 4 33333 36 87 01 12 930 945 0 150 0 102 49 1060 10392 11 1 108824 36 87 01 12 930 945 0 300 0 102 68 437 4284 F 26471 36 87 01 12 930 945 0 460 0 102 115 633 6206 2 3 22549 36 87 01 12 2945 1000 0 160 0 102 52 988 9686 8 6 84314 36 87 01 12 2945 1000 0 300 0 102 58 741 7265 5 8 56863 36 87
98. wise maxima are compared with observations because the observations refer to 15 minute averages 3 Note that the format for time is different from that for Kincaid but is similar to that of Copenhagen E g 1015 means 10 15 whereas Kincaid and Indianapolis data are given for integer values of hour Perform model calculations for the 14 distances represented in the Lillestrem data set 0 14 0 15 0 16 0 19 0 30 0 41 0 43 0 44 0 46 0 47 0 49 0 81 0 82 0 90 km Dump your output in a file Here we will call the file Modelled lil you may use another name The format of Modelled lil should be the following e There is a one line heading e There should be seven columns or more separated by blanks the values are read using free format input e There must be a line for all arc periods 8 periods times 14 distances altogether 112 lines e The distances must be those indicated above An example of the first fifteen lines of a Modelled lil file is shown below The date and time is indicated then the distance then the normalised arcwise maximum concentration ARCMAX Q and finally the normalised crosswind integrated concentration CY Q YR MO DY HRE DIST ARCMAX Q cy Q 87 1 10 945 0 140 6930 313 87 10 945 0 150 9439 457 87 10 945 0 160 12020 620 87 10 945 0 190 19044 1167 87 10 945 0 300 27819 2691 87 10 945 0 410 25066 3314 87 10 945 0 430 24351 3376 87 10 945 0 440 23968 3400 87 10 945 0 460
99. ype of comparison however does not have a straightforward interpretation Olesen H R 1998 Local scale regulatory dispersion models Initiatives to improve modelling culture Proceedings of the 10th Joint Conference on the Applications of Air Pollution Meteorology with the A amp WMA American Meteorological Society Boston pp 49 53 Contains a brief overview of model evaluation activities Olesen H R 2000 The Model Validation Kit Status and Outlook Int J Environment and Pollution Vol 14 Nos 1 6 pp 65 76 Reviews status for the Model Validation Kit A change of the methodology of the kit is considered based on the concept of near centreline concentrations The paper examines some consequences of such a potential change in methodology Olesen H R 2001 Model Validation Kit Recent Developments Int J Environment and Pollution Vol 16 Nos 1 6 pp 129 136 Paper presented at the 6th International Conference on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes in Rouen October 1999 Supplements the paper on status and outlook mentioned above Certain problems with the determination and use of so called near centreline concentrations NCC s are identified and discussed Olesen H R 2001 A platform for model evaluation 7th international conference on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes Belgirate Italy May 28 31 2001 Av

User's Guide to the Model Validation Kit

Contents

Download Pdf Manuals

Related Search

Related Contents