RClimdex User Manual
Contents
1. days where RRi 20mm 19 22 Rnn Let RRibe the daily precipitation amount on day iin period j If nn represents any reasonable daily precipitation value then count the number of days where RRi nnmm 23 CDD Let RRi be the daily precipitation amount on day iin period j Count the largest number of consecutive days where RRi lt Imm 24 CWD Let RRi be the daily precipitation amount on day iin period j Count the largest number of consecutive days where RRi gt Imm 25 R95pTOT Let RRw be the daily precipitation amount on a wet day w RR 1 0mm in period j and let RRin95 be the 95 percentile of precipitation on wet days in the 1961 1990 period If W represents the number of wet days in the period then W R95 p RRw where RRwj gt RRwn95 w 26 R99p Let RRw be the daily precipitation amount on a wet day w RR 1 0mm in period j and let RRw99 be the 99 percentile of precipitation on wet days in the 1961 1990 period If W represents number of wet days in the period then W R99 p X RRwj where RRw gt RRwn99 w 1 27 PRCPTOT Let RRi be the daily precipitation amount on day iin period j If Z represents the number of days in j then 20 I PRCPTOT RRi i l Appendix D Threshold estimation and base period temperature indices calculation Empirical quantile estimation The quantile of a distribution is defined as QO p F p inf x F x p 1 lt p lt 1 where F x2. 1 Introduction ClimDex is a Microsoft Excel based program that provides an easy to use software package for the calculation of indices of climate extremes for monitoring and detecting climate change It was developed by Byron Gleason at the National Climate Data Centre NCDC of NOAA and has been used in CCI CLIVAR workshops on climate indices fromim 2001 The original objective was to port ClimDex into an environment that does not depend on a particular operating system It was very natural to use R as our platform since R is a free and yet very robust and powerful software for statistical analysis and graphics It runs under both Windows and Unix environments In 2003 it was discovered that the method used for computing percentile based temperature indices in ClimDex and other programs resulted in inhomogeneity in the indices series A fix to the problem requires a bootstrap procedure that makes it almost impossible to implement in an Excel environment This has made it more urgent to develop this R based package RClimDex 1 0 is designed to provide a user friendly interface to compute indices of climate extremes It computes all 27 core indices recommended by the CCI CLIVAR Expert Team for Climate Change Detection Monitoring and Indices ETCCDMI as well as some other temperature and precipitation indices with user defined thresholds The 27 core indices include almost all the indices calculated by ClimDex Version 1 3 This version of RClim
3. R9Sp R99p PRCPTOT Very wet days Extremely wet days Annual total wet day precipitation Annual total PRCP when RR gt 95 percentile Annual total PRCP when RR gt 99 percentile Annual total PRCP in wet days RR gt 1mm 14 Mm mm mm APPENDIX B Input Data Format All of the data files that are read or written are in list formatted format The exception is the very first data file that is processed in the Quality Control step This input data file has several requirements 1 ASCII text file 2 Columns as following sequences Year Month Day PRCP TMAX TMIN NOTE PRCP units millimeters and Temperature units degrees Celsius 3 The format as described above must be space delimited e g each element separated by one or more spaces 4 For data records missing data must be coded as 99 9 data records must be in calendar date order Missing dates allowed Example data Format for the initial data file e g used in the Quality Control step 1901 1 1 9959 S351 6 8 1901 1 2 99 Geli 3 A386 1901 1 3 TI SO Hr 1901 1 4 997 96S TIS 1901 1 7 99 9 41 3 8 74 15 APPENDIX C Indices definitions Definitions for indicators listed in Appendix A For practical reasons in this version of the software not all indices are calculated on a monthly basis Monthly indices are calculated if no more than 3 days are missing in a month while annual values are calculated if no more than 15
4. days are missing in a year No annual value will be calculated if any one month s data are missing For threshold indices a threshold is calculated if at least 70 of data are present For spell duration indicators marked with a a spell can continue into the next year and is counted against the year in which the spell ends e g a cold spell CSDI in the Northern Hemisphere beginning on 31 December 2000 and ending on 6 J anuary 2001 is counted towards the total number of cold spells in 2001 1 FDO Let Tn be the daily minimum temperature on day iin period j Count the number of days where Tni lt 0 C 2 SU25 Let Tx be the daily maximum temperature on day i period j Count the number of days where Txi gt 25 C 3 IDO Let Tx be the daily maximum temperature on day iin period j Count the number of days where Txi lt 0 C 4 TR20 Let Tn be the daily minimum temperature on day iin period j Count the number of days where Tni gt 20 C 5 GSL 16 Let T be the mean temperature on day iin period j Count the number of days between the first occurrence of at least 6 consecutive days with Ty gt 5 C and the first occurrence after 1 July 1 January in SH of at least 6 consecutive days with Tj lt 5C 6 TXx Let Txw be the daily maximum temperatures in month k period j The maximum daily maximum temperature each month is then TXxqy max Txi 7 TNx Let Tny be the daily min
5. is the distribution function Let X a X denote the order statistics of X gt X ie sorted values of X and let p denote the ith sample quantile definition The sample quantiles can be generally written as Q p d 7 X 5 YX iy Hyndman and Fan 1996 suggest a formula to obtain medium un biased estimate of the quantile by letting j int p n 1 p 3 and letting y p n 1 p 3 j where int u is the largest integer not greater than u The empirical quantile is set to the smallest or largest value in the sample when j lt or j gt n respectively That is quantile estimates corresponding to p lt n J are set to the smallest value in the sample and those corresponding to p gt n n are set to the largest value in the sample Bootstrap procedure for the estimation of exceedance rate for the base period It is not possible to make an exact estimate of the thresholds due to sampling uncertainty To provide temporally consistent estimate of exceedance rate throughout the base period and out of base period we adapt the following procedure Zhang et al 2004 to estimate exceedance rate for the base period a The 30 year base period is divided into one out of base year the year for which exceedance is to be estimated and a base period consisting the remaining of 29 years from which the thresholds would be estimated b A 30 year block of data is constructed by using the 29 year base period dat
6. maximum temperature less than daily minimum temperature In addition QC also identifies outliers in daily maximum and minimum temperature The outliers are daily values outside a region defined by the user Currently this region is defined as the mean plus or minus n times standard deviation of the value for the day that is mean n std mean n std Here std represents the standard deviation for the day and n is an input from the user and mean is computed from the climatology of the day Select Load Data and Run QC from the RClimDex Menu to open a window as shown below This allows users to select load the data file from which indices are to be computed Desktop A My Documents My Computer E File name My Network P Files of type Text Files txt 7 The filename should be of the form stationname txt The values in the file should be of the format described in Appendix B In this menu we use data from a station whose data are stored in an ASCII file 21946 txt for the purpose of demonstration A pop up window as shown below will appear once the data for station 21946 are successfully loaded i Data 21946 loaded click OK to continue Error messages will appear in the R console if this step has not been completed successfully This is usually caused by the wrong input data format Please compare your format with our sample data if you see such messages Unreasonable values are identif
7. Dex has been developed under R 1 84 It should run with R 1 84 or a later version A main objective of constructing climate extremes indices is to use for climate change monitoring and detection studies This requires that the indices be homogenized Data homogenization has been planned but is not implemented in this release Current RClimDex only includes a simple data quality control procedure that was provided in ClimDex As in ClimDex we require that data are quality controlled before the indices can be computed This users manual provides step by step instructions on 1 The installation of R and setting up the user environment 2 Quality control of daily climate data 3 Calculation of the 27 core indices 2 Installation and running of R R is a language and environment for statistical computing and graphics It is a GNU implementation of the S language developed by John Chambers and colleagues at Bell Laboratories formerly AT amp T now Lucent Technologies S plus provides a commercial implementation of the S language 2 1 How to install R RClimDex requires the base package of R and graphic user interface TclTk The installation of R involves a very simple procedure 1 Connect to the R project website at http www r project org 2 Follow the links to download the most recent version of R for your computer operating system from any mirror site of CRAN For Microsoft Windows 95 98 2000 and XP download the Windows setup pr
8. RClimDex 1 0 User Manual By Xuebin Zhang and Feng Yang Climate Research Branch Environment Canada Downsview Ontario Canada September 10 2004 Acknowledgement The RClimDex is developed and maintained by Xuebin Zhang and Feng Yang at the Climate Research Branch of Meteorological Service of Canada Its initial development was funded by the Canadian International Development Agency through the Canada China Climate Change Cooperation C5 Project Lisa Alexander Francis Zwiers Byron Gleason David Stephenson Albert Klan Tank Mark New Lucie Vincent and Tom Peterson made important contributions to the development and testing of the package Jose Luis Santos at CIIFEN helped to translate this document into Spanish Earlier versions of RClimDex have been used during CCI CLIVAR ETCCDMI workshops in Cape Town South Africa May 31 June 4 2004 and in Maceio Brazil August 9 14 2004 The lectures and attendees of the workshops provided very valuable suggestions for the improvement of RClimDex TABLE OF CONTENTS 1 Introduction 2 Installation and running of R 2 1 How to install R 2 2 How to run R 3 How to use RClimDex 3 1 Loading of RClimDex 3 2 Data quality control 3 3 Calculation of Indices 4 Known bugs 5 Bug report Appendix A List of Climate Indices Appendix B Input Data Format Appendix C Indices definitions Appendix D Threshold and in base period temperature indices calculation Appendix E R for Windows FAQ
9. a set and adding an additional year of data from the base period i e one of the years in the base period is repeated This constructed 30 year block is used to estimate thresholds c The out of base year is then compared with these thresholds and the exceedance rate for the out of base year is obtained d Steps b and c are repeated for an additional 28 times by repeating each of the remaining 28 in base years in turn to construct the 30 year block e The final index for the out of base year is obtained by averaging the 29 estimates obtained from steps b c and d Reference Hyndman R J and Y Fan 1996 Sample quantiles in statistical packages The American Statistician 50 361 367 22 Zhang X G Hegerl F W Zwiers and J Kenyon 2004 Avoiding inhomogeneity in percentile based indices of temperature extremes J Climate submitted Appendix E R for Windows FAQ A Adobe Acrobat Docun 23
10. csv A pop up window appears once the data QC is complete At the same time four Excel files 21946tempQC csv 21946prepQC csv 21946tepstdQC csv and 21946indcal csv are created in a subdirectory called log The first two files contain information about unreasonable values for temperature and precipitation The third file flags all possible outliers in daily temperature with the dates on which those outliers occur The last file contains the QC d data and will be used for the indices calculation Note that in this file only missing values and unreasonable values are replaced with NA flagged possible outliers are NOT changed For an easy visualization 4 PDF files containing time series plots missing values are plotted as red dots of daily precipitation amount daily maximum minimum temperatures and daily temperature range are also stored in log At this point the user may check the data in the file 21946tepstdQC csv to determine if any value marked as an outlier is really an outlier The file 21946indcal csv can be modified using Excel under Windows and any editor under Unix if any action needs to be taken After the completion of this step the user may Click OK on the following window to proceed with indices calculation i If you have checked data x myxP etccd RClimDex 21946indcal csyv click OK to continue Note that the indices are computed from the QC d data The original input file i
11. e User defined Lower Limit of Day Low allows the calculation of the number of days when daily minimum temperature is below this limit These indices are called SUmm FDmm TRmm IDmm where mm corresponds to user defined value This step includes some data processing so it will take a few seconds to finish E User defined parameters for Indices Calculation First year of base period haro Last year of base period haoo Latitude of this station location eooo Longitude of this station location f User defined upper threshold of daily maximum temperature BE o User defined lower threshold of daily maximum temperature fO User defined upper threshold of daily minimum temperature E User defined lower threshold of daily minimum temperature zz aay User defined daily precipitation threshold aT OK CANCEL Jj Once this step is completed a window will appear to allow the user to select their desired indices for calculation All indices are selected by default i ios 10 Uncheck indices that are not needed then click OK to perform the computation Depending on the indices selected this procedure may take a while o 13 i Check desired indices 5U25 FDO TR20 IDO User Defined SU FD TR ID GSL growing season length TxXx TXn TNx TNn TX10p TX90p TN10p TN90p WSDI CSDI DTR Rxiday RxSday SDII R10mm R20mm Rnnmm CDD CWD R95p R99p PRCPTOT oK CANCEL It may take more than 5 minutes to comp
12. e daily maximum temperature on day iin period j and let 7xin90 be the calendar day 90 percentile centred on a 5 day window calculated using method from Appendix D Then the number of days per period is summed where in intervals of at least 6 consecutive days Txi gt Txin90 15 CSDI Let Tn be the daily minimum temperature at day iin period j and let TxinlO be the calendar day 10 percentile centred on a 5 day window calculated using the method from Appendix D Then the number of days per period is summed where in intervals of at least 6 consecutive days Tni lt TninlO 16 DTR 18 Let Tx and Tni be the daily maximum and minimum temperature respectively on day iin period j If J represents the number of days in j then I Txi Tni DTR amp I 17 RX1day Let RRi be the daily precipitation amount on day iin period j Then maximum 1 day values for period j are Rxlday max RR 18 Rx5day Let RRy be the precipitation amount for the 5 day interval ending k period j Then maximum 5 day values for period j are Rx5day max RRy 19 SDII Let RRw be the daily precipitation amount on wet days w RR Imm in period j If W represents number of wet days in j then 20 R10 Let RRi be the daily precipitation amount on day jin period j Count the number of days where RRi 2 10mm 21 R20 Let RR be the daily precipitation amount on day iin period j Count the number of
13. ex can also be loaded from drop down menu Choose the File from the RGui menu and then select Source R code This will bring a new pop up window within which you can select our R source code rclimdex r from the directory where the program was saved or type http cccma seos uvic ca ETCCDMI RClimDex rclimdex r to download the latest version from the web site 03 The R Foundation for Statistical Computing 2003 11 21 ISBN 3 900051 00 3 e and comes vith ABSOLUTELY NO WARRANTY to redistribute it under certain conditions or licence for distribution details tive project with many contributors lt drs for more information and eitation on how to cite R in publications Type demo for some demos help for on line help or help start for a HTML browser interface to help Type q to quit R gt t IR 1 8 1 A Language and Environment Once the source code is successfully loaded the RClimDex main menu will appear RClimDex 1 0 Load Data and Run QC Indices Calculation Exit 3 2 Load Data and Run QC Data Quality Control is a prerequisite for indices calculations The RClimDex QC performs the following procedure 1 Replace all missing values currently coded as 99 9 into an internal format that R recognizes i e NA not available and 2 Replace all unreasonable values into NA Those values include a daily precipitation amounts less than zero and b daily
14. ied automatically but identification of outliers in temperature data requires input from the user Set Parameters for Data OC Station name or code Criteria number of Standard Deviation 3 The default value for n is 3 Criteria in the Set Parameters for Data QC window but this number may be overwritten by the user As a value of 3 may flag a very large number of values users may wish to start by setting this value to 4 There is no need to fill in Station name or code as this parameter is for a later use After setting the parameter click OK to continue i Data QC 21946 may take 5 minutes click OK to continue In some slower PC s this process may take a few minutes Pop up windows will appear if unreasonable values are found For instance when minimum daily temperature is greater than maximum daily temperature the following message appears RclimDex RClimDex ix I Errors in temperature please check the log file Data QC Indices Calculation Exit If there are any negative values other than missing values coded as 99 9 in the daily precipitation amount the following message will appear E lol x RClimDex eee fs II Errors in prep please check the log file Data OC Indices Calculation Exit If there are outliers the following window appears i Outliers found please check the log file X myXP etccd RClimDex A0g 21946_tepstdQC
15. ile based temperature indices is an example One way to avoid this problem is to add an extra record for the day with values marked as missing just before the beginning of the base period For example if base period is 1961 1990 and the data also starts in 1961 one may add 1960 12 31 99 9 99 9 99 9 as the first line for the input data file 5 Bug report Please report any bugs errors to yang feng ec gc ca with error messages and data being used for the calculation of the indices This will be helpful in producing a better release in the near future We would also appreciate your suggestions for further improvement 12 APPENDIX A List of ETCCDMI core Climate Indices ID FDO SU25 IDO TR20 GSL TXx TNx TXn TNn TN10p TX10p TN90p TX90p WSDI CSDI DTR RXIday RxS5day SDI R10 R20 Rnn CDD CWD Indicator name Frost days Summer days Ice days Tropical nights Growing season Length Max Tmax Max Tmin Min Tmax Min Tmin Cool nights Cool days Warm nights Warm days Warm spell duration indicator Cold spell duration indicator Diurnal temperature range Max 1 day precipitation amount Max 5 day precipitation amount Simple daily intensity index Number of heavy precipitation days Number of very heavy precipitation days Number of days above nn mm Consecutive dry days Consecutive wet days Definitions Annual count when TN daily minimum lt 0 C Annual count when TX daily maxi
16. imum temperatures in month k period j The maximum daily minimum temperature each month is then TNxy max Tn 8 TXn Let Tx be the daily maximum temperatures in month k period j The minimum daily maximum temperature each month is then TXng min Txy 9 TNn Let Tny be the daily minimum temperatures in month k period j The minimum daily minimum temperature each month is then TNng min Tny 10 Tn10p Let Tn be the daily minimum temperature on day iin period j and let TninlObe the calendar day 10 percentile centred on a 5 day window calculated using method from Appendix D The percentage of time is determined where 17 Tni lt TninlO 11 Tx10p Let Tx be the daily maximum temperature on day iin period j and let Txinl0 be the calendar day 10 percentile centred on a 5 day window calculated using method from Appendix D The percentage of time is determined where Txi lt TxinlO 12 Tn90p Let Tn be the daily minimum temperature on day iin period j and let Tnin90 be the calendar day 90 percentile centred on a 5 day window calculated using method from Appendix D The percentage of time is determined where Tni gt Tnin90 13 Tx90p Let Tx be the daily maximum temperature on day iin period j and let 7xin90 be the calendar day 90 percentile centred on a 5 day window calculated using method from Appendix D The percentage of time is determined where Txi gt Txin90 14 WSDI Let Tx be th
17. mum gt 25 C Annual count when TX daily maximum lt 0 C Annual count when TN daily minimum gt 20 C Annual 1st Jan to 31 Dec in NH 1 July to 30 June in SH count between first span of at least 6 days with TG gt 5 C and first span after July 1 January 1 in SH of 6 days with TG lt 5 C Monthly maximum value of daily maximum temp Monthly maximum value of daily minimum temp Monthly minimum value of daily maximum temp Monthly minimum value of daily minimum temp Percentage of days when TN lt 10th percentile Percentage of days when TX lt 10th percentile Percentage of days when TN gt 90th percentile Percentage of days when TX gt 90th percentile Annual count of days with at least 6 consecutive days when TX gt 90th percentile Annual count of days with at least 6 consecutive days when TN lt 10th percentile Monthly mean difference between TX and TN Monthly maximum 1 day precipitation Monthly maximum consecutive 5 day precipitation Annual total precipitation divided by the number of wet days defined as PRCP gt 1 0mm in the year Annual count of days when PRCP gt 10mm Annual count of days when PRCP gt 20mm Annual count of days when PRCP gt nn_mm nn is user defined threshold Maximum number of consecutive days with RR lt 1mm Maximum number of consecutive days with RR gt 1mm 13 UNITS Days Days Days Days Days C C C C Days Days Days Days Days Days C Days Days Days Days
18. ogram Run that program and R will be automatically installed in your computer with a short cut to R on your desktop The TclTk is included in the default installation of R 1 9 0 or later versions It may need to be installed separately if you are running an earlier version of R For Linux download proper precompiled binaries and follow the instruction to install R For other unix systems you many need to download source code and compile it yourself 2 2 How to run R Under the Windows environment double click the R icon on your desktop or launch it through Windows start menu This usually gets you into the R user interface For some computers you may need to first setup an environment variable called HOME See R for Windows FAQ Appendix E for details if you have any problems Under a unix environment just run R to give you the R console Exit from R by entering q in the R console under both Windows and unix Under Windows you may also click File menu and then Exit 3 How to use RClimDex 3 1 Loading of RClimDex Within the R consol prompt gt enter source rclimdex r This will load RClimDex into R environment You may need to include the full path before the filename rclimdex r Or you may download the most recent version from ETCCDMI web site by entering source http cccma seos uvic ca ETCCDMI RClimDex rclimdex r if your computer is connected to the internet Under windows RClimD
19. s not altered in any manner So if a user chose to modify the original data file to correct some of the problematic values the Load Data and Run QC procedure needs to be performed again on the improved data set before the changes can be reflected in the indices calculation 3 3 Indices calculation RClimDex is capable of computing all 27 core indices listed in Appendix A Users may however compute only those indices they require After selecting Indices Calculation from the main menu a user is asked to set up some parameters for the indices calculation The Set Parameter Values window allows the user to enter the first and last years of the base period for the threshold calculation the station latitude Southern Hemisphere is negative to determine in which hemisphere the station is located a user defined daily precipitation threshold P in mm to compute the number of days when daily precipitation amounts exceed this threshold the Rnn indicator and 4 user defined temperature thresholds The User defined Upper Limit of Day High allows the calculation of the number of days when daily maximum temperature has exceeded this threshold The User defined Lower Limit of Day High allows the calculation of the number of days when daily maximum temperature is below this value The User defined Upper Limit of Day Low allows the calculation of the number of days when daily minimum temperature has exceeded this threshold Th
20. ute all the indices Please be patient you will be informed once computations are done A pop up window will appear once the selected indices are computed Calculation Done OK Indices calculation completed Plots are in X myXP etccd RClimDex plots Resulting indices series are stored in a sub directory called indices in Excel format The indices files have names 21946_XXX cvs where XXX represents the name of the index Data columns are separated by a comma For the purpose of visualization we plot annual series along with trends computed by linear least square solid line and locally weighted linear regression dashed line Statistics of the linear trend fitting are displayed on the plots These plots are stored in a sub directory called plots in JPEG format The filenames for plots follow the same rule except that cvs is changed to jpg Select Indices Calculation from the main menu to compute additional indices for the same station For additional stations select Data QC and repeat the above process Select Exit if all required calculations are completed 4 Known bugs There is a known bug in this and earlier versions of RClimDex The program will stop running if the first year of the available data is the same as the first year of the base 11 period This is caused by come computation that requires data beyond the boundary of the base period The calculation of percent
Download Pdf Manuals
Related Search