release_note_2010_updated.txt Release notes for SAFNWC
Contents
1. Only the area germ uses the moving window option It contains intermediate and final output of running the PPS on the testdata included in the acpg subpackage This reference dataset was generated on the production environment at SMHI Redhat Enterprise Linux 5 1 with gcc fortran 4 1 2 Page 8 release_note_2010_updated txt The following local environment was used ACPG Version HDF5 Version 8 4 PROJ Version 7 ZLIB Version AHAMAP Version Pmw version Python version Tkinter version Tk Tcl NumPy version PIL jpeg HL HDF RTTOV GRIB API AAPP SM_COASTALZONE_LIMIT 64 SM_LANDSEA_FRACTION_MAX 255 N H DORPWWN WoO Ww oO oOo PARA HRWO DHOODHPHONNEEPE ARN o AS a an extra bonus to PPS we have included a simple PPS product viewer written in Python using Tk Tcl It may serve as a tool for application managers and researchers It is not intended for use at the forecasters desk on the other hand it may serve as an inspiration for those in charge of developing general purpose including remote sensing satellite image display systems It has a rather neet feature where it is possible to focus on the same object Simultaneously in a PPS product and a AVHRR RGB image The PPS viewer comes as a tar ball and has a small README file ImageViewer r1_26 tgz As this application needs the SMHI training manager tool we have included this as well as well as a small application for extracting NWP profile2. now and in the future In the HDF5 1 8 x release notes it says Improvemens to speed compression and stability Go to a place where you want to keep the source code gt tar xvfz ahamap r1_61 tgz Ex configure command in one line Page 3 release_note_2010_updated txt gt configure prefix local_disk opt AHAMAP 1_61 with proj opt PROJ4 current include opt PROJ4 current lib with python yes with aapp localdisk opt AAPP 6_12 a LE UMP YLT a NUMpy Cope ane huge i gt make gt make check gt make install It is recommended that you use AAPP of version 6 6 or later Earlier it has been recommended to configure without AAPP but now it is recommended to use AAPP It is a necessity if you want PPS to run on global Metop The acpg package is packed in two tar balls i One contains source code and static data and the other one contains test data The unpacking should be done like this standing where you want to have the source distribution located gt tar xvfz acpg r2_32 without_testdata tgz gt tar xvfz acpg r2_32_testdata tgz Build and install the ACPG package Ex configure command in one line gt configure prefix local_disk opt ACPG 2_32 with proj usr local include usr local lib with ahamap local_disk opt AHAMAP 1_61 with hlhdf local_disk opt HLHDF 0_79 with grib_api local_disk opt GRIB_API 1_8_0 with aapp local_disk opt AAPP 6_12 with rttov local_disk opt RTTOV9 9_3 w
3. ifort on linux we will try to see if it is feasable to accommodate the adaptations needed 2 Performance over Antarctica The performance of PPS over Antarctica is problematic we have so far done no validation over that area and there are known problem issues there The landuse land sea mask is generated from the USGS GTOPO30 database as the USGS Global Land Characterization database does NOT include Antarctica There are likely to be inaccuracies in the land sea mask as a consequence of this we do not yet use the OSISAF Ice maps for the southern hemisphere The IR calibration from AAPP sometimes with the current AHAMAP generate nodata values due to very cold temperatures This result in nodata values in the PPS products black dots in images noisy pattern 3 GAC product quality The product quality for GAC is still lower than for the hrpt products read more about this in section 17 PPS version 2010 includes an improved interface to process basic images and cloud products based on the NOAA AVHRR GAC data record i e reduced 4 km resolution global AVHRR data The main improvement in this version consists of implemented calibration corrections for the full series of NOAA satellites i e NOAA 7 to NOAA 19 and Metop in order to produce historically consistent AVHRR GAC radiances Corrections were derived from Simultaneous Nadir Observations comparing AVHRR radiances with MODIS radiances and comparisons with various stable re
4. after a patch while earlier releases eg v2009 does not process GRIB 2 files at all If you are using ECMWF data please be aware of that they will change their products to GRIB 2 format later on PPS 2010 have several minor and major improvements and changes over PPS 2009 The most significant changes improvements are as summarised shortly below Cloud Mask and Cloud Type f a Thresholds are now corrected according to surface emissivity ron Ne calculation of threshold tables done with RTTOV9 and ECMWF 91 profiles Page 1 release_note_2010_updated txt ie and temperature range of threshold tables has been considerably expande Coastal zone information is used even in processing on satellite swath projection Introduced new CM tests spatial coherence over sea and MODIS specific Can use sea ice from NwP files if OSISAF sea ice files are missing CTTH CTTH uses RTTOV 9 now previously RTTOV 7 was used General Processing of MODIS data enabled for PGE01 02 03 but pre processing is still done outside PPS Improved processing of GAC data especially intercalibration of NOAA satellites implemented Though the product quality for GAC is still lower than for the hrpt products read more about this in section 17 The generation of absolute azimuth angles is now possible optional This was a specific requirement from the CM SAF The source_me and the profile_pps files are generated on the fly during configure Some b
5. data from a Hirlam or ECMWF Grib file TrainingManager r2_06 tgz ExtractNwpProfi le r0_15 tgz Please install the above applications in reverse order of listing I e first ExtractNwpProfile then TrainingManager and finally Imageviewer The Training Manager tool comes with a User manuel The SMHI NWCSAF PPS development team very much appreciate if users would contribute to the collection of AVHRR training data using this tool or similar inhouse version in the future The Software Versions Document SVD and the SUM outlines the third party software needed in order to install and run PPS The changes since version 2009 are Numeric has been replaced by NumPy ScientificPython is not used any more Page 9 release_note_2010_updated txt hdf5 has been upgraded from v1 6 x to v1 8 x hl hdf has been upgrade from v0 54 to v0 79 Please note that this change is not compatible you can not run PPS v2010 on old hl hdf s and you can not run PPS v2009 on new hl hdf EMOS has been replaced by GRIB API Though you can still configure for using EMOS If you have GRIB files both would go If you have GRIB2 files you need to use GRIB API but our code is not yet tested with complete GRIB2 files Both disclaimers below were also issued in similar wording with version 2009 1 ifort on Intel No testing of version 2010 with Intel Fortran compilers done we expect that PPS v2010 will not work as is with these compiler suites If anyone needs
6. done NB The module pps Distribute is per default set up to be run just prior to the pps Control module see the module_config py tpl The ppsDistribute py Cunder acpg scr is however adapted to the needs at SMHI and should be turned off or adapted to your local needs for distribution The PPS Task Manager is being configured at least partly during the interactive installation process when invoking the script TIRA During the interactive installation you have to anwser some questions and the parameters you define are being entered into the installed python files It makes use of template files lt python filename gt tpl wh ch are being converted to py files upon installation After installation you can change the configuration settings including the parameters you just specified in a number of configuration files The most frequently visited files and parameters are given here below module_config py You should check and modify according to your local environment and requirements most of the parameters available Parameters which should normally not be changed NOT_ALLOWED_USER_REQUESTS EXCLUSIVE_MODULES TIME_ONLINE_PRODUCTS_FINISHED we run currently with these settings parameters which are changed only seldomly PRODUCTION_PRIORITY 1 NUMBER_OF_SIMULTANEOUS_ORDERS 1 comm_config py It contains only three parameters and these have been set hopefully correctly during your installation However it
7. e running PPS v2009 on local data over northern and central Europe there is no very strong or urgent need for you to upgrade to v2010 provided you solve the bug described in SPR 376 SMR 354 The validation studies done so far with v2010 mainly over European land surface using Synop reports show very little change in performance compared to v2009 and actually a slight degredation though very small of the cloud mask is observed This slight degredation we attribute to a new need for tuning of the thresholds after the threshold calculation has changed with new RTM model and database and use of actual land surface emissivities The major asset of v2010 lies in the global performance where significant improvements are observed especially over barren and desert land areas The cloud mask over sea is also improved picking up more cloud contaminated pixels however this improvement we have not yet quantified Also the new and more consistent and accurate derivation of thresholds both land and sea we think will make a positive impact once we have had time to do a careful tuning This tuning is expected to be ready later this year and if an improvement 1s seen we Witt release these tuned threshold offsets to the PPS users The many technical improvements e g changing from RTTOV 7 to version 9 getting rid of old Numeric and adapting to the new HDF5 1 8 x series are Important changes that we think will improve the overall usefulness of PPS
8. ference targets on the Earth surface Heidinger and Straka 2010 PpS derived cloud products for GAC data are also provided but users should be aware of the need for additional tuning and development activities in order to achieve the same product quality as for HRPT full resolution products For example spatial coherence tests over oceanic surfaces need to be retuned to the coarser GAC resolution since current products give too many clouds over ocean surfaces A PPS software patch including retuned and improved GAC cloud products is expected before the end of 2010 Reference Hedinger A K and W Straka 2010 Deriving and Inter sensor consistent Calibration for the AVHRR Solar Reflectance Data Record Submitted to Int J Remote Sens Page 10 release_note_2010_updated txt Adam Dybbroe and Sara Hornquist SMHI Page 11
9. he directory holding the remapped physiography files which is normally import AUX_data remapped The list of areas regions you want to consider for your local PPS production should be defined in pps_basic_configure py There is a parameter called AREALIST that sets all the area ids of interest Something which is often causing a little confusion is that when you want PPS operated from the Task Manager you will also have to set up the areas in the PPS TM config file module_config py Here the parameter MODULE_PRODUCT_AREAS specify the areas which will be considered for PPS when run from the TM But these area ids should also be available in the pps_basic_configure py One thing which may not seem obvious is that if an area id in the MODULE_PRODUCT_AREAS parameter of the module_config py file is not available in pps_basic_configure py you will still however see that e g AvhrrOnAreas py detects this area as something which should be considered and it will attempt to produce remapped AVHRR data also for that area See the sum for further details 8 Bug corrections and improvements to PPS Task Manager since PPS version 2009 See change history in a separate document available on Helpdesk SW packages amp patches the PPS document page See the also summary above section 1 or consult the list of SPRS SMRsS at the NWCSAF Help Desk release_note_2010_updated txt See change history in a separate document available on Helpdesk SW pac
10. images that will have coast lines and political borders printed on the image grib_config cfg Configure for you local or ECMWF NWP_ model Be sure to point out the correct translation table if such one is needed region_config cfg Do you have all the region definitions needed available in here pps_config_common cfg Normally you would only need to adapt the parameter LOG_COMMUNICATION LOG_COMMUNICATION ax015 WITH_TASK_MANAGER no LOG_DAEMON_PORT_NUM 2961 LOG_DAEMON_SERVER ax015 smhi se Be sure to enter a yes above if you wish to run the ACPG through the task manager And specify the port number and server adress corresponding to where the log daemon is running It could also be that you do not want to use OSISAF ice maps Page 6 release_note_2010_updated txt then you set WITHOUT_SEAICE to yes You find this parameter under PGEO1 profile_pps SM_OVERWRITE 0 SM_LOGGING 0 SM_WITH_CTTH_ST_SHIFT 0 Make sure DIR_NAVIGATION points to the directory where you have the sub directories with AAPP satpos files ephemeris files and orbit parameters E g ana ephe orb_elem satpos tle_db tmp tracking The area region definitions shall be available only at one place namely in the Region contig ctd file Please avoid having permanent definitions in the pcs area plugin files If a new area region is created remapped physiography data needs to be generated first and the file should be put in t
11. is crucial for the TM communication with ACPG that you use correct server names and ports of course Check pps_comm_config cfg from the ACPG installation see elow schedule_config p SATELLITE_PRIORITIES tm_config py Page 5 release_note_2010_updated txt SUPPORTED_SATELLITES ACPG pps_basic_configure py Perhaps you need to adjust the parameter NWP_TIME_RESOLUTION Be sure to have the right prefix NWP_FILE_PREFIX matching your NwP model Maybe you would like to adjust the parameter MAX_NUM_OF_MISSING_DATA aid to allow for less or more no data pixels inside each product area Adjust the switch for moving window option for CTTH semi transparent correction algorithm Example for pltfrm in NOAA_METOP_LIST CTTH_WITH_SHIFT_PLATFORM_AREAS p It frm sswe 1 when set to 1 the moving window option is ON and the result is more valid data on the cost of a longer processing time For running PPS in satellite projection the parameter is CTTH_WITH_SHIFT_PLATFORM Perhaps you want to adjust the dictionary IMAGES controlling which areas and products that will have images generated and in which format Perhaps you want to adjust the parameter THUMBNAIL_IMG_SIZE to control the size of the thumbnail images It is currently set to xsize 3 times ysize 3 where xsize ysize are the dimensions of the original product images Perhaps you want to adjust the parameter OVERLAY controling which areas and product
12. ith numpy usr 1i1b64 python2 5 site packages numpy core inc lude numpy Baie TA Aa sysconfdir local_disk data www make make check make install make install data make instal 1 www VVVVVY Please be aware that in earlier versions of PPS you would configure the use of AAPP like this with aapp local_disk opt AAPP 6_12 AAPP Now the trailing AAPP should be omitted Installation of course provides you have the necessary 3rd party software installed See the software users manual You need hdf5 version 1 8 1 or later You need hl hdf version 0 79 or later while running the configure command the files source_me and profile_pps are build from the content of the configure command If it is the first time you install the TM please refer to the SUM Unpack the T tar ball gt tar xvfz PPS_TaskManager ri1_22 tgz Page 4 release_note_2010_updated txt Install the PPS Task Manager First make sure the install ksh script is executable If not gt chmod u x install ksh If you are really lazy you can copy a previous installation history file e g tm_install_settings xservl6 to the top level directory and edit it as you wish Cor leave it unchanged and then do a re installation install ksh reinstall You Con get at least one question and possibly three Anwser as follows 1 Answer yes y on the first question 2 Answer create c on the second question 3 Answer create c on the third question and you are
13. kages amp patches the PPS document page See the also summary above section 1 or consult the list of SPRS SMRs at the NWCSAF Help Desk See change history in a separate document available on Helpdesk SW packages amp patches the PPS document page See the also summary above section 1 or consult the list of SPRS SMRs at the NWCSAF Help Desk There is a new scientific Validation Report since PPS 2009 For the other documents available there are some updates Please download the documentation from the NWCSAF Help Desk Remaining issues PPS v2010 has not been run with full GRIB 2 files Though parts of GRIB 2 files have been tested and is working If it turns out necessary there will be a patch of PPS v2010 for GRIB 2 ea bee PPS v2010 has not been run on python 2 6 or python 3 0 we plan to have PPS v2011 running on python 2 6 There are a no open Software Problem Reports SPRs Six sets of reference data with final results and intermediate temporary output of PPS running version 2010 on the testdata is available noaal6_20020429_1129_08252_germ_pps v2010_1xutv10 tgz noaal7_20020912_1954_01139_germ_pps v2010_1xutv10 tgz metop02_20070227_0951_01857_sval_pps v2010_1xutv10 tgz metop02_20070227_0951_01857_germ_pps v2010_1xutv10 tgz noaal8_20061121_0159_07751_sval_pps v2010_1Ixutv10 tgz noaa18_20061121_0159_07751_germ_pps v2010_1xutv10 tgz The two datasets with the area sval Svalbard uses OSISAF ice concentration data
14. ments change_history_xxx txt xxx ACPG AHAMAP PPS_TaskManager see also section 10 12 below or consult the list of SPRS SMRS at the NWCSAF Help Desk You can also read the slides pr ea enang v2010 All documents mentioned are available on the NWCSAF Help Desk The changes to PPS as outlined above have the following implications on your environment Page 2 release_note_2010_updated txt das is not more needed Should preferably be removed from your system if possible Scientific Python is not needed any more The HDF5 library should be upgraded to 1 8 x and the latest 1 8 4 at the time of writing is recommended The HLHDF library installed should be at least v0 79 provided from SMHI Numpy version 1 2 0 or later should be installed EMOS lib is not needed anymore GRIB API version 1 8 0 or later should be installed As already mentioned above you can choose to skip the GRIB API for the time being and use the EMOS library instead as before But please be aware that very soon probably later this year you will get a PPS patch to be able to ingest GRIB edition 2 files and then the GRIB API is a requirement So better change now This release is adapted to process GRIB 2 files either as it is or maybe after a patch while earlier releases eg v2009 does not process GRIB 2 files at all If you are using ECMwF data please be aware of that they will change their products to GRIB 2 format later on If you ar
15. release_note_2010_updated txt Release notes for SAFNWC PPS version 2010 23 April 2010 updated 20 August 2010 Contents 0 About this release 1 Summary of main changes since version 2009 2 Changed requirements on 3rd Party Software 3 why you should care 4 Unpacking building and installing the AHAMAP package quick guide 5 Unpacking building and installing the ACPG package quick guide 6 Unpacking building and installing the PPS Task Manager quick guide 7 Configuration of the PPS system 8 Bug corrections and improvements to PPS TaskManager since PPS version 2009 9 Bug corrections and improvements to AHAMAP since PPS version 2009 10 Bug corrections and improvements to ACPG since PPS version 2009 11 Documentation 12 Known open issues 13 Reference data 14 PPS viewer and Training Manager tool 15 Third Party Software 16 Disclaimer 17 GAC product quality This is release 2010 of the SAFNWC PPS software package It contains four sub packages packed in the following four tar balls versioning of the individual packages are CVvS release tags AHAMAP r1 61 ACPG r2 32 without_testdata ACPG r2 32_testdata PPS_TaskManager 1 22 In addition the PPS 2010 release contains a reference dataset and two extra tools to visualise PPS products and for interactive collection of training and validation data see bullet 12 below This release is adapted to process GRIB 2 files either as it is or maybe
16. ug corrections Most importantly The use of wrong threshold table for the feature T37T12 only a problem in region processing mode is now corrected See SPR 376 SMR 354 This bug if when not corrected results in a slight overestimation of cloudiness especially wintertime Changed from Numeric to NumPy and removed dependency on Scientific Python Adaptation to HDF5 library version 1 8 and corresponding HLHDF upgrade Changed from EMOS to GRIB API Processing for GRIB2 files have not been fully tested yet It is still possible to build PPS against the EMOS library as before but in the future this option will removed in favour of the GRIB API software GRIB API is needed in order to be able to read GRIB2 files Many minor improvements and technical changes Some of the most important ones are Removal of remaining smaller memory leaks and making threading in PGEO3 configurable PPS Task Manager with v2010 it is possible to synchronise PPS with the local overpass schedule provided by your HRPT reception station provided you have eaither a MEOS station from Kongsberg Spacetec or a VCS based system These adaptations were previously delivered off line to the users we knew ran PPS via the Task Manager and were equipped with such systems VCS or MEOS If you have another type of satellite reception station please send us a mail and we can see if we are able to work out a solution for you For more details see the separate docu
Download Pdf Manuals
Related Search