Home

AWDP User Manual and Reference Guide

1. NWP SAF Version 2 0 01 Reference Guide Date November 2010 Number Descriptor Parameter Unit 41 021159 ASCAT Sigma 0 Usability Code Table 42 021160 ASCAT Use Of Synthetic Data Numeric 43 021161 ASCAT Synthetic Data Quality Numeric 44 021162 ASCAT Satellite Orbit And Attitude Quality Numeric 45 021163 ASCAT Solar Array Reflection Contamination Numeric 46 021164 ASCAT Telemetry Presence And Quality Numeric 47 021165 ASCAT Extrapolated Reference Function Numeric 48 021166 ASCAT Land Fraction Numeric 49 008085 Beam Identifier Code Table 50 002111 Radar Incidence Angle Degree 51 002134 Antenna Beam Azimuth Degree 52 021062 Backscatter dB 53 021063 Radiometric Resolution Noise Value 54 021158 ASCAT Kp Estimate Quality Code Table 55 021159 ASCAT Sigma 0 Usability Code Table 56 021160 ASCAT Use Of Synthetic Data Numeric 57 021161 ASCAT Synthetic Data Quality Numeric 58 021162 ASCAT Satellite Orbit And Attitude Quality Numeric 59 021163 ASCAT Solar Array Reflection Contamination Numeric 60 021164 ASCAT Telemetry Presence And Quality Numeric 61 021165 ASCAT Extrapolated Reference Function Numeric 62 021166 ASCAT Land Fraction Numeric 63 025060 Software Identification Numeric 64 025062 Database Identification Numeric 65 040001 Surface Soil Moisture Ms 66 040002 Estimated Error In Surface Soil Moisture 67 021062 Backscatter dB 68 021151 Estimated Error In Sigma0 At 40 Deg Incidence Angle dB 69 021152 Slope At 40 Deg Inc
2. 25 3 3 3 Use of full resolution d t 4 i ect tet od eee eta et mathe leeds 25 3 5 4 WVC triplet completion and row merging esee neret eene 25 33 2 Quality s s ERE RENE PRECES ANNE 26 PT EE 26 3 5 7 Ambiguity Removal ied one e Rae eR ERR Rege ete eee 26 3 2 8 MONILOTING is e OR SR RE XR ERE ERR OR sled a tnashdaannusaevande Ee REAPER ERR 27 3 6 DETAILS OF PERFORMANCE eerie Deere deiecit repetito tese tet dedo eee er dade de 27 CHAPTER 4 PROGRAM DESIGN os eo oro rona een onera root eE pua sea EE ane e eR so EUR Ee oe Ease er Eee e ea oae epa 28 4 1 TOP EEVEE DESIGN m examen a a cities Dart atistieeihr 28 4l Matmprogramssh itae eR eed ras eH tete o pe UR ENS 26 42 25 Layered model structure dede d Re REIR RC E each as hed Bests Re resque 29 4 1 3 IRR 30 4 1 4 Quality flagging and error handling eese enne rennen 31 44 5 eDim eee P re cde 31 4 2 MODULE DESIGN FOR GENSCAT LAYER 32 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 4 2 1 RTT DRE 32 4 2 2 Module ambrem eese 32 4 2 3
3. Attribute Type Description nr_of_BUFR_mesasges integer Number of BUFR messages bufr_filename character BUFR file bufr_fileunit integer Fortran unit of BUFR file file_size integer Size of BUFR file file_open logical Open status of BUFR file writemode logical Reading or writing mode of BUFR file is_cray_blocked integer Cray system blocked list_of_BUFR_startpointers integer Pointers to BUFR messages message is valid logical Validity of BUFR messages Table 8 5 Attributes of the bufr file data data type for BUFR files 8 4 Libraries Module BufrMod uses two libraries the BUFR software library of ECMWF and bufrio a small 66 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 library in C for file handling at the lowest level The BUFR software library of ECMWF is used as a basis to encode and decode BUFR data This software library is explained in Dragosavac 1994 Library bufrio contains routines for BUFR file handling at the lowest level Since this is quite hard to achieve in Fortran these routines are coded in C The routines of bufrio are listed in table 8 6 The source file oufrio c is located in subdirectory genscat support bufr Routine Call Description bufr_open open_BUFR_file Open file bufr_split open_BUFR_file Find position of start of messages in file bufr_read_allsections get_BUFR_message bufr_get_section_sizes get_BUFR_messag
4. init inv settings to default init inv output print_message check_input_data print input data of inversion my exit print message convert sigma to zspace calc normalisation calc var sO find minimum cone dist 2 my min my_average my_max get indices lowest local minimum my index max print message do_parabolic_winddir_search get_parabolic_minimum GetSortlndex SortWithIndex calc sign MLE calc_sigma0 fill wind quality code Figure B1 1 Calling tree for inversion routine invert one wvc find minimum cone dist calc cone distance calc sigma0 get_parabolic_minimum Figure B1 2 Calling tree for inversion routine find_minimum_cone_dist 84 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 77 2001 Reference Guide Date November 2010 calc sigma0 read LUT get lun free lun create LUT C VV get_lun calc sigma0 cmod4 Get Br from Look Up Table calc sigma0 cmod5 5 n free lun test for identical LUTs my_exit INTERPOLATE Figure B1 3 Calling tree for inversion routine calc_sigma0 Routine INTERPOLATE is an interface that can have the values interpolate 1d interpolate2d interpolate2dv or interpolate3d There are several equivalent routines to calculate the backscatter like calc sigma0 cmod5 calc sigma0 cmod5 5 calc sigma0 cmod5 n normalize conedist prescat mode check ers ascat inversion data
5. csssssssssssssscssssscccsssseccessnscessens 83 CALLING TREE FOR AR ROUTINES ssscssssssscssssssssssssscssssssccsssnsccesssssecessnsecessans 86 CALLING TREE FOR BUFR ROUTINES ssssscssssssssssssssssssnssccsssnsccsssssecsssnsecessons 90 CALLING TREE FOR GRIB ROUTINES sssssscssssssssssssscsssssescsssnsccesssnscssssnsecessens 92 CALLING TREE FOR PFS ROUTINES esccsssssscssssssscssssscssssssccssssnsccsssssecessnsecessens 94 CALLING TREE FOR ICE MODEL ROUTINES ssssssssssssseesssseesssseesssseeeseseesees 97 ASCAT BUFR DATA DESCRIPTORS wsssscsssssssssssssccsssssccsssnsccesssseccesnsscccssnsccessssnecs 99 ACRONYM S t PRISES 102 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 Preface Software code for processing satellite data may become very complex On the one hand it consists of code related to the technical details of the satellite and instruments on the other hand the code drives complex algorithms to create the physical end products Therefore the EUMETSAT Satellite Application Facility SAF project for Numerical Weather Prediction NWP has included some explicit activities aiming at enhancing the modularity readability and portability of the processing code For several years the KNMI observation research group has been developing processing code to supply Near Real Time NRT level 2 surface wind products based on the ERS and SeaWinds Scatterometer l
6. If the user wants to use a Fortran or C compiler not included in table 2 6 he can make his own version of the use script or set the environment variables for compilation and linking manually AWDP is delivered with a complete make system for compilation and linking under Unix or Linux The make system 1s designed as portable as possible and system dependent features are avoided As a consequence some tasks must be transferred to shell scripts The make system consists of two parts one for AWDP and one for genscat The genscat part should be run first For compilation and linking of the genscat part the user should move to the genscat directory and simply enter make The Makefile refers to each subdirectory of genscat invoking execution of the local Makefile and in cases where a subdirectory contains code as well as a subdirectory containing code Makefile thisdir The Makefiles need supplementary information from the files Objects txt which are present in each directory containing code The settings for the compilers are located in file Makeopt ions in directory genscat This file is generated by the Bourne shell script Set Makeoptions which is called automatically by the genscat make system The local Makefile in subdirectory genscat support bufr calls the script 14 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 make bufr lib for compilation of the BUFR library
7. f lt input file Process a BUFR or PFS input file with name input file AWDP detects if the input file is in BUFR format If not it attempts to read the input as PFS file The BUFR input file should either have the ASCAT or the ERS format The PFS file should contain 25 or 12 5 km level 1b data not full resolution level 1b data Example awdp f ascat 20070426 test 250 11 bufr will process this file The results will be written to a new BUFR file see below in this section for the output file naming convention It is possible to concatenate multiple BUFR input files into one using the Unix cat command but PFS files must be processed one by one nwpfl file Read a list of GRIB file names in the file named file The files in the list are read and the GRIB edition 1 or 2 data are used in the wind processing The most convenient way to construct a file list is to use the Unix command 1s 1 GRIB file pattern file Ifno GRIB data are used only the land masking which is present in the level 1b beam information will be used No ice screening will be performed unless the icemodel option is used Ambiguity removal will be performed only if model winds are already present in the input BUFR file 1 in case of reprocessing of a level2 file or if the armeth 1strank option is used i e selection of the 1 rank wind solution If level 2 data are reprocessed and no NWP data are read the qual sigmaO0 flag which was set i
8. NWP SAF Version 2 0 01 Reference Guide Date November 2010 Attribute Type Description ambig 0 144 ambiguity type Array of wind ambiguities ice icemodel type Ice information stress param nwp stress param type Wind stress information process flag process flag type Processing flag level of input integer Level of input data 1 or 2 Table 4 7 Cell data structure All soil moisture information is read from the input BUFR file into the cell data structure and not used within the program It is written to the output BUFR file at the end of the processing Full resolution data The full res type contains average full resolution data read from a PFS file which are used to replace the 25 km or 12 5 km beam data The attributes are listed in table 4 8 The routine init full res sets the full resolution averaged data to zero The routine print full res may be used to print the full resolution data Attribute Type Description count tot integer Number of full res measurements used lat real Mean value of full res lats lon real Mean value of full res lons count fore integer Number of full res fore beams used incidence fore Mean value of full res values azimuth fore real Mean value of full res values sigma0 fore real Mean value of full res values land frac fore real Mean value of full res values count mid integer Number of full res mid beams used incidence mid Mean value of full res values azimuth mi
9. e Verspeek J 2006 Sea ice classification using Bayesian statistics OSI SAF report e Verspeek J Portabella M Stoffelen A and Verhoef A 2007 ASCAT Calibration and Validation Report SAF OSI CDOP KNMI TEC TN 163 OSI SAF Available on http www knmi nl scatterometer publications e Vogelzang J 2007 Two dimensional variational ambiguity removal 2DVAR Report NWPSAF KN TR 004 UKMO UK Available on http www knmi nl scatterometer publications e Vogelzang J Stoffelen A Verhoef A de Vries J and Bonekamp H 2008 Validation of two dimensional variational ambiguity removal on SeaWinds scatterometer data submitted to J Atm Oceanic Technol e de Vries J and Stoffelen A 2000 2D Variational Ambiguity Removal KNMI Feb 2000 Available on http www knmi nl scatterometer publications e de Vries J Stoffelen A and Beysens J 2005 Ambiguity Removal and Product Monitoring for SeaWinds KNMI Available on http www knmi nl scatterometer publications e Wilson J J W Figa Saldafia J and O Clerigh E 2004 ASCAT Product Generation Function Specification Issue 6 Rev 5 EUMETSAT EUM EPS SYS SPE 990009 Available on http www eumetsat int e WMO 2007 Additions to BUFR CREX Tables for pre operational implementation endorsed by CBS for full operational status on 7 November 2007 updated 04 01 07 pages 55 60 available on http www wmo int web www WMOCodes Updates BUF
10. RMS over N5 vector length between observed and model winds ambiguity Fraction of N5 where the chosen solution is not the one closest to the model wind Table 4 24 Parameters in monitoring output 4 3 5 Module awdp grib The module awdp grib reads in ECMWF GRIB files and collocates the model data with the scatterometer measurements The awdp_grib module uses the genscat module gribio module see subsection 4 2 5 for the interface with the GRIB routine library Table 4 25 provides an overview of the routines and their calls in this module The genscat support routines uv fo speed and uv to dir are used to convert NWP wind components into wind speed and direction Routine Call Description get grib data AWDP Get land mask ice mask and background winds using GRIB data init grib processing grib data Initialise module Table 4 25 Routines in module grib NWP model sea surface temperature and land sea mask data are used to provide information about possible ice or land presence in the WVCs WVCs with a sea surface temperature below 272 16 K 1 0 are assumed to be covered with ice and the ice and qual sigma0 flags in quality are set Note that this step is omitted if the ice screening is used see section 4 3 7 Land presence within each WVC is determined using the land sea mask available from the model data The weighted mean value of the land fractions of all model grid points within 80 km of the
11. WVC centre is calculated and if this mean value exceeds a threshold of 0 02 the qual sigma0 flag in wvc_quality is set The land flag in wvc_quality is set if the calculated land fraction is above zero NWP forecast wind data are necessary in the ambiguity removal step of the processing Wind forecasts with forecast time steps of 3h 6h 36h can be read in The model wind data are linearly interpolated with respect to time and location and put into the model_wind part of each WVC 43 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 4 3 6 Module inversion Module awdp inversion serves the inversion step in the wind retrieval The inversion step is done cell by cell The actual inversion algorithm is implemented in the genscat modules inversion and post inversion see subsection 4 2 1 Table 4 26 provides an overview of the routines and their calls in this module Routine Call Description init inversion invert wvcs Initialisation invert node invert_wvcs Call to the genscat inversion routines invert wvcs AWDP Loop over all WVCs and perform inversion Table 4 26 Routines of module awpd inversion 4 37 Module ambrem Module awdp_ambrem controls the ambiguity removal step of the AWDP program The actual ambiguity removal schemes are implemented in the genscat module ambrem see section 4 2 2 The default method is the KNMI 2DVAR scheme Table
12. gt iceMapWeighted iceMap2scat test cell printlceMap Figure A 8 Calling tree for routine awdpIcemodel first level 79 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 remove ambiguities GetElapsedSystemTime Init mbremModule InitBatchModule Init mbremMethod InitAmbremBGclosest InitTwodvarModule InitDummyMethod GetMaxBatchSize fill batch get distance AllocRowsAndCellsAnd InitBatch AllocAndInitBatchRow InitBatchRow InitBatchCell AllocAndInitBatchCell InitBatchCell InitBatchAmbi speeddir_to_u speeddir_to_v TestBatch TestBatchRow TestBatchCell DoAmbrem select wind TestBatchCell test cell gt DeallocBatch DeallocBatchRows DeallocBatchCells DeallocBatchAmbis ExitAmbremMethod ExitTwodvarModule TDV_Exit Figure A 9 Calling tree for routine remove_ambiguities first level The full name of the 12 routine is AllocRowsAndCellsAndInitBatch 80 DocID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Version 2 0 01 Reference Guide Date November 2010 postprocess GetElapsedSystemTime monitoring speeddir_to_u speeddir_to_v get_lun free_lun get_lun free_lun get_lun free_lun Figure A 10 Calling tree for routine postprocess first level write_bufr_file GetElapsedSystemTime init_bufr_processing set
13. see 2 3 3 It also contains the Fortran program test modules that generates the binary BUFR tables B and D from the ASCII tables already present and is executed automatically by the make system Program test modules can also be used to test the genscat BUFR module The Makefile in subdirectory genscat support bufr bufr tables calls some shell scripts which make symbolic links using the 1n s command of the generic binary BUFR tables B and D under different names There are four different naming conventions in BUFR version 000240 to 000280 and binary files are generated for each of them Symbolic links are not guaranteed to work on each platform e g by some versions of Cygwin under Windows XP so in some cases it may be necessary to replace the 1n s by cp copy Further information on the make system is given in the inline comments in the scripts and Makefiles Compilation and linking of the AWDP part is done in a similar manner go to the awdp directory and enter make As with genscat the make system will execute Makefiles in every subdirectory of awdp The result is the executable awdp in directory awdp src and a symbolic link to this executable in awdp execs AWDP is now ready for use The make system of AWDP doesn t need any further files except the genscat file Makeoptions This is the reason why genscat should be compiled first When recompiling part of AWDP or genscat with the make system for instance when installing a new v
14. 0 01 Reference Guide Date November 2010 Chapter 5 Inversion module 5 1 Background In the inversion step of the wind retrieval the radar backscatter observations in terms of the normalized radar cross sections o s are converted into a set of ambiguous wind vector solutions In fact a Geophysical Model Function GMF is used to map a wind vector specified in term of wind speed and wind direction to the o values The GMF further depends not only on wind speed and wind direction but also on the measurement geometry relative azimuth and incidence angle and beam parameters frequency polarisation A maximum likelihood estimator MLE is used to select a set of wind vector solutions that optimally match the observed o s The wind vector solutions correspond to local minima of the MLE function MLE x o 0 Ol 5 1 N K With N the number of independent o measurements available within the wind vector cell and K the covariance of the measurement error Following a Bayesian approach K is a constant representing the noise in all three ERS or ASCAT beams together Stoffelen and Portabella 2006 This selection depends on the number of independent o values available within the wind vector cell The MLE can be regarded upon as the distance between an actual scatterometer measurement and the GMF in N dimensional measurement space The MLE is related to the probability P that the GMF at a certain wind speed and direction repres
15. 1 size section 3 Kreserved3 integer ksec3 2 reserved ksubsets integer ksec3 3 number of reserved subsets kDataFlag integer ksec3 4 compressed 0 1 observed 0 1 Nsec4 integer ksup 4 dimension section 4 nsec4size integer ksec4 1 size section 4 kReserved4 integer ksec4 2 reserved nelements integer ksup 5 actual number of elements nsubsets integer ksup 6 actual number of subsets nvals integer ksup 7 actual number of values nbufrsize integer ksup 8 actual size of BUFR message ktdlen integer Actual number of data descriptors ktdexl integer Actual number of expanded data descriptors ktdlst integer array List of data descriptors ktdexp integer array List of expanded data descriptors values real array List of values cvals character array List of CCITT IA no 5 elements cnames character array List of expanded element names cunits character array List of expanded element units Table 8 4 Attributes of the BUFR message data type BufrDataType The next step is to bring the section data to actual dimensions descriptions and values of data which can be interpreted as physical parameters Therefore instances of BufrSectionsType are transferred to instances of BufrDataType see table 8 4 The actual data for input or output in a BUFR message should be an instance of the BufrDataType data type Some meta information on the BUFR file is contained in the self explaining bufr_file_attr_data data type see table 8 5
16. 18 21 24 255 m s Figure 2 3 Global coverage of the test run Wind speed results for the 25 km product are shown Directory genscat support bufr contains a test program named test_modules It is invoked by the genscat make system to construct the BUFR tables required by AWDP but it can also be used to test the genscat BUFR module The program is used as follows test modules BUFRinput where BUFRinput is the BUFR input file If omitted the program uses as default input the file testreading bufr in directory genscat support bufr The output is written to a BUFR file named testwriting bufr The directory also contains a shell script named run_test_modules that sets the environment variables required and executes the program Further information can be found in the comment lines of the source code of test_modules Directory genscat support grib contains test programs named test_read_GRIB1 test read GRIB2 and test read GRIB3 The programs can be run from the command line and read in the GRIB file testfile grib in directory genscat support grib Some properties of this file are written to ASCII output files Note that the environment variable SGRIB DEFINITION PATH needs to be set to directory genscat support grib definitions Subdirectories Compiler Features convert ErrorHandler singletonfft file BFGS num sort and datetime of genscat support contain test programs for the mo
17. 4 27 lists the tasks of the individual routines Routine Call Description fill batch remove ambiguities Fill a batch with observations remove ambiguities Main routine of ambiguity removal select wind remove ambiguities Final wind selection Table 4 27 Routines of module awpd ambrem The ambiguity removal scheme works on a so called batch The batch is defined in the fill batch routine For the AWDP program a batch is just a set of rows The size of the batch is determined by the resolution of the structure functions and the number of FFT The genscat routine remove ambiguities performs the actual ambiguity removal Finally select wind passes the selection to the output WVCs 4 3 8 Module awdp_icemodel Module awdp_icemodel performs the ice screening of the wind product The ice screening works on the principle that WVCs over water yield wind solutions which are close to the GMF cone If a WVC is over ice the o triplets from fore mid and aft beam will be close to the so called ice line Hence there is a possibility to discriminate between water wind and ice WVCs The implementation of this principle is described in more detail in Verspeek 2006 The ice screening is done directly after the ambiguity removal step Table 4 28 provides an overview of the routines and their calls in this module Routine Call Description calcIcelineParms nonbayesianIceModel Calculate distance to ice line from given oy s c
18. Module BufrMod is part of the genscat support modules The current version is a Fortran 90 wrapper around the ECMWF BUFR library see http www ecmwf int The goal of this support module is to provide a comprehensive interface to BUFR data for every Fortran 90 program using it In particular BufrMod provides all the BUFR functionality required for the scatterometer processor based on genscat Special attention has been paid to testing and error handling 8 1 Background The acronym BUFR stands for Binary Universal Form for the Representation of data BUFR is maintained by the World Meteorological Organization WMO and other meteorological centres In brief the WMO FM 94 BUFR definition is a binary code designed to represent employing a continuous binary stream any meteorological data It is a self defining table driven and very flexible data representation system It is beyond the scope of this document to describe BUFR in detail Complete descriptions are distributed via the websites of WMO http www wmo int and of the European Centre for Medium range Weather Forecasts ECMWF http www ecmwf int Module BufrMod is in fact an interface On the one hand it contains temporary definitions to set the arguments of ECMWF library functions On the other hand it provides self explaining routines to be incorporated in the wider Fortran 90 program Section 8 2 describes the routines in module BufrMod The public available data stru
19. Module icemodel ee 32 424 Mod le Bufrmod 2523 un ROI D yu dep ERR Ue 32 4 2 5 Module gribio module cesses nsei inin 33 42 6 Support modules it tope es etae e ette e der e eoe e pde 33 4 3 MODULE DESIGN FOR PROCESS LAYER 34 4 3 1 Mod le awdp d ta z s esca e ete ne sieves eec ree E ede 34 4 3 2 Modu le awdp buff dere b ce e iie ete P ere HE ERI REGE aah EUER ce 40 433 JMod le Awdp fs enn on P e a Hie ee ede 41 43 4 Module awdp prepost sae ete eee eter dd ener iri ed tegi eit 41 43 3 Module awdp grib i e er Atte pet tee 43 4 3 6 Module awdp inversion eese esee eese eene 44 4 3 7 Module ambrem 44 4 3 6 Module awdp icemodel eee iea a asana entente enne 44 43 9 Module awdpi tee er tate hele BRR eee eee ee ene eta 45 CHAPTER5 INVERSION MODULE e eee ee eee ene ee etta sese seen ases seen eee eaae eese en sesta 46 5 1 BACKGROUND onde oa E iendae o ntn dU aae 46 5 2 ROUTINES tb edi ed RR E Eee i eos 47 5 3 ANTENNAS DIRECTION e teu eee eee dal Eier 48 6 AMBIGUITY REMOVAL MODULE oeste tn sees see seen ases ena 49 6 1 AMBIGUITY REMOVAL eee etc e eie e eit aie
20. Post processing Setting of flags Monitoring Clean up Deallocation of used memory awdp_grib GRIB file handling Interface to genscat support grib Collocation of GRIB data NWP data are interpolated w r t time and location 29 AWDP User Manual and DocID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Module name Tasks Comments inversion Inversion Interface to genscat inversion awdp ambrem Ambiguity Removal Interface to genscat ambrem icemodel Ice screening Interface to genscat icemodel Table 4 1 AWDP process modules Each module contains code for performing one or more of the specific tasks These tasks are shortly described in table 4 1 A more elaborate description is given in section 4 3 The first module listed awdp data is a general support module This module is used by the other modules of the process layer for the inclusion of definitions of the data structures and the support routines The second module layer is the genscat layer The genscat module classes 1 e groups of modules used in the AWDP program are listed in table 4 2 The genscat package is a set of generic modules which can be used to assemble processors as well as pre and post processing tools for different scatterometer instruments available to the user community A short description of the main interface modules is given in section 4 2 The most important classes of modules are related to the
21. Vogelzang and John de Vries are in alphabetical order the most important contributors Anton Verhoef June 2007 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Chapter 1 Introduction 1 1 Aims and scope The ASCAT Wind Data Processor AWDP is a software package written in Fortran 90 for handling data from the Advanced Scatterometer ASCAT and European Remote Sensing satellite ERS scatterometer instruments Details of these instruments can be found on several web sites and in several other documents see e g Portabella 2002 Stoffelen 1998 and information on the ESA and EUMETSAT web sites AWDP generates surface winds based on ASCAT and ERS data It allows performing the ambiguity removal with the Two dimensional Variational Ambiguity Removal 2DVAR method and it supports the Multiple Solution Scheme MSS The output of AWDP consists of wind vectors which represent surface winds within the ground swath of the scatterometer Input of AWDP is Normalized Radar Cross Section NRCS data These data may be real time The input files of AWDP are in BUFR or Product Format Specification PFS native MetOp format BUFR input may be provided using the BUFR templates for ERS or ASCAT output is always written using the ASCAT BUFR template Besides the nominal 25 km and 12 5 km products AWDP also has the capability to generate a coastal wind product where the backs
22. be found in Stoffelen de Haan Quilfen and Schyberg 2000 de Vries Stoffelen and Beysens 2005 de Vries and Stoffelen 2000 The calculation of the cost function and its gradient is rather complex matter The reader who is only interested in how the 2DVAR scheme is assembled into the genscat module class ambrem is referred to subsection 6 4 2 Readers interested in the details of the cost function calculations and the minimization should also read the subsequent subsections Subsection 6 4 3 forms an introduction to the cost function It is recommended to first read this section because it provides necessary background information to understand the code Subsection 6 4 7 on the actual minimization and subsection 6 4 8 on Fast Fourier Transforms are in fact independent of the cost function itself The reader might skip these subsections 6 4 2 Data structure interface and initialisation The main module of the 2DVAR scheme is TwoDvar Within the genscat ambiguity removal module class the interface with the 2DVAR scheme is set by module Ambrem2DVAR Table 6 6 lists its routines that serve the interface with TwoDvar 53 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Routine Call Description Do2DVARonBatch DoAmbrem Apply 2DVAR scheme on batch BatchInput2DVAR Do2DVARonBatch Fills the 2DVAR data structure with input BatchOutput2DVAR Do2DVARonBatch Fills the batch
23. one must use a free Unix emulator like Cygwin see http www cygwin com for more information and download and section 2 3 6 for some directions To install AWDP the following steps must be taken 1 Copy the AWDP package file AWDP lt version gt tar gz to the directory from which AWDP will be applied and unzip and untar it This will create subdirectories awdp and genscat that contain all code needed see section 2 3 1 and a script called Instal 1AWDP 10 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 for easy compilation 2 Download the ECMWF BUFR library file bufr_000380 tar gz or another version not earlier than 000240 and copy it to directory genscat support bufr See also section 2 33 3 Download the ECMWF GRIB API library file grib api 1 9 0 tar gz or a newer version and copy it to directory genscat support grib See also section 2 3 4 4 Go to the top directory and run the InstallAWDP script The script will ask for the compiler used and it will invoke the make system for compilation and linking of the software see also section 2 3 5 AWDP is now ready for use provided that the environment variables discussed in section 2 3 2 have the proper settings See also sections 2 4 and 2 5 for directions on how to run AWDP 2 3 Directories and files All code for AWDP is stored in a file named AWDP lt version gt tar gz that i
24. quality In the AWDP program two flags are set for every WVC see table 4 3 The flags themselves do not address a single aspect of the data but the flags are composed of several bits each addressing a specific aspect of the data A bit is set to 0 1 in case the data is valid not valid with respect to the corresponding aspect In order to enhance the readability of the code each flag is translated to a data type consisting of only booleans false valid true invalid On input and output these data types are converted to integer values by set and get routines Flag Tasks Description wvc quality Quality checking In BUFR output process_flag Range checking Not in BUFR output Table 4 3 Flags for every WVC attributes of cell type Apart from the flags on WVC level also the beams contain quality indicators Most of them are implemented as real values ranging from 0 to 1 where 0 stands for good quality and 1 for degraded quality See section 4 3 1 for more information on this 4 1 5 Verbosity Every routine in a module may produce some data and statements for the log of the processor To 31 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 control the size the log several modules contain parameters for the level of verbosity The verbosity of the AWDP program may be controlled by the verbosity command line option verbosity In general there are three level
25. statements while processing the GRIB data As said before the underlying encoding and decoding routines originate from the ECMWF GRIB library Appendix B4 shows the calling trees of the routines in module gribio module that are used in AWDP 9 3 Data structures Some meta information on the GRIB file is contained in the self explaining grib file attr data data type see table 9 2 The decoded GRIB messages in the GRIB files with their meta information are contained in the grib message data see table 9 3 Attribute Type Description nr of GRIB messages integer Number of messages in this file grib filename character array Name of GRIB file grib fileunit integer Unit number in file table file size integer Size of GRIB file in bytes file open logical Status flag list of GRIB message ids integer array Message ids assigned by GRIB API list of GRIB level integer array Key to information in messages list of GRIB level type integer array Key to information in messages list of GRIB date integer array Key to information in messages list of GRIB hour integer array Key to information in messages list of GRIB analyse integer array Key to information in messages list of GRIB derived date integer array Key to information in messages list of GRIB derived hour integer array Key to information in messages 70 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date Novem
26. subsequent rows are 3 766 seconds apart The time corrections are rounded to an integer number of seconds Hence in the first row 34 seconds are subtracted from the acquisition time in the second row 30 seconds et cetera until in the last 19 row 34 seconds are added to the acquisition time 4 3 3 Module pfs The module awdp pfs maps the records in a PFS file on the AWDP data structure It also contains a routine to read in a full resolution PFS file and use the data to calculate averaged beam data which are used to replace 25 12 5 km row data Table 4 22 provides an overview of the different routines and their calls in this module Several routines from the pfs ascat module in genscat are called from this module to handle the PFS data Appendix B5 shows the calling trees of the routines in module pfs ascat that are used in AWDP Routine Call Description ascat pfs to row data read pfs file ASCAT PFS record into one row type read full res data AWDP Read full resolution PFS data and replace beam data read pfs file AWDP Read a complete PFS level 1b file into row types Table 4 22 Routines in module awdp_pfs 4 3 4 Module awdp_prepost Module awdp_prepost contains the routines to do all the pre and post processing Pre processing consists of the procedures between the reading of the BUFR input and the wind retrieval for the output product This includes sorting and merging and assessments of the quality of the
27. te eee re deae 49 6 2 MODULE AMBREM sete hatin eee E esee qe e ube te e decet EUR Ree cb de 49 6 3 MODULE BATCHM OD niece tidied en cab eset iie 50 6 4 THEKNMI DVARSCHEME 5 53 6 4 1 THITODUCH ON SNC 53 6 4 2 Data structure interface and initialisation eese eee nennen 53 6 4 5 Reformulation and transformation eese eene etre etre eene tnter trennen 56 6 44 Module CostFunction vccccccccccccccccccccccscssesscscccccsssscscssesscscssssscsssccssssesscessssccscsesscssessccscsesssssaseecs 56 645 X Adjont melliod iie em dio ores E 56 6 4 6 ec ee ERN REP eie eee ER Oe epe eet ates 57 OE MIBIPIZAlIQIUS sse ER Rv Eee ues 57 6 4 8 SingletonFFT Module e tera ertet ient ansa io ee eee EE 58 6 5 THEPRESGAT SCHEME iine n Dm PEE RE DEDISSE 58 CHAPTER7 MODULE ICEMODELMOD ee ee eee eee eene eee eee ease e seen ness eese 59 7 1 BACKGROUND iicet eiie tiec edere fei cese ee eee abes ees diee ea Rad 59 3 2 ROUTINES dietis mete tere at len ra Eee e RE e ai iet te nm oes aia en et 60 7 3 DATA STRUCTURES 5 x ceHaneseihsussscieDegneeeingertaieeeeludar deret 61 7 4 PARAMETERS 5 555 oti etse tee rette e te tertius patte teen tese t tentes ete Ree gus 61 CHAPTER8 MODULE BUFRMODBD e eee eee e
28. the ambiguity removal These steps are performed on the output data The program ends with the post processing step which includes some conversions and the monitoring and the mapping of the output data structure onto BUFR messages of the BUFR output file The different stages in the processing correspond directly to specific modules of the code These modules form the process 28 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 layer see section 4 3 Read input data Read full resolution data Figure 4 1 Baseline of the ASCAT Wind Data Processor 4 1 2 Layered model structure AWDP is a Fortran 90 program consisting of several Fortran 90 modules which are linked after their individual compilation The AWPD program is set up from two layers of software modules The purpose of the layer structure is to divide the code into generic scatterometer processing software and ASCAT specific software Details on the individual modules can be found in sections 4 2 and 4 3 The first layer the process layer consists of modules which serve the main steps of the process Module name Tasks Comments awdp data Definition of data structures awdp bufr BUFR file handling Interface to genscat support bufr awdp_pfs PFS file handling Interface to genscat support pfs awdp_prepost Sorting of input Duplicate rows are merged Quality control Usability of input data is determined
29. 50 seconds before the first data in the level lb WVCs and ending no less than 150 seconds after the last data in the level lb WVCs AWDP needs a high resolution land sea mask in order to determine if a full resolution backscatter measurement is over land or sea This information is used in coastal areas to skip backscatter data over land and to use only backscatter data over sea in the averaging of full resolution data The high resolution land sea mask should be available in GRIB format and should have a resolution of approximately 15 km or better The file containing the land sea mask should be present in the list of GRIB files supplied with the nwpf1 command line option see section 2 4 3 5 4 WVC triplet completion and row merging AWDP sorts the WVC rows in the input file by their acquisition date and time and merges WVC 25 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 information if duplicate rows occur The duplicate information is considered and the output will contain as much useful information as is available in the input WVCs This is especially useful if direct readout data from different ground stations is processed Sometimes a WVC from one ground station contains only fore beam information whereas the corresponding WVC from a second ground station contains only the mid and aft beam information AWDP will combine the information and it will process and write o
30. BUFR file attributes open BUFR file gt InitAndSetNrOfSubsets row to bufr data Bufrint2Real set_beam_collocation set_kp_estim_qual set_wvc_quality save BUFR message gt close BUFR fille gt Figure A 11 Calling tree for routine write bufr file first level Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 gt init cell get wvc quality Figure A 12 Calling tree for routine init cell second level gt test cell test beam Figure A 13 Calling tree for routine fest cell second level print cell print time Figure A 14 Calling tree for routine PrintCell second level 82 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 Appendix B1 Calling tree for inversion routines The figures in this appendix show the calling tree for the inversion routines in genscat All routines are part of genscat as indicated by the black boxes An arrow before a routine name indicates that this part of the calling tree 1s a continuation of a branch in a previous figure The same arrow after a routine name indicates that this branch will be continued in a following figure 83 DocID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Version 2 0 01 Reference Guide Date November 2010 invert one wvc
31. Besides the wind processing some other modes of operation are available If one of the modes is invoked AWDP internally sets some of the options in order to obtain the desired result Note that these modes are always used in combination with the f input file option mononly Write the monitoring file without any processing properties Write some properties of the last row of the input file The acquisition date and time and the sub centre 1d are written to a small ASCII output file properties txt writeonly Write all data to BUFR output without processing This mode is useful to copy an input file to BUFR output without processing Running the command awdp without any command line options will display a list of all available command line options with a short explanation on the console Running the command awdp with an illegal option will produce the same output but preceded by an error message The output will be written into a BUFR file with the name INSTR YYYYMMDD HHMMSS SAT ORBIT srv o SMPL CONT 12 pbufr where e INSTR 15 the instrument ascat or scatt e YYYYMMDD HHMMSS is the acquisition date and time UTC of the first data in the file e SAT is the satellite 6 characters ersl__ ers2__ metopa or metopb e ORBIT is the orbit number 5 digits of the first data in the file 00000 for ERS data e SMPL is WVC sampling cell spacing 250 for 25 km and 125 for 12 5 km e CON
32. BufrDataType to file EncodeBufrData save BUFR message Convert from BufrSectionsType to BufrMessageType CheckBufrData EncodeBufrData Data check FillBufrData EncodeBufrData Convert from BufrDataType to BufrSectionsType bufr msg is valid not used set bufr msg to invalid not used PrintBufrData not used GetPosBufrData not used GetRealBufrData not used GetIntBufrData not used GetRealBufrDataArr not used GetIntBufrDataArr not used GetRealAllBufrDataArr not used CloseBufrHelpers not used missing_real not used missing_int not used int2real not used do_range_check_int not used do_range_check_real not used AddRealDataToBufrMsg not used AddIntDataToBufrMsg not used PrintBufrModErrorCode not used GetFreeUnit encode table b encode table d Get free file unit Table 8 1 Routines of module BufrMod Reading decoding Routine get BUFR message reads a single BUFR message from the BUFR file and creates an instance of BufrDataType Writing encoding Routine save BUFR message saves a single BUFR message to the BUFR file The data should be provided as an instance of BufrDataType Checking and Printing The integer parameter BufrVerbosity controls the extent of the log statements while processing the BUFR file The routines PrintBufrData and CheckBufrData can be used to respectively print and check instances of BufrDataType Open and Close BUFR files The routine open BUFR file opens the BUFR file for either reading writemode fa
33. FR input files The main output consists of BUFR files The output BUFR messages are always in the ASCAT BUFR format for a list of descriptors see appendix C The user may provide arguments and parameters according to Unix command line standards The purpose of the different options is described in the User Manual Chapter 2 When executed the AWDP program logs information on the standard output The detail of this information may be set with the verbosity flag The baseline of processing is described in Figure 4 1 but note that not all of these steps are always invoked Some of them will be skipped depending on the command line options A more detailed representation of the AWDP structure is given in Appendices A and B The first step is to process the arguments given at the command line using the genscat Compiler Features module Next the AWDP program reads the input file specified in the arguments The BUFR messages or PFS records are read and mapped onto the AWDP data structure see subsection 4 1 3 As part of the pre processing a similar AWDP data structure is created for the output Subsequently the input data are sorted with respect to data acquisition time duplicate rows are merged and the output data structure is filled with level 1b related data Then the NWP GRIB data wind forecasts land sea mask and sea surface temperature are read and the data are collocated with the Wind Vector Cells The next steps are the inversion and
34. Invalid satellite instrument id sat motion Invalid satellite direction of motion time Invalid date or time specification latlon Invalid latitude or longitude pixel size hor Invalid cell spacing node nr Invalid across track cell number beam 3 Invalid data in one of the beams model wind Invalid background wind ambiguity Invalid ambiguities selection Invalid wind selection Table 4 17 Cell process flag bits Fortran Table 4 18 provides an overview of all routines and their calls in module awdp data Routine Call Description copy cell Copy all information from one cell into another get beam collocation init cell Convert integer beam collocation to logical structure get kp estim qual init beam Convert integer K estimate quality to logical structure get wvc quality init cell Convert integer WVC quality to logical structure init ambiguity Initialise ambiguity structure init beam init cell Initialise beam structure 39 AWDP User Manualand D901D NWPSAF KN UD 005 NWP SAF 2 Version 2 0 01 Reference Guide Date November 2010 Routine Call Description init cell Initialise cell structure init full res init cell Initialise full resolution structure init icemodel init cell Initialise ice model structure init nwp stress param init cell Initialise NWP stress parameters structure init process flag init cell Initialise process flag structure init time init cell Initialise time structure init_wind init_cell Initi
35. NWP SAF Satellite Application Facility for Numerical Weather Prediction Document NWPSAF KN UD 005 Version 2 0 01 November 2010 AWDP User Manual and Reference Guide Anton Verhoef Jur Vogelzang Jeroen Verspeek and Ad Stoffelen KNMI De Bilt the Netherlands The EUMETSAT Network o Satelite Application Faciities Numerical Weather Prediction Royal Netherlands M M t Offi Tm ECMWF BUR Meteorological Institute Ministry of Transport Public Works and Water Management METEO FRANCE Toujours un temps d avance Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 AWDP User Manual and Reference Guide KNMI De Bilt the Netherlands This documentation was developed within the context of the EUMETSAT Satellite Application Facility on Numerical Weather Prediction NWP SAF under the Cooperation Agreement dated 16 December 2003 between EUMETSAT and the Met Office UK by one or more partners within the NWP SAF The partners in the NWP SAF are the Met Office ECMWF KNMI and M t o France Copyright 2010 EUMETSAT AII Rights Reserved Change record Version Date Author changed by Remarks 1 0j Jun 2007 Anton Verhoef First draft 1 0k Oct 2007 Anton Verhoef Adapted for AWDP version 1 0k 1 0 13 Mar 2008 Anton Verhoef Adapted for AWDP version 1 0 13 1 0 14 Oct 2008 Anton Verhoef First version for ex
36. November 2010 validation can be found in the reports de Vries and Stoffelen 2000 de Vries Stoffelen and Beysens 2005 The performance of 2DVAR with meteorological balance constraints was tested and optimized for ERS data It was found to be superior to other schemes 3 5 8 Monitoring For the automatic ingestion of observations into their NWP systems meteorological centres require quality checks on the NRT products For the ASCAT wind product a monitor flag is under development analogous to the one developed for the SeaWinds Wind Product This flag indicates that several measures on the level of corruption of the output BUFR files are above a specified threshold Onset of the flag indicates that the input should be rejected for ingestion by the NWP system Details on the monitor developed can be found in the NWP SAF document de Vries Stoffelen and Beysens 2005 3 6 Details of performance AWDP is delivered with two example BUFR input files containing data from 26 April 2007 They are named ascat 20070426 test 250 11 bufr 25km cell spacing ascat 20070426 test 125 11 bufr 12 5 km cell spacing and contain approximately half an orbit of data Moreover a set of ECMWF GRIB files containing the necessary NWP output is supplied Table 3 2 gives the approximate times needed for processing these files under various options on a personal workstation with a 2 66 GHz Pentium 4 processor under Linux using the GNU g95 Fort
37. Of Moving Observing Platform Degree True 7 004001 Year Year 8 004002 Month Month 9 004003 Day Day 10 004004 Hour Hour 11 004005 Minute Minute 12 004006 Second Second 13 005001 Latitude High Accuracy Degree 14 006001 Longitude High Accuracy Degree 15 005033 Pixel Size On Horizontal 1 M 16 005040 Orbit Number Numeric 17 006034 Cross Track Cell Number Numeric 18 010095 Height Of Atmosphere Used m 19 021157 Loss Per Unit Length Of Atmosphere Used dB m 20 021150 Beam Collocation Flag Table 21 008085 Beam Identifier Code Table 22 002111 Radar Incidence Angle Degree 23 002134 Antenna Beam Azimuth Degree 24 021062 Backscatter dB 25 021063 Radiometric Resolution Noise Value 26 021158 ASCAT Kp Estimate Quality Code Table 27 021159 ASCAT Sigma 0 Usability Code Table 28 021160 ASCAT Use Of Synthetic Data Numeric 29 021161 ASCAT Synthetic Data Quality Numeric 30 021162 ASCAT Satellite Orbit And Attitude Quality Numeric 31 021163 ASCAT Solar Array Reflection Contamination Numeric 32 021164 ASCAT Telemetry Presence And Quality Numeric 33 021165 ASCAT Extrapolated Reference Function Numeric 34 021166 ASCAT Land Fraction Numeric 35 008085 Beam Identifier Code Table 36 002111 Radar Incidence Angle Degree 37 002134 Antenna Beam Azimuth Degree 38 021062 Backscatter dB 39 021063 Radiometric Resolution Noise Value 40 021158 ASCAT Kp Estimate Quality Code Table 99 AWDP User Manual and Doc ID NWPSAF KN UD 005
38. Pixels integer Number of pixels for the ice map nLines integer Number of lines for the ice map nHist integer Number of historical measurements that is stored 72 pole integer Indicator for Northpole or Southpole timelceNow DateTime Expanded section 3 data description timelcePrev DateTime Expanded section 4 data Xy IcePixel nPixels nLines Pointer to the ice map contents Table 7 3 Attributes for the IceMapType data type 7 4 Parameters There are several parameters involved that control the Bayesian statistics They have sensible default values but most of them are made public so that their value can be overridden in the main program 61 NWP SAF Doc ID Version Date NWPSAF KN UD 005 2 0 01 November 2010 AWDP User Manual and Reference Guide Parameter Description writeRestartFile Logical controlling the writing of a restartfile use WindInfo Logical controlling whether wind information is used useLandpoints Logical controlling whether an ice probability is calculated for land points use_only_windponts use_sst weightS decayTime cutoffTime pClimatelce qClimatelce Class no data Class sea Class ice Class sea or ice Class no sea no ice SubClass a2 SubClass bl SubClass b2 SubClass b3 SubClass d SubClass no data nHist aSdLimit plceGivenX limit sumWeightSTLimit dReflce dRefWind sstLowLimit sstHighLimit Logical controlling whether only points with a va
39. RCREX Preoperational050107 doc 73 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Appendix A Calling tree for AWDP The figures in this appendix show the calling tree for the AWDP program Routines in white boxes are part of AWDP process layer Routines in black boxes are part of genscat An arrow gt before a routine name indicates that this part of the calling tree is a continuation of a branch in a previous figure The same arrow after a routine name indicated that this branch will be continued in a following figure iargc_genscat getarg_genscat write_usage read bufr file gt read pfs file preprocess gt read_full_res_data gt calibrate sO get grib data 2 invert awdplcemodel gt x x remove ambiguities postprocess calibrate sO write bufr file process cleanup GetElapsedSystemTime Figure A 1 Calling tree for program awdp top level White boxes are cut here and will be continued in one of the first level or second level calling trees in the next figures Black boxes with light text indicate genscat routines AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 read bufr file GetElapsedSystemTime init_bufr_processing set BUFR fileattributes open BUFR file get BUFR nr of mess
40. T contents is omitted if the data contains both wind and soil moisture data Otherwise it is set to __ovw Ocean Vector Winds or ssm Example ascat_20070426_095102_metopa_02681_srv_o_250_ovw 12_bufr 2 5 Scripts Directory awdp execs contains a Bourne shell script awdp_run for running awdp with the correct environment variables The script can be invoked with all valid command line options for awdp In the same directory there is also a script awdp_gui py available This script provides convenient graphical user interface and builds and runs an AWDP command line depending on settings of available radio buttons check boxes et cetera This script requires Python to be installed on your system It may be necessary to change some of the environment variables set in the top part of the script 19 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 2 6 Test data and test programs Directory awdp tests contains two BUFR files for testing the AWDP executable File ascat 20070426 test 250 11 bufr contains ASCAT level 1b data from 26 April 2007 9 5 to 10 29 UTC with 25 km cell spacing The same data but on 12 5 km cell spacing is available in file ascat 20070426 test 125 11 bufr The files ECMWF grib contain the necessary NWP data SST land sea mask and wind forecasts to perform the NWP collocation step The user can test the proper functioning of AWDP using the files
41. UFR library AWDP needs the ECMWF BUFR library for its input and output operations Only ECMWF is allowed to distribute this software It can be obtained free of charge from ECMWF at the web page http www ecmwf int products data software bufr html The package contains scripts for compilation and installation The reader is referred to this site for assistance in downloading and installing the BUFR Library Directory genscat support bufr contains the shell script make bufr lib It unzips untars and compiles the BUFR library file which is downloaded from ECMWF and placed into this directory This script is part of the genscat make system and it is automatically invoked when compiling genscat The current version is tested with BUFR version 000380 but later versions or earlier but not earlier than 000240 can be used However AWDP is not tested with later versions BUFR file handling at the lowest level is difficult to achieve Therefore some routines were coded in C These routines are collected in library bufrio see also section 8 4 Its source code is located in file bu rio c in subdirectory genscat support bufr Compilation is done within the genscat make system and requires no further action from the user see 2 3 5 2 3 4 Installing the GRIB API library AWDP needs the ECMWF GRIB API library for its input operations Only ECMWF is allowed to distribute this software It can be obtained free of charge from ECMWF at the web page h
42. ackground surface wind field is meteorologically consistent However background surface winds have their own uncertainty Therefore sophisticated schemes for Ambiguity Removal take both the likelihood of the ambiguities and the uncertainty of the background surface wind into account Examples are the KNMI Two Dimensional Variational 2DVAR scheme and the PreScat scheme The implementation of these schemes is described in sections 6 4 and 6 5 6 2 Module ambrem Module Ambrem is the interface module between the various ambiguity removal methods and the different scatterometer data processors Table 6 1 provides an overview of the different routines 49 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 and their calls A dummy method and the first rank selection method are implemented as part of ambrem More elaborate Ambiguity Removal methods have an interface module see table 6 2 Figure 6 1 shows schematically the interdependence of the various modules for Ambiguity Removal Routine Call Description InitambremModule AWDP Initialization of module Ambrem InitambremMethod Initialization of specified AR scheme DoAmbrem AWDP Execution of specified AR scheme Ambreml stRank DoAmbrem First rank selection method DoDummyMeth DoAmbrem Dummy AR scheme for testing SetDummyMeth DoAmbrem Batch definition of dummy method InitDummyMeth DoAmbrem Initializati
43. ages get_BUFR_message gt ers_bufr_to_row_data init_cell gt get wvc quality BufrReal2int get_beam_collocation test cell 2 ymdajulian julian2ymd ascat bufr to row data init cell gt get wvc quality BufrReal2int get_beam_collocation get_kp_estim_qual test cell 2 close BUFR file 2 ymd2julian Figure A 2 Calling tree for routine read_bufr_file first level 75 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 read pfs file GetElapsedSystemTime get_lun open pfs ascat file ascat pfs to row data read pfs ascat mdr init cell gt get_pfs_ascat_node get_beam_collocation get_kp_estim_qual test_beam init_beam test cell close pfs ascat file free lun ymd2julian Figure Calling tree for routine read_pfs_file first level GetElapsedSystemTime GetSortIndex merge_rows init_cell gt copy_cell test cell 2 pre inversion qc Figure A 4 Calling tree for routine preprocess first level 76 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 read full res data GetElapsedSystemTime get_lun open pfs ascat file init grib processing init GRIB module set GRIB filelist gt inq
44. alcIceCoord 44 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Routine Call Description iceGMF not used Calculate the values from the ice coordinates iceLine iceGMF not used Calculate the ice line origin and slope generalisedIcemodel not used Instrument independent ice model n2s nls calcIcelineParms Convert from two sided to one sided node numbering nls n2s not used Convert from one sided to two sided node numbering coordTransform bayesianIcemodel Calculate coordinates on an SSM I grid scat2iceMap bayesianIcemodel Update the ice map with the information in cell data iceMap2scat bayesianIcemodel Update cell data structure with information in ice map bayesianIcemodel Calculate ice coordinates and distance to ice line bayesianIcemodel awdplcemodel Implementation of the Bayesian ice model nonbayesianlceModel awdpIcemodel Implementation of the basic ice model without history awdpIcemodel AWDP Main routine of ice screening Table 4 28 Routines of module awdp_icemodel 4 3 9 Module awdp Module awdp is the main program of AWDP It processes the command line options and controls the flow of the wind processing by calling the subroutines performing the subsequent processing steps If any process step returns with an error code the processing will be terminated 45 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2
45. alise wind structure print_ambiguity Print ambiguity structure print_beam Print beam structure print_cell Print cell structure print_full_res Print full resolution structure print_icemodel Print ice model structure print_nwp_stress_param Print NWP stress parameters structure print_process_flag Print process flag structure print_time Print time structure print_wind Print wind structure print_wvc_quality Print quality flag structure set_beam_collocation Convert logical beam collocation to integer set_kp_estim_qual Convert logical K estimate quality to integer set_wvc_quality Convert logical WVC quality to integer test_beam test_cell Test validity of beam data test_cell Test validity of cell data test_time test_cell Test validity of time data test_wind test_cell Test validity of wind data Table 4 18 Routines in module awdp_data 4 3 2 Module awdp_bufr The module awdp_bufr maps the AWDP data structure on BUFR messages and vice versa A list of the BUFR data descriptors can be found in appendix C Satellite and instrument identifiers are listed in tables 4 19 and 4 20 Note that the first MetOp mission is MetOp 2 which is also known as MetOp A The awdp_bufr module uses the genscat module BufrMod see subsection 4 2 3 for the interface with the BUFR routine library Satellite Value ERS 1 1 ERS 2 2 MetOp 1 MetOp B 3 MetOp 2 MetOp A 4 MetOp 3 MetOp C 5 Table 4 19 BUFR satellite identifiers Instrument Parameter Val
46. atterometer RMS Root Mean Square SAF Satellite Application Facility SSM Surface Soil Moisture SST Sea Surface Temperature WVC Wind Vector Cell also called node or cell Table D 1 List of acronyms 102
47. ber 2010 Attribute Type Description list of GRIB par id integer array Key to information in messages list of GRIB vals sizes integer array Size of data values arrays Table 9 2 Attributes for the grib file data data type Attribute Type Description message pos in file integer Position of message in GRIB file message id integer Message id assigned by GRIB API date real Date when data are valid time real Time when data are valid derived date real date time 24 derived time real mod time 24 total_message_size integer Size of message vals_size integer Size of data values array is_decoded logical Status flag nr_lon_points integer Information about grid nr_lat_points integer Information about grid nr_grid_points integer Information about grid lat of first gridpoint real Information about grid lat of last gridpoint real Information about grid lon of first gridpoint Information about grid lon of last gridpoint Information about grid lat step real Information about grid lon step real Information about grid real values real array pointer Decoded real data values Table 9 3 Attributes for the grib message data data type Attribute Type Description grib file attributes grib file data GRIB file attributes list of GRIB msgs grib message data array List of messages in file Table 9 4 Attributes of the ist of grib files type data type for GRIB files 9 4 Libraries Module grib
48. c cone distance calc dist to cone center convert sigma to zspace get ers noise estimate calc var s0 get dynamic range get GMF version used calc sigma0 invert one wvc invert one wvc see B 1 AWDP invert one wvc AWDP not used AWDP AWDP invert one wvc invert one wvc do parabolic winddir search find minimum cone dist not used invert one wvc calc var s0 calc normalisation not used not used see B 1 create LUT C VV test for identical LUTSs my mod my min my max my average get indices lowest local minimum my index max my exit print wind quality code print input data of inversion print output data of inversion print in out data of inversion calc sigma0 cmod4 fl Get_Br_from_Look_Up_Table calc_sigma0_cmod5 calc_sigma0_cmod5_5 calc sigma0 cmod5 n calc sigma0 cmod6 calc sigma0 calc sigma0 not used see B 1 see B 1 see B 1 invert one wvc see B 1 see B 1 see B 1 check input data see B 1 not used create LUT C VV calc sigma0 cmod4 calc sigma0 cmod4 create LUT C VV create LUT C VV create LUT C VV create LUT C VV Table 5 1 Routines in module inversion Routine Call normalise conedist ers ascat calc kp ers ascat calc geoph noise ers ascat normalise conedist prescat mode get ers noise estimate check ers ascat inversion data check wind solutions ers ascat remove one solution calc probabilities AWDP normalise
49. catter data from the level 1b files are replaced by box averaged backscatter values from the full resolution level 1 ASCAT product SZF data This mode of operation produces winds that are closer to the coast than the winds from the nominal level 1b data which contain backscatter values that are averaged using a Hamming filter see Portabella et al 2008 Currently end 2010 the SZF data are not available for users in near real time but only off line from the EUMETSAT Data Centre Apart from the ASCAT input data AWDP needs Numerical Weather Prediction NWP model winds as a first guess for the Ambiguity Removal step These data need to be provided in GRIB edition 1 or 2 1 2 Development of AWDP AWDP is developed within the Numerical Weather Prediction Satellite Application Facility NWP SAF and Ocean and Sea Ice Satellite Application Facility OSI SAF programs as code which can be run in an operational setting The coding is in Fortran 90 and has followed the procedures specified for the NWP SAF Special attention has been paid to robustness and readability AWDP may be run on every modern Unix or Linux machine In principle AWDP can also be run on a DocID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Version 2 0 01 Reference Guide Date November 2010 Windows machine if a Unix emulator like Cygwin is installed 1 3 Testing AWDP Modules are tested by test programs and test routines Many test routines or test support rout
50. conedist ers ascat calc kp ers ascat AWDP normalise conedist prescat mode see B 1 AWDP check wind solutions ers ascat AWDP Table 5 2 Routines of module post inversion To establish the MLE function 1 the radar cross section according to the GMF oo must be calculated This is done in routine calc sigma0 The GMF used is read as a Look Up Table LUT from a binary file The value for o GMF 47 is obtained from interpolation of this table The AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 interpolation is done via symbolic routine INTERPOLATE which is set to interpolateld interpolate2d interpolate2dv or interpolate3d depending on the type of interpolation needed For C band at VV polarization the GMF CMODx see Hersbach Stoffen and de Haan 2007 1s given in analytical form routines calc sigma0 cmodxxx If a C band LUT is not present it will be created by routine create VV This routine calls one of the routines calc sigma0 cmodxxx that contain the analytical expressions of the CMOD4 or CMODS algorithm There is a parameter in the inversion settings type that is used to determine which CMOD function is to be used Routines get lun and lun from module LunManager in subdirectory genscat support file are needed when reading and creating the LUTS Note that module post inversion uses some tables for the normali
51. ctures are described in section 8 3 BufrMod uses two libraries the BUFR software library of ECMWF and bufrio a small library in C for file handling at the lowest level These libraries are discussed in some more detail in section 8 4 8 2 Routines Table 8 1 provides an overview of the routines in module BufrMod The most important ones are described below Routine Call Description InitAndSetNrOfSubsets AWDP Initialization routine set BUFR fileattributes AWDP Initialization routine open BUFR file AWDP Opens a BUFR file get BUFR nr of messages AWDP Inquiry of BUFR file get BUFR message AWDP Reads instance of BufrDataType from file get expected BUFR msg size get BUFR message Inquiry of BUFR file ExpandBufrMessage get BUFR message Convert from BufrMessageType to BufrSectionsType 63 AWDP User Manualand 0910 NWESAE EN UD 005 NWP SAF 2 Version 2 0 01 Reference Guide Date November 2010 Routine Call Description PrintBufrErrorCode ExpandBufrMessage EncodeBufrData CheckBufrTables ExpandBufrMessage Data check get_file_size CheckBufrTables Determine size of BUFR file get_bufrfile_size_c get_file_size Support routine in C encode_table_b CheckBufrTables encode_table_d CheckBufrTables FillBufrSecData ExpandBufrMessage Convert from BufrSectionsType to BufrDataType close_BUFR_file AWDP Closes a BUFR file BufrReal2Int AWDP Type conversion Bufrint2Real AWDP Type conversion save BUFR message AWDP Saves instance of
52. d Do2DVAR Write analysis field to file Table 6 12 Routines of module CostFunction 6 4 5 Adjoint method The minimization of cost function is done with a quasi Newton method Such a method requires an accurate approximation of the gradient of the cost function The adjoint method is just a very economical manner to calculate this gradient For introductory texts on the adjoint method and adjoint coding see e g Talagrand 1991 Giering 1997 For detailed information on the adjoint 56 DocID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Version 2 0 01 Reference Guide Date November 2010 model in 2DVAR see Vogelzang 2007 6 4 6 Structure Functions Module StrucFunc contains the routines to calculate the covariance matrices for the stream function v and the velocity potential y Its routines are listed in table 6 13 Routine Call Description SetCovMat Do2DVAR Calculate the covariance matrices InitStrucFunc SetCovMat Initialize the structure functions StrucFuncPsi SetCovMat Calculate w StrucFuncChi SetCovMat Calculate y Table 6 13 Routines of module StrucFunc Routine nitStrucFunc sets the structure function parameters to a default value 6 4 7 Minimization The minimization routine used is LBFGS This is a quasi Newton method with a variable rank for the approximation of the Hessian written by J Nocedal A detailed description of this method is given by Liu and Nocedal 1989 Routine LBFGS is f
53. d real Mean value of full res values sigma0 mid real Mean value of full res values land frac mid real Mean value of full res values count aft integer Number of full res aft beams used incidence aft real Mean value of full res values azimuth aft real Mean value of full res values sigmaO0 aft real Mean value of full res values land frac aft real Mean value of full res values Table 4 8 Full res data structure Ice model data The icemodel type contains information related to the ice screening The attributes are listed in table 4 9 The routine init icemodel sets the ice model data to missing values The routine print icemodel may be used to print the ice data Attribute Type Description class integer Code for WVC being ice or wind ii integer Coordinate on the ice map jj integer Coordinate on the ice map b real Ice coordinate c real Ice coordinate dIce real Distance to the ice line Table 4 9 Ice model data structure 36 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 NWP stress parameter data The nwp stress param type data type contains information relevant for the ice screening and wind stress calculations stress calculation is not yet implemented in AWDP The attributes are listed in table 4 10 The routine init_nwp_stress_param sets the NWP stress parameter data to missing values The routine print_nwp_stress_param may be used to print the
54. data structure with output Set Orientations BatchInput2DVAR Sets the observation orientation GetBatchSize2DVAR Determine maximum size of batch Table 6 6 Routines of module Ambrem2DVAR These routines are sufficient to couple the 2DVAR scheme to the processor The actual 2DVAR processing is done by the routines of module TwoDvar itself These routines are listed in table 6 7 Figures B2 1 B2 6 show the complete calling tree of the AR routines Routine Call Description InitTwodvarModule Initialization of module TwoDvar Do2DVAR Do2DVARonBatch Cost function minimization PrintObs2DVAR BatchInput2DVAR Print a single 2DVAR observation ExitIwodvarModule ExitAmbremMethod Deallocation of module TwoDvar Table 6 7 Routines of module TwoDvar The Obs2dvarType data type is the main data structure for the observed winds Its attributes are listed in table 6 8 The TDV_Type data type contains all parameters that have to do with the 2DVAR batch grid dimensions sizes and derived parameters These data structures are defined in module TwoDvarData and the routines in this module are listed in table 6 10 Attribute Type Description alpha Real Rotation angle cell Integer Store batch cell number row Integer Store batch row number igrid Integer Row index jgrid Integer Node index lat Real Latitude to determine structure function Wil Real Weight lower left Wir Real Weight lower right Wul Real Weight upper left Wur Rea
55. detail in section 9 4 9 Routines Table 9 1 provides an overview of the routines in module gribio module The most important ones are described below Routine Call Description init GRIB module AWDP Initialization routine dealloc_all_GRIB_messages AWDP Clear all GRIB info from memory and close GRIB files set_GRIB_filelist AWDP Open all necessary GRIB files get_from_GRIB_filelist AWDP Retrieve GRIB data for a given lat and lon get_colloc_from_GRIB_filelist inquire_GRIB_filelist AWDP Inquiry of GRIB file list 68 NWP SAF AWDP User Manual and Reference Guide Doc ID Version Date NWPSAF KN UD 005 2 0 01 November 2010 Routine Call Description get colloc from GRIB filelist get GRIB msgnr display req GRIB msg properties display GRIB message properties open GRIB file read GRIB header info extract data from GRIB message get GRIB data values dealloc GRIB message get analyse dates and times check proximity to analyse get field from GRIB file get from GRIB file add to GRIB filelist get analyse dates and times get colloc from GRIB filelist AWDP get field from GRIB file get from GRIB file get from GRIB filelist inquire GRIB filelist get GRIB msgnr get from GRIB filelist get GRIB msgnr get from GRIB filelist get field from GRIB file get from GRIB file set GRIB filelist add to GRIB filelist open GRIB file get from GRIB file get fro
56. distance real Distance of solution to the GMF Table 4 5 Ambiguity data structure Beam data Every WVC contains three beams The information of every beam is stored in the data type beam type The attributes are listed in table 4 6 Most of the attributes are explained in detail in Wilson Figa Saldafia and O Clerigh 2004 The routine init beam sets all beam data to missing and the routine test beam checks if the data in the beam are within valid ranges The routine print beam may be used to print all beam data Attribute Type Description identifier integer Beam number fore 2 mid 3 aft incidence real Incidence angle degrees 0 is vertical 90 is horizontal azimuth real Radar look angle degrees counted clockwise from the south sigma0 real Radar backscatter in dB noise val real Noise value in 96 kp estim qual kp estim qual type Flag related to the quality of the Kp estimate sO usability integer Usability of 0 good usable 2 bad synt data quantity real Amount of synthetic data in op 0 1 synt data quality real Quality of used synthetic data in 0 1 orbit quality real Satellite orbit and attitude quality 0 1 solar reflec real Solar array reflection contamination in 0 1 telemetry real Telemetry quality 0 1 extrapol ref pres real Presence of extrapolated reference functions 0 1 land frac real Land fraction in 0 1 Table 4 6 Beam data struct
57. dule in that subdirectory The test programs write their result to the standard output In some cases a copy of the output is contained in the output files for comparison Table 2 6 gives an overview of the genscat test programs 21 AWDP User Manualand 2 NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Subdirectory Program name Output file Remarks bufr test_modules testwriting bufr Part of make system grib test_read_GRIB several GRIB handling Compiler Features TestCompiler Features Command line handling convert test convert test convert output Wind speed conversion ErrorHandler TestErrorHandler Error handling singletonfft TestSingleton Fast Fourier Transform file TestLunManager TestLunManager output File management BFGS Test BFGS Minimization num test numerics test numerics output Numerical issues pfs test pfs ascat Read PFS file sort SortModTest SortModTest output Array sorting datetime TestDateTimeMod TestDateTimeMod output Date and time conversion Table 2 6 Test programs in genscat support 2 7 Documentation Directory awdp doc contains documentation on AWDP including this document Further information can be found in the readme text files and in the comments in scripts Makefiles and source code 22 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Chapter 3 AWDP prod
58. dvar 2DVAR see section 6 4 The default is 2dvar par ana tc varqc ocf research orpm Various options intended for research activities More information can be found in the Fortran code of AWDP and genscat binof file Write selected data of each WVC to a binary output file Data are written to a binary file lt file gt This option is intended for research activities More information on the file format can be found in the Fortran code of AWDP writeall Write all data to BUFR output including level 2 input data In the normal near real time processing a mixture of level Ib and recent level 2 data is fed into AWDP in order to provide more data which is beneficial for ambiguity removal Only those data rows which were level 1b input are written to the level 2 output file This option overrides this behaviour and writes all rows to the output file handleall Perform NWP collocation inversion ambiguity removal and output writing in all WVCs By default these steps are done only for WVCs which are level 1b input see the description at the writeall option This option is useful for reprocessing level 2 data nws N Write N wind solutions in BUFR output default 2 The number of wind solutions to be written into the ASCAT BUFR format is flexible due to the use of the so called delayed replication and can be chosen between 1 providing only the selected wind solution and 144 providing all wind solutions in MSS p
59. e bufr_swap_allsections get_BUFR_message save_BUFR_message bufr_write_allsections save_BUFR_message bufr_close close_BUFR_file bufr_error see appendix B 3 Read BufrMessageType from BUFR file Optional byte swapping Write BufrMessageType to BUFR file Error handling Table 8 6 Routines in library bufrio 8 5 BUFR table routines BUFR tables are used to define the data descriptors The presence of the proper BUFR tables is checked before calling the reading and writing routines If absent it is tried to create the needed BUFR tables from the text version available in genscat 8 6 Centre specific modules BUFR data descriptors are integers These integers consist of class numbers and numbers for the described parameter itself These numbers are arbitrary To establish self documenting names for the BUFR data descriptors for a Fortran 90 code several centre specific modules are created These modules are listed in table 8 7 Note that these modules are just cosmetic and not essential for the encoding or decoding of the BUFR data They are not used in AWDP Module Description WmoBufrMod WMO standard BUFR data description KnmiBufrMod KNMI BUFR data description EcmwfBufrMod ECMWF BUFR data description Table 8 7 Fortran 90 BUFR modules 67 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Chapter 9 Module gribio module Module gribio module i
60. e id sm integer Soil moisture information database id integer Soil moisture information surface sm real Soil moisture information surface sm err real Soil moisture information sigma0 40 real Soil moisture information sigma0 40 err real Soil moisture information slope 40 real Soil moisture information slope 40 err real Soil moisture information sm sensitivity real Soil moisture information dry backscatter real Soil moisture information wet backscatter real Soil moisture information mean surface sm real Soil moisture information rain fall detect real Soil moisture information sm corr flag integer Soil moisture information sm proc flag integer Soil moisture information sm quality real Soil moisture information snow cov frac real Soil moisture information froz land frac real Soil moisture information inund wet frac real Soil moisture information topo complexity real Soil moisture information software id wind integer Software identification of level 2 wind processor generating app integer Generating application of model information model wind wind type Model wind used for Ambiguity Removal ice prob real Probability of ice ice age real Ice age A parameter wvc quality wvc quality type WVC quality flag num ambigs integer Number of ambiguities present in WVC selection integer Index of selected wind vector skill real Parameter used for PreScat Ambiguity Removal 35 AWDP User Manual and Doc ID NWPSAF KN UD 005
61. e a 3 order polynomial iceLine calcSubClass iceMapWeighted Calculate the ice subclass getClass updatelcePixel Get the ice class sea or ice nonbayesianIceModel printClass not used Print the ice class sea or ice getLatest iceMapWeighted Get the indices of the latest measurement calcAave calcSubclass updatelcePixel printIcePixel getPrevious iceMapWeighted Get the indices of the previous measurement calcAave GetQxGivenlce updatelcePixel Quotient of wind probability and ice probability iceMapWeighted bayesianIcemodel Calculate the ice a posteriori probability bayesianIcemodel Initialise ice map ExpandDateTime iceMapWeighted Converts a date time to a real calcAave updatelcePixel MAPLL coordTransform Convert from lat lon to polar stereographic coordinates MAPXY not used Convert from polar stereographic to lat lon coordinates wT iceMapWeighted calcAave Calculate the moving time average function logit not used Calculate the logit of p In p 1 p inv logit not used Calculate the inverse of the logit of p 1 1 printIcePixel updatelcePixel Print an ice pixel scat2iceMap RW IceMap bayesianIcemodel Read or write an ice map from to a binary restart file updatelcePixel scat2iceMap Update an 1ce pixel with the contents of a BUFR message 60 AWDP User Manual and PocID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Routine Call Description printlceMap baye
62. e a routine name indicates that this part of the calling tree is a continuation of a branch in a previous figure The same arrow after a routine name indicates that this branch will be continued in a following figure updatelcePixel getLatest ExpandDateTime getClass GetQxGivenlce printlcePixel gt Figure B6 1 Calling tree for routine updatelcePixel second level iceMapWeighted getLatest getPrevious ExpandDateTime calcAave getLatest getPrevious ExpandDateTime calcSubclasslce getLatest Figure B6 2 Calling tree for routine iceMapWeighted second level 97 DocID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Version 2 0 01 Reference Guide Date November 2010 printlceMap printice printppmcolor printppmvar Figure B6 3 Calling tree for routine printIceMap second level gt printlcePixel getLatest Figure 4 Calling tree for routine printIcePixel second level 98 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Appendix C ASCAT BUFR data descriptors Number Descriptor Parameter Unit 1 001033 Identification Of Originating Generating Centre Code Table 2 001034 Identification Of Originating Generating Sub Centre Code Table 3 025060 Software Identification Numeric 4 001007 Satellite Identifier Code Table 5 002019 Satellite Instruments Code Table 6 001012 Direction Of Motion
63. e connections from module ambrem to module BatchMod and from module Ambrem2DVAR to convert are not drawn BatchRowType BatchCellType BatchQualFlagType BatchAmbiType Figure 6 2 Schematic representation of the batch data structure BatchType Attribute Type Description NrRows Integer Number of rows in batch Row BatchRowType Array of rows BatchRowType Attribute Type Description RowNr Integer Row number within orbit 51 Doc ID Version Date NWPSAF KN UD 005 2 0 01 November 2010 NWP SAF AWDP User Manual and Reference Guide NrCells Integer Number of cells in batch max 76 Cell BatchCellType Array of cells within row BatchCellType Attribute Type Description NodeNr Integer Node number within orbit row lat Real Latitude lon Real Longitude ubg Real u component of background wind vbg Real v component of background wind NrAmbiguities Integer Number of ambiguities Ambi BatchAmbiType Array of ambiguities BatchAmbiType Attribute Type Description selection Integer Index of selected ambiguity uana Real u component of analysis wind vana Real v component of analysis wind f Real Contribution of this cell to cost function gu Real Derivative of f to u gv Real Derivative of f to v qualflag BatchQualFlagType Quality control flag Table 6 3 Batch data structures To check the quality of the batch a quality flag is introduced for instances of the BatchCellType The f
64. eant to form the key component of the observation operator for surface winds in data assimilation systems The general scheme of AWDP and any other wind scatterometer data processor is given in figure 2 1 The input of the AWDP program is the EUMETSAT ASCAT level 1b BUFR or PFS wind product combined with ASCAT level 1 full resolution data in PFS in the case of coastal wind product processing or the ESA ERS level 2 BUFR wind product Besides this GRIB input containing land sea mask sea surface temperature or soil temperature on level 0 and first guess winds over the globe is necessary The AWDP processing chain contains several steps see figure 2 1 1 Pre processing The input file is decoded and the radar backscatter values are written in the data structures of AWDP In the case of coastal product processing the full resolution SZF backscatter data are read and averaged Some quality control on the input data is done 2 Collocation with NWP data The GRIB edition 1 or 2 files containing NWP data are read and the values for land fraction sea surface temperature and first guess winds are interpolated and stored with the information of each WVC 3 Inversion The o values are compared to the Geophysical Model Function GMF by means of a Maximum Likelihood Estimator MLE The wind vectors that give the best description of the o values the solutions are retained The MLE is also used to assign a probability to each AWDP Us
65. ection 0 indicator ksec1 40 integer Expanded section 1 identification ksec2 4096 integer Expanded section 2 optional ksec3 4 integer Expanded section 3 data description ksec4 2 integer Expanded section 4 data Table 8 3 Attributes for the BufrSectionsType data type Attribute Type Description NsecO integer ksup 9 dimension section 0 nsecOsize integer 0 1 size section 0 nBufrLength integer 0 2 length BUFR nBufrEditionNumber integer 0 3 Nsecl integer ksup 1 dimension section 1 nseclsize integer ksecl 1 size section 1 kEditionNumber integer ksecl 2 Kcenter integer ksecl 3 kUpdateNumber integer ksecl 4 kOptional integer ksecl 5 ktype integer ksecl 6 ksubtype integer 1 7 local use kLocalVersion integer ksecl 8 kyear integer ksecl 9 century year kmonth integer 1 10 integer 1 11 khour integer ksecl 12 kminute integer 1 13 kMasterTableNumber integer ksecl 14 65 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Attribute Type Description kMasterTableVersion integer ksecl 15 ksubcenter integer 1 16 klocalinfo integer ksec1 17 40 Nsec2 integer ksup 2 dimension section 2 nsec2 size integer ksec2 1 size section 2 key 46 integer ksec2 2 key Nsec3 integer ksup 3 dimension section 3 nsec3size integer ksec3
66. ee eee ene aee seen ee eee seen Pese eese ee seen aee eaa 63 8 1 BAGKGROUND eie ee tee ec ed e aie 63 8 2 ROUTINES 5 eee a ehe teat ee eo caet 63 8 3 DATA STRUCTURES imet DNE RE 65 8 4 LIBRARIES trn utere teris eed ipee es dee 66 8 5 BUFR TABLE ROUTINES ccccccssssceceessececsecsececsnececesaececsesaececseeeeceesaececsesaeeeceeeeeessaeeecsesaeeesseeeeees 67 8 6 CENTRE SPECIFIC MODULEBS Gto feet ce rece Fe teras dee d e ee be 67 CHAPTER9 MODULE GRIBIO MODULE eee ee eee ee eee eee seen Peste en etae eese enne essa 68 9 1 BACKGROUND aes iti ond ee D at n E RE ae een 68 9 2 OUI EEE 68 9 3 DATA STRUGTURES tette et ette EI t eto nie vasti ue eue te chee 70 9 4 LIBRARIES ee 71 REFERENCES nrnna eres ciitavessasepecsesesevosesdecdosvsucedeosscessoussucesessasdaveveeeusescsesbuckcsuessasesvenecsceecdsdeseesoes 72 APPENDIX A CALLING TREE FOR AWDDB eee ee eren eee eene eee tn oeste toss eee tn ases seen esee 74 NWP SAF AWDP User Manual and Doc ID NWPSAF KN UD 005 APPENDIX B1 APPENDIX B2 APPENDIX B3 APPENDIX B4 APPENDIX B5 APPENDIX B6 APPENDIX C APPENDIX D Version 2 0 01 Reference Guide Date November 2010 CALLING TREE FOR INVERSION ROUTINES
67. ee table 4 13 The routine init wind sets the wind vector to missing The routine print wind may be used to print the wind vector The routine test wind tests the validity of the wind specification see also the cell process flag 37 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Attribute Description speed real Wind speed dir real Wind direction Table 4 13 Wind data structure Some special data types are introduced for the data quality flags These are discussed below Beam collocation flag The beam collocation type data type is used to indicate whether data of the three beams is originating from a single ground station or from multiple ground stations collocated data This is relevant for so called direct readout data from different ground stations which maybe merged into one single product In a WVC e g the fore beam information from one ground station may be combined with the mid and aft beam information from another ground station in order to make a complete WVC The attributes are listed in table 4 14 The routine get_beam_collocation converts an integer value to the logical beam collocation structure The routine set_beam_collocation converts a logical beam collocation structure to an integer value Attribute Bit 28 Description missing Flag not set all bits on collocation 0 1 Beam information originates from differen
68. egion Each time the MetOp satellite passes over the pole region the corresponding ice map is updated with the new ASCAT data A spatial and temporal averaging is performed in order to digest the new information After the overpass at the end of processing an entire BUFR file the updated information on the ice map is put back into the BUFR structure Optionally graphical maps are plotted which can be controlled by optional input parameters for routine printIceMap The graphical filenames have encoded the North Pole South Pole the date time as well as the parameter name The most important ones are print file N S yyy ymmddhhmms s ppm contains the ice subclass and the a ice parameter on a grey scale for points classified as ice print t file NI S yyy yrnmddhhmms s t ppm contains the ice class print sst file NIS yyyymmddhhmmss sst ppm contains the sea surface temparature print postprob file N S yyyymmddhhmms s postprob ppm contains the a posteriori ice probability Typically at least two days of ASCAT data are needed to entirely fill the ice map with data and give meaningful ice model output Because AWDP handles only one BUFR file at a time a script is needed that calls AWDP several times After each AWDP run a binary restart file is written to disk containing the information of an icemap latestIceMapN rst for the North Pole and latestIceMapS rst for the South Pole With the next call of awdp these res
69. ejected WVCs The blue and violet arrows are a 6 hours forecast from the KNMI HIRLAM model DocID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Version 2 0 01 Reference Guide Date November 2010 2 2 Modes of using AWDP There are several modes to assimilate the ASCAT data in NWP models using AWDP Anyway the first thing to assure oneself of is the absence of biases by making scatter plots between ASCAT and NWP model first guess for at least wind speed but wind direction and wind components would also be of interest to guarantee consistency The operational ASCAT wind product available as a deliverable from the EUMETSAT OSI SAF project could be the starting point for NWP assimilation 1 The unique solution at every WVC may be assimilated as if it were buoy data This is the fastest way and one exploits the data to a large extent For a small advantage AWDP could be installed to provide 2DVAR solutions based on the local first guess 2 The AWDP software may be used to modify the 3DVAR or 4DVAR data assimilation system to work with the ambiguous wind solutions and their probabilities at every WVC This represents some investment but is applicable for all scatterometer data The advantage with respect to option in the ambiguity removal occurs only occasionally but always in dynamic atmospheric cases storms or cyclones that are really relevant Both options can be based on AWDP in standard or MSS mode and at various resoluti
70. en 0 and 1 The land fraction of the WVC is calculated as the average of the grid land fractions where each grid land fraction has a weight of 1 7 r being the distance between WVC centre and the model grid point Routine get colloc from GRIB filelist returns an interpolated value four surrounding grid points from the GRIB data in the list of files messages for a given GRIB parameter latitude longitude and time The list of messages must contain a sequence of forecasts e g 3 hrs 6 hrs 9 hrs et cetera At least three forecasts need to be provided ideally two lying before the sensing time and one after 69 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 In this diagram the 1 2 and 3 mean the three forecast steps with intervals of three hours between them The is the sensing time The software will perform a cubic time interpolation Note that the 1 2 and 3 in the diagram may correspond to 3 6 and 9 forecasts but also e g to 9 12 and 15 If more forecasts are provided e g like this 1 2 4 5 the software will use forecast steps 2 3 and 4 i e it will pick the most usable values by itself If one forecast before and two after are provided N e 2 3 the software will still work and use all three forecasts Checking and Printing The integer parameter GribVerbosity controls the extent of the log
71. endent data representation system but it contains binary data so care must be taken in reading and writing these data under different operating systems A BUFR message record contains observational data of any sort in a self descriptive manner The description includes the parameter identification and its unit decimal and scaling specifications The actual data are in binary code The meta data are stored in 24 DocID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Version 2 0 01 Reference Guide Date November 2010 BUFR tables These tables are therefore essential to decode and encode the data BUFR tables are issued by the various meteorological centres The largest part of the data descriptors specified in the BUFR tables follows the official BUFR descriptor standards maintained by the World Meteorological Organization WMO http www wmo int However for their different observational products meteorological centres do locally introduce additional descriptors in their BUFR tables Appendix C contains a listing of the data descriptors of the BUFR data input and the BUFR data output of the AWDP program in the ASCAT BUFR product format For more details on BUFR the reader is referred to Dragosavac 1994 ECMWF maintains a library of routines for reading writing and decoding encoding the binary BUFR messages This library forms the basis of the genscat BUFR module and hence the AWDP program BUFR interface see Chapte
72. ents the measurement by MLE Poe 5 2 Therefore wind vectors with low MLE have a high probability of being the correct solution On the other hand wind vectors with high MLE are not likely represented by any point on the GMF Details on the inversion problem can be found in Stoffelen and Portabella 2006 Portabella 2002 The AWDP program includes the Multiple Solution Scheme MSS see Portabella and Stoffelen 2001 46 NWP SAF AWDP User Manual and Reference Guide Doc ID Version Date 2 0 01 NWPSAF KN UD 005 November 2010 5 2 Routines The inversion module class contains two modules named inversion and post inversion They are located in subdirectory genscat inversion Tables 5 1 and 5 2 list all routines in the modules Appendix B 1 shows the calling tree for the inversion routines Routine Call Routine Call invert one wvc AWDP INTERPOLATE generic fill wind quality code invert one wvc interpolateld calc sigma0 save inv input not used interpolated2d calc sigma0 read inv input not used interpolate2dv calc sigma0 save inv output not used interpolate3d calc sigma0 do parabolic winddir search one wvc read LUT calc sigma0 calc normalisation calc sign MLE print message init inv input init inv output init inv settings to default write inv settings to file get inv settings set inv settings check input data find minimum cone dist get parabolic minimum cal
73. er 2010 these scripts The scripts are available in Bourne shell extension sh and in C shell extension csh Note that if one of the environment variables is not set the default 90 and cc commands on the Unix platform will be invoked Note that in the top directory a script called InstallAWDP is provided that asks the user which compiler he wants to use and invokes the appropriate use script step 1 above after which the compilation in the genscat and awdp directories is performed steps 2 and 3 above Variable Function SGENSCAT F77 Reference to Fortran 77 compiler SGENSCAT F90 Reference to Fortran 90 compiler S GENSCAT CC Reference to C compiler SGENSCAT LINK Reference to linker for Fortran objects SGENSCAT CLINK Reference to linker for C objects SGENSCAT SHLINK Reference to linker for shared objects Table 2 4 Environment variables for compilation and linking Script Fortran C Remarks compiler compiler use g95 g95 gcc GNU compilers by A Vaught use_gfortran gfortran gcc GNU GCC 4 0 compiler collection use_ifort ifort ice Intel Fortran and C compilers use_pgf90 pgf90 gcc Portland Fortran compiler Table 2 5 Properties of the use_ scripts Example To select the GNU g95 compiler under Bourne Bash or Korn shell type use_g95 bsh the dot being absolutely necessary in order to apply the compiler selection to the current shell Under C shell the equivalent command reads source use g95 csh
74. er Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 wind vector The normal scheme allows 2 solutions at most but in the Multiple Solution Scheme MSS the maximum number of solutions is 144 4 Quality Control Solutions that lie far away from the GMF are likely to be contaminated by e g sea ice or confused sea state During Quality Control these solutions are identified and flagged 5 Ambiguity Removal This procedure identifies the most probable solution using some form of external information AWDP uses a two dimensional variational scheme 2DVAR as default A cost function is minimized that consists of a background wind field and all solutions with their probability using meteorological balance mass conservation and continuity as constraints 6 Quality Monitoring The last step is to output quality indicators to an ASCII monitoring file and to write the results in a BUFR format output file Figure 2 1 AWDP processing scheme The wind vectors and their probabilities after Quality Control may be fed directly in the Data Assimilation step of a Numerical Weather Prediction model Step 1 2 and 6 of the processing chain are rather trivial the real work is done in steps 3 4 and 5 Note that an undesirable dependency arises if the output wind field is assimilated into a numerical weather prediction NWP model in the 2DVAR Ambiguity Removal step 1 a background wind field
75. ersion of the BUFR library one should be sure to enter make clean first To recompile part of the software invoke the make system where needed The compiler settings from file Makeoptions in directory genscat will be used again If a change in these settings is necessary type make clean in the genscat directory and Makeoptions will be removed Don t forget to rerun the use commands to select the right compiler 2 3 6 Some remarks for Cygwin users AWDP can be used under Cygwin a Unix emulator running under Windows Installing and running AWDP under Cygwin is almost the same as under Unix or Linux but the following points may be helpful for Cygwin users e The GNU g95 compiler comes standard with Cygwin version 1 5 25 11 and later so it is always possible to install AWDP using g95 e Cygwin has its own path naming convention for example C awdp under Windows becomes cygdrive c awdp under Cygwin e Don t forget to run the dos2unix command on scripts edited under Windows otherwise Cygwin won t recognize the file as a script 2 4 Command line options The AWDP program is started from directory awdp execs with the command awdp options modes f lt BUFR PFS file gt nwpfl lt file gt with lt gt indicating obligatory input and indicating non obligatory input The following command line options are available 15 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010
76. etonFFT2D 89 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Appendix B3 Calling tree for BUFR routines The figures in this appendix show the calling tree for the BUFR file handling routines in genscat Routines in black boxes are part of genscat Routines in grey boxes with names completely in capitals belong to the ECMWF BUFR library Other routines in grey boxes belong to the bufrio library in C An arrow before a routine name indicates that this part of the calling tree is a continuation of a branch in a previous figure The same arrow after a routine name indicates that this branch will be continued in a following figure open BUFR file bufr error bufr split Figure B3 1 Calling tree for BUFR file handling routine open BUFR file close BUFR file bufr close bufr error Figure B3 2 Calling tree for BUFR handling routine close BUFR file 90 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 get_BUFR_message get_expected_BUFR_msg_size bufr_read_allsections bufr_error bufr_get_section_sizes bufr_swap_allsections ExpandBufrMessage PrintBufrErrorCode CheckBufrTables get_file_size encode_table_b encode table d BUFREX FillBufrSecData BUSEL Figure B3 3 Calling tree for BUFR handling routine get_BUFR_message save_BUFR_mes
77. evel 1b Normalized Radar Cross Section data oo This work is coordinated and supervised by Ad Stoffelen In the beginning only an adaptation of his ERS code existed Later Marcos Portabella and Julia Figa added modifications and extensions to improve e g the wind retrieval and quality control algorithms In 2003 John de Vries finished the first official release of a processor within the NWP SAF This processor was called the QuikSCAT Data Processor QDP Meanwhile Jos de Kloe has been updating the code for ERS scatterometer wind processing For many parts of the process steps e g the BUFR handling and part of the wind retrieval a large overlap with SeaWinds Data processing coding exists The KNMI Scatterometer Team is working towards generic NRT scatterometer processing As a result a new modular processing code for SeaWinds data was developed within the NWP SAF the SeaWinds Data Processor SDP as successor of QDP Based on the generic code already available for SeaWinds and ERS processing a new ASCAT Wind Data Processor AWDP was developed This document is the corresponding reference manual I hope this manual will strongly contribute to the comprehension of future developers and of users interested in the details of the processing Many persons contributed directly or indirectly to the development of the scatterometer software at KNMI Hans Bonekamp Jos de Kloe Marcos Portabella Ad Stoffelen Anton Verhoef Jeroen Verspeek Jur
78. factor Table 6 15 Fourier transform routines Table 6 15 gives an overview of the available routines The figures in Appendix B2 shows the calling tree of the FT routines relevant for 2DVAR Remark the 2DVAR implementation can be made more efficient by using a real to real FFT routine rather than a complex to complex one as implemented now Since AWDP satisfies the requirements in terms of computational speed this has low priority 6 5 The PreScat scheme The PreScat ambiguity removal scheme can be invoked within AWDP by the use of command line option armeth prescat More information on this scheme can be found in Stoffelen de Haan Quilfen and Schyberg 2000 Currently the PreScat scheme can be used only in combination with ERS data 58 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Chapter 7 Module iceModelMod Module iceModelMod is part of the genscat support modules It contains all the Bayesian statistics routines including the routines for spatial and temporal averaging It also contains all the routines for initialising and printing of the SSM I grids for the North Pole and South Pole region 7 1 Background The icemodel option in AWDP basically fills the fields Ice Probability BUFR item 87 and Ice Age BUFR item 88 Also it can output graphical maps of ice model related parameters on an SSM I grid for the North Pole and for the South Pole r
79. get ers noise estimate Figure B1 4 Calling tree for inversion routine normalize conedist prescat mode normalise conedist ers ascat check ers ascat inversion data calc kp ers ascat calc_geoph_noise_ers_ascat Figure B1 5 Calling tree for inversion routine normalize_conedist_ers_ascat check wind solutions ers ascat remove one solution calc dist to cone center calc sigma0 Figure B1 6 Calling tree for inversion routine check wind solutions ers ascat 85 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 Appendix B2 Calling tree for AR routines The figures in this appendix show the calling tree for the Ambiguity Removal routines in genscat All routines are part of genscat as indicated by the black boxes An arrow before a routine name indicates that this part of the calling tree is a continuation of a branch in a previous figure The same arrow after a routine name indicates that this branch will be continued in a following figure InitTwodvarModule TDV Init Set_HelmholzCoefficients Figure B2 1 Calling tree for AR routine Init wodvarModule 86 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 DoAmbrem TestBatch TestBatchRow TestBatchCell AmbRem1stRank DoAmbremBGclosestOnBatch uv to dir DoAmbremPreScatOnBatch DoAmb
80. he observational term Jo s and the background term J The solution the analysis surface wind field may be denoted as v Being just a weighted least squares term the background term may be further specified as Jp O v T B I vy 6 2 where B is the background error covariance matrix The Joss term of the 2DVAR scheme is not simply a weighted least squares term Such a formulation does not closely match the code of the 2DVAR scheme In fact for scientific and technical reasons several transformations are applied to reformulate the minimization problem Description of these transformations is essential to understand the different procedures within the code The interested reader is referred to Vogelzang 2007 6 4 4 Module CostFunction Module CostFunction contains the main procedure for the calculation of the cost function and its gradient It also contains the minimization procedure Table 6 12 provides an overview of the routines Routine Call Description Jt Minimise Total cost function and gradient Jb Jt Background term of cost function Jo Jt Observational term of cost function JoScat Jo Single observation contribution to the cost function Unpack_ControlVector Jo Unpack of control vector Pack_ControlVector Jo Pack of control vector or its gradient Uncondition Jo Several transformations of control vector Uncondition_adj Jo Adjoint of Uncondition Minimise Do2DVAR TwoDvar Minimization DumpAnalysisFiel
81. heir acquisition date and time or the wind vector solutions according to their probability 33 Doc ID Version Date NWPSAF KN UD 005 2 0 01 November 2010 AWDP User Manual and Reference Guide NWP SAF 4 3 Module design for process layer The process layer consists of the modules awdp data awdp_bufr awdp_pfs awdp prepost awdp grib awdp inversion awdp_icemodel and ambrem The routines present in these modules are described in the next sections 4 3 1 Module awdp_data The module awdp_data contains all the important data types relevant for the processing Elementary data types are introduced for the most basic data structures of the processing These are e g wind_type and time_type Using these data types and of course the standard types as integer real etc more complex composed data types are derived Examples are beam_type ambiguity_type cell_type and row_type A complete description of all types is given below The attributes of all these types have intentionally self documenting names Ambiguity data The ambiguity_type data type contains information on an individual ambiguity wind vector solution The attributes are listed in table 4 5 The routine init_ambiguity sets all ambiguity data to missing The routine print_ambiguity may be used to print all ambiguity data Attribute Type Description wind wind type Wind vector solution prob real Probability of wind vector solution cone
82. idence Angle dB Degree 70 021153 Estimated Error In Slope At 40 Deg Incidence Angle dB Degree 71 021154 Soil Moisture Sensitivity dB 72 021062 Backscatter dB 73 021088 Wet Backscatter dB 74 040003 Mean Surface Soil Moisture Numeric 75 040004 Rain Fall Detection Numeric 76 040005 Soil Moisture Correction Flag Flag Table 77 040006 Soil Moisture Processing Flag Flag Table 78 040007 Soil Moisture Quality 79 020065 Snow Cover 80 040008 Frozen Land Surface Fraction 81 040009 Inundation And Wetland Fraction 82 040010 Topographic Complexity 83 025060 Software Identification Numeric 84 001032 Generating Application Code Table 85 011082 Model Wind Speed At 10 m m s 86 011081 Model Wind Direction At 10 m Degree True 87 020095 Ice Probability Numeric 88 020096 Ice Age A Parameter dB 89 021155 Wind Vector Cell Quality Flag Table 90 021101 Number Of Vector Ambiguities Numeric 91 021102 Index Of Selected Wind Vector Numeric 92 031001 Delayed Descriptor Replication Factor Numeric 93 011012 Wind Speed At 10 m m s 94 011011 Wind Direction At 10 m Degree True 95 021156 Backscatter Distance Numeric 100 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Number Descriptor Parameter Unit 96 021104 Likelihood Computed For Solution Numeric 97 011012 Wind Speed At 10 m m s 98 011011 Wind Direction At 10 m Degree True 99 021156 Backscatter Distance Numeric 100 021104 Like
83. if the mon command line option is set Parameter Description observation Number of Wind Vector Cells in output land Fraction of WVCs with land flag set ice Fraction of WVCS with 1ce flag set background Fraction of WVCs containing model winds backscatter info Fraction of WVCs containing sufficient valid for inversion N2 42 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Parameter Description knmi flag Ratio number of WVCs with KNMI QC flag set N2 wind retrieval Fraction of N2 that actually contains wind solutions N3 wind selection Fraction of N3 that actually contains a wind selection N4 big mle Number of WVCs containing a wind solution but no MLE value avg mle Averaged over N4 MLE value of 1 wind selection var qc Fraction of N4 that has the Variational QC flag set rank 1 skill Fraction of N4 where the first wind solution is the chosen one avg wspd diff Averaged over N4 difference between observed and model wind speeds rms diff wspd RMS over N4 difference between observed and model wind speeds wspd ge 4 Fraction of N4 where the selected wind speed is gt 4 m s 5 rms diff dir RMS over N5 difference between observed and model wind directions rms diff u RMS over N5 difference between observed and model wind u components rms diff v RMS over N5 difference between observed and model wind v components rms diff vec len
84. in the awdp tests directory To do this first create a small file containing a list of NWP files ls 1 ECMWF 200704260000 0 nwpflist Then run AWDP on 25 km and 12 5 km cell spacing execs awdp run f ascat 20070426 test 250 11 bufr mon calval nwpfl nwpflist execs awdp run f ascat 20070426 test 125 11 bufr mon calval nwpfl nwpflist The result should be two ASCAT level 2 files in BUFR format called ascat 20070426 095102 metopa 02681 srv o 250 ovw 12 bufr and ascat 20070426 095100 metopa 02681 srv o 125 ovw 12 bufr respectively Figure 2 3 shows the global coverage of the test run on 25 km The colours indicate the magnitude of the wind speed as indicated by the legend The result on 12 5 km should be very similar to this Directory awdp tests also contains an ERS BUFR file in ESA format called scatt_20070426_test_250 11 bufr in ESA BUFR format The data are from the same date as the ASCAT data in this directory and they can be processed using the same ECMWF files execs awdp run f scatt 20070426 test 250 11 bufr mon nwpfl nwpflist The result should be an output file in ASCAT BUFR format called scatt 20070426 063627 ers2 00000 srv o 250 ovw 12 bufr 20 DocID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Version 2 0 01 Reference Guide Date November 2010 0 3 6 9 12 15
85. ines are part of the modules themselves Test programs can be compiled separately For the AWDP program the description of the test programs and the results of the testing are reported in Verhoef et al 2008 1 4 User Manual and Reference Guide This document is intended as the complete reference book for AWDP Chapter 2 is the user manual UM for the AWDP program This chapter provides the basic information for installing compiling and running AWDP Chapter 3 contains the Product Specification PS of the AWDP program Reading the UM and the PS should provide sufficient information to the user who wants to apply the AWDP program as a black box The subsequent chapters are of interest to developers and users who need more specific information on how the processing is done The Top Level Design TLD of the code and the Module Design MD of the AWDP code can be found in Chapter 4 Several modules are very generic for NRT scatterometer data processing Examples are the modules for the BUFR and GRIB handling ambiguity removal and parts of the wind retrieval These generic modules are part of the generic scatterometer genscat layer and are described in Chapter 5 to Chapter 9 The appendices of this document contain a complete calling tree of the AWDP program up to and including the genscat layer The appendices also contain a list of ASCAT BUFR data descriptors and a list of acronyms 1 5 Conventions Names of physical quantities e g
86. input data Post processing consists of the procedure between the ambiguity removal step and the BUFR 41 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 encoding of the output The post processing includes the monitoring of the wind data and the setting of some of the flags in the output product Routine Call Description calibrate_sO preprocess Apply oo calibration during the cal val period postprocess merge rows preprocess Merge the data of two input rows monitoring postprocess Monitoring postprocess AWDP Main routine of the post processing pre inversion qc preprocess Perform quality checks on input data preprocess AWDP Main routine of the pre processing process cleanup AWDP Memory management write binary output postprocess Write WVC data to a binary output file write properties postprocess Write some properties of the data into a text file Table 4 23 Routines of module prepost Table 4 23 lists the tasks of the individual routines AWDP calls preprocess to sort the rows with respect to the acquisition data and time It also checks on the appearance of double rows that 1s rows which are less than half the nominal cell distance pixel size on horizontal in the input data apart If preprocess finds a double row it merges the two rows into one row In that case the number of input rows will be reduced Once the input rows are sorted and
87. inversion processing step Chapter 5 the Ambiguity Removal step Chapter 6 the BUFR file handling Chapter 8 and the GRIB file handling Chapter 9 The genscat modules are located in subdirectory genscat In addition genscat contains a large support class to convert and transform meteorological geographical and time data to handle file access and error messages sorting and to perform more complex numerical calculations on minimization and Fourier transformation Many routines are co developed for ERS ASCAT and SeaWinds data processing Module class Tasks Description Ambrem Ambiguity Removal 2DVAR and other schemes see Chapter 6 Inversion Wind retrieval Inversion in one cell see Chapter 5 IceModel Ice screening Uses ice line and wind cone for ice discremination Support BUFR support BufrMod based on ECMWF library PFS support Reading of PFS files GRIB support gribio module based on ECMWF library FFT minimization Support for 2DVAR Error handling Print error messages File handling Finding opening and closing free file units Conversion Conversion of meteorological quantities Sorting Sorting of ambiguities to their probability Date and time General purpose Table 4 2 genscat module classes 4 1 3 Data Structure Along track the ASCAT swath is divided into rows Within a row across track the ASCAT orbit is divided into cells also called Wind Vector Cells WVCs or nodes This division in rows and cells forms the basis
88. io module uses two libraries from the GRIB API software library of ECMWF libgrib api a and libgrib api f90 a The GRIB API software library of ECMWF is used as a basis to decode GRIB data This software library is explained on http www ecmwf int 71 DocID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Version 2 0 01 Reference Guide Date November 2010 References e Dragosavac M 1994 BUFR User Guide and Reference Manual ECMWF Available on http www ecmwf int e Figa Saldafia J and Wilson J J W 2005 ASCAT Level 1 Product Format Specification Issue 6 Rev 5 EUMETSAT EPS MIS SPE 97233 Available on http www eumetsat int e Giering R 1997 Tangent linear and Adjoint Model Compiler Users manual Max Planck Institut fuer Meteorologie e Hersbach H Stoffelen A and de Haan S 2007 An improved C band scatterometer ocean geophysical model function CMODS Journal of Geophysical Research 112 e Liu D C and Nocedal J 1989 On the limited memory BFGS method for large scale optimization methods Mathematical Programming 45 pp 503 528 e UK Met Office 2001 ERS Products WMO FM94 BUFR Format ER IS UKM GS 0001 Version 4 Issue 2 e Portabella M 2002 Wind field retrieval from satellite radar systems PhD thesis University of Barcelona Available on http www knmi nl scatterometer publications e Portabella M and Stoffelen A 2001 Rain Detection and Quali
89. iption azimuth 21 2097152 Poor azimuth diversity among oo kp 20 1048576 Any beam noise content above threshold monflag 19 524288 Product monitoring not used monvalue 18 262144 Product monitoring flag knmi qc 17 131072 KNMI quality control fails var qc 16 65536 Variational quality control fails land 15 32768 Some portion of wind vector cell is over land ice 14 16384 Some portion of wind vector cell is over ice inversion 13 8192 Wind inversion not successful large 12 4096 Reported wind speed is greater than 30 m s small 11 2048 Reported wind speed is less than or equal to 3 m s rain fail 10 1024 Rain flag not calculated rain_detect 9 512 Rain detected no background 8 256 No meteorological background used redundant 7 128 Data are redundant gmf distance 6 64 Distance to GMF too large Table 4 16 Wind Vector Cell quality flag bits Fortran Cell process flag Besides a cell quality flag every WVC contains a process flag The process flag checks on aspects that are important for a proper processing but are not available as a check in the cell quality flag The cell process flag is set by the routine test cell which calls routines test fime test beam and test wind Table 4 17 lists the attributes of the process flag type The process flag is only available internally in AWDP The routine print process flag may be used to print the bit values of the flag Attribute Description satellite id Invalid satellite id sat instruments
90. is used and ii meteorological balance constraints causing spatially correlated error Therefore it is recommended to feed the wind solutions and their probabilities directly into the NWP data assimilation step after Quality Control as indicated in figure 2 1 If this is done the Ambiguity Removal step can be skipped and consequently no forecast winds are necessary in the NWP input No impact tests have been performed to date by assimilating AWDP outputs after ambiguity removal As further detailed in Chapter 3 AWDP profits from developments in Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Version 2 0 01 Reference Guide Date November 2010 e Inversion and output of the full probability density function of the vector wind Multiple Solution Scheme MSS e Quality Control QC e Meteorologically balanced Ambiguity Removal 2DVAR e Quality monitoring e Capability to process ASCAT data on both 25 km and 12 5 km cell spacing A complete specification of the AWDP program can be found in the Product Specification in Chapter 3 The program is based on generic genscat routines for inversion ambiguity removal and BUFR and GRIB file handling These routines are discussed in more detail in Chapter 5 to Chapter 9 EUMETSAT KNMI Figure 2 2 AWDP wind field retrieved for 15 April 2007 approximately 13 h UTC at 25 km cell spacing near Newfoundland overlaid on a GOES IR satellite image The yellow dots are r
91. l Weight upper right ubg Real Background EW wind component vbg Real Background NS wind component NrAmbiguities Integer Number of ambiguities incr AmbilncrType Ambiguity increments uAnalncr Real Analysis increment vAnalncr Real Analysis increment selection Integer Selection flag QualFlag TwoDvarQualFlagType Quality control flag f Real Cost function at observation gu Real gv Real df dv Table 6 8 The Obs2dvarType data structure 54 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Attribute Type Description delta Real 2DVAR grid size in position domain delta p Real 2DVAR grid size in frequency domain delta q Real 2DVAR grid size in frequency domain NI Integer Dimension 1 of 2DVAR grid HI Integer NI 2 Integer H1 1 number of nonnegative frequencies N2 Integer Dimension 2 of 2DVAR grid H2 Integer N2 2 K2 Integer H2 1 number of nonnegative frequencies Ncontrol Integer Size of control vector Table 6 9 The TDV Type data structure Routine Call Description TDV Init InitTwodvarModule Initialization of 2DVAR grid and preparations Set HelmholzCoefficients Init Set Helmholz transformation coefficients Set CFW TDV Init Set cost function weights TDV Exit ExitTwodvarmodule Deallocate memory InitObs2dvar BatchInput2DVAR Allocation of observations array BatchOutput2DVAR DeallocObs2dvar BatchOutput2DVAR Deallocation of ob
92. lag is set by routine TestBatchCell The attributes of this flag of type BatchQualFlagType are listed in table 6 4 Module BatchMod contains a number of routines to control the batch structure The calls and tasks of the various routines are listed in table 6 5 The batch structure is allocatable because it is only active between the wind retrieval and the ambiguity removal step Attribute Description Missing Quality flag not set Node Incorrect node number specification Lat Incorrect latitude specification Lon Incorrect longitude specification Ambiguities Invalid ambiguities Selection Invalid selection indicator Background Incorrect background wind specification Analysis Incorrect analysis Threshold Threshold overflow Cost Invalid cost function value Gradient Invalid gradient value Table 6 4 Batch quality flag attributes Routine Call Description AllocRowsAndCellsAndInitBatch AllocAndInitBatchRow AllocAndInitBatchCell Processor AllocRowsAndCellsAndInitBatch AllocAndInitBatchRow Allocation of batch Allocation of batch rows Allocation of batch cells 52 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Version 2 0 01 Reference Guide Date November 2010 Routine Call Description AllocRowsOnlyAndInitBatch not used InitBatchModule Ambrem Initialization module InitBatch InitBatchRow InitBatchCell InitbatchAmbi AllocRowsAndCellsAndInitBatch InitBatch InitBatchR
93. le 6 14 Routines in module BFGSMod Some of the error returns of the line search routine MCSRCH have been relaxed and are treated as a normal return Further details can be found in the comment in the code itself 57 AWDP User Manual and Doc ID NWPSAF KN UD 005 Version 2 0 01 NWP SAF Reference Guide Date November 2010 Routines daxpy and ddot were rewritten in Fortran 90 These routines originally written by J Dongarra for the Linpack library perform simple operations but are highly optimized using loop unrolling Routine ddot for instance is faster than the equivalent Fortran 90 intrinsic function dot product 6 4 8 SingletonFFT Module Module SingletonFFT Module in directory genscat support singletonfft contains the multi variate complex Fourier routines needed in the 2DVAR scheme A mixed radix Fast Fourier Transform algorithm based on the work of R C Singleton is implemented Routine Call Description SingletonFFT2d SetCovMat Uncondition 2D Fourier transform Uncondition adj fft SingletonFFT2d Main FFT routine SFT Permute Sit Permute the results SFT_PermuteSinglevariate SFT Permute Support routine SFT PermuteMultivariate SFT Permute Support routine SFT PrimeFactors Get the factors making up N SFT Base2 2 SFT Base3 3 SFT Base4 Base 4 FFT SFT Base5 5 SFT BaseOdd fft General odd base FFT SFT_Rotate fft Apply rotation
94. lid wind solution are used Logical controlling wheter sea surface temperature is used in the ice model algorithm Matrix given the weight factors for the spatial averaging Defines the decay time hours of the temporal weighting function Defined the cutoff time hours for the temporal weighting function This is the ice probability when no a priori information is available This is the odds on ice related to pClimatelce Class no data Class sea wind Class ice Class sea or ice indecisive Class unknown outlier SubClass sea SubClass sea ice weight lt weightSTLimit SubClass probably ice SD a gt aSdLimit SubClass ice SubClass unknown outlier SubClass no data Number of historical measurement that are stored 2 Upper limit for the standard deviation of the a iceparameter Lower limit of p ice X for classifying a pixel as ice Lower limit for the total weight of all measurements involved in the temperal and spatial averaging Upper limit for the distance to ice line Upper limit for the distance to wind cone Lower limit for sea surface temperature K Below this limit pixels are classified as ice Upper limit for sea surface temperature K Above this limit pixels are classified as sea wind Table 7 6 Parameters in the Bayesian statistics 62 DocID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Version 2 0 01 Reference Guide Date November 2010 Chapter Module BufrMod
95. lihood Computed For Solution Numeric Table C 1 List of data descriptors Note that descriptor numbers 93 96 can be repeated to 144 times depending on the value of the Delayed Descriptor Replication Factor descriptor number 92 101 NWP SAF DocID NWPSAF KN UD 005 Version 2 0 01 November 2010 AWDP User Manual and Reference Guide Date Appendix D Acronyms Name Description AMI Active Microwave Instrument scatterometer on ERS 1 and ERS 2 satellites AR Ambiguity Removal ASCAT Advanced SCATterometer on MetOp BUFR Binary Universal Form for the Representation of data C band Radar wavelength at about 5 cm ERS European Remote Sensing satellites ECMWF European Centre for Medium range Weather Forecasts EUMETSAT European Organization for the Exploitation of Meteorological Satellites genscat generic scatterometer software routines GMF Geophysical model function HIRLAM High resolution Local Area Model KNMI Koninklijk Nederlands Meteorologisch Instituut Royal Netherlands Meteorological Institute Ku band Radar wavelength at about 2 cm Llb Level 1b product LSM Land Sea Mask LUT Look up table MetOp Meteorological Operational Satellite MLE Maximum Likelihood Estimator MSS Multiple Solution Scheme NRCS Normalized Radar Cross Section NWP Numerical Weather Prediction OSI Ocean and Sea Ice PFS Product Format Specification native MetOp file format QC Quality Control RFSCAT Rotating Fan beam Sc
96. lse or writing writemode true Routine set BUFR fileattributes determines several aspects of the BUFR file and saves these data in an instance of bufr file attr data see table 8 5 Routine get BUFR nr of messages is used to determine the number of BUFR messages in the file Finally routine close BUFR file closes the BUFR file 64 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 As said before the underlying encoding and decoding routines originate from the ECMWF BUFR library Appendix B3 shows the calling trees of the routines in module BufrMod that are used in AWDP 8 3 Data structures The data type closest to the actual BUFR messages in the BUFR files is the BufrMessageType see table 8 2 These are still encoded data Every BUFR message consists of 5 sections and one supplementary section After decoding expanding the BUFR messages the data are transferred into an instance of BufrSectionsType see table 8 3 which contains the data and meta data in integer values subdivided in these sections Attribute Type Description buff integer array BUFR message all sections size integer Size in bytes of BUFR message nr of words integer Idem now size in words Table 8 2 Attributes for the BufrMessageType data type Attribute Type Description ksup 9 integer Supplementary info and items selected from the other sections ksec 3 integer Expanded s
97. m GRIB filelist get field from GRIB file get from GRIB file get from GRIB filelist open GRIB file all messages get field from GRIB file get colloc from GRIB filelist get colloc from GRIB filelist not used not used not used Retrieve time interpolated GRIB data for a given lat and lon Inquiry of GRIB file list Prints GRIB message info Prints GRIB message info Open GRIB file and get some header information from all messages in this file Read header part of a GRIB message Interpolate data from four surrounding points for a given lat and lon Read all data from GRIB message Clear GRIB message from memory Helper routine Helper routine Table 9 1 Routines of module gribio module Reading Routine set GRIB filelist reads GRIB messages from a list of files decodes them and makes the data accessible in a list of GRIB messages in memory Retrieving Routine get from GRIB filelist returns an interpolated value four surrounding grid points from the GRIB data in the list of files messages for a given GRIB parameter latitude and longitude It is also possible to get a weighted value of all grid points lying within a circle around the latitude and longitude of interest This is used in the land fraction calculation in AWDP The land fraction is calculated by scanning all grid points of the land sea mask lying within 80 km from the centre of the WVC Every grid point found yields a land fraction betwe
98. merged an output row structure is allocated ant the input data are copied into the output rows The routine inversion qc which is called by preprocess performs land flagging and checks the setting of flags in the level 1b beam information If the input data is of inferior quality the qual sigma0 flag in the wvc quality is set which prevents further processing of this WVC Also the land fractions present in the beam information in the level 1b product are considered if any land fraction in the fore mid or aft beam exceeds 0 02 the qual sigma0 flag in quality is set as well The and flag in wvc quality is set whenever any level 1b land fraction is above zero The next step 15 the calibration of s in calibrate 50 This is done during the cal val period of ASCAT Once the level 1b products are fully calibrated this step can be removed Note that the calibration is done again in the reverse order after the post processing in order to write the o s to output as plain copies of the input oy s More information about the calibration can be found in Verspeek 2007 The monitoring which is performed as part of the post processing calculates some statistics from the wind product and writes them to an ASCII file called monitoring report txt The monitoring parameters are listed in table 4 24 They are calculated separately for three different regions of each swath left and right Note that the monitoring is invoked only
99. mpiler Features Compiler specific routines mainly command line handling support convert Conversion between wind speed direction and u and v support datetime Date and time conversion routines support ErrorHandler Error handling routines 11 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Name Contents support file File handling routines support grib GRIB file handling routines support num Numerical definitions and number handling routines support pfs PFS file handling routines support singletonfft FFT routines needed in minimization support sort Sorting routines Table 2 2 Contents of directory genscat Directories awdp and genscat and their subdirectories contain various file types e Fortran 90 source code recognizable by the F 90 extension e C source code recognizable by the c extension e Files and scripts that are part of the make system for compilation like Makefile_thisdir Makefile use Objects txt and Set Makeoptions see 2 3 4 for more details e Scripts for the execution of AWDP in directory awdp execs e Look up tables and BUFR tables needed by AWDP e Files with information like Readme t xt After compilation the subdirectories with the source code will also contain the object codes of the various modules and routines 2 3 2 Environment variables AWDP needs a number of environment variables to be set These are lis
100. n the initial processing is evaluated and it will be used to determine if a WVC contains suitable backscatter data for wind inversion Several options for the processing can be invoked szffl file Read a list of full resolution PFS file names in the file named file The files in the list are read and the full resolution PFS data SZF are used to replace the 25 km 12 5 km beam data This option is intended to produce a coastal wind product The beam data values incidence and azimuth angles which are read from the BUFR or PFS level 1b input file are replaced by average values of the data from the full resolution file which are located within a certain radius typically 10 to 20 kilometers from the WVC location See section 3 5 3 for more information stressparam Get stress related parameters derived from GRIB files This option is intended for research activities More information can be found in the Fortran code of AWDP noinv Switch off inversion default is switched on icemodel IM Choose ice screening method to be used 0 default 1 or 2 The value 0 results in no ice screening except when a GRIB file containing sea surface temperature is read The value 1 invokes a simple non Bayesian ice model which does not keep history of the water or ice state of each location Value 2 invokes the Bayesian ice model which keeps the history of each location and uses this history to determine the 16 NWP SAF AWDP User Ma
101. ne WVC containing all three beams 3 5 5 Quality control The quality of every WVC is controlled Before processing the beam data checks are done on the completeness and usability of the op data After the wind inversion step the distance of the first rank wind solutions to the GMF is considered If this value is too large the wind solutions are flagged The K values are also considered If one of the three beam K values is above a threshold which is wind speed dependent the wind information is flagged Stoffelen 1998 3 5 6 Inversion In the inversion step of wind retrieval the radar backscatter observations in terms of the Normalized Radar Cross Sections are converted into a set of ambiguous wind vector solutions In fact a Geophysical Model Function GMF is used to map a wind vector specified in term of wind speed and wind direction to a oy value The GMF depends not only on wind speed and wind direction but also on the measurement geometry relative azimuth and incidence angle and beam parameters frequency and polarization Currently the CMOD5 GMF which was developed for ERS is in use see Hersbach Stoffelen and de Haan 2007 but improvements are under study The AWDP program also includes the Multiple Solution Scheme MSS In MSS mode a large number of wind vector solutions is produced typically 144 The wind vector solutions are ranked according to their probability based on the MLE and constitute the full wind
102. ninin iiser a ii 7 2 2 MODES OFUSING AW DB i sss eti ebbe aru BON IT n ORO PRISE RA RUP 10 2 3 INSTALLING A ID 10 23 1 Directories and files i RR RERO RU Nen wees scab esse EUIS 11 2 3 2 Environment variables 12 2 3 3 nstallingAhe BUFR libr ty i eo e de or d ee e gets 13 2 3 4 Installing the GRIB library essent rennen eerte nennen 13 2 3 5 Compilation Gnd lmking eis dae o ape eder dede 13 2 3 6 Some remarks for Cygwin users nennen ren eene ene ie entente tenete nennen 15 2 4 COMMAND LINE OPTIONS hee ge entere euh eis 15 2 5 EE 19 2 6 TEST DATA AND TEST PROGRAMS een b o reete e wai ath 20 2 7 DOCUMENTATION 3 5 2 6 e tte ege OI re Pio i aie or pte eati tere US 22 AWDP PRODUCT SPECIFICATION e eere eee to se etta eese tone etta 23 3 1 PURPOSE OF PROGRAM AW DPs inia E EE EE E AAE a 23 3 2 OUTPUTSPEGCIEICA TION AEA E OA m e onem a v A iter 23 3 3 INPUT SPECIFICATION eee i Fede ee ro ee e ete rete bee dede eee i eee YER eh 24 3 4 SYSTEM REQUIREMENTS eee RE Mas Ch ees Ce e eta te eee Cogita te ce qus oe ste 24 3 5 DETAILS OF FUNCTIONALITY 3s 0 252 tins esse tec ote re rated deett E Foe eee oet diee 24 BUFR IQ and coding iih emen gehe eoe es oe ett pedea 24 3 5 2 Product resolution
103. not yet exist Alternatively the user may specify a file name including path in the environment variable SLUT FILENAME C VV If SLUT FILENAME C VV is present it will be used to store the GMF A second file with the same name and extension zspace is also generated Note the old GMF files need to be removed if new files need to be generated 1 e if a different GMF version is requested Ignore setting of usability flags and perform o calibration During the ASCAT calibration and validation period all o usability flags in the level 1b product are set i e all beams are marked as invalid These flags are ignored if this option is used Moreover a calibration of the op values is performed See Verspeek Portabella Stoffelen and Verhoef 2007 for more details Use the Multiple Solution Scheme for Ambiguity Removal If the Multiple Solution Scheme MSS is switched on AWDP internally works with 144 different solutions for the wind vector If MSS is switched off AWDP calculates two solutions at most MSS is switched off as default Choose ambiguity removal method Valid methods are 1st rank the wind solution with the lowest distance to GMF residual is selected bgc losest the wind solution closest to the background model wind is selected prescat see section 6 5 17 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 2
104. nual and Doc ID NWPSAF KN UD 005 Version 2 0 01 Reference Guide Date November 2010 noamb nowrite ignorellflags cmod N calval mss armeth meth sea or ice state of a WVC The ice screening can be combined with the ice screening which is done in the GRIB collocation In this case the SST of the GRIB file will be used to assign a WVC as surely ice when the SST is below a certain value or surely water when the SST is above a certain value Switch off ambiguity removal default is switched on This option is useful if the selection of the scatterometer wind solution is left to the data assimilation procedure of a Numerical Weather Prediction model In other words the NWP model is fed with a number of solutions and their probability and finds the best value when comparing with other data sources Do not produce BUFR output default is switched on Ignore the setting of level 1b o related flags in BUFR input If this option is switched on the value of the flags and quality indicators in the beam information including the sigma0 usability and land fraction is neglected Choose CMOD version 4 5 55 5n default or 6 With this option the user can choose between GMF version CMODA CMODS CMODS 5 CMODS 0 5 m s CMOD5n CMODS for neutral winds or CMOD6 The GMF table is generated by the program and written to a binary file named c vv2 dat in the current directory if it does
105. of the main data structures within the AWDP package In fact both the input and the output structure are one dimensional arrays of the row data structure row type These arrays represent just a part of the swath Reading and writing decoding and encoding ASCAT BUFR files corresponds to the mapping of a BUFR message to an instance of the row type and 30 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 vice versa The main constituent of the row type is the cell data structure cell type see figure 4 2 Since most of the processing is done on a cell by cell basis the type is the pivot data structure of the processor row type cell type ambiguity type Figure 4 2 Schematic representation of the nested data definitions in the row type data structure The related level 1b data of a cell are stored in a data structure called beam type Every cell contains three instances of the beam type corresponding to the fore middle and aft beams A cell may also contain an array of instances of the ambiguity type data structure This array stores the results of a successful wind retrieval step the wind ambiguities level 2 data Details of all the data structures and methods working on them are described in the next sections 4 1 4 Quality flagging and error handling Important aspects of the data processing are to check the validity of the data and to check the data
106. on of dummy method InitDummyBatch not used ExitAmbremMethod AWDP Deallocation of memory Table 6 1 Routines of module Ambrem Routine Description Documentation Ambrem2DVAR Interface to KNMI 2DVAR method Section 6 4 AmbremBGClosest Interface to Background Closest method Section 6 1 AmbremPrescat Interface to Prescat method Section 6 5 Table 6 2 Interface modules for different Ambiguity Removal schemes 6 3 Module Batch Mod After the wind retrieval step the Ambiguity Removal step is performed on selections of the available data In general these selections are just a compact part of the swath or a compact part of the world ocean The batch module BatchMod facilitates these selections of data In fact a batch data structure is introduced to create an interface between the swath related data and the data structures of the different AR methods Consequently the attributes of the batch data structures are a mixture of swath items and AR scheme items Figure 6 2 gives a schematic overview of the batch data structure Descriptions of the attributes of the individual batch data components are given in table 6 3 50 DocID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Veian 2 0 01 Reference Guide Date November 2010 AmbremPreScat AmbremBGclosest BatchMod Ambrem2DVAR CostFunction convert TwoDvarData BFGSMod SingletonFFT Figure 6 1 Interdependence of the modules for Ambiguity Removal Th
107. ons MSS is somewhat more dependent on the first guess and balance constraints in 2DVAR than the standard AWDP but much less noisy A noticeable advantage is thus obtained by using option 2 and potentially the full hi res benefit of the ASCAT data is achieved At the moment the 12 5 km data are experimental since at KNMI we are now objectively evaluating the added value of MSS and 2DVAR at 12 5 km The mode of using AWDP thus depends on the opportunities experience and time the user has to experiment with ASCAT data in the NWP system under consideration The AWDP program can of course also be used to create a stand alone wind product e g for nowcasting purposes Such a stand alone ASCAT wind product is a deliverable of the OSI SAF project More information on this project can be found on http www osi saf org 2 3 Installing AWDP AWDP is written in Fortran 90 with a few low level modules in C and is designed to run on a modern computer system under Linux or Unix AWDP needs a Fortran 90 compiler and a C compiler for installation AWDP comes along with a complete make system for compilation In some cases the Makefiles call installation scripts which are written in Bourne shell to enhance portability When compiled AWDP requires about 100 150 Mb disk space In principle AWDP may also run under Windows However it needs the BUFR and GRIB API libraries from ECMWF and this poses some restrictions on the systems supported Under Windows
108. ow InitBatchCell Initialization of batch Initialization of batch rows Initialization of batch cells Initialization of batch ambiguities DeallocBatch Processor Deallocation of batch DeallocBatchRows DeallocBatch Deallocation of batch rows DeallocBatchCells DeallocBatchRows Deallocation of batch cells DeallocBatchAmbis DeallocBatchCells Deallocation of batch ambiguities TestBatch Processor Test complete batch TestBatchRow TestBatch Test complete batch row TestBatchCell TestBatchRow Test batch cell TestBatchQualFlag Processor Print the quality flag getBatchQualFlag not used setBatchQualFlag not used PrnBatchQualFlag not used Table 6 5 Routines of module BatchMod 6 4 The KNMI 2DVAR scheme 6 4 1 Introduction The purpose of the KNMI 2DVAR scheme is to make an optimal selection provided the modelled likelihood of the ambiguities and the modelled uncertainty of the background surface wind field First an optimal estimated surface wind vector field analysis is determined based on variational principles This is a very common method originating from the broad discipline of Data Assimilation The optimal surface wind vector field is called the analysis Second the selected wind vector field the result of the 2DVAR scheme consists of the wind vector solutions that are closest to the analysis wind vector For details on the KNMI 2DVAR scheme formulation the reader is referred to Vogelzang 2007 Information on 2DVAR can also
109. r 8 3 5 2 Product resolution An important feature of the AWDP program is that it may produce a level 2 wind product on different resolutions The resolution of the level 2 wind product is the same as that of the level 1b input product ASCAT data are available in two different resolutions 50 km resolution with 25 km cell spacing also known as the ASCAT operational product SZO and 25 km resolution with 12 5 km cell spacing also known as the ASCAT research product SZR Of course there is a trade off between the resolution and the statistical error of the mean wind vectors The statistical error of the wind vectors for the higher resolution is a topic for further testing 3 5 3 Use of full resolution data AWDP offers the possibility to replace the backscatter values in the level Ib product by box averaged o data that are acquired using the full resolution ASCAT level 1 data SZF The beam data o values incidence and azimuth angles which are read from the BUFR or PFS level 1b input file are replaced by average values of the data from the full resolution file which are located within a certain radius typically 10 to 20 kilometers from the WVC location In this way a coastal product can be created See Portabella et al 2008 for more information on how the box averaged data are composed It is important to notice that the full resolution data which are fed into AWDP using the sz 1 option must span a time starting at least 1
110. r specific issues mainly with respect to command line argument handling The Makefile in this directory compiles on of the available source files depending on the Fortran compiler used Subdirectory convert contains module convert for the conversion of meteorological and geographical quantities e g the conversion of wind speed and direction into u and v components and vice versa Subdirectory datetime contains module DateTimeMod for date and time conversions AWDP only uses routines GetElapsedSystemTime for calculating the running time of the various processing steps and julian2ymd and ymd2julian for conversion between Julian day number and day month and year Module DateTimeMod needs modules ErrorHandler and numerics Subdirectory ErrorHandler contains module ErrorHandler for error management This module is needed by module DateTimeMod Subdirectory file contains module LunManager for finding opening and closing free logical units in Fortran AWDP uses only routines get_lun and free_lun for opening and closing of a logical unit respectively Subdirectory num contains module numerics for handling missing values for instance in the BUFR library This module is needed by module DateTimeMod and is used in the test program test_modules Subdirectory pfs contains module pfs_ascat for opening reading and closing of files in PFS format Subdirectory sort finally contains module SortMod for sorting the rows according to t
111. ran compiler Cell spacing m MSS Inversion AR BUFR IO GRIB IO Total seconds seconds seconds seconds seconds 25000 No 33 8 3 1 46 12500 No 132 16 8 3 160 12500 Yes 156 91 8 3 263 Table 3 2 Approximate times needed by AWDP to process example BUFR files under various input resolutions and options As can be seen from table 3 2 the use of MSS results in slightly larger processing times needed for inversion and in much larger processing times needed for AR The computation time of course increases with increasing resolution The choice of platform compiler and compiler settings will generate a large variation in the processing times Using the Portland pgf90 compiler rather than the GNU g95 compiler in the examples in table 3 2 will result in processing times that are 25 to 50 smaller 2 AWDP User Manual and Doc ID NWPSAF KN UD 005 Version 2 0 01 NWP SAF Reference Guide Date November 2010 Chapter 4 Program Design In this chapter the design of the AWDP program is described in detail Readers to whom only a summary will suffice are referred to the Top Level Design TLD in section 4 1 Readers who really want to know the very detail should not only read the complete chapter but also the documentation within the code 4 1 Top Level Design 411 Main program The main program AWDP file awdp in the awdp src directory is a Unix Linux executable which processes ASCAT BUFR or PFS or ERS BU
112. reeware and can be obtained from web page http www netlib org opt index html file 1bfgs um shar The original Fortran 77 code has been adjusted to compile under Fortran 90 compilers Routine LBFGS and its dependencies are located in module BFGSMod F90 in directory genscat support BFGS Table 6 14 provides an overview of the routines in this module Routine LBFGS uses reverse communication This means that the routine returns to the calling routine not only if the minimization process has converged or when an error has occurred but also when a new evaluation of the function and the gradient is needed This has the advantage that no restrictions are imposed on the form of routine Jt calculating the cost function and its gradient The formal parameters of LBFGS have been extended to include all work space arrays needed by the routine The work space is allocated in the calling routine minimise The rank of LBFGS affects the size of the work space It has been fixed to 3 in routine minimise because this value gave the best results lowest values for the cost function at the final solution Routine Call Description LBFGS minimise Main routine LBFGS Printing of output switched off daxpy LBFGS Sum of a vector times a constant plus another vector with loop unrolling ddot LBFGS Dot product of two vectors using loop unrolling MCSRCH LBFGS Line search routine MCSTEP MCSRCH Calculation of step size in line search Tab
113. remBGclosestOnBatch uv_to_dir Do2DVARonBatch BatchInput2DVAR TestBatchCell InitObs2DVAR gt Set_WVC_Orientations rotuv PrintObs2DVAR Do2DVAR 2 BatchOutput2DVAR rotuv InitObs2DVAR gt DeallocObs2DVAR DoDummyMeth Figure B2 2 Calling tree for AR routine DoAmbrem gt InitObs2dvar InitOneObs2dvar TestObs2dvar set2DVARQualFlag Figure B2 3 Calling tree for AR routine nitObs2dvar 87 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 gt Do2DVAR TestObs2dvar set2DVARQualFlag Prn2DVARQualFlag SetCovMat StrucFuncPsi StrucFuncChi SingletonFFT2d 2 Jt 2 Minimise Jt 2 LBFGS daxpy ddot LB1 MCSRCH MCSTEP TestObs2dvar set2DVARQualFlag DumpAnalysisField Figure B2 4 Calling tree for AR routine Do2DVAR Unpack ControlVector Uncondition SingletonFFT2d gt JoScat Uncondition_adj SingletonFFT2d gt Pack_ControlVector Figure B2 5 Calling tree for AR routine Jt calculation of cost function 88 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 SingletonFFT2d SFT_PrimeFactors SFT_Permute SFT_PermuteSinglevariate SFT_PermuteMultivariate SFT_Base2 SFT_Base3 SFT_Base4 SFT_Base5 SFT_BaseOdd SFT_Rotate Figure B2 6 Calling tree for AR routine Singl
114. rocessing subc SC Set id of sub centre in each WVC of the BUFR output to SC By default it is copied from input mon Switch on the monitoring function The monitoring results are written in an ASCII file with the name monitoring report txt By default no monitoring file is produced verbosity L Setthe verbosity level to L default is 0 If the verbosity level is 1 or smaller no output is written to the standard output except error messages If the verbosity level equals 0 only some top level processing information is written to output If the verbosity level is 1 or greater also additional information is given The normal mode of operation of AWDP is wind processing i e a BUFR or PFS file is read and the various processing steps are performed Note that by default AWDP does not recalculate data that are already present in the input For example if a WVC already contains model winds then the GRIB collocation will not be done for this WVC if a WVC already contains wind solutions then the wind inversion will not be performed This behaviour is desired when near real time processing is performed and a mixture of level 1b and level 2 files is fed into AWDP If one wants 18 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 to perform reprocessing of level 2 files the behaviour of AWDP can be changed by the command line options e g the 1 11 option
115. rogram the icemodel module is only used in the awdp icemodel module see section 4 3 8 4 2 4 Module Bufrmod Genscat contains several support modules In particular the BufrMod module is the Fortran 90 wrapper around the BUFR library used for BUFR input and output It is located in subdirectory 32 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 genscat support bufr Details of this module are described in Chapter 8 In the AWDP program the BufrMod module is only used in the awdp_bufr module see subsection 4 3 2 4 25 Module gribio module The gribio module module is the Fortran 90 wrapper around the GRIB library used for GRIB input and collocation of the NWP data with the scatterometer data It is located in subdirectory genscat support grib Details of this module are described in Chapter 9 In the AWDP program the gribio module module is used in the awdp_grib and pfs modules see subsection 4 3 5 4 2 6 Support modules Subdirectory genscat support contains more support modules besides Bufrmod and gribio module The KNMI 2DVAR Ambiguity Removal method requires minimization of a cost function and numerical Fourier transformation These routines are located in subdirectories BFGS and singletonfft respectively and are discussed in more detail in section 6 4 Subdirectory Compiler Features contains module Compiler Features for handling some compile
116. routine open pfs ascat file read pfs ascat read string from file get uint Figure B5 2 Calling tree for PFS file handling routine read pfs ascat mdr 94 NWPSAF KN UD 005 Version 2 0 01 Reference Guide Date November 2010 AWDP User Manualand 2 10 NWP SAF get pfs ascat node get szo node get_time get_ushort get_uint ymd2julian julian2ymd get_ushort1 get_ushort get_int1 get_int get_uint1 get_uint get_int3 get_int get_ushort3 get_ushort get_short3 get_short get_uchar3 get szr node calc asc Figure B5 3 Calling tree for PFS file handling routine get pfs ascat node The calling tree for get szr node is identical to to the one of get szo node 95 AWDP User Manual and DocID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 get pfs ascat szf node get_time get_ushort get_uint ymd2julian julian2ymd get_int1 get_int get_ushort1 get_ushort get_uint1 get_uint get_uchar1 calc asc Figure B5 4 Calling tree for PFS file handling routine get pfs ascat szf node 96 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand kion 2001 Reference Guide Date November 2010 Appendix B6 Calling tree for ice model routines The figures in this appendix show the calling tree for the ice model routines in genscat All routines are part of genscat as indicated by the black boxes An arrow befor
117. s made available in the framework of the NWP SAF project This file should be placed in the directory from which AWDP is to be run After unzipping with gunzip AWDP lt version gt tar gz and untarring with tar xf AWDP lt version gt tar the AWDP package is extracted in subdirectories awdp and genscat which are located in the directory where the tar file was located Subdirectories awdp and genscat each contain a number of files and subdirectories A copy of the release notes can also be found in the directory awdp docs Tables 2 1 and 2 2 list the contents of directories awdp and genscat respectively together with the main contents of the various parts Depending on the distribution more directories may be present but these are of less importance to the user Name Contents doc Documentation including this document execs Link to awdp executable shell script for running AWDP sre Source code for AWDP program and supporting routines test Example BUFR and GRIB input files for testing purposes Table 2 1 Contents of directory awdp Name Contents ambrem Ambiguity removal routines ambrem twodvar KNMI 2DVAR ambiguity removal routines inversion Inversion and quality control routines main Dummy subdirectory to facilitate the make system support General purpose routines sorted in subdirectories support BFGS Minimization routines needed in 2DVAR support bufr BUFR tables in subdirectory and file handling routines support Co
118. s of verbosity specified lt 1l be as quiet as possible 0 only report top level processing information gt 1 report additional information Of course errors are logged in any case Table 4 4 gives a incomplete list of verbosity parameters They are not all set by the command line option as some of them serve testing and debugging purposes Module Verbosity parameter Ambrem2Dvar TDVverbosity AmbremBGclosest BGverbosity BatchMod BatchVerbosity Ambrem AmbremVerbosity awdp_bufr BufrVerbosity awdp_grib GribVerbosity Table 4 4 Verbosity parameters 4 2 Module design for genscat layer 4 2 1 Module inversion The module inversion contains the genscat inversion code Module post inversion contains some routines specific for ERS and ASCAT inversion and quality control The modules are located in subdirectory genscat inversion Details of this module are described in Chapter 5 In the AWDP program the inversion module is only used in inversion module see section 4 3 6 4 23 0 Module ambrem The module ambrem is the main module of the genscat Ambiguity Removal code It is located in subdirectory genscat ambrem Details of this module are described in Chapter 6 In the AWDP program the ambrem module is only used in ambrem module see section 4 3 7 4 2 3 Module icemodel The module icemodel contains the genscat ice screening code It is located in subdirectory genscat icemodel In the AWDP p
119. s part of the genscat support modules The current version is a Fortran 90 wrapper around the ECMWF GRIB API library see http www ecmwf int The goal of this support module is to provide a comprehensive interface to GRIB data for every Fortran 90 program using it In particular gribio module provides all the GRIB functionality required for the scatterometer processor based on genscat Special attention has been paid to testing and error handling 9 1 Background The acronym GRIB stands for GRIdded Binary GRIB is maintained by the World Meteorological Organization WMO and other meteorological centres In brief the WMO FM 92 GRIB definition is a binary format for efficiently transmitting gridded meteorological data It is beyond the scope of this document to describe GRIB in detail Complete descriptions are distributed via the websites of WMO http www wmo int and of the European Centre for Medium range Weather Forecasts ECMWF http www ecmwf int Module gribio module is in fact an interface On the one hand it contains temporary definitions to set the arguments of the ECMWF library functions On the other hand it provides self explaining routines to be incorporated in the wider Fortran 90 program Section 9 2 describes the routines in module gribio module The available data structures are described in section 9 3 The gribio module uses two libraries from the GRIB software library of ECMWF This is discussed in some more
120. sage EncodeBufrData CheckBufrData FillBufrData PrintBufrErrorCode Figure B3 4 Calling tree for BUFR file handling routine save BUFR file 9 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Appendix B4 Calling tree for GRIB routines The figures in this appendix show the calling tree for the GRIB file handling routines in genscat Routines in black boxes are part of genscat Routines in grey boxes belong to the ECMWF GRIB API library An arrow before a routine name indicates that this part of the calling tree is a continuation of a branch in a previous figure The same arrow after a routine name indicates that this branch will be continued in a following figure set GRIB filelist open GRIB file read GRIB header info Figure B4 1 Calling tree for GRIB file handling routine set GRIB filelist inquire GRIB Ffilelist get GRIB msgnr display req GRIB msg properties display GRIB message properties Figure B4 2 Calling tree for GRIB file handling routine inquire GRIB filelist 92 AWDP User Manual and Doc ID NWPSAF KN UD 005 Version 2 0 01 NWP SAF Reference Guide Date November 2010 gt get_from_GRIB_filelist get_GRIB_msgnr display_req_GRIB_msg_properties display_GRIB_message_properties display_req_GRIB_msg_properties display_GRIB_message_properties get_GRIB_data_val
121. sation of MLEs and noise values These tables are read from ASCII files which are present in direction genscat inversion The environment variable 5 TNVERSION LUTSDIR should contain the proper directory name 5 3 Antenna direction The output wind direction of inversion routines are generally given in the meteorological convention see table 5 3 The inversion routine uses a wind direction that is relative to the antenna direction The convention is that if the wind blows towards the antenna then this relative wind direction equals to 0 Therefore it is important to be certain about the convention of your antenna azimuth angle For ERS and ASCAT the radar look angle antenna angle or simply azimuth equals 0 if the antenna is orientated towards the south The radar look angle increases clockwise Therefore the antenna angle needs a correction of 180 degrees Meteorological Oceanographic Mathematical u v Description 0 180 270 0 1 Wind blowing from the north 90 270 180 1 0 Wind blowing from the east 180 0 90 0 1 Wind blowing from the south 270 90 0 1 0 Wind blowing from the west Table 5 3 Conventions for the wind direction 48 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Chapter 6 Ambiguity Removal module 6 1 Ambiguity Removal Ambiguity Removal AR schemes select a surface wind vector among the different surface wind vector sol
122. servations array InitOneObs2dvar InitObs2dvar Initialization of single observation TestObs2dvar Do2DVAR Test single observation Prn2DVARQualFlag Do2DVAR Print observation quality flag set2DVARQualFlag TestObs2DVAR Convert observation quality flag to integer get2DVARQualFlag not used Convert integer to observation quality flag Table 6 10 Routines in module TwoDvarData The quality status of an instance of Obs2dvarType 1s indicated by the attribute QualFlag which is an instance of TwoDvarQualFlagType The attributes of this flag are listed in table 6 11 Attribute Description missing Flag values not set wrong Invalid 2DVAR process Lat Invalid latitude Background Invalid background wind increment Ambiguities Invalid ambiguity increments Selection Invalid selection Analyse Invalid analysis wind increment Cost Invalid cost function specification gradient Invalid gradient specification weights Invalid interpolation weights grid Invalid grid indices Table 6 11 Attributes of 2DVAR observation quality flag 55 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 6 4 3 Reformulation and transformation The minimization problem to find the analysis surface wind field the 2D Variational Data Assimilation problem may be formulated as minJ 0 7 0 ty 9 6 1 where v is the surface wind field in consideration and J the total cost function consisting of t
123. sianIcemodel Print one or more ice map variables to graphical ppm files printIce printIceMap Print the ice parameter a and the ice classes to a ppm file printppmcolor printiceMap Print variable on ice map to ppm file using colour index printppmvar Print variable on ice map to ppm file mapped on gray scale Table 7 1 Routines of module iceModelMod 7 3 Data structures There are two important data structures defined in this module The first contains all relevant data of one pixel on the ice map IcePixel The second one contains basically a two dimensional array of ice pixels and represents an entire ice map IceMapType This could be either an ice map of the North Pole region or the South Pole region Attribute Type Description subClass integer Ice subclass alceAves real Average of the a iceparameter aSd real a iceparameter standard deviation Number of measurement classO integer Ice class Pice real a priori ice probability Qice real a priori odds on ice plceGivenX real a posteriori ice probability gqIceGivenX real a posteriori odss on ice sumWeightST real Sum of weight factors sst real Sea surface temperature K alce real nhist a iceparameter qXgivenlce real nhist Odds on measurement X given ice timelce DateTime nhist Date time of measurement class integer nhist Ice class Table 7 2 Attributes for the cePixel data type Attribute Type Description n
124. stress data Attribute Description Eastward zonal wind component v real Northward meridional wind component t real Air temperature q real Specific humidity sst real Sea surface temperature chnk real Charnok parameter Sp real Surface pressure Table 4 10 NWP stress parameter data structure Row data The data of a complete row of the swath is stored in the data type row_type see table 4 11 A complete row corresponds to a single BUFR message in the AWDP output The level 1 BUFR data may contain more than one row per BUFR message Attribute Type Description time stamp integer Time stamp of row data in seconds used for sorting num_cells integer Actual number of WVC s Cell 82 cell type Array of Wind Vector Cells Table 4 11 Row data structure Time data The time type data type contains a set of 6 integers representing both the date and the time see table 4 12 The routine init time sets the time entries to missing values The routine test time tests the validity of the date and time specification see also the cell process flag The routine print time can be used to print the time information Attribute Type Description year integer 19 or20XX month integer 1 12 day integer 1 31 hour integer 0 23 minute integer 0 59 second integer 0 59 Table 4 12 Time data structure Wind Data The wind type data type contains the wind speed and wind direction s
125. t ground stations Table 4 14 Beam collocation flag bits K estimate quality flag The kp_estim_qual_type data type contains the flag indicating the quality of the K estimate Each one of the three beams in a WVC contain an instance of this flag The attributes are listed in table 4 15 The function get_kp_estim_qual interprets an integer flag BUFR input to an instance of kp_estim_qual_type The function set_kp_estim_qual transforms an instance of kp_estim_qual_type to an integer flag Attribute Bit 2 Description missing Flag not set all bits on estim qual 0 1 Inferior quality of K estimate Table 4 15 estimate quality flag bits Fortran Wind Vector Cell quality flag Every WVC contains a flag for its quality Therefore the cell type contains an instance of the wvc quality type Table 4 16 gives an overview of its attributes The function get wvc quality interprets an integer flag BUFR input to an instance of wvc quality type The function get_wvc_quality transforms an instance of quality type to an integer flag The routine print quality may be used to print the bit values of the flag Attribute Bit 25 Description missing Flag not set all bits on qual sigma0 22 4194304 Not enough good o available for wind retrieval 38 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Attribute Bit 2 Descr
126. tart files are read in again Environment variable SRESTARTDIR contains the directory for the ice model restart files Optionally sea surface temperature SST data from GRIB files can be used to further improve the 39 Doc ID Version Date NWPSAF KN UD 005 2 0 01 November 2010 NWP SAF AWDP User Manual and Reference Guide quality of the ice algorithm the use sst logical must be turned on Processing llb input with the use of NWP data and SST data can be done with the following command line options bufr file icemodel 2 mon awdp f lt gribfilelist gt handleall nwpfl Reprocessing of level 2 input with only running the ice model on top of it can be done with the following command line options noinv noamb mon handleall gt awdp f bufr file icemodel 2 The SSM I grids are widely used for representation of ice related parameters A good description as well as some software routines can be found on the website of the National Snow and Ice Data Centre NSIDC http www nsidc org data docs daac ae si25 25km tb and sea ice gd html A more detailed description of the Bayesian statistics method and ice model is given in Verspeek 2006 7 2 Routines Table 7 1 provides an overview of the routines in module iceModelMod Routine Call Description calcAave iceMapWeighted Calculate the space time averaged ice map parameters calcPoly3 calclcelineParms Calculat
127. ted in table 2 3 together with their possible values Name Value SBUFR_TABLES genscat support bufr bufr_tables SGRIB_DEFINITION_PATH genscat support grib definitions SINVERSION_LUTSDIR genscat inversion SLUT FILENAME C VV Any existing writable path file name Table 2 3 Environment variables for AWDP The SBUFR TABLES variable guides AWDP to the BUFR tables needed to read the input and write the output The SGRIB DEFINITION PATH variable is necessary for a proper functioning of the GRIB decoding software The variable STNVERSION LUTSDIR should point to a directory containing some look up tables extension asc that are used by the inversion software The necessary tables are delivered with genscat The variable LUT FILENAME VV points AWDP to the correct C band GMF lookup table at VV polarisation It should contain a file name including a valid path If the file does not exist it will be created when the inversion is invoked for the first WVC In order to prevent confusion it is advised to use standard file names lt path gt cmod5 dat path cmod5 5 dat 12 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 path cmod5 n dat or path cmod6 dat since the inversion software uses the file name to determine which CMOD version is used 2 3 3 Installing the B
128. ternal review 1 0 16 Dec 2008 Anton Verhoef Modified according to DRI comments 1 1 Jan 2010 Anton Verhoef Removed a few typo s and corrected some of the diagrams in the appendices for AWDP version 1 1 2 0 Aug 2010 Anton Verhoef Modified for AWDP version 2 0 added section 3 5 3 changed sections 2 3 2 3 4 2 4 Chapter 9 and Appendix B4 2 0 01 Nov 2010 Anton Verhoef Modified according to DRI comments Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 Contents CONTENDS ep dono ni ep aue ee paa seo rp o LESE aUe PE EE NN Een ER Pe ERE SR EN Eee aga 1 PREEA GE UE E e eR TEN NN ea ER e EEUU REN EVEN N RUN VETE Re NEUSS UAE ERES 4 CHAPTER 1 INTRODUCTION eee o roe sto ro e eren poe ei ene Enea aa oo ee ba rane rea nn oen esae ERR Tuae ep Tae o a eps 5 1 1 NEP WEIST UD 5 1 2 DEVELOPMENTOEAXWDDB t tee dtes tectae hee Eee tens 5 1 3 TESTING AW DP a seca AE AE 6 1 4 USER MANUAL AND REFERENCE GUIDE eseeeseeeeeeee eene enne enne ener tenentes enne 6 1 5 CONVENTIONS 5 5 13 ee ER Ee Ra Uta te Ov iet UR don EP AER oe 6 CHAPTER 2 AWDP USER MANUAL eee esae se eot he ea F orbita rb obo pond eee o oa ia ean ale ense Ue 7 2 1 WHY USING THE AWDP PROGRAM mersi
129. tional parameters are stored These are e g latitude longitude time information orbit and node numbers background wind vector cell quality flag and information on the scatterometer beams including and K data The output file is structured according to the same conventions as the ASCAT level Ib input also 1f ERS data are processed The ASCAT BUFR format consist of three main sections one section containing level 1b information which is copied from the input data one section containing Surface Soil Moisture SSM level 2 information which is also copied from the input and one section containing level 2 wind data which is calculated in AWDP The ASCAT BUFR data descriptors are listed in Appendix C 23 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 3 3 Input specification Input of AWDP is the ASCAT level 1b BUFR or PFS Data Product These products are created by EUMETSAT see WMO 2007 and Figa Saldaf a and Wilson 2005 Alternatively the ERS scatterometer wind product in BUFR can be used as input see UK Met Office 2001 It is also possible to reprocess level 2 ASCAT or ERS data in ASCAT BUFR format and treat them as if they are level 1b data To achieve this some command line options need to be set see section 2 4 Apart from the scatterometer data GRIB files containing NWP output with global coverage are necessary for the wind processing At least three
130. ttp www ecmwf int products data software grib api html The package contains scripts for compilation and installation The reader is referred to this site for assistance in downloading and installing the GRIB API Library Directory genscat support grib contains the shell script make grib lib It unzips untars and compiles the GRIB API library file which is downloaded from ECMWF and placed into this directory This script is part of the genscat make system and it is automatically invoked when compiling genscat The current version is tested with GRIB API version 1 9 0 but later versions can be used However AWDP is not tested with later versions 2 3 5 Compilation and linking Compilation and linking of AWDP under Linux or Unix is done in three steps 1 Set the compiler environment variables according to the choice entered on request This can be done by running the appropriate use scripts in directory genscat 2 Go to directory genscat and type make 3 Go to directory awdp and type make to produce the executable awdp in directory awdp src Before activating the make system some environment variables identifying the compiler should be set These variables are listed in table 2 4 The environment variables in table 2 4 can be set by using one of the use scripts located in directory genscat Table 2 5 shows the properties of 13 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date Novemb
131. ty Control of SeaWinds Journal of Atm Oceanic Technol 18 pp 1171 1183 e Portabella M and Stoffelen A 2004 A probabilistic approach for SeaWinds Data Assimilation Quart J Royal Meteor Soc 130 pp 127 152 e Portabella M Verhoef A and Stoffelen A 2008 ASCAT coastal AWDP prototype report NWPSAF KN VS 007 Available on http www knmi nl scatterometer publications e Stoffelen A and M Portabella 2006 On Bayesian Scatterometer Wind Inversion IEEE Transactions on Geoscience and Remote Sensing 44 6 1523 1533 doi 10 1109 TGRS 2005 862502 e Stoffelen A de Haan S Quilfen Y and Schyberg H 2000 72 DocID NWPSAF KN UD 005 NWP SAF AWDP User Manual and Version 2 0 01 Reference Guide Date November 2010 ERS scatterometer ambiguity removal scheme comparison OSI SAF report Available on http www knmi nl scatterometer publications e Stoffelen A C M 1998 Scatterometry PhD thesis University of Utrecht ISBN 90 393 1708 9 Available on http www knmi nl scatterometer publications e Talagrand O 1991 The use of adjoint equations in numerical modeling of the atmospheric circulation In Automatic Differentiation of Algorithms Theory Implementation and Application A Griewank and G Corliess Eds pp 169 180 Philadelphia Penn SIAM e Verhoef A Vogelzang J Verspeek J and Stoffelen A 2008 AWDP Test Report Report NWPSAF KN TV 005 UKMO UK
132. uct specification 3 1 Purpose of program AWDP The ASCAT Wind Data Processor AWDP program has been developed to fully exploit oo data from the ASCAT scatterometer instrument on the MetOp satellites or the AMI scatterometer instrument on the European Remote Sensing ERS satellites to generate surface winds AWDP may be used for real time data processing The main application of AWDP is to form the core of an Observation Operator for ASCAT scatterometer data within an operational Numerical Weather Prediction System Program AWDP is also a level 2 data processor It reads data from the EUMETSAT level 1b ASCAT BUFR or PFS product or from the ESA ERS scatterometer BUFR product AWDP applies algorithms for inversion quality control and Ambiguity Removal at various spatial resolutions These methods are mainly developed and published by KNMI The output of AWDP is a BUFR file in ASCAT BUFR format 3 2 Output specification The wind vectors generated by AWDP represent the instantaneous mean surface wind at 10 m anemometer height in a 2D array of Wind Vector Cells WVCs with specified size 25 x 25 km or 12 5 x 12 5 km depending on the cell spacing of the input product These WVCs are part of the ground swath of the instrument In conventional mode the wind output for every WVC consists of up to 4 ambiguities wind vector alternatives with varying probabilities The selected wind vector is indicated by a selection index For every WVC addi
133. ue AMI scatt sat instr ers 142 ASCAT sat instr ascat 190 Table 4 20 BUFR instrument identifiers 40 Doc ID NWPSAF KN UD 005 NWP SAF AWDP User Manualand 2001 Reference Guide Date November 2010 Table 4 21 provides an overview of the different routines and their calls in this module The genscat support routines ymd2julian and julian2ymd are used to provide each row in AWDP with a date time stamp that can be used for sorting easily Routine Call Description ascat bufr to row data read bufr file ASCAT BUFR message into one or more row types ers bufr to row data read bufr file ERS BUFR message into 19 row types init bufr processing read bufr file Initialise module write bufr file read bufr file AWDP Read a complete BUFR file into row types row to bufr data write bufr file AWDP row type into ASCAT BUFR message write bufr file AWDP Write all row types into a complete BUFR file Table 4 21 Routines in module awdp bufr Note that the acquisition date and time of ERS data are modified when they are read in routine ers bufr to row data An ERS BUFR message contains 19 rows of data which all have the same date and time of acquisition This would cause problems in AWDP when the rows are sorted with respect to the acquisition date and time Therefore the date and time of each ERS row are recalculated assuming that the 10 middle row of the ERS BUFR message contains the true acquisition time and that
134. ues get_angle_distance extract_data_from_GRIB_message Figure B4 3 Calling tree for GRIB file handling routine get_from_GRIB_filelist get colloc from GRIB filelist convert to derived datetime conv date to daycount get_analyse_date_and_times inquire_GRIB_filelist gt check_proximity_to_analyse conv_date_to_daycount inquire_GRIB_filelist gt get_from_GRIB_filelist 2 Figure B4 4 Calling tree for GRIB file handling routine get_colloc_from_GRIB_filelist dealloc all GRIB messages dealloc GRIB message grib close file Figure B4 5 Calling tree for GRIB file handling routine dealloc all GRIB messages 93 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Appendix B5 Calling tree for PFS routines The figures in this appendix show the calling tree for the PFS native MetOp format file handling routines in genscat All routines are part of genscat as indicated by the black boxes An arrow gt before a routine name indicates that this part of the calling tree is a continuation of a branch in a previous figure The same arrow after a routine name indicates that this branch will be continued in a following figure open pfs ascat file read rec read string from file get_uint get_str get_num strne skip nrec skip rec read string from file get uint streq Figure B5 1 Calling tree for PFS file handling
135. uire GRIB filelist 2 read pfs ascat get pfs ascat szf node get_distance get from GRIB filelist gt test cell gt close pfs ascat file free lun Figure A 5 Calling tree for routine read full res data first level get grib data GetElapsedSystemTime init_grib_processing init GRIB module set_GRIB_filelist gt inquire GRIB filelist gt get from GRIB filelist get colloc from GRIB filelist gt test cell gt dealloc all GRIB messages Figure A 6 Calling tree for routine get grib data first level 77 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 gt invert_wvcs GetElapsedSystemTime init_inversion init_inv_settings_to_default get_inv_settings set_inv_settings init_inv_input invert one wvc normalise conedist prescat mode normalise conedist ers ascat check wind solutions ers ascat calc_probabilities GetSortIndex test cell 2 Figure A 7 Calling tree for routine invert first level 78 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 awdplcemodel GetElapsedSystemTime getClass initlceMap RW IceMap mapll SetIntegerDate SetlntegerTime updatelcePixel printlcePixel gt print wvc quality print cell
136. ure 34 NWP SAF AWDP User Manual and Reference Guide Doc ID Version Date NWPSAF KN UD 005 2 0 01 November 2010 Cell Data The cell_type data type is a key data type in the AWDP program because many processing steps are done on a cell by cell basis The attributes are listed in table 4 7 The routine init_cell sets the cell data to missing values Also the flags are set to missing The routine test_cell tests the validity of data This routine sets the cell process flag The routine print_cell may be used to print the cell data Attribute Type Description centre_id integer Identification of originating generating centre sub_centre_id integer Identification of originating generating sub centre software id lIb integer Software identification of level 1 processor satellite id integer Satellite identifier sat instruments integer Satellite instrument identifier sat motion real Direction of motion of satellite time time type Date and time of data acquisition lat real Latitude of WVC lon real Longitude of WVC pixel size hor real Distance between WVCs meters orbit nr integer Orbit number node nr integer Across track cell number height atmosphere Height of atmosphere used loss unit lenght real Loss per unit length of atmosphere beam collocation beam collocation type Beam collocation flag beam 3 beam type Beam data full res full res type Averaged full resolution data softwar
137. utions per cell for the set of wind vector cells in consideration The goal is to set a unique meteorological consistent surface wind field The surface wind vector solutions per cell simply called ambiguities result from the wind retrieval process step Whenever the ambiguities are ranked a naive scheme would be to select the ambiguity with the first rank e g the highest probability the lowest distance to the wind cone In general such a persistent first rank selection will not suffice to create a realistic surface wind vector field scatterometer measurements tend to generate ambiguous wind solutions with approximately equal likelihood mainly due to the 180 invariance of stand alone scatterometer measurements Therefore additional spatial constraints and or additional external information are needed to make sensible selections A common way to add external information to a WVC is to provide a background surface wind vector The background wind acts as a first approximation for the expected mean wind over the cell In general a NWP model wind is interpolated for this purpose Whenever a background wind is set for the WVC a second naive Ambiguity Removal scheme is at hand the Background Closest BC scheme The selected wind vector is just the minimizer of the distance e g in the least squares sense to the background wind vector This scheme may produce far more realistic wind vector fields than the first rank selection since the b
138. vector probability density function Subsequently the 2DVAR Ambiguity Removal method see e g section 3 5 7 is applied with a much larger set of wind vector solutions The output BUFR format can accommodate any number of wind solutions due to the use of the so called delayed descriptor replication Details on the KNMI inversion approach can be found in Portabella 2002 For SeaWinds MSS compares better to an independent NWP model reference and buoys than conventional two or four solution schemes Portabella and Stoffelen 2004 Vogelzang et al 2008 but for ERS and ASCAT this needs to be investigated further Technical information on the KNMI inversion approach can be found in Chapter 5 3 5 7 Ambiguity Removal The Ambiguity Removal AR step of the wind retrieval is the selection of the most probable surface wind vector among the available wind vector solutions the so called ambiguities Various methods have been developed for AR More information on Ambiguity Removal is given in Chapter 6 The default method implemented in AWDP is the KNMI 2DVAR AR scheme A description of its implementation can be found in section 6 4 The Multiple Solution Scheme MSS offers the possibility to postpone AR to the NWP step in order to treat all information from models and measurements in the same manner Further details on the algorithms and their 26 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date
139. wind forecasts with forecast time intervals of 3 hours are necessary to perform interpolation with respect to time and location Apart from this GRIB fields of Sea Surface Temperature and Land Sea Mask are necessary for land and ice masking 3 4 System requirements Table 3 1 shows the platform and compiler combinations for which AWDP has been tested However the program is designed to run on any Unix Linux based computer platform with a Fortran compiler and a C compiler The equivalent of a modern personal computer will suffice to provide a timely NRT wind product AWDP requires about 100 150 MB disk space when installed and compiled Platform Fortran compiler C compiler Suse Linux work station Portland pgf90 GNU gcc GNU 095 GNU gfortran SunOS Unix Sun Fortran Sun C SGI Irix MIPSpro Fortran compiler MIP Spro C compiler SGI Altix Intel Fortran compiler Intel C compiler Windows XP PC with Cygwin GNU g95 GNU gcc Table3 1 Platform and compiler combinations for which AWDP has been tested AWDP may also run in other environments provided that the environment variables discussed in section 2 2 are set to the proper values and that the BUFR library is properly installed For Windows a Unix emulator like Cygwin is needed 3 5 Details of functionality 3 5 1 BUFR IO and coding Data sets of near real time meteorological observations are generally coded in the Binary Universal Form for Representation BUFR BUFR is a machine indep
140. wind speed components u and v modules e g BufrMod subroutines and identifiers are printed italic Names of directories and subdirectories e g awdp src files e g awdp F90 and commands e g awdp f input are printed in Courier Software systems in general are addressed using the normal font e g AWDP genscat Hyperlinks are printed in blue and underlined e g http www knmi nl scatterometer References are in square brackets with the name of the author italic e g Stoffelen 1998 AWDP User Manual and Doc ID NWPSAF KN UD 005 NWP SAF Version 2 0 01 Reference Guide Date November 2010 Chapter 2 AWDP User Manual This chapter is the user manual of the AWDP program Sections 2 1 and 2 2 give general information about AWDP Section 2 3 provides information on how to install compile and link the AWDP software The command line arguments of AWDP are discussed in section 2 4 Section 2 5 gives information on a script for running AWDP Please note that any questions or problems regarding the installation or use of AWDP can be addressed at the NWP SAF helpdesk at http www nwpsaf org 21 Why using the AWDP program Scatterometers provide valuable observational data over the world s oceans Therefore successful assimilation of scatterometer data in numerical weather prediction systems generally improves weather forecasts The AWDP program has been developed to fully exploit scatterometer data It is m

AWDP User Manual and Reference Guide

Contents

Download Pdf Manuals

Related Search

Related Contents