Home

OASIS3-MCT User Guide OASIS3-MCT 1.0

Contents

1. Writes the grid cell corner longitudes and latitudes counterclockwise sense Longitudes must be given in degrees East in the interval 360 0 to 720 0 Latitudes must be given in degrees North in the interval 90 0 to 90 0 Note also that cells larger than 180 0 degrees in longitude are not supported Writing of corners is optional as corner information is needed only for SCRIPR CONSERV see section 4 3 If called needs to be called after oasis prism write grid CALL oasis write angle cgrid nx ny angle or ny CALL prism write angle cgrid nx angle cgrid CHARACTER 4 IN grid name prefix nx INTEGER IN first grid dimension x ny INTEGER IN second grid dimension y angle REAL Obsolete in OASIS3 MCT as vector interpolation is not supported See SCRIPR CONSERV in section 4 3 DIMENSION nx ny IN array of angles CALL oasis write mask cgrid nx ny mask or CALL prism write mask cgrid nx ny mask cgrid CHARACTER 4 IN grid name prefix nx INTEGER IN first grid dimension x ny INTEGER IN second grid dimension y mask INTEGER DIMENSION nx ny IN mask array be careful about the OA SIS historical convention 0 not masked 1 masked Writes the model grid mask CALL oasis write area cgrid nx ny area or ny CALL prism write
2. SCHANNEL 24 CHAPTER 3 THE CONFIGURATION FILE NAMCOUPLE SNFIELDS 4 JOBNAME SNBMODEL 3 ocemod atmmod chemod 55 70 99 SRUNTIME 432000 SINIDATI SMODINFO SNLOGPRT 2 CI SCALTYP Ht Ht Ht EE A aE aE aE aE aE aE aE HE HEE EE HE aE aE aE EEE SHEE EE ERE Second section SSTRINGS GI Field 1 SOSSTSST SISUTESU 1 86400 5 sstoc nc EXPORTED 182 149 128 64 toce atmo LAG 14400 SEQ 1 P2P 0 LOCTRANS CHECKIN MAPPING BLASNEW CHECKOUT AVERAGE INT 1 map toce atmo 120315 nc src opt CONSTANT 273 15 INT Field 2 CONSFTOT SOHEFLDO 6 86400 4 flxat nc EXPORTED atmo toce LAG 14400 SEQ 2 POP 2 LOCTRANS CHECKIN SCRIPR CHECKOUT ACCUMUL INT 1 BILINEAR LR SCALAR LATLON 1 INT 1 Field 3 COSENHFL SOSENHFL 37 86400 1 flda3 nc IGNOUT atmo atmo LAG 7200 LOCTRANS 32 FIRST SECTION OF NAMCOUPLE FILE 25 AV ERAGE Field 4 SOALBEDO SOALBEDO 7 86400 0 SOALBEDO nc INPUT FE TE a aE aE aE aE EE A EE AE EE EE EE SE 3 2 First section of namcouple file The first section of namcouple uses some predefined keywords prefixed by the sign to locate the related information The sign must be in the second column The first ten keywords are described hereafter SSEO
3. e CALL prism get localcomm proto local comm ierror local comm INTEGER OUT value of local communicator ierror INTEGER OUT returned error code If needed this routine may be called by the component processes to get the value of a local com municator to be used by the component for its internal parallelisation This may be needed as all component models started in a pseudo MPMD mode with MPII share au tomatically the same MPI COMM WORLD communicator Another communicator has to be used for the internal parallelisation of each component OASIS3 MCT creates this local communicator based on the name given to oasis init comp prism init comp proto routine its value is returned as the first argument of the routine Local_comm The model may call MPI Init explicitly but if so has to call it before calling prism init comp proto in this case the model also has to call MPI Finalize explicitly but only after calling prism terminate proto 8 CHAPTER 2 INTERFACING A COMPONENT MODEL WITH OASIS3 MCT 2 2 3 Coupling through a subset of the component model processes If only a subset of the component processes participate in the coupling e g a component is setup to run on 80 processes but 16 of those processes are associated with a distinct task like I O a communicator gathering only these processes must be defined with either oasis prism create couplcomm or oasis prism set couplcomm
4. e CALL prism get intracomm new comm cdnam kinfo new comm INTEGER OUT mpi intracomm communicator cdnam CHARACTER IN other model name kinfo INTEGER OUT OPTIONAL returned error code This routine sets up an MPI intracomm communicator between the root processors of two compo nents the local component and the component associated with cdnam This must be called by both components at the same time otherwise a deadlock will occur In addition this call is collective across the tasks of the two components but other components are not involved 2 10 Coupling algorithms SEQ and LAG concepts Using the OASIS3 MCT coupling library the user has full flexibility to reproduce different coupling algo rithms In the components the sending and receiving routines respectively oasis put prism put proto and oasis get prism get proto can be called at each model timestep with the appropriate date argument giving the actual time at the beginning of the timestep expressed in number of seconds since the start of the run see section 2 7 1 This date argument is automatically analysed by the coupling library and depending on the coupling period and the lag LAG chosen by the user for each coupling field in the configuration file namcouple different coupling algorithms can be reproduced without modifying anything in the component model codes themselves With OASIS3 MCT the SEQ index is no longer
5. If such communicator does not exist yet in the code the component processes should use to create it e CALL oasis create couplcomm icpl local comm coupl comm kinfo e CALL prism create couplcomm icpl local comm coupl comm kinfo icpl INTEGER IN coupling process flag local comm INTEGER IN MPI communicator with all processes of the component coupl comm INTEGER OUT returned MPI communicator gathering only component processes participating in the coupling kinfo INTEGER OUT OPTIONAL returned error code This routine creates a coupling communicator for a subset of processes It must be called by all com ponent processes with icpl 1 for processes participating in the coupling and with icp1 MPI_UND for the others Argument 1ocal comm is the MPI communicator associated with all processes of the component The new coupling communicator is returned in coupl_comm argument and the internal coupling communicator is also set to that value If this communicator already exist in the code the component should use to provide it to OASIS3 MCT e CALL oasis set couplcomm coupl comm kinfo e CALL prism set couplcomm coupl comm kinfo coupl comm INTEGER IN MPI communicator gathering only component processes participating in the coupling kinfo INTEGER OUT OPTIONAL returned error code This routine allows
6. 3 5 H EH 3 OASIS3 MCT User Guide OASIS3 MCT_1 0 Edited by S Valcke T Craig L Coquart CERFACS CNRS SUC URA No1875 CERFACS TR CMGC 12 49 July 2012 Copyright Notice Copyright 2012 by CERFACS All rights reserved No parts of this document should be either reproduced or commercially used without prior agreement by CERFACS representatives How to get assistance Assistance can be obtained as listed below Phone Numbers and Electronic Mail Adresses Name Phone Affiliation e mail Laure Coquart CNRS oasishelp at cerfacs fr Sophie Valcke CERFACS How to get documentation The documentation can be downloaded from the OASIS web site under the URL http oasis enes org Contents 1 Introduction Step by step use of OASIS3 MCT OASIS3 MCT sources Licenses and Copyrights T4 1 2 1 3 2 1 2 2 2 3 2 4 2 5 2 6 2 7 2 8 2 9 2 10 3 1 32 33 4 1 1 3 1 1 3 2 MCT copyright statement 1 3 3 The SCRIP 1 4 license copyright statement 2 2 1 OASIS3 MCT license and copyright statement Interfacing a component model with OASIS3 MCT Use of OASIS3 MCT library Initialisation Coupling initialisation 2 2 2 Communicator for internal parallelisation 2 2 3 Coupling through a subset of the component model processes Grid data file definition Partition definition 2 4 1 2 4 3 2 7 1 Auxiliary routines 3 3 1 3 3 2 Sec
7. are collective and must be called by all processes New routines oasis get debug and oasis set debug New routines oasis get debug and oasis set debug are now available to respectively re trieve the current OASIS3 MCT internal debug level set by SNLOGPRT in the namcouple or to change it see section 2 9 Functionality not offered anymore SCRIPR BICUBIC and SCRIPR CONSERV SECOND remappings As in OASIS3 3 the SCRIP library can be used to generate the remapping interpolation weights and addresses and write them to a mapping file All previous SCRIPR remapping schemes available in OASIS3 3 are still supported besides BICUBIC and CONSERV SECOND because these remapping involve at each source grid point the value of the field but also the value of the gradients of the field which are not known or calculated See also section 4 3 Vector field remapping Vector field remapping is not and will not be supported see Support of vector fields with the SCRIPR remappings in section 4 3 Automatic calculation of grid mesh corners in SCRIPR CONSERV For SCRIPR CONSERV remapping grid mesh corners will not be compute automatically if they are needed but not provided Other transformations not supported The following transformations are not available in OASIS3 3 and will most probably not be implemented as it should be not too difficult to implement the equivalent operations in the component models th
8. To use OASIS3 MCT library a user needs to add in his code e USE mod_oasis e USE mod_prism Both use statements are valid and use of just one or the other is recommended in a particular component model A single use statement now provides all the methods that required multiple use statements in pre vious OASIS3 versions The methods datatypes and capabilities are identical for both the mod_prism or mod_oasis interfaces The only difference is the name of the interface The interface in module mod_prism is provided for backwards compatability with prior versions of OASIS3 Use of module mod_oasis is now recommended provides access to a set of updated routine names that will con tinue to evolve in the future always ensuring backward compatibility In the following sections both the mod_prism and mod_oasis interface names is defined and a single description of the interface arguments is provided 2 2 Initialisation 2 2 1 Coupling initialisation e CALL oasis_init_comp compid model_name ierror e CALL prism_init_comp_proto compid model name ierror I compid INTEGER OUT component model ID model name CHARACTER 6 IN component model name as in namcouple ierror INTEGER OUT returned error code This routine must called by all component processes to initialise the coupling 2 2 Communicator for internal parallelisation e CALL oasis get localcomm local comm ierror
9. f1d2 put f1d2 In model2 fid2 depends on fldl If this dependency is known then there is a benefit in using SEQ 1 for fld1 and SEQ 2 for fid2 At runtime if the sequencing of both modell and model do not match the above diagram the model will abort gracefully For instance if model2 has the dependency shown above but modell does not have consistent ordering of the put and get as required by model2 then if SEQ is unused the model will deadlock and hang If SEQ is set properly the coupling layer will detect that the required sequence has not been followed and will exit gracefully with an error message 22 CHAPTER 2 INTERFACING A COMPONENT MODEL WITH OASIS3 MCT Again the SEQ namecouple setting is only diagnostic and is not required Chapter 3 The configuration file namcouple The OASIS3 MCT configuration file namcouple contains below pre defined keywords all user s defined information necessary to configure a particular coupled run The namcouple is a text file with the following characteristics e the keywords used to separate the information can appear in any order e the number of blanks between two character strings is non significant e all lines beginning with are ignored and considered as comments e blank lines are not allowed The first part of namcouple is devoted to configuration of general parameters such as the number of models involved in the simulation or the number of fields The second part gat
10. model startup this is a bug fix new in OASIS3 MCT 2 7 2 Receiving a coupling or I O field In the model time stepping loop each process receives its part of the coupling field CALL oasis get var id date field array info CALL prism get proto var id date field array info var id INTEGER IN field ID from corresponding prism def var proto date INTEGER IN number of seconds in the run at the time of the call by convention at the beginning of the timestep field array REAL OUT I O or coupling field array info INTEGER OUT returned info code OASIS Recvd 23 if the field was received from another model OASIS FromRest 210 if the field was read from a restart file only OASIS Input 211 if the field was read from an input file only OASIS RecvOut 212 if the field was both received from another model and written to an output file OASIS FromRestOut 213 if the field was both read from a restart file and written to an output file OASIS OK 20 otherwise and no error occurred 2 8 TERMINATION 17 This routine may be called by the model at each timestep The date argument is automatically analysed and the receiving action is actually performed only if date corresponds to a time for which it should be activated given the period indicated by the user in the namcouple An exchange at the beginning of the run at time 0 is expected 2 8 Termination e CALL
11. CHAPTER 2 INTERFACING A COMPONENT MODEL WITH OASIS3 MCT Proc 1 Proc 2 Proc 3 local offset 0 local offset 4 local offset 10 local size 4 local size 6 local size 5 Figure 2 1 Apple partition It is assumed here that the index start at 0 in the upper left corner 2 4 PARTITION DEFINITION 13 Proc 1 Proc 2 local offset 0 local offset 2 local x extent 2 local x extent 3 local y extent 2 local y extent 2 global x extent 5 Proc 3 local offset 10 local x extent 5 local y extent 1 Figure 2 2 Box partition It is assumed here that the index start at 0 in the upper left corner 14 CHAPTER 2 INTERFACING A COMPONENT MODEL WITH OASIS3 MCT Proc 1 1 segment offset 0 E H a L nbr of segments 3 1 segment size 5 2nd segment offset 7 2nd segment size 2 a a 3rd segment offset 10 B B B 3rd segment size 3 Figure 2 3 Orange partition for one process It is assumed here that the index start
12. at 0 in the upper left corner 2 4 4 Orange partition Each partition is an ensemble of segments of the global domain Each segment is described by its global offset and its local extent In this case we have ig paral 1 N where N 2 2xnumber of segments e ig_paral 1 3 indicates a Orange partition e ig paral 2 the total number of segments for the partition limited to 200 presently see note for ig paral 4 for Box partition above e ig paral 3 the first segment global offset e ig paral 4 the first segment local extent e ig paral 5 the second segment global offset 6 e ig paral the second segment local extent e ig paral N 1 the last segment global offset e ig paral N the last segment local extent Figure 2 3 illustrates an Orange partition with 3 segments for one process The other process partitions are not illustrated 2 5 Coupling field declaration All component processes declares the fields sent or received by the component during the simulation This is true even for processes not implied in the coupling 2 6 END OF DEFINITION PHASE 15 e CALL oasis def var var_id name il part id var nodims var actual shape var type ierror or kinout e CALL prism def var proto var id name il part id var_nodims kinout var actual shape vartype ierror var id INTEGER OUT coupling field ID name CHARACTER 8 IN field symbolic name a
13. e CALL ierror oasis terminate ierror prism terminate proto ierror INTEGI ER OUT returned error code All processes of the component model must terminate the coupling by calling this routine normal termination 2 9 Auxiliary routines Not all auxiliary routines that were in OASIS3 3 are currently available e CALL e CALL compid oasis abort compid routine name abort message prism abort proto compid routine name abort message INTEG ER IN component model ID from oasis init comp or prism init comp proto routine_name CHARACTER IN name of calling routine abort message CHARACTER IN message to be written out If a process needs to abort voluntarily it should do so by calling oasis abort prism abort proto This will ensure a proper termination of all processes in the coupled model communicator This routine writes the name of the calling model the name of the calling routine and the message to the process debug file see SNLOGPRT in section 3 2 This routine cannot be called before prism init comp proto e CALL e CALL debug value oasis get debug debug value prism get debug debug value INTEGER OUT debug value This routine may be called at any time to retrieve the current OASIS3 MCT internal debug level see SNLOGPRT in section 3 2 This is useful if the user wants to return the
14. en de Recherche et Formation Avanc e en Calcul Scientifique CERFACS 4 CHAPTER 1 INTRODUCTION This software and ancillary information called OASIS3 MCT is free software CERFACS has rights to use reproduce and distribute OASIS3 MCT The public may copy distribute use prepare derivative works and publicly display OASIS3 MCT under the terms of the Lesser GNU General Public License LGPL as published by the Free Software Foundation provided that this notice and any statement of authorship are reproduced on all copies If OASIS3 MCT is modified to produce derivative works such modified software should be clearly marked so as not to confuse it with the OASIS3 MCT version available from CERFACS The developers of the OASIS3 MCT software are researchers attempting to build a modular and user friendly coupler accessible to the climate modelling community Although we use the tool ourselves and have made every effort to ensure its accuracy we can not make any guarantees We provide the software to you for free In return you the user assume full responsibility for use of the software The OASIS3 MCT software comes without any warranties implied or expressed and is not guaranteed to work for you or on your computer Specifically CERFACS and the various individuals involved in development and maintenance of the OASIS3 MCT software are not responsible for any damage that may result from correct or incorrect use of this software 1 3 2 MCT
15. happen and to exit grace fully see section 2 10 2 The I O library mpp io is no longer used to write the restart and output files Appendix C Coupled models realized with OASIS Table C 1 Use of OASIS3 centres coupled models and computing platforms Centre Coupled model Platform CERFACS FR CERFACS FR CERFACS FR CERFACS FR M t o France FR M t o France FR M t o France FR M t o France FR IPSL FR IPSL FR OMP FR LGGE FR ECMWF MPI M DE MPI M DE Met Office UK Met Office UK NCAS Reading UK NCAS Reading UK IFM GEOMAR DE CMCC IT CMCC IT CMCC IT CMCC IT ENEA IT LMNCP IT SMHI SE SMHI SE KNMI NL KNMI NL DMI DK U Bergen NO ICHEC IE ICHEC IE NUI Galway IE ARPEGE4_T63 NEMO ORCA2 LIM TRIP ARPEGE_T63 NEMIX ORCA2 ARPEGE_T359 NEMO ORCA0O12 ARPEGE_T799 NEMIX ORCA025 ARPEGE_V4 6 NEMO ORCA2 NEMOmed8 ALADIN Climat NEMOmed8 TRIP ARPEGE V5 NEMOV3 ORCA 1 TRIP ARPEGE V5 1 NEMOI LMDZ NEMO ORCAJ LMDZmed NEMOmed8 LMDz 144x142 NEMO ORCA2 MESO NH S YMPHONIE MAR NEMO LIM IFS_T399 NEMO ORCAI1 ECHAMS MPIOM REMO MPIOM UM Atm 192x145 NEMO ORCAI UM Atm 432x325 NEMO ORCA025 ECHAM4 NEMO ORCA2 HadAM3 NEMO ORCA2 ECHAMS T63 NEMO ORCA2 ECHAMS T31L31 0PA8 2 ORCA2 ECHAMS T159L31 0PA8 2 ORCA2 ECHAMS T63L95 OPA8 2 ORCA2 ECHAMS T159 OPAS8 2 ORCAJ NEMOMed 1 16 RegCM MITgcm WRF ROMS EC Earth IFS_T159 NEMO ORCA1 EC
16. needed in the namcouple input to specify the sequencing order of different coupling fields The sequence SEQ index in the namcouple file now provides the coupling layer with an ability to detect a deadlock before it happens and exit The lag concept and indices are explained in more detail below and some examples are provided 2 10 1 Thelag concept The lag LAG value tells the coupler to modify the time at which that data is sent put by the amount of the lag The lag must be expressed in number of seconds and can be positive or negative but should never be larger in absolute magnitude than the coupling period of any field due to problems with restart ability and dead locking When a component model calls a oasis put prism put proto the value of the lag is automatically added to the value of the date argument and the put is actually performed when the sum date lag is a coupling time in the target component this put will match a oasis get prism get proto for which the date argument is the same coupling time The lag only shifts the time data is sent It cannot be used to shift the time data is received yet When the lag is positive a restart file must be available to initiate the coupling and in those cases the restart file is then updated at the end of the run A positive lag acts like a send occurred before the model started In fact for a field with positive lag the source component model automatically reads the field in the r
17. original debug value after changing it or if a user wants to key off the oasis debug level for model debug diagnostics e CALL e CALL debug value oasis set debug debug value prism set debug debug value INTEGER IN debug value This routine may be called at any time to change the debug level in oasis This method allows users to vary the debug level at different points in the model integration e CALL e CALL new comm INT cdnam CHARACTER IN other model name oasis get in prism get in tercomm new comm cdnam kinfo tercomm new comm cdnam kinfo EG ER OUT mpi intercomm communicator If the process called MPI Init before calling oasis init comp or prism init comp proto it must also call MPI_Finalize explicitly but only after calling oasis terminate proto orprism terminate proto 18 CHAPTER 2 INTERFACING A COMPONENT MODEL WITH OASIS3 MCT kinfo INTEGER OUT OPTIONAL returned error code This routine sets up an MPI intercomm communicator between the root processors of two compo nents the local component and the component associated with cdnam This must be called by both components at the same time otherwise a deadlock will occur In addition this call is collective across the tasks of the two components but other components are not involved e CALL oasis get intracomm new comm cdnam kinfo
18. quality of the remapping for any specific couple of grids e Coupling field CF standard names The file cf name table txt is not needed or used anymore The CF standard names of the coupling fields are not written to the debug files e Binary auxiliay files All auxiliary files besides the namcouple must be NetCDF binary files are not supported anymore B 4 New functionality offered Better support of components for which only a subset of processes participate in the coupling In OASIS3 3 components for which only a subset of processes participated in the coupling were supported in a very restricted way In fact this subset had to be composed of the N first processes and N had to be specified in the namcouple Now the subset of processes can be composed of any of the component processes and does not have to be pre defined in the namcouple New routines oasis_create_couplcomm and oasis set couplcomm are now available to create or set a coupling communicator gathering only these processes see section 2 2 3 Exact restart for LOCTRANS transformations If needed LOCTRANS transformations write partially transformed fields in the coupling restart file at the end of a run to ensure an exact restart of the next run see section 4 1 Therefore fields of type OUTPUT now support a restart file specification in the namcouple to be used for LOCTRANS transformations Support to couple multiple fields via a single communication This is suppor
19. sstoc nc sstat nc EXPORTED 182 149 P2P 0 128 64 toce atmo LAG 14400 SEQ 1 where the different entries are e Field first line SOSSTSST symbolic name for the field in the source model CHARACTER 8 It has to match the argument name of the corresponding field declaration in the source model see oasis_def_var or prism_def_var_proto in section 2 5 SISUTI ESU symbolic name for the field in the target model CHARACTI ER 8 It has to match the argument name of the corresponding field declaration in the target model see oasis_def_var or prism_def_var_proto in section 2 5 1 UNUSED but still required for parsing 86400 coupling and or I O period for the field in seconds 5 number of transformations to be performed by OASIS3 on this field sstoc nc name of the coupling restart file for the field CHARACTI no coupling restart file is effectively used for more detail see section 5 2 Sstat nc UNUSED but still required for parsing ER 8 mandatory even if 3 3 SECOND SECTION OF NAMCOUPLE FILE 27 EXPORTED field status e Field second line 182 number of points for the source grid first dimension optional 149 number of points for the source grid second dimension optional 128 number of points for the target grid first dimension optional 64 number of points for the target grid second dimension option
20. 21 overlays the grid cells with i imax the latter must be masked If this is not the case given the mask defined in masks nc OASIS3 MCT must be compiled with the CPP key TREAT OVERLAY which will ensure that these rules are respected This CPP key was introduced in OASIS3 3 A target grid cell intersecting no source cell either masked or non masked at all i e falling in a hole of the source grid will in all cases get a zero value Ifa target grid cell intersects only masked source cells it will still get a zero value unless the FRACNNEI normalisation option is used in which case it will get the nearest non masked neighbour value Note that the option of having the value 1 0E 20 assigned to these target grid cell intersecting only masked source cells for easier identification is not yet availble in OASIS3 MCT Support of vector fields with the SCRIPR remappings Vector mapping is NOT supported is not and will not be supported by OASIS3 MCT For proper treatment of vector fields the component model has to send the 3 components of the vector projected in a Cartesian coordinate system INTERP UNUSED MOZAIC UNUSED note that MAPPING see above is the NetCDF equivalent to MOZAIC NOINTERP UNUSED FILLING UNUSED The post processing stage CONSERV CONSERV ensures a global modification of the coupling field This analysis requires the source and target grid mesh areas to be transfered to the coupler with oasis wri
21. CALL oasis write grid cgrid nx e CALL prism write grid cgrid nx ny lon lat or ny lon lat cgrid CHARACTER 4 IN grid name prefix see 3 3 nx INTEGER IN first grid dimension x ny INTEGER IN second grid dimension y lon REAL DIMENSION nx ny lat REAL DIMENSION nx ny IN array of longitudes degrees East IN array of latitudes degrees North Writes the model grid longitudes and latitudes Longitudes must be given in degrees East in the interval 360 0 to 720 0 Latitudes must be given in degrees North in the interval 90 0 to 90 0 Note that if some grid points overlap it is recommended to define those points with the same number e g 90 0 for both not 450 0 for one and 90 0 for the other to ensure automatic detection of overlap by 10 CHAPTER 2 INTERFACING A COMPONENT MODEL WITH OASIS3 MCT OASIS3 MCT which is essential to have a correct conservative remapping SCRIPR CONSERV see section 4 3 CALL oasis write corner nx cgrid ny nc clon clat or ny CALL prism write corner cgrid nx nc clon clat cgrid CHARACTER 4 INTEGER ny INTEGER nc INTEGER lon REAL lat REAL IN grid name prefix nx IN first grid dimension x IN second grid dimension y IN number of corners per grid cell always 4 in the version
22. CALL MPI COMM Split localComm icpl 1 couplComm ierror provide this communicator to OASIS3 MCT CALL The oasis set couplcomm couplComm ierror call to MPI COMM Split and oasis set couplcomm could be replaced by CALL oasis create couplcomm icpl localComm couplComm ierror CALL oasis def partition CALL oasis def var CALL oasis enddef do loop if couplingpe CALL oasis put if couplingpe CALL oasis get enddo CALL oasis terminate 2 3 Grid data file definition With OASIS3 MCT the grid data files grids nc masks nc and areas nc are required only for certain op erations see also section 5 1 i e grids nc and masks nc for SCRIPR see section 4 3 and masks nc and areas nc for CONSI ERV see section 4 4 These grid data files can be created by the user before the run or can be written directly at run time by the master process of each component model with the following routines These routines can be used by the component models to add grid fields to the grid files but grid fields are never overwritten in the grid files These routines have to be called just after prism init comp e CALL oasis start grids writing f e CALL prism start grids writing f flag INTEGER OUT not used lag or lag Obsolete in OASIS3 MCT exists however for upward compatibility e
23. Earth IFS_T799 NEMO ORCA025 ECHAMS MPIOM EC Earth IFS_T159 NEMO ORCA2 ECHAM global HIRLAM reg MMS ROMS EC Earth IFS_T159 NEMO ORCA1 ROMS WRE ECHAMS REMO MPI OM Linux Cluster CRAY XD1 VPP5000 NEC SX6 SX8 CRAY XDI NEC SX9 Bullx Altix ICE NEC SX8 NEC SX9 NEC SX9 IBM Power 6 NEC SX8 Linux Opteron cluster IBM Power 6 IBM Power 6 SUN Linux IBM Power 575 IBM Power6 NEC SX6 IBM Power6 NEC SX6 SX8 NEC SX8 NEC SX6 SX8 SX9 NEC SX9 IBM Power6 IBM SP5 Linux Cluster Ekman AMD Opteron NEC SX 8 SGI Altix IBM Power NEC SX6 SGI Altix ICE Bull clust Continued on next page 45 Table C 1 continued from previous page Centre Coupled model Platform ETH CH COSMO CLM CLM ETH CH COSMO CLM ROMS U Castille ES PROMES U Madrid ocean IBM Power 6 Intel Itanium NHM Service RS ECHAMS MPIOM Linux Fedora core 14 CMC CA GEM NEMO IBM Power5 UQAM CA GEMDM 3 3 2 NEMOv 2 3 0 Linux PC MM MA ARPEGE Climat_V5 1 NEMO ORCA1 IBM INMT TN ARPEGE Climat_V5 1 NEMO ORCA2 IBM Regata Series Oregon St U USA PUMA UVic Linux cluster Hawaii U USA ECHAM4 POP Linux cluster JAMSTEC JP ECHAM OPA8 2 NEC SX8 JAMSTEC JP ECHAMS T106L31 NEMO_0 5 NEC SX8 Met Nat Center CN GRAPES 201x161 ECOM si IBM cluster TAP CN CREM reg POM2000 SGI IAP CN ECHAM MPIOM Linux x64 CSIRO AU ACCESS UMv7 3 MOMApI CICE SUN SGI clusters BoM AU BAM3 ACOM2 NEC SX6 SUN BoM AU
24. FACS Toulouse France 2004 Valcke et al 2003 Valcke S A Caubel D Declat and L Terray OASIS3 Ocean Atmosphere Sea Ice Soil User s Guide Technical Report TR CMGC 03 69 CERFACS Toulouse France 2003 Valcke et al 2000 Valcke S L Terray and A Piacentini OASIS 2 4 Ocean Atmosphere Sea Ice Soil User s Guide and Reference Manual Technical Report TR CMGC 00 10 CERFACS Toulouse France 2000
25. MODE UNUSED SCHANNEL UNUSED SNFIELDS On the line below this keyword is the total number of field entries in the second part of the namcouple If more than one field are described on the same line new in OASIS3 MCT see section B this counts as only one entry SJOBNAME UNUSED SNBMODEL On the line below this keyword is the number of models running in the given experi ment followed by CHARACTERx6 variables giving their names which must correspond to the name announced by each model when calling oasis init comporprism init comp proto sec ond argument see section 2 2 Then the user may indicate on the same line the maximum Fortran unit number used by the models In the example Fortran units above 55 70 and 99 are free for respectively the ocean atmosphere and atmospheric chemistry models In all cases OASIS3 MCT library assumes during the initialization phase that units 1025 and 1026 are free and temporarily uses these units to read the namcouple and to write corresponding log messages to file nout 000000 After the initialization phase OASIS3 MCT will still suppose that units above 1024 are free unless maximum unit numbers are indicated here in the namcouple SRUNTIME On the line below this keyword is the total simulated time of the run expressed in seconds SINIDATE UNUSED SMODINFO UNUSED SNLOGPRT The line below this keyword refers to the amount of information that will be written to the OASIS3 MCT d
26. Power3 JAMSTEC JP ECHAMA OPA 8 2 NEC SX5 BMRC AU BAMG ACOM2 NEC SX6 Bibliography Cassou et al 1998 Cassou C P Noyret E Sevault O Thual L Terray D Beaucourt and M Imbard Dis tributed Ocean Atmosphere Modelling and Sensitivity to the Coupling Flux Precision the CATHODe Project Monthly Weather Review 126 No 4 1035 1053 1998 Guilyardi et al 1995 Guilyardi E G Madec L Terray M D qu M Pontaud M Imbard D Stephenson M A Filiberti D Cariolle P Delecluse and O Thual Simulation coupl e oc an atmosph re de la variabilit du climat C R Acad Sci Paris t 320 s rie IIa 683 690 1995 Jacob et al 2005 Jacob R J Larson and E Ong MxN Communication and Parallel Interpolation in CCSM3 Using the Model Coupling Toolkit Int J High Perf Comp App 19 3 293 307 2005 Jones 1999 Jones P Conservative remapping First and second order conservative remapping Mon Weather Rev 127 2204 2210 1999 Larson et al 2005 Larson J R Jacob and E Ong The Model Coupling Toolkit A New Fortran90 Toolkit for Building Multiphysics Parallel Coupled Models Int J High Perf Comp App 19 3 277 292 2005 Noyret et al 1994 Noyret P E Sevault L Terray and O Thual Ocean atmosphere coupling Proceedings of the Fall Cray User Group CUG meeting 1994 Pontaud et al 1995 Pontaud M L Terray E Guilyardi E Sevault D B Stephenson and O Thual C
27. RPOLATIONS SMAPSTRATEGY is optional and can be either bfb sum or opt In b b mode the map ping is done using a strategy that produces bit for bit identical results regardless of the grid decompositions without leveraging a partial sum computation With sum the transform is done using the partial sum approach which generally introduces roundoff level changes in the results on different processor counts Option opt allows the coupling layer to choose either approach based on an analysis of which strategy is likely to run faster Usually partial sums will be used if the source grid has a higher resolution than the target grid as this should reduce the overall communication Note that if SCRIPR see below is used to calculate the remapping file MAPPING can still be listed in the namcouple to specify a name for the remapping file generated by SCRIPR different from the default and or to specify a SMAPLOC or SMAPSTRATEGY option SCRIPR SCRIPR gathers the interpolation techniques offered by Los Alamos National Laboratory SCRIP 1 4 library Jones 1999 SCRIPR routines are in oasis3 mct lib scrip See the SCRIP 1 4 documentation in oasis3 doc SCRIPusers pdf for more details on the interpolation algorithms In the current implementation each pair of source and target points in the MAPPING file can be linked by only one weight i e remappings such as SCRIPR BICUBIC involving at each source grid point the value of the field of the gradien
28. SED EXPORTED exchanged between component models and transformed by OASIS3 MCT EXPOUT exchanged transformed and also written to two debug NetCDF files one before the sending action in the source model below the oasis put or prism put proto call after local ns LOCTRANS and BLASOLD if present and one after the receiving action in the target model below the prism get proto call after all transformations This status should be used when debugging the coupled model only The name of the debug NetCDF file one per field is automatically defined based on the field and component model names transformatio IGNORED with OASIS3 MCT this setting is equivalent to and converted to EXPORTED IGNOUT with OASIS3 MCT this setting is equivalent to and converted to EXPOUT INPUT read in from the input file by the target model below the oasis getorprism get proto call at appropriate times corresponding to the input period indicated by the user in the namcouple See section 5 3 for the format of the input file OUTPUT written out to an output debug NetCDF file by the source model below the oasis put or prism put proto call after local transformations LOCTRANS and BLASOLD at appropriate times corresponding to the output period indicated by the user in the namcouple 3 3 1 Second section of namcouple for EXPORTED and EXPOUT fields The first 3 lines for fields with status EXPORTED and EXPOUT are as follows SOSSTSST SISUTESU 1 86400 5
29. STATES DEPARTMENT OF ENERGY AND THEIR EMPLOYEES 1 DISCLAIM ANY WARRANTIES EXPRESS OR IMPLIED INCLUDING BUT NOT LIMITED TO ANY IMPLIED WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE TITLE OR NON IN FRINGEMENT 2 DO NOT ASSUME ANY LEGAL LIABILITY OR RESPONSIBILITY FOR THE ACCURACY COMPLETENESS OR USEFULNESS OF THE SOFTWARE 3 DO NOT REPRESENT THAT USE OF THE SOFTWARE WOULD NOT INFRINGE PRIVATELY OWNED RIGHTS 4 DO NOT WARRANT THAT THE SOFTWARE WILL FUNCTION UNINTER RUPTED THAT IT IS ERROR FREE OR THAT ANY ERRORS WILL BE CORRECTED 1 3 LICENSES AND COPYRIGHTS 5 5 LIMITATION OF LIABILITY IN NO EVENT WILL THE COPYRIGHT HOLDER THE UNITED STATES THE UNITED STATES DEPARTMENT OF ENERGY OR THEIR EMPLOYEES BE LIABLE FOR ANY INDIRECT INCIDENTAL CONSEQUENTIAL SPECIAL OR PUNITIVE DAMAGES OF ANY KIND OR NATURE INCLUDING BUT NOT LIMITED TO LOSS OF PROFITS OR LOSS OF DATA FOR ANY REASON WHATSOEVER WHETHER SUCH LIA BILITY IS ASSERTED ON THE BASIS OF CONTRACT TORT INCLUDING NEGLIGENCE OR STRICT LIABILITY OR OTHERWISE EVEN IF ANY OF SAID PARTIES HAS BEEN WARNED OF THE POSSIBILITY OF SUCH LOSS OR DAMAGES 1 3 3 The SCRIP 14 license copyright statement The SCRIP 1 4 copyright statement reads as follows Copyright 1997 1998 the Regents of the University of California This software and ancillary infor mation herein called SOFTWARE called SCRIP is made available under the terms described here The SOFTWARE
30. TCLAPS MOM NEC SX 6 U Tasmania AU Data atm model MOM4 SGI O3400 Compaq5 Table C 2 Use of OASIS2 centres coupled models and computing platforms Centre Coupled model Platform CERFACS FR ARPEGE3 OPA8 2 LIM 2deg VPP5000 CERFACS FR ARPEGE3 OPA8 1 VPP700 CERFACS FR ARPEGE2 OPAICE CRAY C90 IPSL FR LMDz 96x71x19 OPA ORCA2 VPP5000 IPSL FR LMDz 72x45x19 OPA ORCA4 VPP5000 IPSL FR LMDZ 120X90 OPA ATL3 1 3 IPSL FR LMDZ 120X90 OPA ATLI 1 deg LODYC FR IFS T195L31 OPA8 1 LODYC FR ECHAMA T30 T42 L14 OPA ORCA2 M t o France FR ARPEGE medias OPAmed 1 8 deg VPP5000 M t o France FR ARPEGE3 OPA 8 1 Gelato VPP5000 M t o France FR ARPEGE 2 T31L19 OPA8 TDH CRAY J90 ECMWF IFS T63 T255 E HOPE 2deg 1deg IBM Power 4 ECMWF IFS Cy23r4 T159L40 E HOPE 256L29 VPP700 ECMWF IFS Cy23r4 T95LA0 E HOPE 256L29 VPP700 ECMWF IFS Cy15r8 T63L31 E HOPE 128L20 VPP300 MPI DE ECHAMS T42 L19 C HOPE T42 L20 NEC SX MPI DE PUMAT42 L19 C HOPE 2deg GIN NEC SX MPI DE EMAD E HOPE T42 L20 CRAY C 90 MPI DE ECHAMS T42 L19 E HOPE T42 L20 NEC SX MPI DE ECHAMA T30 L19 E HOPE T42 L20 CRAY T90 MPI DE ECHAMA T30 L19 E HOPE T42 L20 CRAY C90 SMHI SE RCA HIRLAM reg RCO OCCAM reg CGAM UK HadAM3 2 5x3 75 L20 OPA ORCA2 T3E SOC UK Interm Atm GCM OCCAM Lite NOC UK FORTE IGCM3 128 x 64 MOMA 180 x 88 JPL USA TRIDENT QTCM Linux workstation IRI USA ECHAM4 MOM3 SGI Origin IBM
31. ag needs to be defined the coupling field produced by model A at the coupling timestep can be consumed by model B without causing a deadlock situation As in the first example the oasis get prism get proto performed at the beginning of the run for F and F gt will automatically receive data read from their coupling restart files and the last oasis put prism put proto performed at the end of the run automatically write them to their coupling restart file For F5 no coupling restart file is needed nor used as at each coupling period the coupling field produced by model A can be directly consumed by model B We see here how the introduction of appropriate LAG indices results in get in the target model coupling fields produced by the source model the timestep before this is in some coupling config urations essential to avoid deadlock situations 2 10 2 The sequence concept The order of coupling operations in the system is determined solely by the order of calls to send put and receive get data in the models in conjunction with the setting of the lag in the namcouple Data that is received get is always blocking while data that is sent put is non blocking with respect to the model making that call It is possible to deadlock the system if the relative orders of puts and gets in different models are not compatible With OASIS3 MCT the sequence SEQ index in the namcouple file now provides the coupling layer with an abil
32. al toce prefix of the source grid name in grid data files see section 5 1 CHARACTER x4 atmo prefix of the target grid name in grid data files CHARACTER 4 LAG 14400 optional lag index for the field expressed in seconds SEQ 1 optional sequence index for the field see section 2 10 e Field third line P source grid first dimension characteristic P periodical R regional 2 source grid first dimension number of overlapping grid points P target grid first dimension characteristic P periodical R regional 0 target grid first dimension number of overlapping grid points The fourth line gives the list of transformations to be performed for this field In addition there is one or more configuring lines describing some parameters for each transformation These additional lines are described in more details in the chapter 4 3 3 2 Second section of namcouple for OUTPUT fields The first 2 lines for fields with status OUTPUT are as follows COSHFTOT COSHFTOT 7 86400 0 fldhftot nc OUTPUT atmo atmo where the different entries are as for EXPOUT fields except that the source symbolic name must be re peated twice on the field first line the restart file name is needed only if LOCTRANS transformations are present there is no output file name on the first line and no grid dimensions and no LAG or SEQ index on the second line The name of the output file i
33. ang SGI Germany Li Yan CERFACS We also would like to thank the following people for their help and suggestions in the design of the OASIS software in alphabetical order with the name of their institution at the time of their contribution to OASIS Dominique Astruc IMFT Chandan Basu NSC Sweden Sophie Belamari M t o France Dominique Bielli M t o France Gilles Bourhis IDRIS Pascale Braconnot IPSL LSCE Sandro Calmanti M t o France Christophe Cassou CERFACS Yves Chartier RPN Jalel Chergui IDRIS Philippe Courtier M t o France Philippe Dandin M t o France Michel D qu M t o France Ralph Doescher SMHI Jean Louis Dufresne LMD Jean Marie Epitalon CERFACS Laurent Fairhead LMD Uwe Fladrich SMHI Marie Alice Foujols IPSL Gilles Garric CERFACS Eric Guilyardi CERFACS Charles Henriet CRAY France Pierre Herchuelz ACCRDI Maurice Imbard M t o France Luis Kornblueh MPI M Stephanie Legutke MPI M amp D Claire L vy LODYC Olivier Marti IPSL LSCE S bastien Masson IPSL LOCEAN Claude Mercier IDRIS Pascale Noyret EDF Andrea Piacentini CER FACS Marc Pontaud M t o France Adam Ralph ICHEC Ren Redler MPI M Tim Stockdale ECMWF Rowan Sutton UGAMP V ronique Taverne CERFACS Jean Christophe Thil UKMO Nils Wedi ECMWF Chapter 1 Introduction In 1991 CERFACS started the development of a software interface to couple existing numerical Gen
34. approximation see details in chapter 4 of SCRIP 1 4 documentation SCRIPusers pdf For BILINEAR Logically Rectangular LR and Reduced D source grid types are sup ported x Masked target grid points the zero value is associated to masked target grid points Non masked target grid points having some of the source points normally used in the bilinear or bicubic interpolation masked a N nearest neighbour algorithm using the re maining non masked source points is applied Non masked target grid points having all bilinear neighbours masked by default the nearest non masked source neighbour is used logical 11 nnei hard coded to t rue in oasis3 mct lib scrip src remapbilinear F this is NOT the same default behaviour as OASIS3 3 to have the same default behaviour as in OASIS3 3 put ll nnei false The configuring line is CHAPTER 4 TRANSFORMATIONS AND INTERPOLATIONS SCRIPR for BILINEAR SCMETH CGRS SCFTYP SREST SNBIN where SCMETH BILINEAR SCGRS is the source grid type LR or D SCFTYP SNBIN are as for DISTWGT SREST is as for DISTWGT except that only LATITUDE is possible for a Reduced D source grid BICUBIC as opposed to OASIS3 3 OASIS3 MCT does not currently support this map ping option because it is higher order CONSERV performs 1st order conservative remapping which means that the weight of a source cell is proportional to area intersected by the target ce
35. area cgrid nx area cgrid CHARACTER 4 INTEGER INTEGER REAL IN grid name prefix nx IN first grid dimension x IN second grid dimension y DIMENSION nx ny ny I area IN array of grid cell areas Writes of the model grid cell areas Needed only for CONSERV operation see section 4 4 DIMENSION nx ny nc IN array of corner longitudes in degrees East DIMENSION nx ny nc IN array of corner latitudes in degrees North CALL prism_terminate_grids_writing CALL oasis terminate grids wri ting Terminates grids writing required if some of the above grid writing routines are called 2 4 PARTITION DEFINITION 11 2 4 Partition definition The coupling fields sent or received by a component model are usually scattered among the different component processes With OASIS3 MCT all processes exchanging coupling data have to define their local partition in the global index space If only a subset of the processes actually exchange coupling data the processes not implied in the coupling have to call this routine to describe a null partition i e with ig paral 0 e CALL oasis def partition il part id ig paral ierror or e CALL prism def partition proto il part id ig paral ierror il part id INTEGER OUT partition ID ig paral INTEGER DIMENSION IN vector of integers describi
36. ated grid pair For first order mapping this is 1 This field in a netCDF dimension src address is a one dimensional array of size num links It contains the integer source address associated with each weight This field is a netCDF variable dst address is a one dimensional array of size num links It contains the integer destination address associated with each weight This field is a netCDF variable remap matrix is a two dimensional array of size num links num wgts It contains the real weight value s associated with the source and destination address This field is a netCDF variable Chapter 6 Compiling running and debugging 6 1 Compiling OASIS3 MCT Compiling OASIS3 MCT can be done in directory oasis3 MCT util make dir with Makefile TopMakefileOasis3 which must be completed with a header file make your platform specific to the compiling platform used and specified in oasis3 MCT util make_dir make inc One of the header files distributed with the release can by used as a template The root of the OASIS3 MCT tree can be anywhere and must be set in the variable COUPLE in the make your platform file The following commands are available e make f TopMakefileOasis3 ormake oasis3_psmile f TopMakefileOasis3 for backward compatibility compiles all OASIS3 MCT libraries mct scrip and psmile e make realclean f TopMakefileOasis3 removes all OASIS3 MCT compiled sources and librairies Log and error messages from compila
37. copyright statement Modeling Coupling Toolkit MCT Software Copyright 2011 UChicago Argonne LLC as Operator of Argonne National Laboratory All rights re served Redistribution and use in source and binary forms with or without modification are permitted provided that the following conditions are met 1 Redistributions of source code must retain the above copyright notice this list of conditions and the following disclaimer 2 Redistributions in binary form must reproduce the above copyright notice this list of conditions and the following disclaimer in the documentation and or other materials provided with the distribution 3 The end user documentation included with the redistribution if any must include the following acknowledgment This product includes software developed by the UChicago Argonne LLC as Operator of Argonne National Laboratory Alternately this acknowledgment may appear in the software itself if and wherever such third party acknowledgments normally appear This software was authored by e Argonne National Laboratory Climate Modeling Group Mathematics and Computer Science Division Argonne National Laboratory Argonne IL 60439 e Robert Jacob tel 630 252 2983 E mail jacob mcs anl gov e Jay Larson E mail larson mcs anl gov e Everest Ong e Ray Loy 4 WARRANTY DISCLAIMER THE SOFTWARE IS SUPPLIED AS IS WITHOUT WARRANTY OF ANY KIND THE COPYRIGHT HOLDER THE UNITED STATES THE UNITED
38. coupling restart values or input data values or the remapping weights and addresses 5 1 Grid data files With OASIS3 MCT the grid data files grids nc masks nc and areas nc are required only for certain oper ations i e grids nc and masks nc for SCRIPR see section 4 3 and masks nc and areas nc for CONSERV see section 4 4 These grid data files can be created by the user before the run or can be written directly at run time by the master process of each component model using the grid data definition routines see section 2 3 These routines can be used by the component models to add grid fields to the grid files but grid fields are never overwritten in the grid files These files are netCDF The arrays containing the grid information are dimensioned nx ny where nx and ny are the grid first and second dimension Unstructured grids are supported by setting the ny dimension to 1 and then nx is the total number of grid points 1 grids nc contains the model grid longitudes and latitudes in single or double precision REAL arrays depending on OASIS3 MCT compilation options The array names must be composed of a prefix 4 characters given by the user in the namcouple on the second line of each field see section 3 3 and of a suffix 4 characters this suffix is lon or lat for respectively the grid point longitudes or latitudes If the SCRIPR CONSERV remapping is used longitudes and latitudes for the source and target
39. d where SAVALUE is the scalar to be added to the field CONSTANT SAVALUE The remapping interpolation MAPPING The MAPPING keyword is used to specify an input file to be read and used for mapping ie regrid ding or interpolation the MAPPING file must follow the SCRIPR format This is an alternative method to SCRIPR for setting the mapping file In the current implementation each pair of source and target points in the MAPPING file can be linked by only one weight i e remappings such as SCRIPR BICUBIC involving at each source grid point the value of the field of the gradients and the cross gradient or second order conservative remapping are not supported Besides this restriction MAPPING can be used with any mapping file produced by any regridding algorithm This transformation requires at least one configuring line with one filename and two optional string values SMAPNAME SMAPLOC MAPSTRATEGY SMAPNAME is the name of the mapping file to read This is a netcdf file consistent with the SCRIPR map file format see section 5 3 SMAPLOC is optional and can be either src or dst With src the mapping will be done in parallel on the source processors before communication to the destination model and proces Sors this is the default With dst the mapping is done on the destination processors after the data is sent from the source model on the source grid 30 CHAPTER 4 TRANSFORMATIONS AND INTE
40. e CALL oasis put var id date field array info e CALL prism put proto var id date field array info var_id INTEGER IN field ID from corresponding oasis def var prism def var proto see section 2 5 date INTEGER IN number of seconds in the run at the time of the call by convention at the beginning of the timestep field array REAL IN coupling or I O field array info INTEGER OUT returned info code OASIS Sent 24 if the field was sent to another model OASIS _LocTrans 5 if the field was only used in a time transformation not sent not output Parameters OASIS In PRISM In mct lib psmile src mod oasis parameters F90 OASIS Out PRISM Out are defined in oasis3 16 CHAPTER 2 INTERFACING A COMPONENT MODEL WITH OASIS3 MCT OASIS ToRest 26 if the field was written to a restart file only OASIS Output 27 if the field was written to an output file only OASIS SentOut 28 if the field was both written to an output file and sent to another model directly or via OASIS3 main process OASIS ToRestOut 9 if the field was written both to a restart file and to an output file OASIS OK 20 otherwise and no error occurred To ensure a proper use of the oasis put prism put proto one has to take care of the following aspects This routine may be called by the model at each timestep The convention for the date argument is
41. e fields must be provided on the source grid in single or double precision REAL arrays and as the grid data files must be dimensioned nx ny where nx and ny are the grid first and second dimension except for fields given on unstructured for which the arrays are dimensioned nt 1 where nt is the total number of grid points The shape and orientation of each restart field and of the corresponding coupling fields exchanged during the simulation must be coherent with the shape of its grid data arrays 5 3 Input data files Fields with status INPUT in the namcouple will at runtime simply be read in from a NetCDF input file by the target model below the oasis get or prism get proto call at appropriate times corresponding to the input period indicated by the user in the namcouple The name of the file must be the one given on the field first configuring line in the namcouple see sec tion 3 3 3 There must be one input file per INPUT field containing a time sequence of the field in a single or double precision REAL array named with the field symbolic name in the namcouple and dimen sioned nx ny time or nbr pts 1 time The time variable has to be an array t ime time expressed in seconds since beginning of run The time dimension has to be the unlimited dimension 5 4 Transformation auxiliary data files 5 4 1 Files containing the remapping weights and addresses for MAPPING The mapping files to be read using the MAPPING
42. ebug files for each model and processor The value of NLOGPRT may be 0 one file debug root xx is open by the master proces of each model and one file de bug notroot xx is open for all the other processes of each model to write only error infor mation 1 one file debug root xx is open by the master proces of each model to write information equivalent to level 10 see below one file debug notroot xx is open for all the other processes of each model to write error information 2 one file debug xx xxxxxx is open by each process of each model to write normal production diagnostics 5 as for 2 with in addition some initial debug info 10 as for 5 with in addition the routine calling tree 12 as for 10 with in addition some routine calling notes 15 as for 12 with even more debug diagnostics 20 as for 15 with in addition some extra runtime analysis 30 full debug information 26 CHAPTER 3 THE CONFIGURATION FILE NAMCOUPLE This value can be changed at runtime with the oasis_set_debug or prism_set_debug routine see section 2 9 e SCALTYPE UNUSED 3 3 Second section of namcouple file The second part of the namcouple starting after the keyword SSTRINGS contains coupling information for each coupling or I O field Its format depends on the field status given by the last entry on the field first line EXPORTED IGNOUT or INPUT in the example above The field may be AUXILARY UNU
43. emselves CORRECT FILLING SUBGRID MASKP LOCTRANS ONCE is not explicitely offered as it is equivalent to defining a coupling period equal to the total runtime The following transformations are not available as they were already deprecated in OASIS3 3 see OASIS3 3 User Guide REDGLO INVERT REVERSE GLORED MASK and EXTRAP are not available but the corresponding linear extrapolation can be re placed by the more efficient option using the nearest non masked source neighbour for tar get points having their original neighbours all masked This is now the default option for SCRIPR DISTWGT GAUSWGT and BILINEAR interpolations It is also included in SCRIPR CONSERV if FRACNNEI normalization option is chosen see section 4 3 INTERP interpolations are not available SCRIPR should be used instead MOZAIC is not available as MAPPING should be used instead 42 APPENDIX B CHANGES BETWEEN PREVIOUS OASIS3 3 AND NEW OASIS3 MCT NOINTERP does not need to be specified anymore if no interpolation is required Field combination with BLASOLD and BLASNEW these transformations only support mul tiplication and addition terms to the fields see section 4 2 e Using the coupler in interpolator only mode This is not possible anymore as OASIS3 MCT is now only a coupling library However it is planned in a further release to provide a toy coupled model that could be use to check the
44. eness portability and flexibility are OASIS3 MCT key design concepts At run time there is no longer a separate coupler executable OASIS3 MCT acts as a coupling library which main function is to interpolate and exchange the coupling fields between the component models of a coupled system OASIS3 MCT supports coupling of general two dimensional fields Unstructured grids are also supported using a one dimension degen eration of the two dimensional structures Thanks to MCT all transformations including regridding are executed in parallel on the set of source or target component processes and all couplings are now executed in parallel directly between the components via Message Passing Interface MPI OASIS3 MCT also supports parallel file I O using netcdf In spite of the significant changes in underlying implementation usage of OASIS3 MCT in the codes has largely remained unchanged with respect to OASIS3 3 To communicate with another model or to perform I O actions a component model needs to include few specific calls to the OASIS3 MCT coupling library which Application Programmig Interface used in component models is unchanged The use state ment has been updated and now requires a single use mod prism or use mod oasis statement instead of the various use statements required in prior OASIS3 versions The namcouple configuration file is also largely unchanged relative to OASIS3 although several options are either not used or not suppo
45. eral Circulation Models of the ocean and of the atmosphere Today the OASIS3 3 coupler which is the result of more than 20 years of evolution is used by about 30 modelling groups in Europe Australia Asia and North America on the different computing platforms The list of coupled models realized with OASIS3 and OASIS2 can be found in tables C 1 and C 2 in Appendix C OASIS sustained development is ensured by a collaboration between CERFACS and the Centre National de la Recherche Scientifique CNRS and its maintainance and user support is presently reinforced with additinal resources coming from IS ENES project funded by the EU FP7 GA no 228203 The current OASIS3 MCT version was significantly refactored with respect to OASIS3 3 OASIS3 MCT is now interfaced with the Model Coupling Toolkit Larson et al 2005 Jacob et al 2005 developed by the Argonne National Laboratory in the USA MCT implements fully parallel regridding as a parallel ma trix vector multiplication and parallel distributed exchanges of the coupling fields based on pre computed regridding weights and addresses Its design philosophy based on flexibility and minimal invasiveness is close to the OASIS approach MCT has proven parallel performance and is most notably the underlying coupling software used in National Center for Atmospheric Research Community Earth System Model 1 NCAR CESM1 OASIS3 MCT is a portable set of Fortran 77 Fortran 90 and C routines Low intrusiv
46. estart file during the coupling initialization phase below the oasis enddef prism enddef proto and send the data to match the oasis get prism get proto performed at time 0 in the target compo nent model The final coupling data on the source side will then be automatically written to the restart file for use in the next run When there is a lag the first and last instance of the source field in the debug netCDF file EXPOUT fields see section 3 3 always correspond respectively to the field read from and written to the restart file 2 10 COUPLING ALGORITHMS SEQ AND LAG CONCEPTS 19 Model B timestep 6 0 6 12 18 24 30 120 A M 2 SEIE L L F2 F1 F2 F2 F1 F2 Fl Fi ds F2 Fi F2 rif rf rife Fife eel rf rf Fij F2 pj rf rf Fl eee Pe Sa JI X 4b RER EURE 411 X LL 1 e Ae coe 1 2 F2 0 4 8 12 6 0 24 28 120 Model A timestep 4 p prism put proto prism get proto leading to writing reading to from coupling restart file Cpl period Fi 12 prism put proto prism get proto leading to sending receiving actions Cpl period F2 24 prism put proto prism get proto not leading to LAG F1 4 sending receiving actions LAG F2 6 Figure 2 4 LAG concept first example 1 LAG concept first example A first coupling algorithm exploiting the LAG concept is illustrated on figure 2 4 On the 4 figures in this section short black a
47. field therefore this option does not ensure global conservation of the field but ensures that the energy received is proportional to the non masked surface of the target grid and it does not change the sign of the field SCONSOPT is an optional argument specifying the algorithm SCONSOPT can be bfb or opt The bfb option enforces a bit for bit transformation regardless of the grid decomposition or process count The opt option carries out the conservation using an optimal algorithm using less memory and a faster approach Option bfb is the default setting SUBGRID UNUSED BLASNEW BLASNEW performs a scalar multiply or scalar add to any destination field This is the equivalent of BLASOLD on the destination side The prior feature that supported linear combinations of the current coupling field with any other fields after the interpolation has been deprecated This analysis requires the same input line s as BLASOLD MASKP UNUSED REVERSE UNUSED CHECKOUT CHECKOUT calculates the global minimum the maximum and the sum of the the target field values and prints them to the OASIS3 MCT debug file for the master process of the target component model only This operation does not transform the field The generic input line is as for CHECKIN see above GLORED UNUSED Chapter 5 OASIS3 MCT auxiliary data files OASIS3 MCT may use auxiliary data files e g defining the grids of the models being coupled containing the field
48. grid corners must also be available in the grids nc file as arrays dimensioned nx ny 4 or nbr pts 1 4 where 4 is the number of corners in the counterclockwize sense The names of the arrays must be composed of the grid prefix and the suffix clo or cla for respectively the grid corner longitudes or latitudes As for the other grid information the corners can be provided in grids nc before the run by the user or directly by the model through specific calls see section 2 3 Longitudes must be given in degrees East in the interval 360 0 to 720 0 Latitudes must be given in degrees North in the interval 90 0 to 90 0 Note that if some grid points overlap it is recommended to define those points with the same number e g 360 0 for both not 450 0 for one and 90 0 for the other to ensure automatic detection of overlap by OASIS3 MCT The corners of a cell cannot be defined modulo 360 degrees For example a cell located over Greenwich will have to be defined with corners at 1 0 deg and 1 0 deg but not with corners at 359 0 deg and 1 0 deg Cells larger than 180 0 degrees in longitude are not supported 2 masks nc contains the masks for all component model grids in INTEGER arrays Be careful to note the historical OASIS convention 0 not masked i e active or 1 masked i e not active 36 CHAPTER 5 OASIS3 MCT AUXILIARY DATA FILES for each grid point The array names must be composed of the grid prefix and the suffix ms
49. has been approved for release with associated LA CC Number 98 45 Unless otherwise in dicated this SOFTWARE has been authored by an employee or employees of the University of California operator of Los Alamos National Laboratory under Contract No W 7405 ENG 36 with the United States Department of Energy The United States Government has rights to use reproduce and distribute this SOFTWARE The public may copy distribute prepare derivative works and publicly display this SOFT WARE without charge provided that this Notice and any statement of authorship are reproduced on all copies Neither the Government nor the University makes any warranty express or implied or assumes any liability or responsibility for the use of this SOFTWARE If SOFTWARE is modified to produce derivative works such modified SOFTWARE should be clearly marked so as not to confuse it with the version available from Los Alamos National Laboratory Chapter 2 Interfacing a component model with OASIS3 MCT At run time the OASIS3 MCT coupling layer supports coupling data between two components as well as interpolation and transformation of coupling fields Different communication techniques have been historically developed in OASIS With OASIS3 MCT communication is performed by MCT based on Message Passing Interface 1 protocol the keyword CHANNEL in the configuration file namcouple has to be MPI1 see chapter 3 For a practical test case using the OASIS3 MCT library
50. hbours is applied Non masked target grid points having all of the N source nearest neighbours masked by default the nearest non masked source neighbour is used logical 11_nnei hard coded to true in oasis3 mct lib scrip src remap_distwgt F same default behaviour as OASIS3 3 The configuring line is SCRIPR for DISWGT SCMETH SCGRS CFTYP SREST SNBIN NV SASSCMP SPROJCART SCMETH DISTWGT SCGRS is the source grid type LR D or U see annexe A SCFTYP is the field type SCALAR The option VECTOR which in fact leads to a scalar treatment of the field as in the previous versions is still accepted VECTOR I or See also http climate lanl gov Software SCRIP and the copyright statement in appendix 1 3 3 4 3 THE REMAPPING INTERPOLATION 31 VECTOR J i e vector fields are not supported anymore in OASIS3 MCT See Support of vector fields with the SCRIPR remappings below SREST is the search restriction type LATLON or LATITUDE see SCRIP 1 4 documen tation SCRIPusers pdf SNBIN the number of restriction bins see SCRIP 1 4 documentation SCRIPusers pdf Note that for D or U grid the restriction may influence sligthly the result near the borders of the restriction bins SNV is the number of neighbours used SASSCMP UNUSED vector fields are not supported anymore in OASIS3 MCT See Support of vector fields with the SCRIPR remappings below SPROJCART UNUSED vector fields are
51. he weight of a source cell is proportional to area of the source cell intersected by target cell Using the divergence theorem the SCRIP library evaluates this area with the line integral along the cell borders enclosing the area As the real shape of the borders is not known only the location of the 4 corners of each cell is known the library assumes that the borders are linear in latitude and longitude between two corners This assumption becomes less valid closer to the pole and for latitudes above the north thresh or below the south thresh values see oasis3 mct lib scrip remap_conserv F the library evaluates the intersection between two border segments using a Lambert equivalent azimuthal projection Problems were observed in some cases for the grid cell located around this north thresh or south thresh latitude Another limitation of the SCRIP 1st order conservative remapping algorithm is that is also supposes for line integral calculation that sin latitude is linear in longitude on the cell 4 4 THE POST PROCESSING STAGE 33 44 borders which again is in general not valid close to the pole For a proper consevative remapping the corners of a cell have to coincide with the corners of its neighbour cell with no holes between the cells If two cells of one same grid overlay at least the one with the greater numerical index must be masked for a proper conservative remapping For example if the grid cells with i
52. hers specific information on each coupling or I O field e g their coupling period the list of transformations or interpolations to be performed by OASIS3 MCT and associated configuring lines described in more details in chapter 4 etc In OASIS3 MCT several namcouple inputs have been deprecated but for backwards compatibility they are still allowed These inputs will be noted in the following text using the notation UNUSED and not fully described Information below these keywords is obsolete in OASIS3 MCT it will not be read and will not be used In the next sections a simple namcouple example is given and all configuring parameters are described The additional lines containing the different parameters required for each transformation are described in section 4 A realistic namcouple can be found in oasis3 mct examples tutorial data oasis3 directory 3 1 An example of a simple namcouple The following simple namcouple configures a run into which an ocean an atmosphere and an atmospheric chemistry models are coupled The ocean provides only the SOSSTSST field to the atmosphere which in return provides the field CONSFTOT to the ocean One field COSENHFL is exchanged from the atmosphere to the atmospheric chemistry and one field SOALBEDO is read from a file by the ocean HH HH HH HHH HE EH HH EE EH HO HH EE EH HE EE HE EH HO EE HE EE HH EE EH OE EH HE EE First section SSEQMODE
53. ired to adapt each coupling field from its source model grid to its target model grid on this basis prepare OASIS3 MCT configuring file namcouple see chapter 3 OASIS3 MCT supports different interpolation algorithms as is described in chapter 4 Regridding files can be compute online using the SCRIP options or offline and read using the MAPPING trans formation 4 Generate required auxiliary data files see chapter 5 5 Compile OASIS3 MCT the component models and start the coupled experiment Chapter 6 de scribes how to compile and run OASIS3 MCT and the tutorial toy coupled model If you need extra help do not hesitate to contact us see contact details on the back of the cover page 1 2 OASIS3 MCT sources OASIS3 MCT and tutorial toy coupled model sources are available from CERFACS SVN server To obtain more detail on how to download the sources please fill in the registration form at https verc enes org oasis download oasis registration form OASIS3 MCT directory structure is the following one oasis3 mct lib psmile OASIS3 MCT coupling library scrip SCRIP interpolation library mct Model Coupling Toolkit Coupling Software oasis3 mct doc OASIS3 MCT User Guide oasis3 mct util make dir Utilities to compile OASIS3 MCT oasis3 mct examples tutorial Environment to run the tutorial toymodel 1 3 Licenses and Copyrights 1 3 1 OASIS3 MCT license and copyright statement Copyright 2012 Centre Europ
54. ity to detect a deadlock before it happens and exit It does this by tracking the order of get and put calls in models compared to the SEQ specified in the namcouple If there are any inconsistencies the model will abort gracefully with a useable error message before the system deadlocks If there are any coupling dependencies in the system use of the SEQ index is recommended for diagnosis but has no impact on the ultimate solution and is NOT required Take the following two examples In both examples there are two models each put a field to the other at every coupling period without any lags In the first case there is no dependency and each model sends puts the data first and then receives the data second at each timestep modell model2 put f1d1 put 1d2 get f1d2 get fldl The put from modell for fld1 is received by the get in model2 and the put from model for fld2 is received by the get in modell In this case there is no sequencing dependency and the value of SEQ must be identical or unset in the namcouple description of the fld1 and fld2 coupling If SEQ is set to 1 for fld1 and 2 for fld2 in this case then the model will abort because at runtime the coupling will detect in model 2 that fld2 was sent before fld1 was received which is out of sequence as defined by the SEQ settings In the next example there is a dependency in the sequencing modell model2 put f1d1 get fldl fld2 g fld1 get
55. k This file masks or masks nc is mandatory 3 areas nc this file contains mesh surfaces for the component model grids in single or double pre cision REAL arrays depending on OASIS3 MCT compilation options The array names must be composed of the grid prefix and the suffix srf The surfaces may be given in any units but they must be all the same This file areas nc is mandatory for CONSERV GLOBAL GLBPOS BASBAL and BASPOS it is not required otherwise 5 2 Coupling restart files At the beginning of a coupled run some coupling fields may have to be initially read from their coupling restart file on their source grid see the LAG concept in section 2 10 When needed these files are also automatically updated by the last active oasis_put or prism put proto call of the run see section 2 7 1 Warning the date is not written or read to from the restart file therefore the user has to make sure that the appropriate coupling restart file is present in the working directory The coupling restart files must follow the NetCDF format The name of the coupling restart file is given by the 6th character string on the first configuring line for each field in the namcouple see section 3 3 Coupling fields coming from different models cannot be in the same coupling restart files but for each model there can be an arbitrary number of fields written in one coupling restart file In the coupling restart files th
56. led by model B For F the oasis put prism put proto performed at time 18 by model B then leads to a put action as 18 6 24 which is a coupling period that matches the get action performed at time 20 CHAPTER 2 INTERFACING A COMPONENT MODEL WITH OASIS3 MCT id B dm pa 30 120 X 18 EEEE 24 et 6 c Lc 2 T fr ny fe of F2 Fi F3 F2 Fi F3 F2 F3 JE p Fi F2 F5 if E rf Fi F2 F5 jr Te 3 ef 15 L___1_i_ __1___t_1_ __L______ L __ EST b 18 24 12 Model A timestep 6 prism_put_proto prism_get_proto leading to writing reading to from coupling restart file Cpl_period F1 12 prism_put_proto prism_get_proto leading to Cpl period F2 12 sending receiving actions Cpl period F3 12 prism put proto prism get proto NOT leading to LAG F1 2 6 sending receiving actions LAG F2 6 LAG F3 0 Figure 2 5 LAG concept second example 24 below the oasis_get prism_get_proto called by model A At the beginning of the run as their LAG index is greater than 0 the firstoasis get prism get proto of F and F5 will automatically be fulfilled with fields read from their respective coupling restart files The user therefore has to create those coupling restart files before the first run in the experi ment At the end of the run F having a lag greater than 0 is automatically written to its coupling restart file below the la
57. les for SCRIPR 37 6 Compiling running and debugging 38 6 1 Compiling OASIS3 MCT 38 6 2 iDeb geing 5 2 dog Renton eMe eese ar Peg EN RS E ID ee 38 6 2 1 Running OASIS3 MCT 38 A Thegrid types for the transformations 39 B Changes between previous OASIS3 3 and new OASIS3 MCT 40 B 1 General architecture 40 B 2 Changes in the coupling interface in the component models 40 B 3 Functionality not offered anymore 41 B 4 New functionality offered 42 B 5 Changes in the configuration file namcouple 43 B 6 Other differences 1 4 4484 done PIER PESE 43 C Coupled models realized with OASIS 44 CONTENTS 1 ACKNOWLEDGMENTS We would like to thank the main past or present developers of OASIS are in alphabetical order with the name of their institution at the time of their contribution to OASIS Arnaud Caubel LSCE IPSL amp FECIT Fujitsu Damien Declat CERFACS Italo Epicoco CMCC Veronika Gayler MPI M amp D Josefine Ghattas CERFACS Jean Latour CERFACS amp Fujitsu Fecit Eric Maisonnave CERFACS Silvia Mocavero CMCC Elodie Rapaport CERFACS Hubert Ritzdorf CCRLE NEC Sami Saarinen ECMWF Eric Sevault M t o France Laurent Terray CERFACS Olivier Thual CERFACS Reiner Vogels
58. ll Note that 2nd order con servative mapping is not supported yet with OASIS3 MCT The configuring line is SCRIPR for CONSERV SCMETH S CGRS SCFTYP REST NBIN NORM SORDER where SCMETH CONSERV SCGRS is the source grid type LR D and U are supported for first order remapping if the grid corners are given by the user in the grid data file grids nc only LR is supported if the grid corners are not available in grids nc and therefore have to be calculated automatically by OASIS3 SCFTYP SREST SNBIN are as for DISTWGT SNORM is the NORMalization option FRACAREA The sum of the non masked source cell intersected areas is used to NORMalise each target cell field value the flux is not locally conserved but the flux value itself is reasonable DESTAREA The total target cell area is used to NORMalise each target cell field value even if it only partly intersects non masked source grid cells local flux conser vation is ensured but unreasonable flux values may result FRACNNEI as FRACAREA except that at least the source nearest unmasked neigh bour is used for unmasked target cells that intersect only masked source cells SORDER FIRST for first order remapping respectively see SCRIP 1 4 documentation Note that 2nd order conservative mapping is not supported yet with OASIS3 MCT Precautions related to the use of the SCRIPR CONSERV remapping For the 1st order conservative remapping t
59. mations e g remappings interpolations in parallel on either the source or target processes and that performs all communication in parallel directly between the component models there is no central coupler executable anymore MAPPING transformation to use a pre defined mapping file With MAPPING OASIS3 MCT has the ability to read a predefined set of weights and addresses mapping file specified in the namcouple to perform the interpolation remapping The user also has the flexibility to choose the location and the parallelization strategy of the remapping with specific MAPPING options see section 4 3 Mono process mapping file generation with the SCRIP library But as before OASIS3 MCT can also generate the mapping file using the SCRIP library Jones 1999 When this is the case the mapping file generation is done on one process of the model compo nents all previous SCRIPR remapping schemes available in OASIS3 3 are still supported besides BICUBIC and CONSERV SECOND see SCRIPR in section 4 3 MPI2 job launching is NOT supported Only MPII start mode is allowed As before with the MPI1 mode all component models must be started by the user in the job script in a pseudo MPMD mode in this case they will automat ically share the same MP I COMM WORLD communicator and an internal communicator created by OASIS3 MCT needs to be used for internal parallelization see section 2 2 2 Changes in the coupling interface in the compone
60. ng the local partition in the global index space has a different expression depending on the type of the partition in OASIS3 MCT 4 types of partition are supported Serial no partition Apple Box and Orange see below ierror INTEGER OUT returned error code 2 4 1 Serial no partition This is the choice for a monoprocess model In this case we have ig_oaral 1 3 e ig paral 1 0 indicates a Serial partition e ig paral 2 0 e ig paral 3 the total grid size 2 4 2 Apple partition Each partition is a segment of the global domain described by its global offset and its local size In this case we have ig paral 1 3 e ig paral 1 1 indicates an Apple partition e ig paral 2 the segment global offset e ig paral 3 the segment local size Figure 2 1 illustrates an Apple partition over 3 processes 2 4 3 Box partition Each partition is a rectangular region of the global domain described by the global offset of its upper left corner and its local extents in the X and Y dimensions The global extent in the X dimension must also be given In this case we have ig paral 1 5 e ig paral 1 2 indicates a Box partition e ig paral 2 the upper left corner global offset e ig paral 4 the local extent in y 2 e ig paral 3 the local extent in x 4 5 e ig paral 5 the global extent in x Figure 2 2 illustrates a Box partition over 3 processes 12
61. not supported anymore in OASIS3 MCT See Support of vector fields with the SCRIPR remappings below GAUSWGT performs a N nearest neighbour interpolation weighted by their distance and a gaussian function All grid types are supported Masked target grid points the zero value is associated to masked target grid points Non masked target grid points having some of the N source nearest neighbours masked a nearest neighbour algorithm using the remaining non masked source nearest neighbours is applied Non masked target grid points having all of the N source nearest neighbours masked by default the nearest non masked source neighbour is used i e logical 11 nnei hard coded to true in oasis3 mct lib scrip src remap gauswgt F this is NOT the same default behaviour as OASIS3 3 to have the same default behaviour as in OASIS3 3 put 11 nnei false The configuring line is SCRIPR for GAUSWGT SCMETH SCGRS SCFTYP SREST SNBIN SNV SVAR where all entries are as for DISTWGT except that SCMETH GAUSWGT SVAR which must be given as a REAL value e g 2 0 and not 2 defines the weight given to a neighbour source grid point as proportional to exp 1 2 d o where d is the distance between the source and target grid points and c V AR d where d is the average distance between two source grid points calculated automatically by OASIS3 MCT BILINEAR performs an interpolation based on a local bilinear
62. nt models Use statement The different OASIS3 3 USE statements were gathered into one USE mod oasis or one USE mod_prism therefore much simpler to use As with OASIS3 3 the put calls are non blocking but the get calls are blocking As before the user has to take care of implementing a coupling algorithm that will result in matching put and get calls to avoid deadlocks see section 2 7 The lag LAG index works as before in OASIS3 3 see section 2 10 B 3 FUNCTIONALITY NOT OFFERED ANYMORE 41 e Support of previous prism xxx and new oasis xxx interfaces B 3 OASIS3 MCT retains prior interface names of OASIS3 3 e g prism put proto to ensure full backward compatibility However new interface names such as oasis put are also available and should be prefered Both routine names are listed in chapter 2 Auxiliary routines not supported yet Auxiliary routines prism put inquire prism put restart proto prism get freq are not supported yet Support of components for which only a subset of processes participate in the coupling New routines oasis create couplcomm and oasis set couplcomm are now available to create or set a coupling communicator in the case only a subset of the component processes partici pate in the coupling see section 2 2 3 But even in this case all OASIS3 MCT interface routines besides the grid definition see section 2 3 and the put and get call per se see section 2 7
63. ond section of namcouple for OUTPUT fields 3 3 3 The configuration file namcouple An example of a simple namcouple First section of namcouple file Second section of namcouple file Serial no partition 2 4 2 Apple partition Box partition 2 4 4 Orange partition Coupling field declaration End of definition phase Sending put and receiving get actions Sending a coupling or I O field 2 7 2 Receiving a coupling or I O field Termination Coupling algorithms SEQ and LAG concepts 2 10 1 The lag concept 2 10 2 The sequence concept Second section of namcouple for EXPORTED and EXPOUT fields Second section of namcouple for INPUT fields Transformations and interpolations Time transformations 23 23 25 26 26 27 27 28 ii CONTENTS 4 2 The pre processing transformations 29 4 3 The remapping interpolation 29 4 4 The post processing stage 33 5 OASIS3 MCT auxiliary data files 35 SL Grid data tiles usu aue EIUSDEM RUE e a PEE ee S 35 5 2 Coupling restart files 5o eB RP ye RR IY S ey uo Rye d 36 5 3 Inp t data files 5 4 ri Ine ue eura bep RUE e e Pe eere 36 5 4 Transformation auxiliary data files ee 36 5 4 1 Files containing the remapping weights and addresses for MAPPING 36 5 4 2 Auxiliary data fi
64. ons e LOCTRANS LOCTRANS requires one configuring line on which a time transformation automatically performed below the call to oasis_put or prism put proto should be indicated LOCTRANS operation STRANSFORM where TRANSFORM can be INSTANT no time transformation the instantaneous field is transferred ACCUMUL the field accumulated over the previous coupling period is exchanged the accu mulation is simply done over the arrays field array provided as third argument to the oasis put or prism put proto calls not weighted by the time interval between these calls AVERAGE the field averaged over the previous coupling period is transferred the average is simply done over the arrays 1eld array provided as third argument to the oasis put or prism put proto calls not weighted by the time interval between these calls T MIN the minimum value of the field for each source grid point over the previous coupling period is transferred T MAX the maximum value of the field for each source grid point over the previous coupling period is transferred ONCE UNUSED not supported in OASIS3 MCT With OASIS3 MCT time transformations are supported more generally with use of the coupling restart file The coupling restart file allows the partial time transformation to be saved at the end of a run for exact restart at the start of the next run For that reason coupling restart filenames are now req
65. option are consistent with the files generated in OA SIS3 3 or OASIS3 MCT using the SCRIP library see 5 4 2 below The files are netcdf and the key fields are num links the number of weights for each grid pair src grid size the 2d size of the source grid dst grid size the 2d size of the destination grid num wgts the total number of weights remap matrix the weights dst address the global destination index for the weight src address the global source index for the weight 5 4 TRANSFORMATION AUXILIARY DATA FILES 37 5 4 2 Auxiliary data files for SCRIPR The NetCDF files containing the weights and addresses for the SCRIPR remappings see section 4 3 are automatically generated at runtime by OASIS3 MCT Their structure is described in detail in section 2 2 3 of the SCRIP documentation available in oasis3 mct doc SCRIPusers pdf In particular they are netCDF files with the following one fields src grid size is a scalar integer indicating the total number of global grid cells for the source grid This field in a netCDF dimension dst grid size is a scalar integer indicating the total number of global grid cells for the destination or target grid This field in a netCDF dimension num links is a scalar integer indicating the total number of associated grid pairs in the file This is typically a large number This field is a netCDF dimension num wegts is a scalar integer indicating the number of weights per associ
66. ou pled ocean atmosphere modelling computing and scientific aspects In 2nd UNAM CRAY supercomputing conference Numerical simulations in the environmental and earth sciences Mexico city Mexico 1995 Sevault et al 1995 Sevault E P Noyret and L Terray Clim 1 2 user guide and reference manual Technical Report TR CGMC 95 47 CERFACS 1995 Terray and Thual 1995b Terray L and O Thual Oasis le couplage oc an atmosph re La M t orologie 10 50 61 1995 Terray and Thual 1993 Terray L and O Thual Coupled ocean atmosphere simulations In High Performance Computing in the Geosciences proceedings of the Les Houches Workshop F X Le Dimet Ed Kluwer Aca demic Publishers B V 1993 Terray et al 1995 Terray L E Sevault E Guilyardi and O Thual OASIS 2 0 Ocean Atmosphere Sea Ice Soil User s Guide and Reference Manual Technical Report TR CGM C 95 46 CERFACS 1995 Terray et al 1995b Terray L O Thual S Belamari M D qu P Dandin C L vy and P Delecluse Climatology and interannual variability simulated by the arpege opa model Climate Dynamics 11 487 505 1995 Terray et al 1999 Terray L S Valcke and A Piacentini OASIS 2 3 Ocean Atmosphere Sea Ice Soil User s Guide and Reference Manual Technical Report TR CMGC 99 37 CERFACS Toulouse France 1999 Valcke et al 2004 Valcke S A Caubel R Vogelsang and D Declat OASIS3 User s Guide oasis3 prism 2 4 PRISM Report No 2 5th Ed CER
67. racteristics of these grids are detailed here LR grid The longitudes and the latitudes of 2D Logically Rectangular LR grid points can be described by two arrays longitude i j and latitude i j where i and j are respectively the first and second index dimensions Streched or and rotated grids are LR grids Note that A B G L Y or Z grids are all particular cases of LR grids U grid Unstructured U grids do have any particular structure The longitudes and the lati tudes of 2D Unstructured grid points must be described by two arrays longitude nbr pts 1 and latitude nbr pts 1 where nbr pts is the total grid size D grid The Reduced D grid is composed of a certain number of latitude circles each one being divided into a varying number of longitudinal segments In OASIS3 the grid data longitudes latitudes etc must be described by arrays dimensioned nbr pts 1 where nbr pts is the total number of grid points There is no overlap of the grid and no grid point at the equator nor at the poles There are grid points on the Greenwich meridian Appendix B Changes between previous OASIS3 3 and new OASIS3 MCT B 1 General architecture e OASIS3 MCT is only a coupling library B 2 Much of the underlying implementation of OASIS3 was refactored to leverage the Model Coupling Toolkit MCT OASIS3 MCT is a coupling library to be linked to the component models and that carries out the coupling field transfor
68. rovided in section 3 2 ED F E B 5 CHANGES IN THE CONFIGURATION FILE NAMCOUPLE 43 B 5 B 6 Changes in the configuration file namcouple The namcouple configuration file of OASIS3 MCT is fully backward compatible with OASIS3 3 However several namcouple keywords have been deprecated even if they are still allowed These keywords are noted UNUSED in sections 3 2 and 3 3 and are not fully described Information below these keywords will not be read and will not be used SSEQMODE CHANNEL JOB NAME INIDATE MODINFO CALTYPE Also the following inputs should not appear in the namcouple anymore as the related functionality are not supported anymore in OASIS3 MCT see above field status AUXILARY time transforma tion ONCE REDGLO INVERT MASK EXTRAP CORRECT INTERP MOZAIC FILLING SUBGRID MASKP REVERSE GLORED To get 2D fields in the debug output NetCDF files the 2D dimensions of the grids must be provided in the namcouple except if the field has the status OUTPUT otherwise the fields in the debug output files will be 1D Other differences IGNORED and IGNOUT fields are converted to EXPORTED and EXPOUT respectively The file areas nc is not needed anymore to calculate some statistics with options CHECKIN and or CHECKOUT see section 4 2 SEQ index is no longer needed to ensure correct coupling sequencing within the coupler Use of SEQ allows the coupling layer to detect potential deadlocks before they
69. rrows correspond to oasis_put prism_put_proto or oasis get prism get proto called in the component model that do not lead to any put or receiving action long black arrows correspond to oasis put prismput proto or oasis get prism get proto called in the component models that do actually lead to a put or get action long red arrows correspond to oasis put prism put proto or oasis get prism get proto called in the component models that lead to a reading or writing of the coupling field from or to a coupling restart file During a coupling timestep model A receives F2 and then sends 71 its timestep length is 4 During a coupling timestep model B receives F and then sends P its timestep length is 6 F1 and F gt coupling periods are respectively 12 and 24 If Fi P5 put action by model A B was used at a coupling timestep to match the model B A get action a deadlock would occur as both models would be initially waiting on a get action To prevent this F1 and F5 produced at the timestep before have to be used to match respectively the model B and model A get actions This implies that a lag of respectively 4 and 6 seconds must be defined for F and F2 For F the oasis put prism put proto performed at time 8 and 20 by model A will then lead to put actions as 8 4 12 and 20 4 24 which are coupling periods that match the get actions performed at times 12 and 24 below the oasis get prism get proto cal
70. rted There is a new transformation in namcouple i e MAPPING which allows a user to specify a mapping file generated externally Some features like vector mapping and second order mapping have been delayed in implementation while other new features like parallel mapping have been added And currently only MCT see www mcs anl gov research projects mct 1 1 STEP BY STEP USE OF OASIS3 MCT 3 MPII job launching is supported First tests done with up to 8000 cores on the Bullx Curie machine at the TGCC are very encouraging and it is therefore very likely that OASIS3 MCT will provide an efficient and easy to use coupling solution 1 4 Step by step use of OASIS3 MCT To use OASIS3 MCT for coupling models and or perform I O actions one has to follow these steps 1 Obtain OASIS3 MCT source code see chapter 1 2 2 Identify the coupling or I O fields and adapt the component models to allow their exchange with the OASIS3 MCT coupling library based on MPI1 message passing The OASIS3 MCT coupling library uses NetCDF and therefore can be used to perform I O actions from to disk files For more detail on how to interface a model with OASIS3 MCT see chapter 2 The tutorial coupled model gives a practical example of a coupled model the sources are given in directories examples tutorial more detail on the tutorial and how to compile and run it can be found in chapter 6 3 Define all coupling and I O parameters and the transformations requ
71. s automatically defined based on the field and component model names The third line is LOCTRANS if this transformation is chosen for the field Note that LOCTRANS is the only transformation supported for OUTPUT fields 3 3 3 Second section of namcouple for INPUT fields The first and only line for fields with status INPUT is SOALBEDO SOALBEDO 17 86400 0 SOALBEDO nc INPUT where the different entries are e SOALBEDO symbolic name for the field in the target model CHARACTER 8 repeated twice e 17 index in auxiliary file cf_name_table txt see above for EXPORTED fields e 86400 input period in seconds e 0 number of transformations always 0 for INPUT fields e SOALBEDO nc CHARACTER 32 giving the input file name for more detail on its format see section 5 3 e INPUT field status The 2D dimensions of the grids must be provided in the namcouple so to have 2D fields in the debug files otherwise the fields in the debug files will be 1D Chapter 4 Transformations and interpolations Different transformations and 2D interpolations are available in OASIS3 MCT to adapt the coupling fields from a source model grid to a target model grid In the following paragraphs a description of each transformation with its corresponding configuring lines is given Features that are now deprecated non functional compared to prior versions will be noted with the string UNUSED but not described 4 1 Time transformati
72. s in the namcouple il part id INTEGER IN partition ID see section 2 4 var nodims INTEGER DIMENSION 2 IN var nodims l is the rank of field ar ray 1 or 2 var nodims 2 is the number of bundles always 1 for OASIS3 MCT kinout INTEGER IN OASIS_In or PRISM_In ie 15 for fields received by the model OASIS Out PRISM Out i e 14 for fields sent by the model 2 var actual shape INTEGER DIMENSION 2xvar nodims 1 IN vector of integers giving the minimum and maximum index for each dimension of the coupling field ar ray for OASIS3 the minimum index has to be 1 and the maximum index has to be the extent of the dimension var type INTEGER IN typeofcoupling field array put OASIS RealorPRISM Real i e 4 for single or double precision real arrays All coupling data is treated as double pre cision in the coupling layer but conversion to or from single precision data is supported in the interface ierror INTEGER OUT returned error code 2 6 End of definition phase All component processes close the definition phase e CALL oasis enddef ierror e CALL prism enddef proto ierror ierror INTEGER OUT returned error code 2 7 Sending put and receiving get actions 2 7 4 Sending a coupling or I O field In the model time step loop each process sends its part of the coupling or I O field
73. see the sources in examples tutorial and more details in chapter 6 To communicate with another component model or to perform I O actions a component model needs to be interfaced with the OASIS3 MCT library which sources can be found in oasis3 mct lib psmile directory The OASIS3 MCT library supports parallel communication between parallel component models an ability to couple a component on a subset of it s processeses only automatic sending and receiving actions at appropriate times following user s choice indicated in the namcouple time integration or accumulation of the coupling fields some transformations such as mapping interpolation between grids I O actions from to files To adapt a component model to OASIS3 MCT specific calls of the following classes have to be imple mented in the code DO I QN tA BW NY Initialisation section 2 2 Grid data file definition section 2 3 Partition definition section 2 4 coupling I O field declaration section 2 5 End of definition phase section 2 6 coupling I O field sending and receiving section 2 7 Termination section 2 8 Optional auxiliary routines section 2 9 Finally in section 2 10 different coupling algorithms are illustrated and details on how to reproduce them with OASIS3 MCT are provided More information on the LAG and SEQ indices are also given in that section 2 1 USE OF OASIS3 MCT LIBRARY 7 2 1 Use of OASIS3 MCT library
74. st F oasis put prism put proto as the date lag equals the total run time The analogue is true for F2 These values will automatically be read in at the beginning of the next run below the respective oasis get prism get proto 2 LAG concept second example A second coupling algorithm exploiting the LAG concept is illustrated on figure 2 5 During its timestep model A receives P5 sends F3 and then F3 its timestep length is 6 During its timestep model B receives F receives F3 and then sends F gt its timestep length is also 6 F1 F2 and F3 coupling periods are both supposed to be equal to 12 For F and F the situation is similar to the first example If F F gt put action by model A B was used at a coupling timestep to match the model B A get action a deadlock would occur as both models would be waiting on a get action To prevent this F1 and F gt produced at the timestep before have to be used to match the model A and model B get actions which means that a lag of 6 must be defined for both F and F2 For both coupling fields the oasis put prism put proto performed at times 6 and 18 by the source model then lead to put actions as 6 6 12 and 18 6 24 which are coupling periods that match the get action performed at time 12 and 24 below the oasis get prism get proto called by the target model 2 10 COUPLING ALGORITHMS SEQ AND LAG CONCEPTS 21 For F3 sent by model A and received by model B no l
75. te area prism write area see section 2 3 For a correct CONSERV operation overlapping grid cells on the source grid or on the target grid must be masked In the namcouple CONSERV requires one input line with one argument and one optional argument CONSERV operation SCMETH CONSOPT where SCMETH is the method desired with the following choices with SCMETH GLOBAL the field is integrated on both source and target grids with out considering values of masked points and the residual target source is uniformly distributed on the target grid this option ensures global conservation of the field with SCMETH GLBPOS the same operation is performed except that the residual is distributed proportionally to the value of the original field this option ensures the global conservation of the field and does not change the sign of the field with SCMETH BASBAL the operation is analogous to GLOBAL except that the non masked surface of the source and the target grids are taken into account in the calculation of the residual this option does not ensure global conservation of the field but ensures that the energy received is proportional to the non masked surface of the target grid 34 CHAPTER 4 TRANSFORMATIONS AND INTERPOLATIONS with SCMETH BASPOS the non masked surface of the source and the target grids are taken into account and the residual is distributed proportionally to the value of the original
76. ted through colon delimited field lists in the namcouple for example ATMTAUX ATMTAUY ATMHFLUX TAUX TAUY HEATFLUX 1 3600 3 rstrt nc EXPORTI in a single namcouple entry All fields will use the namcouple settings for that entry In the com ponent model codes these fields are still sent put or received get one at a time Inside OASIS3 MCT the fields are stored and a single mapping and send or receive instruction is executed for all fields This is useful in cases where multiple fields have the same coupling transformations and to reduce communication costs by aggregating multiple fields into a single communication Matching one source filed with multiple targets A coupling field sent by a source component model can be associated with more than one target field and model get In that case the source model needs to send put the field only once and the corresponding data will arrive at multiple targets as specified in the namcouple configuration file Different coupling frequencies and transformations are allowed for different coupling exchanges of the same field The inverse feature is not allowed i e a single target get field CANNOT be associated with multiple source put fields The debug files The debug mode was greatly improved as compared to OASIS3 3 The level of debug information written out to the OASIS3 MCT debug files for each model process is defined by the NLOGPRT value in the namcouple All details are p
77. tion are saved in the files COMP log and COMBerr in make dir During compilation a new compiling directory defined by variable ARCHDIR is created After successful compilation resulting executables are found in the compiling directory in bin libraries in 1ib and object and module files in build The following CPP keys are coded in OASIS3 MCT and can be specified in CPPDEF in make your_platform file e TREAT_OVERLAY To ensure in SCRIPR CONSERV remapping see section 4 3 that if two cells of the source grid overlay at least the one with the greater numerical index is masked they also can be both masked this is mandatory for this remapping For example if the grid line with i 1 overlaps the grid line with i imax it is the latter that must be masked when this is not the case with the mask defined in masks nc this CPP key forces these rules are to be respected 6 2 Debugging If you experience problems while running your coupled model with OASIS3 you can obtain more infor mation on what is happening by increasing SNLOGPRT in your namcouple see section 3 2 6 2 1 Running OASIS3 MCT A practical example on how to run OASIS3 MCT and running it in a coupled system is provided in oasis3 mct examples tutorial For more details see the README there in Appendix A The grid types for the transformations As described in section 4 the different SCRIP remappings OASIS3 MCT support different types of grids The cha
78. to indicate the time at the beginning of the timestep The sending is actually performed only if the time obtained by adding the field lag LAG in the namcouple to the date corresponds to a time at which it should be activated given the coupling or I O period indicated by the user in the namcouple see section 3 If the convention for date is followed the first coupling of a run should occur at time 0 and the final coupling should occur at time runtime cpl period where runtime is the total time of the run and cpl period is the coupling period The total run time should match an integer number of coupling periods If acoupling field has a positive lag the coupling field that matches the oasis get prism get proto at time 0 will come from a coupling restart file written by the last active oasis put prism put proto of the previous run see section 2 10 For a coupling field with a positive lag the coupling restart file see section 5 2 is automatically overwritten by the oasis put prism put proto when the time i e dat e LAG equals to the total runtime If a local time transformation is indicated for the field by the user in the namcouple INSTANT AVERAGE ACCUMUL T_MIN or T MAX see section 4 it is automatically performed and the resulting field is finally sent at the coupling or I O frequency For non instant transformations partially transformed fields will be written to the restart file at the end of the run for use on the next
79. to provide a local coupling communicator to OASIS3 MCT given that it already exists in the code The value of coup1 comm must be the value of this local coupling communicator for the processes participating to the coupling and it must be MP I COMM NULL for processes not involved in the coupling These routines should be called after the oasis init comp prism init comp proto but before the grid partition or coupling field declaration All OASIS3 MCT interface routines besides the grid definition see section 2 3 and the puts and gets per se see section 2 7 are collective and must be called by all processes In particular the coupling fields sent or received by the component must be declared by all component processes with oasis def var prism def var proto see section 2 5 and the field partition must be described across all coupling processes with oasis def partition prism def partition proto but with ig paral 0 for the processes not involved in the coupling see section 2 4 Here is a coding sample of how to use these routines CALL oasis init comp comp id comp name ierror CALL oasis get localcomm localComm ierror create communicator gathering coupling processes every other CALL MPI Comm Rank localComm mype ierror couplingpe false if mod mype 2 0 couplingpe true icpl MPI_UNDEFINED if couplingpe icpl 1 EFINED 2 3 GRID DATA FILE DEFINITION
80. ts and the cross gradient or second order conservative SCRIPR CONSERV SECOND remapping are not supported When the SCRIP library performs a remapping it first checks if the file containing the corre sponding remapping weights and addresses exists If it exists it reads them from the file if not it calculates them and store them in a file The file is created in the working directory and is called rmp_srcg t o tgtg INTTYPE NORMAOPT nc where srcg and tgtg are the acronyms of respetively the source and the target grids INTTYPE is the interpolation type i e DISTWGT GAUSWGT BILINEAR not BILINEA as in OASIS3 3 or CONSERV see below and NOR MAOPT is the normalization option i e DESTAREA FRACAREA or FRACNNEI for CONSERV only see below One has to take care that the remapping file will have the same name even if other details like the grid masks are changed When reusing a remapping file one has to be sure that it was generated in exactly the same conditions than the ones it is used for The following types of interpolations are available DISTWGT performs a distance weighted nearest neighbour interpolation N neighbours All types of grids are supported x Masked target grid points the zero value is associated to masked target grid points Non masked target grid points having some of the N source nearest neighbours masked a nearest neighbour algorithm using the remaining non masked source nearest neig
81. uired for all namcouple transformations that use LOCTRANS with non INSTANT values In this mode OASIS3 MCT will exit gracefully with an error message if a restart filename is not provided This is the reason an optional restart file is now provided on the OUTPUT namcouple input line 4 2 THE PRE PROCESSING TRANSFORMATIONS 29 4 2 4 3 The pre processing transformations REDGLO UNUSED INVERT UNUSED MASK UNUSED EXTRAP UNUSED CHECKIN CHECKIN calculates the global minimum the maximum and the sum of the the source field values and prints them to the OASIS3 MCT debug file for the master process of the source component model only This operation does not transform the field The generic input line is as follows even if SNINT has no impact in OASIS3 MCT CHECKIN operation SINT SNINT CORRECT UNUSED BLASOLD BLASOLD allows the source field to be scaled and allows a scalar to be added to the field The prior ability to perform a linear combination of the current coupling field with other coupling fields has been deprecated in OASIS3 MCT This transformation occurs before the interpolation per se This transformation requires at least one configuring line with two parameters BLASOLD operation SXMULT SNBFIELDS where XMULT is the multiplicative coefficient of the source field SNBFIELDS must be 0 if no scalar needs to be added or 1 if a scalar needs to be added In this last case an additional input line is require

Download Pdf Manuals

Related Search

Related Contents

Tripp Lite Full-Motion Desk Mount for 10" to 32" Flat-Screen Displays  NS2503-24P/2C User Manual  les principes de l`ionisation cuivre/argent  Advantech ADAM-4500 Network Card User Manual  Tascam DP-004 Bedienungsanleitung Deutsch  MANUAL DE UTILIZAÇÃO  Portal User Manual – Employer  Atomic Accessories PCA.35 game console accessory  AOCEM 320 YESOS  Homelite UT08012 Blower User Manual  

Copyright © All rights reserved. Failed to retrieve file