Home

Australian Climate Ocean Model (AusCOM) Users Guide

1. The actual building location is compile build MPI1 nt62 and the generated executable matm_MPI1 exe_nt62 is also moved to AusCOM1 0 bin 5 3 3 CICE cd AusCOM1 0 submodels cice4 0 compile comp auscom cice VAYU nP N Note the compiling script must be provided with an integer N such as 1 2 3 6 12 etc specifying how many processors are to be used to run CICE The processor usage is determined at compile time because the CICE code does not support dynamic MPI partitioning Consequently the generated CICE executable must be labeled with this number to distinguish it from other CICE executables compiled for using a different number of processors Following the convention the building directory is compile build MPI1 Np and the executable cice MPIl exe NP is moved to AusCOM1 0 bin where one may soon find more than one CICE executable having been built for different numbers of processors This happens because for different applications AusCOM may need to use different computing resources Also processors allocated for CICE and MOMA should be adjusted to achieve a reasonable load balance for the best computing efficiency Note however because of the different scalabilites of CICE and MOMA on different platforms processor allocations for these two models may vary largely for achieving a suitable load balance From experience we find that MOMA needs to use 10 20 times more processors than CICE In addition CICE4 0 code doe
2. Is Doc bin exp forcing input output submodels e Doc you can find a file named AusCOM1 0_usersguide pdf which is this document e bin a few files are located here environs vayu nci org au do mppncombine ksh calendar F90 e exp three sample experiments are prepared for you i e ciaf2 xx ciaf2 xy and cnyf2 xx with a README file describing these experiments e forcing only one small script file link forcing ksh is located here e input four sub directories are here containing the preprocessing auxiliary files for each of the components some of which are templates for run time adaption 40 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland e output nothing is here until your experiment proceeds e submodels the full source code packages of the 4 components are located here 6 2 Establishing Links to the Available Forcing Datasets cd AusCOM1 0 forcing link_forcing ksh Is la CIAF gt short p66 sjm599 COREv2 26JAN2010 CIAF CNYF v2 gt short p66 sjm599 COREv2 15JUNE2009 CNYF v2 link forcing ksh We see the links point to the physical locations of the forcing data e CIAF CORE2 interannual forcing e CNYF v2 CORE2 normal year i e climatology forcing 6 3 Building the component executables Each component system tree has a directory named compile where the compilation script is located All successfully generated executables will be automatically m
3. external GFDL module for surface flux calculations otherwise using the CICE built in boundary module e using the preprocessed CORE runoff regrided currently the annual mean climatology only data instead of that read in by MATM and interpolated by OASIS3 Doing so ensures reasonable re distribution spreading of the river runoff into ocean avoiding potential model crashes due to unrealistic freshwater accumulation and assocaied abnormal heating at some river outlets The penalty is that model loses all temporal variablities of the runoff forcing e ocean and sea ice start from the observed temperature and salinity namely a cold start Otherwise the ocean sea ice system will use a warm start as described below A warm start means an experiment is initialised with the full set of restart data from an existing AusCOM spin up run So in the case of cold start 0 the user must set a path to an initial condition source for example owner dhb599 expname cnyf2 01 ic_year 500 access owner if run under owner s home directory access short p66 Sowner if run under short p66 disk These example lines indicate that restart files of end of year 500 from user dhb599 s spin up run cnyf2 01 will be used to start this experiment Following the above pre definition all data files will be copied to the working directory i e rundir R 3 2 Adapting or Creating Configuration Files 37 This sub
4. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Bureau of Meteorology CSIRO Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland CAWCR Technical Report No 027 August 2010 www cawcr gov au Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland The Centre for Australian Weather and Climate Research a partnership between CSIRO and the Bureau of Meteorology CAWCR Technical Report No 027 August 2010 ISSN 1835 9884 National Library of Australia Cataloguing in Publication entry Author Daohua Bi and Simon Marsland Title Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland ISBN 978 1 921605 92 5 PDF Series CAWCR technical report 27 Notes Included bibliography references and index Subjects Ocean atmosphere interaction Australia Simulation methods Dewey Number 551 5246 Enquiries should be addressed to Daohua Bi Centre for Australian Weather amp Climate Research GPO Box 1289 Melbourne Victoria 3001 Australia dave bi csiro au Copyright and Disclaimer 2010 CSIRO and the Bureau of Meteorology To the extent permitted by law all rights are reserved and no part of this publication covered by copyright may be reproduced or copied in any form or by any means except with the written permission of CSIRO and the Bureau of Meteo
5. 4 Fig 5 Fig 6 ACCESS AusCOM Coupled Ocean and Sea Ice Model Framework and Components nnne eet env esie e se ette uda ie Ale Bathymetry m for the AusCOM ocean and sea ice model Grid lines indicate each fourth row in the orthogonal curvilinear zonal and meridional directions Top northern hemisphere projection showing the tripolar grid over the Arctic Bottom southern hemisphere projection showing the Mercator meridional grid over the Southem Ocean nitet teo hte ete fo cert o e ecd Em es bene AusCOM ocean and sea ice grid orthogonal curvilinear zonal grid spacing dx top km orthogonal curvilinear meridional grid spacing dy middle km and grid aspect ratio dx dy bottom dimensionless seeeee AusCOM ACCESS coupling approach CICE acts as the coupling media Schematic diagram of the mono processor coupling algorithm in AUSGOM ACCESS ih isis edere aeri a eiae saad eee drea deu ead a enu aia Edda Schematic diagram of the multi processor coupling algorithm in AusCOM AGCESS eee he tei utin bid redi reed rt rub naaa cred LIST OF TABLES Table 1 8 The 31 coupling fields of the AusCOM system occur in four classes there are 10 fields passed from the atmosphere model to the sea ice model a2i seven fields passed from ocean to sea ice 02i one field passed from sea ice to atmosphere i2a and 13 from sea ice to ocean i20 Where appropria
6. 4 Coupling Strategy nimodo Sai Era nno E e EX HEC sn Taani 12 4 1 iGo pling fields ortae d eee t diede 12 42 Coupling Interfaces gnat tias 12 4 2 1 pristri init eie er Aeneas e i HE ee TEETH ais 14 4 2 2 O hee p REAL He Potted gates Dea oleh 14 4 2 3 E A oh ethly git E 14 4 2 4 TOM OCN PER as 15 4 2 5 INTO FOCI uu ev tenu e epit tut oeste 15 4 2 6 into AtMit EE 15 4 2 7 coupler termination ooooonnonccccccnncncconononccnnnnncnnnnnncnncnnnnnnnnnnnnncancnnnnnn nn ncannnnncnnnns 15 4 3 Coupling Approach sie A TR RES 16 4 3 1 Gode Execution him 16 4 3 2 Coupling Algorithm Mono processor vs Parallel Coupling 18 5 A Technical Guide to the AusCOM System eene 21 5 1 PIAtOnms t 2 24s seek UTILI 21 5 2 AusCOM 1 0 System Infrastructure esirinnas aana eee 21 5 2 1 ME ea eee Ae dee nee ee ee ee 22 5 2 2 OX Ph m tole 22 5 2 3 Tisi c p 22 5 2 4 submodels 58 2 ne toc ER RO do tere 23 5 2 5 A C m xt 24 5 2 6 OUfplll a d tocco roter een EUN OD Se E ae Ava ated Se echo fond Gane nuevos e soo Mets too of 25 5 3 AusCOM GCombpllatlOFi 2 2nd etri xt qi rent tete teens Radi o ete tud taeda ada 25 5 3 1 PSMILe libraries and OASISS cccccceceesecceceeeeeeeceaeeeeeeesesecaeeeeeeeseseeneeaeeeeess 25 5 3 2 MATM 2i eot elato fibt e e ERE ld e fenes 25 5 3 3 GIG Boe Satie i A od 26 5 3 4 MONA 5n
7. AusCOM R 1 2 Run Path Definition This part sets all paths associated with this experiment or segment run specifically paths to the preprocessed input files model outputs and the working directory which on VAYU should all be located on the short disk system For example The actual run directory of the sample run ciaf2 xx 1s Srundir short Sproject Suser RUNNING AusCOM1 0 ciaf2 xx and its outputs are stored at Soutputdir short Sproject Suser OUTPUT AusCOM1 0 ciaf2 xx 34 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland Users are not supposed to make any change here All required directories will be created by this script when the experiment is started i e at the beginning of the initial run Useful tips e When a job is running a symbolic link to the physical location of the working directory rundir will be established under the submitting directory jobdir named as Running dir which helps the user get direct access to rundir without caring about where it is actually located to monitor simulation progress e Also symbolically linked is the model archive directory which can be accessed directly from say AusCOM1 0 output ciaf2 xx This helps the user locate the model outputs very efficiently R 2 Exp Run Time Control This is one of the most frequently visited parts R 2 1 Runtime Control for the Whole Experiment and a Segment Run iniyear 1948 finalyear 1950 inimonth
8. g NDAY s HNDAY nday g HSYEAR s SYEAR iniyear g HSMON s SMON inimonth g SDAY s HSDAY S iniday g HCAL_TYPE s CAL_TYPE cal_type g DT_CPL s DT_CPL dt_cpl_io g DT_OCE s HDT_OCE dt _oce g HLAYOUT s LAYOUT layout g VLIMIT s VLIMIT truncate velocity g FVWARN s FVWARN S truncate _verbose g SST_restoring s SST_restoring temp_restore_tscale g SSS_restoring s SSS_restoring salt restore tscale g Freezing simple s Freezing_simple Simple_frazil g Freezing accurate s Freezing_accurate Accurate_frazil g TL_frazil_only s TL_frazil_only TL_frazil g DIFF_CBT_IW s DIFF_CBT_IW diff_cbt_iw g VISC_CBU_IW s VISC_CBU_IW vise cbu iw g CONVECTION s HCONVECTION convection g HAREDI s HAREDI aredi g HAGM s HAGM agm g RICR s RICR ricr g USE_waterflux s HUSE_waterflux use_waterflux w q eof base_time is read in from diag_table and must NOT be changed during the exp if jobnum 1 then cp f Sinputdir mom4 diag table MOMA input diag table ed MOMA input diag table eof g SYEAR s SYEAR year g SMON s SMON month g HSDAY s HSDAY day w q eof fi b namelist for coupling purpose icemlt_factor 1 0 if lt 1 reduce the potential ice melt 67 only usable when POP icediag f frazil factor 0 5 mom4 uses two level frog time stepping but cice uses forward time stepping see comm
9. 6 previniday echo previnidate cut c 7 8 echo echo first day of this run date echo last day of this run fenddate echo initial date of the experiment inidate echo final date of the experiment finaldate echo day before the initial date previnidate echo last day of the previous run prevdate echo first day of the next run nextdate echo echo number of days in this run days in runj echo number of days since beginning of the experiment days since start echo runtime days in run 86400 duration of this run in seconds echo duration of this run in seconds runtime VGHUIIEEBHERHHIIUIIBEERBEBBHIIIBERBEBHHIDUNIBBERSHERBHHIUNIBBERHERIGURUNIB E H 3 Getting All Files into the Run Directory SUBBHHHEZSBERHHRHIGUUUNIBBEERHERHIAIIGHIEBEREHEHHHEURHHRBERHHIEIBBESRHHRIHIDUNIEE H 4 3 1 Grids IC forcing executables and some preprocessed auxiliary files following setup needs two things be decided first boundary layer gfdl lt how to calculte surface fluxes decided here runoff data core lt regrided core rundoff instead of that interpolated by oasis cold start 1 1 0 this experiment starts from scratch spinup if cold start 0 then ie warm start use existing AusCOM run restart to initialise this experiment for jobnum 1 next 4 5 lines specify the restart location owner dhb599 expname cnyf2 01 i
10. AusCOM MOMA in an offline manner These include the pre and post processing tools including grid generator initialisation mpp combining functions etc 4 COUPLING STRATEGY 4 1 Coupling fields AusCOM has a total of 31 2 dimensional coupling fields exchanged between CICE and MATM and between CICE and MOMA via OASIS see Table 1 All coupling fields are delivered by OASIS into or from CICE the coupling media and there is no direct coupling between MATM and MOMA Note the atmospheric near surface variables 2m air tempterature and 10m velocity are received by the CICE model and converted there into heat and momentum fluxes i e latent and sensible heat fluxes windstress by the standard NCAR boundary layer bulk formulae Large and Yeager 2004 2009 4 2 Coupling Interfaces One of the key tasks of building a coupled model under the OASIS3 framework is to develop for each sub model a coupling interface that connects to the coupler These interfaces are coded under the OASIS3 philosophy using the PRISM CLIM communication technique based on MPI and associated functions provided by the PSMILe libraries These subroutines control data transition between the sub models and the coupler They include a series of PSMILe subroutine calls which can either be scattered in the model code mainly within the time loop or be grouped as a set of functions for better code readablity and the convenience in code maintainance In the AusCOM sub mode
11. bet Gi fot UE MALUI GE las ete ks o de 26 5 4 OASIS3 Configuration File sssrin arindamai anai iia Anaia Saa 27 5 4 1 Introductions E E E tee bates id 27 5 4 2 Configuring Coupling Fields oooonoococcnnncnnnnocccccnnnccconononcconcnnnnnano nc cn nnncnnnnnnn cnn 28 5 5 The Runscript run auscom VAYU ssssssssesseeeeeee eee 33 5 5 1 INTO UCI Dn 33 5 5 2 Steps for Setting up an Aus COM Run seem 33 6 AusCOM Quick start on NCI VAYU eese nnne 40 6 1 Getting the AusCOM1 0 Release Package seem 40 6 2 Establishing Links to the Available Forcing Datasets sees 41 6 3 Building the component executables sssseeee eene nenne 41 6 4 Running A Sample Experiment sesessssssssssseseeeenene emen 42 6 5 Monitoring the RUn eee eee de eee ed ete dea min e OA eee us 42 6 6 Verifying the QUIDUts tdt tene E e D ed Lee t beat did 43 7 Acknowledgments cnc iii aee dx nor RR BER T Ka Ra nnna 44 8 References 5 1 er II Ier erri pP ee ree inen 44 PAD DOINGIIX inscritas 46 A A Sample OASIS3 Coupling Configuration File namcouple 46 B A Sample AusCOM RunscCript ccccccccccccccccccccccccccncnnnnnnnncnnncnnnnnnnnnnnennnnnnnnnnnen 55 ii Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland LIST OF FIGURES Fig 1 Fig 2 Fig 3 Fig
12. date year month day echo gt gt expid log echo date jobnum date starting pre processing gt gt expid log use an external tool to work out the run date inforamtion there must be a easier way for this but it works ok anyway cat gt calendar in lt lt EOF inidate date nyear nmonth nday caltype EOF AusCOMHOMB bin calendar exe lt calendar in gt calendar out prevdate cat calendar out cut c 2 9 enddate cat calendar out cut c 11 18 nextdate cat calendar out cut c 20 27 previnidate cat calendar out cut c 29 36 days in run cat calendar out cut c 38 45 days since start cat calendar out cut c 47 54 date in days cat calendar out cut c 56 63 days this year cat calendar out cut c 65 67 rm calendar in calendar out 58 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland prevyear echo prevdate cut c 1 4 prevmonth echo prevdate cut c 5 6 prevday echo prevdate cut c 7 8 endyear echo Senddate cut c 1 4 endmonth echo enddate cut c 5 6 endday echo enddate cut c 7 8 nextyear echo nextdate cut c 1 4 nextmonth echo nextdate cut c 5 6 nextday echo nextdate cut c 7 8 previniyear echo previnidate cut c 1 4 previnimonth echo previnidate cut c 5
13. eof b namelist for coupling purpose POP ICEDIAG true use POP approach for ice formation melting GFDL FLUXES false if boundary_ layer gfdl then GFDL FLUXES true use GFDL code for surface flux calculation cat input ice gfdl nml eof amp surface flux nml no neg q false use virtual temp true alt gustiness false old dtaudv false use mixing ratio false gust const 1 0 gust min 0 0 ncar ocean flux true ncar ocean flux orig false raoult sat vap false amp ocean rough nml roughness mom 5 8e 5 roughness heat 5 8e 5 roughness moist 5 8e 5 roughness min 1 0e 6 charnock 0 032 rough scheme beljaars do highwind false do cap40 false zcohl 0 0 zcogl 0 0 i eof cat gt input ice monin nml lt lt eof amp monin obukhov nml neutral true amp end eof fi if boundary_layer gfdl cat gt input_ice nml lt lt eof amp coupling nml init date iniyear inimonth iniday caltype caltype 65 jobnum jobnum inidate date runtime0 truntime0 runtime runtime dt_cpl_ai dt_cpl_ai dt_cpl_io dt_cpl_io dt_cice dt_ice pop_icediag POP_ICEDIAG ice_pressure_on true ice_fwflux true use_ocnslope false use_umask false rotate_winds true limit_icemelt false meltlimit 200 0 use_core_runoff true precip_factor 1 0 cst_ocn_albedo true ocn_albedo 0 1 gfdl surface flux GFDL FLUXES chk gfdl rough
14. fields 4 matm table COREv2 forcing data table e get core2 IA ksh script called by runscript to set COREv2 Inter Annual forcing 22 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland e get core2 NY ksh script called by runscript to set COREv2 Normal Year forcing under input cice cice grid nc CICE model grid information file cice kmtu nc CICE model grid land sea masks cice4 0 in nml CICE model namelist file for experiment design and control SSTS nc SST and SSS data used to initialise the sea ice properties if required A2I time0_10fields nc air to ice forcing data at the beginning of an experiment to be read in by CICE for two time level forcing purpose core runoff regrid nc CORE runoff data regrided on the CICE grid used as replacement of the runoff received from coupler if required e u star t0 nc u data required by CICE if the GFDL module is used to calculate surface fluxes under input mom4 grid spec nc MOMA grid information data file mom4 in nml namelist file for MOMA physical configuration and experiment control diag table MOMA diagnostic output table e field table MOMA field table data table MOMA data table not used in AusCOM but existence is required e ocean temp salt nc MOM4 ocean initial condition for temperature and salinity salt sfc restore nc 12 monthly SSS data for surface salinity relaxation temp sfc restore nc 1
15. interfaces are coded and called under the same philosophy and their data exchange operations are a lot simpler than that in CICE because they both are coupled to CICE only Therefore they both have only into ice and from ice for the data exchange operations 15 4 3 Coupling Approach 4 3 1 Code Execution Under the OASIS3 coupling framework all sub models are recoded mainly in their initialisation and time stepping loop to implement calls to the above discussed coupling interface routines for data exchange and model execution synchronisation As a result all of the sub models are running concurrently and exchanging data with OASIS3 at certain coupling time points in an order determined by the chosen coupling algorithm We again take the coupling media CICE model as an example to explain the AusCOM coupling approach and demonstrate how it is realised during the model code execution Following is a symbolic flow of CICE code execution coordinated by the coupler via the data exchange operations initialisation read in a2i and i20 fields saved at the end of last run read in o2i fields saved by ocean model at the end of last run time sec 0 begin atmosphere ice coupling iterations DO icpl ai 1 num cpl ai call from atm time sec begin ice ocean coupling iterations Do icpl io 1 num cpl io call into ocn time sec cice time loop within each ice ocean coupling interval do itap 1 num ice io update atmospheric data u
16. oce to 1800 half an hour only for the crashing run often helps to get around this problem R 2 2 Processor usage nproc cpl 1 nproc_atm 1 ncplproc atm 1 nproc_ice 6 ncplproc ice 1 nproc_oce 120 ncplproc oce 1 ntproc nproc atm nproc ice nproc oce nproc cpl These variables declare how many processors should be allocated for each model and the coupler Note the coupler and MATM are both hardwired to use 1 processor each but CICE and MOMA processor usage can vary Also declared here is the number of coupling processors for each sub model As already mentioned AusCOM 1 0 currently only supports mono cpu coupling R 2 4 Calendar Date Control Attention needs be paid to these two lines they are forcing dataset dependent caltype 1 0 1 or n eg 30 for 30 day months cal type julian Zfor caltype 1 thirty day for caltype 30 The following if block if f exp date determines status of the current job initial run jobnum 1 or continue run jobnum larger than 1 by existence of file exp date e g ciaf2 xx date and sets the required date and time control information for this job accordingly This exp date file has an important role in controlling the experiment A newly set up experiment should only have the runscript in position After the initial run is successfully completed file exp date is created containing start time and date information for the second run When the sec
17. particular format which depends on the field status given by the last entry on the first line of each field s configuring block Note in namcouple all comment lines start with and empty lines are not allowed AusCOM s 31 coupling fields are divided into 4 groups namely i2a i20 021 and a2i We choose one field from each group to demonstrate how they are configured under the OASIS3 coupling philosophy See Appendix A for a complete list of the coupling fields The atmosphere to ice a21 fields are formulated as follows Piste Stes a Atmosphere gt Ice 4 Field 01 downward short wave radiation swfld ai swfld i367 21600 4 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 POPO LOCTRANS CHECKIN SCRIPR CHECKOUT INSTANT INT 0 CONSERV LR SCALAR LATLON 10 FRACNNEI FIRST INT 0 This coupling field has 8 configuring lines with different numbers of entries 1 The first line includes 7 entries e swfld ai symbolic name of this field as defined in the source model MATM character 8 e swfld i symbolic name of this field as defined in the target model CICE character 8 e 367 index in auxiliary file cf name_table txt used by OASIS3 and PSMIL e to identify proper description for this coupling field just for printing purpose i e surface net downward shortwave flux W m Note this number and the associated description have no modelling significance and thus may not necessarily reflect th
18. share the disk systems and users need to make sure the forcing data are available for both machines 5 4 OASIS3 Configuration File For users who are keen to configure their own version of the AusCOM system or run their own designed experiments using the standard AusCOM1 0 release some of the provided auxliary files may need to be modified or even regenerated Please check the relevent documentation of the sub models and coupler or consult the AusCOM developers for advice to make sensible changes There are two key namelist files required for AusCOM physical configuration They are cice4 0 in nml for CICE and mom4 in nml for MOMA adapted from the CICE4 0 and MOMApl release Some minor changes have been made especially to mom4 in nml for flexible model management and tuning Users are encouraged to tackle physics issues in MOMA and CICE by switching the supported parameterization schemes and changing phyical parameters within a reasonable range in these two namelist files for achieving better modelling performance of the coupled ocean and sea ice system Here we focus on the core namelist file for AusCOM coupling control namcouple 5 4 1 Introduction namcouple is the key OASIS3 namelist file configuring and controling all coupling operations of the system in the course of modelling for a particular set up It provides the coupler with all required information about coupling such as the associated component models and their
19. 1 finalmonth 12 iniday 1 finalday 31 These lines set the simulation period for the experiment ciaf2 xx from the first day of year 1948 to the last day of year 1950 i e 3 complete years For a datatype NY CORE Normal Year experiment the iniyear should be set to 1 and finalyear can be any sensible number 1 10 500 up to 9999 nyear 0 nmonth 12 nday 0 The above lines set the simulation period for a segment run of this experiment namely 12 months Note nyear must always be 0 for some unreasonable reason in the calendar management program code curently used not essential for its function though and for a multi year segment say 2 year run we can set nmonth 24 Theoretically any combination of nmonth and nday are acceptable such as nmonth 7 nday 13 but in practice we always set whole months and in short test cases such as a 3 day test one may set nyear 0 nmonth 0 nday 3 The following self meaning variables set time step for all sub models atm ice and ice ocean coupling intervals As is shown atm ice coupling occurs once per 6 hour whilst ice couples with ocean every time step 1 e once per hour dt cpl ai 21600 dt cpl 10 3600 35 dt atm 3600 dt ice 3600 dt 0ce 3600 A useful tip e In some circumstances users may encounter a MOM4 model crash due to free surface penetrating into rock a sign of instability caused by a number of reasons Reducing dt
20. 1200 800 400 Bathymetry m for the AusCOM ocean and sea ice model Grid lines indicate each fourth row in Fig 2 and meridional directions Top northern hemisphere projection zonal showing the tripolar grid over the Arctic Bottom southern hemisphere projection showing the Mercator meridional grid over the Southern Ocean the orthogonal curvilinear 40 N 0 40 8 115 80 N 110 40 N 75 0 40 8 80 N 40 N 0 40 8 Fig 3 AusCOM ocean and sea ice grid orthogonal curvilinear zonal grid spacing dx top km orthogonal curvilinear meridional grid spacing dy middle km and grid aspect ratio dx dy bottom dimensionless 8 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland 3 COMPONENTS OF THE AUSCOM SYSTEM 3 1 Numerical Coupler OASIS3 The Ocean Atmosphere Sea Ice Soil version 3 OASIS3 is a numerical coupler developed at CERFACS France Valcke 2006 under the concepts of portability and flexibility At run time OASIS3 acts as a separate mono process executable It receives interpolates and sends coupling fields 2D only between the sub models It hooks the stand alone sub models via a few specific OASIS3 PRISM Model Interface Library PSMILe calls implemented in sub models OASIS3 is in charge of all data flows coupling between sub models and controls model synchronization via MPI functions Cu
21. 2 monthly SST data for surface temperature relaxation under input oasis3 grids nc grid lat lon and rotation angle information of all sub model grids e masks nc land sea masks for all sub model grids areas nc grid cell areas for all sub model grids e namcouple OASIS3 configuration file containing all coupling fields e cf name table txt a text file for OASIS3 to obtain coupling fields description e a2i nc 021 nc i20 nc i2a nc preprocessed restart files for OASIS3 5 2 4 submodels It contains full packages of codes of the sub models and coupler submodels matm cice4 0 mom4p1 oasis3_prism_2 5 Each sub model has its own sub directory trees some of which are very deep matm source bld compile cice4 0 csm_share 23 doc drivers auscom input templates mpi rundir serial source compile mom4p1 bin exp auscom_cpl compile oasis3_ prism_2 5 prism Linux data src util compile For compatibility the three imported sub systems CICE MOM4 and OASIS3 retain their own original infrastructure and contents as determined from the public release package which will allow for more straightforward system upgrading using subversion svn in the future Note however the addition of the extra driver directories where the bulk of the coupling code can be found e CICE AusCOM1 0 submodels cice4 0 drivers auscom MOM4 AusCOMI 0 submodels mom4pl src auscom c
22. 6 and p73 AusCOM development greatly benefitted from many helpful discussions with international collaborators and scientists at NOAA GFDL LANL CERFACS and the UKMO We acknowledge contributions to the development of AusCOM made by the CAWCR ACCESS Coupled and Ocean Modelling Team Xiaobing Zhou at BoM Michael Bates at UNSW Nathan Bindoff Jason Roberts Petra Heil and Roger Stevens at the University of Tasmania and Antarctic Climate and Ecosystems Cooperative Research Centre collaborators in the CAWCR BoM Seasonal Prediction Group Russ Fiedler Richard Matear Matthew Chamberlain and Andrew Lenton at CSIRO Marine and Atmospheric Research Hobart and the many others involved in ACCESS model development Notice of usage conditions New users are requested to contact Dave Bi or Simon Marsland prior to usage of the model This 1s in order to maintain a user data base by which users will be notified of upgrades bug fixes etc Users are requested to cite this User Guide in publications which include results from AusCOM1 0 This standard reference for AusCOM1 0 may be updated in the future and users on the database will be advised AusCOM1 0 is free software you can redistribute it and or modify it under the terms of the GNU General Public License as published by the Free Software Foundation either version 3 of the License or at your option any later version This program is distributed in the hope that it will be useful but WITHOUT ANY
23. 60x300 logically rectanglar horizontal mesh This is a typical IPCC class resolution In the vertical direction AusCOM1 0 implements the z coordinate available in MOMA with 50 model levels covering 0 6000 meters with a resolution ranging from 10 meters in the upper layers 0 200 meters to about 333 meters for the abyssal ocean It is worth mentioning that the conventional z height coordinate models have previously represented free surface variations by a variable thickness upper layer The z coordinate Adcroft and Campin2004 however rescales the height coordinate and treats the time dependent free surface as a coordinate surface The finite volume method for discretizing the model within the z coordinate framework allows an accurate representation of topography by means of shaped volumes shaved cells or variable bottom layer thickness partial cells and has been demonstrated to overcome the inadequacies of height coordinates in representing topography Additionally the z coordinate allows for more accurate treatment of sea ice in the model by removing the problem of disappearing levels when the sea ice thickness exceeds the thickness of the upper levels 6 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland 000 5600 5200 3600 3200 2800 re 2400 2000 A A P A 1600 1200 800 400 i P o o o re 5200 3600 3200 2800 2400 2000 1600
24. AusCOM experiment the following minimum number of files are located here e oasis3 MPlIl exe the OASIS3 coupler executable e matm MPII exe the data atmospheric model MATM executable cice MPll exe np the sea ice model CICE executable compiled for using n processors e mom4 MPII exe the ocean model MOM4p1 executable calendar exe a calendar tool used in the runscript for calendar management mppnccombine exe a post processing tool used to combine the domain processor based MOMA history files do mppncombine ksh a script calling mppnccombine e environs vayu nci org au loads modules for the AusCOM compilation environment 5 2 2 exp All AusCOM experiments such as test 01 ciaf2 99 should be set under this directory Jobs are sumbmitted under their own run directories through a preprocessed runscript located there for example in one of the sample runs cd ciaf2 xx Is run auscom VAYU Once the job is submitted a new directory Running dir will be created here which is a symbolic link to the directory where the job is actually running see section 6 5 2 3 input Is input cice matm mom4 oasis3 These directories store the preprocessed data files namelist files etc required by each sub model and the coupler for system initialisation configuration and experiment control The following lists show the files that are essential for running an AusCOM experiment under input matm e core2
25. BHHBHHBHHHHHHBHHBHHBHHBHHHHHHBHHRHHBHHHRHHER SINIDATE This is the initial date of the run This is important only if FILLING analysis is used for a coupling field in the run The format is YY YYMMDD such as 19910101 19500101 SEND TPHERHHEHHHHHHHHHBHHBHHBHHHHHHBHHBHHBHHHHHHHHHBHHBHHHHHHHHHBHHRHHBHHHHH NR MODINFO Indicates if a header is encapsulated within the field brick in binary restart files for all communication techniques and for coupling field exchanges for PIPE SIPC and GMEM YES or NOT NOT END TPHERHHEHHHHHHHHHBHHBHHBHHHHHHBHHBHHBHHHHHHBHHBHHBHHBHHHHHHHBHHBHHBHHHHHHER NLOGPRT Index of printing level in output file cplout 0 no printing 1 main routines and field names when treated 2 complete output 1 SEND TPHRRHHBHHHHHHHHHBHHBHHBHHHHHHBHHBHHBHHBHHHHBHHBHHBHHBHHHHHHHBHHRBHHBHHHHHHER SCALTYPE Calendar type 0 365 day calendar no leap years 1 365 day or 366 days for leap years calendar n 21 n day month calendar This is important only if FILLING analysis is used for a coupling field in the run 0 END TPHERHHEHHHHHHHHHBHHBHHBHHHHHHBHHBHHBHHHHHHBHHBHHBHHBHHHHHHBHHRHHBHHHHHHER SSTRINGS The above variables are the general parameters for the experiment Everything below has to do with the fields being exchanged TPHEHHEHHHHHHHHHBHHBHHBHHHHHHBHHBHHBHHHHHHBHHBHHBBHHHHHBHHRBHHBHHHBRHNE note file cf name table txt does not include all the coupling flds listed bel
26. D WIND SPEED 1 UWND UWNDI ms 3 10M NORTHWARD WIND SPEED 1 VWND VWNDI ms 4 DOWNWARD SHORTWAVE RADIATION wa SWFLD SWFLXI m 5 DOWNWARD LONGWAVE RADIATION s LWFLD LWFLXI m 6 2M AIR SPECIFIC HUMIDITY n QAIR QAIRI g kg 7 RAINFALL RATE Lx i RAIN RAINI kgm s 8 SNOWFALL RATE 2 I SNOW SNOWI kgm s 9 PRESSURE Pa PRESS PRESSI 10 RIVER RUNOFF LEE RUNOF RUNOFI kgm s 02i Ocean to sea ice coupling fields 11 SEA SURFACE TEMPERATURE E SST SSTO 12 SEA SURFACE SALINITY psu SSS SSSO 13 ZONAL WATER SPEED E SSU SSUO ms 14 MERIDIONAL WATER SPEED 1 SSV SSVO ms 15 ZONAL SEA SURFACE GRADIENT 1 SSLX SSLX mm 16 MERIDIONAL SEA SURFACE GRADIENT 1 SSLY SSLY mm 17 POTENTIAL ICE FREEZE MELT HEATFLUX Goes FRZMLT PFMICE m i2a Sea ice to atmosphere coupling fields 18 SEA ICE SURFACE TEMPERATURE n ISST SST i20 Sea ice to ocean coupling fields 19 ZONAL ICE OCEAN STRESS peta IOSTRSU IOSTRSU gm s 20 MERIDIONAL ICE OCEAN STRESS NTE IOSTRSV IOSTRSV gm s 21 RAINFALL RATE ES IORAIN IORAIN kgm s 22 SNOWFALL RATE icd IOSNOW IOSNOW gm s 23 SALT FLUX 2 I IOSTFLX IOSTFLX kgm s 24 ICE MELTING HEAT FLUX S IOHTFLX IOHTFLX m 25 SHORTWAVE PENETRATING TO OCEAN wa IOSWFLX IOSWFLX m 26 LATENT HEAT FLUX DOWN wa IOQFLUX IOQFLUX m 27 SENSIBLE HEAT FLUX DOWN Wm IOSHFLX IOSHFLX m 28 LONGWAVE HEAT FLUX DOWN Wu IOLWFLX IOLWFLX m 29 RIVER RUNOFF ur IORUNOF IORUNOF gm s 30 PRESSURE ANOMALY Pa IOPRESS IOPRESS 31 SEA ICE CONCENTRATION
27. HECKIN INTERP CHECKOUT INSTANT INT 0 BICUBIC G SCALAR INT O0 IHHHHEHHHHHHHBHHHHEHHHHHHBHHHHHBHHHHBHHHHBHHHHBHHBHHBHHHHBHHHHHBHHBHHBHHBHHHHI H it ICE gt gt gt OCEAN Rn Te ae Sn e dos TERT Field 11 ice ocean interface stress x direction THERE strsu_io strsu_o 170 3600 1 120 nc IGNORED cice cice LAG 0 SEQ 1 H LOCTRANS INSTANT TIGBRHEHHUE Field 12 ice ocean interface stress y direction HET EH strsv_io strsv o 175 3600 1 120 nc IGNORED cice cice LAG 0 SEQ 1 H LOCTRANS INSTANT TIGBRHEHHUE Field 13 freshwater flux 50 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland TRHHHHHHHEE rain io rain o 27 3600 1 i20 nc IGNORED cice cice LAG 0 SEQ 1 it LOCTRANS INSTANT THT Field 14 freshwater flux THETA snow_io snow_o 28 3600 1 i20 nc IGNORED cice cice LAG 0 SEQ 1 it LOCTRANS INSTANT HATE Field 15 salt flux no ref no for saltflux yet THERE stflx io stflx_o 454 3600 1 i20 nc IGNORED cice cice LAG 0 SEQ 1 LOCTRANS INSTANT TIGBRHEHHUE Field 16 next heat flux heatflux into ocean 42 not right TIBBRHEHHUE htflx 1o htflx o 42 3600 1 120 nc IGNORED cice cice LAG 0 SEQ 1 it LOCTRANS INSTANT THAT Field 17 swflux penetrating through ice into ocean THETA swflx io swflx o 367 3600 1 120 nc IGNORED cice cice LAG 0 SEQ 1 H LOCTRANS INSTANT THERE Field 18 latent heat flux THERE qflux io q
28. HHEHHHHHHHHHBHHBHHBHHHHHHBHHBHHBHHHHHHBHHBHHBHHHHHHHHHBHHBHHBHHHRHHE CHANNEL The communication technique you want to use Choices are MPI1 or MPI2 NONE SIPC GMEM if you use MPI1 or MPI2 message passing CLIM library based on MP or MPI2 you must write MPI or MPI2 on one line one line per model giving for the model the total number of procs the number of procs implied in the coupling and for MPI2 only an optional launching argument MPIl NOBSEND 61 11 120 1 o NFIELDS This is the total number of fields being exchanged 10 fields atm gt ice HH 13 fields ice gt ocn Hi 7 fields ocn gt ice 1 fields ice gt atm 31 SEND ett EERE EERE EEE EEE EAE EEE AAAA AAA AAR HHHHHHH SJOBNAME This is an acronym for this run 3 characters OIA END TPHERHHEHHHHHHHHHBHHBHHBHHHHHHBHHBHHBHHHHHHBHHBHHBHHHHHHBHHBHHRBHHBHHHRHHE NBMODEL This gives you the number of models running in this experiment their names character 6 exactly in option the maximum Fortran unit number used by each model 1024 will be used if none are given 46 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland 3 cicexx matmxx mom4xx 99 99 99 END TPHERHHEHHHHHHHHHBHHBHHBHHHHHHBHHBHHBHHHHHHBHHBHHBHHBHHHHHHBHHRHHBHHHHHHNR SRUNTIME This gives you the total simulated time for this run in seconds 4 86400 1 day 2678400 31 days 31536000 SEND TPHRHHBHHHHHHHHHBHHBHHBHHHHHH
29. HHHHBHHBHHBHHHHHHNR Set up the running environment and run the auscom coupled model weh gi TE GE GE AusCOM is a coupled ocean and sea ice model consisting of 3 components 1 matm a data atmospheric model providing atmospheric forcing it 2 cice4 0 LANL sea ice model 3 mom4pl GFDL ocean model H built under the OASIS3 PRISM_2 5 framework This sample run is set up on the NCI VAYU platform for 128 processes 1 for cpl 1 for matm 6 for cice and 120 for mom4 mono cpu cpling TPHERHHBHHHHHHHHHBHHBHHHHHHBHHBHHBHHHHHHBHHBHHBHHHHHHBHHBHHBHHHHHHBHHRHHBNE H 0 Prologue TPHRHHBHHHHHHHHHBHHBHHBHHHHHHBHHBHHBHHBHHHRHHBHHBHHBHHHHHHBHHBSHBHHHHHHNR PBS P p66 PBS q normal PBS l walltime 1 40 00 PBS 1 vmem 128GB PBS I ncpus 128 PBS l software vampir PBS l other rms PBS M dave bi csiro au PBS N ciaf2 xx PBS wd kb date set e set xv ulimit s unlimited ulimit a Export System depending variables export MPIPROGINF DETAIL export F PROGINF detail export F FTRACE YES export MPLARGS export F SETBUF06 50000 export F SETBUF07 50000 export F SETBUF08 50000 export ATM COMPARAL 1 export F SYSLEN 300 export F SETBUF00 50000 MPIEXPORT F PROGINF F SYSLEN MPIEXPORT MPIEXPORT MPIPROGINF F FTRACE MPISUSPEND MPIEXPORT MPIEXPORT F SETBUF00 F SETBUF06 F SETBUF07 F SETBUFOS export MPI MULTITASKMIX ON 55 export MPIEXPORT MPIEXPOR
30. HRHUHRHEHRBHEHHHHEHHUHRHHHHRHEHHHBEHHBHRHHNBRHEHBRSNE H 2 Exp Run Time Control etc UGEBBEHHBBRHEHRHHEHHUHRHHHBRHEHHHBEHHUBRHEHHRHBEHHHBEHRHUHRHEHRRHEBHHBEHHRSHE H 2 1 Runtime control for the whole exp and this segment run H Initial and final date of the experiment iniyear 1948 finalyear 1950 typeset Z4 iniyear finalyear inimonth 1 finalmonth 12 typeset Z2 inimonth finalmonth 56 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland iniday 1 finalday 31 typeset Z2 iniday finalday Duration of this run maybe the most often visited place for test short runs nyear 0 number of years ALWAYS 0 change nmonth etc nmonth 12 number of months nday 0 number of days nmonth 0 if not a whole month set length nday xx below nday 5 days of this run important for quick tests Time steps dt_cpl_ai 21600 air ice coupling interval in seconds dt_cpl_io 3600 ice ocn coupling interval in seconds dt_oce 3600 oce model timestep dt_atm 3600 atm model timestep dt_ice 3600 ice model timestep H 2 2 Processor usage for this run Processor for each executable nproc cpl 1 Zalways 1 nproc_atm 1 1 nproc_ice 6 Zchangable nproc oce 120 Zchangable Total number of procs for this job must lt requested in the PSB line ntproc nproc atm nproc ice nproc oce nproc cpl Currently AusCOM is hardwired for mono cpu coupling ncplproc atm 1 ncplpro
31. MOMA on an identical global tripolar grid By doing so coupling CICE to MOM4 becomes relatively straightforward because of grid compatibility All coupling fields can be exchanged directly between the two models with no need of transformation e g vector rotation grid remapping interpolation etc by the coupler main process which to some extent enhances the model computational performance by reducing the workload of the mono cpu OASIS3 coupler Further MOMA and CICE both being Arakawa B grid models makes the coupling between them more efficient exchanged fields need no additional treatment such as grid point shifting before data sending in the source model or after data receiving in the target model CICE plays a very special role in the AusCOM coupled system As to be revealed in section 4 it functions as a coupling buffer or media where all coupling fields are gathered processed if required and then delivered to their receivers 3 4 Ocean Model MOM4 The MOMA ocean model is developed at and supported by the NOAA Geophysical Fluid Dynamics Laboratory GFDL MOMA is a community code with contributions by many scientists from collaborating instituations around the world including researchers at CSIRO Marine and Atmospheric Research A description of the MOMA framework and elements can be found in the technical report Griffies et al 2004 A detailed introduction to the theoretical aspects of the oceanic physics dynamics and various par
32. NSTANT INT 0 CONSERV LR SCALAR LATLON 10 FRACNNEI FIRST INT 0 HHT Field 06 runoff HATH runof ai runof 1 297 21600 4 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 POPO LOCTRANS CHECKIN SCRIPR CHECKOUT INSTANT INT 0 CONSERV LR SCALAR LATLON 10 FRACNNEI FIRST INT 0 TPERHHHHHHBE Field 07 near surface 2m air temp HiT tair_ai tair_i 110 21600 4 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 POPO LOCTRANS CHECKIN SCRIPR CHECKOUT INSTANT INT 0 CONSERV LR SCALAR LATLON 10 FRACNNEI FIRST INT 0 HATH Field 08 2m air humidity THERHHHHHHBE qair ai qair 1 339 21600 4 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 POPO LOCTRANS CHECKIN SCRIPR CHECKOUT INSTANT INT 0 CONSERV LR SCALAR LATLON 10 FRACNNEI FIRST INT 0 HHT Field 09 10m wind u HHT uwnd_ ai uwnd 1 56 21600 3 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 P 0P 0 HH ZCHECKIN SCRIPR CHECKOUT THE INT 1 DISTWGT LR VECTOR I LATLON 10 4 vwnd ai INT 1 49 uwnd ai uwnd i 56 21600 4 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 POPO it LOCTRANS CHECKIN INTERP CHECKOUT INSTANT INT 0 BICUBIC G SCALAR INT 0 THT Field 10 10m wind v THRE vwnd_ai vwnd 1 56 21600 3 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 P0P0 He CHECKIN SCRIPR CHECKOUT HH INT 1 DISTWGT LR VECTOR J LATLON 10 4 uwnd ai INT 1 vwnd_ai vwnd_i 56 21600 4 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 POPO it LOCTRANS C
33. Q must be set properly in the context of the coupling strategy to avoid coupling deadlock See Valcke 2006 for detailed discussions The third line contains 4 entries P source grid first dimension characteristic P for periodical and R for regional 0 source grid first dimension number of overlapping points none for nt62 P target grid first dimension characteristic 0 target grid first dimension number of overlapping points none for cice 4 Four transformations for this field are listed in line 4 LOCTRANS time transformation CHECKIN preprocessing namely calculating the field mean and extreme values on the source grid and printing them to the coupler log file cplout The other available and useful preprocessing transformations in OASIS3 include MASK EXTRAP CORRECT 29 e SCRIPR interpolation transformation OASIS3 also supports BLASOLD INTERP MOZAIC NOINTERP and FILLING for different applications e CHECKOUT postprocessing namely calculating the field mean and extreme values on the target grid and printing to cplout 5 Line 5 configures the time transformation LOCTRANS e INSTANT the instantaneous field is transfered i e no time transformation The supported time transformations include INSTANT ACCUMUL ACERAGE T MIN T_MAX and ONCE See Valcke 2006 for full descriptions Note for all the AusCOM coupling fields the only time transformation needed is time average and it is done in sub mo
34. RIP A spherical coordinate remapping and interpolation package LA CC Number 98 45 27pp Large W and Yeager S 2004 Diurnal to decadal global forcing for ocean and sea ice models the data sets and flux climatologies CGD Division of the National Center for Atmospheric Research NCAR Technical Note NCAR TN 460 STR Large W G and Yeager S 2009 The global climatology of an interannually varying air sea flux data set Climate Dynamics 33 doi 10 1007 500382 008 0441 3 Murray R J 1996 Explicit generation of orthogonal grids for ocean models Journal of Computational Physics 126 251 73 Roberts J L Heil P Phipps S J and Bindoff N L 2007 AusCOM The Australian community ocean model Journal of Research and Practice in Information Technology 39 2 137 150 Valcke S 2006 OASIS3 User Guide PRISM Support Initiative Report No 3 64pp Winton M 2001 FMS sea ice simulator NOAA Geophysical Fluid Dynamics Laboratory 11pp 45 APPENDIX A A SAMPLE OASIS3 COUPLING CONFIGURATION FILE NAMCOUPLE This is a typical input file for OASIS 3 0 using netCDF format for restart input files Oasis reads this file at run time Any line beginning with is ignored Blank lines are not allowed SEQMODE This keyword concerns the coupling algorithm Put here the maximum number of fields that have to be at one particular coupling timestep necessarily exchanged sequentially in a given order 1 END TPHR
35. S3 coupling framework see Fig 1 This document describes the AusCOM system framework the coupling strategy and the interfaces designed for using the OASIS3 25 coupler Section 2 introduces the AusCOM framework along with details of the horizontal and vertical discretisation in the default setup The coupler along with each of the component models are briefly introduced in Section 3 AusCOM users are directed to the user guides provided by the developers of the MOMA ocean code Griffies et al 2004 and the CICE sea ice code Hunke and Lipscomb 2008 for more detailed scientific and technical descriptions of these ocean and sea ice component codes Section 4 is for users who are keen to fully understand the coupling logic re mapping algorithms and associated Message Passing Interface MPI technology used in the AusCOM system It should be read in conjunction with the OASIS3 25 coupler user guide Valcke 2006 the SCRIP package user guide Jones 1997 and any MPI reference Section 5 gives technical details of the AusCOM infrastructure information for compilation of the system components a description of various configuration files and notes on the model run script A quick start guide is provided in Section 6 The CAWCR ACCESS Coupled and Ocean Modelling Team provides users with the AusCOM1 0 release which is a compressed tar file containing full packages of the four components their associated auxiliary input files and a few sample exper
36. STANT THETA Field 27 northward sea surface water velocity TERT ssv_oi ssv_i 261 3600 1 o2i nc IGNORED cice cice LAG 0 SEQ 1 it LOCTRANS INSTANT THETA Field 28 frazil ice fromation energy J m 2 441 is upward_sea_ice_basal_heat_flux W m 2 THETA pfmice_ oi pfmice 1441 3600 1 021 nc IGNORED ptice oi pfmice 1441 3600 1 o2i nc IGNORED cice cice LAG 0 SEQ 1 H LOCTRANS INSTANT THETA Field 29 sea surface slope x m m H 203 is height THT sslx oi sslx_1 203 3600 1 o2i nc IGNORED cice cice LAG 0 SEQ 1 LOCTRANS INSTANT THETA Field 30 sea surface slope y m m 310 is sea surface elevation THETA ssly oissly 1310 3600 1 o2i nc IGNORED cice cice LAG 0 SEQ 1 it LOCTRANS INSTANT E ICE gt gt gt ATMOSPHERE LLL EE THT Field 31 ice ocean surface temp no ref no 331 is snow temp THETA isst la isst a 331 21600 4 i2a nc EXPORTED cice nt62 LAG 3600 SEQ 1 POPO it LOCTRANS CHECKIN SCRIPR CHECKOUT INSTANT INT 0 53 CONSERV LR SCALAR LATLON 10 FRACNNEI FIRST INT 0 TIHHHHBHHHHBHHHHBHHHHBHHBHHHHBHHHHBHBHHBHBHHHHBHHHHBHHHHHHBHHHHBHHHHBHHHNE SEND 54 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland B A SAMPLE AUSCOM RUNSCRIPT bin ksh TPHRHHEHHHHHHHUHBHHBHHBHHHHHHBHHBHHBHHBHHHHRHHBHHBHHBHHHHHHBHHBHHBHHHHHHRR run auscom VAYU TPHRHHBHHHHHHHUHBHHBHHBHHHHHHBHHBHHBHHBHHHHHHBHHBHHBHH
37. T MPI MULTITASKMIX export MPI BUFFER MAX 5000000 A AAA A RAAR AARAA HARARE EREHEHH H 1 Primary Setups GEUBHHHBHRHEHRHHEHHUHBRHHHBRHEHHHHEHHUBRHHHHRHBEHHHBEHRUHRHEHRHRHEHHHBEHHRUHE H 1 1 Define experiment ID etc it project p66 short disk owned by project e g p66 jobid PBS JOBID job id assigned by PBS the queue sys job PBS JOBNAME name of this script chan MPI1 Message Passage MPI1 MPI2 expid ciaf2 xx change expid for each new experiment atmdata core2 choose the atmospheric forcing dataset atm forcing atmdata ncep2 era40 core2 or um96 etc datatype IA NY IA Normal Year Interannual Annual year data end 2007 data NOT available after this year 4 1 2 Define all associated paths Location where jobs are submitted and this script is located cd pwd AusCOMHOME pwd model AusCOMHOME the model name i e AusCOM1 0 jobdir AusCOMHOME exp expid Location of preprocessed input files for the coupled model inputdir AusCOMHOME input Location where the model executables are stored bindir AusCOMHOME bin Location where outputs are to be stored datahome short project SUSER OUTPUT outputdir datahome model Sexpid restdir outputdir restart histdir outputdir history Location where the sub models and coupler are actually runing workhome short project SUSER rundir workhome RUNNING model Sexpid VIBHBHHUBRHEHRHHEHHHHBE
38. WARRANTY without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE See the GNU General Public License for more details http www gnu org license AusCOM1 0 incorporates the CICE 4 0 software from the Los Alamos National Laboratory Use of CICE is covered by the following copyright O Copyright 2008 LANS LLC All rights reserved Unless otherwise indicated this information has been authored by an employee or employees of the Los Alamos National Security LLC LANS operator of the Los Alamos National Laboratory under Contract No DE AC52 06NA25396 with the U S Department of Energy The U S Government has rights to use reproduce and distribute this information The public may copy and use this information without charge provided that this Notice and any statement of authorship are reproduced on all copies Neither the Government nor LANS makes any warranty express or implied or assumes any liability or responsibility for the use of this information Beginning with version 4 0 the CICE code carries Los Alamos Software Release number LA CC 06 012 8 REFERENCES Adcroft A and Campin J M 2004 Rescaled height coordinates for accurate representation of free surface flows in ocean circulation models Ocean Modelling 7 269 84 44 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland Bi D Marsland S J Collier M and Griffies S M 2010 Benchmarking the Australian C
39. a Ice Model CICE The Los Alamos Sea Ice Model CICE Hunke and Lipscomb 2008 Hunke 2001 Hunke and Dukowicz 1997 was initially designed as a computationally efficient sea ice component for a fully coupled atmosphere ice ocean land global climate model especially compatible with the Los Alamos Parallel Ocean Program POP ocean circulation model POP which is the ocean component of the National Centre for Atmospheric Research NCAR Community Climate System Model CCSM fully coupled model In recent years through collaboration with other institutions such as NCAR and UK Met Office the CICE4 0 release has been largely enhanced in terms of its technical and physical compatibilities with different models In particular CICE now supports tripolar grids and a Zero Layer thermodynamic configuration to allow the overlaying atmospheric model to calculate the ice surface temperature These developments have made it possible or at least easier for other users to adopt CICE into their own coupled systems CICE consists of several interacting componments a thermodynamics module computing local growth rates of snow and ice due to vertical conductive radiative and turbulent fluxes an ice dynamics module predicting the ice pack velocity field based on a model of the ice material strength a transport module describing advection of ice concentration volume and other properties and a ridging module transfering ice among thickness categories default 5 c
40. a coupling field including a symbolic name as defined in the namcouple file b partition identification c array dimensions d stamp for sending or receiving e data type etc This subroutine also allocates all coupling field arrays and the associated temporary arrays which are all defined in a separate module cpl arrays 4 2 3 from atm This routine integrates all atmospheric data receiving operations Only one PSMILe routine is involved prism get proto it is called repeatedly to receive each atmospheric field declared in cpl init and listed in the OASIS coupling namelist file namcouple 14 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland Routine from atm also supports a post processing operation namely rotating the received vector wind from the regular atmospheric latitude longitude mesh grid onto the CICE tripolar grid in case the two wind components are passed in as scalar arrays defined in the namcouple file Doing this and other kinds of transformations in a sub model instead of in OASIS3 reduces the workload of the mono processor coupler and enhances the coupling efficiency 4 2 4 from ocn This subroutine integrates all oceanic data receiving operations As with from atm the PSMILe routine prism get proto is called repeatedly to receive each oceanic field declared in cpl init and listed in the namcouple file 4 2 5 into ocn This subroutine handles all o
41. ameterizations realised in the model numerics can be found in Griffies 2004 We have adopted the MOMA model as the AusCOM ACCESS ocean component for these reasons MOM is a recognised leading world class ocean models code there is a long history of MOM usage in Australia with associated development of in house expertise synergy is maintained with the other major CAWCR ocean modelling theme involving Bluelink and associate projects which focus on eddy permitting regional to global scale modelling using MOM4 there is a long and fruitful history of collaboration between CSIRO and GFDL scientists on ocean modelling which we wish to continue and enhance MOMApl is originally configured within the GFDL Flexible Modelling System FMS please see www gfdl noaa gov allowing a wide range of convenience in modelling management such as pre and post processing time control coupler and data over riding diagnostic managing and so on It is already coupled to the SIS sea ice model Winton 2001 via the FMS coupler in the standard MOMA4p1 release package 11 For connecting MOMA to CICE via the OASIS3 coupler we drop the FMS coupling framework at compile time and replace it with a new coupling interface in the model code that links MOMA to OASIS3 Controls over modelling time and coupling frequency etc are also handed over to the OASIS3 coupler However large parts of the functions provided by the FMS package can still be used by
42. ategories based on energetic balances and rates of strain The original motivation for using CICE as the sea ice component of AusCOM ACCESS was on the basis of it being a recognised leading sea ice code However as the Met Office Unified Model UM has been chosen to be the atmospheric component of the ACCESS coupled model and UM is specifically configured to be coupled to the NEMO CICE ocean sea ice model our using CICE becomes a natural choice The CICE code can be run in a standlone mode with an active boundary module handling the required atmospheric and oceanic forcing or directly coupled to an ocean model via a supported external flux coupler i e the NCAR CCSM coupler It can also function as a sea ice module subroutine in an ocean model such as the ocean component NEMO of the Met Office HadGEM3 coupled system In AusCOM CICE is coupled to MOM4 and MATM via the OASIS3 coupler allowing for relatively flexible data exchange and coupling frequency control For doing so the original data exchange functions in CICE supporting the NCAR CCSM coupler have been dropped and a new coupling interface which consists of a set of subroutines dealing with PSMILe libraries 10 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland has been developed and implemented in the code to facilitate connections between CICE and MATM and between CICE and MOM4 via OASIS3 As shown in Section 2 AusCOM configures CICE and
43. bal gathering and scattering in the sub models and therefore does not slow down the sub models However in parallel coupling all processors of all sub models communicating with the coupler does not necessarily enhance the computing efficiency of the coupled system In contrast because the coupler takes over all the gathering and scattering duties which would otherwise be undertaken by sub model local communicators in the mono processor coupling case the system coupling efficiency actually becomes lower highlighting the coupling bottleneck This is especailly true when the system such as the ACCESS fully coupled model has a large number of coupling fields and requires higher coupling frequency This is the reason why we have chosen the mono processor approach over the parallel coupling approach for our AusCOM and ACCESS coupled systems 20 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland 5 A TECHNICAL GUIDE TO THE AUSCOM SYSTEM This section is a full technical manual for users to install the AusCOM system design experiments and set up runs for their own applications 5 1 Platforms A majority of the AusCOM development was undertaken on the National Computational Infrastructure National Facility NCI NF SGI cluster XE and Sun Constellation cluster VAYU platforms The AusCOMI 0 model with IPCC class configuration has been running on VAYU the current peak system at NCI for many multi century s
44. box ES ae eh ae ee A at EE O a i ee bsend yes needs larger buffer size for MPI Bsend operation to make it work we ve doubled il bufsendsize in oasis3 src inicmc F bsend no this one works fine and is recommended if bsend no then nobsend NOBSEND else nobsen fi if chan MPI1 then argl arg2 arg3 fi Ln get and adapt file namcouple cd rundir cp f inputdir oasis3 namcouple_31fields namcouple ed namcouple lt lt eof g Channel s Channel chan nobsend g Modlprocs s Mod1procs npt1 npcl arg1 g Mod2procs s Mod2procs npt2 npc2 arg2 g Mod3procs s Mod3procs npt3 npc3 arg3 g Mod1_name s Mod1_name nam1 g Mod2_name s Mod2_name nam2 g Mod3_name s Mod3_name nam3 g Runtime_sec s HRuntime_sec runtime g Inidate s Inidate date g Caltype s Caltype caltype g HNLOGPRT s HNLOGPRT nlogprt g CPL_intv_ai s CPL_intv_ai dt_cpl_ai g CPL_intv_io s CPL_intv_io dt_cpl_io g DT_OCE s HDT_OCE dt_oce g DT_ATM s HDT_ATM dt_atm 63 g DT_ICE s HDT_ICE dt_ice wW q eof 3 2 2 namelist for matm coupling cat gt input atm nml lt lt eof amp coupling init date iniyear inimonth iniday inidate date truntime0 truntime0 runtime runtime dt_cpl dt_cpl_ai dt_atm dt_atm dataset atm_forcing runtype datatype caltype caltype days per year days this yea
45. c ice 1 ncplproc oce 1 Decide ocean domain MPI partitioning pattern oce nx 8 oce ny 15 Zoce nx x oce ny nproc oce HH 2 3 Names of the 4 executables oa3 exe oasis3 atm exe matmxx ZThese 3 sub model exe names must be same as ice exe cicexx defined in namcouple and model code oce exe mom xx character 6 2 4 Calendat date control Calendar type available calendar options 0 No leap year 365 days per year H 1 Gregorian 365 366 days per year n Equal months of n days 30 for 30 day months 57 Default set as below for era40 and ncep2 forcing caltype 1 0 1 or n eg 30 for 30 day months cal type julian Hor caltype 1 thirty day for caltype 30 For core and core2 forcing we must use below if Satm forcing core Satm forcing core2 then caltype 0 cal_type NOLEAP fi Dates in format YYYYMMDD inidate iniyear inimonth iniday finaldate finalyear finalmonth finalday cd jobdir typeset Z4 year typeset Z2 month day if f expid date then year iniyear month inimonth day iniday jobnum 1 1 for initial run gt 1 for continue runs truntime0 0 total accumulated runtime by the end of last run if f expid log then rm expid log fi echo date Beginning of Experiment expid gt expid log else read year month day jobnum truntime0 lt expid date fi
46. c year 500 access owner if run under HOME access short p66 owner if run under short p66 disk typeset Z4 ic year ic yearpl ic_year 1 typeset Z4 ic yearpl rest_date_oasis ic_year 1231 rest_date_mom4 ic_year 1231 59 rest date cice ic yearp1j0101 ic_location access model output expname restart mom4 ic ic location mom4 cice ic ic location cice oasis ic ic location oasis3 fi if jobnum 1 J then initial run rm rf restdir rm rf histdir rm rf AusCOMHOME output Sexpid mkdir p restdir cice restdir mom4 restdir oasis3 mkdir p histdir cice histdir mom4 In s outputdir AusCOMHOME output rm fr rundir mkdir p rundir In s rundir Sjobdir Running dir cd rundir mkdir MATM input Zsubdirs for MATM mkdir CICE input CICE restart CICE hist subdirs for CICE mkdir MOMA input MOMA restart MOMA hist Zsubdirs for MOMA get the executables cp f bindir oasis3 Schan exe 0a3 exe cp f bindir mom4 chan exe oce exe cp f bindir cice Schan exe nproc_ice p ice exe cp f bindir matm chan exe nt62 atm exe get input files for oasis3 a ref and grids data cp f Sinputdir oasis3 cf name table txt cp f inputdir oasis3 grids cice nt62 nc grids nc cp f Sinputdir oasis3 masks cice nt62 nc masks nc cp f Sinputdir oasis3 areas cice nt62 nc areas nc b restart if cold start 1 then cold start the pre processed coupling restart files
47. ciences community including the Bureau of Meteorology CSIRO and the Australian universities for ocean climate research and applications It was conceived in discussions and informal workshops involving wide representation from the Australian ocean climate modelling community during 2003 and 2004 and initial prototype components were implemented at the Tasmanian Partnership for Advanced Computing TPAC Roberts et al 2007 Heil et al 2005 The current configuration was developed at the Centre for Australian Weather and Climate Research CAWCR a partnership between the Commonwealth Scientific and Industrial Research Organisation and the Australian Bureau of Meteorology primarily by staff in the Ocean and Coupled Modelling Team of the CAWCR Earth System Modelling Program AusCOM comprises the Geophysical Fluid Dynamics Laboratory Modular Ocean Model version 4 1 GFDL MOM4p1 the Los Alamos National Laboratory Sea Ice Model version 4 0 LANL CICE4 0 and a data atmospheric model MATM Numerical coupling i e data exchange between the sub models is strictly constrained by the PRISM 2 5 OASIS3 coupler hereafter refered to as the OASIS3 25 coupler or simply OASIS3 AusCOM is the core component for climate purposes of the Australian Community Climate and Earth System Simulator ACCESS Specifically the ACCESS model is built by coupling the UK Met Office Unified Model UM atmosphere and other sub models as required to AusCOM via the OASI
48. cp f inputdir oasis3 a2i_10fields nc a2i nc cp f Sinputdir oasis3 021 7fields nc o2i nc cp f inputdir oasis3 i20_13fields nc 120 nc cp f inputdir oasis3 i2a_1fields nc i2a nc else warm start rstart from an existing run spinup cp f oasis_ic a2i nc rest_date_oasis a2i nc cp f o0asis_ic o2i nc rest_date oasis o2i nc cp f oasis ic i2o nc rest date oasis i20 nc cp f oasis ic i2a nc rest date oasis i2a nc fi 60 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland get input files for matm to be done in section 4 input files for cice if runoff data core then cp f Sinputdir cice core runoff regrid nc CICE input fi a grid data and surface cp f inputdir cice cice grid nc CICE input grid nc cp f inputdir cice cice_kmtu ne CICE input kmt nc b IC restart if cold start 1 then cold start runtype initial Lrestart false ice_ic default cp f Sinputdir cice A21 time0 10fields nc CICE input A2I time0 nc cp f Sinputdir cice SSTS 12Jans nc CICE input monthly sstsss nc if Sboundary layer gfdl then cp f inputdir cice uu_ star t0 nc CICE input u star nc fi else warm start runtype continue Lrestart true ice ic default ice restart cice ic iced rest date cice cp f ice restart CICE restart iced echo iced gt CICE restart ice restart file cp f cice_ic A2I timel nc rest date oasis CICE input A2I time0 nc if Sboundar
49. d log echo Error code at end of simulation TEHBHBHHHHHHHHBHHHHHBHHBRHHHHHHHHHHBHHHHHHHHHHHHHHHBHHBRHSHHBHHHHHHBHHHHBHHBE f 5 Postprocessing Saving the Output Data HU THBEHBHHBHHBHBHHHHHHHHBHHHHHHHBEHBHHHHHHHHBHHHHBHHHHHHHHHHHHHBRHHSHHHHHHHHBHE cd rundir rm MATM input Sendyear nc remove the used forcing data 4 5 1 Output files of the coupler OASIS3 Restart files for resfile in Is 2 nc do mv resfile restdir oasis3 resfile enddate done H 5 2 Output files of the ocean mom4 H Restart files cd rundir MOM4 restart for restfile in ls ocean do mv restfile restdir mom4 restfile enddate done History files cd rundir MOM4 hist sdir rundir MOM4 hist tdir histdir mom4 tool bindir do_mppncombine ksh archjob qsub tool v bdir bindir sdir sdir tdir tdir idate enddate 5 4 Output files of the ice cice H cd rundir Restart files mv CICE input A2I timel nc frestdirj cice A21 timel nc enddate mv CICE restart ice restart file restdir cice ice restart file enddate mv CICE restart iced restdir cice if fu star nc then mv u star nc restdir cice u_star nc enddate fi History files iceh_ nc mv CICE hist histdir cice 69 H 5 5 Store coupling fields output if any e g fields 120 in ice nc etc i if Is fields nc wc w gt 0 then for tmpfile in Is fields
50. dels Communication between a sub model and OASIS3 only occurs at actual coupling time points avoiding unnecessary data exchanges and time transformations in OASIS3 and therefore reduing its workload 6 Line 6 configures the preprocessing transformation CHECKIN e INT 0 1 or 0 whether or not calculating and printing the source field integral 7 Line 7 configures the interpolation transformation SCRIPR SCRIPR in PSMILe adopts the LANL CSRIP1 4 interpolation software Jones 1997 and offers the following types of interpolation DISIWGT GAUSWGT BILINEAR BICUBIC and CONSERV Each of them has advantages and disadvantages and may be used for different circumstances For AusCOM the CONSERV algorithm is chosen to perform interpolations for all fields although it is only necessary for those involving mass and energy fluxes e CONSERV performs first or second order conservative remapping The following entries further configure this interpolation algorithm e LR the source grid type Logically Rectangular e SCALAR field type e LATLON search restriction type e 10 number of restriction bins e FRACNNEI normalisation option for the SCRIPR CONSERV interpolation algorithm The other two alternative options are FRACAREA and DESTAREA Extra caution needs to be taken for choosing the proper one otherwise the model might not behave due to abnormal products of the interpolation FRACNNEI is our choice for AusCOM after numerio
51. e real definition 28 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland of the coupling field In fact filed swfld ai is actually defined and used as total surface downward shortwave flux W m y in the AusCOM model code 21600 coupling interval in seconds for this field 4 number of transformations to be performed by coupler on this field These 4 transformations are specified below line 4 here followed by their own configurations one line each in order a2i nc coupling restart filename character 8 In AusCOM coupling fields in the same group share one restart file EXPORTED field status indicating that this field is to be exchanged between component models and transformed by OASIS3 main processor OASIS3 supports a number of field status all named in capital AUXLIARY EXPORTED EXPOUT IGNORED IGNOUT INPUT and OUTPUT standing for different treatments for the coupling field Please see the OASIS3 User Guide Valcke 2006 for detailed descriptions 2 The second line has 4 entries nt62 4 character string source grid name in grid datafiles where all grid information variables must use their grid name as the prefix of the variable names e g nt62 lon cice ang etc cice 4 character string target grid name cice is the AusCOM MOM4 CICE tripolar grid LAG 3600 lag index for this field 3600 is the MATM time step in seconds SEQ 1 sequence index LAG and SE
52. eed to use PSMILe libraries Therefore we should first compile the PRISM_ 2 5 libraries and generate the OASIS3 executable The three sub models can then be built in any order As already mentioned a building directory compile has been set up for each of the sub models and each sub model can be built using the provided script under compile in a very simple way 5 3 1 PSMILe libraries and OASIS3 cd AusCOM1 0 submodels oasis3_ prism 2 S prism compile comp oasis325 VAYU Note the compiled PRISM 2 5 librares and object files are stored at Linux lib build The OASIS3 executable oasis3 MPIl exe is moved to AusCOM1 0 bin immediately after being generated where the sub model executables are also stored 5 3 2 MATM cd AusCOM1 0 submodels matm compile comp auscom matm VAYU Note as mentioned earlier MATM resolution is dataset dependent and must be determined at compile time For example to handle the CORE forcing fields which are on the nt62 grid on a 192x94 mesh MATM must be compiled specifically for this grid This can be achieved by setting two variables GRID and RES in the compiling script comp auscom matm VAYU Grid resolution setenv GRID nt62 setenv RES 192x94 25 Consequently the generated MATM executable is labelled with the grid name nt62 to distinguish it from other MATM executables compiled for different grids such as the um96 UKMO Unified Model grid used in the ACCESS fully coupled model
53. ents in code frazil factor 1 0 CH MOMA and CICE use same two timelevel stepping cat gt input oce nml lt lt eof amp coupling runtime runtime dt cpl dt cpl io dt mom4 dt oce pop icediag POP ICEDIAG do ice once false kmxice 5 fixmeltT false Tmelt 216 use_ioaice true aice_cutoff 0 15 chk_i20_fields false chk_02i_fields false icemlt_factor icemlt_factor frazil factor frazil factor iceform adj salt false sign stflx 1 0 amp end eof TIBHHHHRHHHHHRHHHHRHHBHRHHBHRHHBHHHHRRHHHHRHHBHRHHBHRHHBHRHH RH HH H 4 Launch Execute the AusCOM Coupled Model on VAYU HOHBHHHBHHBHRHHBHRHHBHHHHRRHHHHRHHBHRHHBHRHHBHRHHBRHHHHHRHHBHRHHBHRHHBRHEH E set e echo date jobnum date starting mpirun mpiexec gt gt jobdir expid log echo echo mpirun mpiexec started at date echo Z David Singleton s solution to the bad perforamce of model export PATH opt anumpirun 2 1 16a bin SPATH mpirun n nproc_cpl 0a3 exe n nproc_ice ice exe A n nproc atm atm exe n nproc oce oce exe mpirun mca mpi paffinity alone 1 n nproc cpl 0a3 exe n nproc ice ice_exe n nproc_atm atm_exe n nproc_oce oce_exe echo echo job completed at date 68 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland echo echo date jobnum enddate done mpirun mpiexec gt gt jobdir expi
54. flux o 452 3600 1 120 nc IGNORED cice cice LAG 0 SEQ 1 H LOCTRANS INSTANT THETA Field 19 Sensible heat flux THERE shflx io shflx o 362 3600 1 i20 nc IGNORED cice cice LAG 0 SEQ 1 it LOCTRANS INSTANT 51 TIGBRHEHHUE Field 20 LW radiation flux down THERE Iwflx io lwflx o 366 3600 1 i20 nc IGNORED cice cice LAG 0 SEQ 1 it LOCTRANS INSTANT TIGBRHEHHSE Field 21 runoff THT runof 1o runof o 297 3600 1 i20 nc IGNORED cice cice LAG 0 SEQ 1 H LOCTRANS INSTANT THETA Field 23 surface pressure THT HE press io press o 33 3600 1 i20 nc IGNORED cice cice LAG 0 SEQ 1 it LOCTRANS INSTANT TTA Field 23 ice concentration TTA aice io aice o 44 3600 1 i20 nc IGNORED cice cice LAG 0 SEQ 1 LOCTRANS INSTANT VIHHHHHBHRHEHBHHEHHUHHHHHBRHEHRHHEHHHHRHHHBRHBEHRHHEHRUBRHBEHBRHEHHRHBEHHNE H OCEAN gt gt gt ICE Hee ee ae A WIGBRHEHHUE Field 24 Sea surface temperature Celsius in MOM4 BIBBRHEHHUE sst oisst 11 3600 1 o2i nc IGNORED cice cice LAG 0 SEQ 1 LOCTRANS INSTANT TIBBRHEHHUE Field 25 Sea surface salinity psu THHHHHHHHBE sss oi sss 1312 3600 1 o2i nc IGNORED cice cice LAG 0 SEQ 1 it LOCTRANS INSTANT THETA Field 26 eastward sea surface water velocity THETA ssu oissu 1181 3600 1 o2i nc IGNORED 52 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland cice cice LAG 0 SEQ 1 it LOCTRANS IN
55. for the i20 fields Those explanations for all parameters apply here except for the source and target names being swapped Finally the ice to atmosphere 12a field is configured as follows Field 31 ice ocean surface temperature isst ia isst a 1 21600 4 i2a nc EXPORTED cice nt62 LAG 3600 SEQ 1 POPO LOCTRANS CHECKIN SCRIPR CHECKOUT INSTANT INT 0 32 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland CONSERV LR SCALAR LATLON 10 FRACNNEI FIRST INT 0 This only i2a field which is also the last coupling field listed in the namcouple file is configured the same way as that for the a2i fields discussed earlier except for the source and target names being swapped It must be stressed again that due to the complexity of the OASIS3 coupling philosophy and a wide range of choices for coupling algorithms creating a functional coupling configuration file especially the coupling fields configuration part is not an easy task This sample namcouple file works fine but is not necessarily a very reasonable and efficient design for the AusCOM system s high standard coupling performance AusCOM users with good experience in OASIS3 are strongely encouraged to explore deeper into the interpolation algorithms and other aspects and test their own preferred schemes With all the auxiliary files preprocessed the design of an AusCOM experiment is finalised and the model is well almost ready to run 5 5 T
56. gain a few missing files 39 6 AUSCOM QUICK START ON NCI VAYU For the quickest start possible a short script Setup AusCOM1 0 ksh is provided for users to set up their own independent self contained AusCOM1 0 system on the NCI VAYU platform Under your home directory recommended or other places where you want to install the AusCOM1 0 system run the script home 599 dhb599 Setup AusCOMI 0 ksh It should only take 20 to 25 minutes to instal and compile the component models The fully compiled AusCOM1 0 system takes about 427 MB of disk space Now everything is in place and the model is ready to run The following Sections 6 1 3 give a more detailed description of the steps taken in the installation script The remainder of Section 6 gives advice on running the sample experiments provided with the package 6 4 monitoring the progress of runs 6 5 and finding the model outputs at the end of a run 6 6 6 1 Getting the AusCOM1 0 Release Package The whole package of the AusCOM version 1 0 release is a compressed tar file named AusCOM1 0 tar gz about 92 MB located at VAYU home 599 dhb599 It contains the whole AusCOM system except for the supported CORE forcing datasets due to size limitations Copy it to your home directory recommended or somewhere on the short disk and unpack it cp home 599 dhb599 AusCOM1 0 tar gz gunzip AusCOM1 0 tar gz tar xvf AusCOM1 0 tar wm AusCOM1 0 tar cd AusCOM1 0
57. he Runscript run auscom VAYU 5 5 1 Introduction A single runscript named run auscom VAYU is used to set up and run AusCOM on the NCI VAYU platform This UNIX script is written in the Korn shell ksh programming language and contains sections performing various management tasks including system initialisation run time calendar control IO management job re submission and so on In addition it supports run time modifications to some key configuration files for flexible experiment and run setup which of course unavoidably increases the file length This is proven very helpful in many circumstances especially for test runs that need a multitude of tuning work A sample runscript file is provided in this document as an important part Appendix B for users to examine before they conduct any AusCOM experiments In the AusCOM1 0 release package it is located under the three sample run directories Although it is quite self descriptive some key parts of this multi hundred line script need more detailed explanation This introduction is aimed to help users to set up their own experiments Therefore any parts associated with a specific run setup will be highlighted for users attention Those parts that are required for general or common use may just be mentioned or simply skipped In addition here we follow the section sequence in the script for reader s convenience of target locating but prefix the section numbers with characters R to distingui
58. he indicated core part AusCOM when runing alone as AusCOM1 0 actually includes the data atmospheric model MATM in the place of UM plus different coupling interface for CICE multi layer sea ice physics configuration Sea Ice CICE 4 0 Fig 1 ACCESS AusCOM Coupled Ocean and Sea Ice Model Framework and Components Horizontally the AusCOM ocean model MOM4p1 is configured with an orthogonal curvilinear grid Fig 2 A singularity at the north pole is avoided by using a tripolar grid following Murray 1996 This approach also provides reasonably fine resolution in the Arctic Ocean which enhances the computational efficiency and accuracy of the model The AusCOM sea ice model CICE is configured to share this tripolar grid with MOMA An important advantage of doing so in terms of ice ocean coupling performance will be addressed below The orthogonal curvilinear spatial discretization is shown in Fig 3 Along the curvilinear zonal direction AusCOM has a regular spaced grid with 1 resolution In the meridional direction the grid spacing is nominally 1 resolution with the following three refinements tripolar Arctic north of 65 north e equatorial refinement to 1 between 10 S and 10 N anda Mercator cosine dependent implementation for the Southern Hemisphere ranging from 0 25 at 78 S to 1 at 30 S For the AusCOM1 0 release documented here MOM4p1 and CICE4 0 are horizontally configured on a global tripolar grid with a 3
59. he sub model coupling namelist files users can find a number of parameters associated with run time coupling field checks In input ice nml for example we have the default setting chk_i20_fields false If it is set to true CICE will record all the ice to ocean fields at every coupling interval in a NetCDF file named fields 120 in ice nc This file and similarly others if also written can be very useful for checking if the coupling fields cause abnormal behaviour of the models which happens quite often when a new experiment is started using new model code new coupling algorithms and or new forcing datasets etc These checks add computational expense and should be turned off i e set to false once the AusCOM system is shown to be robust Now the preparation is completed and the model is ready to go Users thus do not need touch the remaining sections R 5 and R 6 but might like to familiarise themselves with these nevertheless R 4 Launch Execute the AusCOM Coupled Model on VAYU 38 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland mpirun n nproc cpl 0a3 exe n nproc ice ice_exe n nproc atm atm exe n nproc oce 0ce exe R 5 Postprocessing Saving the Output Data Upon the successful completion of a segment run output data history and restart is ready to be written to the storage disk space This section handles all output files moving them to the destinitio
60. ice restart_file prevdate CICE restart ice restart file cp restdir cice cat CICE restart ice restart file CICE restart cp restdir cice u_star nc prevdate CICE input u star nc for mom4 for restfile in ls restdir mom4 ocean prevdate do ncfile restfile cp restfile MOM4 input ncfile done fi initial or continue run prepare the atm_ forcing dataset needed for this run cd rundir MATM_ input typeset Z4 yl y2 y 1 endyear y2 endyear if datatype NY then yl 1 for NY forcing yl should be always 0001 fi inputdir matm get_ atmdata datatype ksh y1 y2 AusCOMHOME y2 expr endyear 1 yl y2 if datatype NY then y1 1 fi if Sendyear year data end then 62 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland y 1 endyear fi inputdir matm get_ atmdata datatype ksh y1 y2 AusCOMHOME 3 2 Adapting or creating configuration files 3 2 1 namelist for oasis3 nlogprt 1 cplout writing control 0 no 1 medium 2 full output npt1 nproc_ice npcl ncplproc_ice argl ice_exe naml ice_exe npt2 nproc_atm npc2 ncplproc_atm arg2 atm_exe nam2 atm_exe npt3 nproc_oce npc3 ncplproc_oce arg3 oce_ exe nam3 oce_ exe buffered MPI Send for coupling communication yes buffered send for MPI or MPI2 without mailbox it no simple send for MPI2 with big enough mail
61. ices After that the model will advance quickly Normally it takes slightly more than 1 hour to complete a 1 year run Useful tip e The runscript will re submit itself until the whole integration period of the experiment is completed unless you wish to stop it by doing something like mv run auscom VAYU run auscom VAYU stop To resume the run simply change the runscript name back 6 6 Verifying the Outputs A symobolic link to the phyical location of the model output storge has now been built under the AusCOM home Is AusCOM1 0 output ciaf2 xx When the first run is completed you can check and or process the outputs of the ice and ocean models cd AusCOM1 0 output ciaf2 xx history mom4 Is ocean layer nc 19481231 ocean param nc 19481231 ocean surf nc 19481231 ocean month nc 19481231 ocean snap nc 19481231 ocean trans nc 19481231 cd AusCOM1 0 output ciaf2 xx history cice Is iceh 1948 01 nc iceh 1948 04 nc iceh 1948 07 nc iceh 1948 10 nc iceh 1948 02 nc iceh 1948 05 nc iceh 1948 08 nc iceh 1948 11 nc iceh 1948 03 nc iceh 1948 06 nc iceh 1948 09 nc iceh 1948 12 nc 43 7 ACKNOWLEDGMENTS This work has been undertaken as part of the Australian Climate Change Science Program funded jointly by the Department of Climate Change and Energy Efficiency the Bureau of Meteorology and CSIRO We are grateful for super computing support from the National Computing Infrastructure NCI under projects p6
62. iments ready for the user to start with This AusCOMI 0 release package has been extensively tested on the National Computational Infrastructure NCI Sun Constellation cluster VAYU the super computing platform on which AusCOM has been developed Several test cases are included that utilise both climatological and interannually varying forcing datasets of Large and Yeager 2004 Large and Yeager 2008 A 500 year spin up run benchmarks AusCOM against the models used in the Coordinated Ocean ice Reference Experiment CORE of Griffies et al 2009 and is documented in our second technical report on the AusCOM development Bi et al 2010 Users who want to spend as little time as possible in learning to set up and run AusCOM1 0 at NCI can skip directly to Section 5 for the technical guide or Section 6 for the quick start shortcut In fact users who have access to the NCI VAYU disk short p66 only need type in one simple command wait for less than 25 minutes then can run this independent self contained AusCOM1 0 system on VAYU 4 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland 2 AUSCOM FRAMEWORK AND THE OCEAN SEA ICE HORIZONTAL GRID The AusCOM ocean sea ice coupled model is a four executable system that consists of three sub models MATM CICE and MOM4 and a numerical coupler OASIS3 25 Figure 1 shows the components of the ACCESS coupled model initial implementation and the coupling framework T
63. imulations and can achieve the computational performance of at least 20 model years per day on 128 processors i e 16 nodes This manual provides guidance for AusCOM users who have access to the XE and VAYU machines We assume users know about the machine configuration and disk system and also have access to data under project p66 For users who have no access to short p66 we may provide the required forcing data for them to store in their own short space For users who wish to set up AusCOMI 0 on another computing platform the model IO management compiling and run control etc introduced here may all need to be changed according to the specific infrastructure of their platforms 5 2 AusCOM1 0 System Infrastructure Ideally but not necessarily the AusCOM1 0 base directory should be set up under the user s home directory It contains the following sub directories AusCOM1 0 bin exp input submodels forcing output In practice the first four sub directories can be phyically located here whilst the other two may be just symbolic links to their physical locations on other disks that allow for large data storage such as the short system on XE and VAYU This arrangement is based on the fact that the HOME space quota is always very limited and should be used for files that need permanent storage 21 5 2 1 bin It stores model and coupler executables and some auxiliary tools used for various purposes For running an
64. ion strsu io strsu o 170 3600 1 i20 nc IGNORED cice cice LAG 0 SEQ 1 LOCTRANS INSTANT Coupling field No 11 is configured by only four lines of parameters 1 The first line has 7 entries e strsu 1o symbolic name of this field as defined in the source model CICE character 8 e strsu o symbolic name of this field as defined in the target model MOMA character 8 e 170 index in cf name table txt used by OASIS3 to obtain description for this field e 3600 interval in seconds of ice ocean coupling 31 e l only 1 time transformation to be performed by coupler e i20nc coupling restart filename character 8 for all ice to ocean fields e IGNORED field status indicating that this field is to be exchanged directly from CICE to MOM4 without being transformed by OASIS3 main process This is the advantage of CICE and MOMA sharing the same tripolar grid 2 The second line has 4 parameters e cice source CICE grid e cice target MOMA grid e LAG 0 no lag for ice ocean coupling e SEQ 1 sequence number 3 Line 3 and Line 4 configure the only time transformation the same as that for field 1 The other 12 i20 fields are configured the same way The ocean to ice fields 021 are formulated as follows Field 24 sea surface temperature sst_oi sst_i 1 3600 1 o2i nc IGNORED cice cice LAG 0 SEQ 1 LOCTRANS INSTANT The 7 021 fields are all configured the way shown above It s basically the same as that
65. limate Ocean Model AusCOM with a coordinated ocean ice reference experiment CAWCR CSIRO Marine and Atmospheric Research CAWCR Technical Report No xx in press Griffies S M 2004 Fundamentals of ocean climate models Princeton University Press Princeton USA 496pp Griffies S M Biastoch A Boening C W Bryan F Chassignet E England M Gerdes R Haak H Hallberg R W Hazeleger W Jungclaus J Large W G Madec G Samuels B L Scheinert M Gupta A S Severijns C A Simmons H L Treguier A M Winton M Yeager S and Yin J 2009 Coordinated ocean ice reference experiments COREs Ocean Modelling 26 1 46 doi 10 1016 j ocemod 2008 08 007 Griffies S M Harrison M J Pacanowski R C and Rosati A 2004 A Technical guide to MOM4 NOAA Geophysical Fluid Dynamics Laboratory GFDL Ocean Group Technical Report No 5 337pp Heil P Phipps S J and Roberts J L 2005 User guide for the TPAC coupled ocean sea ice model TPAC Technical Report 45pp Hunke E C 2001 Viscous plastic sea ice dynamics with the EVP model Linearization issues J Comput Phys 170 19 38 Hunke E C and Dukowicz J K 1997 An elastic viscous plastic model for sea ice dynamics Journal of Physical Oceanography 27 1849 67 Hunke E C and Lipscomb W H 2008 CICE the Los Alamos sea ice model documentation and software user s manual version 4 0 LA CC 06 012 72pp Jones P W 1997 A User s Guide for SC
66. ls all major coupling functions are coded in a coupling module e g cpl interface For each of the sub models that link to OASIS the coupling module code can be found in the associated driver directory e CICE AusCOM1 0 submodels cice4 0 drivers auscom MOM4 AusCOM1 0 submodels mom4p1 src auscom_cpl e MATM AusCOM1 0 submodels matm source Each module consists of a group of interfacing subroutines which are called by the model at different stages to accomplish various coupling tasks Taking the coupling media CICE as the example the subroutine calls are as follows 12 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland Table 1 The 31 coupling fields of the AusCOM system occur in four classes there are 10 fields passed from the atmosphere model to the sea ice model a2i seven fields passed from ocean to sea ice 02i one field passed from sea ice to atmosphere i2a and 13 from sea ice to ocean i2o Where appropriate zonal and meridional denote local direction on the tripolar orthogonal curvilinear grid The source and target entries refer to actual variable names in the broadcasting and receiving models respectively No Coupling field units source target a2i Atmosphere to sea ice coupling fields 1 2M AIR TEMP UE TAIR TAIRI 2 10M EASTWAR
67. n space and labelling them with proper model date information end date of this run Note a PBS copyq queue job do mppncombine ksh is submitted here requesting for 1 processor to combine the MOMA history NetCDF files which are processor dependent when written out by MOMA Doing so can obviously speed up the post processing operation avoiding possible long time holding of all the processors e g 128 processors allocated for the current job while only one of them is actually working This is especially true for the XE machine on which the MOM4 post processing mppnccombine operation takes minutes R 6 Submission of the next job This section updates the exp date file and submits the next job if the simulation has not reached the final date as defined in Section R 1 That s all for the runscript Users can try AusCOM out by submitting one of the sample jobs say ciaf2 xx qsub run ausCOM VAYU One last useful tip e When this runscript is adapted for a new experiment which usually involves use of different input files please run it first in interactive mode to check if all input files are in place namely run auscom VAYU e fitreaches the model launching line mpirun then go ahead and qsub it Otherwise check what s missing and do it again and again until everything is in place The beneifit of doing so is obvious it avoids wasting time waiting for the job to get on the PBS queue only to find frustratingly a
68. nc do mv f tmpfile restdir oasis3 tmpfile enddate done fi DHA EE EU HELE RUPEE EL EEUU AAAHHH H 6 Submission of the next job SUBBBHIEIIBHEBHERBHIIUHUIIBESRBHEBEHIIUUIBESRBERIUUNIBBESHERIGUGUNIBBESRHERHAHRR cd jobdir H Number of the next job H nextjob jobnum 1 update date and log file if f expid date then mv expid date expid date_ jobnum fi truntime0 expr truntime0 runtime echo nextyear nextmonth nextday nextjob truntime0 gt expid date echo date jobnum enddate done post processing gt gt expid log Check whether final date is reached If not keep going if nextdate gt finaldate then echo Experiment over echo date Experiment over gt gt expid log else next jobid qsub W depend after archjob p 10 run auscom VAYU echo date New run is submitted fi GRHHIBBHEHHBHHEURHEEHERHHIIU AAAHHH H 7 Epilogue E AAAHHH date exit 70 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland 71 72 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland The Centre for Australian Weather and Climate Research is a partnership between CSIRO and the Bureau of Meteorology
69. ness false chk frzmlt sst false chk_i20_fields false chk_02i_fields false chk_i2a_fields false chk_a2i_fields false amp end eof 3 2 4 namelists for mom4pl a standalone mode input namelist file cp f inputdir mom4 mom4_in nml mom4 in nml alap 1 0e5 truncate_velocity true truncate_verbose true if year gt Siniyear then truncate_velocity false truncate_verbose false fi temp_restore_tscale 30 0 sst restoring time scale of 30 days temp restore tscale 1 0 NO SST restoration salt_restore_tscale 60 0 sss restoring time scale of 60 days salt_restore_tscale 15 0 strong SSS relaxation as recommended salt_restore_tscale 1 NO SSS restoration use_waterflux true layout oce nx oce ny mpi partitioning pattern Simple frazil false simple temp frazil if f use complicated scheme and allow multi layer frazil Accurate_frazil true accurate temp frazil must be t if Simple_frazil f vice versa 66 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland TL frazil false top layer frazil if f multi layer frazil diff cbt_iw 0 1e 6 f background diffusion when BL profile is NOT used 1 e 4 m2 s is the default value visc_cbu_iw 1 0e 4 BG vertical vicocity convection true aredi 600 agm 100 ricr 0 3 ed mom4 in nml lt lt eof g HNMONTH s HNMONTH nmonth
70. normalised IOAICE IOAICE 13 4 2 1 prism init This subroutine is called by CICE on start up to initialise PSMILe and MPI This call opens a channel for the model to communicate with the coupler Major MPI and PSMILe library calls include MPI init initialise MPI if required prism init comp proto called by all processors to initialise the coupling MPI Buffer Attach determine the model MPI buffer size prism get localcomm proto called by all processors to obtain the value of the local communicator to be used by the model for its internal parallelisation In a mono coupling case this local communicator is also in charge of data exchange between model and coupler 4 2 2 cpl init This subroutine is also called at the model initilisation stage to determine coupling strategy mono or multi processor coupling and define coupling field names etc in the context of the OASIS coupling control namelist in the file namcouple see Section 5 for details Major PSMILe library calls include prism def partition proto called by all coupling processors to obtain the model MPI partition information OASIS3 supports 4 types of partition Serial no partition which is the current choice for AusCOM coupling Apple Box and Orange prism def var proto called by all coupling processors to declare each coupling field to be sent or received in the course of a simulation It passes to the coupler all the required information about
71. ond run is finished the previously used exp date is renamed to exp date 2 and a new exp date is created for the third run and so on Therefore as an experiment progresses more and more exp date files will be created labelled with job numbers and saved for possible future use A useful Tip 36 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland e fusers wants to restart a run from a certain time point in the run history for whatever reason they can simply rename the associated exp date file to exp date and submit the job under one condition though the associated restart files for all sub models and the coupler have to be there R 3 Getting All the Files into the Run Directory This section puts in place all preprocessed auxiliary files initial condition data files and sub model and OASIS executables Some configuration files that control run time execution will be edited to reflect particular features of the experiment and control over the to be performed run segment Run time configuration of namelist files are also created for each of the sub models R 3 1 Grids Initial Condition Executables and some Preprocessed Auxiliary Files boundary layer gfdl Z gfdl or non gfdl runoff data core core or non core cold_start 1 1 0 cold warm start In this section probably only these top lines need close attention They decide some features of an experiment e using the
72. oral resolutions and often use different variable names for the same fields In conjunction with an external data table and a datafile selection script both are preprocessed for a specific dataset MATM reads in all the required atmospheric forcing fields with proper scaling and offsetting treatment whenever applicable and passes them into CICE Currently the MATM model resolution is dataset dependent and must be determined at compile time i e static array definition Therefore datasets of different horizontal resolution have to be handled with different MATM executables In the next release MATM will be upgraded to support dynamic array definition and therefore all datasets will be handled using a single MATM executable Another feature of MATM worthy of mention is that for providing the ocean and sea ice system with better timing diurnal foring MATM is required to send forcing fields of one coupling interval e g 6 hours ahead of the coupling time For example at a coupling time point say 00 00AM MATM sends fields of 06 00AM instead of 00 00AM into CICE via OASIS CICE already has the 00 00AM data by reading in preprocessed data at the very beginning of an experiment or passed from MATM at the last coupling time point in the middle of a run CICE interpolates these two sets of forcing data to obtain the right time forcing for every time step of the sea ice model within this coupling interval 00 00 to 06 00AM 3 3 Se
73. oved to AusCOM1 0 bin e PSMILe libraries and OASIS3 executable cd AusCOM1 0 submodels oasis3_ prism 2 S prism compile comp oasis325 VAYU This script compiles all the PSMILe libaries first and then the OASIS3 executable Also generated is the calendar tool executable i e calendar VAYU e MATM executable cd AusCOM1 0 submodels matm compile comp auscom matm VAYU e CICE executable cd AusCOM1 0 submodels cice4 0 compile comp auscom cice VAYU np 6 Note this builds the CICE executable running on 6 processors e MOMA executable cd AusCOM1 0 submodels mom4p1 compile comp auscom mom4 VAYU 41 It also compiles the MOM4 history file combining tool mppnccombine exe Congratulations You have just successfully set up the standard AusCOM1 0 system Now you can test the sample runs and hopefully soon design your own experiments for various applications 6 4 Running A Sample Experiment cd AusCOM1 0 exp Is ciaf2 xx ciaf2 xy ciaf2 xx Three sample experiments are available here for you to test e ciaf2 xx CORE2 interannual forcing experiment with cold start i e ocean and ice are initialised with the observed temperature and salinity only It is set to start from date 19480101 and end on 19501231 with 1 year duration for each run segment e ciaf2 xy CORE2 interannual forcing experiment with warm start i e ocean and ice are initialised with an existing ocean ice spinup Same run time a
74. ow so the fld number 367 for swfld etc may not point to the realistic longname of the variable may modify this file to add more fields in the future TPHERHHEHHHHHHHHHBHHBHHBHHHHHHBHHBHHBHHHHHHBHHBHHBHHHHHHHHHBHHBHHBHHHRHHE 47 it ATMOSPHERE gt gt gt ICE He ones ET Te Ee TT Field 01 swflx down HHHHHHHHHH swfld ai swfld_i 367 21600 4 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 POPO LOCTRANS CHECKIN SCRIPR CHECKOUT INSTANT INT 0 CONSERV LR SCALAR LATLON 10 FRACNNEI FIRST INT 0 THETA Field 02 lwflx down THERE TE Iwfld ailwfld 1366 21600 4 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 POPO LOCTRANS CHECKIN SCRIPR CHECKOUT INSTANT INT 0 CONSERV LR SCALAR LATLON 10 FRACNNEI FIRST INT 0 TERT Field 03 rainfall THRE rain_airain_i 26 21600 4 a2inc EXPORTED nt62 cice LAG 3600 SEQ 1 POPO H LOCTRANS CHECKIN SCRIPR CHECKOUT INSTANT INT 0 CONSERV LR SCALAR LATLON 10 FRACNNEI FIRST INT 0 THT HE Field 04 snowfall HHH snow ai snow 126 21600 4 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 POPO it LOCTRANS CHECKIN SCRIPR CHECKOUT INSTANT INT 0 CONSERV LR SCALAR LATLON 10 FRACNNEI FIRST INT 0 TERETE Field 05 surface pressure THERE press_ai press 133 21600 4 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 POPO 48 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland LOCTRANS CHECKIN SCRIPR CHECKOUT I
75. perations that send forcing fields into the ocean The involved PSMILe routine is prism put proto called in an ice to ocean loop to send each forcing field required by the ocean model which is declared in cpl init and listed in namcouple The above three data transfering routines are the core of the coupling interface connecting CICE to OASIS3 in AusCOM 4 2 6 into atm This routine contains only one call to the PSMILe routine prism put proto sending one ice ocean variable into the MATM data model although not needed there It is included as mentioned above to mimic a complete cycle of data exchange required in a fully coupled model such as ACCESS In AusCOM and ACCESS ice ocean coupling is allowed to occur more frequently than ice atmosphere coupling depending on experimental design and therefore the subroutines from ocn and into ocn can be called more frequently than from atm and into atm in the course of a simulation In fact it is highly recommended that for AusCOM runs ice ocean coupling occur once per time step common to both MOMA and CICE and that ice receive data from atmosphere once per 6 hours being the highest temporal resolution of the standard forcing of LargeYeager 2009 4 2 7 coupler termination This subroutine calls the PSMILe routine prism def partition proto to terminate connection between model and coupler It also deallocates all coupling fields For the other two models MATM and MOM4 coupling
76. pl There are additional places in the modelling code where various modifications are made for coupling purposes These are indicated by ifdef AusCOM precompiler statements Also for each component we set a new subdirectory compile as shown above in which scripts are provided for compiling each sub model and also the coupler 5 2 5 forcing Under this directory symoblic links should be set to the physical locations where the available atmospheric forcing datasets are stored For example we need Is l forcing n CIAF gt short p66 sjm599 COREv2 26JAN2010 CIAF seid CNYF v2 gt short p66 sjm599 COREv2 15JUNE2009 CNYF v2 The scripts input matm get core2 IA ksh etc mentioned above will pick up atmospheric forcing data from these directories Users should note that the CORE forcing is updated from time to time As such the get core2 ksh scripts may also be updated 24 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland 5 2 6 output Under this directory we can find symbolic links to outputs of experiments For example Is l output ciaf2 xx gt short p userid OUTPUTS AusCOM1 0 ciaf2 xx Is output ciaf2 xx ciaf2 01 history cice mom4 ciaf2 01 restart cice mom4 oasis3 These links are established by the runscript when the initial run of an experiment is conducted 5 3 AusCOM Compilation To build the AusCOM system under the OASIS3 framework all sub models n
77. processor usages run time and calendar option full list of coupling fields and associated prerequisites including the souce and target grids coupling algorithm coupling frequency remapping interpolation options etc The namcouple file released with AusCOM1 0 is designed for the sample experiments based on the example namcouple used for the OASIS3 ToyClim model It was prepared with extreme care after the coupling approach was determined and the coupling interface in each sub model 27 was set up All items listed in this file some of which are to be discussed below must be consistent with their coresponding part defined in each of the sub models or in other preprocessed auxiliary data files such as the OASIS3 grid information files As required by OASIS3 namcouple predefines 11 groups of parameters for coupling control Each group contains one or more parameters starts with a SKEYWORD line e g SRUNTIME and ends with a SEND line As shown in Appendix A which is the complete namcouple file used for the example run ciaf2 xx the first 10 groups are general parameters and are all well described inside the file Therefore the following introduction focuses only on the last large group of parameters which configures all the coupling fields one by one 5 4 2 Configuring Coupling Fields This part is found in namcouple between lines STRING and SEND It contains multi line configuring information for each of the coupling fields with a
78. r chk_a2i_fields false chk_i2a_fields false amp end eof get and adapt the forcing pointer file cp f inputdir matm atmdata fields datatype table data 4 matm table ed data 4 matm table lt lt eof g Y EAR s Z Y EAR Sendyear g tFORCING s ZFORCING MATM input w q eof 3 2 3 namelists for cice a standalone mode input it npt_cice expr runtime dt_ice if nmonth 0 then ie nmonth 1 a whole month run om histfreq m hist avg true dumpfreq m dumpfreq n nmonth else ie nmonth 0 an nday run histfreq d hist avg true dumpfreq d dumpfreq n nday fi hist_avg false would output snapshot hist mixedocean false use or not use the mixed ocean layer cp f inputdir cice cice4 0_in nml cice in nml ed cice_in nml lt lt eof g DAYS per _year s HDAYS per year days this year g HYEAR_init s YEAR_init iniyear g DT_CICE s HDT_CICE dt_ice g NPT s NPT npt_cice g HRUNTYPE s HRUNTYPE runtype 64 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland g ZHISTFREQ s ZHISTFREQ histfreq g HIST_ AVG s HIST AVG S hist avg g ZDUMPFREQ s 7ZDUMPFREQ dumpfreq g DUMPFR_N s HDUMPFR_N dumpfreq_n g ZRESTART s ZRESTART S Lrestart g ICE_IC s ICE_IC ice_ic g HFYEAR_init s FYEAR_init iniyear g MIXEDOCN s HMIXEDOCN mixedocean g NPROCS s HNPROCS nproc_ice wW q
79. rology CSIRO and the Bureau of Meteorology advise that the information contained in this publication comprises general statements based on scientific research The reader is advised and needs to be aware that such information may be incomplete or unable to be used in any specific situation No reliance or actions must therefore be made on that information without seeking prior expert professional scientific and technical advice To the extent permitted by law CSIRO and the Bureau of Meteorology including each of its employees and consultants excludes all liability to any person for any consequences including but not limited to all losses damages costs expenses and any other compensation arising directly or indirectly from using this publication in part or in whole and any information or material contained in it CONTENTS iSt Of FAAO es Fae cashed adnate mee avira ea Ee araara aae a aAa ROO iii List of Tables e iii 1 Introduction 3 2 AusCOM Framework and the Ocean Sea Ice Horizontal Grid 5 3 Components of the AusCOM System eese 9 3 1 Numerical Coupler OASIS3 sssssssssssssessseseeeeneere nennen ennt 9 3 2 Data Atmospheric Model MATM ssssssssssseseeeneeee rennen enne nnne 9 3 9 Sea lce Model CICE ireen one tec eel eoo bete eb i as 10 3 4 Ocean Model MOMA sssssssssssssssese eene enne n nens n nennen 11
80. rrangement as in ciaf2 Xx e cnyf2 xx CORE2 normal year forcing experiment cold start Please compare the runscripts run auscom VAYU under the job directories to see the setup differences Make sure to replace project p66 in the script with your own project name If you are on p66 this script should work immediately for you You can choose any one of them to start your very first AusCOM test run For example First test run the job interactively to test all necessary files are available for reasons described in Section R 6 above cd AusCOMI 0 exp ciaf2 xx Is run auscom VAYU Kill the interactive run when it reaches the model launching line mpirun Now submit the job to the PBS queue qsub run auscom VAYU 6 5 Monitoring the Run nqstat grep user 12345 R userid p66 ciaf2 xx 42 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland When you see R in the above job status line a symbolic link to the working directory rundir has already been established here called Running dir Also created is a log file ciaf2 xx log which will record the run history of this experiment Is Running dir ciaf2 xx log run auscom VAYU You can monitor the model progress less Running dir cplout Or tail f Running dir cplout Note the first couple of minutes see little progress because OASIS3 is calculating the huge interpolation coefficient matr
81. rrently OASIS3 has been adopted by around 20 climate modelling groups in the world as a coupling framework for their coupled models and run on various computing platforms Earlier versions of OASIS have been used in Australia for some years When the Australian climate sciences community started building its own new generation coupled climate model at first the coupled ocean sea ice AusCOM then the fully coupled model ACCESS which combines component models from different providers around the world OASIS3 was chosen to be the numerical coupler because of the following advantages Modular architecture it is relatively easy for alternative sub model replacement ndividual executables for sub models easy code management and compilation Grid independence conservative remapping between different grids at the 2D interface allows each sub model to use its own preferred grid In practice we make a choice to simplify the coupling computation as discussed in sections 3 3 and 3 4 System expandability additional sub models can be relatively easily added into the system 3 2 Data Atmospheric Model MATM The AusCOM atmospheric component MATM is a mono processor data model designed for the purpose of handling various atmospheric forcing datasets It 1s equiped with a set of data reading subroutines and modules used for different datasets such as NCEP2 ERA40 CORE and UM AMIP run outputs etc which may be of different spatial and temp
82. rther discussed below for users to better understand the AusCOM coupling stragegy MATM CICE MOM4 Fig 4 AusCOM ACCESS coupling approach CICE acts as the coupling media Firstly AusCOM allows for different frequencies of coupling atmosphere to sea ice and sea ice to ocean MATM CICE coupling occurs once per 6 hours i e 4 times daily which is the most frequent temporal resolution of atmospheric fields in the available datasets such as NCEP ERA40 and CORE CICE MOM4 coupling interval can be flexible depending on model time steps and experiment design usually 1 2 3 or 6 hours In practice we choose 1 hour as the ice ocean coupling interval which is usually the ice and ocean model time step for the purpose of minimising coupling lag Secondly CICE functions as a coupling buffer or media where all coupling fields are gathered processed if required and then delivered to their receivers Atthe beginning of a certain ice atmosphere coupling period IACP CICE receives from MATM the raw atmospheric fields then starts the ice ocean coupling cycles IOCP within this IACP At the beginning of an IOCP CICE first sends the prepared i20 fields to MOM4 then starts its own time stepping loop 17 At every time step in an ice internal time loop CICE updates the atmospheric fields by time interpolation using them and some oceanic fields most recently received from MOMA to calculate surface flu
83. s including itself 18 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland Model A OASIS Model B Tw Hu T n Fig 5 Schematic diagram of the mono processor coupling algorithm in AusCOM ACCESS In this coupling process MPI global gathering and scattering operations are needed in the source and target models respectively to some extent slowing down the sub models execution This drawback may become significant if large number of processors are used for the sub models Figure 6 demonstrates how the multi processor coupling works for the coupling field F At coupling time each of the coupling processors in model A usually all the processors allocated for this model sends their own partitioned information of F to the coupler The coupler receives this partitioned field and gathers it automatically onto the global domain of the source grid following the partitioning pattern defined in model A s coupling interface Then the coupler re maps this global field F onto the global domain of the target grid by interpolation divides it into partitions following the partitioning pattern defined in model B s coupling interface and sends each partition of field F directly to model B s corresponding coupling processors Model A OASIS Model B Fig 6 Schematic diagram of the multi processor coupling algorithm in AusCOM ACCESS 19 Obviously this parallel coupling does not involve any MPI glo
84. s not support dynamic array allocation Grid resolution must be specified in the compiling script by setting variable GRID and RES as following Grid resolution setenv GRID tp1 setenv RES 360x300 and because of its dependence on the MOM4 grid whenever changes are made to the MOM4 grid horizontal resolution the above two lines must be changed accordingly and CICE be re compiled 5 3 4 MOM4 On VAYU AusCOM MOM4 can be compiled straight away cd AusCOM1 0 submodels mom4p1 compile comp_auscom_mom4p1 VAYU 26 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland On XE however MOM4 compilation is somewhat more complicated cd AusCOM1 0 submodels mom4p1 compile qsub q express lwalltime 01 00 00 vmem 2Gb jobfs 4Gb ncpus 1 wd I Isoftware intel fc intel cc comp auscom mom4pl XE The complication on the XE comes about because we need to request more memory usage for the mono processor compiling operation which would otherwise fail in generating the ocean boundary condition module ocean obc mod that requires large memory It is worth mentioning that XE and VAYU are compatible in their operating systems Executables compiled on one platform can be readily executed on the other platform Therefore users who have accounts on both XE and VAYU may compile the AusCOM system on only one of these two machines and set up runs on both of them The only inconvenience is that XE and VAYU do not
85. section handles all configuration files They are either edited from their own template file by replacing some adjustable i e experiment run dependent parameters or created at run time These files are e namcouple OASIS3 coupling configuration namelist file edited e input atm nml MATM run and coupling control file created e data 4 matm table definition of names of forcing variables and files created e cice in nml CICE configuration namelist file edited e input ice gfdl nml ocean rough nml and input 1ce monin nml namelists for the GFDL module used by CICE to calculate surface fluxes optionally created e input ice nml CICE run and coupling control file created e mom4 in nml MOM4 configuration namelist file edited e MOMA input diag table MOMA namelist specifying the diagnostic ocean outputs edited e input oce nml MOMA run and coupling control file created Useful tips e The sub model configuration template files especially mom4 in nml provided in this AusCOM1 0 release contain a number of parameters editable at run time by the runscript namely those strings prefixed with ff need be replaced with sensible values when job is submitted This flexible modelling approach encourages the tuning of physical parameterisations in particular that required by MOMA Users who have similar intentions can create their own template to allow for more or less run time tuning freedom e In t
86. sh them from the section sequence of this document 5 5 2 Steps for Setting up an AusCOM Run R 0 Prologue 33 This part requests computing resouces through the PBS system and sets up some environment variables associated with MPI and Fortran IO control Frequently changed are the following key PBS lines PBS P p66 PBS q normal PBS l ncpus 128 PBS 1 vmem 128GB PBS 1 walltime 1 30 00 PBS N ciaf2 xx They define project p66 user dependent queue type normal number of processors to be used 128 memory request 128GB wall time needed 1 5 hours and the job name ciaf2 XX R 1 Primary Setups R 1 1 Experiment ID and Forcing project p66 This defines the project that the user belongs to on the NCI system so that the user can use the short project disk to run the model and store model outputs It s not necessarily the same as that in the PBS P p line if the user belongs to more than one project expid ciaf2 xx atmdata core2 datatype IA year data end 2007 These lines define experiment ID run name same as that defined by the PBS N ciaf2 xx line the atmospheric forcing dataset forcing type IA for interannual NY for normal year and up to which year the data is available for a NY forcing run can be set up to 9999 Although MATM also supports other forcing such as NCEP2 ERA40 and UM96 UM AMIP run outputs the CORE2 dataset is the recommended standard atmospheric forcing for
87. sing time interpolation calculate forcing fields required for ocean and ice ice time stepping time sec time sec dt ice enddo call from ocn time sec End Do call into atm time sec dt offset lag update i2a fields for the beginning of next a2i coupling END DO save the last calculated i20 fields which have not been sent into ocean save the last received a2i fields for use at the beginning of next run save other fields required for next run finalisation The CICE integration is completed by three time loops which advance the ice time stepping the ice ocean coupling and the ice atmosphere coupling respectively The number of ice steps within an Ice Ocean Coupling Period IOCP number of IOCP within an Ice Atmosphere Coupling Period IACP and number of IACP for the job are determined by the ice time step the ice ocean coupling frequency the ice atmosphere coupling frequency and the integration length of the job These four time intervals are defined in the AusCOMI 0 runscript see sub section 5 5 16 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland Under the concepts of OASIS3 data exchange nature of air ice ocean coupling and the atmospheric forcing temporal resolution AusCOM is configured to perform coupling between sub models and progress simulations in the way shown by the above CICE code execution flow which although quite self explaining is illustrated by Fig 4 and fu
88. te zonal and meridional denote local direction on the tripolar orthogonal curvilinear grid The source and target entries refer to actual variable names in the broadcasting and receiving models respectively iti tert eiit HE re ien tres iv Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland ABSTRACT A user guide for the Australian Climate Ocean Model AusCOM is presented AusCOM is the ocean and sea ice component of the Australian Community Climate and Earth System Simulator ACCESS model AusCOM is an IPCC class coupled ocean and sea ice climate model with its current configuration developed primarily within the Ocean and Coupled Modelling Team at the Centre for Australian Weather and Climate Research CAWCR It comprises the NOAA Geophysical Fluid Dynamics Laboratory Modular Ocean Model version 4 1 GFDL MOMAp1 the Los Alamos National Laboratory Sea Ice Model version 4 0 LANL CICE4 0 and a data atmospheric model MATM Numerical coupling 1 e data exchange between the sub models utilises the Message Passing Interface within the PRISM 2 5 OASIS3 coupler A standard version is released as AusCOM1 0 an independent self contained and easily setup package 2 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland 1 INTRODUCTION The Australian Climate Ocean Model AusCOM is an IPCC class coupled ocean and sea ice model aimed to serve the Australian climate s
89. us tests mainly because it results in a reasonable flux value although local flux conservation is not guaranteed e FIRST the conservation order 8 The last line configures the postprocessing transformation CHECKOUT e INT 0 Specifies to not calculate or print the target field integral 30 Australian Climate Ocean Model AusCOM Users Guide Daohua Bi and Simon Marsland The other 9 i20 fields are configured the same way This means that even the two wind components are treated as scalars By doing so we in effect reduce the workload of the coupler which would otherwise be required to perform vector rotation transformations As already expressed we let the sub models take care of the vector rotation before or after coupling whenever required For users who do want the vector rotation to be done in OASIS3 the wind components fields No 9 and 10 may be configured as below Field 09 10m wind u uwnd ai uwnd i 56 21600 3 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 POPO CHECKIN SCRIPR CHECKOUT INT 1 DISTWGT LR VECTOR I LATLON 10 4 vwnd ai INT 1 Field 10 10m wind v vwnd ai vwnd i 56 21600 3 a2i nc EXPORTED nt62 cice LAG 3600 SEQ 1 POPO CHECKIN SCRIPR CHECKOUT INT 1 DISTWGT LR VECTOR J LATLON 10 4 uwnd ai INT 1 Our tests show that this approach and the recommended give very close model results The ice to ocean 120 fields are formulated as follows Field 11 ice ocean interface stress x direct
90. usCOM and ACCESS use a mono prorcessor coupling algorithm for all sub models although multi processor parallel coupling is seemingly more efficient Our arguement is that under the mono processor OASIS3 coupling framework efficiency of data exchange between sub models via the coupler largely depends on workload of the coupler At a coupling time letting the coupler communicate with all processors of each sub model 1 e do all the global gathering and scattering work for all coupling fields may take more time than the time saved by each sub model s avoiding global gathering and scattering operations Figure 5 illustrates the fate of a certain coupling field F when it 1s passed from the source model A to a target model B via the OASIS3 coupler using a mono processor coupling algorithm Let s assume F is partitioned into 2 x 3 and 2 x 2 sub domains in A and B respectively At coupling time model A s coupling processor local communicator gathers from all slave processors the partitioned information of F places F onto the global domain of the source grid and then sends this global field to the coupler If required which is the case for most coupling operations but not for the AusCOM ice ocean coupling the coupler re maps this global field onto the global domain of the target model grid by suitable interpolation Then the coupler sends F into model B where the local communicator receives the global field F and scatters it onto all slave processor
91. xes required by sea ice and ocean This calculation is performed by either a built in boundary layer module or an external module adopted from the GFDL FSM system based on standard NCAR bulk formula of Large and Yeager 2004 Those i20 fields are time averaged and weighted by ice coverage if applicable for the next IOCP e After finishing the ice time loop CICE receives 021 fields from MOMA and then starts the next IOCP and so on until reaching the end of the curent IACP Then CICE transfers the i2a field s to MATM and starts the next ice atmosphere coupling cycle Note the ice to atmosphere sending operation is not really necessary for AusCOM In practice we let CICE send one field ocean ice surface temperature into MATM only for mimicking a complete cycle of data exchange in a fully coupled model such as ACCESS The above is basically the same coupling approach used for the ACCESS fully coupled model except ACCESS requires a lot more ice atmosphere data exchange and obviously does not allow for atmospheric data update within an 1ce atmosphere coupling interval For MATM and MOMA the code execution occurs in a similar manner but with only 2 level time loops and the data exchange operations occuring there must be in accordance with their CICE counterparts in every aspect i e coupling fields and coupling time otherwise the system would enter a deadlock 4 3 2 Coupling Algorithm Mono processor vs Parallel Coupling A
92. y layer gfdl then cp f cice ic u star nc rest date oasis CICE input u star nc fi fi get input files for mom4 cd MOMA input cp f Sinputdir mom4 field table field table cp f Sinputdir mom4 data table data table cp f inputdir mom4 grid spec auscom nc grid spec nc cp f inputdir mom4 salt_sfc_restore nc salt sfc restore nc cp f inputdir mom4 temp sfc restore nc temp sfc restore nc if cold start 1 then get ocean initial condition only T S cp f Sinputdir mom4 ocean temp salt nc ocean temp salt res nc else for restfile in Is mom4_ic ocean_ rest_date_mom4 do newfile restfile cp restfile newfile done ystart ic yearpl if Systart It 10 then typeset Z1 ystart elif ystart It 100 then typeset Z2 ystart elif ystart It 1000 then typeset Z3 ystart 61 elif ystart It 10000 then typeset Z4 ystart fi ed ocean_solo res lt lt eof g Systart s ystart iniyear w q eof fi cold_start else for continue runs cd rundir rm f out weights prt clean up prepare restart files for oasis3 for resfile in ls restdir oasis3 2 nc prevdate do sresfile resfile take away the front path name cp resfile sresfile take away the appendix YY Y YMMDD done for cice runtype continue Lrestart true ice 1c default cp Srestdir cice A21 timel nc prevdate CICE input A2I time0 nc cp restdir cice

Australian Climate Ocean Model (AusCOM) Users Guide

Contents

Download Pdf Manuals

Related Search

Related Contents