Training Course - ComIntro
Contents
1. Browse queues local Member State Browse basket account s You need Submit new job Hostname host meteo ms tomanta an association with an Directory Ers my id from ecmwi MS user Job submissions Comment File transfers Browse history Default destination genericFtp ECtrans uses the specified information ECtrans setup Get certificate Complementary information to perform unattended file Please use the data content editor transfers to from your local Member Log off usl ectrans sizeTimeOut ftp commTimeOut 60000 ftp dataTimeOut 60000 ftp lowPort no ftp mkdirs yes ftp passive yes ftp portTimeout 60000 This is optional ftp prefix information which ftp suffik tmp can be used by an ftp usetmp no ECtrans module for Daea nana example to perform some extra security check e g Login information certificate Login my local 10 Basewonli Proce With the data content editor you F can either enter your Create this MS user So State account COM INTRO 2014 ECaccess ECtrans Association Web Maintenance Ay ECMWF gateway service gt ECtrans gt Setup Microsoft Internet Explorer O x File Edit View Favorites Tools Help Bak gt O A A Qseach Favorites Zmeda lt 4 4x SAN Links kd EChome files ECscratch files ECfs files Browse queues Browse basket Submit new job Job submissions File transfers Brows2. mshost echost ecaccess ectrans restart 389 mshost echost ecaccess ectrans restart overwrite 389 mshost echost ecaccess ectrans restart target NEW NAME 389 mshost echost ecaccess ectrans delete 389 SS ECMWF ECaccess Pointers e http www ecmwf int services ecaccess User s manual Administrator s manual Registration centre for local ECaccess gateways SS ECMWF
3. mshost ecaccess certificate create Please enter your user id tr Your passcode mshost ecaccess certificate list submitJob 168h Mar 11 11 25 job submission getJobList 168h Mar 11 11 25 job list deleteJob 168h Mar 11 11 25 delete a job e Certificate valid for 7 days but this can be extended for special requirements e Certificate only needed for file job and transfer management NOT required for unattended transfers see later ectrans e Certificate not needed for ECMWE local use of ECtools lt SS ECMWF ECtools File Management ecaccess file chmod ecaccess file copy ecaccess file delete ecaccess file dir ecaccess file get ecaccess file mget ecaccess file mkdir ecaccess file modtime ecaccess file move ecaccess file mput ecaccess file put ecaccess file rmdir ecaccess file size Change ECaccess File Mode Bits Copy an ECaccess File Remove an ECaccess File List ECaccess Directory Contents Download an ECaccess File Download Multiple ECaccess Files at once Make a Directory on the ECaccess File System Show the Last Modification Time of an ECaccess File Move or Rename ECaccess Files Upload Multiple Local Files at once Upload a File on the ECaccess File System Remove a Directory on the ECaccess File System Show the Size of an ECaccess File SS ECMWF ECtools Job Management e With ECtools you can submit shell scripts and LoadLeveler command files to the LoadLeveler queues of ecgate and the su
4. Ecaccess Gateway Tools and ectrans Dominique Lucas User Support advisory ecmwf int TL ECMWF Content e ECaccess concepts e Ecaccess tools e Ectrans e Tutorial SS ECMWF ECaccess Concepts e ECaccess provides a portal to access ECMWF archiving and computing facilities e Strict authentication via Actividentity tokens and X509 certificates e Data integrity confidentiality guaranteed by SSL SS ECMWF ECaccess Concepts e ECaccess provides File and job management in batch or interactive mode through an extended FTP SFTP server File and job management through a Web browser A secure ssh access to ECMWF A secure X11 access to ECMWF Various services via the web e All this via Internet or RMDCN SS ECMWF ECaccess Architecture eMuitiple tiers gt ECaccess Client ECaccess Gateway ECaccess Server ECaccess ecgate MS Gateway COM INTRO 2014 ECaccess 5 lt S AECMWEF ECaccess Architecture e ECaccess client Standard FTP SFTP SSH HTTP S or NX clients Standard X Server not needed for NX and FTP Server ECaccess tools new 4 0 0 and old versions 3 3 0 e ECaccess gateway Entry point for all ECaccess users Verifies ECaccess users authentication certificates or passcodes Implements FTP SFTP and HTTP S protocols Secure tunnels through firewalls to ECaccess Server Either local ECaccess gateway installed at your site OR Gateways locat
5. bmit new job Job submissions File transfers Browse history ECtrans setup Get certificate Log off usl r Association association can be User Full name used by this user in order to perform List of ECtrans associations unattended transfers Dominique Lucas x2386 on your behalf r from_ecmwf Carsten Maass x2389 room224 Paul Dando Umberto Modigliani x2382 Pam Prior x2384 John Greenaway x2385 ECMWF user name Grant association s e MS associations are specific to each ECaccess gateway ECMWF Disclaimer zi OB internet n lt SECMWF ECtrans Command at ECMWEF echost ectrans help usage ectrans gateway name remote association protocol get put source ec ectmp filename args gateway arg access gateway name default ecaccess ecmwf int remote arg association and optional transfer protocol default none source arg source file name target arg target file name default same as source mailto arg target email address default current user put interactive synchronous transfer no spool get interactive synchronous pull rather than push file onsuccess mail sent on successful transfer onfailure mail sent when transfer has failed onretry mail sent when transfer is retried verbose verbose mode on COM INTRO 2014 ECaccess 22 w ECMWF ECtrans Command at ECMWEF ecgate e
6. ctrans gateway ecaccess ecmwf int remote from_ecmwf genericFtp source data grib target data grib verbose echost ecaccess ecmwf int ecport 644 ecuser usl source data grib target data grib keep false option reject File to upload 585 bytes 10129250216245 e Users have to maintain associations on the particular gateway used via the new ECtools or via the web interface e ectrans is available for transfers in both directions SS ECMWF ECtools Transfer Management e You can also request an ectrans transfer from your site using the ECtools e In this case the transfer will still be initiated from ECMWF mshost ecaccess ectrans request overwrite onFailure onRetry tr assoc ec testfile txt SS ECMWF ECtools Transfer Management mshost echost ecaccess ectrans list 389 STOP tr00 test teaccess ecmwf int Mar 09 18 29 387 DONE tr00 test teaccess ecmwf int Mar 09 14 39 mshost echost ecaccess ectrans list 389 Copyid 389 MS user tr00 test Hostname teaccess ecmwf int Status STOP Last error Target file already exist Schedule Mar 09 18 29 Source wshome ectrain tr00 testfile Target testfile File size 11 e Target file does already exist COM INTRO 2014 ECaccess 25 Te ECMWF ECtools Transfer Management e Either remove target file from destination and restart OR e Restart and overwrite target file OR e Specify different target file name OR e Delete transfer
7. e history ECtrans setup Get certificate Log off usl ECMWF gateway service gt ECtrans gt Your ECMWF account Dominique Lucas x2386 Login User id Group id Home directory Your ECtrans configuration Access methods associated with ECMWF user usl oO Name List of ECtrans associations m from ecmwf Total 1 Name List of ECtrans destinations P genericSftp P genericFtp genericFile Add associaton Setup usl 1107 1212 home us usl Comment Enabled Module Grant association s From your ECMWF account ECuser usl you can use ECtrans to perform unattended file transfers to or from one of your local Member State accounts You need to create associations with these MSusers The access method needs to be specified in the remote option of the ectrans command in the format association destination where association specifies the ECtrans association and destination specifies the ECtrans destination Click the expand button to the left of an ECtrans association entry to COM INTRO 2014 ECaccess A l internet ECtrans Association Web Maintenance A ECMWF gateway service gt ECtrans gt DestinationDetails Microsoft Internet Explorer File Edit view Favorites Tools Help Back gt O A A Asearch yFavorites Smeda 4 B A E JM EChome files ECscratch files ECfs files Browse queues Browse basket Submit
8. ecaccess association get ecaccess association list ecaccess association protocol ecaccess association put ecaccess ectrans delete ecaccess ectrans list ecaccess ectrans request ecaccess ectrans restart Delete Association Get the Association Descriptive File List your ECtrans associations List the supported ECtrans Protocol Update Create an Association Delete ECtrans List all ECtrans transfers Request a new ECtrans transfer Restart an existing ECtrans transfer SS ECMWF ECtrans Association Maintenance e ECtrans associations can be generated with e New ECtools ecaccess association put e ECaccess gateway web interface e Creation and use of an association template using new ECtools mshost ecaccess association get template assoc assoc txt mshost SEDITOR assoc txt EDIT your template mshost ecaccess association put password assoc txt SS ECMWF ECtrans Association Web Maintenance a ECMWF gateway service gt ECtrans gt Setup Microsoft Internet Explorer aj x File Edit View Favorites Tools Help Back gt Q A Qsearch Favorites media A Eb S St S Links 2 EChome files ECscratch files ECfs files Browse queues Browse basket Submit new job Job submissions File transfers Browse history ECtrans setup Get certificate Log off usl ECMWF gateway service gt ECtrans gt Setup O Your ECMWF account Dominique Lucas x2386 Login usl User
9. ed at ECMWF ecaccess ecmwf int for Internet and msaccess ecmwf int for RMDCN LO ECMWF ECaccess Architecture e ECaccess Server Located at ECMWF ecaccess ecmwf int or msaccess ecmwf int Entry point for all ECaccess gateways Keeps track of users activity e ECaccess ecgate Runs the ECaccess Certificate Authority CA Provides job management through LoadLeveler c2a SLURM ecgate or PBS cca Provides access to home scratch ECFS and host filesystems Keeps job input output and error files Manages spool for unattended file transfers SS ECMWF ECaccess local MS gateways e Use local ECaccess gateways if installed e Internet ECaccess gateways available at most NMSs and at several other sites 44 gateways in total e 11 ECaccess Gateways available over RMDCN e If local ECaccess gateway is not available then use ECMWF gateways ecaccess ecmwf int Internet or msaccess ecmwf int RMDCN TLO ECMWF ECtools ECaccess tools These tools must be installed at your end on each platform for which access to ECMWF services are required Two versions available at the moment 2 Version 4 0 0 has been available for 2 year 2 Version 3 3 0 will disappear 2 Version 4 0 0 of the tools requires version 4 0 0 of gateway to be installed Check with your administrator You can use ECMWF gateways in the meantime ECtools are also available on ECMWF computers LO ECMWF ECtools eccert Command
10. id 1107 Group id 1212 Home directory home us usl Your ECtrans configuration Access methods associated with ECMWF user usl Name Comment Enabled List of ECtrans associations No ECtrans associations Total 0 Name Module Active List of ECtrans destinations gt el gt gt genericFile file genericFtp ftp generic Sftp sftp true true true Total 3 Add associaton From your ECMWF account ECuser usl you can use ECtrans to perform unattended file transfers to or from one of your local Member State account s You need to create associations with these MSusers The access method needs to be specified in the remote option of the ectrans command in the format association destination where association specifies the ECtrans association and destination specifies the ECtrans destination Click the expand button to the left of vi an Ftrans bone Ty fg Internet Li COM INTRO 2014 ECaccess lt s ECMWF ECtrans Association Web Maintenance 3 ECMWF gateway service gt ECtrans gt Create Microsoft Internet Explorer lel x File Edit View Favorites Tools Help Bak gt Q A Qsearch Favorites media KEY sak enko MS user association details A From your ECMWF acne as MS user to be associated EEEN you can use ECtrans ECfs files Association name from ecmwf to perform unattended file enabled yi transfers to or from a
11. new job Job submissions File transfers Browse history ECtrans setup Get certificate Log off usl ECMWF gateway service gt ECtrans gt DestinationDetails The genericFtp ECtrans destination and your authorized MS users ECtrans destination details Destination name genericFtp ECtrans module name ftp Comment Generic FTP access from the gateway Parameter name Source Comment target ectrans at ECMWF target location target filename msuser login gateway database MS user login msuser passwd gateway database MS user password msuser dir gateway database MS user directory msuser host gateway database MS user host name Total 5 Active destination true Restrict access false Resolve path true Authorized MS users Comment Enabled List of MS users Total 1 ECtrans uses the listed parameters and the ftp ECtrans module with which the genericFtp destination is associated to access your local Member State account All the MS users granted access through the genericFtp ECtrans destination are listed here COM INTRO 2014 ECaccess ECMWF Disclaimer ECtrans Association Web Maintenance e ECMWF gateway service gt ECtrans gt DoGrant Microsoft Internet Explorer 305 ECMWF gateway service gt ECtrans gt DoGrant Association s you have selected d Grant an ECMWF user see aT EChome files ECscratch files ECfs files Browse queues Browse basket Su
12. percomputer ecaccess job delete Delete an ECaccess Job ecaccess job get Download a Job Output Input Error File ecaccess job list List all ECaccess Jobs ecaccess job restart Restart an ECaccess Job ecaccess job submit Submit a new ECaccess Job ecaccess queue list List available queues e If you submit a shell script you will have to use the option noDirectives which will add default LoadLeveler directives COM INTRO 2014 ECaccess 12 Te ECMWF Data transfers from ECMWF ectrans e Unattended file transfers e Automatic retrial on failure e Synchronous or asynchronous e Downloads or uploads e Secured transfers using ftp or sftp e Ectrans associations can be set up for various destinations e Association name is unique on each gateway e Association has to be created on the particular gateway in use e Email notification e Transfers are always initiated from ECMWF TL ECMWF ECtrans Command e It is preferable to use your local ECaccess gateway if available Alternatively use the gateway at ECMWF file MS Server running the gateway Shared File System ke Y MS Network ectrans gateway ecaccess ecmwf int ecgate or remote association protocol cla source location filename target location filename COM INTRO 2014 ECaccess 14 Toe ECMWF ECtools Association and Transfer Management ecaccess association delete
Download Pdf Manuals
Related Search