Grid user guide - Institute of Nuclear and Particle Physics
Contents
1. cd myRECO ls echo do FClistPFN FClistPFN u file ORCA XML q owner Digis873 echo W Ck ck Kk 3l echo Starting FCrename for PFN in FClistPFN u file ORCA XML q owner Digis873 5 do echo SPFN FCrenamePFN u file ORCA XML p SPFN n basename SPFN if 0 then echo Problems renaming PFNs exit 1021 fi done end of xml paths modification Run ORCA dst export ORCASTEER dstrc echo gt which writeAllDigis which writeDST echo gt Run ORCA dst date writeDST c SORCASTEER gt amp SORCASTEER log echo Done with ORCA dst date Wrap up output axe Rr tar cvf myRECO output tar myRECO echo gt Saving output to SE T echo lcg cr vo cms d arxiloxos2 inp demokritos gr 1 lfn grid cms SDST OUTPUT LFN file SPWD myRECO output tar export GUID 2 lcg cr vo cms d arxiloxos2 inp demokritos gr 1 lfn grid cms SDST OUTPUT LFN file S PWD myRECO output tar echo GUID SGUID date echo end of grid job wrapper 32 4 Modify the following lines in dst rc Modify the following lines MaxEvents 2 FilePath InputFileCatalogURL file pfc myRECO xml InputCollections System Digis873 myDIG Ou tFileCatalogURL file pfc2. 4 Copy and register your input ntuple to the Storage Element arxiloxos2 Q inp demokritos gr and in the Replica Catalog 34 For the above example leg cr vo cms file my workdir kat su05 TZt bjj 1 ntpl d arxiloxos2 inp demokritos gr 1 lfn grid cms kat su05 TZt bjj 1 ntpl 5 Modify the reco sh for your input ntuple and your output files ui templates OSCAR gt more reco sh bin esh f echo Running grid job wrapper f The Basics echo Environment date hostname cat proc cpuinfo uname a echo VO CMS SW DIR ls 1 VO CMS SW DIR f44S4 source SVO CMS SW DIR cmsset default csh source VO CMS SW DIR cmsset slc3 ia32 gcc323 csh pwd echo End of env testing Setup environment mkdir myRECO cp oscarrc myRECO cp orcarc myRECO Input ntuple name Ls c echo locate lcg cp locate lcg cp echo retrieve ntuple from SE lcg cp vo cms lfn grid cms kat su05 TZt Dbjj 1 ntpl file PWD myRECO kat su05 TZt bjj 1l ntpl 35 ls 1 Setup OSCAR Here starts OSCAR step echo Setup OSCAR date scram list scram project OSCAR OSCAR 3 6 5 setenv OSCARDIR OSCAR 3 6 6 5 cd SOSCARDIR eval scram runtime csh 4 Run OSCAR not needed source src Workspace writeTrigger csh cd myRECO setenv OSCARSTEER oscarrc ec
3. RetryCount 0 Requirements other GlueCEUniqueID xg009 inp demokritos gr 2119 jobmanager lcgpbs ocoms 7 Submit the job to the grid ui edg job submit vo cms o reco output reco jdl 8 Retrieve the output ui gt leg cp vo cms lfn grid cms myRECO output tar file your path file to create 37 9 Submit a OSCAR DIGI DST job to the INP Grid cluster In case you want to run OSCAR and then DIGI and DST job you can use the template files found in the directory ui templates OSCAR DIGI DST 1 Copy the template files into your working directory rwXr r 2535 Feb 27 15 45 reco dst sh 320 Feb 27 15 45 reco dst jdl 4068 Feb 27 15 45 orcarc 4428 Feb 27 15 46 oscarrc 4211 Feb 27 15 46 dstrc EW SS oa SEW SA TLL FEWN a e ct ct ct ct A 2 Follow the steps of the previous section In this case you should modify additionally the dstre file 4 Modify the following lines MaxEvents 3 FilePath 7 InputFileCatalogURL Q file pfc myRECO xml InputCollections System Digis873 myRECO OutputFileCatalogURL file pfc myRECO xml OutputDataSet System DST873 myRECO 38 10 Useful Links i eerie eames Management Data Management Information ne GLUE d ttps x edms cern ch file 454439 LCG 2 Userguide pdf TtU docu atio 2 Userguide pdf 3 pe
4. myRECO xml OutputDataSet System DST873 myDST 5 Submit the DST job to the grid ui gt edg job submit vo cms o dst output dst jdl 6 retrieve the output tar file ui gt leg cp vo cms lfn grid cms DST_OUTPUT_LFN file your_path lt file_to_create gt tar 33 8 Submit an OSCAR DIGI job to the INP Grid cluster In case you want to run OSCAR and then DIGI jobs you can use the template files found in the directory ui templates OSCAR DIG 1 Copy the template files into your working directory 4068 Feb 27 14 24 orcarc 4428 Feb 27 14 24 oscarrc 284 Feb 27 14 26 reco jdl 2322 Feb 27 14 26 reco sh ZEW oS SCW a a OEW I EL ct ct cr oct CIINEU LI 2 Modify the oscarrc Modify the following lines NumberOfEventsToBeProcessed 5 FilePath EventNtplReader NtplFileName kat su05 TZt bjj 1l ntpl OutputFileCatalogURL file pfc myRECO xml OutputDataSet System SimHits365 myRECO VCalShowerLibrary Fil vcal5x5 rz VCalShowerLibrary Fil CMS_PATH cmsim cmdb vcal 3 Modify the orcarc Modify the following lines MaxEvents 5 FilePath InputFileCatalogURL file pfc myRECO xml InputCollections System SimHits365 myRECO OutputFileCatalogURL file pfc myRECO xml OutputDataSet System Digis873 myRECO
5. Institute of Nuclear Physics Grid deployment group C Filippidis C Karafasoulis C Markou D Loukas K Zachariadou Grid Site User Guide DRAFT 0 A GR 05 DEMOKRITOS Grid Site 3 B User guide 4 1 Get a digital certificate 5 2 Registering with LCG 2 8 2 1 Virtual Organizations 8 3 Submit a job to the INP Grid Cluster 9 3 1 Job without input output data streams 9 3 2 Job with input output data streams 12 3 2 1 small input output data files 12 3 2 2 big input output data files 13 4 submit a CMKIN job 18 5 submit an OSCAR job 22 6 submit an ORCA job 26 7 submit a DST job 30 8 submit an OSCAR ORCA job 34 9 submit an OSCAR ORCA DST job 38 10 Useful links 39 A GR 05 DEMOKRITOS Grid Site Domain inp demokritos gr Monitoring DEMOKRITOS Web Site http erid demokritos er GR 05 DEMOKRITOS Phedex PubDB T2 Demokritos Grid deployment Group GdG arxiloxos1 n CERT L A VOBox VOs Cms E xg006 Atlas I LIB ds Lhcb ie LT xg010 Demokritos In the framework of the analysis of CMS data as well as the production of simulated events to be used in current and future analyses the Institute of Nuclear Physics is developing a PC Linux computing farm with high capacity and fast response At the present time the farm is still at an early stage of development and under continuous upgrade and expansion At the time of writing this guide the
6. an empty file usercert request pem usercert pem 3 Transfer the file usercert request pem to your computer and send it by e mail to he llasgrid ca physics auth gr 4 Ask Christos Markou to send a fax to HellasGrid CA verifying that you are a member of the IPF Demokritos 5 HellasGrid CA will send you by e mail your personal certificate username inp demokritos gr pem 6 Tranfer the file username inp demokritos gr pem to the UI ui inp demokritos gr and locate it in your globus directory 7 Goto globus directory and rename it ui mv username inp demokritos gr pem usercert pem 8 In order to import your private key and certificate in your browser you must create a pkcs12 bundle This can be achieved by issuing the command ui username globus gt openssl pkcs12 export in globus usercert pem inkey globus userkey pem name My Certificate out mycertificate p12 After issuing the above command you will be asked to enter the pem pass phrase This is the pass phrase you entered during the initial process of creating the certificate request Next you will have to enter an export password for the pkcs12 bundle and you will have to use it during the import procedure 9 Transfer the pkcs12 bundle to your computer 10 Use Internet Explorer recommended and follow the instructions Step 1 Importing HellasGrid CA Certificate to the browser 11 Open http pki physics auth gr hellasgrid ca C
7. Oct 26 13 28 usercert pem mocms production 1339 Oct 20 11 42 usercert request pem mocms production 963 Oct 20 11 42 userkey pem You will need a proxy certificate This proxy is your credential which authenticates you in every secure interaction on Grid and has a temporal life In order to create a proxy ui home your username grid proxy init If the command is successful you will receive as output Your identity C GR O HellasGrid OU inp demokritos gr CN Katerina Zachariadou Enter GRID pass phrase for this identity you must introduce your password Creating proxy Done Your proxy is valid until Thu Nov 10 02 06 53 2005 To submit a job you must first create a jdl file stating the executable image and the output files as in the following minimal test jdl Executable tjob sh A j file l l pre guuenee aa If your job has an input StdOutput tjob out 8i sear StdError tjob ierr md ile named inputiile InputSandbox tjob sh InputSandbox myexecutable inputfile OutputSandbox tjob out tjob err Requirements other GlueCEUniqueID xg009 inp demokritos gr 2119 jobmanager lcgpbs cms Find a template job t job sh and a jdl file t job jd1 in the directory tests General and copy them to your working directory Find also a minimal jdl file t ortran jdl for s
8. S A S M MK Kx A Kk KR Kx X X koX Wait for done status flag When the job status is Done then one can retrieve the results ui home your username edg job get ui home democms tests edg job get output https arxiloxosl inp demokritos gr 9000 Tr7Muof36saOHK6 7ynr6Tw Retrieving files from host arxiloxosl inp demokritos gr for https arxiloxosl inp demokritos gr 9000 Tr7Muof36saOHK6 7ynr6Tw kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk JOB GET OUTPUT OUTCOME Output sandbox files for the job https arxiloxosl inp demokritos gr 9000 Tr7Muof36saOHK6 7ynr6Tw have been successfully retrieved and stored in the directory tmp jobOutput democms Tr7Muof36saOHK67ynr6Tw 11 meaning that the system has decided to put the output files into the directory tmp jobOutput democms Tr7Muof36saOHK67ynr6Tw A final step could be to copy the files to your home directory ui home your username cp tmp jobOutput democms Tr7Muof36saOHK67ynr6Tw HOME Output Directory If you want to delete the job ui home your username edg job cancel JID 3 2 Job with input output data streams Important notice The input and output sandboxes are intended for relatively small files few megabytes like scripts standard input and standard output streams If you are using large input files or generating large output files you should instead directly read from or write to a storage element Abu
9. available infrastructure consists of 2 Computing Elements two fileservers 2 5TB disk space overall and 34 working nodes 22 PentiumIV 120Gb HD 6 PentiumllI and 6 Xeon 80Gb double processor configured so that they form a Grid Site which is part of the LCG Grid The GR 05 Demokritos Grid Site is a Phedex Physics Experimental Data Export node controlling and maintaining 3TB of data for CMS accessible from CERNs PubDB B User guide In what follows we have tried to record the series of steps which a user has to take in order to use the available infrastructure in order to perform various computational tasks both in the context of a general user as well as in the context of CMS data production and analysis I Important Notice This document is under construction and may be incomplete Please provide feedback to Katerina Zachariadou zacharia inp demokritos gr on missing inaccurate or unclear information 1 Get a digital certificate 1 Login to the INP Grid cluster User Interface UI ui inp demokritos gr 2 Create your personal key and certificate request ui grid cert request int You will be asked to create a password for your grid proxy Then you will be asked to insert the following parameters Country name gt GR Level 0 Organization HellasGrid Level 0 Unit inp demokritos gr Name your name A directory will be created named 91obus Inside the globus directory you will find the files usercert pem
10. A in your browser 12 Click on Click here to import HellasGrid CA certificate directly to your browser 13 A Window will pop up Click on the button open 14 Select the option Install Certificate Click on Next two times and the Finish 15 Answer YES to the question Do you want to ADD the following certificate to the Root Store You have successfully imported the HellasGrid CA Certificate into Internet Explorer Step 2 Import your pkcs12 bundle to the browser 16 Open an Internet Explorer Window 17 Go to File gt Open Browse to the pkcs12 bundle you have transferred to the computer and open the file 18 A new window will popup Press Next to the following two screens 19 I n the 3rd screen you will be asked to enter your password This is the export password you entered previously 20 Enable Enable strong private key protection You will be prompted every time the private key is used by an application if you enable this option DO NOT enable Mark this key as exportable This will allow you to backup or transport your keys at a later time 2 Select Next in the following 2 screens and then Finish 22 Select OK to the following window Congratulations you have imported your certificate and private key to the browser Go to Microsoft Outlook 23 Goto Tools gt Options gt Security and press Setup Secure e mail Select OK Go back to Security and select the Options a add digital signatu
11. E EE HE EH HH HE Executable t_large2 sh InputSandbox home democms tests t large2 sh home democms tests te st in out exe utputSandbox t large2 out tL large2 err tdOutput t large2 out tdError t large2 err Requirements other GlueCEUniqueID xg009 inp demokritos gr 2119 jobmanager lcgpbs ocoms To run the above example copy the appropriate files test in out exe t large2 sh t large2 jdl in your working directory modify the paths and submit the t large2 jdl to the grid ui edg job submit vo cms o tlarge2 t large2 jdl You can retrieve the output file eg with LEN output17 from the RLS using the command ui gt leg cp vo cms l1fn testout17 file PWD local filename Important notice If you need to run the same job more than once be sure to register the output file with a different LEN or to delete from the RLS the LEN of the output before re submit the job 16 Useful commands To delete a registered file from the Replica Catalog ui gt leg del vo VO name lfn logical file name To find the guid of a registered file ui gt leg lg vo cms lfn 1fn name gt 17 4 Submit a CMKIN job to the INP Grid cluster under construction Version CMKIN 4 4 _0 is installed go to your working directory and create a directory eg mkdir cmkin get your personal copy of the cmkin examples from the repositor
12. ar 20 16 05 dst sh 2 Modify the dst jd1 ui templates DST gt more dst jdl Executable dst sh tdOutput dst out tdError dst err InputSandbox dst sh dst jdl dstrc OutputSandbox dst out dst err RetryCount 0 Requirements other GlueCEUniqueID xg009 inp demokritos gr 2119 jobmanager lcgpbs cms 30 3 Modify the dst sh ui templates DST more dst sh usr local bin zsh modify the following lines export ORCA XML pfc myRECO xml export ORCA OUTPUT LFN2myRECO 1231 output tar export DST_OUTPUT_LFN myRECO 1111 output tar echo Input variables echo gt LFN for ORCA output XML S ORCA XML echo LFN for ORCA output tar file S SORCA OUTPUT LFN echo gt LFN for DST output tar file DST_OUTPUT_LFN The Basics echo Environment date hostname cat proc cpuinfo uname a echo VO CMS SW DIR source VO CMS SW DIR cmsset slc3 ia32 gcc323 sh pwd echo End of env testing Setup environment echo retrieve DIGI output from SE lcg cp vo cms lfn grid cms SORCA OUTPUT LFN file SPWD SORCA OUTPUT LFEN tar xvf SORCA OUTPUT LFN cp dstrc myRECO Js Setup ORCA echo Setup ORCA date scram project ORCA ORCA_8_7_3 export ORCADIR ORCA_8_7_3 cd ORCADIR eval scram runtime sh modify the ORCA xml paths
13. arxiloxos2 inp demokritos gr export OutputSE DIR storage cms export SEPrefixDir Katerina export VO cms export RUNTIME AREA pwd export myexe test_in_out exe export OutFileName testout18 Ht KK KK IK IK KK Ck kCkck ck ck ck k ck kk ck kk ck kck kk ck kk ck k ck kckck kk ck k kk ls al 14 Get InputFile from RLS echo About to copy file from SE echo gt gt lcg cp v vo cms t 300 lfn S InputFile file SRUNTIME AREA SInputFile lcg cp v vo cms t 300 lfn InputFile file RUNTIME AREA InputFile echo Working directory listing after staging Runtime area listing SRUNTIME AREA job runs here o Where am I o pwd o myexe begin at date od ugot tx Smyexe myexe o job end at date Oo mw Directory Listing S gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt gt Stageout owas echo gt gt lcg cr v vo cms t 300 file SPWD testout d SOutputSE 1 lfn SOutFileName leg cr v vo cms t 300 file PWD testout d OutputSE 1 lfn OutFileName 15 Put the name of the script eg t large2 sh and of the executable eg test in out exe inthe jdlfile eg t 1arge2 jdl t large2 jdl UserReq other GlueCEUniqueID xg009 inp demokritos gr 2119 jobmanager lcgpbs cms HE HH HE HEH EHH HEE EE HE EH HH EEE E E E E E E E HE
14. ble to the grid create a cmkin jdl and a cmkin sh file find template files in the directory templates CMKIN CMKIN 4 4 0 grid Card file O grid cmkin sh O0 grid cmkin jdl 0 grid eg gj with hpt piO txt O0 grid kine make ntpl ck run l pyt6227 exe gz CMKIN run script to create a CMKIN executable MUST N be zipped HEPEVT ntuple 19 cmkin sh bin csh setenv OutputNtuple eg gj with hpt piO pt20 30 ntpl setenv OutLFN test cmkinl02 ntpl setenv OutputSE arxiloxos2 inp demokritos gr setenv OutputSE_DIR storage cms setenv VO cms setenv SEPrefixDir Katerina set PWD pwd echo SPWD ft I KK IK OK IK KK IK IK KK IK Ok ck k ck kCk kckck IK ck kk kckck ck k KK Kk kK if VO CMS SW DIR then echo VO CMS SW DIR not set echo Setting it manually to opt exp software cms setenv VO CMS SW DIR opt exp software cms else echo VO CMS SW DIR is already set echo VO CMS SW DIR VO CMS SW DIR endif if f SVO CMS SW DIR cmsset slc3 ia32 gcc323 csh then echo Try to set the CMS Environmet by running the following script ls 1 S VO CMS SW DIR cmsset slc3 ia32 gcc323 csh source VO CMS SW DIR cmsset slc3 ia32 gcc323 csh else echo CMS Environment could NOT be set Exiting exit 10 endif gunzip kine_make_ntpl_pyt6227 exe gz chmod ugot x kine make ntpl pyt6227 exe ch
15. cms d arxiloxos2 inp demokritos gr 1 lfn grid cms S OSCAR OUTPUT LFN file SPWD myRECO output tar echo GUID GUID date echo end of grid job wrapper 4 In oscarrc modify the following lines Modify the following lines NumberOfEventsToBeProcessed 2 FilePath EventNtplReader NtplFileName kat_su05_TZt_bjj_1 ntpl OutputFileCatalogURL file pfc myRECO xml OutputDataSet System SimHits365 myRECO VCalShowerLibrary FileName vcal5x5 rz VCalShowerLibrary FilePath CMS PATH cmsim cmdb vcal 3 Copy and register your input ntuples to the Storage Element arxiloxos2 Q inp demokritos gr and register the file in the Replica Catalog lcg cr vo cms file my workdir myntuple ntp d arxiloxos2 linp demokritos gr 1 grid cms Alogical File Name 24 For the above example leg cr vo cms file my workdir kat su05 TZt bjj 1 ntpl d arxiloxos2 inp demokritos gr 1 lfn grid cms kat su05 TZt bjj 1 ntpl 4 Submit the OSCAR job ui gt edg job submit vo cms o oscar output oscar new2 jd1l To check the output use the command ui gt leg lr vo cms l lfn grid cms OutfileName Where OutfileName is defined in oscar_new2 sh To get the output file from the Replica Catalog use the command ui gt leg cp vo cms lfn grid cms OutfileName file your path file t
16. ep server 1 infn it workload grid docs DataGrid 01 TEN 0142 workload grid docs DataGrid 01 TEN 0102 0 2 Document pdf 6 Replica Catalog commands oa age 7 T5 used by lcg utils http grid deployment web cern ch grid is GFAL gfal 3 html 39
17. ho printenv printenv echo gt which oscar which oscar echo Run OSCAR date oscar c SOSCARSTEER gt amp SOSCARSTEER log echo gt Done with OSCAR date Move back in relative mode cd ho Setup ORCA Here ORCA starts echo gt Setup ORCA date scram project ORCA ORCA 8 7 3 setenv ORCADIR ORCA 8 7 3 cd ORCADIR eval scram runtime csh Run ORCA digi cd myRECO setenv ORCASTEER orcarc echo printenv printenv echo gt which writeAllDigis which writeAllDigis Put the lfn for echo Run ORCA digi your output date writeAllDigis c SORCASTEER gt amp SORCASTEER log echo Done with ORCA digi Wrap up output GO Sud tar cvf myRECO output tar myRECO echo Saving output to SE echo lcg cr vo cms d arxiloxos2 inp demokritos gr 1 lfn grid cms myRECO output tar file SPWD myRECO output tar setenv GUID lcg cr vo cms d arxiloxos2 inp demokritos gr 1 lfn grid cms myRECO output tar file S PWD myRECO output tar echo GUID SGUID date 6 Modify if you want the reco jd1 ui templates OSCAR more reco jdl xecutable reco sh tdOutput r vOut dO t reco out tdError reco err InputSandbox reco sh reco jdl oscarrc Worcane s OutputSandbox reco out reco err
18. llowing lines setenv INPUT NTPL kat_su05_TZt_bjj_l ntpl setenv OSCAR OUTPUT LFN myRECO639 0utput tar echo Input variables echo gt LFN for Input Ntuple SINPUT_NTPL echo LFN for OSCAR output tar file SOSCAR_OUTPUT_LFN The Basics echo Environment date hostname cat proc cpuinfo uname a echo VO CMS SW DIR ls 1 VO CMS SW DIR source VO CMS SW DIR cmsset slc3 ia32 gcc323 csh pwd echo End of env testing Setup environment mkdir myRECO cp oscarrc myRECO Is echo gt locate lcg cp locate lcg cp echo retrieve CMKIN ntuple from SE lcg cp vo cms lfn grid cms SINPUT_NTPL file S PWD myRECO S INPUT NTPL ls 1 Setup OSCAR echo Setup OSCAR date scram list scram project OSCAR OSCAR 3 setenv OSCARDIR OSCAR 3 6 5 UM B cd SOSCARDIR eval scram runtime csh 23 Run OSCAR cd myRECO setenv OSCARSTEER oscarrc echo printenv printenv echo gt which oscar which oscar echo Run OSCAR date oscar c SOSCARSTEER gt amp SOSCARSTEER 1log echo gt Done with OSCAR date DU ua tar cvf myRECO output tar myRECO echo Saving output to SE echo lcg cr vo cms d arxiloxos2 inp demokritos gr llfn grid cms SOSCAR OUTPUT LFN file SPWD myRECO output tar setenv GUID lcg cr vo
19. mod ugot x kine make ntpl ck run SPWD kine make ntpl ck run pythia ls al pwd set PWD pwd echo lcg cr v vo VO t 300 file PWD SOutputNtuple d SOutputSE 1 lfn grid cms OutLFN lcg cr v vo VO t 300 file SPWD SOutputNtuple d SOutputSE 1 lfn grid cms S OutLFN leg cp vo cms lfn grid cms SOutLFN file home democms tests CMKIN SOutLFEN 20 emkin jdl HEH HH HEE HE HHH HEE EE HH EH HH HE EE E E EH EH E E E E EE E E EE EH HH HE User defined part of the jdl data test test cmkin jdl HE HT HE HEH EHH HEE EE EH E E HH EEE E E EE E E EE E E E EE EE EH HH HE UserReq other GlueCEUniquel xg009 inp demokritos gr 2119 jobmanager lcgpbs cms HE HH HE HEH HH HH EE EE EH EH HH E E EEE E E EE E E E E E E EEE EE EE EH Executable cmkin sh CMKIN datacard file InputSandbox home democms test s CMKIN cmkiac fh home democms tests CMKIN eg gj with hpt piO txt home democms tests CMKIN kine make ntpl ck run home democms tests CMKIN kine make ntpl pyt6227 exe gz utputSandbox cmkin job out cmkin job err j tdOutput cmkin job out tdError cmkin job err RetryCount 3 CPUTime other GlueCEPolicyMaxCPUTime gt 1440 Memory other GlueHostMainMemoryRAMSize gt 500 Requirements Member VO cms ORCA_8_7_3 other GlueHostApplicationSoftwareRunTimeEnvironme
20. nt amp amp CPUTime amp amp Memory amp amp UserReq To copy the CMKIN output to your local area use the command ui gt leg cp vo cms lfn grid cms OutfileName file your path file to create For the above example ui gt leg cp vo cms lfn grid cms test cmkin102 ntpl file your path file to create 21 5 Submit a OSCAR job to the INP Grid cluster You can find template files for submitting OSCAR jobs to the INP grid cluster in the directory ui templates OSCAR 1 Copy the template files into your working directory ui templates OSCAR 11 total 24 rw r r 1 roo root 899 Dec 20 14 19 oscar new2 jdl rwXrI Xr x 1 roo root 3731 Dec 20 14 16 oscar_new2 sh rw r r 1 roo root 3941 Dec 20 14 16 oscarrc 2 Modify jdl oscar job2 jdl optional Executable oscar new2 sh tdOutput oscar new2 out tdError oscar new2 err InputSandbox oscar new2 sh oscar new2 jdl oscarrc Jj OutputSandbox oscar new2 out oscar new2 err j RetryCount 0 Requirements other GlueCEUniqueID xg009 inp demokritos gr 2119 jobmanager lcgpbs cms 22 2 oscar new2 sh The script sets up the environment runs OSCAR and saves the output to the SE Set the name of your input ntuple and the LEN for the OSCAR output file tar file bin csh f echo Running grid job wrapper modify the fo
21. o create 25 6 Submit a ORCA job to the INP Grid cluster Find template files for submitting ORCA jobs to the ui templates DIG INP grid cluster in the directory 1 Copy the template files into your working directory ui templates D gt 11 Mar 7 16 19 orca new2 jdl rw r r 1 root root 318 rwXr r 1 root root 1754 Mar 7 16 19 orca_new2 sh rw r r 1 root root 4068 Mar 7 16 20 orcarc 2 Modify the orca new2 jd1 Executable orca new2 sh tdOutput orca new2 0ut tdError orca new2 err OutputSandbox orca new2 out RetryCount 0 InputSandbox orca_new2 sh orca new2 jdl orcarc j orca_new2 err Requirements other GlueCEUniquel xg009 inp demokritos gr 2119 jobn nanager lcgpbs ocoms 26 3 Modify the orca new2 sh usr local bin zsh modify the following lines t OSCAR XML pfc myRECO xml export OSCAR OUTPUT LFN 2myRECO654 0utput tar export ORCA_OUTPUT_LFN myRECO 1231 output tar echo gt Input variables echo LFN for OSCAR output XML OSCAR XML echo LFN for OSCAR output tar file SOSCAR OUTPUT LFN echo gt LFN for ORCA output tar file SORCA_OUTPUT_LFN The Basics echo Environment date hostname cat proc cpuinfo uname a echo VO CMS SW DIR source VO CMS SW DIR cmsset slc3 ia32 gcc323 sh pwd echo gt End of env
22. res to outgoing messages b send clear text signed message Select Apply and OK You are ready to send a digitally signed e mail to hellasgrid ca grid auth gr stating that you accept your certificate and that you adhere to theHellasGrid Certification Policy To whom it may concern With this email I state that 1 I HHHH accept my x509v3 digital certificate with DN CZGR O HellasGrid OU zinp demokritos gr CN HHHHHHE Serial Number 0x011C signed by C GR O HellasGrid CN HellasGrid CA 2 I adhere the HellasGrid CA policy and usage rules found at O I D 1 3 6 1 4 1 13089 2 1 10 1 4 2 Registering with LCG 2 Before a user can use the LCG 2 service registration of some personal data with the LCG registration server hosted at CERN plus some additional steps are required For detailed information please visit the following URL To actually register oneself to the LCG 2 service it is necessary to use a WWW browser with the user certificate installed for the request to be properly authenticated 2 1 Virtual Organizations A second requirement for the user is to belong to a Virtual Organization VO A VO is an entity which corresponds typically to a particular organization or group of people in the real world The membership of a VO grants specific privileges to the user For example a user belonging to the ATLAS VO will be able to read the ATLAS files or to exploit resources reserved to the ATLAS collaboration En
23. se of the input and output sandboxes can fill the storage on the Resource Broker and cause the broker to crash 3 1 a Small input output data files Use Input Output Sandboxes in the jdl file Find a template jdl file t smal11 jdl in the directory templates General The above jdl executes a simple fortran executable test in out exe that reads data from a file testin and writes data to an output file testout 1 Copy the files t_small jdl testin test_in_out f test in out exe to your working directory t small jdl a very simple fortran executable StdError t small err d InputSandbox test in out exe home democms tests testin OutputSandbox t small log t small err testout Executable test in out exe Input data StdOutput t small log Output data 12 2 Create a proxy if you haven t already created ul gt grid proxy init 3 Submit the job ui edg job submit vo cms o tsmall output t_small jdl Wait for done status flag Find JID in the tsmall_output file L ui edg job status D 5 Retrieve the output file testout ui gt edg job get output JID 6 Copy the output testout to your working dir 3 1 b Large input output data files The Replica Catalog can be considered to be temporary storage that is available at each site on the grid store node the Storage Element of the site s grid gateway All large data sets should be inpu
24. t and output to from grid jobs via the Replica Catalog rather than via the InputSandbox or OutputSandbox Once registered with the Replica Catalog files may be referred to via grid logical filenames LFNs that the user associates with the file at the time of registration Suppose that the input data file testin see above is a large data set To register the file with the Replica Catalog use the command ui gt leg cr vo VO name file lt local_path gt file name 1 logical file name d storage element ui gt leg cr vo cms file lt local_path gt testin 1 testin d arxiloxos2 inp demokritos gr 13 The above command returns the grid unique identifier guid of the grid file For example something like that guid 767da655 9d60 444d bacf 3bec6cld35ca The output file is located in the Storage Element arxiloxos2 inp demokritos gr storage cms generated current date Files are accessed using the LEN name therefore users are not involved in the directory location of files at any single SE To input data from the Replica Catalog to the job and vice versa use the following commands in a script eg t large2 sh lcg cp v vo cms t 300 lfn InputFile file SRUNTIME AREA SInputFile leg cr v vo cms t 300 file PWD OutFileName d OutputSE 1 lfn OutFileName t large2 sh script usr local bin zsh echo Job started at date W finger ls 1 export I tFile testin export OutputSE
25. tering the VO of an experiment usually requires being a member of the collaboration the user must comply with the rules of the VO relevant to him her to gain membership Of course it is also possible to be expelled from a VO when the user fails to comply with these rules It is not possible to access the LCG 2 Grid without being member of any VO Every user is required to select his her VO when registering with LCG 2 and the supplied information is forwarded to the VO administration and resource prividers for validation before the registration process is completed This forwarding is accomplished by the registration interface back end automaticaly It generates an email to the VO manager of the selected VO requesting addition of the user to the VO Currently it is only possible to belong to one VO at a time This is fine for most users In the rare case that you need to belong to more than VO then you should contact the LCG registration service whose URL was given before A complete list of the VOs accepted by LCG 2 is available at the URL registrar cern ch virtual_organization html 3 Submit a job to the INP Grid cluster 3 1 Job without input output data streams Login to the INP Grid cluster User Interface UI ui inp demokritos gr Before interacting with the Grid make sure that the files inside your globus directory have the following permissions mocms production 2242 Oct 26 13 31 mycertificate p12 mocms production 4861
26. testing Setup environment echo retrieve oscar output from SE lcg cp vo cms lfn grid cms S OSCAR OUTPUT LFN file PWD S OSCAR OUTPUT LFN pwd ls 1 tar xvf SOSCAR OUTPUT LFN cp orcarc myRECO Setup ORCA echo Setup ORCA date scram project ORCA ORCA 8 7 3 export ORCADIR ORCA 8 7 3 cd ORCADIR eval scram runtime sh modify the OSCAR xml paths cd myRECO echo 1st FClistPFN FClistPFN u file S OSCAR XML q DataType META echo WO k k k kW echo 2nd FClistPFN FClistPFN u file OSCAR XML q DataType EV dataset myRECO echo Nexen 27 echo Starting first FCrename for PFN in FClistPFN u file OSCAR XML q DataType META NEL echo SPFN FCrenamePFN u file S OSCAR XML p SPFN n basename SPFN if UL S 2 0 L s then echo Problems renaming PFNs exit 1021 fi done echo Starting second FCrename for PFN in FClistPFN u file SOSCAR_XML q DataType EV dataset myRECO do echo SPFN FCrenamePFN u file S OSCAR XML p SPFN n basename SPFN if 0 then echo Problems renaming PFNs exit 1022 fi done end of OSCAR xml paths modification Run ORCA digi export ORCASTEER orcarc echo gt which writeAllDigis which writeAllDigis echo Run ORCA digi date writeAllDigis c SORCASTEER gt amp SORCASTEER log echo Done with ORCA digi date Wrap
27. ubmission of a simple test fortran exe fortran executable Submission is performed with the command ui home your username edg job submit vo cms o tjob jdl The edg jobld has been saved in job output file If job submission is successful the middleware displays a message of the following type Selected Virtual Organisation name from vo option cms Connecting to host arxiloxosl inp demokritos gr port 7772 Logging to host arxiloxosl inp demokritos gr port 9002 edg job submit Success The job has been successfully submitted to the Network Server Use edg job status command to check job current status Your job identifier edg jobId is https arxiloxosl inp demokritos gr 9000 Tr7Muof36saOHK67ynr 6Tw The edg jobId has been saved in the following file home democms tests job output Monitor execution and recover output as follows ui home your username edg job status JID 10 For the above example ui home your username edg job status https arxiloxosl inp demokritos gr 9000 Tr7Muof36saOHK6 7ynr6Tw BOOKKEEPING INFORMATION Status info for the Job https arxiloxosl inp demokritos gr 9000 Tr7Muof36s aOHK67ynr6Tw Current Status Done Success E S xit code tatus Reason Zob terminated successfully Destination xg009 inp demokEf itos gr 2119 jobmanager lcgpbs cms reached on Fri Nov 4 12 50 38 2005 Ck CK CK Ck ck KC Ck CEA CC CC 0C CS C
28. up output pwd cd tar cvf myRECO output tar myRECO echo Saving output to SE echo lcg cr vo cms d arxiloxos2 inp demokritos gr 1 lfn grid cms SORCA OUTPUT LFN file PWD myRECO output tar export GUID lcg cr vo cms d arxiloxos2 inp demokritos gr 1 lfn grid cms SORCA OUTPUT LEN file SPWD myRECO output tar echo GUID SGUID date echo end of grid job wrapper 28 4 modify the following lines in orcarc if necessary Modify the following lines MaxEvents 2 FilePath InputFileCatalogURL file pfc myRECO xml InputCollections System SimHits365 myRECO OutputFileCatalogURL file pfc myRECO xml OutputDataSet System Digis873 myDIG 4 submit the ORCA job to the grid ui gt edg job submit vo cms o orca output orca new2 jdl 5 retrieve the output tar file leg cp vo cms lfn grid cms ORCA OUTPUT LEN file your path file to create tar 29 7 Submit a DST job to the INP Grid cluster Find template files for submitting DST jobs to the INP grid cluster in the directory ui templates DST 1 Copy the template files into your working directory ui templates DST gt 11 total 16 rw r r 1 roo 1 265 Mar 20 16 05 dst jdl rw r r 1 roo 1 5142 Mar 20 16 06 dstrc rwxr r 1 root 1 2643 M
29. y templates CMKIN CMKIN 4 4 0 examples 3 Find in the subdirectory make_ntpl_jobs the compilation script kine make ntpl com Modify it by adding in the beginning of the file the following lines p set SCRATCH working dir cmkin examples make ntpl jobs set VERSIO CMKIN 4 4 0 4 The compilation script is used as follows ui kine make ntpl com generator where the first parameter can have one of the following values pythia herwig isajet simple single double simplemulti cosmic comphep alpgen madgraph phase toprex or stagen The optional second parameter hapdf is given when the user wants to choose Les Houches Accord parton density functions Presently lhapdf can be used with pythia toprex and stagen eg if you want to use the pythia generator ui kine make ntpl com pythia eg running the above command you will get the executable kine make ntpl pyt6227 exe 5 Add the following line in the beginning of the file kine make ntpl run set SCRATCH your working dir cmkin examples make ntpl jobs 18 6 Replace the line set DATACARDS cmsl1 Releases CMKIN CMKIN 4 4 0 examples with the line set DATACARDS storage YOUR USERNAME cmkin examples make ntpl job s datacards SDATACARDFILE T7 In case you want to run the cmkin executable interactively ui kine make ntpl run generator eg ui kine make ntpl run pythia In order to submit the cmkin executa
Download Pdf Manuals
Related Search