Home

Java CoG Kit 2.0 Manual, pre-alpha

Contents

1. task setProvider MyProvider 7 4 6 Executing Tasks on an ExecutionResource To execute tasks on a customized execution resource you need to first create an ExecutionResource and register the security contexts and service contacts for providers supported by this resource Listing 7 28 Create an execution resource Create an Execution Resource assigning it a user defined name ExecutionResource resource new ExecutionResourcelmpl myGridResource For each provider register a security context x org globus cog core impl gt2 GlobusSecurityContextIimpl gt2security new org globus cog core impl gt2 GlobusSecurityContextIimpl xselects the default credentials x securityContext setCredentials null this resource setSecurityContext GT2 gt2security org globus cog core impl gt3 GlobusSecurityContextIimpl gt3security new org globus cog core impl gt3 GlobusSecurityContextImpl x selects the default credentials securityContext setCredentials null resource setSecurityContext GT3 gt3security is Register tie service COMAC Or EVER provider serviceType combination Es Set ie seryice comtaci dior GI job excoumion lt ServiceContact gt2Exec new ServiceContactlmpl hot mcs anl gov 5224 resource setServiceContact ACTOS Task JOB_SUBMISSION gt2Exec 60 ls Set Hie serios CONAC HOT El Ob exccuiion lt a ServiceCont
2. gt lt i guirecullhy gt lt setvar name variablel value variable2 gt lt echo message variablel gt All three examples will print the same value blah The Scope of Variables The scope of variables is limited to the element inside which they appear unless they are shadowed in sub elements In such a case the scope of the shadowed variable will be limited to the element in which the 74 _ graph Canvas Java CoG Kit Graph Viewer X File View Help y 7 Y setvar Y setvar Y setvar F setvar Y setvar F setvar F setvar Y setvar F setvar Y Error g javaBean g Error details GSSException Expired credentials detected Location gridExecute Bs gt j gridExecute home mike woidrecovered woddflow templates xml line 28 Actions y javaBean 2 Oo Abort cal O Ignore 167 nodes 176 edges O Restart times B Options C Apply to all errors of this type O Apply to all errors for this element Figure 9 6 Error dialog variable was defined and any sub elements executed at runtime In technical words Karajan is a dynamically scoped language and uses shallow binding The following example illustrates this lt sequential gt lt define the variable var lt setvar name vat value one gt lt print its value on the console lt echo message var gt lt a container lt sequential
3. help Display usage 117 usage cn lt name gt commonname lt name gt service lt service gt host lt FODN gt dir lt dir_name gt prefix lt prefix gt nopw nodes nopassphrase verbose int eractive force Common name of the user Create certificate for a service Requires the host option and implies that the gene key will not be password protected ie imp Create certificate for a host named lt FODN gt Changes the directory the private key and request will be placed in By default user certificates are placed in home user glo certificates are placed in etc grid secur service certificates are place in etc grid security lt service gt Causes the generated files to be named lt prefix gt cert pem lt prefix gt key pem and lt prefix gt cert_request pem Create certificate without a passwd Don t clear the screen lt lt Not used gt gt Prompt user for each component of the DN lt lt Not implemented yet gt gt Overwrites preexisting certifictes 118 Bibliography 1 G von Laszewski and P Wagstrom Tools and Environments for Parallel and Distributed Computing ser Series on Parallel and Distributed Computing Wiley 2004 ch Gestalt of the Grid pp 149 187 Online Available http www mcs anl gov gregor papers vonLaszewski gestalt pdf 9 2 G von Laszewski and K Amin Grid Middleware Wiley 2004 ch Chapter 5 in Middleware for Commnica
4. Defines a token that can be used to guarantee that a set of tasks will be executed on the same resource Arguments name The name of the variable that should be set with the value of the token The token can then be used by the gridExecute and gridTransfer 0819 elements as a host attribute Performs a boolean and operation Arguments Nested arguments The terms Evaluates to the value supplied as argument Arguments value The argument Sets checkpointing parameters or forces the immediate creation of a checkpoint Arguments fileName The name of the file to which the checkpoint will be written interval Sets the interval at which regular checkpoints will be performed The interval is specified in seconds now If set to true causes the immediate creation of a checkpoint This is merely a debugging feature The recommended method is to set a regular interval for checkpointing Tests whether a file contains a certain text Arguments file The file to be searched value The value to be searched 95 9 8 6 default 9 8 7 echo 9 8 8 element 9 8 9 elementDef 9 8 10 else 9 8 11 elseif Typically used to define the default value for an argument in a template It sets the value of the specified variable 1f it is not already defined Arguments name The name of the variable to be defined value The value of the variable Echoes a message on the console Arguments message The message to be
5. Executes an arbitrary Grid job on a remote Grid resource 8 3 1 cog setup 8 3 2 grid authenticate 8 3 3 grid execute grid copy Transfers file between Grid resources This is a tentative list and is by no means final The initial prototype for GridAnt has the functionality for job submission and file transfer Other tasks are under development We release the current version as a technology preview in order to obtain feedback and to engage the community in its further development The cog setup task is a utility task that assists in the quick setup of the Java CoG Kit environment on the client machine It invokes the cog setup component that allows the users to specify several parameters including their user certificates and user keys The grid authenticate task is a utility task that invokes a gui allowing the users to instantiate their grid proxy certificates The grid execute task executes an arbitrary job on a Grid resource It requires the following input parameters specifies a mandatory argument provider Specifies the backend Grid provider for the submission of this job Available options are GT2 and GT3 server Specifies the location of the remote job execution service for the given provider xmlsecurity Specifies the XML security parameters only for GT3 provider Valid op tions are xmlsig and xmlenc for XML signature and XML encryption respec tively The default is XML signature delegation Specif
6. http www globus org research papers html 16 2 2 Globus Toolkit 2 2 1 Globus Toolkit Public License GTPL Globus Toolkit Public License GTPL Version 2 Globus Toolkit Public License Version 2 July 31 2003 Copyright 1999 2003 University of Chicago and The University of Southern California All rights reserved This software referred to as the Globus Toolkit software Software includes voluntary contributions made to the Globus Project collaboration Persons and entities that have made voluntary contributions are hereinafter referred to as Contributors This Globus Toolkit Public License is referred to herein as the GTPL For more information on the Globus Project please see http www globus org Permission is granted for the installation use reproduction modification display performance and redistribution of this Software with or without modification in source and binary forms Permission is granted for the installation use reproduction modification display performance and redistribution of user files manuals and training and demonstration slides Documentation distributed with or specifically designated as distributed under the GTPL Any exercise of rights under the GTPL is subject to the following conditions 1 Redistributions of this Software with or without modification must reproduce the GTPL in 1 the Software or 2 the Documentation or some other similar material which is provid
7. ing Grid implementation enforces its own security requirements therefore making it necessary to abstract a generalized security context Hence the security con text in cog core offers a common construct that can be extended by the different implementations of Grid to satisfy the corresponding backend requirements The task specification represents the actual attributes or parameters required for the ex ecution of the Grid centric task The generalized specification can be extended for common Grid tasks such as remote job execution file transfer and information query The service contact associated with a task symbolizes the Grid resource required to execute it Listing 7 2 Interface definition for Task public interface Task extends ExecutableObject public static final int JOB_SUBMISSION 1 public static final int FILE_TRANSFER 2 public static final int INFORMATION QUERY 3 public void setType int type public int getType public void setProvider String provider 38 public String getProvider public void setSpecification Specification specification public Specification getSpecification public void setSecurityContext SecurityContext security public SecurityContext getSecurityContext public void setServiceContact ServiceContact servicecontact public ServiceContact getServiceContact public void setStdOutput String output public String getStdOutput public void setStdError
8. object An object instance to invoke the method on The newJavaObject element can be used to create such an instance method The name of the method to invoke Nested arguments Arguments to pass to the method Concatenates lists items into a list Arguments Nested arguments Lists or items to be concatenated Concatenates lists items in reverse order Arguments Nested arguments Lists or items to be concatenated Evaluates to the size of the list argument Arguments list A list or an item In the later case the size will be 1 Performs a numeric comparison of the arguments Arguments valuel The first argument value2 The second argument 99 9 8 29 9 8 30 9 8 31 9 8 32 9 8 33 9 8 34 math ge math gt math le math lt math product math quotient Tests if the first argument is greater than or equal to the second argument Arguments valuel The first argument value2 The second argument Tests if the first argument is greater than the second argument Arguments valuel The first argument value2 The second argument Tests if the first argument is lower than or equal to the second argument Arguments valuel The first argument value2 The second argument Tests if the first argument is lower than the second argument Arguments valuel The first argument value2 The second argument Multiplies a set of numbers Arguments Nested arguments The valu
9. version gt3 3 0 2 type job submission url http host 8080 ogsa services base gram MasterForkManagedJobFactorypervice gt lt service 88 venson ais I 0 2 type file transfer url http host 8080 ogsa services base multirft MultiFileRFTFactoryService gt lt host gt lt grid gt SSH The SSH handler requires explicit pointers to the credentials used for authen tication It supports both username password which we do not recommend and public key authentication The following example shows how to use an SSH Core handler with Karajan lt define the task handler to be used lt taskHandler type SSH version ssh gt lt associate with the previously defined gt SE UN COE gt lt securityContex name ssh doe gt lt property name ssh username value johndoe gt lt property name ssh private key value home johndoe ssh identity gt lt property name ssh pass phrase value guessme gt gt lt ssh password could also be used instead of gt lt the ssh private key ssh pass phrase pair gt lt securityContext gt lt taskHandler gt lt li gt debio ans male mid lt grid name default gt lt host name hot mes anl gov cpus 2 gt lt service version ssh type job submission PE REO lt service versiome Ssla type file
10. 1 catch TaskSubmissionException tse logger error TaskSubmission Exception tse System exit 1 catch IllegalSpecException ispe logger error Specification Exception ispe System exit 1 catch InvalidServiceContactException isce logger error Service Contact Exception isce System exit 1 If it is required to monitor the status of the task desired in most interactive tasks then before submitting the task to a handler subscribe to the task for its status 53 changes task addStatusListener this If registered to listen to the status notification of the task implement the sta tusChanged function public void statusChanged StatusEvent event Status status event getStatus logger debug Status changed to status getStatusCode if status getStatusCode Status COMPLETED x Makes sense if SpesssseuredinectediatruenE spec setStdOutput null logger debug Output task getStdOutput System exit 1 y 7 4 2 Executing a Third Party File Transfer task Executing a file transfer is extremely simple with cog core To begin with create a Task with the appropriate attributes Listing 7 19 Create a Task object Create a new file transfer task named myTestTask x Task task new TaskImpl myTestTask Task FILE_TRANSFER Set the desired provider Default options
11. 2 Running 2 p 2 2 4 08 se mop ES eS E EE le ee 105 10 3 Using The Graph Editor u 2 22 88 e er 105 103 1 The Swink Target ar 2228 Kar pack 106 10 3 2 The HTML BEE oscars sas a pe 107 103 3 The PostScript Target Di is a Ge er 108 10 3 4 The Remote Target os 22 258 Br et 108 10 4 Graph file format lt i ss sso eepe e e e e eoh K a 109 SAPI e 6 2 ea O 110 10 6 Sealapitty A 112 11 Testing Framework 113 11 1 Stuc 2 ads a a en a ae eas 113 11 2 Using the Testing Framework o 113 11 2 1 Configuring the Driver Script 113 11 42 The Host List Format e 22 25 4 8b oad ss 114 12 Command Tools 115 13 grid cert request 116 0 1 Revisions e 04 05 2004 Added the GridAnt Chapter Removed the QoS and the portals section from the make process They need to be cleaned e 01 14 2003 Improved the Karajan and graph editor chapters e 07 18 2003 Added the QoS descriptions e 03 14 2003 Released the first version of the manual documenting the work flow 1 Preface This manual contains a number of high level modules of the Java CoG Kit that are not distributed as part of the Globus Toolkit version 2 3 or 4 We believe these components are valuable add on components to any Grid Toolkit The popularity of the Java CoG Kit has lead to the fact that 1t is now distributed in part with the Globus Toolkit Often users of the Globus Toolkit do not know that they use components contributed by the Java
12. 42 9 8 43 9 8 44 9 8 45 9 8 46 9 8 47 9 8 48 9 8 49 9 8 50 9 8 51 9 8 52 9 8 53 9 8 54 9 8 33 9 8 56 ISDOTeBIOTS eek 99 MEME nee ar pe a OR eS 99 myokelavaMethod ociosas 99 hstappend 2 ua e a ee 99 St PIEPER o ke ee aan nr 99 Mala a SEG ee Se en Bee 99 qmiathiequals 2 5 ori 884 ae Be eS 99 MAIER cs ek Rem eA a a A 100 PRUNE BE 52 icp Bs a a AG 100 MANNE 4 A a hE we PRR hae oe 100 MAURI o ms eR A a ee eg 100 cs o a waa an a AG 100 Mmathiquotient gt 2 ee ew ee eee 100 At TEMAMASE s s a pace a ee G 101 MASATE 2 A a a eh 101 WAS GU E i i a e id de EAS 101 inatliisubfracton s aa es A 101 WASG os en an rk 101 pewlavaObiect ooo 2 heb ea 101 noncCheckpomtable e es Pe kann ee 101 HO 6 4 as Sa ea re an ih 102 elo di 6 4 nt he Ee SESE ES 102 OF ie ER ae ee ee A Pew eR Ee aly ae 102 Parallel u 25 na a So a EAS 102 FOIE 2 246 bee E Bee a 102 AI 102 TESTATONEMOS p ee es eS e rr 102 SCNCUBIED 24 554 acht eG hs da dde a 103 SEINE fie adn db hea ee ewe ee Ge ale oad 103 SGCUTILVGCOMIEXE cps 5 a ee Se eR ee a 103 securityContextProperty 103 Seqienttal s cect Ree eae ee bale oad 103 SERVED uns seen nee 103 taskhHlandier 4 io ee sl a a 104 template u oa era eS 104 9837 templateDer 044 0 68a a be 104 o a ect ne ey SA ee a A ee ee ee I 104 O NMI A AE AA ARA a Be 104 A Wal erheben 104 10 Graph Editor 105 10 1 CONASUIME sc isps eee ee ETERS RR He ES 105 19
13. Each such entity is not to be taken literally In general such constructs are explained as they occur throughout the manual The use of such entities is shown in the example below lt machine name gt Here lt machine name gt is to be replaced with an actual machine name gt ping hot mcs anl gov Web links are proceeded by a meaningful name for the link An example is Java CoG Kit Website http www globus org cog Links to code source are proceeded by the repository tag An example is jglobus org globus gram Gram java 1 6 Manual Maintainer A number of people are currently maintaining the manual Part Section Name Preface 1 Gregor von Laszewski Introduction Gregor von Laszewski Licence Gregor von Laszewski Installation 4 Mike Hategan Setup 22 Mike Hategan Contributing 5 Mike Hategan Modules jglobus TBD Util TBD Certrequest TBD Resources TBD Common TBD Grapheditor 10 Mike Hategan Karajan 9 Mike Hategan Core 7 Kaizar Amin Portlet 22 Mike Hategan QoS 22 Rashid Al Ali Command Tools Certrequest 13 Gregor von Laszewski We invite you to contribute to the manual or the code see 1 1 1 7 Contributors Gregor von Laszewski Argonne National Laboratory University of Chicago Kaizar Amin University of North Texas ANL Mike Hategan University of Chicago ANL Shashank Shankar Illinois Institute of Technology ANL Vladimir Silva IBM Jean Claude Cote High Performance Computing National R
14. ID Delays the execution for a period of time or until a specific time Arguments delay The delay in milliseconds to wait until A string representing a date The format of the date is any format accepted by the java util Date class See also http java sun com j2se 1 4 2 docs api java util Date html 104 10 Graph Editor In order to compile the editor execute ant dist in the grapheditor directory A new directory named dist will be created The dist directory will contain the following subdirectories bin contains the launchers used to start programs from the command line etc contains configuration files none at the moment lib contains the jar files needed to run the viewer editor examples a few graph examples and a perl client that can interact with the viewer service 10 1 Configuring The dist etc directory contains the grapheditor properties file which can be used to customize certain aspects of the editor 10 2 Running To run the editor cd to the dist bin directory and execute grapheditor or grapheditor bat on windows The following command line options can be used s lt port gt Starts the editor in server mode listening for incoming connections on the specified port If no port is specified the default 9999 will be used h help Displays a list of options together with brief explanations 1 load lt file gt Loads lt file gt after starting up
15. Listeleigene gt 64 ee Ee 24 88 wa ELS RS 87 9 3 15 Miscellaneous Elements 87 Supported Handlers 2 2 02 oh 882 Re Ee RAS 88 Include Search Pah coso Sy ed 8 wa eee sin 89 Atchttectite 4 Haie 90 961 TheLoading Process a nes Reese et 90 9 62 The Execution Model aa 208 ae o 90 963 Task Scheduling 8n 28 93 Ai ua a re Re ne ut 93 9 7 1 Checkpoint Creation are it 93 9 7 2 Restoring from a Checkpoint 94 Quick Element Reference 2 o 95 93 1 allocatellost ascii a ai 95 RL HON E A eb ee eS 95 98 3 ANERE cos yk Ge A ee ee et 95 934 CRECKPOINE comica a a nit 95 2 83 CONAS ias ea 95 OSG ETE cue ia ek A RA 96 OR EED a AAA a a ee 96 2 3 Cm 2 25 5 bo aig Sew e dads 96 9 89 elemenmtDel 2 454405 bee dew eee be ae wee 96 OR MO EBE fcc cage A A Go Se Bae eA 96 BUE GIGI 5 3 i be eS ee E ea Ge 96 98 12 CQUI S 6 4 dee a She ee Gewese be ae od 97 98 13 EXCOMSISYA ce 4 625 2 0444 54 eh cae E eS 97 She TOR ad Sod ig Sethe eee beh ees 97 98 13 foreach e 42456 e ee Ge ale od 97 9 8 15 EENETRBEROLE es pc nr sauer 97 GAR A e eee A e Sa 98 D818 ENdERERUTE gt 4 oe ae e ae oa 98 9 3 19 Eng Transier as ss ins ser sehe 98 98 20 II eB Se i r 98 A Wee MAS OAR ee ae Ee eae 98 9 8 22 9 8 23 9 8 24 9 8 25 9 8 26 9 8 27 9 8 28 9 8 29 9 8 30 9 8 31 9 8 32 9 8 33 9 8 34 9 8 35 9 8 36 9 8 31 9 8 38 9 8 39 9 8 40 9 8 41 9 8
16. MultiRFT Grid service ssh showcasing the ssh functionality 36 ExecutableObject ServiceContact SecurityContext Dependency Specification JobSpecification FileTransferSpecification QuerySpecification TaskGraphHandler Figure 7 1 Core UML Class Diagram 1 The SSH JobSubmission and FileTransfer examples demonstrate the ability to use SSH and SSH credentials to perform the corresponding tasks misc demonstrating the combination of the gt2 gt3 and ssh platforms with exe cution dependencies The examples in misc package show how to create a directed acyclic graph DAG and visualize it using the grapheditor module It also shows how to create a hierarchical DAG and visualize it After successfully compiling the core module these examples can be executed from the launcher scripts available in modules core dist bin direc tory 7 3 Design One of the most important usage patterns in Grid computing is the execution of a Grid task An extension to this basic Grid execution pattern is a Grid workflow pattern that enable the user to submit a set of Grid tasks along with an execution dependency Therefore the initial design of cog core concentrates on providing the artifacts required to support these important usage patterns Other Grid patterns can be supported by extending the flexible cog core design based on community feedback Figure 7 1 shows the class di
17. THIS SOFTWARE OR DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES 8 The Globus Project may publish revised and or new versions of the GTPL from time to time Each version will be given a distinguishing version number Once Software or Documentation has been published under a particular version of the GTPL you may always continue to use it under the terms of that version You may also choose to use such Software or Documentation under the terms of any subsequent version of the GTPL published by the Globus Project No one other than the Globus Project has the right to modify the terms of the GTPL Globus Toolkit Public License 7 31 03 18 2 2 2 Globus Toolkit Contributor Liceense Grant of Licenses in Globus Toolkit Contributions July 2003 The undersigned licensor LICENSOR has delivered or caused or permitted to be delivered to The University of Chicago as Operator of Argonne National Laboratory and The University of South ern California collectively LICENSEE software and documentation collectively CONTRIBU TIONS created by LICENSOR or by LICENSOR s employees associates contractors or collabo rators Software SOFTWARE contributed by LICENSOR are source and binary software code such as schema bug fixes patches upgrades or other modifications or enhancements of the features functionality or performance of the Globus Toolkit software GLOBUS TOOLKIT Documentation DOCUMEN
18. editor service that the target will try to connect to 10 4 Graph file format The graphs are stored in a simple XML format The simplest graph can be specified as follows lt graph gt lt graph gt Adding nodes can be done using the node element lt graph gt lt node nodeid 1 name the first node gt lt node nodeid 2 name the second node gt lt graph gt Edges can be added using the nodeid s as references lt graph gt lt node nodeid 1 name the first node gt lt node nodeid 2 name the second node gt lt edge from 1 to 2 gt lt graph gt Hierarchical graphs can be created too In such graphs each node can itself contain other graphs lt graph gt lt node nodeid 1 name the first node gt lt node nodeid snl name subnode 1 gt lt node nodeid sn2 name subnode 2 gt lt node nodeid sn3 name subnode 3 gt lt edge from snl to sn2 gt lt edge from sn2 to sn3 gt lt edge from sn3 to snl gt lt node gt lt node nodeid 2 name the second node gt lt edge from 1 to 2 gt lt graph gt Properties are specified as XML attributes There are a few predefined properties that each node can have name String appears as the text in the label used to render the node iconfile String an absolute path to an image that wi
19. gt lt element3 gt lt parallel gt Mixed sequential parallel execution lt sequential gt lt elementl gt lt parallel gt lt element2 gt lt element3 gt lt parallel gt lt element4 gt lt sequential gt 76 Sequential execution with asynchronous ele ment lt sequential gt lt elementl gt lt element2 sync false gt E Iterators are used in Karajan to execute a sequence repetitively All iterators can have either a sequential behavior the default in which an iteration begins ex ecution only after the previous iteration has completed execution or a parallel behavior switched on using the parallel true attribute in which all iterations are executed in parallel The parallel behavior of an iterator does not apply to contained elements If a parallel iterator has two sub elements the elements will execute in sequential order This can be prevented by using an explicit parallel 8 45 container inside the iterator lt element3 gt lt element4 gt lt sequential gt Parallel execution with asynchronous ele ment lt parallel gt lt element gt lt element2 gt lt element3 sync false gt lt parallel gt 9 3 3 Iterators Two iterators exist for and foreach Both have a mandatory at tribute name which represents the name of the iteration variable e The for iterator is used for iterations acr
20. lt grid name default gt lt for name index from 1 to 20 gt lt host name lgOnf index pts uml mov cpus 1 gt lt service version AAA 0 type job submission url host 2119 jobmanager fork gt lt service version gt2 2 4 0 type file transfer tic esti Mf ios 211 lt host gt lt for gt lt grid gt Two types of Grid tasks are available remote execution and transfer For all tasks that require one or more host attributes the hosts may need to exist in the Grid definition such that the handlers can extract service contact information In some cases defaults may work e gridExecute can be used to submit a remote job The attributes are as follows host Specifies the host to which the job will be submitted The host must executable args stdin stdout stderr exist in the grid description such that the handler can extract the cor rect service information If the attribute is not specified it is up to the scheduler to pick a host for this job The executable to be run It must exist on the remote site If it does not it can be transfered beforehand using a transfer task The attributes to be passed to the executable If input redirection is desired this attribute can be used to specify a remote file that will be redirected to the process standard input Can be used to redirect the standard output of t
21. name of l ie Excecutable 1 local spec setLocalExecutable true spec setExecutable bin I1Is Seil for the arguments if any the executable x spec setArguments la x chiesenamessolssthesst lesswihichsssiervieis the input to the executable Sel as io De reires then Iii Mae dora le meca from the local machine 51 spec setLocalExecutable true caf spec setStdInput core testInput Set the name of the file to which the remote output must be stored in If the remote output needs to be redirected to the local machine then Specescunvedinectied tnuic If the remote output needs to be manipulated at the local machine rather than storing it in a lo y Uca SpeeszseHRedimecterdi truen E spec setStdOutput null The output is now available from task getOutput and can be used or displayed as desired spec setStdOutput core testOutput x Set the execution mode of the job spec setBatchJob true x Add additional attributes that are not provided by default These add on attributes will be considered by the handler only if it supports it spec setAttribute runCount 546 Assign this specification to the task task setSpecification spec Next assign the desired security credentials to the task This step assumes you have a valid uses certificate successfully obtained from appropr
22. of dependencies between tasks or the order of the execution of tasks It is up to the workflow engine to do so For the default scheduler once a set of tasks is queued in the scheduler there is no way to know anything about the order in which the execution of these tasks will begin nor about the order in which they will complete their execution Of course other schedulers for which such things are known can be written but the scheduler interface does not define explicit ways for enforcing execution order nor it is required by the workflow engine from the scheduler that such order be known or be specifiable Checkpointing is still an experimental feature in Karajan This section describes the basic workings of checkpointing in Karajan Checkpointing parameters can be adjusted by using the checkpoint element Checkpointing here refers to Karajan checkpointing Only the state of the workflow is saved in a checkpoint The state of grid tasks is not included in the checkpoint Imagine the following scenario in chronological order a Karajan specification creates certain files on a remote resource the execution is checkpointed and interrupted the files then are deleted and the execution state is restored from the checkpoint If the files are further needed and referenced in the specification an error will eventually occur Checkpointing works by dumping the specification and the state to a file The specification consists of the element tr
23. of remote tasks that the scheduler will allow at any given time showTaskList If set to true the scheduler will pop up a window providing a lists of tasks that are being executed and additional task and memory statis tics grid Specified as a nested typed argument through the grid element taskHandler Specified as a nested typed argument through the taskHandle defines the set of resources that will be used for scheduling r 9 8 55 element defines the type of Java CoG Kit Core task handler to be used by the scheduler If there are multiple task handlers for a scheduler they will all be used with a priority set by the order in which they were specified the highest priority going to the first The scheduler should also try to match the handlers against available resources and use a lower priority handler if resources are not available for a higher priority handler taskHandler 8 55 defines a type of Java CoG Kit Core task handler that 1s going to be used by the scheduler The attributes are type which selects the type of the handler for a list of supported handlers consult the Section 9 4 and version which is used by the scheduler to select the appropriate grid resources by matching it with the version attribute in the service 8 50 elements Multiple handlers can be specified In this case all of them will be used the highest priority going to the one listed first A securityContext 5 51 must also be specified
24. the han dler will not work Karajan can pass such settings to Core using generic attributes The following are default Core handlers together with examples that show their usage in Karajan GT2 This handler does not require any specific settings but the Java CoG Kit must be configured properly for the handler to work lt define the task handler to be used lt taskHandler type GT2 version gt2 2 4 0 gt lt associate with the previously defined gt lt l Seeurliy CONLE gt lt securityContext name gt2 gt lt taskHandler gt lt define a small grid lt grid name default gt lt host name cold mcs anl gov cpus 2 gt lt service version gt2 2 4 0 type job submission url host 2119 jobmanager fork gt lt service WERSIOM IA type file transfer Wile geitin Hf nose fee Vays lt host gt lt grid gt GT3 Similar to the GT2 handler the GT3 handler does not require any special parameters The following example shows how the GT3 handler can be used in Karajan lt define the task handler to be used lt taskHandler type GT3 version gt3 3 0 2 gt lt associate with the previously defined lt SEC CONTET lt securityContext name gt3 gt lt taskHandler gt Si detine smell girial gt lt grid name default gt lt host name mild mcs anl gov cpus 2 gt lt service
25. when arguments are equivalent The math sum element is such an example lt math sum gt lt argument value 1 gt lt argument value 2 gt lt argument value 3 gt lt math sum gt This does not necessary mean that changing the order of the arguments will not change the outcome of the execution of the element as it can easily be seen in the following example lt list append gt lt argument value 1 gt lt argument value 2 gt lt argument value 3 gt lt list append gt default arguments can be used for at most one of the arguments A default argument can be expressed as an unnamed nested argument No element can have both a default argument and multiple unnamed arguments The setvar has the value argument as a default argument The following code snippets lead to the same result lt setvar name var value 1 gt 79 lt setvar name var gt lt argument value 1 gt lt setvar gt typed default arguments are an exception to some of the above rules In some instances an element 9 3 6 Templates may expect multiple arguments each having a different type In such cases there is no need to name the arguments since their type will provide the required information that is needed to map the supplied arguments to the ones required Since types exist only internally in Karajan typed default arguments can only be used for internally defined eleme
26. 3 104 setvar 73 79 103 taskHandler 83 103 104 template 81 104 templateDef 80 104 then 78 80 104 UID 87 104 wait 87 104 License 15 bouncycastle 25 eryptix 25 Globus Toolkit 17 GPTL 17 Java CoG Kit 21 JCoGPL 21 junit 25 log4j 25 puretls 25 soaprmill 25 xerces 25 xml4j 25 Mailing List 11 120 Project Registration 15 Testing 113 Website 10 Workflow 69 Karajan 69 121
27. 4 2 and 7 4 3 We then create a TaskGraph and add a dependency between these ExecutableObjects Listing 7 26 Create a TaskGraph with a dependency TaskGraph htg new TaskGraphImpl Give a convenient name to the TaskGraph htg setName testGraph x Add the ExecutableObjects to the TaskGraph x htg add taskl htg add task2 htg add task3 htg add tg x Add dependencies between these ExecutableObjects Dependency is added as ASL gt LAS gt al gt 8 e This implies taskl is executed before task2 and task2 is executed before task3 and task3 is executed before TaskGraph tg htg addDependency taskl getldentity task2 getldentity htg addDependency task2 getldentity task3 getldentity htg addDependency task3 getldentity tg getldentity Next create a TaskGraphHandler and submit the task for execution Listing 7 27 Create a task graph handler TaskGraphHandler handler new TaskGraphHandlerImpl 58 try handler submit htg catch InvalidSecurityContextException ise logger error Security Exception ise System exit 1 catch TaskSubmissionException tse logger error TaskSubmission Exception tse System exit 1 catch IllegalSpecException ispe logger error Specification Exception ispe System exit 1 catch InvalidServiceContactException isce logger error Service Contact
28. CoG Kit We hope that you will find the components described in this manual help you making use of the Grid more easily The Java CoG Kit is a very open project and invites participation by others Thus we have started to involve the community more strongly into the development of the Java CoG Kit If you have components that you like to contribute to the Java CoG Kit please notify us If you like to participate in the development of the Java CoG Kit I recommend that you contact us through a simple e mail as described in Section 1 1 1 1 Participation Opportunities To participate please send a mail with your intend and abilities to Gregor mcs anl gov Please follow the simple subject mail syntax rule APPLICATION CV lt Firstname gt lt Lastname gt where CV is an abbreviation for community volunteer All mail not following this rule will be caught by a spam filter and automatically deleted We have a variety of open projects that could provide ideal opportunities to get engaged in furthering the development of Grid computing Some of these projects could also be given for credit as independent studies or lead to a Masters Thesis project If you decide to integrate them in your curriculum it is best to develop an agreement between the Java CoG Kit project your advisor and yourself We con ducted such activities with volunteers from Canada UK Switzerland and several local and remote students and professionals in the
29. ENDS NO WARRANTIES OF ANY KIND WHETHER EXPRESS OR IMPLIED CONCERN ING THE CONTRIBUTIONS LICENSOR MAKES NO EXPRESS OR IMPLIED WARRANTIES OF MERCHANTABILITY OF SATISFACTORY QUALITY OR FITNESS FOR A PARTICULAR PURPOSE OR USE LICENSOR MAKES NO REPRESENTATION THAT THE CONTRIBUTIONS WILL NOT INFRINGE ANY PATENT COPYRIGHT TRADE SECRET OR OTHER PROPRIETARY RIGHT LICENSOR SHALL HAVE NO LIABILITY WITH RESPECT TO ITS OBLIGATION UN DER THIS GRANT OR OTHERWISE FOR DIRECT INDIRECT SPECIAL INCIDENTAL CON SEQUENTIAL EXEMPLARY OR PUNITIVE DAMAGES OF ANY CHARACTER EVEN IF AD VISED OF THE POSSIBILITY OF SUCH DAMAGES 23 LICENSEE shall not be required to include the name of LICENSOR or any of its employees asso ciates contractors or collaborators in any copyright registration notice or license associated with the CONTRIBUTIONS LICENSOR hereby grants LICENSEE permission to post this Grant on a website associated with the JAVA COG KIT provided that LICENSOR is given the opportunity to review and approve in advance any such posting such approval not to be unreasonably withheld or delayed Except as expressly provided herein nothing in this Grant shall be construed as granting any right or license under any inventions patents copyrights trade secrets or any other intellectual property rights of LICENSOR LICENSOR By Title Date Address Java CoG Kit Contributor License Draft 24 2 4 Other Licences W
30. Error public void setCount int count public Integer getCount public void setBatchJob boolean bool public boolean isBatchJob 40 public void setRedirected boolean bool public boolean isRedirected public void setLocalExecutable boolean bool public boolean isLocalExecutable public void setAttribute String name String value public String getAttribute String name public Enumeration getAllAttributes The FileTransferSpecification provides the commonly used attributes for file trans fers between Grid resources We note once again that not all attributes are sup ported by every Grid implementation Listing 7 5 Interface definition for FileTransferSpecification public interface FileTransferSpecification extends Specification public void setSourceServer String server public String getSourceServer public void setDestinationServer String server public String getDestinationServer public void setSourceDirectory String directory public String getSourceDirectory public void setDestinationDirectory String directory public String getDestinationDirectory public void setSourceFile String file public String getSourceFile public void setDestinationFile String file public String getDestinationFile public void setSource String source public String getSource public void setDestination String destination public String getDestination public void setDirectoryT
31. Exception isce System exit 1 If it is required to monitor the status of the task graph desired in most interactive task graphs then before submitting the task graph to a handler subscribe to the task graph for its status changes htg addStatusListener this If registered to listen to the status notification of the task graph implement the statusChanged function public void statusChanged StatusEvent event Status status event getStatus logger debug Status changed to status getStatusCode if status getStatusCode Status COMPLETED status getStatusCode Status FAILED logger info Task Graph Done System exit 1 7 4 5 Writing a Custom TaskHandler To write a custom TaskHandler create a class say foo bar My Handler that imple ments the org globus cog core interfaces TaskHandler interface To successfully execute an ExecutableObject with this custom handler you need to associate this handler with a provider name say MyProvider Provide the mapping between the provider name and the class name using the cog properties file Instructions for adding an entry in the cog properties file is available in Section 8 2 3 Hence we add the following entry in the cog properties file 59 MyProvider foo bar MyHandler Now in order to use this handler with any ExecutableObject simply associate that ExecutableObject with the provider MyProvider
32. F head etc MANIFEST MF tail lib src The build files for each module have four parts build xml should not be modified at all unless absolutely necessary If there is a fea ture that you would like added to the build system please tell Mike hate ganO mcs anl gov dependencies xml project dependencies are stored here Please modify it to suit your needs An example is given in the modules template directory launchers xml launchers that you want created in the build process Use the example in modules template to see how to use it project properties module properties are specified here The module name must be the same 5 1 2 Libraries as the directory name of the module The last line in this file contains the library dependencies for this module If you don t add the jar files that your project requires there it will not build The format is a comma separated list of files We suggest using lt jar name gt so that licenses and other things belonging to a jar will also be copied Please read below about the libraries Libraries can be found in two places 1 cog lib 2 cog modules yourmodule lib The build system will automatically choose the library from either of the two di rectories If a library exists in both directories priority will be given to the library in the cog 1ib directory This may cause your module not to build Please talk to Gregor or Mike in this case Also please note that the l
33. It is thus prohibited to do so After the checkpoint manager has ensured that the overall state of the execution is in a consistent state it begins writing the specification events and list of currently executing elements to the checkpoint file Each event and running element has an associated stack which will also be serialized It is mandatory that all elements put only Java Beans on the stack otherwise variables on the stack will not be saved leading to an incomplete checkpoint 9 7 2 Restoring from a Checkpoint When invoked with a checkpoint file from the command line Karajan will auto matically detect the checkpoint and restore the state of the execution at the time the checkpoint was taken A checkpoint file is self contained and does not require the original specification The restoration process is done by first loading the specification from the check point file Afterwards pending events are deserialized and posted to the event dispatcher Elements that were executing at the time the checkpoint was taken are also sent a RESTART event using the associated stack that was saved during checkpointing This will effectively put the execution in the state it was at the time the checkpointing was done 94 9 8 Quick Element Reference 9 8 1 allocateHost 9 8 2 and 9 8 3 argument 9 8 4 checkpoint 9 8 5 contains Default arguments are indicated by a plus sign Optional arguments are indi cated by an asterisk
34. Java CoG Kit User Manual Version 4 0 pre alpha MCS Technical Memorandum ANL MCS TM 259 Revision Apr 7 2004 The Java CoG Kit Team Argonne National Laboratory Mathematics and Computer Science Division 9700 S Cass Ave Argonne IL 60439 Corresponding Editor 630 252 0472 gregor mcs anl gov Location of Manual http www globus org cog manual cog2 pdaf Be kind to your environment and do not print this frequently changing manual c Argonne National Laboratory All rights reserved July 20 2004 Contents O1 REVISIONS a ne ee A ne ee Preface 1 1 Participation Opportunities 12 Gmds cis cis rr RRA 13 dntended Audience pa i ae eS a ra is IA RSSOUrees er Ee ae eos 1 4 1 Project Website a4 sda ee eek ee 14 2 Bup Reporting cpc eee eea ee we a 143 Maling Lists 25 4625 e840 ee 1 4 4 Sourcecode Repository 15 Abowtithe Manual 4 224 s ace Wer ee ed a i 1 5 1 Conventions 1 6 Manual Maintainer 20 0000 1 7 COMICIOS cn jek SH here ne ee 1 8 Administrative Contact 444 26 era eee es 1 9 Acknowledgments c pesate 2 bee ERR Oe ES License 2 1 Project Registtation o rs A ee a 22 Globus Toolkit coso 342 02 2 rs RO eee eS 2 2 1 Globus Toolkit Public License GTPL 2 2 2 Globus Toolkit Contributor Liceense 23 JAVACOG Kit cocaina a rd nennen 2 3 1 Java CoG Kit Public License JCOGPL 2 3 2 Java CoG Kit Contributor Lice
35. LAR PURPOSE OR USE ARE DISCLAIMED THE COPYRIGHT HOLDERS AND CONTRIBUTORS MAKE NO REPRESENTATION THAT THE SOFTWARE DOCUMENTATION MODIFICATIONS ENHANCEMENTS OR DERIVATIVE WORKS THEREOF WILL NOT INFRINGE ANY PATENT COPYRIGHT TRADEMARK TRADE SECRET OR OTHER PROPRIETARY RIGHT 21 7 LIMITATION OF LIABILITY THE COPYRIGHT HOLDERS AND CONTRIBUTORS SHALL HAVE NO LIABILITY TO LICENSEE OR OTHER PERSONS FOR DIRECT INDIRECT SPECIAL INCIDENTAL CONSEQUENTIAL EXEMPLARY OR PUNITIVE DAMAGES OF ANY CHARACTER INCLUDING WITHOUT LIMITATION PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOW EVER CAUSED AND ON ANY THEORY OF CONTRACT WARRANTY TORT INCLUD ING NEGLIGENCE PRODUCT LIABILITY OR OTHERWISE ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE OR DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES 8 The Java CoG Kit Project may publish revised and or new versions of the JCOGPL from time to time Each version will be given a distinguishing version number Once Software or Documen tation has been published under a particular version of the JCOGPL you may always continue to use it under the terms of that version You may also choose to use such Software or Doc umentation under the terms of any subsequent version of the JCOGPL published by the Java CoG Project No one other than the Java CoG Project has the right to modify the terms of the JCoGPL Java CoG Kit Public Lic
36. MOEMADON scr Se BARE o ri A A e ee 26 26 26 26 26 27 27 27 27 27 28 28 28 28 29 29 29 29 30 31 31 31 31 32 32 32 32 32 32 32 32 32 32 gt 24 Natables ooo sia dd be 3 25 Instance Variables escusa ASS mio OSL 2 4 de too e re ra rl A o cee bh Ow a ee ee ee DIS TEHE oa oe A Oe RP OA Ae eR ee RA 3 33 Intemationalization ss 2 5 2 48 ea RA 3 34 Library Reuse ociosos II o 6 4 2 8 we OR A Re BR 6 Modules Ol MO u Ge kes be ee eee eB SE ee owe ee e AA bok Ce ee a BAe Ene Se ORS 7 Core Vl o e eccere ee 4 Ks sa aan rer a S 72 Instalateh oscar andre Tal Download 24 ass et SES T Compile s a a a Ee 123 OORIPUTAUION s s e e n eka aoe A ee RO ee iot kat PMA IES Se i eS Se dg Ss ee SBS Ta DES coa a EE EAR ee wa od 13 1 BxertlableObiect oe pond wow ee eR a Woe MOSK hor eae eee ed dl eee oe 1 33 Specificajon coria a aes 134 VaskGiraph 055 lt lt ss Ran an eo DAUS a ee CR ee Ge eS IO Bandes rest esios ee a a hae 13 1 MORESQUICS cosido RE RR rar ee eS 1 38 FXRECIMONRESQUICE oo oepa E a kopa 4 39 FileRessures u ua a an eh Ta Programmers Guide pocepao prie tki ni aS 7 4 1 Executing a Remote Job Execution Task 7 4 2 Executing a Third Party File Transfertask 7 4 3 Executing a Simple TaskGraph DAG 7 4 4 Executing a Hierarchical TaskGraph 34 34 34 7 4 5 Writing a Custom TaskHandler 7 4 6 Executing Tasks on an Executi
37. String error public String getStdError public void setAttribute String name Object value public Object getAttribute String name public void addStatusListener StatusListener listener public void removeStatusListener StatusListener listener public void addOutputListener OutputListener listener public void removeOutputListener OutputListener listener public void fromXML String task public String toXML public void fromString String task public String toString public boolean isUnsubmitted public boolean isActive public boolean isCompleted public boolean isSuspended public boolean isFailed public boolean isCanceled public Calendar getSubmittedTime public Calendar getCompletedTime 7 3 3 Specification Every Grid Task has an associated Specification that dictates the objective of the task and the environment required to achieve the objective The TaskHandler man ages the tasks based on the parameters specified in the task specification 39 Listing 7 3 Interface definition for Specification public interface Specification public static final int JOB_SUBMISSION 1 public static final int FILE TRANSFER 2 public static final int INFORMATION QUERY 3 public void setType int type public int getType public void setSpecification String specification public String getSpecification A task specification is a generalized concept and ca
38. T3 org globus cog core impl gt3 SSH org globus cog core impl ssh SCHEMA_LOCATION lt COG_HOME gt modules core schema GT2 GT3 and SSH are default providers SCHEMA LOCATION indicates the directory location of the schemas required by GT3 and lt COG_HOME gt points to the location where the CoG is installed Hence an example cog core properties file would be similar to Java CoG Kit Core module GT2 org globus cog core impl gt2 GT3 org globus cog core impl gt3 SSH org globus cog core impl ssh SCHEMA_LOCATION home user name cog core modules core schema 7 2 4 Examples Several examples that demonstrate the ease of use and functionality of the Java CoG Kit Core are provided These examples are available in the modules core src org globus cog core examples directory The examples are further divided into the following packages gt2 showcasing the gt2 functionality 1 The GT2 JobSubmission example demonstrates the ability to submit to a PBS batch queue 2 The GT2 FileTransfer example demonstrates the ability to perform a third party file transfer using Grid FTP gt3 showcasing the gt3 functionality 1 The GT3 JobSubmission example demonstrates the ability to submit to a MasterForkManagedJobFactory service It can also be used for PBS managed factory services 2 The GT3 FileTransfer includes examples for third party transfers for single as well as multiple files It uses the
39. TATION contributed by LICENSOR are print and digital media that describe and ex plain such as readme files white papers overviews and tutorials and are useful with the GLOBUS TOOLKIT Acknowledging receipt of LICENSEE s services in distributing the GLOBUS TOOLKIT to a com munity of end users and developers including LICENSOR and LICENSEE s intention to continue such distribution LICENSOR hereby grants to LICENSEE the following licenses collectively with all limitations included herein referred to as this Grant a a non exclusive worldwide royalty free perpetual transferable copyright license to install use re produce modify display perform and prepare derivative works of SOFTWARE to incorporate SOFT WARE in whole or in part and derivative works thereof into the GLOBUS TOOLKIT or into any other computer software and to sublicense copyrights in and distribute SOFTWARE and derivative works thereof in source and binary forms b subject to termination provisions below and to the extent that patent claims licensable by LICENSOR are necessarily infringed by the use or sale of SOFTWARE alone or when combined with the GLOBUS TOOLKIT a non exclusive worldwide royalty free license and right to sublicense under such patent claims with the rights for LICENSEE and its sublicensees to make use sell offer to sell import and otherwise transfer the SOFTWARE in source code and object code form alone and incorporated int
40. US Doe to the nature of volunteering these applications are usually less formal than real applications but we must know your affiliation your address and citizenship To volunteer you ought to be committed It is of no help to us if you volunteer one week and than you drop the project in the next week If you apply for community volunteer positions make sure you provide us with evidence that you can conduct the project you or we suggest Community volunteer projects are a good start for a paid internship or job opportu nities Other paid opportunities for undergraduate and graduate appointments are updated regularly on the Web page Often it is a good idea to have your advisor 1 2 Grids directly talk to us and recommend you over the phone or in one of the many meet ings we participate in Paid assistantships are in general restricted to US citizens and permanent residents All contributors have to submit a contributor license Grids are an important development in the discipline of computer science and en gineering Rapid progress is being made on several levels including the definition of the terminology the design of an architecture and framework the application in the scientific problem solving process and the creation of physical instantiations of Grids on a production level A small overview about the Grid can be found in a draft paper entitled Gestalt of the Grid 1 Article http www mcs anl gov gregor bib papers
41. act gt3Exec new ServiceContactImpl http 127 0 0 1 80 serviceURL resource setServiceContact MEI Task JOB_SUBMISSION gt3Exec Es SG me SERVICES COMCEL or GA Wille ransior cans cn De awil since a Server dle not require on third party transfers ServiceContact gt2Transfer new ServiceContactImpl null resource setServiceContact AGI Task FILE_TRANSFER gt2Transfer x Similarly set service contacts for GT3 file transfer or any other service as required We note that it is not necessary to register service contacts and security contexts for every provider However attempts to execute tasks for unregistered providers will result in errors Next step is to prepare tasks task graphs to be executed Instructions for creating job submission and file transfer tasks and task graphs are available in the previous sections 7 4 1 7 4 2 7 4 4 It is not necessary to associate any security context and service contact with these tasks These values will be automatically assigned by the ExecutionResource based on the provider attribute of the task Assuming we create two tasks namely task and task2 we can execute them by submitting them to the execution resource Listing 7 29 Submit tasks to the execution resource resource submit taskl resource submit task2 Tf it is required to monitor the status of the tasks task graphs submitted to the ex ecution resource then before submitting the task to th
42. after being loaded instructs the loader to parse the included file the contents of which is in lined in the current element tree There are two important notions to remember in Karajan One is the execution element or flow element which as outlined in the previous subsection is con structed from XML elements in the specification The second one is the event Events are used either to notify elements about the status of the execution of other elements or to instruct elements to perform certain actions such as start or restart execution Events also encapsulate the state of the overall execution through a variable stack The stack contains the complete run time state for a specific thread of execution There should be no deterministic difference in the execution of two different instances of the same type of element with the same attribute values that receive equivalent events with identical states Elements are static as far as the execution is concerned Their internal state may change during the execution but the execution state must not be influenced by the internal state of the elements They react to events and can use the stack passed through the events to manipulate the state of the execution Each element that 1s being executed can add a frame to the variable stack The frame can be used to store variables that can represent the state of the element These variables are also accessible to contained elements When the execution of a
43. agram of cog core A detailed listing of the attributes and functions for each class has been omitted for simplicity In the rest of this section we describe the important entities designed and their semantics as a part of the offered functionality 37 7 3 1 ExecutableObject An ExecutableObject provides a high level abstraction for artifacts that can be executed on the Grid It can be specialized as a Grid Task or a TaskGraph An ExecutableObject in cog core has a unique identity and an execution status Listing 7 1 Interface definition for ExecutableObject public interface ExecutableObject public static final int TASK 1 public static final int TASKGRAPH 2 public void setName String name public String getName public void setldentity Identity id public Identity getldentity public int getObjectType public void setStatus Status status public void setStatus int status public Status getStatus 7 3 2 Task A Task is the atomic unit of execution in cog core It represents a generic Grid functionality including remote job execution file transfer request or information query It extends the ExecutableObject hence it has a unique identity and execu tion status It also has a security context a specification and a service contact The task identity helps in uniquely representing the task across the Grid The secu rity context represents the abstract security credentials of the task Every underly
44. ame must be passed by the method argument Nested arguments can be used to indicate arguments to be passed to the method e executeJava will synchronously execute a Java program The main Class attribute can be used to specify the fully qualified class name that contains the main method The class must be present in the classpath of the instance of the Java Virtual Machine that is executing the specification e math sum adds a set of numbers specified as unnamed arguments e math product multiplies a set of numbers specified as unnamed ar guments e math subtraction 835 subtracts two numbers specified by the from and value Effectively calculates from value e math quotient divides dividend by divisor and returns the quotient e math remainder computes the remainder of the division between dividend and divisor e math square computes the square of the default value argument e math sqrt computes the square root of the default value argument e math equals 2 tests if two numbers are equal e math gt tests if value is greater than value2 e math lt tests if valuel is less than value2 e math ge 829 tests if value is greater than or equal to value2 e math le tests if valuel is less than or equal to value2 e and evaluates to the boolean and of the nested unnamed arguments e or calculates the boolean or value of the n
45. are GIA GISAC S El task setProvider GT2 Then create a FileSpecification for the task and set the appropriate attributes as per the task requirements Listing 7 20 Create a task specification x Create a new FileSpecification x FileSpecification spec new FileSpecificationImpl Set the source and destination files x spec setSource gsiftp domain 2811 home filename spec setDestination gsiftp domain 2811 home filename Es Mie te se a cmd mary we Thiele e spec setThirdParyt true x Assign this specification to the task x task setSpecification spec 54 Next assign the desired security credentials to the task This step assumes you have a valid uses certificate successfully obtained from appropriate certificate au thority Listing 7 21 Create security credentials x Since the provider is GT2 create a GlobusSecurityContext If a non globus security context is required then use the SecurityContextImpl class and set the credentials as required by the handler GlobusSecurityContextImpl securityContext new GlobusSecurityContextImpl x Assign the default credentials available as a valid proxy certificate whose location is specified in the COL POPE Lele Smet ile spe sente HOME globus directory To assign non default credentials create a GSSCredential and pass this GSSCredential as the argument instead of null securityContext setC
46. as a nested element security Context 550 is used to define a security context passwords pri vate public keys proxy location etc for the task handler The mandatory attribute is the name attribute A series of nested elements can exist for the securityContext element These nested elements are handler specific and are described in Section 9 4 grid 7 encapsulates a set of resources that will be used by the scheduler Accepts an optional name attribute A series of nested host 2 arguments can be specified host 9 50 designates a single contact point a remote host The mandatory name attribute denotes the hostname of the remote contact A cpus attribute 83 Grid Tasks allows the specification of the number of CPUs the host has This infor mation may be used for scheduling purposes Each host can have multiple nested service 850 elements e service defines a host service The version attribute allows the defini tion of a logical handle that can be used to group multiple services based on technology version The type attribute specifies the service type The current possible values are job submission and file transfer The exact details of the service are expressed in the form of a URL The format and details of the service URLs differ from handler to handler Section 9 4 provides details of all supported handlers and their details The following example illustrates the use of the above elements
47. ask implement the sta tusChanged function public void statusChanged StatusEvent event Status status event getStatus logger debug Status changed to status getStatusCode if status getStatusCode Status COMPLETED status getStatusCode Status FAILED logger info Task Done System exit 1 I 7 4 3 Executing a Simple TaskGraph DAG To create a TaskGraph we assume that we have created three tasks taskl task2 and task3 Instructions for creating job submission and file transfer tasks are avail able in the previous sections 7 4 1 and 7 4 2 We then create a TaskGraph and add a dependency between these tasks Listing 7 24 Create a TaskGraph with a dependency TaskGraph tg new TaskGraphlmpl x Give a convenient name to the TaskGraph tg setName testGraph x Add the tasks to the TaskGraph x tg add taskl tg add task2 56 tg add task3 x Add dependencies between these tasks Dependency is added as AS gt task gt slo This implies taskl is executed before task2 ANG asi iS EXECU e dber oreta SKE a tg addDependency taskl getldentity task2 getldentity tg addDependency task2 getIdentity task3 getIdentity Next create a TaskGraphHandler and submit the task for execution Listing 7 25 Create a task graph handler TaskGraphHandler handler new TaskGraphHandlerImpl try i handler
48. at the compiled module depends on will also be compiled and included in the same directory 4 7 Using the Java CoG Kit version 4 The following is the basic layout for the binary distribution directories whether obtained by downloading the precompiled packages or by compiling the sources bin etc lib The bin directory contains launchers that can be used to start a particular appli cation in the Java CoG Kit The etc directory contains configuration files needed by various parts of the Java CoG Kit The lib directory contains the jar files that belong to the Java CoG Kit together with the libraries required to run various parts of the Java CoG Kit To start a particular CoG application go to the bin directory and choose its respec tive launcher IMPORTANT The Java CoG Kit version 2 automatically detects the COG_INSTALL_PATH If you have the variable set to a specific directory pointing to an older version of the Java CoG Kit unexpected behavior may result Please unset the COG_INSTALL_PATH variable before running any of the applications or set it to either cog dist cog lt version gt or cog modules lt modulename gt dist lt modulename gt lt moduleversion gt 4 8 Downloading JGlobus JGlobus the Globus Toolkit v2 x client API though included as a precompiled version in the Java CoG Kit is also available as a separate download The cur rent JGlobus source code can also be fetched from the source repository by us ing t
49. cumentation and to sublicense copyrights in and distribute DOCUMENTATION and derivative works thereof If LICENSEE or any subsequent transferee each referred to as a RECIPIENT institutes patent litigation against any entity including a cross claim or counterclaim in a lawsuit alleging that the SOFTWARE or the JAVA COG KIT distributed pursuant to this Grant or the Java CoG KitPublic License infringe such RECIPIENT s patent s then such RECIPIENT s rights granted under the patent license in paragraph b above shall terminate as of the date such litigation is filed The foregoing licenses shall be effective so long as the terms under which LICENSEE sublicenses copyrights in and distributes the CONTRIBUTIONS effectively a include compatible patent license termination language as that set forth above and b disclaim liability on behalf of LICENSOR for all damages including direct indirect special incidental exemplary and punitive damages in sub stantially the same form as that included herein and shall apply to any and all CONTRIBUTIONS specifically designated now or in the future as within the scope of this Grant by LICENSOR s autho rized representative LICENSOR represents that to its knowledge it has or has obtained any and all required permissions and authority to make this Grant Except as provided in the preceding sentence LICENSOR PROVIDES THE CONTRIBUTIONS ON AN AS IS BASIS AND MAKES NO REPRESENTATIONS AND EXT
50. curity parameters only for GT3 provider Valid op tions are xmlsig and xmlenc for XML signature and XML encryption respec tively The default is XML signature 66 delegation Specifies the parameters for credential delegation for GSI security Valid options are full partial and none for full delegation partial delegation and no delegation respectively The default is no delegation source Specifies the url of the source file to be copied destination Specifies the url of the destination file parallelStreams Specifies the number of parallel TCP streams desired for the file transfer Default is 1 tcpbuffer Specifies the TCP buffer size desired for the file transfer Default is 16000 blockSize Specifies the transfer block size desired for the file transfer Default is 16000 thirdparty A boolean flag indicating if this is a third party file transfer Default is true dcau A boolean flag indicating if data channel authentication is enabled Default is true For example to transfer from a GridFTP server hot anl gov to another GridFTP cold anl gov using the GT reliable file transfer service at http rft anl gov 8080 RFTFactoryService lt gridCopy provider GT3 server http rft anl gov 8080 RFTFactoryService xmlsecurity xmlenc delegation full source gsiftp hot anl gov home amin from txt destination gsiftp cold anl gov home amin to txt parallelStreams 3 te
51. d against the version labels of the defined task handler s Used as a nested element of taskHandler 855 to define a security context for the handler Arguments type Indicates the type of the security context For details consult Section 9 4 securityContextProperty sequential setvar Defines a property for a security context Arguments name The name of the property value The value of the property Executes the contained elements in sequential order Sets the value of a variable Arguments name The name of the variable value The value of the variable 103 9 8 55 9 8 56 9 8 57 9 8 58 9 8 59 9 8 60 taskHandler template templateDef then UID wait Defines a task handler that can be used by the scheduler to execute tasks Arguments type The type of the handler Valid types are described in Section 9 4 version A label used to match service definitions against handlers Invoked a template that was previously defined using templateDef Arguments name The name of the template to be invoked lt varies gt Arguments to be passed to the template Defines a template Arguments name The name of the template to be defined Encapsulates a series of elements that will be executed if the condition specified prior to the then element evaluates to true Further details about conditional execution are available in Section 9 3 4 Generates a unique numeric
52. d to herein as the JCoGPL For more information on the Java CoG Kit please see http www cogkits org Permission is granted for the installation use reproduction modification display performance and redistribution of this Software with or without modification in source and binary forms Permission is granted for the installation use reproduction modification display performance and redistribution of user files manuals and training and demonstration slides Documentation distributed with or specifically designated as distributed under the JCoGKit Any exercise of rights under the JCoGKit is subject to the following conditions 1 Redistributions of this Software with or without modification must reproduce the JCoGPL in 1 the Software or 2 the Documentation or some other similar material which is provided with the Software if any 2 The Documentation alone or if included with a redistribution of the Software must include the following notice This product includes material developed by the Globus Project http www globus org and the Java CoG Kit http www cogkits org 3 Alternatively if that is where third party acknowledgments normally appear this acknowledg ment must be reproduced in the Software itself 4 Globus Toolkit Java CoG Kit and Globus Project are trademarks of the University of Chicago Any trademarks of the University of Chicago may not be used to endorse or promote software o
53. distinct node icon a separate image will be generated but identical icons will not result in multiple images Also a certain amount of optimization is involved in the generation of edges arrows An exponential scale is used to snap the dimensions of the generated arrow images such that arrows having dimensions that differ by a small percent ratio will be represented by the same image However this image will be further scaled by the JavaScript code in the html file as needed This may result in aliasing and distortion of the rendered page but it is a required compromise The available options for the HTML target are as follows html outputdir The output directory where the html source and images will be generated html graphview layoutengine The layout engine to be used when rendering a graph The value is a fully qualified Java class name The predefined layout engines that the graph edi tor provides are located in the org globus cog gui grapheditor canvas views layouts package Available layout class names are as follows ExtendedSpringLayout A spring layout that does some initial heuristic layouting to reduce the overall layouting time This layout will also skip the springing part for large graphs since the spring layout is O n 107 nocet LevelLayout RadialLayout NonOverlappingRadialLayout Flow Layout 10 3 3 The PostScript Target nodes editable O sub nodes Properties Edit E Expandeel E I
54. e variable2 gt lt 60 ie CONC O C carito de lt readFile name tmp exitcode gt lt setvar gt Variable Expansion Variables can be expanded inside element attributes by en closing them inside curly brackets Nested expansion is also possible but must be used with care Examples more about functions in Section 73 _ graph Canvas Java CoG Kit Graph Viewer X File View Help o f gt Start Graph View v Y setvar y setvar echo echo editable O sub nodes Properties er Edit ee Remove Breakpoint setvar Resume execution O Expanded O Interactive O Keep aspect ratio O Show number of sub nodes 167 nodes 176 edges Figure 9 5 Resuming execution 1 lt setvar name variablel value blah gt lt echo message variablel variablel gt lt variable2 is not defined the value will not be lt expanded lt setvar name variablel value variable2 gt lt setvar name variable2 value blah gt at this point both variablel and variable 2 are defined the first expansion will evaluate variablel to variable the second expansion will evaluate variable to blah lt echo message variablel gt 3 lt setvar name variable2 value blah gt lt variable2 is now defined gt lt variablel will be assigned the value of blah
55. e appropriate handlers for these are avail able For cog core to support Unicore all one needs to do is to add a Unicore handler The TaskHandler provides a simple interface to handle a generic Grid task submit ted to it It is capable of categorizing the tasks and providing the appropriate func tionality for it For example the task handler will handle a remote job execution differently than a file transfer request Cog core does not impose any restrictions on the implementation of the task handler as long as its working is transparent to the end user Listing 7 9 Interface definition for TaskHandler public interface TaskHandler public static final int GENERIC 1 pUbIiG Static iiinal wmt CW 2s public static final int GT3 3 public void setType int type public int getType public void submit Task task throws IllegalSpecException InvalidSecurityContextException InvalidServiceContactException TaskSubmissionException public void suspend Task task throws InvalidSecurityContextException TaskSubmissionException public void resume Task task throws InvalidSecurityContextException TaskSubmissionException public void cancel Task task throws InvalidSecurityContextException TaskSubmissionException public void remove Task task throws ActiveTaskException public Task getAllTasks public Enumeration getActiveTasks public Enumeration getFailedTasks public Enumeration getCompletedTask
56. e distribute a number of other libraries with the Java CoG Kit These libraries come with their own licences We strongly encourage you to inspect these licenses The can be found in the lib directories of the Java CoG Kit 2 4 1 jglobus The jglobus lib directory contains the following licences jglobus jglobus jglobus jglobus jglobus 2 4 2 ogce bouncycastle LICENSE eryptix LICENSE log4j LICENSE junit LICENSE puretls LICENSE The ogce lib directory contains the following licences ogce ogce ogce 2 4 3 Others 2 5 GNU Public Licence soaprmil 1 LICENSE xerces LICENSE xml4j LICENSE Although we prefer the development of code that is non GPL we do have the ability to distribute components under the GNU license or are dependent on code developed with the GNU License For thie purpose we created a seperate CVS archive in which we maintain the GPL based code We will not distribute any GPL based Java CoG Kit code in binary format It must be downloaded compiled and installed seperatly We will include more details to this issue in future One of the Java CoG Kit Codes that will be included is the availability of a GPL based Grid shell At this time we have not yet made this code available as part of the Java CoG Kit 1 Other licenses need to be adde here 25 3 Introduction 3 1 Overview 3 2 History 3 2 1 Metacomputing 3 2 2 CoG Kits directions developed OO develop
57. e resource subscribe to the task graph for its status changes task addStatusListener this If registered to listen to the status notification of the task implement the sta tusChanged function public void statusChanged StatusEvent event ExecutableObject eo event getSource logger debug eo getName Status status event getStatus logger debug 61 62 8 GridAnt 8 1 Introduction This chapter focuses on a client side workflow management system that can or chestrate simple task dependencies It gives an overview of process workflows and workflow engines It further describes the applicability of a client side work flow system for Grid technologies and introduces the functionality of the GridAnt workflow system It provides detailed instructions for the user to install the Gri dAnt system and other dependent packages An introductory set of examples is discussed that helps the end user to understand the working of the GridAnt sys tem Significant research has been conducted in recent years to automate complex busi ness tasks using sophisticated workflow management tools Such tools are ex tremely useful in expressing complicated business activities as a set of independent work units and orchestrating a series of dependencies across these units In other words a workflow management system helps in combining a set of specialized tasks by expressing intricate dependencies between these tasks and exp
58. eceiving the END element and after cleanup is done EXECUTION_FAILED generated by an element when the execution failed The frame created by the element should be popped from the stack before the event is sent EXECUTION RESTARTED generated after receiving a restart event An example of the execution model for a sequential and a parallel container can be seen in Figure 9 8 and Figure 9 9 respectively Workflow Container Event Dispatcher Child Elements T lt sequential gt l M lt element1 gt E lt element2 gt sequential lt sequental gt Figure 9 8 Execution of sequential elements Workflow Container Event Dispatcher Child Elements lt parallel gt 7 lt element1 gt lt element2 gt parallel E lt parallel gt element2 Figure 9 9 Execution of parallel elements For space and readability considerations the EXECUTION_COMPLETED event type was shortened to COMPLETED in the images 92 9 6 3 Task Scheduling 9 7 Checkpointing 9 7 1 Checkpoint Creation Task scheduling on Grid resources is done by using a scheduler that in turn uses the Java CoG Kit Core Grid abstraction layer The gridExecute 819 and grid Transfer 1 elements submit the requests for execution to the scheduler which enqueues these requests and executes them as resources become available It is up the the scheduler to manage both local and remote resources in order to ensure that these resources are not
59. echoed lt inline text gt Can be used instead of the message attribute for larger chunks of text Used to specify a user defined element Consult Section 9 3 7 Arguments name The name of the element that is being defined arguments A list of named arguments that the element accepts vargs Set to true if the element accepts variable unnamed arguments These will be available through the vargs variable Defines a new element using a Java class Arguments className The fully qualified Java class name of the element nodeType The XML element name to be defined Encapsulates a set of elements that will be executed if no other condition evaluates to true inside an if 9 element Details about conditional execution can be found in Section 9 3 4 Encapsulates a series of elements that will be executed if the condition specified prior to the elseif element evaluates to true Further details about conditional execution are available in Section 9 3 4 96 9 8 12 equals 9 8 13 executeJava 9 8 14 for 9 8 15 foreach 9 8 16 generateError Performs an object comparison on two objects It can for string comparisons in particular Arguments valuel The first argument value2 The second argument Executes a Java application in a separate thread The element completes execution when the application completes execution Arguments mainClass The fully qualified name of the class that contains the main method Ne
60. ed with the Software if any 2 The Documentation alone or if included with a redistribution of the Software must include the following notice This product includes material developed by the Globus Project http www globus org 3 Alternatively if that is where third party acknowledgments normally appear this acknowledg ment must be reproduced in the Software itself 4 Globus Toolkit and Globus Project are trademarks of the University of Chicago Any trade marks of the University of Chicago or the University of Southern California may not be used to endorse or promote software or products derived therefrom and except as expressly pro vided herein may not be affixed to modified redistributions of this Software or Documentation except with prior written approval obtainable at the discretion of the trademark owner from info globus org 5 To the extent that patent claims licensable by the University of Southern California and or by the University of Chicago as Operator of Argonne National Laboratory are necessarily in fringed by the use or sale of the Software you and your transferees are granted a non exclusive worldwide royalty free license under such patent claims with the rights to make use sell of fer to sell import and otherwise transfer the Software in source code and object code form This patent license shall not apply to Documentation or to any other software combinations which include the Software No ha
61. ed API based version no success too dificult to use Globus API protocol services Web services without modification Inforgram first futire service 26 4 Installation 4 1 Download 4 2 CVS Release Tags The current releases of jglobus ogce and cog have the common v 4 0 a tag In order to use the Java CoG Kit the Java Runtime Environment version 1 4 avail able from the Java Web site is required Additionally if you plan to compile the Java CoG Kit from sources you will need the full Java Development Kit version 1 4 available from the same Web site and a recent version of the Apache Ant build system At this time we recommend that you use the following packages as we have not yet tested Java CoG with any other Java version 1 ant 1 5 4 The Java CoG Kit requires Apache Ant which can be down loaded from http ant apache org 2 JDK 1 4 2_02 b03 or above Please note that earlier versions of the Java Development Kit contain expired root certificates which means that you will only be able to use public key cryptography in a limited fashion 4 3 Downloading the Java CoG Kit version 4 Before using the Java CoG Kit you will need to download it At this moment the Java CoG Kit is available only in source format and from the source repository To download the sources from the source repository you will need to have a CVS client installed Instructions will be provided for the command line CVS clients available
62. ee and is similar to the initial source after all 93 the include elements have been processed The state of of the execution is composed of two main areas the set of events that are waiting to be delivered and the state of elements that have begun execution but have not yet completed it When a checkpoint is requested the checkpoint manager first locks the event dis patcher in order to guarantee that the state of remains consistent during the check pointing process While the event dispatcher is locked it does not deliver events nor does it accept new events Threads that are trying to post events to the dis patcher are suspended during this time The checkpoint manager also keeps track of elements that have been started but were not completed and also keeps a refer ence to the stack of those elements Since the checkpoint manager does not make a full copy of the element stack for performance considerations it may sometimes be the case that an element can at specific moments modify the stack and leave it in an inconsistent state A special locking mechanism that allows an element to group operations on the stack that should be atomic is provided The checkpoint manager will therefore wait until all elements have completed the execution of blocks that need to be atomic relative to the stack before making the actual checkpoint It can be easily seen that posting an event to the dispatcher inside a atomic block could cause a deadlock
63. endencies It will also create a dist directory containing the distribution of the GridAnt module Due to problems with the inability of Apache Ant classloaders to load signed jars it is required that the user explicitly include all the necessary jar files in their class path This can be done by sourcing the gridant setenv script from the dist gridant 1 0 bin directory cd dist gridant 1 0 bin source gridant setenv Further in order to execute the GridAnt tasks it is required to actually define these tasks lt taskdef name cog setup classname org globus cog gridant GridSetup classpathref classpath gt lt taskdef name grid authenticate classname org globus cog gridant GridAuthenticate classpathref classpath gt lt taskdef name grid execute classname org globus cog gridant GridExecute classpathref classpath gt lt taskdef name grid copy classname org globus cog gridant GridCopy classpathref classpath gt The gridant xml file provides a detailed example of how to define and use these tasks 64 8 3 GridAnt Tasks The following is a partial list of GridAnt tasks that are currently available in the GridAnt module cog setup The Grid environment setup grid authenticate Initializes the proxy certificate to be used by clients grid execute
64. ense 24 Other Licences 2045 4 awe Pee ew eee e e 2AL JETODOS ciora a a ee e Ee ae el 4 2AA DIES cons poe Rh EERE SY SRG ee DAS IE Bee ie es 25 GNU Public Licence isis es VD OO oO o 10 10 11 12 12 12 13 14 14 14 3 Introduction 3 1 3 2 DIVEIVIEW san na a a ee RE a eA PUSE ar UA AA eS 32 1 Metacomputing ce res bea ee Re e Bae CoG RIR con es 2 een a een BA 4 Installation 4 1 42 43 44 4 5 4 6 4 7 4 8 4 9 4 10 4 11 4 12 4 13 Download 2 seor sos ea 22H 0 00 eS eR we CVS Release Tags oo 2 2 0 2 a e y eh we Downloading the Java CoG Kit version 4 Compiling the Java CoG Kit version 4 Compiling the Complete Distribution Compiling Individual Modules Using the Java CoG Kit version4 o Downloading JGlobus oo rs a ES BS Compiling IGIOBUS var a 40 ser ae Using IJGlob s 2 2 2 2 20040 25 2 8 Bann nn Downloading OGCE o ea 25 22 22 8 sea a eee ars Compiling OGUE oo copa era Using OGUE 2556 6 ee eS 5 Contributing Jd 5 2 Creating a Module os o oe 24545504 44550 See a a ILL B ldFilES casi a en 12 LIBRES peee ap en ST SEE lt p e AA A ee E 3 14 Usiig PMB eed aioe A a hw 5 1 5 Documenting the Modules 5 1 6 Maintaining a Module O o Era ILE WEDSTATE 2605 4 cae pede a Coding Guidelines for the Java CoG Kit 321 POS cer al e a da JAA I
65. ense Draft 29 Feb 04 22 2 3 2 Java CoG Kit Contributor Liceense Grant of Licenses in Java CoG Kit Contributions July 2003 The undersigned licensor LICENSOR has delivered or caused or permitted to be delivered to The University of Chicago as Operator of Argonne National Laboratory collectively LICENSEE soft ware and documentation collectively CONTRIBUTIONS created by LICENSOR or by LICEN SOR s employees associates contractors or collaborators Software SOFTWARE contributed by LICENSOR are source and binary software code such as schema bug fixes patches upgrades or other modifications or enhancements of the features functionality or performance of the Java CoG Kit soft ware JAVA COG KIT Documentation DOCUMENTATION contributed by LICENSOR are print and digital media that describe and explain such as readme files white papers overviews and tutorials and are useful with the JAVA COG KIT Acknowledging receipt of LICENSEE s services in distributing the JAVA COG KIT a community of end users and developers including LICENSOR and LICENSEE s intention to continue such distribu tion LICENSOR hereby grants to LICENSEE the following licenses collectively with all limitations included herein referred to as this Grant a a non exclusive worldwide royalty free perpetual transferable copyright license to install use re produce modify display perform and prepare d
66. erface There are two interfaces to Karajan 1 The command line interface accessible through bin karajan provides a very simple interface which is mainly non interactive and does not provide feed back on the execution 2 The graphical interface which can be started through bin karajan gui can display a graphical representation of the workflow and other progress infor mation and statistics It also allows a certain level of interaction The command line interface allows you to start the execution of a specification The syntax is very simple gt karajan spec xml Karajan will then try to load parse and execute the indicated specification Any resulting messages will be printed on the console The graphical interface allows for additional interaction with the execution engine It can be started using bin karajan gui The following command line options are supported help Displays a brief usage summary load filename Can be used to load a script upon starting run Used in conjunction with load will immediately start the execution When the interface is started without any parameters an empty view is presented Figure 9 1 IT graph Canvas Java CoG Kit Graph Viewer TX File View Help Graph View k Figure 9 1 An empty Karajan desktop 70 The File gt Open menu item can be used to load a script After the script is loaded a graph that represents the control flow of the l
67. erivative works of SOFTWARE to incorporate SOFT WARE in whole or in part and derivative works thereof into the JAVA COG KIT or into any other computer software and to sublicense copyrights in and distribute SOFTWARE and derivative works thereof in source and binary forms b subject to termination provisions below and to the extent that patent claims licensable by LICENSOR are necessarily infringed by the use or sale of SOFTWARE alone or when combined with the JAVA COG KIT a non exclusive worldwide royalty free license and right to sublicense under such patent claims with the rights for LICENSEE and its sublicensees to make use sell offer to sell import and otherwise transfer the SOFTWARE in source code and object code form alone and incorporated into the JAVA COG KIT and derivative works This patent license shall apply to the combination of the SOFTWARE and the JAVA COG KIT if at the time SOFTWARE is added by the LICENSEE or its transferees such addition of the SOFTWARE causes such combination to be covered by such patent claims The patent license shall not apply to any other combinations which include the SOFTWARE No hardware per se is licensed hereunder c a non exclusive worldwide royalty free perpetual transferable copyright license to install use reproduce modify display perform and prepare derivative works of DOCUMENTATION to incorpo rate DOCUMENTATION in whole or in part and derivative works thereof into any other do
68. ertrequest 34 7 Core 7 1 Introduction 7 2 Installation 7 2 1 Download The Java Cog Kit core cog core is an add on to the cog jglobus library It in cludes many advanced features to make Grid programming easier The core mod ule provides an abstraction layer for various low level Grid implementations such as Globus Toolkit v2 and v3 A Grid application developer can port Grid appli cations from one implementation GT2 to another GT3 by simply changing the underlying implementation provider Thus all applications developed by using the cog core APIs are compatible with all the underlying Grid implementations sup ported by cog core The current version of the core module provides support for GT2 GT3 and SSH implementations Other platforms will be supported based on availability of resources Further cog core also provides several constructs whereby simple execution de pendencies workflows can be expressed as a directed acyclic graph DAG or hierarchical DAG where each Grid task can interface with a different Grid imple mentation For more sophisticated workflow functionality the reader is directed to the Java CoG Kit Karajan module in Chapter 9 Hence cog core offers the following benefits e develop client applications that will be interoperable across multiple Grid backend implementations e provide reusable code to support rapid prototyping of basic Grid access pat terns e provide an open source and extensib
69. es to be multiplied Divides two real numbers Arguments dividend The dividend divisor The divisor 100 9 8 35 9 8 36 9 8 37 9 8 38 9 8 39 9 8 40 9 8 41 math remainder math sqrt math square math subtraction math sum newJavaObject nonCheckpointable Evaluates the remainder of a division operation Arguments dividend The dividend divisor The divisor Computes the square root of a number Arguments value The number for which the root square is to be calculated Computes the square of a number Arguments value The number to be squared Subtracts two numbers evaluates from value Arguments from The minuend value The subtrahend Adds a set of numbers Arguments Nested arguments The values to be summed instantiates a new Java object Arguments className the class name of the object to be instantiated Has no functional purpose It is generated inside serialized versions of events in the locations where non checkpointable elements are found An example of such an element is include which serves its purpose during the parsing process and has no further function afterwards 101 9 8 42 9 8 43 9 8 44 9 8 45 9 8 46 9 8 47 9 8 48 not numberFormat or parallel project readFile restartOnError Performs a boolean negation Arguments value The argument Formats a number according to the specified pattern Argumen
70. esearch Council Canada If we have forgotten to include your name in the list of contributors please notify us 1 8 Administrative Contact 1 9 Acknowledgments The project is managed by Gregor von Laszewski To contact him please use the information below Gregor von Laszewski Argonne National Laboratory Mathematics and Computer Science Division 9700 South Cass Avenue Argonne IL 60439 Phone 630 252 0472 Fax 630 252 1997 gregor mcs anl gov This work was supported by the Mathematical Information and Computational Science Division subprogram of the Office of Advanced Scientific Computing Re search Office of Science U S Department of Energy under Contract W 31 109 Eng 38 DARPA DOE and NSF support Globus Project research and develop ment This work would not have been possible without the help of Ian Foster and the Globus Project team 14 2 License 2 1 Project Registration The Java CoG Kit is distributed under two licenses The parts that are included in the Globus Toolkit are distributed under the Globus Toolkit Public License GTPL which is listed in Section 2 2 1 The parts that are not distributed in the Globus Toolkit are distributed undert the Java CoG Kit Public License At this tiem the Java CoG Kit License Section 2 3 is a simple copy of the Globus Toolkit License with the Globus Toolkit refernces being replaced by apropiate Java CoG Kit refernces and the institution just being University of Ch
71. ested unnamed arguments e not 2 negates the default value argument 86 9 3 14 List elements 9 3 15 Miscellaneous Elements list append 2 2 appends concatenates the supplied unnamed arguments in a list The arguments can be lists They are appended in the order they are supplied list prepend 92 concatenates the supplied unnamed arguments in re verse order It does not reverse individual arguments list size 8 evaluates to the size of the default list argument If the argu ment is not a list the size will be 1 project 9 4 is the main container of a Karajan specification Any specifi cation that can be executed by Karajan must have project as the root element echo S echoes a message on the console The message can be specified either by using the message attribute or by inlining the text inside the echo 9 8 7 element include 8 2 can be used to include reusable libraries inside a main speci fication The include 2 element acts during the parsing process before the actual execution begins The file attribute specifies a file relative to the main specification file that will be substituted for the include el ement The included file will have its root element ignored Section 9 5 provides details about the include search path wait 86 produces a delay in the execution One of the delay or until attributes must be set The delay attribute indicates wait period in mill
72. esthost localhost destdir tmp gt lt allocateHost gt 9 3 10 Explicit Error Handling In certain cases errors that appear in certain locations are known to have no impact on the overall execution A typical example would be a cleanup process In such cases it may be preferable to be able to simply ignore errors Other operations have particularly high rates of failure However subsequent re executions of such operations may result in a successful result The following elements deal with such cases e ignoreErrors 22 has no attributes and any errors that occur on contained elements are ignored 85 9 3 11 Java Elements 9 3 12 Arithmetic elements 9 3 13 Boolean elements e restartOnError 849 has a numeric mandatory attribute times that spec ifies the number of times the contained elements are restarted when an error occurs before that error is reported e generateError will cause an error to be generated with an associated message specified by the message attribute The Java elements allow limited interaction with the Java Virtual Machine e newJavaObject instantiates a new Java object The class of the ob ject is specified by using the className argument A set of arguments to be passed to the constructor can also be specified through nested arguments e invokeJavaMethod invokes a method on an existing object indicated by the default argument object The method n
73. f the said interfaces are available at org globus cog gui grapheditor generic GenericNode and org globus cog gui grapheditor generic GenericEdge Graphs are displayed in graph canvases which in turn can have various views used to render the graphs in particular ways Views can also have transformations used to algorithmically modify graphs just before they are being displayed These transformations can also be chained The nodes can also contain canvases which in turn can contain other graphs in a recursive manner This allow for the effective use of hierarchical graph structures The following example shows how to build a simple graph and display it in a window I We need to have the JavaDoc of the CoG online 110 create the root node RootNode root new RootNode create a canvas for the root node GraphCanvas canvas root createCanvas Il create a graph structure Graph graph new Graph create three node components GenericNode genericNodel new GenericNode GenericNode genericNode2 new GenericNode GenericNode genericNode3 new GenericNode add the components to the graph and keep references to the node objects Node nodel graph addNode genericNodel Node node2 graph addNode genericNode2 Node node3 graph addNode genericNode3 create two edge components GenericEdge genericEdgel new GenericEdge GenericEdge genericEdge2 new GenericEdge add
74. ged to inspect the JavaDoc documentation We further expect that you are familiar with the Globus Toolkit and have access to a Globus Toolkit installation If you do not the Globus web page provides information about the details and how to install it Globus Toolkit http www globus org The Globus Toolkit development us undergoing some significant changes If you currently use Globus Toolkit 2 4 x we do recommend to evaluate a switch to ver sion 3 2 carefully This is in anticipation the Globus Toolkit version 3 2 will be replaced with Toolkit version 4 x during the year 2004 The Java CoG Kit pro vides so far an abstraction that protects the application user from the differences between these versions In case you develop with the Java CoG Kit APIs a switch between versions of the Globus Toolkit is simplified 1 4 Resources We support our efforts through a web site on which you find a bug tracking system Mailing lists and the code repository 1 4 1 Project Website Online information about the Java CoG Kit can be found on its home page Home page http www globus org cog java Here you can find links to the manual the code and some basic information about the project Besides this page we also maintain a project related Web page that reports on the Java and Python Commodity Grid Kits Project gt http www cogkits org 1 4 2 Bug Reporting We are using the Bugzilla system from mozilla org to track bugs and requests fo
75. gt lt override var lt setvar name var value two gt lt print 116 velo ON te console lt echo message var gt lt sequential gt lt i at tus point var wall be one again lt echo message var gt lt sequential gt lt var does not exist here lt echo message var gt The example will produce the following output Be AAA gt 2 _ JJ lt a 75 9 3 2 Parallelism one two one var In the last echo 7 element Karajan tries to expand var but since it cannot be found it prints the message literally 935 Karajan supports two basic containers through which parallelism can be achieved namely sequential and parallel Both containers are synchronous which means that their execution will terminate when all sub elements have fin ished execution This behavior can be overridden in any element by specifying the sync false attribute The following examples illustrate the use of sequential 09 353 and parallel containers as well as synchronous and asynchronous execution On the right side an image showing the resulting control flow of the specifications on the left is shown deco Sequential execution lt sequential gt lt elementl gt lt element2 gt lt element3 gt lt sequential gt element4 Parallel execution lt parallel gt lt elementl gt lt element2
76. h A boolean flag that specifies if the remote execution is to occur in batch mode If true the client will not be notified of any status changes on the server side It is the responsibility of the client to obtain the final output from the server by some offline asynchronous mechanism If false the client will interactively receive status notifications from the server as well as the final output or error Default value is false For example to remotely execute a job on a GT2 resource say hot anl gov in batch mode lt gridExecute provider GT2 server hot anl gov delegation full executable bin 1s arguments 1 directory home amin stdoutput myOutput txt stderror myError txt redirect false batch true The grid copy task performs file transfers between grid resources For GT2 provider 1t supports direct as well as third party transfers between the client machine and GridFTP servers For GT3 provider it only supports third party file transfers be tween GridFTP servers using the reliable file transfer service of GT3 0 2 This task requires the following input arguments specifies a mandatory arguments provider Specifies the backend Grid provider for the file transfer Available options are GT2 and GT3 server Specifies the location of the remote file transfer service for the given provider not required for the GT2 provider xmlsecurity Specifies the XML se
77. hapter describes the Java CoG Kit testing framework The main reason the framework was created was to ease the process of deploying automated tests of the Java CoG Kit against various Grid services Nevertheless it was also successfully used in testing the deployment and availability of the services themselves The framework generates results in a convenient HTML form that can be accessed through the web The testing framework consists of a set of Java classes and a driver script The Java classes are used to interface with the Grid client side functionality provided by the Java CoG Kit while the driver script performs more mundane tasks such as fetching of code data compiling combining reports and launching test suites 11 2 Using the Testing Framework To use the testing framework you will need to download the driver script The driver script is available in the Java CoG Kit v4 CVS archive Instructions on downloading the Java CoG Kit sources from CVS are available in the installation Section 4 The driver script would then be located in the cog modules testing bin directory The script can then be moved to a convenient location 11 2 1 Configuring the Driver Script The driver script contains a set of configuration variables that can be modified to customize the tests These variables are explained below LOCAL If set to no the drivers script will fetch the latest Java CoG Kit sources from the CVS archive at each run If se
78. he following commands In order to reduce ambiguity a directory named lt download gt will be used as the base of the following operations For further details about JGlobus and Globus Toolkit versions interoperability please consult the Java CoG Kit compatibility matrix athttp www globus org cog java production php 28 1 cvs d pserver anonymous cvs globus org home dsl cog CVS login 2 password Hit Enter 3 cvs d pserver anonymous O cvs globus org home dsl cog CVS checkout r v 4 0 a jglobus 4 9 Compiling JGlobus JGlobus can be compiled by using the following commands 1 cd lt download gt jglobus 2 ant dist The compiled JGlobus files will now be available in the lt download gt jglobus build cog 1 2 directory 4 10 Using JGlobus The command line tools included in JGlobus can be accessed from the lt download gt jglobus build cog 1 2 bin directory Before using any of the JGlobus command line tools you must set the COG_INSTALL_PATH environment variable to point to the lt download gt jglobus build cog 1 2 directory It is preferable to specify COG_INSTALL_PATH as an absolute path 4 11 Downloading OGCE OGCE is a part of the Java CoG Kit that includes a series of experimental graphical interfaces and convenience abstractions over JGlobus OGCE can be obtained from the source repository by using the following commands 1 cvs d pserver anonymous cvs globus org home dsl cog CVS login 2 password Hit E
79. he job to a remote file Used optionally to redirect the standard error stream of the job to a remote file 2 attributes followed by an asterisk are optional 84 e gridTransfer is used to transfer a file from one host to another The accepted attributes are srchost The source host Use localhost for the local machine srcdir The source directory where the file can be found srcfile The name of the file that is to be transfered desthost The destination host Can also be localhost for the local machine destdir The directory on the destination host where the file will be placed destfile Can be used to rename the file during the transfer e It may sometimes be necessary to execute a set of tasks on the same host The allocateHost element can be used for this purpose The name at tribute specifies a variable that can be used inside the element by the various tasks whenever the remote host needs to be referenced A simple example is provided below lt allocateHost name remote gt lt transfer the input data lt gridTransfer srchost localhost sredir tmp Scion desthost remote destdir tmp gt lt do the heavy processing lt gridExecute host remote executable usr bin tac args stdin tmp in stdout tmp out gt lt transfer back the results lt gridTransfer srchost remote sredir tmp srernille oni d
80. hose variables This is still possible only in a sequential context There is no way to propagate information from one thread to the other While this characterizes the workflow execution the applications themselves can still use interthread or interprocess communication or messaging as needed A distinction exists between Karajan threads and Java or OS threads The Karajan threads differ from each other only by the variable stacks they receive No assump tion can be made about the Java or OS thread in which a Karajan thread executes The events that are passed between elements are managed by an event dispatcher which may use more than one Java thread The appearance of parallelism is achieved through the fact that elements either take a short time to execute or they make use of their own Java threads if known to take a longer time to execute The result is the ability to execute a large number of Karajan threads without the over head required by Java OS threads As an example each Java thread requires a minimum of 96 kilobytes of memory just for the thread stack Karajan defines six event types 91 START tells an element it should start execution RESTART tells an element which has not completed execution yet that it should restart its execution EXECUTION_STARTED sent by an element immediately after it has started execution EXECUTION_COMPLETED sent by an element after it completes execution This event is sent as a result of r
81. iate certificate au thority Listing 7 16 Create security credentials Es Siico tie provider 18 GUS create a GlobusSecurityContext If a non globus security context is Req llincd nen use ne SecurityContextlmpl class and set the credentials as required by the handler GlobusSecurityContextlmpl securityContext new GlobusSecurityContextImpl 52 x Assign the default credentials available as a valid proxy certificate whose location is specified in the COM PLOPe nte Sp ES Sent HOME globus directory To assign non default credentials create a GSSCredential and pass this GSSCredential as the argument instead of null securityContext setCredentials null x Assign this security credential to the task x task setSecurityContext securityContext Next assign a ServiceContact to the task This attribute defines the location of the remote Grid resource where the task is to be executed Listing 7 17 Create a service contact ServiceContact service new ServiceContactImp Shite pice ee On One 080 ogsa services base gram MasterForkManagedJobFactoryService task setServiceContact service Next create a TaskHandler and submit the task for execution Listing 7 18 Create a task handler TaskHandler handler new GenericTaskHandlerImpl try handler submit task catch InvalidSecurityContextException ise logger error Security Exception ise System exit
82. ibraries in your module may at any time move to the cog 1ib directory 31 5 1 3 Source 5 1 4 Using PMD The sources for your module Not much to say here We recommend that developers and contributors use PMD http pmd sourceforge net to check their code Many of the complaints that PMD generates should be taken seriously Still there are instances when PMD rules do not apply for a good reason and create false positives To use pmd you need to download it and add all its jar files to the pmd directory Afterwards just run ant pmd in the module you want to check It will generate both an on screen report and an HTML report pmd report htm 5 1 5 Documenting the Modules 5 1 6 Maintaining a Module 5 1 7 Launchers 5 1 8 Webstart README TODO CHANGES PMD 5 2 Coding Guidelines for the Java CoG Kit 5 2 1 Imports 5 2 2 Indentation 5 2 3 Brackets 5 2 4 Variables The Java CoG Kit follows in general the basic coding conventions given in the Sun Coding Conventions for the Java Programming Language http java sun com docs codeconv html CodeConvTOC doc html Additionally we have the following rules All imports must be single class and explicit That is import lt package gt is not allowed All indentation levels should be 4 spaces No editor tabs are allowed unless they are converted to 4 spaces before saving the file In contrast to the OGSA coding guides we allow the use of brackets
83. icago To collaborate with us it is best for now to just sign the Globus Toolkit contributor License and fax it to Gregor von Laszewski at 630 252 1997 We wish that you to notify us about projects that you develop with the help of the Java CoG Kit This will allow us to keep track of the use of the Java CoG Kit as this directly affects our ability to motivate additional coding activities Please be so kind to send an e mail to gregor mcs anl gov with the subject JAVA COG KIT USGAE with the following additional information provided by you Project name Institution Main contact E mail Web page Description of your project References References citing the Java CoG Kit In case you like to cite the Java CoG Kit in your papers we recommend that you use the following paper Gregor von Laszewski lan Foster Jarek Gawor Peter Lane A Java Commodity Grid Kit Concurrency and Computation Practice and Experience Pages 643 662 Volume 13 Issue 8 9 2001 http www globus org cog java We also would like to be notified about your publications that involve the use of the Java CoG Kit as this will help us to document its usefulness We like to feature links to these articles with your permission on our Web site Additional references to Java CoG Kit and other Grid related activities can be found at 15 Some Refernces von Laszewski or Some References Globus Project http www mcs anl gov gregor bib
84. ies the parameters for credential delegation for GSI security Valid options are full partial and none for full delegation partial delegation and no delegation respectively The default is no delegation executable Specifies the command to be executed on the Grid resource localExecutable A boolean flag that specifies if the executable resides on the client machine If true the executable will be automatically staged from the client machine The default is false arguments Specifies the arguments to be provided with the executed command directory Specifies the remote working directory in which the command is to be exe cuted 65 8 3 4 grid copy environment Specifies the environment variables to be set prior to the execution of the command stdoutput Specifies the file name to which the output must be redirected If left blank or not specified the output is streamed to the standard output By default output is streamed to the standard output stderror Specifies the file name to which the error messages must be redirected If left blank the errors are streamed to the standard error By default the errors are streamed to the standard error stdinput Specifies the file name from which the input must be extracted In the current version staging of input files is not supported redirect A boolean flag that specifies if the output and error streams are to be redi rected to the client Default value is true batc
85. in any copyright registration notice or license associated with the CONTRIBUTIONS LICENSOR hereby grants LICENSEE permission to post this Grant on a website associated with the GLOBUS TOOLKIT provided that LICENSOR is given the opportunity to review and approve in advance any such posting such approval not to be unreasonably withheld or delayed Except as expressly provided herein nothing in this Grant shall be construed as granting any right or license under any inventions patents copyrights trade secrets or any other intellectual property rights of LICENSOR LICENSOR By Title Date Address Globus Toolkit Contributor License 7 31 03 20 2 3 Java CoG Kit Note that the Java CoG Kit License is just a draft and is in principle very similar to the Globus Toolkit License At present we recommend to work with us under the Globus Toolkit Public Li cense and the Globus Toolkit Contributor License 2 3 1 Java CoG Kit Public License JCoGPL Java CoG Kit Public License JCoGPL Draft Version Java CoG Kit Public License Draft February 29 2004 Copyright 1999 2003 University of Chicago All rights reserved This software referred to as the Java CoG Kit software Software includes voluntary contributions made to the Globus Project collaboration Persons and entities that have made voluntary contributions are hereinafter referred to as Contributors This Java CoG Kit Public License is referre
86. ion is this filename a directory public boolean isDirectory String dirName throws DirectoryNotFoundException executes a non interactive workflow of commands public void submit ExecutableObject commandWorkflow throws IllegalSpecException TaskSubmissionException public void setAttribute String name Object value public Object getAttribute String name 50 7 4 Programmer s Guide 1 Executing a remote job execution task 7 4 1 Executing a third party file transfer task 7 4 2 Executing a simple TaskGraph DAG 7 4 3 Executing a hierarchical DAG 7 4 4 Writing a custom Task Handler 7 4 5 N PA 2 2 ID Executing tasks on an ExecutionResource 7 4 6 7 4 1 Executing a Remote Job Execution Task Executing a remote job becomes extremely simple with cog core To begin with create a Task with the appropriate attributes Listing 7 14 Create a Task object x Create a new job submission task named myTestTask x Task task new TaskImpl myTestTask Task JOB_SUBMISSION Set the desired provider Default options are GR GBFZOTLZSSH task setProvider GT3 Then create a JobSpecification for the task and set the appropriate attributes as per the task requirements Listing 7 15 Create a task specification Create a new JobSpecification x JobSpecification spec new JobSpecificationImpl ie excouialblle executable then x Set the location and
87. ion In other words for some Grid implementations it may not be possible to suspend and resume remote execution It is easy to associate a simple Task with one of the above mentioned status For example initially the task is unsubmitted its status changes to submitted when it is handled by a handler its status changes to active when it is being executed remotely and so on However it is not apparent how a TaskGraph is mapped to one the supported status Cog core uses the following logic to map a TaskGraph to its appropriate status Listing 7 7 Pseudocode to determine the status of a TaskGraph if any Task in the TaskGraph has failed status failed else if all tasks are unsubmitted status unsubmitted else if any task is suspended 43 7 3 6 Handlers status suspended else if any task is either active or resumed status active else if any task is submitted status submitted i else if every task is either completed or canceled status completed i else impossible to get here the above cases take care of all conditions Listing 7 8 Interface definition for Status public interface Status public static final int UNSUBMITTED 0 public static final int SUBMITTED 1 public static final int ACTIVE 2 public static final int SUSPEND public static final int RESUMED 4 public static final int FAILED 5 public static final int CANCELED 6 public static fi
88. isec onds while the until attribute specifies an absolute date in a format accepted by the java util Date class equals 12 tests if value is identic to value2 Unlike math equals this is not a numeric comparison Comparing 1 0 and 1 will lead to a result of false argument evaluates to the value supplied by the value argument contains S5 determines whether a file contains a specific sequence of characters The file attribute points to the file to be checked while the value attribute specifies the value to be searched numberFormat allows the formatting of a decimal number The pattern attribute indicates the patter to be used for formatting as used by the java text DecimalFormat class The value attribute holds the decimal value that is to be formatted readFile reads the contents of a file pointed to by the name attribute This is intended for short text files that may possibly hold things like error messages or exit codes The file is completely read into memory therefore this function would not be suitable for manipulation of large files UID 8 59 generates a unique ID 3 This function is not thread safe at the moment of this writing but plans are to correct the problem 87 9 4 Supported Handlers Karajan supports any handler that the Java CoG Kit Core supports However some handlers may require particular security settings which must be known or
89. le architecture that can be built collec tively and incrementally based on community feedback and e access the same set of interfaces implemented in disparate technologies The Java CoG Kit core module can be downloaded from the Java CoG Kit CVS archive Instructions regarding the Java CoG Kit requirements and details on ob taining the Java CoG Kit sources are available in Section 4 We note that cog core is explicitly a client side library The current version of cog core provides support for GT2 GT3 and SSH Hence in order to execute tasks against these implementations the reader is directed to install GT2 4 GT3 0 2 and SSH server For further details on installing the Globus Toolkit please visit the Globus Alliance Web page http www globus org Formerly known as the GridSDK module 35 7 2 2 Compile To compile the core module change the directory to cog modules core and type ant dist This will compile cog core and all its dependencies It will also create a dist directory containing the distribution of the core module Inside the dist directory the bin directory will contain the necessary scripts that can be used to launch several command line clients and example applications 7 2 3 Configuration Cog core can be configured via the cog core properties file in the globus direc tory The user can set the following properties in this configuration file provider class GT2 org globus cog core impl gt2 G
90. le patent license termination language as that set forth above and b disclaim liability on behalf of LICENSOR for all damages including direct indirect special incidental exemplary and punitive damages in sub stantially the same form as that included herein and shall apply to any and all CONTRIBUTIONS specifically designated now or in the future as within the scope of this Grant by LICENSOR s autho rized representative LICENSOR represents that to its knowledge it has or has obtained any and all required permissions and authority to make this Grant Except as provided in the preceding sentence LICENSOR PROVIDES THE CONTRIBUTIONS ON AN AS IS BASIS AND MAKES NO REPRESENTATIONS AND EXTENDS NO WARRANTIES OF ANY KIND WHETHER EXPRESS OR IMPLIED CONCERN ING THE CONTRIBUTIONS LICENSOR MAKES NO EXPRESS OR IMPLIED WARRANTIES OF MERCHANTABILITY OF SATISFACTORY QUALITY OR FITNESS FOR A PARTICULAR PURPOSE OR USE LICENSOR MAKES NO REPRESENTATION THAT THE CONTRIBUTIONS WILL NOT INFRINGE ANY PATENT COPYRIGHT TRADE SECRET OR OTHER PROPRIETARY RIGHT LICENSOR SHALL HAVE NO LIABILITY WITH RESPECT TO ITS OBLIGATION UN DER THIS GRANT OR OTHERWISE FOR DIRECT INDIRECT SPECIAL INCIDENTAL CON SEQUENTIAL EXEMPLARY OR PUNITIVE DAMAGES OF ANY CHARACTER EVEN IF AD VISED OF THE POSSIBILITY OF SUCH DAMAGES 19 LICENSEE shall not be required to include the name of LICENSOR or any of its employees asso ciates contractors or collaborators
91. ll appear as an icon for the rendered node overlayfile String an absolute path to an image that will be overlayed on top of the base icon hue Float specifies an additive adjustment for the hue of the icon Changing this can shift the colors of the icon 109 10 5 API saturation value status Float a multiplicative adjustment for the saturation of the colors in the icon Float a multiplicative adjustment for the value of the colors in the icon Integer a value ranging from 0 to 3 with the following meanings stopped running failed completed This will adjust the HSV color properties of the icon and change the overlay to give a visual representation of a possible state of a task You can also add your own custom properties which will show up in the properties list for a node edge lt graph gt lt node nodeid 1 name the first node iconfile usr share icons myicon png gt lt node nodeid 2 name the second node myproperty myvalue gt lt edge from 1 to 2 gt lt graph gt At the basis of the viewer editor stands the org globus cog util graph Graph class For details please consult the JavaDoc available at el The editor can render nodes and edges that correctly implement the org globus cog gui grapheditor nodes NodeComponent interface respectively theorg globus cog gui grapheditor edges EdgeComponent Generic implementations o
92. mbers from spam bots News about the Java CoG Kit is sent in irregular intervals the frequency is monthly to every four month by means of the following list CoG News cog news globus org 11 Sorted by Thread http www unix globus org mail_archive cog news threads html Sorted by Date http www unix globus org mail_archive cog news maillist html Discussions and Community Developers Discussions and general questions can be send to the high volume e mail list at Java List java globus org Sorted by Thread http www unix globus org mail_archive java threads html Sorted by Date http www unix globus org mail_archive java maillist html 1 4 4 Sourcecode Repository 1 5 About the Manual 1 5 1 Conventions Note that this list may result in daily mails sent by the Java CoG Kit community Please use the bug tracking system for reporting bugs If you use the bug track ing system your message has a higher chance of being answered There is no guarantee that we answer a mail sent to the Java CoG Kit mailing lists We maintain all source code in a CVS repository that can be accessed anony mously You can find more details about this in Section This manual is constantly being improved and your input is highly appreciated Please report suggestion errors changes and new sections or chapters through our bugzilla system When you report bugs please do not use page line or section numbers Remember new sections ma
93. n be further categorized into JobSpecification FileSpecification and QuerySpecification not implemented at this time We note that the specific parameters required in a task specification depend on the underlying Grid implementation used for the execution of the Task For example GT3 has several required parameters that are not supported by GT2 and vice versa However the specification classes in cog core offer some com monly used attributes that can be extended or omitted based on the requirements of the task and specific Grid implementation The JobSpecification mentions all the important attributes needed for the remote job execution Most of the attributes provided by the JobSpecification class are similar to the ones available in the Resource Specification Language RSL sup ported by the Globus Toolkit Nevertheless additional attributes can be added based on specific requirements Listing 7 4 Interface definition for JobSpecification public interface JobSpecification extends Specification public void setExecutable String executable public String getExecutable public void setDirectory String directory public String getDirectory public void setArguments String arguments public String getArguments public void setStdOutput String output public String getStdOutput public void setStdInput String input public String getStdInput public void setStdError String error public String getStd
94. n element ends it destroys the frame that it created and together with it the variables that it contained This behavior is not enforced but it is recommended It is however mandatory that the number of frames is the same before an element begins execution and after the same element has completed execution 90 In the case of parallel containers each parallel thread will start with a copy of the stack The stack copies will internally share frames that are not write accessible to the threads A diagram detailing the stack model can be seen in Figure 9 7 The conventional representation pop a b a for a stack indicates that a and b were present on the stack before the execution of pop and only a was left afterwards frameO frameO lt sequential gt frameO frameO frame1 lt element1 gt frameO framei frameO frame1 frameO framei frame2 a frameO framel frameO framel frame2 b lt parallel gt lt element2 gt frameO framei frame2 a frameO framei frame2 a lt element3 gt frameO framel frame2 b frameO framel frame2 b frameO framel frame2 a frameO frame1 frameO framel frame2 b lt parallel gt lt sequential gt frameO framei frameO Figure 9 7 The stack model The above may in some cases be insufficient Certain variables need to be made accessible to the parent frames such that they can be used by subsequent elements that are not descendants of the element which defined t
95. nal int COMPLETED 7 public abstract void setStatusCode int status public abstract int getStatusCode public abstract void setPrevStatusCode int status public abstract int getPrevStatusCode public abstract void setException Exception exception public abstract Exception getException public abstract void setMessage String message public abstract String getMessage public void setTime Calendar time public Calendar getTime Cog core contains the TaskHandler and the TaskGraphHandler to process a Task and a TaskGraph respectively Once a Task or a TaskGraph is submitted to the appropriate handler the handler interacts with the desired Grid implementation and accomplishes the necessary tasks The handlers in cog core can be viewed as adaptors that translate the abstract definitions of a Task and TaskGraph into implementation specific constructs understood by the backend Grid services For example a GT3 TaskHandler will extract the appropriate attributes from the cog core Task and make the necessary calls to the remote Grid service factory retrieve 44 the Grid service handle and interact with the newly created service instance Sym metric translations would be done for other Grid implementations Intuitively a handler is specific to the backed implementation and is the only part of cog core that needs to be extended for supporting additional Grid implementations Since cog core supports GT2 GT3 and SSH th
96. named arguments and calculates the root mean square of the supplied values The arguments are available as a list that can be iterated in the vargs variable lt element name rms vargs true gt lt math sqrt gt lt math quotient gt lt math sum argumentName dividend gt lt foreach name item in vargs gt lt math square value item gt lt foreach gt lt math sum gt lt list size list vargs lt math quotient gt lt math sqrt gt argumentName divisor lt element gt 81 9 3 8 Recursion If a definition takes only one argument and has no variable unnamed arguments that argument will become a default argument If there are multiple named argu ments but no variable unnamed arguments a default argument can be specified using the defaultArgument attribute In line element invocation is performed simply by using the element name The following examples will invoke the previously defined sum and rms elements and print the resulting values lt setvar name sum gt lt sum min 1 max 10 gt lt setvar gt lt echo message The sum is sum gt lt setvar name rms gt lt rms gt lt argument value 10 gt lt argument value 12 gt lt argument value 3 gt lt argument value 50 gt lt argument value 11 gt lt rms gt lt setvar gt lt echo message The rms is rms gt Due to the fact
97. ndency public void addDependency Identity from Identity to public boolean removeDependency Identity from 42 7 3 5 Status Identity to public void setAttribute String name Object value public Object getAttribute String name public void addStatusListener StatusListener listener public void removeStatusListener StatusListener listener public int getSize public boolean isEmpty public boolean contains Identity id public int getUnsubmittedCount public int getSubmittedCount public int getActiveCount public int getCompletedCount public int getSuspendedCount public int getResumedCount public int getFailedCount public int getCanceledCount Cog core provides two additional utility classes that specialize the functionality of the TaskGraph The Set is a special type of TaskGraph with no dependencies Intuitively it represents a bag of tasks that can be executed in parallel The Queue is another specialized TaskGraph that represents a first in first out FIFO queue The dependencies in a Queue are not set explicitly but are maintained implicitly based on the addition of a Task to the Queue Every ExecutableObject Task or TaskGraph has an associated execution status An ExecutableObject can be in one of the following status unsubmitted submit ted active suspended resumed failed canceled and completed We note that not every status is supported by every Grid implementat
98. not subscribed or you use an email address different from the one you subscribed with your message will not be posted to the list and you will not receive any notification that your mes sage was not posted Hence if you send a message to the list and do not subsequently see your message on the list or in the list archive verify that you are using an email address that is subscribed to the list and then retry your posting Subscribed Lists To verify that you are subscribed to the list send an email message from the Subscription Center News email account you subscribed from to majordomo globus org with the single word which in the body of the message You will receive in re sponse a message listing the lists to which your email address is subscribed If this mailing list does not appear in the list you receive you are probably subscribed to the list under a different address and you will not be able to post messages to the list using your current address If you would like to be notified of CoG Kit release updates visit our convenient subscription center at Subscribe http www globus org cog contact Other Globus related mailing lists can be found on the Globus web page Subscribe http www globus org about subscriptions html Note that you can use these web pages to unsubscribe from the lists All mailing list are maintained with majordomo However we did have to disable the who function in order to protect the me
99. nter 3 cvs d pserver anonymous cvs globus org home dsl cog CVS checkout r v 4 0 a ogce 4 12 Compiling OGCE The OGCE compilation process requires the JGlobus sources see Section 4 8 also to be present Both the JGlobus and the OGCE source package directories must exist in the same directory The following example shows a typical setting lt download gt 3jglobus lt download gt ogce To compile OGCE issue the following commands at the command prompt 1 cd lt download gt ogce 2 ant dist The compiled OGCE package including JGlobus will be available in the lt download gt build cog 1 2 directory or using the above typical example 29 lt download gt build lt download gt 3jglobus lt download gt ogce 4 13 Using OGCE The tools included in OGCE can be accessed from the lt download gt build cog 1 2 bin directory Before using any of the OGCE command line tools you must set the COG_INSTALL_PATH environment variable to point to the lt download gt build cog 1 2 directory It is preferable to specify COG_INSTALL_PATH as an absolute path 30 5 Contributing 5 1 Creating a Module 5 1 1 Build Files It is easy to contribute to the Java CoG Kit through its newly designed module concept A sample module build file can be found in modules template A few requirements have been imposed in order to keep consistency The basic directory structure that must exists for each module is etc MANIFEST M
100. nteractive O Keep aspect ratio E Show number of sub nodes Figure 10 4 The Node Context Menu An O n algorithm which places vertices on vertical levels using the edges as a factor of decision Generally if vertex a has an outgoing edge to vertex b the later will be placed on a level below the former Another O n algorithm which tries to distribute edges for a vertex in such a way that the angles between consecutive edges will be the same This algorithm is a variation of the LevelLayout with the dis tinction that instead of levels the vertices are placed on concentric circles While it tries not to overlap edges it does not always succeed O n algorithm suitable for flow networks It associates bounding boxes to vertices and traversing the graph resizes those bounding boxed according to the sizes of the bounding boxes of connected ver tices Renders the graph in Encapsulated PostScript format Available options are as follows postscript outputdir The directory where the output file will be placed postscript outputfile The name of the file that the output will be written to postscript graphview layoutengine The layout engine used to render the graph Accepts the same values as html graphview layoutengine10 3 2 10 3 4 The Remote Target Forwards API calls to a remote graph editor service 108 Available options are as follows remote contact A host port pair that represents the location of the graph
101. nts It is generally both acceptable and useful to use arguments inside elements that describe execution flow There is a specific set of elements namely sequential 9 8 53 parallel 9 8 45 for 98 14 foreach 8715 then 9 8 58 elseif 9 8 11 and else 81 which do not affect the contained arguments The following example shows how a the sum of the results of two parallel operations can be calculated lt math sum gt lt parallel gt lt operationl gt lt operation2 gt lt parallel gt lt math sum gt It is also possible to use iterators for expressing arguments both in parallel and non parallel mode lt math sum gt lt for name var from 1 to 10 parallel true gt lt argument value varj gt lt for gt lt math sum gt Templates can be used to define reusable code and are somewhat similar to proce dures in other languages Templates can accept named parameters Definition of templates can be done by using the templateDef 8 50 element The mandatory attribute name specifies the name of the template The body of the template can consist of any Karajan elements An additional element default 9 8 6 can be used to designate default values for parameters A simple template definition is shown below lt templatedef name sample gt lt default name argl value default1 gt lt default name arg2 value default2 gt lt echo message a
102. nts themselves to exist on different resources This mechanism also allows an external controller which has access to these events to completely control the execution It also allows a certain level of modification to the elements to be performed at runtime without affecting the execution of other elements As an example suppose a large job requires a transfer of the resulting data after the completion of all calculations Also suppose the specification of the transfer points to a non existing resource as the destination for the data The transfer will fail A tool can be used to intercept the failure notification and present the user with a visual message The user can then modify the bogus specification after which the particular failing element can be restarted by using the state present in the failure event 9 1 1 Obtaining the Source Code 9 1 2 Compiling Karajan Karajan can be downloaded from the Java CoG Kit CVS archive Instructions regarding the Java CoG Kit requirements and details on obtaining the Java CoG Kit sources are available in Section 4 Change directory to cog modules karajan and type ant dist This will compile Karajan and all its dependencies It will also create a dist directory containing the distribution of Karajan Inside the dist directory the bin directory will contain the necessary scripts that can be used to launch Karajan 69 9 2 Using Karajan 9 2 1 Command Line Interface 9 2 2 Graphical Int
103. o the GLOBUS TOOLKIT and derivative works This patent license shall apply to the combination of the SOFTWARE and the GLOBUS TOOLKIT if at the time SOFTWARE is added by the LICENSEE or its transferees such addition of the SOFTWARE causes such combination to be covered by such patent claims The patent license shall not apply to any other combinations which include the SOFTWARE No hardware per se is licensed hereunder c a non exclusive worldwide royalty free perpetual transferable copyright license to install use reproduce modify display perform and prepare derivative works of DOCUMENTATION to incorpo rate DOCUMENTATION in whole or in part and derivative works thereof into any other documentation and to sublicense copyrights in and distribute DOCUMENTATION and derivative works thereof If LICENSEE or any subsequent transferee each referred to as a RECIPIENT institutes patent litigation against any entity including a cross claim or counterclaim in a lawsuit alleging that the SOFTWARE or the GLOBUS TOOLKIT distributed pursuant to this Grant or the Globus ToolkitPublic License infringe such RECIPIENT s patent s then such RECIPIENT s rights granted under the patent license in paragraph b above shall terminate as of the date such litigation is filed The foregoing licenses shall be effective so long as the terms under which LICENSEE sublicenses copyrights in and distributes the CONTRIBUTIONS effectively a include compatib
104. oaded specification will be drawn An example can be seen in Figure 9 2 TM graph Canvas Java CoG Kit Graph Viewer v TIX File View Help Graph View v echo setvar setvar setvar v 167 nodes 176 edges Figure 9 2 A script was loaded The execution can be started by pressing the Start button located on the toolbar Once it is started the status of each node will be visible as an overlayed image over the node icon The following states exist None The node has not yet been executed Running gt The node is being executed Completed The node completed execution successfully Failed A Execution of the node failed Breakpoint Paused Wi Setting A Breakpoint o A breakpoint was set on the node The execution was paused at the current node possibly because of a break point being set on the node Breakpoints can be set by using a node s context menu Clicking on a node with the right mouse button will pop up the menu as it can be seen in Figure 9 3 Whenever the execution reaches the node where a breakpoint was set a message dialog will pop up and the execution of the specific thread branch where the node 1s located will be suspended see Figure 9 4 The execution can then be resumed by using the context menu of the node acces sible by right clicking on the node In the case of a paused node an item that will resume the execution will be
105. ocus on standardizing the interactions with Grid resources rather than the mechanisms for task execu tion A GridResource is an abstract entity in cog core that represents any Grid entity It contains an identity and name Currently cog core supports two types of GridResources the execution resource and the file resource Listing 7 11 Interface definition for GridResource public interface GridResource i public static final int FILE 1 public static final int EXECUTION 2 public static final int INFORMATION 3 public void setName String name public String getName 46 public void setldentity Identity id public Identity getldentity public int getType GTM FET GT2 Execution Transfer Transer Service Service _ Service Figure 7 3 ExecutionResource Abstractions 7 3 8 ExecutionResource An ExecutionResource in cog core represents an abstract Grid entity that maps multiple Grid services implemented by different providers onto a single Grid re source In other words without having to worry about the actual physical location of the services Grid users can construct their own execution resources mapping multiple services onto a single resource See Figure 7 3 Grid users can then sub mit tasks to this ExecutionResource to be executed on a service represented by the given provider Every execution resource manages its own queue for task scheduling Further it has a
106. on most UNIX and Linux machines and CYGWIN 1 cvs d pserver anonymous cvs globus org home dsl cog CVS login 2 password Hit Enter 3 cvs d pserver anonymous cvs globus org home dsl cog CVS checkout r v 4 0 a cog 4 4 Compiling the Java CoG Kit version 4 In compiling the Java CoG Kit you have two options 1 Compiling the whole Java CoG Kit This option may suit you if you plan to use the whole functionality of the Java CoG Kit or if you want to test all the features of the Java CoG Kit 2 Compiling individual modules This option will compile only the neces sary parts needed in order to provide the particular functionality packed in a module 27 4 5 Compiling the Complete Distribution In the main cog directory type ant dist A new directory named dist will be created in the cog directory Inside the dist directory you will find a cog lt version gt directory which contains libraries Jib configuration files etc example files examples and application launchers bin 4 6 Compiling Individual Modules The main cog directory contains a subdirectory named modules which in turn con tains all the modules that compose the Java CoG Kit You can change directory to any of these modules and type the following in order to obtain a binary distribution directory for that module ant dist A dist lt modulename gt lt moduleversion gt directory will be created containing the distribution files Any modules th
107. onResource 8 GridAnt Bl o A 3 2 Installation sa ce p ead eae ee ae ea es 821 Dowload eee kite hE SS eR ee e S22 Compile i pa asas so a BS BRE ae Be 8 2 3 Configuration lt oco ece e me ee a we 83 GndAnt Tasks eop see eee Ee ee ak IL AI 8 3 2 Btid aufhenlteale 23 baaa e BS 833 Smd exGeuie pe cae ba be EE SES ee wee SOA BROS oko aca Bo ee le PS OR e ee SA Workflow Example ac 28 be Saw a ea Ree ES BRS 9 Karajan 9 1 Anstallguon 222 ake RG RRA Mew Ae ee ods 9 1 1 Obtaining the Source Code 912 Compiling Karajan 22 56 5508 80 Gees 92 Using Karajan cocaina ee a eee oS 9 2 1 Command Line Interface 2 4 2 4 spoe 2 922 Graphical Interface soso de ed ee 93 Language Specification os s sesse ee eee 931 COMCSP S 0 54462 5 e ERR SRE wo ere A 93 2 Parallels u 4 2454 Sad are 6 ee 5 934 TEOS vente net 9 3 4 Conditional execution OS AUMENS o oe bie bea eS ee is 9 30 Templates u ek aaa a a A ak da 93 1 Inmineclements css osos be ee eS DIS BECHSION 5 6 o ee Beh ale ie 9 3 9 Grid related Elements 2 0 e 9 3 10 Explicit Error Handling 9311 JavaElement lt s o e hipa 63 63 64 64 64 64 65 65 65 65 66 67 69 69 69 69 70 70 70 72 72 76 77 18 79 80 81 82 83 85 86 9 4 9 5 9 6 9 7 9 8 93 12 Arithmetic elements so sn 624 6 6 bab Bee ewi 86 93 13 Boolean elements cee oR A RR A 86 23 18
108. onditionN gt lt elseif gt lt elseif gt lt else gt lt else gt lt if gt The condition1 through conditionN are elements that evaluate to something Typ ically they are functions such as math equals 9 29 math le 2 99 etc Func tionally the then 9 and elseif are the same They both execute if the last specified condition can be evaluated to a boolean value of true or cause the enclosing if 2 to terminate if the condition evaluates to a boolean value of false 78 9 3 5 Arguments Arguments in Karajan are generally expressed as XML attributes However XML attributes are limited It is impossible to use attributes to express the result of executing other elements The solution is to express such arguments as nested elements There are four main types of arguments named arguments are used to express arguments when there exist multiple arguments for an element and they differ from a semantic perspective The setvar ele ment for example has a name and a value attribute which cannot be com muted without changing the meaning of the program Named arguments can be expressed both as XML attributes and as nested elements with the argumentName attribute lt setvar name var value 1 gt would produce the same result as lt setvar value 1 gt lt argument argumentName name value var gt lt setvar gt variable unnamed arguments are used
109. only as defined in the Java Coding guidelines E g for index 0 index lt length index lt code gt No acronyms or abbreviations should be used For example a b mVarLen should be avoided Instead use totalLength partLength newLength 32 5 2 5 Instance Variables One Liners Logging Testing Internationalization Library Reuse Exceptions Use this prefix when referencing instance variables for example public MyClass ServicePropertiesInterface properties this properties properties publie int foo int locallnt 3 return this instancelnt locallnt Even single line statements should be inside brackets for example return Log4J should be used exclusively System out err println is not allowed Further exceptions should be logged Each component class should have a JUnit test The tests should be put in test directory under each package directory The core framework should be fully internationalized The samples may be inter nationalized The Java 118n L10n Toolkit may be used to verify whether code is international Treat all code as a library and as a reusable component Calls to System exit are disallowed except the main method Use chained exceptions Java CoG Kit provides two simple generic exception classes for chaining multiple exceptions together Look at ChainedException and ChainedIOException 33 6 Modules 61 util 6 2 c
110. osing them as a single complex activity To the heart of any workflow system is the workflow engine The workflow engine is a central controller that handles task dependencies failure recoveries performance analysis and process synchronization Most of the work done in workflow management systems concentrate on the business aspects of the workflow Little consideration is given to the needs of the client in terms of mapping the process flow of the client In the Grid community it is essential that the Grid users have such a tool available to their disposal that enable them to orchestrate complex workflows on the fly without substantial help from the service providers At the same time it is also important that such a workflow system does not burden the Grid user with the intricacies of the workflow system With the perspective of the Grid user in mind a simple yet powerful client side workflow management system has been developed and is named as GridAnt Gri dAnt which makes use of commodity technologies such as Apache Ant and XML GridAnt uses Apache Ant as its workflow engine Apache Ant is a popular build tool that is extensively used in the Java community Its current functionality allows the management of complex dependencies and task flows within the project build process We extend the functionality of Apache Ant by providing customized Ant tasks to access the Grid GridAnt is not claimed as a substitution for more sophisticated and powerful
111. oss integer ranges and takes two numerical attributes representing the first and last value There are two equivalent ways of specifying the two attributes as shown in the following examples lt for name iteration from 1 to 4 gt lt echo message Iteration iteration gt lt for gt and lt for name iteration range 1 4 gt lt echo message Iteration iteration gt lt for gt The result in both cases is as follows AeA UNa 71 9 3 4 Conditional execution e The foreach 81 iterator can be used to iterate across arbitrary values or files in a directory a feature that needs a little polishing Iteration values can be specified by using the in attribute To iterate on the files in a directory you can use the dir attribute Examples lt foreach name iteration in one two three four gt lt echo message Iteration iteration gt lt for gt would produce the following output while lt for name file dir home johndoe gt lt echo message File file gt lt for gt would list all the files contained in the home directory of user johndoe Conditional execution is expressed using the if then 099 elseif 0310 and else 29 10 elements The format is as follows lt if gt lt conditionl gt lt then gt lt then gt lt condition2 gt lt elseif gt lt elseif gt lt c
112. overloaded However certain parameters can be passed to the scheduler using the scheduler 849 element that can alter the way in which the resources are allocated from the defined pool the grid 81 element The scheduler also chooses the proper handlers and services for a given task Tasks may or may not have certain constraints associated with them Some tasks may have predefined resources or handlers that they require For example a cer tain job submission may have a predefined resource that it needs to run on In such a case the scheduler should not attempt to find another resource for the task However when a task does not specify such constraints the scheduler must fill the missing parts required to execute the task The scheduler must also take care of task encapsulation This refers to the case when certain tasks must be executed on the same resource When trying to submit a task the default scheduler cycles through the list of avail able resources and uses the first one that it finds suitable for the given task The resource search for the next task begins with the resource immediately following the last used resource in the list If the end of the list is reached the search con tinues from the beginning of the list If after one complete cycle through all the elements in the list nothing suitable is found the execution is postponed for a later time when some of the resources may become free The scheduler does not take care
113. pbuffer 32000 8 4 Workflow Example This sections demonstrates the sequence of tasks required to execute a simple Grid workflow Lets assume you want to transfer a file input txt from a Grid re source hot anl gov to another Grid resource cold anl gov Then execute the pro gram myJob on cold anl gov with GT2 provider and input txt as its input and out put txt as its output Finally you want to transfer the output txt back to hot anl gov lt target name workflow gt lt sequential gt lt Setup the Grid environment gt lt cog setup gt lt Iiminlalize ine Girl proxy Ceni EiS lt grid authenticate gt lt i Transiet ine Pu lt grid copy provider GT2 delegation partial source gsiftp hot anl gov 1111 home amin input txt gt 67 destination gsiftp cold anl gov 2222 home amin input txt parallelstreams 1 tcpbuffer 32000 gt thirdparty true gt EAS US Jou gt lt grid execute provider GT2 server cold anl gov 3333 delegation partial executable myJob arguments directory environment localExecutable false redirect false stdinput input txt stdoutput output txt stderror gt gt 2999 lt Transfer the output file gt lt grid copy provider GT2 delegation partial source gsiftp cold anl gov 2222 home amin output txt destina
114. present in the menu see Figure 9 5 71 Error Handling T graph Canvas Java CoG Kit Graph Viewer w X File View Help Graph View v I echo editable O sub nodes Properties gt Edit _ A O Interactive Keep aspect ratio a O Show number of sub nodes gt 167 nodes 176 edges Figure 9 3 The node context menu Errors that may occur during the execution which are not explicitly handled in the specification will result in a dialog window that provides several options for dealing with the error A sample error dialog is presented in Figure 9 6 Each error option provided by the error dialog is described below Abort Ignore Restart Apply to all errors of this type Apply to all errors for this element 9 3 Language Specification Passes the error to the execution engine which will result in an error message dump on the console and the immediate termination of the execution Completely ignores the error as 1f it has never occurred Restarts the failed node You can also specify the number of times that the node will be restarted before the execution is aborted Whenever an identical error occurs on any node the same action will be applied automatically All other errors that occur on the node will automatically be treated with the same action 9 3 1 Concepts Elements The Karajan specifications are written in an XML ba
115. put some temporary credential on a MyProxy server See myproxy command line tool for more details on what this means myproxy jpg description of params goes here After a certain amount of time your credential will expire Before this happens your CA will send you a renewal notification with a challenge phrase You can use the certificate renew applet to generate your renewal request certrenew_applet jpg description of params goes here If for some reason your credentials get compromised or you simply dont need them anymore you may want to destroy your credentials and notify your CA that you did so The certificate revocation applet can be used to delete the certificate files and notify your CA certrevocation_appet jpg description of params goes here grid cert request help options SYNOPSIS grid cert request can create user host and LDAP server certificates A certificate request and private key will be created You will be asked to enter a PEM pass phrase This pass phrase is akin to your account password and is used to pro tect your key file If you forget your pass phrase you will need to obtain a new certificate EXAMPLES grid cert request Creating a user certificate grid cert request host my host fqdn Creating a host or gatekeeper certifcate grid cert request service ldap host my host fqdn Creating a LDAP server certificate OPTION version Display version h
116. r enhancements for the Java CoG Kit Bugzilla provides you with an interface that guides you on submitting the bug The link to the bug system is located at CoG Kit Bugzilla http www globus org cog contact bugs In case you like to report bugs for other components of the Globus Toolkit you can use the main link at Globus Toolkit Bugzilla http bugzilla globus org globus 2 we need to make sure that we have in the ant script a publication mechanism of the JavaDoc We need to document how we update the web page and manual E g in the doc directory we say make make publish 10 1 4 3 Mailing Lists Policy To use it you need to first create an account To report a bug you need to be precise in your description and include operating system JVM version and other information that can be used to better identify or replicate the condition of your error This also includes the version of Globus Toolkit services you use We have established a number of mailing lists to simplify the communication with the group of developers and users Restrictions on the use of the mailing list are outlined below No Advertisements We do not allow you to use the mailing lists in any form of advertisement for your products or services In response to spam mail on this mailing list we have disabled the ability to post messages to this list if you are not subscribed to it Subscription Required If you send a message to the list and are
117. r products derived therefrom and except as expressly provided herein may not be affixed to modified redistributions of this Software or Documentation except with prior written approval obtainable at the discretion of the trademark owner from info O globus org 5 To the extent that patent claims licensable by the University of Chicago as Operator of Argonne National Laboratory are necessarily infringed by the use or sale of the Software you and your transferees are granted a non exclusive worldwide royalty free license under such patent claims with the rights to make use sell offer to sell import and otherwise transfer the Software in source code and object code form This patent license shall not apply to Documentation or to any other software combinations which include the Software No hardware per se is licensed hereunder If you or any subsequent transferee a Recipient institutes patent litigation against any entity including a cross claim or counterclaim in a lawsuit alleging that the Software infringes such Recipient s patent s then such Recipient s rights granted under the patent license above shall terminate as of the date such litigation is filed 6 DISCLAIMER SOFTWARE AND DOCUMENTATION ARE PROVIDED BY THE COPYRIGHT HOLD ERS AND CONTRIBUTORS AS IS AND ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABIL ITY OF SATISFACTORY QUALITY AND FITNESS FOR A PARTICU
118. ransfer boolean bool public boolean isDirectoryTransfer public void setThirdParty boolean bool public boolean isThirdParty public void setAttribute String name Object value public Object getAttribute String name 41 7 3 4 TaskGraph Figure 7 2 A TaskGraph can represent multiple levels of hierarchical DAG A TaskGraph provides a building block for expressing complex dependencies be tween tasks All significantly advanced applications require mechanisms to exe cute client side workflows that process the tasks based on user defined dependen cies Hence the data structure representing the TaskGraph aggregates a set of Exe cutableObjects Tasks and TaskGraphs and allows the user to define dependencies between these tasks In graph theoretical terms a TaskGraph provides the artifacts to express workflows as a hierarchical directed acyclic graph see Figure 7 2 A TaskGraph can theoretically contain infinite levels of hierarchy However practi cally it is constrained with the availability of resources memory on a particular system Listing 7 6 Interface definition for TaskGraph public interface TaskGraph extends ExecutableObject i public void add ExecutableObject graphNode public ExecutableObject remove Identity id public ExecutableObject get Identity id public ExecutableObject toArray public Enumeration elements public void setDependency Dependency dependency public Dependency getDepe
119. rdware per se is licensed hereunder If you or any subsequent transferee a Recipient institutes patent litigation against any entity including a cross claim or counterclaim in a lawsuit alleging that the Software infringes such Recipient s patent s then such Recipient s rights granted under the patent license above shall terminate as of the date such litigation is filed 6 DISCLAIMER SOFTWARE AND DOCUMENTATION ARE PROVIDED BY THE COPYRIGHT HOLD ERS AND CONTRIBUTORS AS IS AND ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABIL ITY OF SATISFACTORY QUALITY AND FITNESS FOR A PARTICULAR PURPOSE OR USE ARE DISCLAIMED THE COPYRIGHT HOLDERS AND CONTRIBUTORS MAKE NO REPRESENTATION THAT THE SOFTWARE DOCUMENTATION MODIFICATIONS ENHANCEMENTS OR DERIVATIVE WORKS THEREOF WILL NOT INFRINGE ANY PATENT COPYRIGHT TRADEMARK TRADE SECRET OR OTHER PROPRIETARY RIGHT 7 LIMITATION OF LIABILITY THE COPYRIGHT HOLDERS AND CONTRIBUTORS SHALL HAVE NO LIABILITY TO LICENSEE OR OTHER PERSONS FOR DIRECT INDIRECT SPECIAL INCIDENTAL CONSEQUENTIAL EXEMPLARY OR PUNITIVE DAMAGES OF ANY CHARACTER INCLUDING WITHOUT LIMITATION PROCUREMENT OF SUBSTITUTE GOODS OR 17 SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOW EVER CAUSED AND ON ANY THEORY OF CONTRACT WARRANTY TORT INCLUD ING NEGLIGENCE PRODUCT LIABILITY OR OTHERWISE ARISING IN ANY WAY OUT OF THE USE OF
120. redentials null x Assign this security credential to the task task setSecurityContext securityContext Note that for GT2 file transfers there is no need to assign a ServiceContact since the source and destination file names implicitly contain the remote machine names However for other providers it may be required to specify the ServiceContact Listing 7 22 Create a service contact ServiceContact service new ServiceContactImpl 127 0 0 1 task setServiceContact service Next create a TaskHandler and submit the task for execution Listing 7 23 Create a task handler TaskHandler handler new GenericTaskHandlerImpl try handler submit task catch InvalidSecurityContextException ise logger error Security Exception ise System exit 1 55 catch TaskSubmissionException tse logger error TaskSubmission Exception tse System exit 1 catch IllegalSpecException ispe logger error Specification Exception ispe System exit 1 catch InvalidServiceContactException isce gt logger error Service Contact Exception isce System exit 1 If it is required to monitor the status of the task desired in most interactive tasks then before submitting the task to a handler subscribe to the task for its status changes task addStatusListener this If registered to listen to the status notification of the t
121. rgl is argl gt 80 9 3 7 In line elements lt echo message arg2 is arg2 gt lt echo message arg3 is arg3 gt lt templatedef gt Template invocations can be made via the template element which ac cepts the name attribute plus any number of other arguments that are passed to the template lt template name sample argl valuel arg2 value2 arg3 value3 gt Alltemplates are re entrant as long as no external resources are involved Variables defined or overridden inside templates are considered local Elements can also be defined similar to templates Since version 0 22 this is prefer able to templates In line element definition is achieved using the element element The name attribute specifies the name that the definition will be bound to The argu ments attribute is a comma separated list of argument names that can be passed to the definition The vargs attribute if set to true indicates that the element should rather take a variable length list of unnamed arguments than a set of named argu ments The following element definition creates and element that sums all numbers from min to max lt element name sum arguments min max vargs false gt lt math sum gt lt for name var range min max gt lt argument value var gt lt for gt lt math sum gt lt element gt The following element definition makes use of un
122. s public Enumeration getSuspendedTasks public Enumeration getResumedTasks public Enumeration getCanceledTasks 45 7 3 7 GridResource The TaskGraphHandler provides a similar functionality as the task handler inter face However it has an additional responsibility of enforcing the dependency on the graph like task sets submitted to it It can be implemented as an advanced work flow engine coordinating the execution of tasks on corresponding Grid resources honoring the user defined dependencies Listing 7 10 Interface definition for TaskGraphHandler public interface TaskGraphHandler public void submit TaskGraph taskgraph throws IllegalSpecException InvalidSecurityContextException InvalidServiceContactException TaskSubmissionException public void suspend throws InvalidSecurityContextException TaskSubmissionException public void resume throws InvalidSecurityContextException TaskSubmissionException public void cancel throws InvalidSecurityContextException TaskSubmissionException public Task getAllTask public Enumeration getActiveTasks public Enumeration getFailedTasks public Enumeration getCompletedTasks public Enumeration getSuspendedTasks public Enumeration getResumedTasks public Enumeration getCanceledTasks The core module offers an alternate level of abstraction to Grid applications via its resource model The resource abstractions in cog core f
123. s priority over the JDKSDIR variable If not set the JDKs detected using the JDKSDIR variable will be used Points to the location of the CVS archive for the Java CoG Kit It should not be necessary to change the value of this variable Indicates the cog properties configuration file to be used for the test runs The default should work in most of the cases Represents a space separated list of hosts services sets to be used for the test ing Each item can either be a local file or a URL If a URL is encountered the wget utility will be used to attempt to download the list The format of such a list is detailed in Section 11 2 2 The maximum amount of time in seconds to wait for an individual test to complete The host list contains a list of machines and services that will be used for the testing Each line in the list represents a service Fields in each line are separated by a semi colon The fields are as follows Host name OS CPU Memory Service Port The name of the machine containing the service The operating system installed on the machine The type speed number of CPUs the machine has The amount of memory present The service type and version The service types that the framework recog nizes are gram and gsiftp The version is separated from the type by a space The port that the service is running on There can be multiple services of the same type on one machine given that the ports are differen
124. scheduler will choose a resource If a resource token see allocateHost is used the job will be executed on the resource that the token resolves to stderr A path to a file on the remote resource to which the standard error stream of the executable is to be redirected stdin A path to a file on the remote resource that will be redirected to the standard input of the executable stdout A path to a file on the remote resource that will be used to redirect the standard output stream of the job 9 8 19 gridTransfer Used to transfer a file on the Grid Arguments destdir The destination directory destfile The name of the destination file desthost The destination resource srcdir The source directory srcfile The source file srchost The source resource 9 8 20 host Describes one resource in the Grid definition Arguments cpus The number of CPUs that the resource has name The host name of the resource 9 8 21 if Allows conditional execution Section 9 3 4 provides additional details 98 9 8 22 9 8 23 9 8 24 9 8 25 9 8 26 9 8 27 9 8 28 ignoreErrors include invokeJavaMethod list append list prepend list size math equals Causes any errors generated by contained elements to be ignored Parses the contents of a file inserting the elements after the position of the include 9 8 23 element Arguments file The file to be included Invokes a method on a Java object Arguments
125. se deployment methods are appealing because they dont require any installation of OGSA or CoG by the client The deployment mechanism will install the necessary jars to run the certificate management tools This module consists of java files for the actual applets and of jsp and html pages to launch the Java Applets and jnlp files to launch the WebStart Applets These files contain parameters that can be changed by an administrator to change such things as the Certificate Authority MyProxy server location and background color of the applet These applets need to be signed explain how this is done A user must trust the entity that signed the applets warning jpg At the moment the jce jar is not signed properly by Bouncy Castle Once we use the latest jar it will be ok but for the moment you will get this dialog Simply press the X not the abort button jce jar Once the user grants the applets security access it will start The first step a new grid user will have to do is get credentials To do this you need to generate a certificate request and have it signed by your certificate authority Use the CertReqApplet to do this certreq jpg description of params goes here Once you get a response from your CA place it in your usercert pem as described in the email You can now use the certificate info applet to see your local credentials cerinfo jpg 116 Supposing you were going away on a business trip you may want to
126. sed language Extensive in formation about XML is available from http www w3 org XML XML has the advantage of a very strict and well defined structure The building block of a Karajan specification is an XML element The structure of Karajan specifications is very similar to that of structured languages such as C 72 Variables graph Canvas Java CoG Kit Graph Viewer File View Help 9 7 o Start Graph View v D start Y setvar gt point Breakpoint reached Y setvar amp echo setvar setvar z setvar v 167 nodes 176 edges Figure 9 4 A Breakpoint Was Reached Java or Pascal Most elements can also act as containers for other elements Each element performs a specific function or describes how contained elements relate to each other Variables can be used in Karajan to store temporary values values that can change and appear often in the specification as counters for iterators and so forth Defining Variables Variables can be defined explicitly by using the setvar 8 5 element which takes two attributes name and value The following example as signs the value blah to the variable named variable lt setvar name variablel value blah gt If the value attribute is not specified setvar will use the value of the default return variable This can be used for getting values from functions lt setvar nam
127. source extends GridResource MW Set ne most mame Or de Tomoi TeSOureE public void setHost String host public String getHost set the port of the remote resource public void setPort String port public String getPort She protocol indicates wt this sa i Grids iP eres ounce ON bames ounce or a custom FileResource public void setProtocol String protocol public int getProtocol EAT O II CI ATAR A public void setURL URL url throws IllegalHostException 48 public URL getURL set the appropriate SecurityContext public void setSecurityContext SecurityContext securityContext public SecurityContext getSecurityContext Authenticate with the resource and start the resource for command executions public void start throws IllegalHostException IllegalProtocolException InvalidSecurityContextException public void stop equivalent to cd command public void setCurrentDirectory String directory throws DirectoryNotFoundException public String getCurrentDirectory equivalent to ls command public Enumeration list public Enumeration list String directory throws DirectoryNotFoundException equivalent to mkdir and rmdir public void mkdir String directory public void rmdir String directory boolean force throws DirectoryNotFoundException equivalent to rm command public void rmfile String file throws FileNotFoundException equivalen
128. specific security context for every provider and a service contact for every individual service mapped by it Listing 7 12 Interface definition for ExecutionResource public interface ExecutionResource extends GridResource public void submit ExecutableObject executableObject throws InvalidProviderException public void remove Identity identity public void setQueue Queue queue 47 public Queue getQueue public void setServiceContact String provider int serviceType ServiceContact serviceContact public ServiceContact getServiceContact String provider int serviceType public void setSecurityContext String provider SecurityContext securityContext public SecurityContext getSecurityContext String provider public Enumeration getAllSubmittedTasks public void setAttribute String name Object value public Object getAttribute String name 7 3 9 FileResource The FileResource represents a file server typically FTP server or GridFTP server It provides convenient APIs for managing and accessing files hosted on this re source The FileResource abstraction offers a unified interface to all file hosting servers FileResource allows file transfers between the remote server and the local machine However third party file transfers cannot be performed by FileResource abstraction and requires the task model to accomplish it Listing 7 13 Interface definition for FileResource public interface FileRe
129. sted arguments Arguments to be passed to the constructor Iterates across a range of integer values Arguments from Used in conjunction with the fo attribute indicates the first value of the iteration name The name of the variable that is set with the current iteration value parallel If set to true the iterations will be executed in parallel otherwise they will be executed sequentially range Arange of the form n m describing all integers between n and m inclusive to Used together with the from attribute indicates the last value of the iteration Iterates across a sequence of discrete values Arguments dir Points to a directory The iteration will be performed by using the files in the specified directory in A comma separated list of strings that will be used as iteration values parallel If set to true the iterations will be executed in parallel otherwise they will be executed sequentially Causes an error to be generated Arguments message Sets the message associated with the error 97 9 8 17 grid Encapsulates a collection of Grid resources that are used by the scheduler to sched ule tasks 9 8 18 gridExecute Executes a job on the Grid Arguments args The arguments to be passed to the executable directory The directory on the remote resource to execute the job in executable A path to an executable on the remote resource host A resource on which the job will be executed If left empty the
130. submit tg catch InvalidSecurityContextException ise logger error Security Exception ise System exit 1 catch TaskSubmissionException tse logger error TaskSubmission Exception tse System exit 1 catch IllegalSpecException ispe logger error Specification Exception ispe System exit 1 catch InvalidServiceContactException isce logger error Service Contact Exception System exit 1 SCI If it is required to monitor the status of the task graph desired in most interactive task graphs then before submitting the task graph to a handler subscribe to the task graph for its status changes tg addStatusListener this If registered to listen to the status notification of the task graph implement the statusChanged function public void statusChanged StatusEvent event Status status event getStatus 29 logger debug Status changed to 57 status getStatusCode if status getStatusCode Status COMPLETED status getStatusCode Status FAILED i logger info Task Graph Done System exit 1 7 4 4 Executing a Hierarchical TaskGraph To create a hierarchical TaskGraph we assume that we have created three tasks and 1 TaskGraph task1 task2 task3 and tg Instructions for creating job submission and file transfer tasks and simple TaskGraphs are available in the previous sec tions 7 4 1 7
131. t target lt target gt Starts on the specified rendering target The following targets exists as of the writing of this manual swing 10 3 1 Uses the Java Swing graphical interface It is currently the only target that supports interactive editing html 10 3 2 Produces a HTML file together with any necessary images postscript 10 3 3 Renders the graph in an Encapsulated PostScript file remote 10 3 4 Can be used to forward the display of a graph to a remote viewer The API can be used as if working with local rendering and the remote renderers will take care of forwarding the events to the remote viewer It is unlikely that this target would be of any use when invoking the graph editor from the command line Additional options can be specified in the etc grapheditor properties file These options are generally particular to every target and thus explained in the target descriptions subsections 10 3 Using The Graph Editor This section will describe how each target of the graph editor can be used 105 10 3 1 The Swing Target When started in the Swing default target an empty frame shown in Figure 10 3 is displayed Three essential elements can be identified Menu Bar can be used for various operations like loading and saving of graphs The menus are dynamic which means that depending on certain factors 1 e the selected view their structure can change The View menu can be used to select the active view A snapsho
132. t The OS CPU and Memory fields serve only as informal items If a line cannot be parsed it will simply be excluded from the tests An example is shown below hot mcs anl gov Mandrake a 2 2 4 17 PIII 866 MHz x2 512 MB gram 2 4 2 5242 hot mcs anl gov Mandrake 7 2 2 4 17 PIII 866 MHz x2 512 MB gram 2 4 3 2119 hot mcs anl gov Mandrake a 2 2 4 17 PIII 866 MHz x2 512 MB gsiftp 2 4 2 6242 hot mcs anl gov Mandrake 7 2 2 4 17 PIII 866 MHz x2 512 MB gsiftp 2 4 3 2811 cold mcs anl gov Solaris 9 Sparc 900 Mhz x2 4 GB gram 2 4 3 2119 cold mcs anl gov Solaris 9 Sparc 900 Mhz x2 4 GB gsiftp 2 4 3 2811 114 12 Command Tools 115 13 grid cert request The certificate management module is a set of tools that make it easier for users to manage their certificates For instance there are tools to generate a certificate re quest store credential on MyProxy server view local credential renew certificates and revoke certificates Administrators have to choice of deploying these tools as signed Java Applets and or as signed Java WebStart Applets The benefits of signed Java Applet are integration into web pages however they required Java capable browsers Java singed WebStart Applets do not have this requirement The WebStart mechanism also has the advantage of caching the jars used by the applets One disadvantage to the WebStart mechanism is that the Applet will not integrate into web page Both of the
133. t showing the list of available views is shown in Figure 10 1 C Graph Wiew Layout O List View E Dwe ri ew O Tree View Fl Anti Aliasing Figure 10 1 Available views The graph view also supports a number of layout algorithms These al gorithms can be seen in the View Layouts menu when the Graph View is selected see Figure 10 2 Re Layout O Overview Persistence O Anti Aliasing Ci ini Lagat O Radial Layout O Non Overlapping Radial Layout a Level Layout O Flow Layout Figure 10 2 Layout algorithms Tool Bar contains icons representing various components that can be created in a graph canvas The icons may not be present if the selected view does not support editing View Display is the main panel where each view renders graphs 106 10 3 2 The HTML Target IT graph Canvas Java CoG Kit Graph Viewer IX File View Help Figure 10 3 An empty graph editor window When a graph is loaded or created each component can have a set of options or actions accessible through a context menu A snapshot of such a menu can be seen in Figure 10 4 The HTML target is used to render an HTML file that displays a graph In order to be able to properly display a generated HTML graph your browser will need to support JavaScript and transparent PNG images The HTML target has been tested with Mozilla 1 4 and up The output is optimized to produce a relatively small amount of data For each
134. t to yes a local copy of the Java CoG Kit sources will be used at each run In the later case the BUILDDIR variable must point to a valid Java CoG Kit source tree COMPILE If set to yes the Java CoG Kit code involved in the testing process will be compiled before each test run is executed If set to no no compilation will be performed The later option assumes that the testing module is already compiled For details about compiling a Java CoG Kit module please refer to Section 4 6 BUILDDIR Points to the directory where the Java CoG Kit is present or will be down loaded depending on the value of the LOCAL variable LOGFILE Points to a file where a detailed log of the testing process will be created HTMLOUTDIR Indicates the root of the tree where the HTML reports will be generated An index file will automatically be created in the specified directory 113 JDKSDIR JDKS CVSROOT COG_PROPERTIES HOSTLISTS TIMEOUT 11 2 2 The Host List Format Points to a directory containing a set of Java Development Kits that will be used for the testing Symbolic links can exist in the directory However each JDK version will only be used once regardless of the number of links pointing to it Only valid JDKs from that directory will be used therefore specifying something like usr local is safe Enumerates a set of Java Development Kits to be used for testing separated by spaces This variable ha
135. t to cp copy command public void getFile String remoteFilename String localFileName throws FileNotFoundException public void putFile String localFilename String remoteFileName throws FileNotFoundException equivalent to the mv command public void getDir String remoteDirname String localDirName throws DirectoryNotFoundException 49 public void putDir String localDirname String remoteDirName throws DirectoryNotFoundException changes the permissions on the file if authorized to do so public void chmod String filename int mode throws IllegalModeException InvalidSecurityContextException public void makeReadable String filename boolean readable throws FileNotFoundException InvalidSecurityContextException public boolean isReadable String filename throws FileNotFoundException public void makeWritable String filename boolean writable throws FileNotFoundException InvalidSecurityContextException public boolean isWritable String filename throws FileNotFoundException public void setLastModified String filename long time throws FileNotFoundException InvalidSecurityContextException public long getLastModified String filename throws FileNotFoundException Hf REURIGWES Me size Ol m6 wile meme public long size String filename throws FileNotFoundException N A ES tais ile CGE ON te PESQUPOS Y public boolean exists String filename throws FileNotFoundExcept
136. that side effects in Karajan have a limited scope an element cannot change the environment of its parent except for strictly controlled circumstances recursion is necessary to address a certain class of problems that cannot be solved otherwise Recursion is possible using element definition element The following definition computes the n th value in a Fibonacci series lt element name fibonacci arguments n gt lt if gt lt math le valuel n value2 2 gt lt then gt lt argument value 1 gt lt then gt lt else gt lt math sum gt lt fibonacci gt lt math subtraction from n value 1 gt lt fibonacci gt lt fibonacci gt lt math subtraction from n value 2 gt lt fibonacci gt lt math sum gt lt else gt if its lt element gt 82 9 3 9 Grid related Elements Karajan contains a series of elements that are divided into two main categories Grid resource description and configuration and Grid tasks Grid Resource Description and Configuration Elements scheduler 9 is used to select a scheduler type and specify various pa rameters for 1t Currently only one scheduler is available named default The arguments are type The type of the scheduler desired Only default is available at this time jobsPerCpu Sets the maximum number of tasks that the scheduler will allocate for one CPU maxSimultaneousJobs Sets the total maximum number
137. the edge components to the graph the first edge will go from nodel to node2 while the second one will go from nodel to node3 graph addEdge nodel node2 genericEdgel graph addEdge nodel node3 genericEdge2 tell the canvas what graph it is supposed to deal with canvas setGraph graph choose a view for the canvas canvas setView new GraphView create a frame that displays everything GraphFrame frame new GraphFrame root false 0 activate the frame frame activate start the main loop frame run Node components and edge components in short graph components can have properties These properties can be used to change the appearance or behavior of the components A list of meaningful properties for a GenericNode can be found in Table 10 4 The nodeComponent getPropertyValue String name and nodeComponent setPropertyValue String name Object value methods can be used to query modify the value of a property programmatically The property changes will show immediate results on the screen The example below will change the icon used for a node and de saturate it 111 10 6 Scalability When using the viewer with large graphs please note that each element node or edge will take a total of about 1 3KB of memory when fully rendered This means that a 50 000 node 150 000 edge graph will consume a total of about 300 MB 112 11 Testing Framework 11 1 Structure This c
138. tion gsiftp hot anl gov 1111 home amin output txt parallelstreams 1 tcpbuffer 32000 gt thirdparty true gt lt sequential gt lt target gt 68 9 Karajan 9 1 Installation Karajan is a Grid parallel task management language and an execution engine It aims to provide the scientific community with an easy to use tool to define com plex jobs on computational Grids while keeping scalability and offering some ad vanced features such as failure handling checkpointing dynamic execution and distributed execution Specifications in Karajan are defined by using a structured language based on XML and is extensible through Java The building block of the language is the element which loosely translates into an XML element container Various elements are included such as elements for parallel processing parallel iterators and Grid el ements i e job submission and file transfer Common tasks can be grouped by using templates and can be reused from multiple locations The execution engine in Karajan is based on an event model which allows effec tive separation between the specification and the runtime state Elements react to events received from other elements and generate their own events These events provide notification of status changes within the execution or can be used to con trol the execution of elements The complete runtime state is contained within the events which allows the eleme
139. tions to be published Online Available http www mcs anl gov gregor papers vonLaszewski grid middleware pdf 9 119 Index Acknowledgments 14 Administrative Contact 14 Bugs 10 Contact 14 Grid Scripting 69 GridAnt 63 cog setup 65 grid authenticate 65 grid copy 66 grid execute 65 Tasks 65 Karajan 69 Methods allocateHost 85 95 98 and 86 95 argument 87 95 checkpoint 93 95 contains 87 95 default 80 96 echo 76 87 96 element 81 96 elementDef 96 else 78 80 96 elseif 78 80 96 equals 87 97 executeJava 86 97 for 77 80 97 foreach 77 78 80 97 generateError 86 97 grid 83 93 98 gridExecute 84 93 95 98 gridTransfer 85 93 95 98 host 83 98 if 78 96 98 ignoreErrors 85 99 include 87 89 90 94 99 101 invokeJavaMethod 86 99 list append 87 99 list prepend 87 99 list size 87 99 math equals 78 86 87 99 math ge 86 100 math gt 86 100 math le 78 86 100 math lt 86 100 math product 86 100 math quotient 86 100 math remainder 86 101 math sqrt 86 101 math square 86 101 math subtraction 86 101 math sum 79 86 101 newJavaObject 86 99 101 nonCheckpointable 101 not 86 102 numberFormat 87 102 or 86 102 parallel 76 77 80 102 project 87 102 readFile 87 102 restartOnError 86 102 scheduler 83 93 103 securityContext 83 103 securityContextProperty 103 sequential 76 80 103 service 83 84 10
140. transfer url host 22 gt lt host gt lt grid gt 9 5 Include Search Path When the include 2 element is used the specified file is first searched in the directory where the main specification file is located If the requested file is not found the include search path is iterated until the file is found The include search path is defined in etc karajan properties The list of directories is separated by 89 9 6 Architecture 9 6 1 The Loading Process 9 6 2 The Execution Model colons A special token classpath indicates Karajan should try to find the file in the JVM class path By default Karajan starts with a very bare set of elements defined To access most of the elements described in this manual you should include the sysdefaults xml file in the beginning of the specification lt project name myproject gt lt include file sysdefaults xml gt lt project gt This section explains the main architectural characteristics of Karajan Only basic structural and syntactic validation is being performed at load time Se mantic validation is performed individually at execution time by each execution element A one to one mapping of the XML document elements and flow elements is done by using an element map which provides the correspondence between XML ele ment names and fully qualified flow element class names An exception applies to the include 2 element which immediately
141. ts pattern The pattern according to which the number is formatted The pattern has the syntax used by java text DecimalFormat value The value to be formatted See also http java sun com j2se 1 4 2 docs api java text DecimalFormat html Performs a boolean or operation Arguments Nested arguments The arguments Executes the contained elements in parallel The root container of a main workflow file Arguments name The name of the project Reads a file and stores the contents into a variable This function is intended for small files Arguments file The file to be read Restarts the execution of the contained elements if an error is generated Arguments times Indicates the maximum number of times the contained elements should be restarted in case of an error before the error is reported 102 9 8 49 9 8 50 9 8 51 9 8 52 9 8 53 9 8 54 scheduler service securityContext Specifies the type and parameters for the scheduler that is going to be used to schedule Grid tasks Arguments type Indicates the type of the scheduler Details about available schedulers can be found in Subsection 9 6 3 lt varies gt Attributes to be passed to the scheduler Defines a service for a resource Arguments type The type of the service Currently the accepted values are job submission and file transfer url A URL indicating the location of the service version A version label that is matche
142. vonLaszewski gestalt pat This article provides an overview of important influences developments and tech nologies that are shaping state of the art Grid computing What motivates the Grid approach What is a Grid What is the architecture of a Grid Which Grid research activities are performed How do researchers use a Grid What will the future bring A slightly differnt focus on middleware is presented in a paper entitled Grid Mid dleware 2 Article http www mcs anl gov gregor papers vonLaszewski gridmiddleware pdf Other CoG Kit related papers can be found at References von Laszewski http www mcs anl gov gregor bib 1 3 Intended Audience 1 This manual is intended for the intermediate Grid programmer that would like to access the Globus Toolkit functionality through Java We assume that the reader of this manual is familiar with Java If not general information about Java is available through the Web site at SUN Microsystems or at IBM SUN http java sun com IBM http www ibm com java l the bib file needs to be updated Also there is a collection at www cogkits org In general this manual serves as a basic introduction to a subset of functionality provided by the Java CoG Kit This manual does not explain every package class and method This manual is intended to show you that the Java CoG Kit provides an effective way of accessing the Grid through Java Developers are encoura
143. work flow engines that map complex business processes Nevertheless applications with simple process flows tightly integrated to work with the Grid technology can ben efit from GridAnt without having to endure any complex workflow architectures The philosophy adopted by the GridAnt project is to use the workflow engine avail able with Apache Ant and develop a Grid workflow vocabulary on top of it For an advanced Grid workflow management system please see Section 9 dedicated to the Karajan Workflow Framework 63 8 2 Installation 8 2 1 Download 8 2 2 Compile 8 2 3 Configuration GridAnt tasks are built on top of cog core see Section 7 thus they have the ability to submit Grid tasks multiple Grid providers GT2 and GT3 The Java CoG Kit GridAnt module can be downloaded from the Java CoG Kit CVS archive Instructions regarding the Java CoG Kit requirements and details on obtaining the Java CoG Kit sources are available in Section 4 We note that GridAnt is explicitly a client side library The current version of GridAnt provides support for GT2 and GT3 Hence in order to execute tasks against these implementations the reader is directed to install GT2 4 and GT3 0 2 For further details on installing the Globus Toolkit please visit the Globus Alliance Web page http www globus org To compile the GridAnt module change the directory to cog modules gridant and type ant dist This will compile GridAnt and all its dep
144. y appear due to community contributions Instead please quote the section title or make corrections by hand and FAX it to us Even better submit a corrected document as you can check out the manual through our CVS archive If you see a or a in the text there is no reason to send us a report on it It simply means that the section to which we refer has not yet been integrated in this manual Comments that indicate issues that needs to be don are included as footnotes New text that has not yet been reviewed may be in a different color Regular text is written using the Times font Code examples are highlighted in shaded blocks Interactive commands issued by a user in a shell are preceded with a gt at the beginning of the line In case interactive commands exceed the 79 character limit they are wrapped into the next line and are not proceeded by the gt character A backslash is included at the end of such lines to explicitly indicate that the command ins continued on the next line A CN Y EI AN LOTO A ha OnivinuUcd A next lines he wlicadine m blanks ine theme xt lanes are 10 le mio noned gt echo This is a new command References to variables or other important text that is part of a program or shell script is written in Courier To illustrate this on an example Hence a reference to the variable int a form our previous example uses also the Courier font Generic entities are wrapped between angle brackets

Download Pdf Manuals

Related Search

Related Contents

    Manual de instrucciones  WStt - Manual PDF  User`s Manual - Drake Software  G DATA ANTIVIRUS  

Copyright © All rights reserved. Failed to retrieve file