Home

IST-FP6-034286 SORMA D5.3 Integrated SORMA system & System

Contents

1. e Verify the installation This test application has three components e The ExchangeService which runs the exchange service e The Auctioneer which runs a continuous double sided auction e Two clients a seller and a buyer which ask offer on the auction To start the application follow the next steps 1 Be sure that the emm libraries are in the lib directory 2 Be sure that the following files are in your current directory e gmm properties basic configuration of the emm e Exchange properties configuration to launch the exchange service e client properties configuration for clients including the protocol executor 3 Launch the Exchange Service with the command ant f launch xml exchange You should see the message in the console Initializing Exchange Service Agent 4 Launch the protocol executor with the command ant f launch xml auctioneer Dapp args lt url gt lt port gt lt auction gt where e lt ur is the url to contact the exchange service e lt port gt is the local port used to listen to messages from the exchange e lt auction gt is the name of the auction You should see the following message in the exchange console Creating session lt auction gt where lt auction gt is the auction name as passed to the auctioneer Example ant f launch xml auctioneer Dapp args http localhost 8090 8099 theAuction 5 Launch the clients Seller
2. 83 4 8 2 SORMA SLA Administration GUI The SLA Administration GUI is packed with the SORMA ContractManagement component and provides the capabilities for a system administrator to observe the state of active and complete SLAs in the SORMA Market Infrastructure Here the state of an SLA corresponds to violations in the agreement and also which life cycle state the SLA is in 4 8 3 SORMA Security Management This section describes how to use the administrative GUI to perform management of users credentials and trusted identity providers The administrative GUI used here is the GAARDS UI developed by the caGrid a Starting the Administration UI 1 First download the Admin ui zip package from the SORMA DVD ivy sorma eu sormaproject security LATEST admin ui zip 2 Extract the ZIP files and cd into the admin ui directory S unzip admin ui zip cd admin ui 3 Lastly run the command to start the UI bin start ui This will bring up the UI as shown in Figure 47 File MyAccount Account Management Trust Fabric Group Management Window Help a Login Credential Manager My Groups Figure 47 Security initial GUI b Setting the Security Service Endpoint When the admin UI is first launched you need to set the URL endpoint of the Security services 1 Click on the Window menu and then select Preferences A new window will be displayed 2 Under Preferences select Grid User Management the
3. Property hosting config service exchange service exchange name service exchange builder service exchange class service exchange param channel service exchange config agent agent exchange name agent exchange builder agent exchange class agent exchange param channel agent exchange param cspaceHost agent exchange param cspacePort agent exchange param cspaceService agent exchange param conversation Default value service exchange ExchangeService edu upc cnds gmm p2pagent configuration builders ServiceBuilder edu upce cnds gmm coreservices exchange impl MarketExchangeService HttpChannel agent exchange ExchangeServiceA gent edu upc cnds gmm p2pagent configuration builders AgentBuilder eu sormaproject gmm coreservices exchange cspace imp ExchangeServiceA gent HttpChannel N A 8095 cspace services ProtocolExecutorAdapterService N A Description Includes the exchange service as a Service in the gmm middleware Name of the Exchange Service Define the factory for this service Specifies the communication transport to be used by the exchange service Adds the Exchange Service s agent to the gmm middleware Name of exchange service s agent Factory class used to create the agent Class that implements the exchange service s agent Specifies the communication transport to be used by clients to contact the exchan
4. 5 2 3 Sorma Ivy Portlet Developer Guide When implementing a JSR 168 portlet the compoent s build xml should import following standard template lt import file build bootstrap dir portlet build xml1 gt In addition to the standard SORMA directory structure a webapp directory needs to be present in the top level of your component s source tree that follows the standard JSR 168 layout of css html js jsp WEB INF In addition the component s build properties are extended to include the following app name gridrmPortlets app version 0 9 2 ivy organisation org gridrm Is this a JSR 168 or GridSphere WebSphere webapp jsr or gs use jsr jsr location of gridsphere source and build directory Override in your build properties gridsphere home home gms bin gridsphere 2 2 8 gridsphere build gridsphere home build The following are required to workaround a bug in the build script and will be redundant when the bug has been resolved For now just include the following 3 properties verbatim build dir basedir build build classes build dir classes build test build dir test The Gridsphere properties reference a Gridsphere 2 2 8 installation that must be installed before the portlet build will succeed To deploy the portlets to a local Gridsphere installation the following target is used ant deploy The standard targets are used for publishing the portlet
5. Integration tests are executed with the following requests java cp agentServiceClient jar lib eu sormaproject web client AgentServiceTest 147 83 30 215 16789 ConsumerEJSDLPrivateData 2 5 5 xml java cp agentServiceClient jar lib eu sormaproject web client AgentServiceTest 147 83 30 215 16789 ProviderEJSDLPrivateData 2 5 5 xml When the request are successful you will see on both consoles the received match document in form of XML File or receive a status report of the cause why it was not successful b Use pre defined EJSDLPrivate templates in case of an application based invocation Examples of pre defined EJSDLPrivate templates can be found on the SORMA DVD at e g src sorma trunk shared message test ProviderEJSDLPrivateData xml c Install and configure the Supply Modelling Portlet in case of a web based invocation The source code for the portlets can be found on the DVD at src sorma trunk intelligentTools provider providerPortlets c 1 Prerequisites e Installed Apache Tomcat Application Server Version 5 5 or higher e Installed Gridsphere Portlet Container Version 2 2 10 c 2 Installation Deploy the Technical Resource Modelling Portlet TRMP The portlet war file can be found on the DVD at ivy sorma eu sormaproject technical resource modelling portlet LATEST technical resource modelling portlet war Copy the war file to CATALINA HOME webapps and restart the
6. where mis properties is the direction to the properties file Alternatively you can type the following command line java cp marketInformation jar lib eu sormaproject coreMarketServices marketInformation Test mis properties where mis properties is the direction to the properties file d Configuration The following table describes the configuration parameters for the MIS see mis properties file within the configuration package of the download section depending if the MIS is using a database or an overlay Using the database allows to connect via HTTP to firewall protected machines and allows and easier setup of the system 66 Property overlay overlay port overlay bootport overlay IP overlay type db mis db subscriptio n delay db table db jdbcDriver db connection db filenames db username db createTable test test max_instance S test node test query test publish log log file Default value 20000 20000 147 83 34 222 ScribeSimulator 1000 MIS_Table org hsqldb jdbcDri ver jdbc hsqldb http p cmargenat ac upc e defines the connection direction du 8880 misdb sa false true false log4j properties 4 6 3 Payment Service Description Comments Configuration parameters when using an overlay as communication protocol Includes the exchange service as a the port must be open not service in the gmm middleware behind a firewall o
7. xmins xmins xmins xmins xmins xmins xmins xmins lt ValueExpression xsi type bean MarketMessage Type bean http www sormaproject eu message ejsdl beans gt lt bean currency xsi nil true gt lt bean paymentType gt EITHER lt bean paymentType gt lt bean clearingPrice gt 3 3333333333333335 lt bean clearingPrice gt lt ValueExpression gt lt Reward gt lt BusinessValueList gt lt GuaranteeTerm gt lt GuaranteeTerm ws Name IndividualPhysicalMemory ws Obligated ServiceProvider gt lt ServiceScope ws ServiceName myJobId1234 gt lt ServiceLevelObjective gt lt KPITarget gt lt KPIName gt jsdl IndividualPhysicalMemory lt KPIName gt lt CustomServiceLevel xsi type jsdl RangeValue_ Type jsdl http schemas ggf org jsd1 2005 11 jsd1 add http schemas xmlsoap org ws 2004 03 addressing gt lt jsdl UpperBoundedRange gt 300 0 lt jsdl UpperBoundedRange gt lt jsdl1 LowerBoundedRange gt 200 0 lt jsdl1 LowerBoundedRange gt lt CustomServiceLevel gt lt KPITarget gt lt ServiceLevelObjective gt lt BusinessValueList gt lt Importance gt 1 lt Importance gt lt Penalty gt lt AssessmentInterval gt lt TimeInterval gt PT30S lt TimeInterval gt lt AssessmentInterval gt lt ValueExpression xsi type bean Penalty Type jsdl http schemas ggf org jsd1 2005 11 jsdl1 add http schemas xmlsoap org ws 2004 03 addressing bean http www sormaproject eu message ejsdl bean
8. Again standard output is redirected to nohup out and the nohup command means that the gateway will continue running in the background when you log out of your terminal session Make sure that the gateway started successfully by inspecting nohup out e g cat nohup out fora static snapshot of the file or tail f nohup out to see changes to the file in real time nohup out will contain a number of messages ending with INFO Driver registration successful uk ac port dsg gridRM info providers gangliaGridRMv2 GangliaGridRMv2Driver 07 Sep 2007 21 47 07 uk ac port dsg gridRM info localLayerCore GridRMDriverManager registerDefaultDrivers INFO KKKKKKKKKKKKKKORTVER REGISTRATION PHASE COMPLETE KKEKKK KK KKK kk kk KK KKK kk kk Ak kk At this point the gateway is ready for action it has registered with the Tycho registry tested access to the internal database successfully loaded the default resource drivers and is waiting to receive client requests c Test the gateway installation The Simple Test Client can be used to test that the GridRM gateway is accessible over the network and to retrieve sample data c 1 Download and unpack the gridrm simple test client from the DVD ivy vendor sorma vendor org gridrm gridrm simple test client LATEST gridrm simple test client tar gz c 2 Edit the config properties file to point to your gateway the file is extensively commented c 2 Install and run the
9. LowerBoundedRange gt dl IndividualPhysicalMemory gt 1 IndividualDiskSpace gt sd1 UpperBoundedRange gt 45455 0 lt jsdl UpperBoundedRange gt sd1 LowerBoundedRange gt 3445 0 lt jsdl1 LowerBoundedRange gt lt jsdl IndividualDiskSpace gt lt jsdl1 Resources gt lt jsdl DataStaging gt lt jsdl FileName gt fileName lt jsdl FileName gt lt jsdl FilesystemName gt fileSysName lt jsdl1 FilesystemName gt lt jsdl CreationFlag gt append lt jsdl CreationFlag gt lt jsdl DataStaging gt lt jsd1 JobDescription gt lt jsd1 JobDefinition gt lt Penalty gt lt functionName gt DefaultPenalty lt functionName gt lt normalizationConstant gt 0 5 lt normalizationConstant gt lt Penalty gt A K 1 n Q Q 98 lt MarketMessage gt lt clearingPrice gt 33 0 lt clearingPrice gt lt contractId gt contractId lt contractId gt lt ConsumerContext gt lt PartipantId gt consumerlId lt PartipantId gt lt BidId gt bidId lt BidId gt lt ConsumerContext gt lt ProviderContext gt lt PartipantId gt providerlId lt PartipantId gt lt BidIid gt offerId lt BidId gt lt ProviderContext gt lt requestType gt MATCH lt requestType gt lt duration gt 321 lt duration gt lt serviceType gt WEBSERVICE lt serviceType gt lt paymentType gt EITHER lt paymentType gt lt MarketMessage gt lt MarketDefinition gt Example 3 Market message EJSDLMa
10. Smith URE Updated abstract Inserted developer user guide Fixed path references to the DVD Fixes and updates throughout document Table 1 Document History Table of Contents 1 Introduction 1 1 Structure of this Document 1 1 1 Annexed DVD 1 2 SORMA Actors 1 2 1 Market operator 1 2 2 Resource Provider 1 2 3 Consumer 1 3 Note on Security LA Explanatory Notes about WP5 deliverables D5 2 D5 3 D 5 4 2 Technical Description of the SORMA System 2 1 Integrated SORMA System Architecture 2 1 1 SORMA Architecture 2 1 2 SORMA Components 2 2 SORMA Components Behaviour 2 2 1 Resource registration 2 2 2 Batch resource service sale 2 2 3 Resource Usage 2 2 4 Special case Future Markets 2 3 SORMA Deployed System 2 3 1 SORMA Test beds 2 4 SORMA Software Development Methodology 2 4 1 Revision Control 2 4 2 Build mechanisms using Apache ANT and IVY 3 Resource Providers Manuals 3 1 Introduction 3 1 1 Pre requirements 3 2 Installation and Configuration Guide 3 2 1 Provider side intelligent tools 3 2 2 Economically Enhanced Resource Manager EERM 3 3 Providers User Manual 3 3 1 Resource Modelling GUI 3 3 2 Offer Submission GUI 3 3 3 EERM GUI 4 Resource Consumers Manuals 4 1 Introduction 4 2 Intelligent Tools Installation and Configuration Guide 4 3 User Manual 4 3 1 User Registration
11. and brokers utilize software components that interpret the information about the available resources from the resource fabrics and create the service requested User agents and brokers utilize an expressive market language while brokers and resource fabrics adopt a fabric specific language to allow brokers to acquire specific resources The grid market matches the requests with the advertised capabilities reports back to the requesting user and generates a Service Level Agreement SLA Through the data gathering interfaces along all abstraction levels and the powerful service description tools the Open Grid Market reveals information about demand and supply accurately Market or pricing mechanisms are the components that foster information exchange SORMA mechanisms ensure a more rigorous allocation of resources and self organizing resource management A simple management interface for virtual resources is provided the surveillance of the running nodes is made through a shared information gathering layer to simplify the brokers tasks of discovering and keeping track of nodes and their status 1 2 2 Resource Provider SORMA offers to the ICT resource owners outsourcing providers and designated utility providers a complex economically efficient and market based platform for the supply and trading of their available resources This includes computational as well as hardware idle or unused resources over the Open Grid Market for exampl
12. lt jsdl FileName gt textures inc lt jsd1l FileName gt lt jsdl CreationFlag gt overwrite lt jsdl CreationFlag gt lt jsdl DeleteOnTermination gt false lt jsdl DeleteOnTermination gt lt jsdl Source gt lt jsdl URI gt input_files_location pov textures inc lt jsd1 URI gt lt jsdl1 Source gt lt jsdl DataStaging gt lt jsdl DataStaging gt lt jsdl FileName gt povray lt jsdl FileName gt lt jsdl1 CreationFlag gt overwrite lt jsdl CreationFlag gt lt jsdl DeleteOnTermination gt false lt jsd1 DeleteOnTermination gt lt jsdl Source gt lt jsdl URI gt input_files_ location povray lt jsd1 URI gt lt jsdl1 Source gt lt jsdl1 DataStaging gt lt jsdl DataStaging gt lt jsdl FileName gt input_file lt jsdl FileName gt lt jsdl1 CreationFlag gt overwrite lt jsdl CreationFlag gt lt jsdl DeleteOnTermination gt false lt jsd1 DeleteOnTermination gt lt jsdl Source gt lt jsdl URI gt input_files_ location pov input_file lt jsdl URI gt lt jsdl1 Source gt lt jsdl DataStaging gt lt jsdl DataStaging gt lt jsdl FileName gt output_file lt jsdl FileName gt lt jsdl CreationFlag gt overwrite lt jsdl CreationFlag gt lt jsdl Target gt lt jsdl URI gt output_files location output_file lt jsdl URI gt lt jsdl Target gt lt jsdl1 DataStaging gt lt jsdl JobDescription gt lt jsdl JobDefinition gt lt ServiceDescriptionTerm gt lt ServiceProperties ws ServiceName myJobId1234 gt lt
13. touch CATALAINA_HOME webapps gridsphere WEB INF CustomPortal portlets consumerPortlets 53 4 3 User Manual This section explains how to use the SORMA GUI Intelligent Tools and API 4 3 1 User Registration For new users that want to apply to join the SORMA market they need to first apply fora SORMA account To do so they need to use a web browser and go to the URL for the SORMA user registration Going to the URL displays a web page as follows Mozilla Firefox File Edit View History Bookmarks Tools Help _ SORMA User Registration Page Please Fill in all the mandatory field Mandatory Field User ID First Name IT Last Name C_ Organization Phone Number TI Address IT a Address 2 Ir Y City State Zipcode Country Select Country Email Address Submit User has to register by filling out the form with their personal particulars All fields that are marked with are mandatory and therefore cannot be left blank User can choose to use a preferred user ID but subjecting to availability another user ID may be assigned instead Once the form is completed submit the application by clicking on the submit button Doing so will send the application information to a SORMA administration who will then validate the application information 4 3 2 Intelligent Tools GUI The user can use the Intelligent Tools GUI to specify a job description to an existing JSDL resource
14. 11 10T12 00 00 05 00 lt res finishTime gt lt res fixed gt lt res request gt lt res reservation gt lt jsdl JobDefinition gt lt jsdl JobDescription gt lt jsdl JobIdentification gt lt jsdl JobName gt JobName lt jsd1 JobName gt lt jsdl Description gt Description lt jsdl Description gt lt jsdl1 JobAnnotation gt lt jsdl JobProject gt lt jsdl JobIdentification gt lt jsdl Application gt lt jsdl Description gt Appl description lt jsdl Description gt lt jsdl Application gt lt jsdl Resources gt lt jsdl CandidateHosts gt lt jsdl HostName gt www sorma eu lt jsdl HostName gt lt jsdl CandidateHosts gt lt jsdl OperatingSystem gt lt jsdl OperatingSystemType gt lt jsdl OperatingSystemName gt Windows XP lt jsdl OperatingSystemName gt lt jsdl OperatingSystemType gt lt jsdl OperatingSystem gt lt jsdl CPUArchitecture gt lt jsdl CPUArchitectureName gt sparc lt jsd1l CPUArchitectureName gt CPUArchitecture gt 1 IndividualCPUSpeed gt sd1 UpperBoundedRange gt 3333 0 lt jsdl1 UpperBoundedRange gt sd1 LowerBoundedRange gt 233 0 lt jsdl1 LowerBoundedRange gt dl IndividualCPUSpeed gt 1 IndividualCPUCount gt sd1 UpperBoundedRange gt 5 0 lt jsdl UpperBoundedRange gt sd1 LowerBoundedRange gt 2 0 lt jsd1 LowerBoundedRange gt dl IndividualCPUCount gt 1 IndividualPhysicalMemory gt sdl UpperBoundedRange gt 433434 0 lt jsd1 UpperBoundedRange gt dl LowerBoundedRange gt 3455 0 lt jsdl
15. 4 3 2 Intelligent Tools GUI 4 4 Consumers Developer Kit 4 4 1 SORMA API 4 4 2 _SORMA Messages O O uo o Go D NN N kA m 11 11 12 15 15 16 17 17 18 19 24 24 25 27 27 27 27 27 31 46 46 47 49 53 53 53 54 54 54 58 58 59 SORMA Market Administrators Manuals 4 5 Introduction 4 6 Installation and Configuration Guide 4 6 1 Market Exchange Service 62 62 62 62 4 6 2 Market Information Service 66 4 6 3 Payment Service 67 4 6 4 Trading Management 69 4 6 5 Contract Management and Billing 72 4 6 6 SORMA Global Logging 74 4 7 How to Apply Security Infrastructure 4 7 1 Prerequisites 76 76 4 7 2 Deploying the Services 76 4 7 3 Configure to Trust the CA 78 4 7 4 Get Host Credential 78 4 7 5 Configure HTTPS for Tomcat 79 4 7 6 Testing the Security services with a Sample Client 80 4 7 7 Deployment of User Registration Web Form 81 4 8 SORMA Administrator User Guide 4 8 1 SORMA Logging GUI 82 82 4 8 2 SORMA SLA Administration GUI 84 4 8 3 SORMA Security Management 84 5 SORMA System developer guide 5 1 Getting Started with Ivy in SORMA 5 1 1 Configure the environment 88 88 88 5 1 2 Building and publishing an example component library 5 2 Developing for SORMA 5 2 1 Applications 89 92 94 5 2 2
16. Bee Let Administrators j wid iderttty S Om ate Cal Obey 2 OU c SECH eh beer AC been 16 Civelang JESO E TT ets iB Veit Arm P Add Aden Remove Admn Figure 51 Administrator User Interface 2 From the Administrator window you are allowed to List all the users with administrative access This will list the grid identities of all the entities with administrative access in the Administrators table at the bottom of the screen Grant a user administrative access Click the add Admin button This will bring up the Add Administrator window which will prompt you to enter the grid identity of the user you wish to grant administrative access to You can enter it directly or you can click the Find button to search for the user you wish to grant administrative access to Revoke a user s administrative access This can be done by select the user from the Administrators table and click the Remove Admin button To grant a user administrative access click the Add Admin button 87 5 SORMA System developer guide This section provides practical information that will be useful to developers contributing to the SORMA system code base Section 2 4 of this document titled SORMA Software Development Methodology provides an overview of the development methodology used in SORMA and should be read prior to proceeding with this section 5 1 Getting Started with Ivy in SORMA This section explains how to get started using SORMA s
17. Buyer are launched with the command ant f launch xml client Dapp args lt file gt lt url gt where e lt file gt name of the xml file used to describe the terms of the proposal e lt url gt url to access the exchange service You should see the following message in the exchange server console Message Handling negotiation from lt user gt where lt user gt is the user passed to the client Once two clients with matching bids are started the following message should appear in the console of each client 65 Response received Example ant f launch xml client Dapp args ask xml http localhost 8090 ant f launch xml client Dapp args offer xml http localhost 8090 4 6 2 Market Information Service a Pre requisites e Java 1 6 e Apache ant 1 6 and up to use the component launching scripts b Download The configuration files and the library which are necessary for the testing of the market information service can be downloaded from the SORMA DVD ivy sorma eu sormaproject marketInformation LATEST marketInformatio n config tar gz The Market Information Service package is available on the SORMA DVD ivy sorma eu sormaproject marketInformation LATEST marketInformatio n jar c Testing Go to the folder with downloaded marketInformation jar and run the following command java jar marketInformation jar mis properties
18. CandidateHosts lt KPIName gt lt CustomServiceLevel xsi type jsdl CandidateHosts Type xmlns jsdl http schemas ggf org jsd1 2005 11 jsdl xmlns add http schemas xmlsoap org ws 2004 03 addressing gt lt jsdl HostName gt 10 10 10 10 lt jsd1 HostName gt lt CustomServiceLevel gt lt KPITarget gt lt ServiceLevelObjective gt lt BusinessValueList gt lt Importance gt 1 lt Importance gt lt Penalty gt lt AssessmentInterval gt lt TimeInterval gt PT30S lt TimeInterval gt lt AssessmentInterval gt lt ValueExpression xsi type bean Penalty Type xmlns jsdl http schemas ggf org jsd1 2005 11 jsdl xmlns add http schemas xmlsoap org ws 2004 03 addressing xmlns bean http www sormaproject eu message ejsdl beans gt lt bean functionName gt DefaultPenalty lt bean functionName gt lt bean normalizationConstant gt 1 0 lt bean normalizationConstant gt lt ValueExpression gt lt Penalty gt lt Reward gt lt ValueExpression xsi type bean MarketMessage Type xmlns bean http www sormaproject eu message ejsdl beans gt lt bean currency xsi nil true gt lt bean payment Type gt EITHER lt bean paymentType gt lt bean clearingPrice gt 3 3333333333333335 lt bean clearingPrice gt lt ValueExpression gt lt Reward gt lt BusinessValueList gt lt GuaranteeTerm gt lt GuaranteeTerm ws Name IndividualCPUCount ws Obligated ServiceProvider gt lt ServiceScope ws ServiceName myJobId1234 gt l
19. Engine name Catalina defaultHost localhost gt lt Valve className org globus tomcat coyote valves HTTPSValve55 gt 4 In the HTTPS Connector step 2 above the default port used is 8443 If there is a need to use other port number i e 443 you also need to modify the Security component s web xml file located at CATALINA HOME webapps sorma security WEB INF web xml Change the defaultPort value highlighted in bold to the number you used in the HTTPS Connector configuration 79 lt web app gt lt servlet gt lt servlet name gt WSRFServlet lt servlet name gt lt display name gt WSRF Container Servlet lt display name gt lt servlet class gt org globus wsrf container AxisServlet lt servlet class gt lt init param gt lt param name gt defaultProtocol lt param name gt lt param value gt https lt param value gt lt init param gt lt init param gt lt param name gt defaultPort lt param name gt lt param value gt 443 lt param value gt lt init param gt lt load on startup gt true lt load on startup gt lt servlet gt lt web app gt Now the security services should be running To verify start a browser window and point it to https localhost 8443 sorma security services change hostname and port as appropriate and you should see a list of services Scroll through list and look for the SecurityManager service see the following screenshot Eile Edi
20. Gateway details Enter the GridRM Gateway Tycho name gridrm gateway portals ace Enter the Tycho mediator socket address socket localhost 3535 Enter the Tycho mediator group name sorma eerm Enter your Distinguished Name DN garry Enter your passphrase eccccee Save Details Find Q Lakhoo Next Previous fe Highlight all _ Match case Figure 19 Configuring Portlet behaviour using the edit mode d 5 Testing the GridRM Portlets A good test to check Portlet gateway communications is to try and subscribe for events using the Event Portlet see Figure 19 Select an event type and click the Save Details button If you receive the Initialisation of Tycho blocking API failed error message as shown in Figure 20 the Portlet was unable to contact a mediator If this occurs check that the standalone mediator is running either on the local machine or is available across the local network Check firewall rules to from the machine hosting the Portlet container and the machine hosting the standalone mediator A quick check of CATALINA_HOME logs catalina out will confirm that the standalone mediator could not be contacted Attempting to register with the standalone mediator at socket localhost 3535 with a hostname of kat acet rdg ac uk 21 59 16 852 EVENT Started SocketServer on 0 0 0 0 38569 Registering with Tycho registry 21 59 16 854 WARN ConnectException in socketSend
21. System and CPU Architecture are chosen via drop down menus CPU Speed CPU Count and Disk Space take integer values Filename represents the name of the JSDL file you want to create as shown Provide technical resource description Operating System Windows XP CPU Architecture x86_64 y Individual CPU Speed 2000 Total CPU Count 4 Total Disk Space 500000 Total Physical Memory 4096 Filename sample_host jsdl Create JSDL Figure 23 Sample values A click on the Create JSDL button creates the document After a document was created it can be viewed and opened for further editing To open a document click on the small pencil shaped icon in the upper left corner of the Technical Resource Modelling view The state of the will change to the view shown in Figure 24 46 Provide technical resource description Content of sample_host jsdl lt jsdl JobDefinition xmlns jsdl http schemas ggf org jsdl 2005 11 jsdl gt lt jsdl JobDescription gt sample_host jsdl lt jsdl JobIdentification gt lt jsdl Application gt lt jsdl1 POSIXApplication xmlns jsdll http schemas ggf org j sdl 2005 11 jsdl posix gt lt jsdl Application gt Figure 24 JSDL source view On the left hand side of the view all files contained in the JSDL Resource directory of the TRMP are listed currently sample_host jsdl is the only file If you click the name of a sp
22. Test parameters are passed to the unit testing code via propertysets that are defined in the component s build xml file more details below The component level build properties file contains component specific parameters required to build and deploy the component Required properties are given below with example values 7 http xmlunit sourceforge net 92 Short name that is used to refer to this component app name exampleComponent Used to identify the component s owning organisation to Ivy ivy organisation eu sormaproject Provides version information for the build and Ivy repository app version 0 2 Further properties can be included in build properties as required by the build xml The Go vm file defines e where in the repository the component will be published to i e organization and component names e the component version number read from build properties e which other SORMA components the component depends on and e which 3rd party libraries your component relies on as first level dependencies lt dependency gt An example ivy xml looks like this lt ivy module version 2 0 gt lt info organisation eu sormaproject module exampleComponent gt lt dependencies gt lt dependency org eu sormaproject name sSorma common rev latest integration gt lt dependencies gt lt ivy module gt The minimum build xml to build test and deploy a component is shown nex
23. The CA certificate is obtained in step 2 of section Configure to Trust the CA of this guide filename is d2e41d5e 0 This certificate is required to configure the security APIs to trust the CA Copy this file that contains the certificate to the certs directory cp d2e41d5e 0 certs 7 Finally run the sample client again with the service URL java eu sormaproject security examples LoginClient https lt serviceurl gt lt port gt sorma security services SecurityManager 4 7 7 Deployment of User Registration Web Form The registration form is written in PHP and can be deployed in any PHP application server To deploy the web form 1 First download the file registration php from the Sorma DVD src sorma trunk openGridMarket securityManagement userAccountR egistration registration php 2 Open the PHP file with a text editor and look for lt CHANGE ME SORMA COM gt as below lt php Sadmin_email lt CHANGE ME SORMA COM gt Ssuccess 0 3 Modify and add a proper email address that belongs to a designated SORMA administrator This email address will receive the registration request when someone submits an application lt php 81 Sadmin_email register sorma com Ssuccess 0 4 Copy the PHP file to the web directory of a PHP application server For example for an Apache web server copy the file to httpd htdocs sorma directory 5 Start
24. a specified time slot The Bidding Agent will forward this question to the EERM which will respond as appropriate in case of no will provide an alternative allocation for this concrete petition Following this process Trading Management will confirm deny the reservation If both parties arrive at an agreement the match is returned and the normal workflow of SORMA continues If not the 17 Trading Management component will try to perform a reservation with the next pair of matched bid offers Client Bidding Agent Trading Management Provider Bidding Agent Sends EJSDL wih time requirements Sends Resource Description without time specifications l ae Performs technical Matchmaking without having time into account l ye zez zc zi For every match In the list Sends the user technical and time requirements l pl l Accepts Denies the proposal l Return match i Return match 7 1 7 4 BE E EES Figure 8 interactions for reservation of resources 2 3 SORMA Deployed System Figure 9 shows how the components described in the previous sections are deployed from a logical point of view The figure shows the three main stakeholders Consumers Resource Providers and the SORMA Grid Market and the components that are required for each of them e Consumers will be running the client side Intelligent Tools and or having their o
25. can be found on the DVD under ivy vendor riotfamily mirror Local A private repository located on each developers machine Used to publish artifacts for testing locally as part of single developer integration and unit testing Cache Not strictly a repository the cache is located on each developer s machine and holds copies of artifacts that have previously been downloaded from each of the repositories listed above The cache removes the need to repeatedly download artifacts that have not changed from the remote repositories Apart from saving bandwidth the cache allows developers to continue working when they are offline e g using a laptop at an airport or on a train Further details of the build mechanisms can be found in Section 5 SORMA System developer guide http ant apache org ivy history latest release index html 26 3 Resource Providers Manuals 3 1 Introduction A Provider who wishes to offer their resources on the SORMA market needs to install and run the following components e Provider Side Intelligent Tools e Economically Enhanced Resource Manager EERM All components can be found on the SORMA system DVD as described in section 1 1 1 of this document 3 1 1 Pre requirements The Economically Enhanced Resource Manager and supporting components have been tested on the following platform e Linux kernel 2 6 or later e Sun Java Development Kit 1 6 A number of network ports need to be open thro
26. compile and execute the unit tests ant If we want to take a closer look at the dependency resolution that took place the following target can be called which results in an HTML file being written under the build directory e g Users gms workspace sorma trunk exampleIvyComponent build logs ivy eu sormaproject exampleComponent default html ant report Pointing a Web browser at the aforementioned HTML file results in something similar to the screenshot shown in Figure 52 When all unit tests pass the component s binary can be published to your local repository for local integration testing ant publish local The publish local command builds from source tests packages and then publishes the binary version of the component to the local Ivy repository that resides on your workstation Once in the local repository other components that rely on the component just published can be built and tested When all local integration tests pass the component s can then be published to the SORMA wide 90 integration repository for further testing in conjunction with other developers The command to achieve this is ant publish Any other SORMA components owned by other SORMA developers that reference this component in their ivy xml dependency file will now be able to retrieve the component from the integration repository as part of the their standard build Components may eith
27. document with the resource description and some economic auxiliary data The EJSDL document is sent to the provider s Bidding Agent and the Resource Provider is now ready to start a negotiation for the sale of the resource or service 15 2 2 2 Batch resource service sale Bidding Agent Trading Mgmt Contract Mgmt SLA Enforcement Payment Bidding Agent EERM T Economic Preferences H Send Omg JSDL T I Technical Requirements r gt i I I I 1 1 Send BidEJSDL U Create EUSDY a Match BidgOffers Send Matching 4 Se e Q Create Cont Return Contract Send Contract 1 Perform payment l l l l l l l I l l I l l It t I I l I l I l perform Payment Send Created WS Agreement EE oe ee pee ae pe Bee Figure 6 Resource Sale sequence diagram Figure 6 can be seen as the continuation of Figure 5 and shows the process that occurs upon the submission of an EJSDL document to the Provider s Bidding Agent Once the Provider s Bidding Agent gets the resource information with some economic data it creates the offer and sends it to the market concretely to the Trading Management component In an external and independent process the Client s Bidding Agent creates a bid from the Economical and Technical Preferences of the end user and also sends it to Trading Management Trading Manageme
28. gt lt jsdl Description gt Description lt jsdl Description gt lt jsdl1 JobAnnotation gt lt jsdl JobProject gt lt jsdl JobIdentification gt lt jsdl Application gt lt jsdl Description gt Appl description lt jsdl Description gt lt jsdl Application gt lt jsdl Resources gt lt jsdl CandidateHosts gt lt jsdl HostName gt www sorma eu lt jsdl HostName gt lt jsd1 CandidateHosts gt lt jsdl OperatingSystem gt lt jsdl OperatingSystemType gt lt jsdl OperatingSystemName gt Windows XP lt jsdl OperatingSystemName gt lt jsd1l OperatingSystemType gt lt jsdl OperatingSystem gt lt jsdl CPUArchitecture gt lt jsdl CPUArchitectureName gt sparc lt jsd1 CPUArchitectureName gt lt jsdl CPUArchitecture gt lt jsdl IndividualCPUSpeed gt lt jsdl UpperBoundedRange gt 1000 0 lt jsdl UpperBoundedRange gt lt jsdl LowerBoundedRange gt 100 0 lt jsd1 LowerBoundedRange gt lt jsd1 IndividualCPUSpeed gt lt jsdl IndividualCPUCount gt lt jsdl UpperBoundedRange gt 5 0 lt jsdl1 UpperBoundedRange gt lt jsdl LowerBoundedRange gt 2 0 lt jsdl LowerBoundedRange gt lt jsdl IndividualCPUCount gt lt jsdl IndividualPhysicalMemory gt lt jsdl UpperBoundedRange gt 1000 0 lt jsdl UpperBoundedRange gt lt jsdl LowerBoundedRange gt 500 0 lt jsdl LowerBoundedRange gt lt jsdl IndividualPhysicalMemory gt lt jsdl IndividualDiskSpace gt lt jsdl UpperBo
29. gt root lt username gt lt password gt lt password gt lt DatabaseConfiguration gt lt AutoCreate gt lt CASubject gt C EU O SORMA OU Security CN SORMA CA lt CASubject gt lt CAKeySize gt 2048 lt CAKeySize gt lt lifetime years 25 months 0 days 0 hours 0 minutes 0 seconds 0 gt lt AutoCreate gt 77 d Certificate Authority Subject Name It is important that the subject of the CA certificate is unique and meaningful to your deployment To set the subject of the certificate authority for your deployment edit the CASubject element The default value of C EU O SORMA OU Security CN SORMA CA is provided as an example 4 7 3 Configure to Trust the CA The Security component includes a CA that will be acting as the SORMA CA In order to use the security services with this CA you need to configure for the services to trust the CA To do this a utility script is provided Note that you should run the script as the user that starts up the Tomcat service so that the CA certificate will be place in the correct directory 1 Change to CATALINA HOME webapps sorma security bin directory 2 Run trustcert trustcert Succesfully configured Globus to trust the Dorian CA C EU O SORMA OU Security CN SORMA CA Succesfully wrote CA certificate to root globus certificates d2e41d5e 0 Succesfully wrote CA signing policy to root globus certificates d2e41d5e signing policy N
30. i Signature Registry ZA nformation g egistry CH Query lt lt Component gt gt B lt lt Component gt gt B Market Directory Market Information o Information Publish rs C Protocol Search Execution Figure 2 Core Market Services Open Grid Market Figure 3 details the internal components of the Open Grid Market and their interactions The Trading Manager provides the protocol execution interface and is used to process the bids received from participants and apply the market s allocation protocol A Match Making component allows the Trading Manager to match bids using technical and economical criteria Matches are submitted to the Market Exchange using the Trading interface while the corresponding Agreements are notified to the SLA Manager s AgreementNotification interface The Trading Manager also provides Market Information with anonymized information about the resource allocation using the InformationPublish interface The SLA Manager oversees the fulfilment of resource allocation agreements It uses the Provider s Tools Monitor interface to track the status of the job execution and resource provision It also receives the payment instruction from the application users Based on the agreement conditions and the actual resource utilization information it calculates the required payment and uses the Payment component s ExecutePayment interface to realize it Once the payment has occurred it informs the Provider s Tools of the payment
31. lt A A ER W o ne ne E SORMA Information Society Technologies A IST FP6 034286 SORMA D5 3 Integrated SORMA system amp System Manual Contractual Date of Delivery to the CEC 31 January 2009 Actual Date of Delivery to the CEC 31 January 2009 Author s TXT BSC UKA URE SUN UPC CU Work package 5 Est person months 93 Security public Nature final Version 0 9 5 Total number of pages 104 Abstract This deliverable describes the final integrated SORMA system providing both software and user guides The SORMA prototype is described from a technical point of view that covers the integrated architecture the behaviour of the main components and the integration test beds User guides for the three main categories of end users resource providers resource consumers and SORMA system administrators are provided that detail installation and configuration steps The SORMA prototype software is available on the DVD annexed to this document Keyword list optional Revision Table Revision amp Date Author Comment 0 1 10 12 2008 M Villa TXT Initial ToC 0 2 18 12 2008 M Villa TXT Refined ToC Action List 0 3 30 01 2009 M Villa TXT Added information about relationship with D5 2 and D5 4 section 1 3 Updated action plan 0 4 09 02 2009 M Villa TXT M Macias BSC N Borissov UKA G Smith Uni Reading Contribu
32. lt normalizationConstant gt 0 5 lt normalizationConstant gt lt Penalty gt lt PrivateData gt lt valuation gt 100 lt valuation gt lt strategy gt QStrategy lt strategy gt lt PrivateData gt A K u n Q Q 96 lt Bid gt lt requestType gt OFFER lt requestType gt lt participantId gt participantl lt participantId gt lt timeout gt 1344 lt timeout gt lt duration gt 321 lt duration gt lt serviceType gt WEBSERVICE lt serviceType gt lt paymentType gt EITHER lt paymentType gt lt Bid gt lt PrivateDefinition gt Example 1 EJSDLPrivate message lt BidDefinition bidId bid123 xmlns add http schemas xmlsoap org ws 2004 03 addressing xmlns http www sormaproject eu message ejsdl beans xmlns res http ws sormaproject eu eerm reservation xmlns jsdl http schemas ggf org jsd1 2005 11 jsd1 gt lt add EndpointReference gt lt add Address gt http localhost 18080 EERM lt add Address gt lt add EndpointReference gt lt res reservation gt lt res request gt lt res fixed gt lt res startTime gt 2009 01 10T12 00 00 05 00 lt res startTime gt lt res finishTime gt 2009 01 11T12 10 00 05 00 lt res finishTime gt lt res fixed gt lt res request gt lt res reservation gt lt jsdl JobDefinition id job123 gt lt jsdl JobDescription gt lt jsdl JobIdentification gt lt jsdl JobName gt JobName lt jsd1 JobName
33. missing economic data should be compensated for via the provision of well defined economic extensions E JSDL EJSDL The aim of the SORMA Message API is to provide a central API for all the messages between the SORMA components like Applications EERM Trading Management Market Information System Contract Management and BidGenerator Job EJSDLMarket JobData query job status Resource Provider EERM Application Provider SLAEnforcement job status request EJSDLPrivate match EJSDLMarket match EJSDLMarket request EJSDLPrivate nN amp agreement WSAgreement finalAgreement EJSDLMarket BidGenerator Market Information System Figure 42 Message formats between the SORMA components The main message formats produced and exchanged during the negotiation processes in SORMA are JSDL EJSDLPrivate EJSDL EJSDLMarket and WS Agreement JSDL The plain JSDL message format as proposed by OGF implements the standard JSDL data format In additional to the standard JSDL we added advanced reservation facility in form of start and end times to support providers which offer advanced resource reservations EJSDLPrivate The EJSDLPrivate message format extends EJSDL with additional category of PrivateData which consist of private information like valuation and strategy reported by the consumers and providers to the BidGenerator see Example 1 This in
34. provide a Web based administrative GUI for interacting with GridRM Gateways Note the Portlets are not required for successful EERM operation but they can be helpful for administering resources at a provider s site Operational details can be found in the resource provider s user guide d 1 Download from the SORMA DVD GridRM Portlets v 0 9 2 war file ivy vendor sorma vendor org gridrm gridrmPortlets LATEST gridrmPortlets war d 2 Pre requisites e Gridsphere 2 2 8 or a later 2 2 series installed and working e Sun Java 1 5 or later e A standalone mediator already configured and running you need to use a standalone mediator with the GridRM Portlets Do not try to an embedded mediator from within Tomcat or you will experience issues e Bi directional firewall access between the local Portlet mediator and the mediator for any gateway that you wish to connect to The following Network ports need to be open Port number wr tard Component ServerSocket Port description Container g Wide area communication over HTTPS between Tycho mediators on remote machines Tycho typically 8080 but any single port We typically stick to 8080 80 because most firewalls allow traffic to these ports to Em peddeg Mediator can be used Jetty pass by default Port must be open both ways Tomcat port Typically 80 but any Client Web browsers connect to this Tomcat port in order to log in to Gridsphere and access Apache can be used the Portlets T
35. test client against your gateway e g yabalka2 gridrm simple test client 0 9 2 gms simpleTestClient sh If you see the following at the end of the Simple Test client s standard output or log file then you have successfully managed to contact the gateway and perform a simple query Depending on whether you have just installed the gateway or have been using the gateway for some time you will have 0 or more resources reported In this example we have 0 because we have just installed the gateway and have not registered any resources yet 35 INFO main SimpleTestClient Start off by listing the resources that are already registered at the gatway INFO main SimpleTestClient Requesting the list of resources registered with gateway INFO main MonitoringImp1 Resolving address for gateway gridrm gateway obufki2 lan DEBUG main SimpleTestClient After requesting the list of registered resources DEBUG main SimpleTestClient Resource length is 0 INFO main SimpleTestClient The gateway does not have any resources registered INFO main SimpleTestClient If you experience problems connecting to the gateway please refer to the GridRM troubleshooting guide at https portals rdg ac uk gridrm index php GlobalLayerTroubleshootingGuide d Installing and Testing GridRM Portlets optional task An optional task is the installation of JSR 168 Portlets that
36. that the upload succeeded and displaying the namespace of the uploaded jar file Remember or copy the namespace string as you will need it in order to create a new conversation in the next step 5 Create the CDA Auction On the C Space Manager web page in the section named Create a Conversation modify the Namespace entry if the default value is different from the namespace you received when uploading the CDA protocol jar The other values should be left as they are Click Create Now the SORMA Trading Management should be successfully setup You can reset it by clicking the Reset button on the C Space Manager web page 4 6 5 Contract Management and Billing The Contract Management component is responsible for the generation runtime logic and enforcement of Service Level Agreements SLAs This entails the support of other SORMA components in respect to SLA related issues and the guidance of an SLA through its lifecycle It is implemented as a library with a web service front end This document describes how it can be deployed as a web service 72 This component is a downstream component of e SORMA Trading Management Component e SORMA Intelligent Tools and an upstream component of e SORMA Payment Component e SORMA Market Information e SORMA EERM There are two tasks involved in the installation of the web service and there is no particular order in which they need to be carried out e Runtime Configuration e S
37. the DVD docs A copy of the user guides src sorma A snapshot of the SORMA Subversion source code repository trunk sre third party Existing 3 party source code that was modified under SORMA ivy vendor 3rd party libraries binaries from the Ivy repositories ivy sorma All Sorma specific binary artifacts a snapshot of the SORMA Ivy Integration repository virtual machine images Virtual Machine images that can be used to test the SORMA and MOSIX systems the images come with components already in place but some configuration is still required to take into account e g network settings 1 2 SORMA Actors Brokerage of resources over the Grid will benefit Grid consumers resource owners outsourcing providers and new intermediaries The SORMA platform offers the necessary tools for all the actors providers consumers and market operators involved to join the market participate in transactions and IT resource realization SORMA is therefore a complete package making it possible for all types of users to participate 1 2 1 Market operator SORMA provides the Open Grid Market to perform the brokerage of resources over the Grid match the users and brokers requests and put them into contact with each other In a messaging framework user agents and brokers register their requests and capabilities using the SORMA messaging protocols Here software components create resource requirements specifications on behalf of the user
38. the following Credential cred new Credential BasicAuthenticationCredential bac BasicAuthenticationCredential new bac setUserld jdoe bac setPassword changeme cred setBasicAuthenticationCredential bac The authentication can then be carried out by invoking the following methods String uri https myhost SEET security SecurityManager AuthenticationClient authClient new AuthenticationClient uri cred SAMLAssertion saml authClient authenticate If the authentication is successful a SAML assertion will be return by the service This SAML assertion has an expiry time and is a proof that the user has been authenticated At any time in its validity period the SAML assertion can be presented to the Security service to exchange for a Grid proxy credential b Bid Submission API In order to request or provide resources in an automatic way the application providers consumers as well as the resource providers will use the BidGenerator s Web service interface submitBid Credential userCredential EJSDLPrivate ejsdlPrivate The user credentials like certificate user name and password are specified in the Credential object where the bid requests is submitted in form of an EJSDLPrivate message document described in next section 58 c Payment API The API specified here is provided by the Contract Management Client Interfa
39. the following target will cause the application to be packaged in a tar file that is then compressed using the gzip command resulting in a package with a tar gz extension lt target name package depends targz description package application in a tar gz file gt 5 2 2 Web Services When implementing a Web Service the component s build xml should import the following standard template lt import file build bootstrap dir webapp build xml gt The most important Web Service related targets are webapp builds a standard web application structure webapp war this target jars the web application and named like configured e g war file webapp tomcat setup This target copies specified setup files to the Tomcat s common lib directory deploy deploys a web application on tomcat undeploy undeploys a web application on tomcat To just build the webapp directory structure useful for inspection of the structure ant webapp To create a war file the package target is overridden in the component s build xml eg lt target name package depends webapp war description package application in a tar gz file gt To deploy the Web Service directly to the Tomcat manager the following target is used ant deploy To undeploy ant undeploy The standard targets are used for publishing the war file to the Ivy repository Oe ant publish 94
40. to authorize the utilization of the resource and also to uses the Core Market Service s PublishInformation interface to publish the final price The Payment component is a gateway for actual payment mechanisms for example credit cards online payment systems and virtual currencies Its main function is to control the execution of the payment transactions 13 lt lt Component gt gt Open Market ES Protocol Execution Match Making Li lt lt Component gt gt E lt lt Component gt gt E Trading Trading Manager Match Making Agreement Notification aH 2 Informatio k Publish O Execute Payments lt lt Component gt Payment lt lt Component gt gt IS SLA Manager SLA Query O Request Payment We Monitoring Inform Payment Figure 3 Open Grid Market Provider s Tools Figure 4 shows the internal component of the Provider s Tool The EERM provides the interfaces to submit jobs and monitor their execution It also issues requests to the Bid Generator component to offer a resource in the market as they become available using the BidSubmission interface The EERM must also be informed by the Open Grid Market about the authorization to use the resources using the InformPayment interface lt lt Component gt gt Open Market Bid Submission E lt lt Component gt gt E Bid Generator l
41. up the Apache web server and point the browser to http your_host port sorma registration php The registration page will be displayed as follows Mozilla Firefox Pie Edt View History Bookmarks Tools Help lt SORMA User Registration Page lolx Please Fill in all the mandatory field Mandatory Field User ID TI First Name eeo o o Last Name L Organization L Phone Number L Address Address 2 City KH State TI Zipcode Ir Country Select Country Email Address TI Submit 4 8 SORMA Administrator User Guide The SORMA System can support the Administrator in the following tasks thanks to a set of GUI offered by the following components e Monitoring logging GUI e SLA Administrator GUI e Security Administrator GUI 4 8 1 SORMA Logging GUI SORMA Global Logging provides a functionality to submit redirect special log messages from different and distributed SORMA components to a central logging instance log server Chainsaw is a GUI and server to receive and display log4 log messages Figure 45 shows the main GUI which displays the arriving messages 82 e ja Chainsaw v2 Log Viewer i AR File View Currenttab Connectto Help ap ug W er gea E eicon Dray amp Dro AM HSI KBW 091 089 000 136 hsi2 kabelbw de B b em x ae x 11 EK Receivers ES SE ID Timesta tevei Logger Message g 1 2008 09 21 eu sormaproje VISAGE TXT cli
42. 1 root root 1249 2009 01 29 16 49 d2e41d5e 0 rw r r 1 root root 111 2009 01 29 16 49 d2e41d5e signing policy rw r r 1 root root 1090 2009 01 29 14 17 myhost cert pem Ywr 1 root root 891 2009 01 29 14 17 myhost key pem 4 7 5 Configure HTTPS for Tomcat Communication with the security services has to be over a secured channel therefore it is mandatory to use HTTPS This section describes how to configure HTTPS with Tomcat 5 5 X 1 Use a text editor eg vi and open the Tomcat configuration file at CATALINA HOME conf server xml 2 Add a HTTPS Connector in the lt Service name Catalina gt section as below and update the parameters highlighted in bold appropriately with your local configuration lt Service name Catalina gt lt Connector className org globus tomcat coyote net HTTPSConnector port 8443 maxThreads 150 minSpareThreads 25 maxSpareThreads 75 autoFlush true disableUploadTimeout true scheme https enableLookups true acceptCount 10 debug 0 protocolHandlerClassName org apache coyote httpll HttpllProtocol socketFactory org globus tomcat catalina net BaseHTTPSServerSocketFactory cert root globus certificates myhost cert pem key root globus certificates myhost key pem NOTE The cert and key files are the host cert and key that were obtained in the previous section 3 Add a HTTPS Valve in the lt Engine name Catalina gt section lt
43. Figure 32 50 EERM Panel Job Successful See results below Figure 32 testing batch resources Show properties this opens in a new tab an example is shown in Figure 33 which is added to the general panel of the EERM In this tab the user can see some information about the resource o System description o Monitoring information only CPU at the moment o Timetable with the amount of resources reserved only CPU and memory at the moment Resources Clients Economic policies Properties of 10 0 2 200 Hardware Description Monitoring Information Number of CPUs 4 CPU last hour Memory 1024 0 MB Reservation info Reserved CPU 4 0 3 5 3 0 2 5 2 0 1 5 1 0 0 5 CPUs 0 0 ___ 1 233 500 000 000 1 234 000 000 000 1 234 500 000 000 1 235 000 000 000 1 235 500 000 000 time http www dummy user org Reserved Memory 2 000 000 1 750 000 1 500 000 Figure 33 showing resource properties 51 The other general section of the EERM panel is the one related to the clients information Figure 34 Here you can see the client identifier its reputation and its classification The two last columns are useful for applying some economic policies Resources Clients Economic policies Identifier Reputation Priority http www dummy user org 98 6 Bold Jg http www dummy client org 25 5 Platinum Z Figure 34 clients properties view Figure 35 shows the third s
44. Ivy build mechanism and describes e How to configure the build environment for SORMA e How to build an example component library and publish it to the Ivy repository The prerequisites to use Apache Ant and Apache Ivy are the installation of Java 1 5 or later Apache Ant 1 7 0 or later an SVN client an account on the SORMA SVN and repository server and an Internet connection 5 1 1 Configure the environment Before using the SORMA build mechanisms for the first time you need to configure the development environment by checking out the build bootstrap directory from the SORMA Subversion svn export https portals rdg ac uk sorma svn src build bootstrap NOTE The build bootstrap directory can also be found on the SORMA DVD at src sorma build bootstrap The build bootstrap contains e Standard Apache Ant build files that are shared across SORMA e Apache Ivy configuration settings e An example build properties file which has to be copied to your home directory S user home in Java Ant parlance directory The Ant files contained in build bootstrap are important because SORMA components import these into their own Ant build scripts The use of Ant imports means that the component build scripts are very simple and can contain as little as 7 lines as long as the component adheres to the standard SORMA build layout In order to finalize the configuration of the Apache Ant Ivy building environment the following steps are re
45. OSU CN langella 1 CN 13244 Sat Jun 02 00 20 35 EDT 2007 C U5 0 05U OU BMI OU caGrid Development OU Users OU OSU CN langella 1 Fri May 30 12 58 09 EDT 2008 LS Ka To register anew SORMA user click on Account Management gt Local Accounts gt Registration In the Search Criteria panel you can enter the specific attributes of the users that you want to remove modify For example to search for users who have suspended status select from the User Status drop down box and choose Suspended Then click on the Find User button to execute the search 3 If the search is successful the users that matched your criteria will be displayed To manage a particular user select the user and click on Manage User button or the Remove User button to remove the user e Managing Administrator Only users that have been granted administrative access will be able to access the administrative features of the Security service The administrative features include Account Management Managing Trusted Identity Providers and the ability to grant administrative access to users Administrative access can be managed through the administrator UI 1 From the top menu bar select Account Management gt Grid Account Management gt Administrators This will bring up the administrative window see Figure 51 86 T Admirestratort F eye LRI Mitps 140 254 80 185 8443 neuf aere eil AC t D Credential Globus Def auk Prory D
46. Returns the job identifier or null if the job can not be executed e getJobStatus gets the status of a job o Parameters Identifier of the resource that is executing the job Job identifier o Returns the status of the job running cancelled finished etc and a short explanation of the status 4 4 2 SORMA Messages In the field of Grid based scheduling there are several standardized description languages which we want to adopt in our scenario Prominent examples are the Job Submission and Description Language JSDL Anjomshoaa et al 2005 GLUE Schema Andreozzi et al 2008 Common Information Model CIM DMTF 2008 and WS Agreement Andrieux et al 2007 JSDL and GLUE Schema are standard and more general resource description languages currently used in Grid Middlewares for the technical description of provider resources and technical requirements of the consumer jobs CIM is a resource description model that aims to describe computing systems and their services in more 59 detail WS Agreement is an OGF standard web services protocol for negotiating SLAs between two parties designed for and applied mainly to bilateral SLA Negotiation mechanisms rather than to auctions Furthermore WS Agreement does not offer economic elements for auction based bidding in the level of detail desired like specified in the following JSDL GLUE and CIM aim to support the description of technical preferences for computing resources The
47. SM Universitat Karlsruhe TH Sun Microsystems Swedish Institute of Computer Science TXT e Solutions Universitat Polit cnica de Catalunya University of Reading Barcelona Supercomputing Center Centro Nacional de Supercomputaci n CARDIFF UNIVERSITY PRIFYSCOL Gd FZI a DOVT MYT AV NNT N The Hebrew University of Jerusalem Co Institute of Information Systems and Management TXT e solutions UNIVERSITAT POLITECNICA DE CATALUNYA ES The University of Reading 104
48. VariableSet gt lt Variable ws Name CPUArchitecture ws Metric jsdl CPUArchitecture gt Location gt JobDescription Resources CPUArchitecture lt Location gt lt Variable gt lt Variable ws Name CandidateHosts ws Metric jsdl CandidateHosts gt Location gt JobDescription Resources CandidateHosts lt Location gt lt Variable gt lt Variable ws Name IndividualCPUCount ws Metric jsdl IndividualCPUCount gt lt Location gt JobDescription Resources IndividualCPUCount lt Location gt lt Variable gt lt Variable ws Name IndividualCPUSpeed ws Metric jsdl IndividualCPUSpeed gt lt Location gt JobDescription Resources IndividualCPUSpeed lt Location gt lt Variable gt lt Variable ws Name IndividualDiskSpace ws Metric jsdl IndividualDiskSpace gt Location gt JobDescription Resources IndividualDiskSpace lt Location gt lt Variable gt lt Variable ws Name IndividualPhysicalMemory ws Metric jsdl IndividualPhysicalMemory gt Location gt JobDescription Resources IndividualPhysicalMemory lt Location gt lt Variable gt lt VariableSet gt lt ServiceProperties gt lt GuaranteeTerm ws Name CPUArchitecture ws Obligated ServiceProvider gt lt ServiceScope ws ServiceName myJobId1234 gt lt ServiceLevelObjective gt lt KPITarget gt lt KPIName gt jsd1 CPUArchitecture lt KPIName gt lt CustomServiceLevel xsi type jsdl CPUArchitecture Type xmlns jsdl http schemas ggf org jsd1 2005 11 jsdl xmlns add h
49. Web Services 94 5 2 3 Sorma Ivy Portlet Developer Guide 95 5 3 Further reading Appendix 5 4 Examples of SORMA Messages SORMA Consortium 95 96 96 104 Table of Figures Fioure 1 SORMA Technical Archi bere gen 12 Figure 22 Core Market EIER eege GE eare thee Rassen tia oasavteaasensanaaseers meas 13 Rene 3 Open Gnd Eege ed EE 14 Fig re 4 Providers Ten ams Seest eege choses eae aaea eege ZOE dE Gh oa page eat 14 Figure 5 resource provider registration Sequence diagram cccecceeescceseceseeeeeeeceeeseeeseeeeseenseeenseeees 15 Figure 6 Resource Sale Sequence SEENEN 16 Figure 7 job submission Sequence diagram sscceseesseeseceseeseceecesecseeesecesecceeseceaeseeeeseceseeneeeaeeaeens 17 Figure 8 interactions for reservation REIH gea deed am Warned eiaess 18 Figure 9 high level components deployment Vis wy eietiergenr gert Ee dee Eve 19 Figure 10 Deployed VMs in PC E 20 Fig r 11 Deployed VMs in POZ scsssassdarersnestsh oxdaacedvansetannadedestaanesiztaaiesatanaiiaiabadeadalioasstimsaensesnanstiont 21 Figure 12 CPU Activity of the virtual machines during one bor 22 Figure 13 Traffic in the virtual machines during one bor 23 The software development process in SORMA is executed using a well defined methodology The methodology contains the following parts 24 Fig r 14 Portlet Group Manage rieurs eion ieit iki een EE ESEE EEE ear AK ES o eNi 37 Figure 15 New Resource Monitoring grou
50. ains official SORMA releases that are made available via the SORMA Website Releases are generated automatically in response to a request from the developer leading the release effort The release is built and packaged directly from Subversion and takes transitive dependencies from the Vendor repository The Release repository is read only via HTTP Developers are not able to write artifacts to this Ivy repository directly Integration A repository shared over the network that all developers use to publish retrieve components as part of the integration testing efforts A copy of the integration repository contents can be found on the DVD accompanying this report under the ivy sorma directory Vendor SORMA currently has 3 separate vendor repositories that contain 3rd party libraries These include a Sorma vendor All developers have read write access so that they can publish new libraries jars as required by their components A snapshot can be found on the DVD under ivy vendor sorma vendor b Maven mirror A partial mirror of the public Maven repository with name space and other meta data issues associated with the public Maven repository resolved Frequent down time and incorrect metadata entries at the public Maven repository prompted consortium to import and host our own partial mirror A snapshot can be found on the DVD under ivy vendor maven mirror c Riotfamily mirror A mirror of the Riot Family project s Ivy repository A snapshot
51. allows the management of virtual machines in desktop PCs The implementation of the LRM interacts with Xen for the virtualization of the PC The Local Resource Manager tool provides a menu guided access to the functionalities of Xen which allows the administrator to perform configuration setup and maintenance of the virtual machines Once all virtual machines are set up their status can be checked This is shown in the following figures where information concerning the different virtual machines set up for the SORMA developers can be seen In Figure 10 the information obtained from the LRM is shown for PC1 Figure 11 shows the virtual machines in PC2 It can be seen that information about the operating system status and assigned MHz of the virtual machines is given The column QoS type further indicates one of the 4 operating policies which can be assigned to the virtual machines All of the 4 policies assure to the virtual machines the MHz assigned if this CPU usage is needed by the VMs The QoS levels are then different in terms of additional CPU however which is then assigned to the VMs from the unused idle CPU resource LRM Client Beta 0 2 userl connected to 147 83 30 2 Action hr for help VM ID OS share Status Price QoS MHz QoS type pc2 linux 0 05 running 0 12 500 fixed pc3 Linus 0 04 running 0 10 400 fixed pc4 linux 0 04 running 0 10 400 fixed DCH linux 0 04 running 0 10 460 fixed pc6 linux 0 04 running 0 10 400 fixed pc7
52. and and supply modeling and associated portlets that are required at the consumer and provider sides of the market e openGridMarket contains subdirectories for components required for the SORMA market including Trading Management The Economically Enhanced Resource Manager EERM Matchmaking Payment services Security and Contract Management e pilotApplications contains code and test data for the Correlation Systems pilot application e yresourceFabrics contains subdirectories for components related to the management of resource fabrics e shared contains subdirectories for libraries and services that are shared across multiple SORMA components for example SORMA message protocols shared log4j configuration files and the logging service used to demonstrate internal SORMA state for demonstration and debugging purposes Each SORMA specific component s source code is packaged under the eu sormaproject name space 2 4 2 Build mechanisms using Apache ANT and IVY Apache Ant and Apache Ivy are build and dependency tools respectively that are used to support a common and consistent methodology for the SORMA development process Apache Ant is a Java based build tool allowing development of software across multiple platforms Ant provides flexible configuration mechanisms for the build process by specifying so called targets that are implemented using an XML syntax Targets are defined for tasks such as e g build directory prepara
53. ay Traycing lt jsdl Description gt lt jsdl1 JobAnnotation gt jobPR lt jsdl JobAnnotation gt lt jsdl JobProject gt RT lt jsdl JobProject gt lt jsd1 JobIdentification gt lt jsdl Application gt lt jsdl ApplicationName gt Pov Ray lt jsdl1l ApplicationName gt org jsd1 2005 11 jsdl1 lt POSIXApplication xmlns http schemas ggf org jsdl 2005 11 jsdl posix gt lt Executable gt povray lt Executable gt lt Argument gt 1I input_file lt Argument gt lt Argument gt 0 output_file lt Argument gt lt Argument gt Ws width lt Argument gt lt Argument gt H height lt Argument gt lt POSIXApplication gt lt jsdl Application gt lt jsdl Resources gt lt jsdl DataStaging gt lt jsdl FileName gt colors inc lt jsd1 FileName gt lt jsdl CreationFlag gt overwrite lt jsdl CreationFlag gt lt jsdl DeleteOnTermination gt false lt jsd1 DeleteOnTermination gt lt jsdl Source gt lt jsdl URI gt input_files_location pov colors inc lt jsdl URI gt lt jsdl1 Source gt lt jsdl1 DataStaging gt lt jsdl DataStaging gt lt jsdl FileName gt finish inc lt jsdl FileName gt lt jsdl CreationFlag gt overwrite lt jsdl CreationFlag gt lt jsdl DeleteOnTermination gt false lt jsdl DeleteOnTermination gt lt jsdl Source gt 99 lt jsdl URI gt input_files_location pov finish inc lt jsdl URI gt lt jsdl1 Source gt lt jsdl1 DataStaging gt lt jsdl DataStaging gt
54. behind a proxy 5 1 2 Building and publishing an example component library Checkout the exampleIvyComponent from SVN trunk or copy it from the DVD src sorma trunk exampleIvyComponent to a directory where you have read write permissions The directory structure will look like this doc java docs of the component src source code test source code of unit and integration tests build properties common build properties of a component build xml contains ant based configuration with standardized and custom for building a component ivy xml contains the ivy based configuration of dependencies and publishing information LICENSE txt contains license information README txt contains specific information to building and execution of a component To finding out what ant targets are available to use execute following command ant p The main SORMA targets targets clean Clean up all files and directories that were generated by the build clean cache Clean the ivy cache clean ivy Clean the ivy installation clean local Clean the current module from the local repository compile Compiles the source compile tests Compiles the unit test source enviro Tests that the user s environment is configured to meet SORMA s highlevel requirements package Packages the component into a jar file that is written to the component s build directory 89 publish Publish this
55. c private hidden Select portlets Select portlets that will be made available to the group Users in this group will have the chance to add these portlets to their layout In addition required role levels may be associated with the portlets Offer Generation Portlet Offer Generation Status USER A portlet to view and unregister resources at a GridRM Gateway Query core attribute values A portlet to query standard resource attributes Subscribe and receive GridRM events Add resources to a GridRM Gateway A portlet to add resources to a GridRM Gateway Job Submission Portlet Submit a new job to the Grid Market M List GridRM Gateway resources A portlet that receives events from GridRM gateways CO Find Q Lakhoo Next Previous Highlight all C Match case Figure 14 Portlet Group Manager Figure 15 shows the Resource Monitoring group after it has been created Note the edit users link Administrators can assign particular groups to users so that uses can access the group when they log in to Gridsphere Further groups can be created by clicking the Create new group button and existing groups can be deleted using the associated Delete button 37 000 GridSphere Portal LL ot http portals rdg ac uk 8449 gridsphere gridspheryy Lei Googe Q 4 Free BE framework Resource Monitoring Welcome i Resource Monitoring Portlet Group Manager Core GridSphere gridsphe
56. can use the following command to determine which process is listening on a port gridrm kat bin gridrm tycho gateway 0 9 3 1 fuser n tcp 54321 54321 tcp 22198 gridrm kat bin gridrm tycho gateway 0 9 3 1 ps auxw grep 22198 gridrm 22198 0 0 0 6 1087904 23452 pts 0 lib hsqlidb 1 8 0 7 jar org hsqldb Server gridrm 22275 0 0 0 0 1952 644 pts 0 Sl S4 13 26 0 00 java cp 13 42 0 00 grep 22198 First we used the fuser command to output the ID of the process bound to port 54321 Then we used the ps command with the offending process s ID number 22198 to get details of that process Looking at the output we see that the process was previously started by us wait a minute This is a database from an earlier experiment with a GridRM Gateway We shut the gateway down earlier but forget to stop the database We don t need the remaining database process so let s shut it down so that we can reuse the port We decide to pass the offending process process ID to the ki11 command gridrm kat home users rdg gms kill 22198 gridrm kat home users rdg gms ps auxw grep 22198 gridrm 22306 0 0 0 0 1956 648 pts 0 S 13 54 0 00 grep 22198 Notice that we did not need to use kill KILL which would have killed the process without letting it first clean up 34 b 2 Starting the GridRM Gateway Once the gateway s database is running the gateway can be started nohup gateway amp
57. ce Payment actions are carried out by invoking the following method PaymentResponse makePayment PaymentDetails paymentDetails The PaymentDetails message document encapsulates the information required by the SORMA Payment Component to instigate a monetary transaction The consumer does not supply the price here as this is calculated by the SORMA Contract Management Component so that any SLA related violations which may have occurred can be considered The invocation of this method will return a PaymentResponse message document which describes one of three possible outcomes of the transaction Namely 1 Success 2 Failure 3 Not Required If the payment was successful the response acts as a receipt of the payment detailing the amount paid and when If the payment was unsuccessful the response will contain information describing why the payment failed In some rare circumstances a payment will not be required this will only occur if the calculated price is zero which in turn can only occur if all terms in the SLA were violated Further details of the Payment protocol are presented in section 4 6 3 d Job Submission amp Monitoring API The APIs provided by the EERM are WSJobManager this service implements the functionalities related with the batch jobs e sendJob sends a job for being executed in the resource fabrics o Parameters Identifier of the resource acquired to execute the job JSDL with the job description o
58. component in the SORMA Integration Ivy Repository publishes binary version only for use by other developers publish local Publish this component in the local ivy repository on this machine report Generate a report of dependencies processed by Ivy point your Web browser at build logs test Execute Unit tests The targets are fairly explanatory clean local and clean cache are useful for cleaning out local artifacts They has to be called when someone has published a new version of their component without changing the version id before publishing The clean target is used to clean up all the artifacts generated by a component s build The enviro target currently tests to see if the java version is 1 5 or 1 6 Over time the environment tests will evolve as further prerequisites required for SORMA are identified across components An example environment check ant enviro Look inside the iv ml for the list of dependencies the component requires In this example only the sorma common library is required as shown in the following example lt ivy module version 2 0 gt lt info organisation eu sormaproject module exampleComponent gt lt dependencies gt lt dependency org eu sormaproject name Sorma common rev latest integration gt lt dependencies gt lt ivy module gt Use the following command to prepare the build directory resolve dependencies listed in the ivy xml compile the source and then
59. d Provider s Tools for trading the Open Grid Market that provides the context upon which the exchange of resources takes place the Core Market Services which provide the supporting services for the actual trade to occur and the Security Manager which provides the cross system security mechanisms On the consumer s side of the market the Consumer s Intelligent Tools provide the grid applications with the interface SubmitBid to express its preferences both technical and economical for resources To fulfil the application s requirements the Provider s Intelligent Tools use the aggregated market information like price ranges provided by the Core Market Service s Information Publish interface and the Trading interface to submit a bid If this bid is successful the Grid Application receives the credentials to access the resource obtained by the market and uses the SubmitJob interface of the Provider s Intelligent Tools to initiate its utilization The Provider s Intelligent Tools also provide interfaces to monitor the execution of jobs monitoring and to inform of the valid resource allocations The Core Market Services component provides interfaces used by the intelligent tools to Search for markets query for aggregated information about the market and submit bids Trading The Security component provides interfaces for authentication and digital signature to Core Market Services which use them to provide a secure trade environment B
60. d and write disk access In order to assess the resource usage of the virtual machines we have recorded the activity on each of the virtual machines in each of the three time granularities For illustration the next figure shows the CPU resources used by each virtual machine of PC2 during a one hour timeframe is shown It can be seen that the CPU usage of the virtual machines has been quite low in this hour and that the virtual machines could easily cope with the activities made by the deployed components 100 T T Domain 0 90 POO riss 80 70 60 a Den 40 30 20 10 00 00 10 10 00 1020 00 10 30 00 10 40 00 10 50 00 110 Figure 12 CPU Activity of the virtual machines during one hour The next figure shows the traffic per virtual machine in PC2 during a one hour timeframe Some of the peaks which are not high compared to the available bandwidth could be due to the downloading of software The communication within the deployed distributed application is characterized up to now by the exchange of short and not too frequent messages between the components Up to now no performance problem with bandwidth was identified Given the PC s network connection bandwidth larger traffic could take place and should not produce bandwidth problems Extreme scenarios like heavy concurrent communication activities in all the ten virtual machines which could potentially bring communication problems have not yet been part of the u
61. d see your machine in the main panel g Ready Your machine is now ready to accept and execute jobs if you connect via web to the EERM and see your machine in the main panel EERM Panel Mozilla Firefox lt 2 gt Archivo Editar Ver Historial Marcadores Herramientas Ayuda d E http 1147 83 30 246 1500 7 a7 AJEERM local amp J EERM PC5 Barcelona ai GG Supercomputing SORMA EERM Panel Resources Clients Attached Resources 10 0 2 200 4 CPU 1048576 0 MB Figure 21 EERM main panel 45 3 3 Providers User Manual This section describes how Resource Provider should use the SORMA components in order to be able to trade their resources on the SORMA Grid market The components involved are e Intelligent Tools to model resources and bidding strategies and to submit offers e EERM to monitor job executions resource usage etc 3 3 1 Resource Modelling GUI The Technical Resource Modelling Portlet TRMP is used to create JSDL based resource definition files so to describe a specific host system Figure 22 Provide technical resource description Operating System CPU Architecture x86 Individual CPU Speed Total CPU Count Total Disk Space Total Physical Memory Filename Create JSDL Figure 22 Standard view of the Technical Resource Modelling The values for Operating
62. der a simple case to get you up and running with the SORMA system as a whole Download Tycho GridSAM from the DVD ivy sorma eu sormaproject TychoGridSAM LATEST TychoGridSAM tar gz Unpack the file Create a new text file call it for example custom properties and update the properties to reflect the settings you used for your GridRM Gateway and EERM In order to utilise the Ganglia agent you need to make sure that gridrm driver type is set to ganglia and gridrm driver port is either set to 0 for the ganglia default of 8649 or set to the appropriate port number for your Ganglia daemon An example of the config file is shown next The name of the GridRM gateway that this instance of the TychoGridSAM associates with gridrm gateway name gridrm gateway localhost The driver that the GridRM Gateway needs to use to monitor the resource that this Tycho GridSAM points to gridrm driver type ganglia gridrm driver type snmp gridrm driver type gridrm_ agent gridrm driver port 0 Set the tycho mediator url if you want to connect to a standalona mediator Default is to use an embedded mediator Examples for pointing to a standalone mediator tycho mediator url socket dom03 lister acet rdg ac uk 3535 tycho mediator url tycho mediator url socket 10 0 2 200 3535 The tycho mediator group name must be set correctly in order for th mbedded mediator to work Make sure you set this value to the same group name that is bei
63. e as services Sorma provides the means for resource providers to join and leave the market by enabling the resource fabrics to register their presence in an Information Gathering Infrastructure Here brokers collect the messages about the available resources and their status that the resource fabric node sent to the Information Gathering Infrastructure are able to interpret the information correctly and communicate back to the user agent when a broker has received an accept on an offer from the user agent it communicates to the resource fabric market reservation service in order to acquire resources in accordance with the generated SLA The SORMA platform provides resource owners with economically sound sustainable and customizable business models by providing methods and tools e to express the business model of the resources owners e for capturing users reserve prices for resources e for determining the bidding strategy consisting of various economic pricing models e to estimate and monitor the quality of the resource management 1 2 3 Consumer SORMA offers an economically efficient and market based platform to the consumers of Grid computing power or Grid storage infrastructures for the identification and acquisition of the needed resources Acquisition and trading support is provided for low level resources as well as for higher level services with dependencies for applications requiring continuous real time allocation of
64. e code directory has 3 subdirectories branches Contains component code or component level code branches e g for release branches developer trials branches for non trivial bug fixes tags For tagging a particular release of SORMA https portals rdg ac uk sorma svn sre http ant apache org 4 http ant apache org ivy 5 http cruisecontrol sourceforge net http www junit org http www junit org taxonomy term 8 http www bugzilla org 24 e trunk The latest development version of the SORMA source code and support files are located under this directory NOTE The trunk and build bootstrap directories are located on the accompanying DVD under the srce directory The SORMA SVN is available at https portals rdg ac uk sorma svn src The structure of the code trunk SVN folder conforms to the layers of the SORMA architecture e common a library that provides functionality required by all components across the whole of SORMA For example base logging classes and custom unit testing assertions coreMarketServices contains subdirectories for the following components marketExchange marketInformation logging and marketDirectory e exampleIvyComponent an example component that is used to help introduce developers to the build and component publishing mechanisms e aintelligentTools contains subdirectories for components that comprise the intelligent tools including agent services the bid generator dem
65. e upper left corner of the window The Portlet will now return to the previous view with a file selected for further editing You can see the name of the file you are currently editing on top of the Job Description frame see Figure 38 33 Provide job description Current File sample_host jsdl Job Description Job Name Description Job Annotation Job Project m Data Staging Filesystem Name Filename Source Target Creation Flag append Delete on Termination 7 Application Application Name Application Version POSIX Executable Arguments Create JSDL Figure 38 Standard view with file selected To add job description information fill in the corresponding fields of each frame You can add only one Job Description but multiple Data Staging an Application elements Therefore you have to save each Data Staging and Application element individually by clicking on the corresponding save button After clicking a save button the element will be listed in the drop down list to the right side of the button When you have entered all information you can create the JSDL file by clicking the create JSDL button The Portlet will then safe the file and return to standard view with no file selected Step 2 The description of the economic preferences is supported by the Job Econom
66. ecific file its content will be shown in the window on the right side of the view Also the file will be selected for further editing If you want to edit the values of a selected file just click on the small arrow icon in the upper left corner and the Portlet will switch back to the base view with the values loaded from the file to the corresponding fields 3 3 2 Offer Submission GUI The Resource Modelling section 3 3 1 and provider Offer Submission Portlets support the technical description of the offered resources and the expression of the economic preferences The Portlets are part of the intelligent tools that complement the OfferGenerator BidGenerator and OfferGenerator implement a similar behaviour and thus we implemented one bidding framework for both cases called BidGenerator The usage of Portlets describes the case when providers want to use a Web GUI interface in order to prepare and submit offers for their resources The Portlets are implemented and deployed in the portal framework GridSphere The process of manual offering of computing resources to the SORMA Market contains several steps supported by the implemented provider Portlets Step 1 The first step is the technical description of a computing resource section 3 3 1 which result in the creation of a JSDL document res jsdl Step 2 The description of the economic preferences is supported by the Economic Resource Description Portlet Figure 25 In addition t
67. ection which allows the administrator to specify the economic preferences of the EERM for example the goal of the policies revenue maximization weight workload etc the negotiations strategies with varying degrees of aggressiveness and the application of the client classification Resources Clients Economic policies Parameter Value Goal Weighted workloads z Negotiation strategy Medium z Client classification Linear d No client classification Linear Exponential l anarithmia Figure 35 specifying preferences for resource policies 32 4 Resource Consumers Manuals 4 1 Introduction A consumer who wants to access resources from the SORMA market is required only to install and run the consumer side Intelligent Tools These tools enable consumers to specify resource requests to execute a given job and to submit bids to the SORMA Market that will return one or more providers matching the selection criteria The same tools allow enable the submission of the job to the selected Provider and to access the result of the execution These tools provide a user interface but it is also possible for the end user to by pass the standard interface and to write a custom SORMA client application using the SORMA APIs 4 2 Intelligent Tools Installation and Configuration Guide The Intelligent Tools consist of the components Consumer Preferences Modelling Demand Modelling and BidGenerator The Consumer Preferences Modelling compon
68. eement document which enables each participant to access the SLA maintained by Contract Management see D2 2 if they wish SORMA Message Examples Examples of SORMA messages are reported in the Appendix of this document 61 SORMA Market Administrators Manuals 4 5 Introduction The SORMA market is composed by a set of server side components Some of these components offer their own user interface GUI that allows SORMA System Administrators to maintain the system and to monitor what is happening in the SORMA system All components can be found on the SORMA system DVD as described in section 1 1 1 of this document 4 6 Installation and Configuration Guide This section of the user guide describes how administrators can install configure and test SORMA The components to be installed are Market Exchange Service Market Information Service Payment Service Trading Management Contract Management and Billing SORMA Global Logging De NS The security infrastructure is a horizontal feature and it is explained in the next paragraph 5 3 4 6 1 Market Exchange Service a Pre requisites e Linux based platform i e Ubuntu e Sun Java Development Kit 1 6 e Apache ant 1 6 and up to use the component launching scripts b Core Market Services Port Requirements Components that make up the core market services require a number of network ports to be open Defaults are given in the following table Where this component i
69. eir own organization obtaining an SAML assertion and use this assertion in exchange for SORMA credential i e proxy certificate 1 4 Explanatory Notes about WP5 deliverables D5 2 D5 3 D 5 4 In order to avoid misunderstandings or ambiguities it is useful to remember which deliverable is going to deliver the Final software prototypes and related documentation e D 5 2 Economic Grid middleware Prototype Report Description and implementation of the prototypical economic Grid middleware D5 2 contains the technical details of the Core Market Services prototype and the EERM and Resource Fabrics which are to be installed at the Provider side This deliverable IS NOT going to deliver such components user guides which will be delivered into D5 3 s D5 3 Integrated SORMA system amp System Manual Prototype Report Description of the integrations undertaken detailed definition of all prototypical system components and manuals are available This very deliverable which includes the description and technical overview of the integrated SORMA system technical architecture the role of the components while their detailed description can be found in D4 x deliverables and user manuals technical guides and user guides about ALL SORMA components excluding Mosix which will be delivered in D5 4 This deliverable also includes the software prototype of the whole SORMA market and user agents Economic Grid middleware prototyp
70. ender Default org apache 1log4j RollingFileAppender log4j appender Default Threshold DEMO eu sormaproject logging level DEMOLevel l log4j appender Default File DEMOLevel log log4j appender Default layout org apache log4j PatternLayout log4j appender Default layout ConversionPattern d S dbUserid c 1 p x smn Do not Truncate if it already exists log4j appender Default Append true log4j appender Default MaxFileSize 1500KB Archive log files one backup file here log4j appender Default MaxBackupIndex 10 log4j properties log4j rootCategory INFO Default SORMA The Socket Appender log4j appender SORMA org apache log4j net SocketAppender log4j appender SORMA Threshold DEMO eu sormaproject logging level SORMALevel log4j appender SORMA RemoteHost 147 83 30 215 log4j appender SORMA Port 16999 log4j appender SORMA ReconnectionDelay 5000 log4j appender SORMA LocationInfo true 75 d Example Following code snipped is an example for submitting of a log message to the configured logging server If the log4j properties are not already bound they have to initialized The special logs have to be submitted using the DEMO logging level PropertyConfigurator configure some path log4j properties logger log DEMOLevel DEMO Client submits a bid request to AgentService ee OLevel DEMO AgentService uses the BiddingFramework to genera
71. ent provides a message format for describing economic and technical preferences The technical and economic preferences are specified in EJSDLPrivate templates which are stored locally and can be individual for each application Each time an application needs a further resource it invokes the BidGenerator Web services through the submission of an EJSDLPrivate document The Demand component offers a Web based GUI Portlet to enable the simplified description of technical requirements for jobs which can be imported and exported in the JSDL data format These components are comparable to the Providers Intelligent Tools so installation instructions are exactly the same it is possible to refer to section 3 2 1 Provider side intelligent tools The only difference to be reported is in the deployment of the Technical Resource Portlet Deploy the Technical Resource Portlet and Consumer Portlets for Bid Submission as described in section 3 2 2 d Copy the war files of the Technical Resource Portlet and Consumer Portlets for Bid Submission to CATALINA HOME webapps and restart the servlet container Tomcat Tomcat will extract the webapps in a directory with a name like technicalResourceModeling and consumerPortlets Enter the following command to enable the Technical Resource Portlet and Consumer Portlets for Bid Submission touch CATALAINA_HOME webapps gridsphere WEB INF CustomPortal portlets technicalResourceModeling and
72. ent submit a bid request with valuation of 10 and preferred bidding strate 2 2008 09 1 eu sormaproje AgentService uses the BiddingFramework and generates a bid price of 8 for the given re 3 2008 09 1 eu zomaprole Market Exchange receives the bid and addresses the bid message to the auction theAu 4 2008 09 1 eu sormaproje Trading Management receives the bid and trys to match it with existing offers 5 2008 09 1 eu sormaproje BSC s EERM registers new computational resource and generates the corresponsing req 6 2008 09 1 eu sormaproje gentService uses the BiddingFramework and generates a bid price of 5 and submitts it t 7 2008 09 1 eu sormaproje Bid price 8 generated and submitted to Market Exchange 8 2008 09 1 eu sormaproje Trading Management receives the offer and trys to match it with existing bids a 2008 09 1 eu sormaproje After a match is established the established SLA with contract ID is submitted to the tar 10 2008 09 1 eusormaproje The TXT VISAGE Client receives the URI of the target EERM and submits the job to it 11 2008 09 1 eu sormaproje The EERM receives the SLA from the a notification as a result of the AgentService call an 12 2008 09 1 eusormaproje TXTDemand Visage client submits the job to the EERM which is executed on a allocated 13 2008 09 1 eu sormaproje EERM messages 14 2008 09 1 eu sormaproje Monitoring m
73. er use an explicit version number to identify the dependency revision they require or they can always opt to use the very latest version that is available by specifying Latest integration in their Ivy dependency line ot File Edit View History Bookmarks Tools Help eh O file tmp examplelvyComponent build logs ivy eu sormaproject exampleComponent default html G Netbeans Apache Ivy Getting Started Latest Headlines O Editing Getting started wit U Ivy report examplec GB a exampleComponent by eu sormaproject E resolved on 2008570817 37 31 default Dependencies Stats Modules 4 Revisions 4 1 searched 0 downloaded A 0 evicted 0 errors OI Artifacts 4 0 downloaded O failed Artifacts size 690 kB 0 kB downloaded 690 kB in cache Dependencies Overview Module Revision Status Resolver Default Licenses Size sorma common by eu sormaproject 1 0 integration H local false ii UH Ki zz junit dep 44 i release sorma vendor false A389 KB SE junit b 4 4 i release fi sorma vendor false 158 kB log4j by org 1 2 15 release sorma vendor false 383 kB Details sorma common by eu sormaproject Revision 1 0 EE Status integration Publication 20081 104144231 Resolver local Configurations default Artifacts size 11 kB 0 kB downloaded 11 kB in cache Required by Organisation Name Revision In Configurations Asked Revis
74. ervice Deployment a Prerequisites No other SORMA components need to be installed prior to the installation and deployment of the Contract Management component The following SORMA components must however be installed prior to the invocation of this component e SORMA Payment Component e SORMA Market Information e One or more SORMA EERM instances The following third party software must be installed before installing the Contract Management Web Service e Sun Java 1 6 e Apache Tomcat 5 5 with a JAX WS installation b Service Deployment To deploy the web service obtain ContractManagement war from the SORMA DVD ivy sorma eu sormaproject ContractManagement LATEST ContractManagement war Place the war file in the webapps directory of the Tomcat server Tomcat will now explode the war and install the service c Runtime Configuration The war file on the SORMA DVD comes with a pre packaged configuration file it is not expected that the configuration will remain consistent over time and therefore this must be modified To do this a new version of the properties file must be supplied It is very important that the properties file is placed in the correct location otherwise ContractMangement will be unable to find it and therefore function incorrectly The properties file should be placed in the current directory where the Tomcat server was started or a relevant subdirectory anywhere in this directory tree ContractMangement will
75. ervice provides the interfaces InformationQuery and InformationPublish to manage the aggregated information of the market such as maximum and minimum prices in a certain market The Market Exchange provides the Trading interface to allow participants to engage in negotiations exchanging proposals bids and eventually reaching an agreement Bids received from consumers and providers are forwarded to the Open Grid Market s ProtocolExecutor interface to be processed by the market s allocation protocol It also requires the Authentication and DigitalSignature interfaces provided by the Security Management component to protect the interaction among participants The Market Directory offers the Search interface to look for existing markets which are registered using the Registry interface by the Market Exchange To be available to trading agents markets must register themselves using the Trade interface 12 The Logging service offers a secure persistent record of the transactions for auditing and dispute resolution It is also used to recover the state of negotiations in case of a failure of the Market Exchange service Trading L T lt lt Component gt gt I Core Market Services lt lt Component gt gt IS O lt lt Component gt gt E Authentication k Sery x Logging Logging Service Market Exchange Digital ii
76. es are delivered in D5 2 e D 5 4 Market based Grid OS amp System Manual Prototype Report Description of the implementation and modifications undertaken detailed definition of prototypical system components and manual are available This deliverable will include software prototypes and user manuals about Mosix OS 10 2 Technical Description of the SORMA System This chapter provides an integrated view on the SORMA system its final technical architecture and its main components section 2 1 how they behave section 2 2 how they were integrated and how the whole system was deployed section 2 3 in order to prepare for testing 2 1 Integrated SORMA System Architecture This section describes the final SORMA system architecture from a technical point of view A high level view of the SORMA system can be found in the latest deliverable D2 2 section 2 2 1 1 SORMA Architecture The SORMA system is composed of a set of components that provide the required functionality to allow the exchange of resources on an open market across multiple organizations These components interact following complex communication patterns in a distributed setup In this section we present the SORMA s technical architecture Figure 1 shows a high level view of the SORMA architecture How each of those components fulfil the interfaces it exposes will be discussed in the following sections The architecture s main components are the Consumer s an
77. esetsessessrsresessessesesses 55 Figure 37 File selection view with sample file selected 0 0 0 ee eeceeseeeceeseeesececeeseceseceeeeaeeeaeeneeeaeeeaeens 55 Figure 38 Standard view E HEET 56 Figure 39 Economic description of the ER eege Eege eege EE 56 Figure 40 Submission of the bid request to BidGenerator ssssssesssesssessesesessesesessesstsesesseseesesses 57 Fig r 41 Stats te E 57 Figure 42 Message formats between the SORMA component cccecceseceseeseceteeeeeeeeeseeeeeeeeeaeens 60 Figure 43 C Space Protocol Execution Mamager cscsssssscssesseccstsensestsenscesssssconnssnesencengseseeseooss 12 Se ee e EE 80 Fig re45 Chainsaw Oe UN EE 83 Fig re 46 Chainsaw Log Properties srs ouennn are tetera bac N E E E 83 Pau re Al Security mita Cp ace ict e EE E ana E E E aay nse a S 84 LEE TEE 85 Fig r 49 Security E EE 85 Figure 50 Proxy Manager eet 86 Figure 51 Administrator User Interface EE 87 Figure 52 Screenshot of the HTML page generated by the report target showing dependency PESOLUCLON Information siro an arn Een EEEE Eaa EATA baad E ET E vated 91 1 Introduction This deliverable describes the final integrated SORMA system providing both software code and user guides The final integrated SORMA prototype is described from a technical point of view by defining its integrated architecture by showing how the main components behave and how the system is currently deployed for testin
78. esources to EERM If you click the icon of any resource with the left button it will appear a contextual menu see Figure 30 with some actions to perform upon the resource 10 0 2 200 Lo 4 4c Send test job Reserve resource 104857 Show Properties gt 4 Delete resource Figure 30 managing virtual machines The actions that can be performed are e Reserve Resource a small form Figure 31 will appear to allow the EERM administrator to manually perform a reservation Reservations can be specified as fixed points or as intervals the start and end date and specify one or more resources to reserve currently CPU and RAM memory Reservation for r source 10 0 2 200 Fixed reservation O Interval reservation an 8 Finish o o 4 feb 2009 b ene feb mar abr may jun jul ago sep oct nov dic Client http www di lun mar mi jue vie sab dom CPUs 1 Memory 26 27 28 29 30 31 1 2 3 t 5 6 7 8 9 10 11 12 88 14 15 26 Et GENEE 20 2E 22 23 24 25 26 27 28 1 Figure 31 reserving resources manually e Visit URL only for services This enables the user visit to the URL endpoint of the service using the EERM proxy e Send test job only for batch resources The administrator can send a simple ray tracer job to check that the resource fabrics are working correctly If the job is successful the user will see the result a generated image as shown in
79. essage gt e clearingPrice The specified monetary value assigned to an artifact on market clearing also market clearing price This price is determined by the auction based on the bid and offer process of providers and consumers interested in trading the artifact e contractld the id of the contract between a provider and a consumer s bidId the id of the bid e offerld the id of the offer e consumerld the unique id of the consumer e providerld the unique id of the provider WS Agreement The EJSDLMarket document is the first result of the technical and economic match of a consumer bid and provider offer A WS Agreement document is the second result of a match and represents the final Service Level Agreement SLA between both parties see Example 4 and is composed of three main elements e Context the identification of who is participating in the SLA and their roles i e consumer or provider the validity period of the SLA can also be defined here e Service Terms what is being provided and how it is accessed e Guarantee Terms the promises made by both parties These are observable properties and are used to identify the outcome of the agreement Here the economic properties associated to the terms are also quantified For example the penalty if the promise is broken The EJSDLMarket document is returned to the participants of a match and therefore also of the SLA contains the contract id of the generated WS Agr
80. essage layer and the information service it is only necessary to provide values for properties with these prefixes Now point your web browser to the C Space Manager web page at http localhost 8080 cspace NOTE You should change the port number to your own tomcat s HTTP port You will see a page like the screen shot shown in Figure 43 Click Browse in the section named Configure C Space Protocol Executor select the properties file you just modified and click Configure You should now see a message saying that the Protocol Executor has been configured 71 Mozilla Firefox e File Edit View History Bookmarks Tools Help C Space Protocol Executor Manager Configure C Space Protocol Executor Properties file to upload Browse Configure Upload a Conversation Protocol IS JAR file containing protocol to upload Browse Upload Create a Conversation Namespace Class Name Conversation ID KayVL232AiW ZNz8MhgCFpeHYpw eu sormaproject cspace CDA Protocol theAuction Create Reset Protocol Executor Reset Figure 43 C Space Protocol Execution Manager 4 Upload the CDA Conversation Protocol Download the CDA protocol JAR file Again point your web browser to C Space Manager web page Click Browse in the Upload a Conversation Protocol section and select the JAR file you just downloaded Finally click the Upload button You should now see a message confirming
81. essages 15 2008 09 1 eu sormaproje TXTDemand Visage client submits the job to the EERM which is executed on a allocated up 2008 09 1 e eu sormaproje EERM messages Property Value active faise 18 2008 09 1 eu sormaproje TXTDemand Visage client catches the result from the execution class dass or connect listener zt T o oss name SORMA D x mn Paused false E 0 hidden loggers fq I gt port em Refine ocun om E Ze 8 eme All Receivers have been re started Figure 45 Chainsaw Log GUI 18 18 9 0 5 j Select manual configuration The property windows which appears by adding a new receiver gt SocketAppender The only options to be set are name port and threshold the other settings are default class org apache log4j net SocketReceiver org apache log4j Hierarchy 79bca4 SORMA 16999 ALL Figure 46 Chainsaw Log properties In order to submit and maintain special messages we defined a new log4j level called DEMO The required additional log4j configurations for both log4j server and log4j client SORMA components are shown in the following The server properties are only required when starting the loggingService instead of the Chainsaw Log GUI the most important properties to be configured on each client are the log4j client properties log4j properties
82. et localhost 8649 You should see a large XML in the console Keep in mind the host name bscib12 bsc es in the next XML lt GANGLIA XML VERSION 2 5 7 SOURCE gmond gt lt CLUSTER NAME unspecified LOCALTIME 1232101290 OWNER unspecified LATLONG unspecified URL unspecified gt lt HOST NAME bscibl12 bsc es TP 84 88 50 91 REPORTED 1232101271 TN 19 TMAX 20 DMAX 0 LOCATION unspecified GMOND STARTED 1232101235 gt lt METRIC NAME Cpu_nice VAL 0 4 TYPE float UNITS TN 55 TMAX 90 DMAX 0 SLOPE both SOURCE gmond gt The host name of ganglia MUST be the same that the Linux hostname If you type the command hostname in the console the output must be exactly the same in that example bscib12 bsc es If not 43 you MUST change it by eg adding an entry to etc hosts or by changing etc hostname or by ensuring that you DHCPCD is configured to correctly set the hostname f 2 Tycho GridSAM for batch resources Tycho GridSAM provides a mechanism to bind GridSAM exposed resources to the EERM GridSAM is a Grid middleware provided by OMMI UK under the UK eScience programme In SORMA we have extracted the core GridSAM functionality which we then wrap in a Tycho connector to give us a flexible event based front end to GridSAM GridSAM provides plugins to interact with a range of middleware In this example we consi
83. executed When the EERM receives an EJSDL from the Client Application the EERM must check the correct identity of the Client Application and that it has attached a valid ContractId The EERM may need to contact Contract Management to assure the validity of a contract After sending the job from EERM to the Resource the resource notifies continuously about the changes in the status of the execution and EERM forwards them to the Client Application In addition the EERM periodically response to requests for monitoring data from SLA Enforcement to enable the monitoring of SLA adherence by the provider Specifically if the terms in the SLA are being fulfilled correctly and in case they are not take reactive measures such as renegotiation or ask the EERM for a redistribution of the assigned resources Finally when the Resource notifies to the EERM that the task has been finished the EERM forwards this notification to the Client Application and the SLA Enforcement The SORMA cycle is finished here 2 2 4 Special case Future Markets Since the EERM allows the advanced reservation of resources future markets can be implemented in SORMA The main change to support reservations must be implemented in Trading Management component see Figure 8 In the technical economical matchmaking process when a pair of bids offers matches Trading Management component must ask to the Provider s Bidding Agent if a concrete amount of resources could be allocated in
84. ey can be used they need to be configured to connect to a GridRM Gateway You can achieve this by entering the Portlet edit mode by clicking the pen icon as shown Figure 18 and setting appropriate values for the gateway you want to connect to 39 000 GridSphere Portal GID Oe Cs heor ere Most Visited Getting Started Latest Headlines 3 0 Rec Find Q Lakhoo hext Previous Highlight ai 7 Figure 18 All Portlets have an edit mode which is accessed via the pen icon All Portlets have an edit mode section titled Configure Gateway see Figure 19 Each Portlet can be set individually to connect to a different gateway so you will need to edit the settings for each Portlet you wish to use GridRM Gateway Tycho name The Tycho name of the gateway you want to connect to This name corresponds to the tycho gridrm gateway name property that was set in the gateway s configuration file Tycho mediator socket address This is the socket address of the standalone mediator that you are using for the Portlets Tycho mediator group name Set this to the group name used by the standalone Tycho mediator Distinguished Name DN Your user name for the gateway Passphrase Your passphrase Depending on the Portlet the edit mode may also contain other parameters that you can set For example the Portlet shown in next figure has a sec
85. file The standard view of the section is shown in Figure 36 54 d Provide job description BO Job Description Job Name Description Job Annotation Job Project m Data Staging Filesystem Name Ir S Filename Source SY Target Ir Creation Flag append lt Delete on Termination C Nei m Application Application Name Application Version POSIX Executable Arguments Nei Create JSDL Figure 36 Standard view of the Job Description section In order to add a job description to a specific JSDL file you first have to open the file in the job description section To do this click on the pencil shaped icon in the upper left corner and the file selection view will open as seen in the technical resource section Select a file and its content will be shown in a window as shown Provide job description Content of sample_host jsdl lt jsdl JobDefinition xmlns jsdl http schemas ggf org jsdl 2005 11 jsdl gt lt jsdl JobDesc ription gt lt jsdl Resources gt sample_host jsdl lt jsdl OperatingSystem gt lt jsdl Description gt Windows XP lt jsdl Description gt lt jsdl OperatingSystem gt lt jsdl CPUArchitecture gt lt jsdl CPUArchitectureName gt x86_64 lt j sdl CPUArchitectureName gt Figure 37 File selection view with sample file selected To return to the edit view click on the arrow icon in th
86. formation remains private and is only considered by the bid generation processes of BidGenerator 60 EJSDL The economically extended JSDL called also EJSDL extends the standard JSDL with the economic category Bid which adds economic parameters in order to support the bidding process The EJSDL message format is used to describe the bid offer messages send from the BidGenerator to the TradingManagement see Example 2 Economic extensions in element lt Bid gt e type the type of bid message bid offer match e participant unique identifier of the consumer or provider e bidPrice the generated bid price of the consumer or provider e timeout the time in milliseconds a consumer or provider bid is valid in the order book e duration the upper bound duration in milliseconds for the provided or requested resource e serviceType the type of the requested service batch job or WebService e paymentType the type of payment before or after job execution e currency currency e penalty type contains the applied penalty function functionName and normalizationConstant EJSDLMarket The EJSDLMarket message format defines an additional category MarketMessage for describing the market data like clearing price and contractld which is created on match by the TradingManagement The EJSDLMarket message is than submitted back to the target consumer and provider over the BidGenerator component see Example 3 Elements of lt MarketM
87. g User guides for three main categories of end users resource providers resource consumers and SORMA system administrators are included Each user guide provides install and configuration instructions and a user manual In addition a guide for SORMA system developers is included The final prototype software is available on the DVD annexed to this document 1 1 Structure of this Document This document is structured in the following way Chapter 1 provides relevant information about this document and its relationship within Work package 5 and provides an overview of the SORMA system s actors Chapter 2 describes the final SORMA prototype architecture with a special focus on integration Chapter 3 provides manuals for the resource providers specifying how to install configure and use the SORMA tools in order to offer resources on the SORMA Grid market Chapter 4 provides information for resource consumers specifying what to install and how to access the SORMA market via GUIs or how to develop a custom client using the SORMA API Chapter 0 provides information for the SORMA market operators specifying how to install the SORMA Grid market and how to configure and maintain it Chapter 5 provides guides for SORMA system developers Appendix contains code and message examples 1 1 1 Annexed DVD The annexed DVD contains the SORMA Integrated System The DVD content is structured in the following way README txt How to use
88. ge service host name used to contact the CSpace Port used by the CSpace s web services container Name of the CSpace services as deployed in the web services container Name of a conversation to be automatically registered at start up Comments Don t change Don t change Don t change Don t change unless you are providing an alternative implementation of the service Can be changed to any of the transports installed in the gmm e g PastryChannel Don t change Informative value Can be changed to any name Don t change Can be changed to use an alternative implementation of the service Can be changed to any of the transports installed in the gmm e g PastryChannel Change to adapt to your actual deployment of SORMA components Change to adapt to your actual deployment of SORMA components Change to adapt your actual deployment of the SORMA components Must match the name of an existing conversation in CSpace The default name of this file is erchange properties and is expected to be in the exchange s working directory the directory it was launched from This can be changes passing the following command line arguments when launching exchange 64 ant exchange Dapp args config file lt config file path gt Any property can be overridden and new properties can be added passing them in the command line ant exchange Dapp args lt property gt lt value gt
89. gt Here lt cspace host gt is the host where the Trading Management is being installed lt mysql user gt and lt mysql password gt should be consistent with the corresponding entries in the cspace properties file see below 2 Deploy the C Space Protocol Executor Web Application Start your Tomcat server CATALINA_ HOME bin catalina sh start Copy the cspace war to your Tomcat s webapps directory This should automatically deploy the service 3 Configure the Protocol Executor Download the properties template file and open it in a text editor You should see something like this MAD C Space Properties MAD AAA Public address of Protocol Executor AAA cspace public protocol http cspace public host 193 10 67 191 espace public port 8080 TEEPE TEER TE TERETE RETA TTT TT Database connection properties TEETER TE TERETE TEE TTT TT cspace mysql host localhost cspace mysql username cspace_executor cspace mysql password mammamia cspace mysql port 3306 DEELER Information Service DEELER cspace informationService namespace cspace informationService sorma MIS Information Service cspace informationService sorma factory className eu sormaproject cspace mis MISInformationServiceFactory cspace informationService sorma host pcmargenat ac upc edu cspace informationService sorma port 8880 Test Information Service cspace informationService test factory className eu sormaproject cspace information NullInforma
90. he following advantages e With virtual machines the users can be granted if needed root access Linux or administrator access Windows to their virtual machine The root or administrator permissions are sometimes needed to install software and deploy certain components which is not possible with the more 19 restricted user access Traditional approaches not based on virtualization must often apply non trivial solutions to overcome these limitations e The isolation of virtual machines between each other is an important feature since then the activities of some users do not influence on the resources available to other users In order to achieve this isolation the virtualization tools allow making strict resource assignments to each virtual machine While with the traditional approach without virtualization where many users work within their account simultaneously on the same operating system and the same physical PC a computer intensive activity of a certain user does usually affect the resources available to the other users e Virtualization makes it possible to boot different operating systems in the virtual machines A share of the physical resources of the PC is assigned to each virtual machine The practical usage is that some of the PC s virtual machines may be configured to have Linux as the operating system while others may have Windows The Local Resource Manager is a software infrastructure developed at UPC which
91. he IVY is adopted in order to manage and resolve dependencies between the components and 3rd party java libraries Furthermore the components test clients and test files are uploaded versioned and kept up to date in SORMA IVY For the purpose to automate the continuous integration build and test processes we adopted Cruise Control Continuous Testing of the SORMA Components In order to verify the component s functionality and to speed up the integration processes each component has to implement it s own unit and integration tests The unit tests JUnit verify the functionality of the target component To test an individual component that depends on interaction with other components the other components are implemented as a Mock Objects or Mock Service Objects Debugging and Issue Tracking In order to report resolve and document the solutions of issues arose during the software development processes we selected to adopt Bugzilla Revision Control The SORMA Subversion revision control repository is structured into the following top level directories meta Tools and scripts for project administrators developers do not need to access these build bootstrap Developers must export this directory to be able to use the standard SORMA build mechanisms code The root of the SORMA source code notes This is documentation that is interesting for the project but not required by users as part of a source code download Th
92. ic Description Portlet Figure 39 In addition to the technical job description job jsdl the consumer enters its economic preferences like when to submit the offer Date to submit the duration for which the resource is required Requested duration preferred bidding strategy valuation maximum willingness to pay and the bid s Time to Live Bid validity 8 gridsphere portal framework Welcome Administration sorma Job Economic Description Technical job description job1jsdl Date for submit 27 01 2009 Requested duration hours Select bidding strategy Strategy Valuation cents 150 Bid validity seconds Figure 39 Economic description of the job 56 Step 3 The entered preferences are shown one more time Figure 40 before the consumer submits the bid request to the BidGenerator section 4 2 which initiates the negotiation process based on the preferences gridsphere portal framework Welcome Administration sorma Ee Submit Bid Bo Technical job description job1jsdl Date for submit 27 01 2009 Requested duration hours f Bidding strategy lQSategy o Valuation cents ben Bid validity seconds Dm SSS Figure 40 Submission of the bid request to BidGenerator Step 4 Figure 41 shows the result of the bid request If there is an allocation the economic data for example the provider resource reference expected completion time average re
93. ictions are not an issue and we keep a clear separation between our component s source code and the test code The actual component code should not know anything about the test code A corresponding test directory for the example component looks like this test eu sormaproject provider example AllMockServiceIntegrationTests java test eu sormaproject provider example AllUnitTests java test eu sormaproject provider example IntegrationTestSormaWorld java test eu sormaproject provider example TestHelloSormaWorld java Unit tests and Integration tests are logically grouped according to the AllUnitTests java and AllMockServiceIntegrationTests java classes This is because e Unit tests only test local classes and use local mock objects therefore a test environment does not have to be specially created as testing never leaves the local JVM e Integration tests require the test environment and remote Mock Service Objects to be initialized Integration tests could take quite a while to complete where as unit tests should execute rapidly so that e g thousands of tests can be executed in seconds Whereas unit test class names are prefixed with Test e g TestHelloSormaWorld java by convention we prefix integration test classes with IntegrationTest e g IntegrationTestSormaWorld java XMLUnit Tests are unit tests offering methods to test data specified in XML format The various SORMA message protocols are tested using XMLUnit
94. ids are delivered by the Core Market Services to the Open Grid Market s ProtocolExecutor interface to be processed by the corresponding market allocation mechanism e g auction protocol The Open Grid Market also offers interfaces to manage the SLAs which enforce and define the utilization of the resources and to make the payments for such resources 1 It is important to note that this search interface allows the location of markets i e ongoing auctions but not for the specific items being traded in them 11 lt lt Component gt gt E Grid Application Job oe Inform Payment Q _ lt lt Component gt gt lt lt Component gt gt 5 Ce Submit Bid Consumer s Intelligent Provider s Tools Intelligent Tools Monitoring I D sl C O Information Query lt lt Component gt gt E d information Publish Core Market Services C Digital Signature CO E Protocol o lt lt C t gt gt lt lt Component gt gt E Er tca omponen E Authentication Security Open Grid Market SLA Query Request Payment Figure 1 SORMA Technical Architecture 2 1 2 SORMA Components Core Market Services Figure 2 details the internal components of the Core Market Services It is formed by four services the Market Information Market Exchange Market Directory and Logging The Market Information S
95. imple Test Client An optional task is the installation of JSR 168 Portlets that provide a Web based administrative GUI for interacting with GridRM Gateways Note the Portlets are not required for successful EERM operation a Installation of the GridRM Gateway a 1 Pre requisites e Bi directional firewall access for the Tycho port if wide area communication is required e The following Network ports are required Port number Anere Component ServerSocket Port description Container GridRM Each GridRM gateway has an internal Hypersonic database that it uses Currently the ateway s internal 54321 but canbe database is running as a standalone component good for development and debugging Hypersonic pees any port purposes later we will embed the database into the gateway so that a port is not required DB By default the database is located on the same machine as the gateway Wide area communication over HTTPS between Tycho mediators on remote machines Tycho Mediator typically 8080 but single port We typically stick to 8080 80 because most firewalls allow traffic to these ports Einbaded any can be used Jetty to pass by default Port must be open both ways GridRM Gatewa dynamically Local area socket communications between the GridRM gateway and a Tycho mediator Embedded y assigned port either located within the same JVM or on the LAN Jetty a 2 Download from the SORMA DVD gridrm tycho gateway tar gz version ivy vendor
96. ion eu sormaproject exampleComponent working obufki default latest integration Done o Figure 52 Screenshot of the HTML page generated by the report target showing dependency resolution information If you successfully resolved dependencies compiled tested and published the exampleIvyComponent to the SORMA integration repository then you have successfully configured your build environment for SORMA As a next step you could also look at Ivy plugins for your favourite Integrated Development Environment IDE and execute the ant targets from within the IDE e Plugins for Eclipse can be found at http ant apache org ivy ivyde e Plugins for Netbeans can be found at http wiki netbeans org Faqlvy 91 5 2 Developing for SORMA In this section we aim to provide further detail about the development process The src directory contains the component s source code All code should be located under the eu sormaproject root To date we have been determining package structure based on where a component appears in the logical architecture Examples include eu sormaproject openGridMarket eerm eu sormaproject coreMarketServices marketExchange The test directory contains unit testing JUnit and integration test code The package structure under test is the same as that under src which results in the test code being located in a parallel tree to the source code The parallel test tree means that package level visibility restr
97. ity of components executing solely on physical machines provides the opportunity to compare component behaviour against that observed in virtual machines for e g identifying race conditions Furthermore the components at UKA execute exclusively on separate hardware so that performance testing can take place in a controlled environment without the risk of interference from other SORMA components on the same machine contaminating the measurements Details of wide area testing will be reported as part of WP6 23 2 4 SORMA Software Development Methodology The software development process in SORMA is executed using a well defined methodology The methodology contains the following parts 2 4 1 Development Best Practices Survey of literature links and common rules for software development Specification of a common methodology and rules for software development Management of the SORMA Components source code The source code is structured and stored in the SORMA SVN Subversion Specification of a common methodology for structuring the source code of the components stable and trial versions naming and versioning conventions Management of the Build Test and Deployment Processes The build test and deployment processes of the SORMA components are managed and executed using Apache ANT based scripts which incorporate best practices as well as standardize and simplify the processes of building testing and deployment Apac
98. ix jar 4 cryptix asn1 jar et ee ee 76 cryptix32 jar jee jdk 13 125 jar jgss jar log4j 1 2 8 jar puretls jar SiO a a Deploy the WAR The security services are packaged as a generic WAR file which can be deployed in any web application server In this guide Apache Tomcat 5 5 26 is used Before going through the installation make sure that Tomcat is installed and running properly 1 Download the WAR file from the SORMA DVD ivy sorma eu sormaproject security LATEST sorma security war 2 Copy the WAR file sorma security war to the Tomcat webapps directory This should automatically deploy the service in the sorma security directory b Configure the Service The services are configured through a single configuration file which is located at CATALINA HOME webapps sorma security WEB INF etc SORMASecurityService Dorian dorian conf xml For simple deployments only the following configuration elements need to be modified 1 Database Configuration 2 CA Subject Name c Database Configurations To modify the database configuration to interact with MySQL database please set the values of the host port username and password elements highlighted in bold below lt xml version 1 0 encoding UTF 8 gt lt DorianConfiguration xmlns http cagrid nci nih gov dorian conf lt DatabaseConfiguration gt lt host gt localhost lt host gt lt port gt 3306 lt port gt lt username
99. java net ConnectException Connection refused Dest localhost 3535 An error occurred when registering with the Tycho registry 7792405484 DEBUG PortletLayoutEngine java actionPerformed 245 41 In fact in this example the mediator had not been started Fix the problem in this case start the mediator and try again e000 GridSphere Portal Se CE Cx tr C26 beer portas rdg ac uk 8449 gridsphere gridsphere gs_modyy Gly Google Q Most Visited Getting Started Latest Headlines gridsphere portal framework f Charts f Resource Monitoring f Welcome f Administration Resource Monitoring 24 GridRM Gateway Events Initialisation of Tycho blocking API failed You can register to receive events from a particular gateway Select the events you wish to receive make sure the gateway d H t H H 1 f H 1 H i l correctly and then press the Save details button NOTE Currently you cannot cancel event registrations by clicking on an already selected event This functionality will appear Event subscriptions Select the events you wish to receive from the gateway x Event Type Description o New resouce registration Subscribe to this event to be notified when new resources are added to a GridRM gate Io Resource unregistered Subscribe to this event to be notified when a resource is unregistered from the gatew New driver registration Subscribe to
100. lProvider Properties channel http param protocol http channel http param baseUrl a 6 Test the Installation Download the AgentService client from the SORMA DVD in order to test the AgentService by submitting EJSDLPrivate based provider and consumer requests The AgentService client can be found on the DVD at ivy sorma eu sormaproject BidGenerator LATEST AgentServiceClient tar gz To test the correctness of the installation and configuration unpack and run the test client AgentServiceTest with following parameters for each of the consumer and provider requests e IP Address of the AgentService e Port Number of the AgentService e The example EJSDLPrivate file To test the AgentService you should open two command line consoles for each the consumer request and provider request and do following call 29 java cp agentServiceClient jar lib eu sormaproject web client AgentServiceTest lt IP Address AgentService Machine gt lt PORT NUMBER gt lt EJSDLPrivate FILEPATH gt Local tests are executed with the DUMMY requests where the AgentService returns a DUMMY response java cp agentServiceClient jar lib eu sormaproject web client AgentServiceTest 147 83 30 215 16789 ConsumerEJSDLPrivateDataDUMMY xml java cp agentServiceClient jar lib eu sormaproject web client AgentServiceTest 147 83 30 215 16789 ProviderEJSDLPrivateDataDUMMY xml
101. lt bean paymentType gt lt bean clearingPrice gt 3 3333333333333335 lt bean clearingPrice gt lt ValueExpression gt lt Reward gt lt BusinessValueList gt lt GuaranteeTerm gt lt GuaranteeTerm ws Name IndividualDiskSpace ws Obligated ServiceProvider gt lt ServiceScope ws ServiceName myJobId1234 gt lt ServiceLevelObjective gt lt KPITarget gt lt KPIName gt jsd1 IndividualDiskSpace lt KPIName gt lt CustomServiceLevel xsi type jsdl RangeValue_ Type xmlns jsdl http schemas ggf org jsd1 2005 11 jsdl xmlns add http schemas xmlsoap org ws 2004 03 addressing gt lt jsdl1 UpperBoundedRange gt 2000000 0 lt jsd1 UpperBoundedRange gt lt jsdl1 LowerBoundedRange gt 100000 0 lt jsdl1 LowerBoundedRange gt lt CustomServiceLevel gt lt KPITarget gt lt ServiceLevelObjective gt lt BusinessValueList gt lt Importance gt 1 lt Importance gt lt Penalty gt lt AssessmentInterval gt lt TimeInterval gt PT30S lt TimeInterval gt lt AssessmentInterval gt lt ValueExpression xsi type bean Penalty Type xmlns jsdl http schemas ggf org jsd1 2005 11 jsdl xmlns add http schemas xmlsoap org ws 2004 03 addressing xmlns bean http www sormaproject eu message ejsdl beans gt lt bean functionName gt DefaultPenalty lt bean functionName gt lt bean normalizationConstant gt 1 0 lt bean normalizationConstant gt lt ValueExpression gt lt Penalty gt lt Reward gt 102
102. mponent LATEST payment war Copy the war archive in the proper folder of the Web Server c Usage The Payment Component has been implemented as a web service so in order to use the component it is necessary to build a client and invoke the needed methods Since the client can be built in several ways this point will be skipped in this manual Instead the following section will explain the methods and how to invoke them String makeSubscription This method returns the URL of the chosen payment tool in string format It is used to redirect the user to the page of the payment tool to fulfil those operations that must be performed using the tool s proprietary web pages String executePayment String strXmLPayment This method takes in input a string in XML format and performs an automatic payment to the selected payment tool The input string has such format lt pay PaymentDetails xmIns pay http www sormaproject eu message sla payment gt lt pay AgreementId gt job123 Provider123 lt pay AgreementId gt lt pay Consumer gt UKA lt pay Consumer gt lt pay Provider gt CU lt pay Provider gt lt pay Price gt 10 0 lt pay Price gt lt pay CreditCardType gt Visa lt pay CreditCardType gt lt pay CreditCardNumber gt 4059042064101342 lt pay CreditCardNumber gt lt pay ExpiryDate gt 102010 lt pay ExpiryDate gt lt pay C VV2 gt 962 lt pay CVV2 gt lt pay PaymentDetails gt The output string varies depending if the operation i
103. n Value in milliseconds main main main openServerSocket main at java net PlainSocketImpl socketBind Native Method at java net PlainSocketImpl bind PlainSocketImpl java 359 at java net ServerSocket bind ServerSocket java 319 at java net ServerSocket lt init gt ServerSocket java 185 at java net ServerSocket lt init gt ServerSocket java 97 at org hsqldb HsgqlSocketFactory createServerSocket Unknown Source at org hsqldb Server openServerSocket Unknown Source at org hsqldb Server run Unknown Source at org hsqldb Server access 000 Unknown Source at org hsqldb ServerSServerThread run Unknown Source Server 16a9d42 Thread HSQLDB Server 16a9d42 5 main shutdown entered Server 16a9d42 Initiating shutdown sequence Server 16a9d42 Thread HSQLDB Server 16a9d42 5 main releaseServerSocket entered Server 16a9d42 Thread HSQLDB Server 16a9d42 5 main releaseServerSocket exited Server 16a9d42 Shutdown sequence completed in 8 ms Server 16a9d42 2009 01 15 13 34 50 551 SHUTDOWN System exit is called next Server 16a9d42 Thread HSQLDB Server 16a9d42 5 main shutdown exiting The output confirms that the port is already in use and therefore the database cannot start You need to either free up the port or find a different port to use edit server properties appropriately and change to a free port number On Linux you
104. n Dorian Service s 3 Inthe Add Remove Value s text box enter the URL of the Security services e g https lt host gt lt port gt sorma security services SecurityManager and click Add 84 3 Preferences x Y Gf Preferences A Dorian Service s k amp Grid User Management ei Authentication Service s Description Dorian Sewice s The URL s of available Dorian Service s b Grid Trust Fabric b amp Group Management Values ti ti i b D EENS Dalagatian Suwica https localhost 8443 sorma security services SecurityManager Mp move up BH H Add Remove Value s https localhost 8443 sorma security services SecurityManager Save Cancel Figure 48 Security preferences K ID 4 Click on Save to commit the changes c Logging onto the Grid Before performing any administration tasks the administrator has to be authenticated 1 Click on login in the task menu A login menu will be displayed Login Dog reate Prox Dorian Service https dorian cagrid org 6443 werf services cagrid Dorian X Lifetime 12 v hs 0 e min 0 v sec Delegation Path Length 0 Authentication Service https dorian cagrid org 6443 wsrf servicesj cagrid Dorian X Login Information User Id Password bn Authenticate Close Figure 49 Security Log in 2 From the Dorian service drop down menu select the URL pointing to your Security service 3 Similarly from Authentication Service dr
105. n PC1 and PC2 respectively The typical usage of the virtual machines in our use case is for the deployment of the already developed components such that the interaction between them can be tested on the test bed The development of the software components itself is done locally at the machines of each SORMA partner Test runs are therefore the main activity in the test bed VM Component type OS pcO Admin Linux pel Admin Linux pc2 Trading Management Linux pe3_ Security Service Linux pc4 Market Exchange Linux pc5 Bid Generator Linux pc6 Bid Generator Linux pc7 Trading Management Windows pc8 Testing tools Windows pc9 Bid Generator Windows Table 2 SORMA components installed on the virtual machines of PC1 21 VM Component type OS pcO Admin Linux pel Admin Linux pc2 TXT application Linux pe3_ MIS Linux pc4 Correlation Sys application Linux pes EERM Linux pc6 EERM Linux pc7 Bid Generator Windows DCH Correlation Sys application Windows pc9 TXT application Windows Table 3 SORMA components installed on the virtual machines of PC2 Monitoring of the virtual machine resource usage The LRM tool offers activity recording of each virtual machine with different time granularity hours days weeks and about different type of actions CPU bandwidth in terms of incoming and outgoing traffic and rea
106. nessValueList gt lt GuaranteeTerm gt lt GuaranteeTerm ws Name IndividualCPUSpeed ws Obligated ServiceProvider gt lt ServiceScope ws ServiceName myJobId1234 gt lt ServiceLevelObjective gt lt KPITarget gt lt KPIName gt jsd1l IndividualCPUSpeed lt KPIName gt lt CustomServiceLevel xsi type jsdl RangeValue_ Type xmlns jsdl http schemas ggf org jsdl 2005 11 jsdl xmlns add http schemas xmlsoap org ws 2004 03 addressing gt lt jsdl UpperBoundedRange gt 2000 0 lt jsd1 UpperBoundedRange gt lt jsdl1 LowerBoundedRange gt 200 0 lt jsd1 LowerBoundedRange gt lt CustomServiceLevel gt lt KPITarget gt lt ServiceLevelObjective gt lt BusinessValueList gt lt Importance gt 1 lt Importance gt lt Penalty gt lt AssessmentInterval gt lt TimeInterval gt PT30S lt TimeInterval gt lt AssessmentInterval gt lt ValueExpression xsi type bean Penalty Type xmlns jsdl http schemas ggf org jsd1 2005 11 jsdl xmlns add http schemas xmlsoap org ws 2004 03 addressing xmlns bean http www sormaproject eu message ejsdl beans gt lt bean functionName gt Default Penalty lt bean functionName gt lt bean normalizationConstant gt 1 0 lt bean normalizationConstant gt lt ValueExpression gt lt Penalty gt lt Reward gt lt ValueExpression xsi type bean MarketMessage Type xmlns bean http www sormaproject eu message ejsdl beans gt lt bean currency xsi nil true gt lt bean paymentType gt EITHER
107. ng used by the EERM GridRM Gateway and Custom GridRM Agent tycho mediator group name Do not change this the following value tycho gridsam suffix gridsam To run Tycho GridSAM just type java jar TychoGridSAM jar properties custom properties 44 f 3 Tycho Services Connector for Web Services The Tycho Services connector is used to bind Web Services to the EERM First you need to have running your Web Service in your application container Tomcat Apache Jboss This is a very basic operation that will depend on your service so there are not instructions for this Download the Tycho Services Connector from the DVD ivy sorma eu sormaproject TychoServices LATEST TychoServices tar gz Unpack it and create a new text file call it for example custom properties and add the next properties gridrm gateway name gridrm gateway upc es tycho mediator group name sorma eerm upc es The values of these properties MUST be exactly the same as the properties with the same name in the GridRM gateway and previously described eermConfig properties file To run Tycho Services Connector just type java jar TychoServices jar properties custom properties url http your service port service path Change ttp vour service port service path by the URL to access locally to your service Your machine is ready to accept and serve jobs if you connect via web to the EERM an
108. nstall the SORMA Trading Management is necessary to follow these steps 1 Configure a MySQL user allowed to create and modify the Trading Management database Do NOT create the database manually Deploy the C Space Protocol Executor web application Configure the Protocol Executor Upload a JAR file containing the CDA protocol to the Protocol Executor Create a CDA conversation ge a a Prerequisites No other SORMA components need to be installed prior to the installation and deployment of the Trading Management component The following SORMA components must however be installed prior to the execution of the Trading Management e SORMA Market Exchange e SORMA Market Information The following third party software must be installed before installing the Trading Management e Sun Java 1 6 e MySQL 5 e Apache Tomcat 5 5 e MySQL can be installed on a remote machine b The Installation Process Dowload the following files from the DVD e ivy sorma eu sormaproject cspace LATEST cspace war e ivy sorma eu sormaproject cspace LATEST cspace properties e ivy sorma eu sormaproject cspace LATEST cda protocol jar 1 Configure MySQL Login to MySQL mysql u root p 69 To configure a MySQL user for the Trading Management execute the following line using a mysql client grant select insert update delete create on cSpaceDB to lt mysql user gt lt cspace host gt identified by lt mysql password
109. nt will search for matches between all the bids and offers and when a match occurs it notifies the Contract Management component which will create an SLA and later send the generated SLA to SLA Enforcement which will monitor the constitutional terms on the SLA to ensure that all terms are adhered to Trading Management then sends the contract to the consumer and provider the SLA participants to notify them of the match and consequent SLA In this graph we assume that Client s Bidding Agent has previously been authenticated by the Identity Provider and that all the inter domain messages incorporate SAML assertions for authenticated communications 16 2 2 3 Resource Usage Client Application Identity Provider SLA Enforcement Contract Management EERM Resource Fabrics l Send Job Call Service l i i i d 1 l Check Client Identity I 1 t 1 1 I l i EEA dee een D 1 I l Check Contract Validity 1 i l l k l l 1 1 i 1 i l l echte i l l Send Job l 1 1 1 D 1 i Update Job Status Lisi Job Status ie x 1 i i 7 Notify Monitoring i l et i Notify Monitoring EN l 1 Update Job Status END Update Job Status END l 1 i 1 EEN 4 Notify Job Completion 1 I 1 Figure 7 job submission sequence diagram Figure 7 which again can be seen as a continuation of Figure 6 shows the interaction between the SORMA components when a job is being
110. o the technical resource description res jsdl the provider enters its economic preferences like when to submit the offer Date to submit the duration for which the resource will be offered Provided duration preferred bidding strategy reservation price minimum requested price and the bid s Time to Live Bid validity IT http www gridsphere org 47 e gridsphere portal framework Welcome I Administration sorma Economic Resource Description Technical re source de scription resl jsdl Date for submit 27 01 2009 Provided duration hours e g 3 D Select bidding strategy QStrategy Reservation price cents ng S Bid validity seconds s000 O Next Figure 25 Economic description of the offered resource Step 3 The entered preferences are shown one more time Figure 26 before the provider submits the offer request to the BidGenerator section 3 2 1 which initiates the negotiation process based on the preferences gridsphere portal framework Welcome I Administration sorma Offer Submission Technical re source de scription res1 jsdl Date for submit 27 01 2009 Provided duration hours e g 3 8 Strategy QStrategy Reservation price cents 100 Bid validity seconds 6000 Figure 26 Submission of the offer request to BidGenerator Step 4 Figure 27 shows the result of the offer request If there is an all
111. o use a text editor to view the file Look for the parts with startup sequence completed and HSQLDB server 1 8 0 is online If the database fails to start you will see output similar to gridrm kat bin gridrm tycho gateway 0 9 3 1 tail 6 nohup out Server 16a9d42 Initiating shutdown sequence Server 16a9d42 Thread HSQLDB Server 16a9d42 5 main releaseServerSocket entered Server 16a9d42 Thread HSQLDB Server 16a9d42 5 main releaseServerSocket exited Server 16a9d42 Shutdown sequence completed in 8 ms Server 16a9d42 2009 01 15 13 34 50 551 SHUTDOWN System exit is called next Server 16a9d42 Thread HSQLDB Server 16a9d42 5 main shutdown exiting At this point you need to inspect nohup out a little more closely there may already be another process listening on the port that you want to use 33 Server 16a9d42 Thread HSQLDB Server 16a9d4 3000 3 seconds 600000 10 minutes Server 16a9d42 Thread HSQLDB Server 16a9d4 server no system exit fals Server 16a9d42 Thread HSQLDB Server 16a9d4 server restart_on_shutdown false Server 16a9d42 Thread HSQLDB Server 16a9d4 entered Server 16a9d42 Thread HSQLDB Server 16a9d4 run openServerSocket java net BindException Address already in use 2 5 2497 25 37 EE Se mai
112. ocation it shows the economic data like the reserve price the generated bid the calculated payment by the market allocated job and consumer 48 82 gridsphere portal framework Welcome Administration sorma get Allocation Reservation price Bid Price PaymentJobld Consumer 100 110 120 job1 jsdijc132 23 134 105 Back Figure 27 Status information 3 3 3 EERM GUI The EERM GUI offers a control panel that can support three basic functionalities e Resources list e Clients information e Economic policies Figure 28 shows the aspect of the Resources Panel All the attached resources are listed with some related information The left icon of the image represents a resource that executes a web service and the right icon represents a batch resource Resources Clients Economic policies Add new i Attached Resources 10 0 2 169 10 0 2 200 4 CPU 4 CPU 1048576 0 MB 1048576 0 MB Figure 28 EERM resources list If you click the Add new button a window see Figure 29 for adding a new virtualized resource to the EERM will be displayed You only have to specify the desired properties and the EERM will automatically contact the Virtualization Manager for creating a custom Virtual Machine 49 Add a new fake resource Web Service O Batch Resource Service URL http www example com Resource name example CPUs a Memory MB 1024 Figure 29 adding r
113. omcat 36 d 3 Install the GridRM Portlets Copy the gridrmportlets 0 9 2 war to CATALINA HOME webapps and restart the servlet container Tomcat Enter the following command to enable the Gridrm Portlets touch SCATALAINA HOME webapps gridsphere WEB INF CustomPortal portlets gridrmPortlets Login to GridSphere and create a new Portlet group The menu can be found under the Administration tab This will allow portal users to subscribe to the GridRM Portlets For this example we use the group name of Resource Monitoring In Figure 14 the Portlet group is created by providing a group name and selecting the Portlets that should be part of the group In this instance we only select the GridRM monitoring Portlets but you can combine Portlets from different sources within a single group 82 gridsphere portal framework f f Li el Charts ACET Resources H Welcome H Portlet Group Manager HO Group Creation Wizard Group information Enter group name Resource Monitoring Enter a brief description of group Resource Monitoring Group visibility Select if group should be public or private Anyone can add themselves to a public group while private groups require administrator approval A hidden group is not displayed to users Only a portal administrator may add a user to a hidden group Please make sure a valid group administrator with valid e mail is added to the group to approve membership requests publi
114. op down menu select the same URL 4 If this is the first time you are using the Administrator UI you have to use the default administrator account to login In the User Id text box enter dorian In the Password text box enter DorianAdmin 1 7 Click the Authenticate button a 85 If the login is successful you will be authenticated and be given your credential A window similar to that shown in Figure 50 should pop up containing the details of your credential Proxy Manager ER C US O 0SU OU BMI OU caGrid Development OU Users OU OSU CN langella 1 Proxy Information Subject C US 0 0SU OU BMI OU caGrid Development OU Users OU OSU CN langella 1 CN 1324467984 Issuer C US O O0SU OU BMI OU caGrid Development OU Users OU OSU CN langella 1 Identity C US O OSU OU BMI OU caGrid Development OU Users OU OSU CN langella 1 Strength 1024 bits Expires Sat Jun 02 00 20 34 EDT 2007 Certificate Chain Subject Expires Figure 50 Proxy Manager d User Account Management e Register a New User Ne A window should pop up with a form to enter the user s information 3 Click on Apply to save the information to the database and create the user account e Removing or Modifying a User Select Account Management gt Local Accounts gt Local Account Management Ne 5 View Certificate aad Save Proxy Delete Proxy SA Set Default Close C US O OSU OU BMI OU caGrid Development OU Users OU
115. ote When this script is run the command also creates the necessary database tables in MySQL 4 7 4 Get Host Credential In order to run the security services the host that the services are deployed on need to generate host certificate and private key To obtain the certificate and key a utility script is provided Note that you should run the script as the user that starts up the Tomcat service so that the certificate and key files will be owned by that user 1 Change to CATALINA _HOME webapps sorma security bin directory 2 Run the gethostcred script and enter the appropriate values gethostcred Enter the hostname myhost Enter the directory to write the host certificate and private key to C EU O SORMA OU Security OU SORMA IdP CN dorian Successfully created the host certificate Subject C EU O SORMA OU Security OU Services CN host myhost Created Thu Jan 29 16 29 09 SGT 2009 Expires Wed Jan 29 16 29 09 SGT 2014 Succesfully wrote private key to opt apache tomcat 5 5 27 webapps sorma security bin myhost key pem Succesfully wrote certificate to opt apache tomcat 5 5 27 webapps sorma security bin myhost cert pem 3 The credential files are generated in the current directory Move the certificate and key PEM files to a secure location 78 mv pem root globus certificates 4 Verify permissions ls 1 total 16 rw r r
116. p created 38 Figure 16 Creating a new tab to host the Portlets from the Resource Monitoring Portlet group 38 Figure 17 The Resource Monitoring tab populated with Portes 39 Figure 18 All Portlets have an edit mode which is accessed via the pen Ven 40 Figure 19 Configuring Portlet behaviour using the edit mode 4 Figure 20 Initialisation of Tycho blocking API fale 42 Figure 21 BER M tain panel aeiiae arsit Eeer EE tobe sakavaav enna tans EATA a 45 Figure 22 Standard view of the Technical Resource Modellmg eeeeseeseeseceneeeeeeneeeseeeeeeeeeeaeens 46 Fig re 23 Semple yale EE 46 LE 24 ERT 47 Figure 25 Economic description of the offered resource saiis s cisansestqayssascvvncosiadecbestvanusesesesaventeaouredandess 48 Figure 26 Submission of the offer request to Bdtienerator ce eeeeseeeecesececeeeeeeseeeeeeeeeseeneeeeeeaeens 48 E 27 Status ur Na e 49 Fig re28 EE 49 Figure 29 adding resources to BERN sneci sats cataecianhauceus eles cuanconsoatd vas canacnaseaancbvdeanapsnavsexehh Eiaa aie iaaa 50 EE EE EINEN 50 Figure 31 reserving resources manually ege ett eege Eed E 50 Figure 32 testing DAC e 51 Figure 33 showing resource properties tee See EE 51 Figure 342 leese ee enee Ee ATE E a EEE eds aul oe eas 52 Figure 35 specifying preferences for resource policies sesssssesesssesesstsrtsesesrestsessessrsresessessesesses 52 Figure 36 Standard view of the Job Description section ssesssssssesssssssstsresresess
117. project logging service LogServer lt PORT NUMBER gt logserver properties 74 Example call java cp loggingService jar lib eu sormaproject logging service LogServer 16999 logserver properties c Configuration The sorma common java package contains two additional logging levels based and extending the log4j Level interface SORMALevel and DEMOLevel The SORMALevel can be used from the components to submit important messages of the running SORMA System in order to document the various processes whereas SORMADemo can be used when SORMA is used in DEMO mode server properties Following properties describes the configuration of the SORMALevel and DEMOLevel in the server properties file log4j rootCategory INFO Default DEMO The default file appender log4j appender Default org apache log4j RollingFileAppender log4j appender Default Threshold SORMA eu sormaproject logging level SORMALevel log4j appender Default File SORMALevel log log4j appender Default layout org apache log4j PatternLayout log4j appender Default layout ConversionPattern d S dbUserid c 1 p Sx Smon Do not Truncate if it already exists log4j appender Default Append true log4j appender Default MaxFileSize 1500KB Archive log files one backup file here log4j appender Default MaxBackupIndex 10 The DEMO file appender log4j app
118. quired 1 Copy build bootstrap build properties to the home directory S user home and edit the file appropriately the file contains extensive comments 2 Properties that must be set in user home build properties a build bootstrap dir The absolute path to the location of the checked out build bootstrap directory e g build bootstrap dir home user workspace SORMA build bootstrap 88 b svn ssh related properties for interacting with the Integration and Vendor repositories Account details are required NOTE We only support the use of DSA certificate based authentication to connect to the Ivy repositories using Ant 3 Check that the ant classpath dir property points to a directory on the ant class path Leave the property commented out to use the default of user home ant lib 4 Properties that should be set if behind a Corporate Proxy Server proxyhost proxyport proxyuser proxypassword aaow In order to turn on off Web proxy support just un comment the proxyhost property To configure the build environment properties and automatically download and insert the required jars to the ant classpath change to the build bootstrap directory and execute following command ant f configure xml The required Jars will now have been installed on your classpath and you are ready to perform a build If this step fails then make sure your proxy settings are correct if you are
119. r NAT Includes the exchange service as a the port must be open not service in the gmm middleware behind a firewall or NAT Includes the exchange service as a Please consider that service in the gmm middleware localhost does not work possible types are Kad Scribe and ScribeSimulator Defines the type of the used overlay Database configuration parameters Defines the reloading of the subscription service for new update name of the table Defines the sql connection driver It uses a http connection as default to pass firewalls Name of the database file username for the connection Attention existing tables will Defines if a table is created be overwritten Configuration parameters to test and evaluate the MIS defines the number of created MIS instances nodes defines the instance node which executes the testing actions like a query Defines if the test node executes a query to all nodes Defines if the test node does a publishing Defines properties for the logging Defines the path to the file with the logging properties 67 a Prerequisites The following software must be installed in order to setup and use the Payment Component 1 Java 1 6 0 2 Web Server Java based Tomcat JBoss Apache etc 3 Axis 1 2 integrated in web server b Installation To install the Payment Component you first have to get the file from the SORMA DVD ivy sorma eu sormaproject paymentCo
120. re Select portlets Edit Users Group Resource i eier Monitoring Select portlets Edit Users Resource Monitoring Create new group Edit default groups February 1 2009 Find Q Chrext_ _ Previous Highlight all Match case Figure 15 New Resource Monitoring group created Once the Resource Monitoring group has been created users including the administrator can then create a new tab and select the particular Portlets that they wish to use After logging in users can navigate to the Welcome tab layout option see Figure 16 and create a new tab in which to host the Portlets they have access to access is role based e g USER or ADMIN as alluded to in Figure 14 Notice in Figure 16 that Portlets can be laid out in a tab using 1 2 or 3 columns The width of the Portlets and amount of data displayed as well as user preference affect which column configuration is selected gridsphere portal framework crans Resource Monitoring H Layout Manager oo Theme configuration Select a theme default x Create new tab Enter new tab title O one column Two columns Three columns Display existing tabs Tabname Edittabname Delete tab Charts Charts Save Delete Resource Monitoring Resource Monitoring Save Delete February 1 2009 a fee Quins mate case Figure 16 Creating a new tab to host the Portlets from the Resource Monitoring Por
121. resources pre planned batch job allocation and dynamic re allocation In addition client demand can be satisfied not only within an organization but also among multiple administrative domains SORMA ensures the means for submitting jobs and monitoring submitted jobs The user no longer has to be concerned on which resources their jobs are being completed as long as they are performed like specified in the consumer preferences and on time The user agents act on behalf of the user being capable to create resource specifications on behalf of the user communicate with the resource brokers match the requirements with the capabilities available on the grid market and get the services needed by the user within a market based business model The SORMA platform provides resource users with intelligent tools to access the Open Grid Market using methods and tools to express the business model of the users e for capturing users preferences of each task requiring resources e for determining the bidding strategy consisting of various economic models e to estimate and monitor the quality of the resource management 1 3 Note on Security The main concept of SORMA security is the ability to have distributed identity management and single sign on This is realized in SORMA by using the SAML protocol for the exchange of security messages and interoperating with other authentication authorities The vision is that SORMA customers can authenticate in th
122. rket lt AgreementOffer ws AgreementId consumerlId providerlId 1234892098656 xmlns http schemas ggf org graap 2007 03 ws agreement xmlns ws http schemas ggf org graap 2007 03 ws agreement xmlns xsi http www w3 org 2001 XMLSchema instance gt lt Context gt lt AgreementInitiator xsi type bean Context_Type xmlns bean http www sormaproject eu message ejsdl beans xmlns jsdl http schemas ggf org jsd1 2005 11 jsdl xmlns add http schemas xmlsoap org ws 2004 03 addressing gt lt bean PartipantId gt consumerlId lt bean PartipantId gt lt bean BidId gt bidId lt bean BidId gt lt AgreementInitiator gt lt AgreementResponder xsi type bean Context_Type xmlns bean http www sormaproject eu message ejsdl beans xmlns jsdl http schemas ggf org jsd1 2005 11 jsd1 xmlns add http schemas xmlsoap org ws 2004 03 addressing gt lt bean PartipantId gt providerld lt bean PartipantId gt lt bean BidId gt offerlId lt bean BidId gt lt AgreementResponder gt lt ServiceProvider gt AgreementResponder lt ServiceProvider gt lt Context gt lt Terms gt lt All gt lt ServiceDescriptionTerm ws ServiceName myJob1Id1234 ws Name JSDL gt lt jsdl JobDefinition id myJobId1234 xmlns jsdl http schemas ggf xmlns add http schemas xmlsoap org ws 2004 03 addressing gt lt jsdl JobDescription gt lt jsdl JobIdentification gt lt jsdl JobName gt POVRAYJob lt jsd1 JobName gt lt jsdl Description gt R
123. s LEE Port number conceptually located Consumer Port description d ServerSocket side Provider side Trusted P Market machines Portal enn Port used by clients to Gees typically 8090 but Trusted Market Exchange invoke exchange service SE any can be used using web services to submit bids c Installation and Configuration The SORMA Market Exchange forms the backbone of SORMA The Market Exchange can be installed by following the instructions in this section 62 The Market Exchange can be installed by downloading the binaries from the SORMA DVD ivy sorma eu sormaproject market exchange LATEST market exchange tar gz To install decompress the binaries tar xvzf market exchange tar gz You should get the directory market exchange lt version gt created in the current directory Its structure should be e lib a directory with the exchange service code in the market exchange lt version gt jar file and its dependencies e samples a directory with the sample applications e gmm properties basic configuration for the GMM e exchange properties default configuration for the exchange service Must be adapted to the current execution environment e log4j properties configuration of message logging e launch xml ant file to launch the exchange d Configuration Steps The Exchange Service can be configured using a properties file The properties that can be configured are listed on the following page 63
124. s gt lt bean functionName gt Default Penalty lt bean functionName gt lt bean normalizationConstant gt 1 0 lt bean normalizationConstant gt lt ValueExpression gt lt Penalty gt lt Reward gt lt ValueExpression xsi type bean MarketMessage Type bean http www sormaproject eu message ejsdl beans gt lt bean currency xsi nil true gt lt bean paymentType gt EITHER lt bean paymentType gt lt bean clearingPrice gt 3 3333333333333335 lt bean clearingPrice gt lt ValueExpression gt lt Reward gt lt BusinessValueList gt lt GuaranteeTerm gt lt ServiceReference ws Name EERMEndpoint ws ServiceName Job ExecutionALL gt lt add EndpointReference xmlns jsdl http schemas ggf org jsd1 2005 11 jsdl add http schemas xmlsoap org ws 2004 03 addressing gt lt add Address gt http localhost 18080 EERM lt add Address gt lt add EndpointReference gt lt ServiceReference gt lt A11 gt lt Terms gt lt AgreementOffer gt Example 4 WS Agreement example of the contract 103 SORMA Consortium This document is part of a research project partially funded by the IST Programme of the Commission of the European Communities as project number IST FP6 034286 Barcelona Supercomputing Center BF M Bayreuth Universitat Bayreuth Cardiff University Correlation Systems Ltd FZI Forschungszentrum Informatik Hebrew University Institut f r Informationswirtschaft und management II
125. s successful or not If the operation is successful then this XML string is returned lt pay PaymentResponse xmlIns pay http www sormaproject eu message sla payment gt lt pay Outcome xsi type pay SuccessfulOutcome_Type xmlns xsi http www w3 org 2001 XMLSchema instance gt lt pay Timestamp gt 2009 01 14T02 57 19 185 08 00 lt pay Timestamp gt lt pay TransactionId gt 4PV764397R589480U lt pay TransactionId gt lt pay Outcome gt lt pay PaymentResponse gt If there are some errors then the following XML string is returned 68 lt pay PaymentResponse xmlIns pay http www sormaproject eu message sla payment gt lt pay Outcome xsi type pay ErrorOutcome_Type xmlns xsi http www w3 org 2001 XMLSchema instance gt lt pay Timestamp gt 2009 01 14T02 49 15 857 08 00 lt pay Timestamp gt lt pay ErrorCode gt ERROR_ID lt pay ErrorCode gt lt pay Description gt Internal Error lt pay Description gt lt pay Outcome gt lt pay PaymentResponse gt Note that the SORMA Message API providers parsers for this document format 4 6 4 Trading Management The SORMA Trading Management TM is a web application designed to run within a Tomcat server The TM uses a MySQL database to hold data associated with the protocols it is able to execute the so called common storage Additionally Trading Management communicates with the SORMA Market Exchange and the SORMA Market Information System To i
126. s war file to the Ivy repository i e ant publish 5 3 Further reading For further details please refer to the SORMA wiki at https portals rdg ac uk sorma mwiki index php Apachelvy 95 Appendix 5 4 Examples of SORMA Messages lt PrivateDefinition requestId request123 xmins http www sormaproject eu message ejsdl beans xmlns add http schemas xmlsoap org ws 2004 03 addressing xmlns res http ws sormaproject eu eerm reservation xmlns jsdl http schemas ggf org jsd1 2005 11 jsd1 gt lt add EndpointReference gt lt add Address gt http 10 12 13 14 18080 EERM lt add Address gt lt add EndpointReference gt lt res reservation gt lt res request gt lt res fixed gt lt res startTime gt 2009 01 10T12 00 00 05 00 lt res startTime gt lt res finishTime gt 2009 01 11T12 10 00 05 00 lt res finishTime gt lt res fixed gt lt res request gt lt res reservation gt lt jsdl JobDefinition id job123 gt lt jsdl JobDescription gt lt jsdl JobIdentification gt lt jsdl JobName gt JobName lt jsd1 JobName gt lt jsdl Description gt Description lt jsdl Description gt lt jsdl1 JobAnnotation gt lt jsdl JobProject gt lt jsdl1 JobIdentification gt lt jsdl Application gt lt jsdl Description gt Appl description lt jsdl Description gt lt jsdl Application gt lt jsdl Resources gt lt jsdl CandidateHosts gt lt jsdl HostName gt www sorma eu lt jsdl HostName gt l
127. sage pattern of the deployed components in the virtual machines 22 25000 Domain 0 20000 15000 bytes sec 10000 5000 1030 00 Figure 13 Traffic in the virtual machines during one hour Test bed evaluation Taking into account that the test bed has been operational for several months now we found that building the development test bed with the virtual machine approach was an appropriate solution for the project s purpose It could be set up very quickly is extensible and allows flexibly to changing demands regarding the virtual machine characteristics The virtual machines of the test bed allow all components to be networked and make them accessible from the Internet B Wide area distributed test bed A distributed test bed has been set up with machines contributed by Consortium partners in order to support testing over the wide area The distributed test bed has been constructed in an incremental manner with sites such as BFM and UKA starting out as separate test beds for exercising the entire SORMA system in parallel The sites are now used for scalability testing on the wide area test bed but also revert back to independent test beds as necessary to investigate issues with different SORMA configurations For example currently most of the components are deployed at UKA and are running on different physical machines in contrast to the use of virtual machines for the development test bed The availabil
128. search for the properties file if it cannot initially find it However in order to save unnecessary disk traversal during the search process ContractMangement will search only subdirectories of the current directory and will go up the directory tree only once to search other sibling directories of the current directory The name of the properties file is also important it must be called SLARuntime properties and follow the structure as outlined below 73 There are in principal two sections of the SLARuntime properties file Firstly those which are not expected to change in the short term and secondly those which are installation specific The fixed properties are presented below and define the implementation of ContractManagement In the event that the functionality of the component is to be changed it is the values specified here that will enable future developers to modify the internal implementation of the component Implementation of the SLARegistry to use SLARegistry eu sormaproject sla registry impl SLARegistryHD The Servicename of the EERM s JobManagement Servic EERM JobManager Servicename WSJobManagerService The Servicename of the EERM s SLAEnforcement Servic EERM SLAEnforcement Servicename WSSLAEnforcementService The Servicename of the EERM s EconomyManager Service EERM EconomyManager Servicename WSEconomyManagerService The namespace URI for the EERM EERM namespaceURI http ws sormaproject eu eerm ContractManagemen
129. servlet container Tomcat Tomcat will extract the Portlet project in webapps in a directory like technicalResourceModeling Enter the following command to enable the Technical Resource Modelling Portlet Portlets If the name of the directory is called technicalResourceModeling the command to be executed is like touch CATALAINA HOME webapps gridsphere WEB INF CustomPortal portlets technicalResourceModeling c 3 Administration On first start up the TRMP creates a java properties file called resources prop in the webapp directory of tomcat This file contains a variable defining the path where the created JSDL files of the Portlet are stored If you want your JSDL files to be saved in a specific location you can set the path to the directory here However the path provided must be relative to the base directory of the TRMP 30 3 2 2 Economically Enhanced Resource Manager EERM The EERM utilises resource and process monitoring services in order to fulfil its role The steps to install and configure the monitoring mechanisms are described in this section They consist of a Installation of the GridRM Gateway b Starting the GridRM c Testing the gateway installation d Installing and testing the GridRM Portlets optional task e Installing and running the EERM f Installation of the Resource Fabrics g Ready We suggest Administrators first install the GridRM Gateway then test the gateway using the S
130. sorma vendor org gridrm gridrm tycho gateway LATEST gridrm tycho gateway tar gz a 3 Extract gridrm tycho gateway tar gz e g gridrm kat binS ls gridrm tycho gateway tar gz gridrm tycho gateway tar gz md5 gridrm kat bin tar xzf gridrm tycho gateway tar gz gridrm kat bin ls gridrm tycho gateway 0 9 3 1 gridrm tycho gateway tar gz gridrm tycho gateway tar gz md5 ch HI t 31 The directory structure of the unpacked file will look like gridrm kat bin cd gridrm tycho gateway 0 9 3 1 gridrm kat bin gridrm tycho gateway 0 9 3 1 ls F classes dbservermode sh gateway sh LICENSE txt README txt sqltool sh dbmanager sh doc lib log4j properties server properties working a 4 Set the execute bit on the shell scripts gridrm kat bin gridrm tycho gateway 0 9 3 1 chmod 744 sh a 5 Edit the server properties configuration file The file is self documenting Note Gateways interact over the network with clients via a Tycho mediator There are two ways to bind a gateway to a mediator 1 Embedded The gateway starts its own embedded Tycho mediator as part of the gateway boot sequence default behaviour 2 Standalone The gateway connects to a standalone mediator that is already executing in the gateway s LAN The associated settings are explained next a 6 Properties that you must assign values to are as follows if these are not set the gate
131. source utilization valuation bid price and payment is displayed Finally depending on the executed market mechanisms the market is executing the allocation and is responding to the consumer over BidGenerator with the target provider resource where to submit the job However there exist also market mechanisms which respond with a bundle of offers to a given consumer request and leave the allocation decision to be done by the consumer s BidGenerator based on the submitted consumer preferences 82 gridsphere portal framework Welcome Administration sorma Submit Job Get Allocation Provider Resource Completion time Utilization Valuation Bid Price Payment Selection Network com 110 123 120 200 2009 01 27T 1 2 00 00 05 00 67 8 150 130 120 Job job1jsdl Back Submit Job Figure 41 Status information a 4 4 Consumers Developer Kit This section describes how end users can build or integrate their own application with the SORMA market and interface it to Resource Providers using the SORMA API and Messages 4 4 1 SORMA API SORMA API can be grouped into four main logical areas a login API b Bid Submission API c Payment API d Job Submission amp Monitoring API a Login API The Login API provides a programmatic way of authenticating to the Security service and retrieve the SAML assertion Before starting the authentication process the user s credential has to be created like
132. stall and configure the Supply Modelling Portlet in case of a web based invocation S a Install configure and test the installation of the Offer Generator a 1 Prerequisites e Tomcat 5 5 with installed JAX WS copy JAX WS jars into Tomcat s directory common lib e Jakarta s commons io jar and commons fileupload jar copy both jars into Tomcat s common lib a 2 Download from the SORMA DVD the AgentService war ivy sorma eu sormaproject AgentService LATEST AgentService war a 3 Deploy the AgentService on an installed and pre configured tomcat server As described in Prerequisites using the Tomcat WAR file deploy manager check if the AgentService is available on http lt HOSTNAME gt lt PORT gt Ag ntService AgentServic This should return information similar to that shown below Web Services Endpoint Information Service http server web sormaproject eu Address http 147 83 30 215 16789 AgentService AgentService oe e d erver web r roject e SCH Zeg oi Ay http 1 47 83 30 215 16789 AgentService ee EH ct eu Wee Stee Port Name a See Se SES d i ort name j AgentServicePort 7 aa eu sormaproject web server AgentService a 4 Setup the agent properties file in the Tomcat directory CATALINA_HOME webapps A gentService WEB INF e AgentService Set the IP and port of the AgentService e Core Market Services Set the ZP and port of the Core Market Services e Market Information Ser
133. t in this case the component build e adheres to the standard SORMA build layout e does not require custom properties to be passed to the unit integration test code notice the empty unit test properties and integration test properties propertyset elements e does not provide mock object injection see later in this document lt project name exampleComponent default test basedir gt lt property file user home build properties gt lt property file build properties gt lt import file build bootstrap dir sorma lib build xml gt lt import file build bootstrap dir ivy build xml1 gt lt propertyset id unit test properties gt lt propertyset id integration test properties gt lt project gt Any properties that have been set in your user home build properties take precedence over your component s own build properties which in turn take precedence over any properties that are set directly in the component s build xml file For example the value for the S build bootstrap dir property is retrieved from the developer s S user home build properties file see also 5 1 1 93 5 2 1 Applications When implementing an application the build xml should import the following standard template lt import file build bootstrap dir sorma lib build xml gt and override the package target as required for your application s needs For example
134. t jsdl CandidateHosts gt lt jsdl OperatingSystem gt lt jsdl OperatingSystemType gt lt jsdl OperatingSystemName gt Windows XP lt jsdl OperatingSystemName gt lt jsdl OperatingSystemType gt lt jsdl OperatingSystem gt lt jsdl CPUArchitecture gt lt jsdl CPUArchitectureName gt sparc lt jsd1 CPUArchitectureName gt CPUArchitecture gt 1 IndividualCPUSpeed gt sd1 UpperBoundedRange gt 3333 0 lt jsdl1 UpperBoundedRange gt sdl1 LowerBoundedRange gt 233 0 lt jsdl1 LowerBoundedRange gt dl IndividualCPUSpeed gt 1 IndividualCPUCount gt sd1 UpperBoundedRange gt 5 0 lt jsdl1 UpperBoundedRange gt sd1 LowerBoundedRange gt 2 0 lt jsd1 LowerBoundedRange gt dl IndividualCPUCount gt 1 IndividualPhysicalMemory gt sd1 UpperBoundedRange gt 433434 0 lt jsdl1 UpperBoundedRange gt dl LowerBoundedRange gt 3455 0 lt jsdl LowerBoundedRange gt dl IndividualPhysicalMemory gt 1 IndividualDiskSpace gt 1 UpperBoundedRange gt 45455 0 lt jsdl UpperBoundedRange gt 1 LowerBoundedRange gt 3445 0 lt jsdl1 LowerBoundedRange gt lt jsdl IndividualDiskSpace gt lt jsdl1 Resources gt lt jsdl DataStaging gt lt jsdl FileName gt fileName lt jsdl FileName gt lt jsdl FilesystemName gt fileSysName lt jsd1 FilesystemName gt lt jsdl CreationFlag gt append lt jsdl CreationFlag gt lt jsdl DataStaging gt lt jsdl1 JobDescription gt lt jsd1 JobDefinition gt lt Penalty gt lt functionName gt DefaultPenalty lt functionName gt
135. t lt Component gt gt E EERM Inform Payment Figure 4 Provider s Tools Monitor Job Submit 14 2 2 SORMA Components Behaviour This section shows how the components previously described interact in the SORMA workflow 2 2 1 Resource registration Identity Provider Bidding Agent EERM GridRM Resource J T Authenticat usernam e passwd Resource Functional Inf t l Register Resource HW Infd K nne 5 em zm vm 3 U Send_info P ricing HW E a co Figure 5 resource provider registration sequence diagram Figure 5 shows the workflow for the resource registration where the bold grey discontinuous line represents domain separation The figure shows how the EERM is authenticated in the Identity Provider to be allowed to send secure messages to SORMA Grid Market domain When a new hardware Resource is attached it notifies the EERM of its existence and sends some functional information for example its host name and if the resource hosts a Web Service or a GridSAM middleware for batch executions The resource must also be registered on GridRM gateway and then GridRM will notify the EERM with the availability of a new Resource The EERM is now able to query the Hardware description and Monitoring Data With the data provided by the resource and the GridRM gateway the EERM is now able to create an EJSDL
136. t Interface to use ContractManagement eu sormaproject sla impl ContractManagementImpl SLAEnforcement Interface to use SLAEnforcement eu sormaproject sla enforcement impl SLAEnforcementImpl The implementation specific properties are presented below and define the settings which are context specific Note that it is necessary to know where the payment component is prior to the first invocation of the ContractManagement component Location of the Payment Component ayment location http 147 83 30 246 12080 SORMA Payment services Payment Location of the SLA Files for different operating systems this is important as t defines where the component will output all documents related to the SLA ifecycle LARegistryHD location Linux home scmsjc LARegistryHD location Windows XP D WH Pe ID H 4 6 6 SORMA Global Logging a Prerequisites e log4j e Optional Chainsaw v2 GUI for displaying the log4j messages which is running like a log server and listening to a specified port b Installation Steps e Download and unpackage the LoggingService from the SORMA DVD ivy sorma eu sormaproject loggingService LATEST loggingService tar g Z e Configure the logserver properties see section Configuration e Start the LogServer java cp loggingService jar lib eu sorma
137. t ServiceLevelObjective gt lt KPITarget gt lt KPIName gt jsd1l IndividualCPUCount lt KPIName gt lt CustomServiceLevel xsi type jsdl RangeValue_ Type xmlns jsdl http schemas ggf org jsd1 2005 11 jsdl xmlns add http schemas xmlsoap org ws 2004 03 addressing gt lt jsdl UpperBoundedRange gt 4 0 lt jsdl1 UpperBoundedRange gt lt jsdl LowerBoundedRange gt 2 0 lt jsd1 LowerBoundedRange gt lt CustomServiceLevel gt lt KPITarget gt lt ServiceLevelObjective gt lt BusinessValueList gt lt Importance gt 1 lt Importance gt lt Penalty gt lt AssessmentInterval gt lt TimeInterval gt PT30S lt TimeInterval gt lt AssessmentInterval gt lt ValueExpression xsi type bean Penalty Type xmlns jsdl http schemas ggf org jsd1 2005 11 jsdl xmlns add http schemas xmlsoap org ws 2004 03 addressing xmlns bean http www sormaproject eu message ejsdl beans gt lt bean functionName gt DefaultPenalty lt bean functionName gt 101 lt bean normalizationConstant gt 1 0 lt bean normalizationConstant gt lt ValueExpression gt lt Penalty gt lt Reward gt lt ValueExpression xsi type bean MarketMessage Type xmlns bean http www sormaproject eu message ejsdl beans gt lt bean currency xsi nil true gt lt bean paymentType gt EITHER lt bean paymentType gt lt bean clearingPrice gt 3 3333333333333335 lt bean clearingPrice gt lt ValueExpression gt lt Reward gt lt Busi
138. t view History Bookmarks Jools Help E hips ilocaihost e443 soma secuntyiservices https localho curity services La gorse o alterSecurityDesc o setAnonymousAuthz getSecurityProperty e authenticate 7 o registerWithldP o findidPUsers o updateldPUser o removeldPUser o authenticateWithidP o createProxy getCACertificate o getTrustedidPs o addTrustedidP o updateTrustedidP o removeTrustedidP o findiFSUsers o updatelFSUser o removelFSUser o renewlFSUserCredentials o getlFSUserPolicies addAdmin o removeAdmin o getAdmins o requestHostCertificate o getOwnedHostCertificates a annraveHnetCertificate z localhost8443 Figure 44 Security screenshot 4 7 6 Testing the Security services with a Sample Client 1 Download the binaries of the sample client from the SORMA DVD ivy sorma eu sormaproject security LATEST examples zip 2 Unzip the archive and cd into the directory 80 S unzip examples zip cd examples 3 Set up the Java classpath by sourcing the setenv sh file in the etc directory etc setenv sh 4 Test that your classpath is correct by running the sample client java eu sormaproject security examples LoginClient Please specify the service URL 5 Create a certs directory in the examples directory mkdir certs 6 Obtain the public certificate of the CA used by the Security service
139. te a bid and submit it to Market Exchange ee tees OLevel DEMO Market Exchange receives the bid and addresses the bid message to the auction theAuction a logger log DEMOLevel DEMO Trading Management receives the bid and tries to match it with a stored offer 4 7 Howto Apply Security Infrastructure The Security component is implemented as a set of services built on top of the Dorian security framework The security services are implemented as WSRF services using Globus WS Core framework This guide describes the procedures for deploying the security services in an Apache Tomcat web container The operating system is assumed to be Linux 4 7 1 Prerequisites 1 Java JDK 1 5 0_17 or later version 2 Ant 1 6 5 3 MySQL 5 0 4 Apache Tomcat 5 5 X 4 7 2 Deploying the Services Before deploying the services we need to copy several Jar files from the Globus WS core framework to some of the Tomcat directories From here on in this guide we refer CATALINA HOME as the directory where Apache Tomcat is installed 1 Download the required Jar files from the SORMA DVD ivy sorma eu sormaproject security LATEST sorma security tomcat lib zip Unzip the file to a temp directory Copy cog tomcat jar to CATALINA HOME server lib Copy xalan jar to CATALINA_HOME common endorsed Copy the following files to CATALINA _HOME common lib 1 cog jglobus jar 2 xml apis jar 3 crypt
140. ternal database user name gridrm gateway internal database user password gridrm gateway internal database url server b Starting the GridRM gateway This section describes how to execute the GridRM Gateway on a Linux platform using the provided shell scripts If you wish to execute the gateway on a non Linux UNIX platform see the section below detailing how to execute using the ant scripts b 1 Starting the database For the moment the GridRM gateway uses a database that is started separately from the gateway the database is to be embedded within the gateway itself in the future For now start the database using the following command gridrm kat bin gridrm tycho gateway 0 9 3 1 nohup dbservermode sh amp 1 22198 nohup appending output to nohup out Note The nohup command means that the database will continue running when you log out of your terminal session The standard output will be saved in the file nohup out You can inspect the file to ensure that the database started OK e g gridrm kat bin gridrm tycho gateway 0 9 3 1 tail 6 nohup out Server 16a9d42 Startup sequence completed in 543 ms Server 16a9d42 2009 01 15 13 26 08 779 HSQLDB server 1 8 0 is online Server 16a9d42 To close normally connect and execute SHUTDOWN SQL Server 16a9d42 From command line use Ctrl C to abort abruptly Server 16a9d42 Thread main 5 main start exiting You could als
141. this event to be notified when a new driver is added to a GridRM gatewa Io New user registration Subscribe to this event to be notified when a new user is added to a GridRM gateway IMofault driver undatad A racnurco e dof driver hac haan channad tn a diffarant dri Za Segen Ka Figure 20 Initialisation of Tycho blocking API failed After the first test passes successfully use the Query Core Attributes Portlet to perform a query against a gateway If you do not receive any results then check the following e The specified gateway is running e Bi directional communications between the gateway s mediator and the standalone mediator used by the Portlets e Check with the gateway administrator to determine mediator settings both mediators should be in the same group e Check the CATALINA_HOME logs catalina out for signs of problems e Installing and running the EERM e 1 Pre requirements e Apache Ant 1 7 e Apache Tomcat 5 5 or higher e Java 6 SDK You should also ensure that the following environment variables are correctly configured export JAVA _HOME your jdk6 path export CATALINA HOME your tomcat path 42 e 2 Download from the SORMA DVD EERM war ivy sorma eu sormaproject EERM LATEST EERM war e 3 Unpack EERM war to CATALINA_HOME webapps directory e 4 Edit file SCATALINA_HOME webapps EERM WEB INEF classes eermConfig properties and set the next properties property name description agent t
142. tion code compilation and packaging Adopting best practices for Apache Ant based development SORMA offers various standard templates build files for building libraries applications Web Services and JSR 168 portlets The standard build templates located in the src build bootstrap directory on the DVD are imported into component specific build files resulting in the simplest case in developers only having to create a simplistic 7 line long build file in order to build package and publish their component to the SORMA wide integration testing repository The standard build templates reduce redundancy across the project and do not require developers to become experts in using Apache Ant http ant apache org manual index html 25 Apache Ivy is a tool that enables the automated management recording tracking resolving and reporting of software dependencies Apache Ivy is powerful tool which is tightly integrated with ant and enables a flexible configuration of dependencies It provides various Ant tasks ranging from dependency resolution to dependency reporting and artifact publication An important feature is that Ivy resolves transitive dependencies between components and libraries In SORMA we have configured a number of Ivy repositories that contain both SORMA developed components as well as 3 party dependencies that are required to build and execute SORMA components Repositories include l Release Cont
143. tion titled Event Subscriptions where you can subscribe to receive events from the associated gateway 40 24 GridRM Gateway Events BOs You can register to receive events from a particular gateway Select the events you wish to receive make sure the gateway details are filled in correctly and then press the Save details button NOTE Currently you cannot cancel event registrations by clicking on an already selected event This functionality will appear soon Event subscriptions Select the events you wish to receive from the gateway X Event Type Description m New resouce Subscribe to this event to be notified when new resources are added to a GridRM registration gateway O Resource unregistered Subscribe to this event to be notified when a resource is unregistered from the gateway m New driver Subscribe to this event to be notified when a new driver is added to a GridRM registration gateway m F Subscribe to this event to be notified when a new user is added to a GridRM _ New user registration gateway Io Default driver updated A resource s default driver has been changed to a different driver m Resource failing to Subscribe to this event to be notified when a resource failed to respond when respond queried m Metric collection Subscribe to this event to be notified when a client complains that certain metrics partial failure are not populated in the response to a resource query
144. tionService 70 MEA Message Layer MEA Exchange Message Layer Note cspace messageLayer sorma exchange client port is the first port the client will try to open for incoming connections from the service host If it fails to open this one it will increment the port number by and try again etc cspace messageLayer sorma factory className eu sormaproject cspace exchange ExchangeMessageLayer Factory cspace messageLayer sorma exchange service host 147 83 30 215 cspace messageLayer sorma exchange service port 14080 cspace messageLayer sorma exchange client port 8090 Test Message Layer cspace messageLayer test factory className eu sormaproject cspace message TestMessageLayer Factory cspace messageLayer test port 8888 Edit the values to suit your setup The properties with prefix cspace public are used to determine how external clients should connect to the C Space service Properties with prefix cspace mysql are used to setup a connection to the database you configured previously Make sure the two following properties are set as follows cspace messageLayer namespace cspace messageLayer sorma cspace informationService namespace cspace informationService sorma This will tell C Space to use the properties with prefix cspace messageLayer sorma for setting up the messaging layer and properties with prefix cspace informationService sorma for the information service Therefore regarding the m
145. tions to section 1 Added BSC contributions section 2 3 Added Reading contribution to sect 1 2 Added Provider s section 3 0 5 13 02 2009 M Koh SUN G Smith URE F Freitag UPC N Borissov UKA Added Security components info Added section 1 1 1 DVD structure Added contribution to section 2 Added Consumer s Intelligent Tool to chapt 4 Added SORMA Administrator guides 0 6 23 02 2009 N Borissov UKA M Macias BSC Updated 5 4 1 Logging GUI Added EERM GUI for Providers sect 3 3 2 0 7 25 02 2009 M Villa TXT N Borissov UKA S Caton CU Completed introduction Completed sections 4 3 2 New section 3 3 2 Contributions to section 4 4 2 New sections 5 2 5 and 5 4 2 0 8 5 03 2009 M Macias BSC S Caton CU P Chacin UPC M Villa TXT Contributions to sect 4 4 1 Contribution to section 2 1 2 2 Update to setion 2 3 0 9 6 03 2009 N Borissov UKA P Chacin UPC updated 3 1 2 c 2 Installation Updated section 4 2 Updated sect 2 1 core market services 0 9 1 amp 2 10 03 09 N Borissov UKA M Macias BSC M Koh SUN Small updates in sections 3 3 2 and 4 3 2 Updated section 2 2 Updated 4 4 1 Added new section 5 3 7 0 9 3 11 03 09 A Anandasivam Updated section 2 3 1 0 9 4 11 03 09 S Caton CU Setting of anchors and one new comment for M Macias 0 9 5 11 03 09 G
146. tlet group 38 Figure 17 shows the Resource Monitoring tab in action Notice the controls at the bottom of the page for adding Portlets to the tab in a user defined order GridRM Gateway List Resources Current gateway gridrm gateway portals acet rdg ac uk Registered resources Name Default Driver Registered on Registered by Remove GridRM Gateway Query resource core attribu Current gateway gridrm gateway portals acet rdg ac uk Operating System any bei Memory MB CH any EZ Processor Loads Last 1 min ES CH any e Gateway Cache Policy Entries lt 64 minutes old ES Submit query to the GridRM Gateway dd portlets Subscribe and receive GridRM events ES Add February 1 2009 GridSphere Portal IG Google Q Logout Welcome administrator GridRM Gateway Events Current gateway gridrm gateway portals acet rdg ac uk _Recent event notifications Timestamp Type Message GridRM Gateway Add Resource Current gateway gridrm gateway portals acet rdg ac uk Add new resource SNMP v1 E Admin Name Driver Port if not default Agent user name Agent passphrase Add portlets Subscribe and receive GridRM events 1 Add C Match case Find Q Lakhoo Figure 17 The Resource Monitoring tab populated with Portlets d 4 Configure the GridRM Portlets The resource monitoring Portlets are now installed but before th
147. ttp schemas xmlsoap org ws 2004 03 addressing gt lt jsdl CPUArchitectureName gt x86 lt jsd1l CPUArchitectureName gt lt CustomServiceLevel gt lt KPITarget gt lt ServiceLevelObjective gt lt BusinessValueList gt lt Importance gt 1 lt Importance gt lt Penalty gt lt AssessmentInterval gt lt TimeInterval gt PT30S lt TimeInterval gt lt AssessmentInterval gt A A A A 100 lt ValueExpression xsi type bean Penalty Type xmlns jsdl http schemas ggf org jsd1 2005 11 jsdl xmlns add http schemas xmlsoap org ws 2004 03 addressing xmlns bean http www sormaproject eu message ejsdl beans gt lt bean functionName gt DefaultPenalty lt bean functionName gt lt bean normalizationConstant gt 1 0 lt bean normalizationConstant gt lt ValueExpression gt lt Penalty gt lt Reward gt lt ValueExpression xsi type bean MarketMessage Type xmlns bean http www sormaproject eu message ejsdl beans gt lt bean currency xsi nil true gt lt bean paymentType gt EITHER lt bean paymentType gt lt bean clearingPrice gt 3 3333333333333335 lt bean clearingPrice gt lt ValueExpression gt lt Reward gt lt BusinessValueList gt lt GuaranteeTerm gt lt GuaranteeTerm ws Name CandidateHosts ws Obligated ServiceProvider gt lt ServiceScope ws ServiceName myJobId1234 gt lt ServiceLevelObjective gt lt KPITarget gt lt KPIName gt jsd1
148. ugh the provider s organisational firewall in order that the provider side components can communicate with the core SORMA services that are running on the SORMA test bed In addition a number of ports are required to be open between the machine s hosting the SORMA provider services During individual components installation instructions ports will be specified whenever necessary 3 2 Installation and Configuration Guide 3 2 1 Provider side intelligent tools The provider side Intelligent Tools consist of Business Modelling Supply and the Offer Generator components The Business Modelling component provides a message format for describing economic and technical preferences The technical and economic preferences are specified in EJSDLPrivate templates see section 4 4 which are stored locally and can be individual for each resource type Each time a resource is free the local provider EERM invokes the Offer Generator Web services through submitting of an EJSDLPrivate document The Supply Modelling component offers a Web based GUI Portlet to enable simplified description of technical requirements for jobs imported and exported in JSDL data format The same GUI is used for a technical description of a provider resource offer Installation instructions consist of 27 ef Install configure and test the installation of the Offer Generator Use pre defined EJSDLPrivate templates in case of an application based invocation c In
149. undedRange gt 10000 0 lt jsd1 UpperBoundedRange gt lt jsdl LowerBoundedRange gt 5000 0 lt jsdl LowerBoundedRange gt lt jsdl IndividualDiskSpace gt lt jsdl1 Resources gt lt jsdl DataStaging gt lt jsdl FileName gt fileName lt jsdl FileName gt lt jsdl FilesystemName gt fileSysName lt jsd1l FilesystemName gt lt jsdl CreationFlag gt append lt jsdl CreationFlag gt lt jsdl DataStaging gt lt jsd1 JobDescription gt lt jsd1 JobDefinition gt lt Bid gt lt bidPrice gt 50 0 lt bidPrice gt 97 lt requestType gt BID lt requestType gt lt participantId gt participantl lt participantId gt lt timeout gt 100000 lt timeout gt lt duration gt 2000 lt duration gt lt serviceType gt WEBSERVICE lt serviceType gt lt paymentType gt EITHER lt paymentType gt lt Bid gt lt BidDefinition gt Example 2 the bid message EJSDL lt MarketDefinition marketMessageId bidId offerId xmlns http www sormaproject eu message ejsdl beans xmlns add http schemas xmlsoap org ws 2004 03 addressing xmlns res http ws sormaproject eu eerm reservation xmlns jsdl http schemas ggf org jsdl 2005 11 jsd1 gt lt add EndpointReference gt lt add Address gt http localhost 18080 EERM lt add Address gt lt add EndpointReference gt lt res reservation gt lt res request gt lt res fixed gt lt res startTime gt 2008 11 10T12 00 00 05 00 lt res startTime gt lt res finishTime gt 2008
150. vice Set the ZP and port of the Market Information Service e Trading Management Set the target auction name running on Trading Management This should be later automated by requesting a description of the registered auctions from the Core Market Services a 5 Configuration Edit the following values to suit your setup 28 HEREH EEE HHH Hitt BidGenerator specific properties FE AE AE TE AE AE FE FE E H H FEF Public address of the AgentService Request Executor bidgenerator public host localhost bidgenerator public port 8080 bidgenerator policy dir policy Public address to the Market Information Service bidgenerator mis host pcmargenat ac upc edu bidgenerator mis port 8880 Static properties for Trading Management bidgenerator auctionName theAuction Address to which the Core Market Services Market Exchange respond to the BidGenerator bidgenerator exchange service host localhost bidgenerator exchange service port 8090 Core Market Services Market Exchange specific properties client channel channel http Public address to the Core Market Services Market Exchange to which the bid is submitted channel http param localAddress localhost channel http param port 8181 channel http param verbose true Define Channe
151. way will refuse to start tycho gridrm gateway name tycho mediator group name tycho gridrm gateway name This is the Tycho name this gateway is registered with To ensure uniqueness it is best to use a scheme similar to the Internet Domain Name System fully qualified hostnames Note The name you choose does NOT already have to be registered in the DNS If the name below is not unique within the Tycho environment then Tycho will produce a unique name However if you provide a unique name here then your clients will have an easier time of locating a specific gateway when they perform a search tycho mediator group name The tycho mediator group name is the name used by Tycho mediators All mediators that want to be able to find each other have to be in the same group a 7 Optional properties that you may wish to set include tycho mediator socket address socket lt standalone mediator name gt 3535 This tells the gateway to connect to the standalone mediator listening at the specified address instead of using the default embedded mediator If a standalone mediator has been selected then the tycho mediator group name property is ignored The gateway uses a hypersonic database for internally storing data and configuration data Optionally you can update the following database properties for example if you find that the port required by the database is already in use 32 server port gridrm gateway in
152. windows 0 04 running 0 10 400 fixed DCH windows 0 04 running 0 10 400 fixed DCH windows 0 04 running 0 10 400 fixed Figure 10 Deployed VMs in PC1 20 LRM Client Beta 0 2 userl connected to 147 83 30 2 Action hr for help VM ID OS share Status Price QoS MHz QoS type DCH linux 0 10 shutdown 0 25 1000 fixed pcl linux 0 10 shutdown 0 25 1000 variableLow pc2 windows 0 10 running 0 25 1000 variableMedium pc3 Linus 0 10 running 0 25 1000 variableHigh pc4 linux 0 04 running 9 10 400 fixed DCH Linux 0 05 running 0 12 500 variableMedium pc6 linux 0 07 running 0 17 700 variableHigh pc7 windows 0 07 running 0 17 700 variableLow DCH windows 0 07 running 0 17 700 fixed DCH windows 0 07 running 9 17 700 fixed Figure 11 Deployed VMs in PC2 Deployed SORMA components SORMA developers have basic access with ssh to the virtual machines both Linux and Windows Alternatively the access with a graphical interface to the Windows virtual machines is possible using the Remote Desktop tool which is available both in Windows and Linux The graphical access to Linux virtual machine can be achieved with tools similar to VNC The virtual machines are assigned to the developers of SORMA where each developer works on one or more virtual machine The components of the software project are networked Some apply the client server model others use a Peer to Peer infrastructure for communication Table 3 and Table 4 indicate the components deployed o
153. wn client application accessing the SORMA Market through the SORMA API see next section 4 4 1 e Resource Providers will be running the provider side Intelligent Tools and the EERM in order to offer their resources to the SORMA market and to Consumers e SORMA Market will be running all the other components namely the Core Market Components and the Security Services 18 Consumers Client Intelligent Tools Trusted Market Exchange EERM VEIG Trading Contract Inform Mgmt Mgmt Provider VET cis Payment Market Intelligent Tools Director Comp Logging User Si Registration User Profiles Sorma Grid Market Resource Provider s Figure 9 high level components deployment view 2 3 1 SORMA Test beds In order to test integration components have been deployed onto two main test beds 1 the development test bed and 2 the wide area distributed test bed A Development test bed For the deployment and testing of components a test bed consisting of virtual machines was set up in the second year of SORMA at the UPC The test bed consists of two PCs PC1 and PC2 each with a quad core processor 4 GB of RAM and 250 GB of disk For the virtualization Xen is used The operating system is Linux Fedora 8 For the management of the virtual machines in each PC a tool developed at UPC called the Local Resource Manager LRM is used This type of test bed for the purpose of providing quickly a flexible test bed for SORMA has t
154. ycho name put some agent name for the EERM e g eerm vm txt it http localhost 18080 EERM WSProxy lt change localhost by the IP address of your tomcat server and 18080 by the port number Put the name of the gridRM gateway It must be the same as specified in the property called tycho gridrm gateway name of the file server properties in your GridRM gateway directory eerm proxy service gridrm gateway name tycho mediator group name some name the tycho mediator group for example sorma eerm txt it http 147 83 30 246 17567 AgentService AgentService wsdl lt Change the IP and the port of the agentservice url AgentService with the address of the AgentService machine to use e 5 Now to run EERM you only have to run the tomcat server CATALINA HOME bin catalina sh start f Installation of the Resource Fabrics f 1 Resource monitoring daemon There are a number of different ways that resources can be monitored depending on their type and any existing agents that are present For the purposes of this guide we will describe how to monitor using Ganglia the popular cluster monitoring tool used in many Grid installations First of all you need to have installed a ganglia monitoring daemon In Ubuntu or Debian it s easy to install it sudo apt get install ganglia monitor gmetad To start it run as root the next commands gmond start gmetad start To check that Ganglia is running type teln

Download Pdf Manuals

Related Search

Related Contents

Manuel d`utilisation  Preguntas frecuentes.  ASM S/390 NOAM User`s Guide  Prêparation / Bereidingswijze  IC-756PROⅡ - TECHNICAL REPORT  Istruzioni operative per l`uso del KXE  Samsung SGH-Z240 User Manual  Logstep  

Copyright © All rights reserved. Failed to retrieve file