ASSIST2CCM User Manual
Contents
1. Figure 2 which contatins the IDL3 files of the component its source code and the compiled output A sub directory named ASSIST2CCM TASKCCM demoleventi if we are using events Figura 2 which contains the IDL3 files for the events the source code and the compiled output The compiled ASSIST program extended with the CCM input and CCM output modules in the target directory chosen in the ast rc file ASSIST2CCM TASKCCM demo eventi componenteG Event IDL3 files Component IDL3 Compiled files for the Compiled files for the event component Figure 2 Directories created within the demo sub tree by assist2ccm sh 12 4 Some Example CCM Applications Regression Tests In the SASSIST2CCM TASKCCMl demolarchivio sub directory the user will find the archived regression tests Each test is a CCM application with a component that will be completely created by the assist2ccm sh script starting from an ASSIST program and with one or more CCM components communicating sending and receiving events with the CCM component that wraps the ASSIST program 4 1 REGRESSION 1IN The first test is composed by two CCM components the first ev moduloA sends data to a second component solo inG which wraps an ASSIST program The code of this test is made up of A directory named assist da analizzare which contains the ASSIST program code solo in ast Using the assist2ccm sh script assist2ccm sh solo in ast solo inG 0 we gener2. OpenORB 1 4 0 2 specify ORB home dir I disc 1 OpenORB 3 Run build sh install build sh install 4 Run again build sh to compile OpenCCM build sh 5 Run again build sh install to perform the final installation of OpenCCM build sh install Last three more variables need to be set properly 1 The OpenCCM_HOMEDIR variable is set to the directory where CCM executables can be found export OpenCCM_HOMEDIR CCM_HOME opencem build 2 The OpenCCM CONFIG DIR variables tells where temporary files created during the process of compiling and running CCM programs must be stored This temporary directory is created and removed at each compilation or run export OpenCCM CONFIG DIR CCM HOMBE openccm tmp 3 The path of CCM executable files must be added to the PATH export PATH PATH CCM_HOME openccm build bin 1 2 Installing OpenORB OpenORB is distributed in precompiled packages JAR Java Archives and is ready to be used The only manual intervention is moving two JARs from the NamingService installation directory to the OpenORB tree cp NamingService lib openorb tns 1 4 0 jar OpenORB lib cp NamingService lib endorsed openorb ns omg 1 4 0 jar OpenORB lib endorsed Then we need to set two environment variables 1 The OpenORB LIB variable tells where OpenORB JARs are stored export OPENORB LIB TCOO HOME OpenORB lib 2 TOOLS is used as a pointer to OpenORB tool JARs export TOOLS TCOO_HOME tools l
3. ASSIST2CCM User Manual Sonia Campa Massimo Coppola Silvia Magini Laura Potiti Diego Puppin September 2004 Table of Contents 1 Installing OpenCCM 0 8 1 and OpenORB 1 4 0 BETA eene 3 Lol Installing Open M stretto portis RU Petras p ea EM MEO vac a Md pode t cused OO Ida iN adde EE 4 1 2 Installing OpenOR B tace tiat edo uias au dat ud ses po ipe EV DU a etu an 5 1 3 Cluster pianossa and C luster o l i ieu eR ra e RESI v leo Rasse ns US dE Dua oae RUNE d Up 5 2 Installing and Using ASSIST2CCM 1 0 ated teet ae oed te aint sac bsp reote ved cake 6 2 1 The LASKCEM directory iiie ceto ee oe PUER URS cag cade Code VOR E ARA E OR MERECE ES Rde vents 7 2 1 Available SCEDpts iii e me oe P e REY Ra a QUT CE Verdi ue Pet E dae UPS nde ga 8 Zl E The env C CM SCHDE soos Datei Urso adu taped ES 8 2 1 1 2 Scripts to Compile Component IDL3 and Java Files enses 9 2 1 1 3 Scripts to Launch and Stop the CCM Application eeeeeeeeene 9 Zw T Dh ASSISI2 COMMIS HD OTT cases pea ac a eX S Maece dele atop nd oe ea ded edle ie 10 2 1 2 Tested Programs Cassist Solo caSSist SEQ i7 sober PREISE at aei S RO EUPRRRISITPAU SR eMe seacdedsoese 10 2 1 3 The CDR IO and CDRServer classes SOCKet cccsssscccceceeeesessessecececeesesessssaeeeeeceens 10 2 14 The XML Patser parser X NIE sio sedis on ande to NIE RON ER EON NE Dem EU e te ERN MR RUE 11 2 1 5 The demo Directory an
4. In the following tables we describe the timing of the sequential running and of the CCM applications varying the number of virtual processors for the ASSIST module SEQUENZIALE 717 96 Table 4 Sequential timing of the benchmark 15 Figure 5 Timing of the CCM application Speed up co eo eo 7 00 Speed up 6 00 v Speed up ideale I I I I I I I I I I I 1 2 3 4 5 6 7 8 9 1011 12 13 14 1 NVPM T4 5 16 Figure 6 Speed up of the CCM application Below we report the bandwidth that we can reach by changing the granularity of communications o3 c9 40 3496 160 censes 640 4325950 2560 3504097 sto 3544677 10240 4374958 Figure 7 Bandwidth with the event implementation 16 5 2 Performance RMI 5 RMI requests 5 RMI requests Figure 8 Structure of the CCM application The test is made up of three components The first and the second component C1 and C2 send their requests to the Assist module using the RMI method esegui assist The total number of requests is 10 Requests from each client are executed sequentially so in order to exploit communication computation parallelism within the wrapped application more than one client is needed Two clients are enough in this settings to reach good speed up 2 The third component wraps the ASSIST program It accepts requests from the first an
5. ate and compile the used events formerly ASSIST streams and the solo inG component which wraps the ASSIST program A directory named ev moduloA which contains the code IDL3 and Java source code of the ev moduloA component A directory named mymain with the main code of the application 4 2 REGRESSION 1IN 1OUT The second test is composed by three CCM components the first ev moduloA sends data to a second component bench dpG which wraps an ASSIST program The latter forwards them to a third component ev moduloB The code of this test is made up of A directory named assist da analizzare which contains the ASSIST program code bench dp ast Using the assist2ccm sh script asynchronous assist2ccm sh bench dp ast bench dpG 0 synchronous assist2ccm sh bench dp ast bench dpG 1 we generate and compile the used events formerly ASSIST streams and the bench dpG component which wraps the ASSIST program Two directories named ev moduloA and ev moduloB which contain the code IDL3 and Java source code of the two components A directory named mymain with the main code of the application 13 4 3 REGRESSION 2IN 1OUT The third test is composed by four CCM components two components ev moduloA and ev moduloC send data to a component bench dpG which wraps an ASSIST program The latter forwards them to the last component ev moduloB The code of this test is made up of A directory named assist da anal
6. ble to generate the executable code for the CCM wrapper component and for the ASSIST program extended with the CCM input and CCM output modules In ASSIST2CCM Implementation more details on this script can be found It can be launched with 2 3 or 4 parameters 1 the first parameter is the name of the ASSIST file to be analyzed 2 The second one is the name of the ASSIST module sequential parmod or generic that the user wants to be wrapped with CCM This is also the name of the CCM module generated by the tool 3 The third parameter optional let the user choose between a synchronous 1 or asynchronous 0 component implementation Default is asynchronous It is not possible to use a synchronous implementation when the number of input events streams is different from the number of output events streams 4 The fourth parameter optional is the name of the ASSSIST RC file ast rc to be passed to the ASSIST compiler If this file is not specified then the default ast rc will be used that is the one available in the HOME directory 2 1 2 Tested Programs assist solo assist seq Three directories of tested examples are offered with taskccm In the seq directory seq users will find the source code of a sequential C implementation of bench dp In the solo assist directory users will find the source code of bench dp and prod matrice written with pure ASSIST and the used ast rc file In the assist directory users wi
7. d the second component elaborates them and sends the results back In the following we will call the cluster nodes u3 u24 The CCM and ASSIST processes are allocated as follows CCM input Assist CCM output Assist and the second component CCM are on node ul2 the ZSM process and one VPM Assist are on node u13 the OSM process and one VPM Assist are on node u 4 the client components ClientA and ClientB CCM are on node u23 the other VPMs ASSIST are allocated one on each of the unused nodes In the following tables we describe the timing of the sequential running and of the CCM applications varying the number of virtual processors for the ASSIST module SEQUENZIALE 717 96 Table 9 Sequential timing of the benchmark 17 Speed up NVPM Bench dp 649 08 326 40 17295 8 s 55 69 Table 10 Timing of the test CCM application T DT T og 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 VPM Table 11 Speed up of the CCM application 18 Speed up v Speed up ideale
8. d Archived Tests demo demo archivio 11 J USME PS 2 COM aac ack oti Ra cei eun redis ntu nus sates RSA estes fs 11 4 Some Example CCM Applications Regression Tests ccessccceceseecesesneeceeeeeeessnaeeeeenaeeceeeeeeeee 13 dU REARS LOIN IN needed tto p eei mde eaa atc EE 13 T2 REGRESSION TIN IOUT ninenin annan te cedag bee epar ee e basta denda atu ieee 13 4 3 REGRESSIGN 2IN TOUT aime eerta dod fodit taies qe aim dites 14 4 4 REGRESSIONC2IN OE Distineo ae oh cto e aki esca Sao tastan de du Eais 14 auos le PENNE dead Sedat as C EE 15 3 L Fertormance s Events ot ascona aono socios sid ats Wales cadet ata Seca eii sciens 15 5 2 Performance RME i ris a eaceti ue sags ace as dac oe Ea ten PL rUE pois 17 List of Figures Figura 1 Structure of the ASSIST2CCM 41 0 tree 6 Figure 2 Directories created within the demo sub tree by assist2ccm sh 12 Figure 3 Structure of the test CCM application 15 Table 4 Sequential timing of the benchmark 15 Figure 5 Timing of the CCM application 16 Figure 6 Speed up of the CCM application 16 Figure 7 Bandwidth with the event implementation 16 Figure 8 Structure of the CCM application 17 Table 9 Sequential timing of the benchmark 17 Table 10 Timing of the test CCM application 18 Table 11 Speed up of the CCM application 18 1 Installing OpenCCM 0 8 1 and OpenORB 1 4 0 BETA1 In the following we assume bash shell is used and that JAVA
9. ee Chapter 4 and many tested CCM applications In order to run these tests must be unpacked into the demo directory 3 Using ASSIST2CCM We summarize here how to run ASSIST2CCM version 1 0 1 Set the ASSIST2CCM environment variable Set the ASSIST2CCM environment variable to point to the directory where the package is installed export ASSIST2CCM HOME ASSIST2CCM 2 Compile the analyzer tool Perform make within the ASSIST2CCM analizzatore directory to compile the analyzer cd ASSIST2CCM analizzatore make 3 Customize the env_CCM script Using the env CCM c1 and env_CCM_pianosa scripts as an example they are described in Paragraph 2 1 1 1 the user must write an env CCM script to set up the work environment source envy CCM 4 Customize CCM start sh and CCM_stop sh Again using the example scripts see Paragraph 2 1 1 3 the user will define two scripts CCM_start sh and CCM stop sh customized with the names of the machines where the NamingService and the ComponentServers will run 11 5 Run assist2ccm sh Now we are ready to wrap an ASSIST program into a CCM component by using the assist2ccm sh script 1 Compiling using asynchronous events assist2ccm sh filename ast componentG 0 file ast rc 2 Compiling using synchronous events assist2ccm sh filename ast componenteG 1 file ast rc During its running the assist2ccm sh script will produce A sub directory named ASSIST2CCM TASKCCM demolcomponenteG
10. en in pure ASSIST it contains programs written in ASSIST extended with the CCM modules CCM_input and CCM_output it contains the configuration file for the environment variables env_CCM along with some scripts to compile the IDL3 files CCM_genera sh CCM_compila sh CCM_compila_main sh It also stores scripts to launch and stop an application CCM_start sh CCM_stop sh and a script to generate a CCM component starting from an ASSIST program assist2ccm sh it stores the code for CDR_IO h and CDRServer java it stores the file demo xml needed to compile the main code of the application an example of a main file mymain java and the relative XML file mymain xml here the user can find scripts that parse and modify the ast out xml file generated by ASSIST They are used to reconfigure the ASSIST application it contains the regression test and further examples of CCM components 2 1 1 Available Scripts Within the script sub directory the user can find several packages useful to deal with OpenCCM 1 A script to configure the OpenCCM environment env CCM 2 A script to compile the component IDL3 and Java source files CCM genera sh CCM compila sh etc 3 Scripts to launch and stop a CCM application CCM start sh CCM_stop sh 4 Scripts to create the CCM component that wrap the ASSIST application assist2ccm sh 2 1 1 1 The env CCM Script This script is used to configure the OpenCCM and OpenORB environm
11. ent variables This release features two configuration scripts as used in our tests env CCM cl gt to configure the environment on the c cluster where we installed OpenCCM version 0 8 0 and OpenORB version 1 3 1 env CCM pianosa to configure the environment on the pianosa cluster where we installed OpenCCM version 0 8 1 and OpenORB version 1 4 0 BETAI The user needs to modify one of these scripts according to the configuration of his own cluster or machine Then the script can be added to the bashrc file so it is loaded every time the user logs in The bash source command can be used to this purpose A line like the following can be added to the bashrc file source HOME TASKCCM script env_CCM Within these two scripts OpenCCM and OpenORB variables are set TCOO_HOME CCM_DIR OpenCCM CONFIG DIR OpenCCM HOMEDIR OPENORB LIB TOOLS PATH the Java CLASSPATH variable is set to include the directories where Java classes for the components can be found some symbolic links are created to the customized versions of some scripts CCM start sh CCM stop sh and demo xml 2 1 1 2 Scripts to Compile Component IDL3 and Java Files In the taskccm package users can find some useful tools to simplify the compilation process CCM_genera sh This is used to compile the IDL3 files In ASSIST2CCM Implementation more details on this script can be found The name of the test to be compiled has to be provided as an in
12. enti documents Here the user will find the documents called TASK CCM DOC 1 11 describing the step by step evolution of this project and some summary documents These documents are written in Italian they were prepared during our experiments with OpenCCM and ASSIST thus they are well detailed but they also contain preliminary results and discussion of discontinued features In the release sub directory there are three final documents part of the first release of this tool 1 ASSIST2CCM Requirements 2 ASSIST2CCM User Manual 3 ASSIST2CCM Implementation 4 Example This directory stores some example ASSIST applications to be used as a test for the tool 1 bench dp with synchronous and asynchronous events 2 prod matrice i e matrix multiply with synchronous and asynchronous events 3 mandelbrot with synchronous and asynchronous events or RMI 4 regression tests LIN 2IN 1OUT 2IN 2OUT and bench dp 1IIN 1OUT with synchronous and asynchronous events 5 TASKCCM For this directory please refer to Paragraph 2 1 2 1 The TASKCCM directory Within the taskccm sub tree the user will find The seq directory The solo assist directory The assist directory The script directory The socket directory The code directory The parserXML directory gt The demo directory gt gt gt gt with tested programs written in C sequential tests it contains tested programs writt
13. ib 1 3 Cluster pianosa and Cluster cl In the following when describing the installation scripts and commands we will refer to two different clusters pianosa and c1 Their configurations are slightly different on cluster c we have OpenCCM 0 8 0 and OpenORB 1 3 1 on cluster pianosa we have OpenCCM 0 8 1 and OpenORB 1 4 0 BETAI 2 Installing and Using ASSIST2CCM 1 0 Figure 1 describes the overall architecture of the ASSIST2CCM 1 0 tool The software suite is organized into sub directories as follows ASSIST2CCM 1 0 et adds These files are Script esempi analyzes the used by the Documentation Some example par bald analyzer to build programs l OpenCCM ASSIST program ihe scat anda files files about bench dp and generates of the CCM the project Bu d mati k IDL3 files and development and source files for Companen summaries Mandelbrot and the the wrapping component lo incapsula regression tests esl es2 es3 Figura 1 Structure of the ASSIST2CCM 41 0 tree The directories are so organized 1 Analizzatore analyzer Here the user will find the source code of our ASSIST analyzer which is described in detail in the document ASSIST2CCM Implementation To install this tool set the ASSIST2CCM environment variable to point to this directory and perform make 2 Code This directory stores some files used by the analyzer to build the souce code of the wrapping CCM component 3 Docum
14. is installed in the directory usr java The version we tested is JAVA 1 4 2_03 which is available at http java sun com First set the JAVA_HOME environment variable to the right path then add the Java compiler path to the PATH variable as follows Download the OpenCCM installation package from the OpenCCM site http openccm objectweb org You need OpenCCM 0 8 1 tar gz From the OpenORB site http openorb sourceforge net another three packages are required 1 OpenORB 1 4 0 BETA 1 tgz 2 NamingService 1 4 0 BETA 1 tgz 3 Tools 1 4 0 BETAI tgz Unpack them in a common directory such as discl and then set the TCOO HOME variable to the path where OpenORB and its tools can be found cd I disc1 tar xzvf OpenCCM O0 8 1 tar gz tar xzvf OpenORB 1 4 0 BETA 1 tgz tar xzvf NamingService 1 4 0 BETA l tgz tar xzvf Tools 1 4 0 BETA1 tgz export TCOO HOME l discl Four directories will be created OpenCCM 0 8 1 OpenORB NamingService tools 1 1 Installing OpenCCM Set the CCM HOME variable to point to the path where OpenCCM was installed export CCM_HOME I disc 1 OpenCCM 0 8 1 Then enter the CCM_HOME directory and 1 Run build sh to create a file named build properties build sh 2 Modify build properties just created uncomment the row with the name of the used ORB ORB name and specify the absolute path of the ORB ORB home dir Changes to build properties 1 uncomment the line ORB name
15. izzare which contains the ASSIST program code inout ast Using the assist2ccm sh script asynchronous gt assist2ccm sh inout ast inoutG 0 synchronous assist2ccem sh inout ast inoutG 1 we generate and compile the used events formerly ASSIST streams and the inoutG component which wraps the ASSIST program Three directories named ev moduloA ev moduloB and ev moduloC which contain the code IDL3 and Java source code of the components A directory named mymain with the main code of the application 4 4 REGRESSION 2IN 20UT The fourth test is composed by five CCM components two components ev moduloA and ev moduloC send data to a component bench dpG which wraps an ASSIST program The latter forwards them to two components ev moduloB and ev moduloD The code of this test is made up of A directory named assist da analizzare which contains the ASSIST program code inout2 ast Using the assist2ccm sh script asynchronous assist2ccm sh inout2 astinout2G 0 synchronous assist2ccem sh inout2 ast inout2G 1 we generate and compile the used events formerly ASSIST streams and the inout2G component which wraps the ASSIST program Four directories named ev moduloA ev moduloB ev moduloC and ev moduloD which contain the code IDL3 and Java source code of the components A directory named mymain with the main code of the application 14 5 Performance Here we present some experiments performed
16. ll find the ASSIST code of bench dp and prod matrice extended with the CCM modules CCM input and CCM output along with the used ast rc file 2 1 3 The CDR_IO and CDRServer classes socket In the socket directory users will find the C CDR_IO class used by ASSIST to exchange data with the Java threads that are part of the CCM component We use sockets to this purpose Also they will find the Java CDRserver class used by the Java threads of the CCM component to communicate with the CCM input and CCM output modules of the ASSIST program More details are available in the document ASSIST2CCM Implementation 10 2 1 4 The XML Parser parserXML Within the parserXML directory we can find some scripts to modify the ast out xml file They are used directly by the wrapping CCM component 2 1 5 The demo Directory and Archived Tests demo demo archivio The demo directory stores the source code for the CCM components Usually a CCM application is made up of a component which wraps ASSIST bench dpG mandelbrotG ev moduloAssist rmi moduloAssist a component managing events eventi some components which interact with the CCM ASSIST component ev moduloA ev moduloB client1 and the component with the main mymain For each component our tool creates a new directory to store IDL3 and source files In the demo directory we also find the archivio archive sub directory Here the user will find the regression tests s
17. put CCM generaRMI sh This is similar to the previous one except the fact that it does not make use of events CCM compila sh CCM_compila sh is used to compile stubs skeletons and component source files Its complete description is available in ASSIST2CCM Implementation The name of the test to be compiled has to be provided as an input CCM compila main sh CCM compila main sh is used to compile the main body of the application The name of the test to be compiled has to be provided as an input It uses the templates available in the code directory it copies the demo xmil file into the directory that contains main CCM genera eventi sh This script compiles both the IDL3 files with the event description and the Java files generated starting from the IDL3 This is automatic because no changes are needed in the Java files 2 1 1 3 Scripts to Launch and Stop the CCM Application In the taskccm package the scripts CCM_start sh_cl and CCM_start sh_pianosa are used to launch the CCM applications on our two clusters Similarly CCM stop sh c1 and CCM_stop sh_pianosa are used to stop it The difference between the two versions is simply in the hard coded machine names The user should modify them to adapt them to their configuration These scripts have two parameters the test name and the number of ComponentServers to be launched default is 2 2 1 1 4 The assist2ccm sh Script This script starting from any ASSIST program is a
18. to measure the performance of ASSIST when wrapped within a CCM component Our tests were performed on a cluster pianosa composed of 23 Intel Pentium III Mobile 800 Mhz with 1 Gbyte RAM and dedicated Fast Ethernet network cards We tested both the RMI implementation and the event implementation In the first case we used OpenCCM 0 8 1 and OpenORB 1 4 0 BETA1 in the secon case we used OpenCCM 0 8 1 and OpenORB 1 3 1 The ASSIST benchmark we used is a synthetic data parallel algorithm which works on very large matrices integer matrices 350x350 elements N 350 The task consists of running a parallel for loop N 10 times The body of the loop is a transcendental function 5 1 Performance Events Comp2 10 events Assist 10 events Figure 3 Structure of the test CCM application The test is made up of three CCM components 1 The first generates an input matrix and sends 10 requests using the event mechanism 2 The second which wraps the ASSIST program accepts the requests and send results to the third 3 The third collects the results In the following we will call the cluster nodes u3 424 The CCM and ASSIST processes are allocated as follows CCM input Assist CCM output Assist and the second component CCM are on node u4 the OSM process and one VPM Assist are on node u the first and the third component CCM are on node u23 the other VPMs ASSIST are allocated one on each of the unused nodes
Download Pdf Manuals
Related Search