The Avalanche Myrinet Simulation Package
Contents
1. at the destination processor the Receive will be invoked by the Myrinet simulation Users of this package must implement this Receive function When the upper level simulation has consumed the packet it MUST schedule the ptask or the Myrinet simulation will not deliver more packets The type signature of the Receive function is as follows Receive task_ptr ptask int src int dest int payload void msg_addr int info size void usr_ptr The parameters of Receive are the same as the parameters in Send References 1 N J Boden D Cohen R E Felderman A E Kulawik C L Seitz J N Seizovic and W K Su Myrinet A gigabit per second local area network IEEE MICRO 15 February 29 36 February 1995 2 L B Stoller R Kuramkote and M R Swanson PAINT PA instruction set interpreter Technical report University of Utah Computer Science Department March 1996 Also available via WWW under http www cs utah edu projects avalanche paint ps 3 J E Veenstra and R J Fowler Mint A front end for efficient simulation of shared memory multiproces sors In MASCOTS 1994 January 19942. 2 2 Topology File 2 3 Routing Table File 0 0 00 00 000000200200 00 0008 3 Interfaces with the Upper Level Simulation Codes 1 Introduction This is the user manual for the configurable Myrinet 1 simulation package that has been developed for the Avalanche project at University of Utah This package requires the use of the PAINT architecture simulator 2 which was evolved as part of the Avalanche effort from the University of Rochester s MINT simulator 3 To use this Myrinet simulation package you must link with the PAINT library and use PAINT to drive the simulation itself Please refer to the Avalanched project home page at http www cs utah edu projects avalanche for more details about the PAINT simulation and to acquire a copy of PAINT Version 2 0 of the Myrinet simulation package was designed to allow a high degree of configura bility of the modeled network Version 1 0 modeled only simple square mesh topologies with 4 port switches and users could specify only a limited number of switch parameters As Myricom released larger and faster versions of their Myrinet switches the V1 0 simulation model became obsolete Users of the V2 0 package can specify arbitrary network topologies composed of Myrinet switches with different number of ports For example 4 port and 32 port switches can be used in a single system Because the V2 0 model supports arbitrary topologies simple X then Y source routing is no longer sufficient to
3. The Avalanche Myrinet Simulation Package User Manual for V2 0 Chen Chi Kuo John B Carter chenchi retrac cs utah edu WWW http www cs utah edu projects avalanche UUCS 96 010 Department of Computer Science University of Utah Salt Lake City UT 84112 September 24 1996 Abstract This is a user manual for Version 2 0 of the Myrinet simulation package Users of the V2 0 pack age can specify arbitrary network topologies composed of Myrinet switches with different number of ports For example 4 port and 32 port switches can be used in a single system Because the V2 0 model supports arbitrary topologies simple X then Y source routing is no longer sufficient to model the required routing Thus users of the V2 0 package must specify the routing table themselves In addition to track improvements to the circuit technologies used in the Myrinet switches the clock rate latency and bandwidth have been parameterized Users can change the parameters in order to meet their simulation needs In the manual the example driven method is used to explain how to build your own Myrinet switch systems This work was supported by the Space and Naval Warfare Systems Command SPAWAR and Advanced Research Projects Agency ARPA Communication and Memory Architectures for Scalable Parallel Computing ARPA order B990 under SPAWAR contract N00039 95 C 0018 Contents 1 Introduction 2 Configuration Files 2 1 System Parameters File
4. ce routing tables explicitly to specify to the simulation how to compose the packet headers A complete routing table file must include N N routing directions in an N processor system one entry for each processor pair Note that the routing need not be symmetric meaning that packets from port X to port Y can take a different path than packets from port Y to port X A partial sample routing table is given below For space purposes only the routes from one processor are given Please refer to the simulation package itself for a complete example The syntax of the routing file is as follows For each processor pair there must be one line specifying the order of switch output ports that a message traveling from the source to the destina tion must take Port numbers are designated via a single character ranging from 0 9 for the first ten ports and then a z for the next 26 ports For the V2 0 product this results in a maximum switch size of 36 ports or realistically 32 ports In the example routing table below for Processor 0 PO to send a packet to Processor 8 P8 the packet will go through port 1 of SO port 1 of S1 port 4 of S4 and finally port 4 of S6 This route is directed by the entry in the routing table PO P8 with the sequence 1144 Simple Mesh Routing Table File sender receiver portnumber sequences po po 0 po pi 10 po p2 110 po p3 111 po p4 112 po p5 113 po p6 1142 po p7 1143 po ps 1144 po p9 1145 po pio 224 po pi
5. ed tfile and the routing file is called rfile Myrinet is a trademark of Myricom Inc For detailed information on Myrinet technology see the Myricom home page at http www myricom com PAINT is designed to model HP PA RISC based multiprocessors while MINT is designed to model MIPS based multiprocessors Although this package requires the use of PAINT a port to the MINT system should be feasible with a limited amount of effort If you perform this port please send it back to us for inclusion in our release for others to use and we will of course give you full credit for the port The reguired format of the three configuration files are explained in Sections 2 1 through 2 3 using the example topologies illustrated in Figures 1 and 2 Figure 1 illustrates a simple mesh topology composed of four and eight node switches while Figure 2 illustrates a chordal ring topology composed of only four node switches 2 1 System Parameters File The System Parameters File specifies the configuration of the Myrinet switches in the system An example is given below with comments to explain the meanings of each parameter Simple Mesh System Parameters File Total number of the processors in the system numOfProcessor 16 Maximum number of ports on any single switch in the system maxNumOfPorts 8 Total number of switches in the system numOfSwitch 7 Link propagation delay in cycles where 1 cycle 10 ns propDelay 4 Time to per
6. f this package must signal this event within the ptask routine e src the processor id of the sending processor e dest the processor id of the destination processor e payload the length of the user message body in bytes e msg addr the memory address of the user message body used to perform DMA transfers e info_size length of the user message header in bytes e usr_ptr an arbitrary pointer to be used by the communicating peers of the upper level simulation which can be to pass information useful for controlling the simulation e do_mem_costs a flag to indicate if this Send call needs to DMA the packet data from the memory which may cost some delay A user define function Memory_read which is explained below will be invoked when do_mem_costs is set If the do_mem_costs is set the function Memory read defined by the users will be invoked in order to simulate the delay caused by DMAing the packet data from the memory The type signature of the Memory_read function is as follows Memory_read task_ptr ptask int src void msg addr int payload The meaning of the parameters is as follows e ptask the PAINT task that MUST be scheduled in the Memory read so the Myrinet simu lation can finish shipping this packet e src the processor id of the sending processor e msg addr the memory address of the user message body used to perform DMA transfers e payload the length of the user message body in bytes Once the packet arrives
7. form taxi translation and cross bar setup for different switch sizes measured in system cycles fallThruDelay4 26 fallThruDelay8 27 fallThruDelay16 30 fallThruDelay32 35 Ratio between CPU and Myrinet switch clock rates For example if the processor speed is 100MHz and the modeled Myrinet system clocks at 50MHz the SpeedFactor is 2 SpeedFactor 2 Myrinet switch slack buffer sizes see Myrinet technical specs for discussion of the kg h and ks values in the buffer buffer_kg 32 buffer_h 16 buffer_ks 32 PO PI P2 P3 A 0 1 117 0 1 3 3 7 PIO a SO I SI a al ge E 7 A P4 a Ho tto 6 S4 ant 3 gt P14 SI gl 88 E i J re o licia ee Lo 2 Dis 5 4 0 y 0 1 n 7 P13 H Pegi E P6 2 2 s5 6 S6 P12 H gi Egr Pi 3 3 5 4 5 4 P11 P10 P9 P8 Figure 1 Example Topology Simple Mesh 1 2 2 1 3 ss 3 SA i i s P4 Figure 2 Example Topology Chordal Ring 2 2 Topology File The topology file specifies the interconnections between individual switches in the system It is used to define the overall system topology The following example file is the topology specification for the simple mesh in Figure 1 The topology file consists of one line per switch that designates where each of that switch s ports are connected either to ports on other switches or to processors Each l
8. ine should have one entry for each port i e the topology entry for a four port switch must have four entries while that for an eight port switch must have eight Simple Mesh Topology File Some definitions 50 1 means port 1 of switch number 0 PO means processor number 0 D means dangling line SO PO 51 3 52 0 P15 Meaning Port 0 of Switch 0 is connected to Processor 0 Port 1 of Switch 0 is connected to Port 3 of Switch 1 Port 2 of Switch 0 is connected to Port 0 of Switch 2 Port 3 of Switch 0 is connected to Processor 15 Si Pi 54 7 53 0 50 1 52 50 2 53 3 55 0 P14 53 51 2 54 6 S5 1 52 1 54 P2 P3 P4 PS 56 1 56 0 53 1 51 1 S5 52 2 53 2 56 7 56 6 P10 pil P12 P13 56 54 5 54 4 P6 P7 P8 P9 55 3 55 2 The following example file is the topology specification for the chordal ring Chordal Ring Topology File SO PO 51 3 53 2 57 1 51 P1 52 3 56 2 50 1 52 P2 53 3 55 2 51 1 53 P3 54 3 50 2 52 1 54 P4 55 3 57 2 53 1 55 P5 56 3 52 2 54 1 56 P6 57 3 51 2 55 1 57 PT 50 3 54 2 56 1 2 3 Routing Table File Myrinet technology uses a static source routing mechanism For simple mesh topologies composed of symmetric switches a simple X then Y routing mechanism suffices to route packets between input and output ports However because the V2 0 simulation package supports arbitrary network topologies and heterogenous switch sizes X then Y routing is no longer sufficient Users must specify the static sour
9. l 225 po p12 226 po p13 227 po p14 23 po pis 3 3 Interfaces with the Upper Level Simulation Codes Traditional PAINT architecture simulations consist of models for the CPU cache controller direc tory controller for scalable DSM models network interface system bus and other components This Myrinet simulation package provides a portion of the router interface to create source routing headers route packets between nodes model the network delays due to internal buffering con straints etc It does not however model input or output buffering within a node i e between the system bus and the network device This level of buffering must be modeled in the archi tecture simulation The network simulation package models packet delivery and flow control at a cycle by cycle level of precision To inject packets into the Myrinet fabric the architecture simulation should invoke the Send function which has the following type sienature Send task_ptr ptask int src int dest int payload void msg addr int info size void usr_ptr int do mem costs The meaning of the parameters is as follows e ptask the PAINT task that will be scheduled by the Myrinet simulation after the last flit of the current packet is injected into the interconnect At that point the architecture simulation can issue another Send to ship the next packet If other pieces of the simulation need to respond to the event of a packet transmission completing users o
10. model the required routing Thus users of the V2 0 package must specify the routing table themselves as described in Section 2 In addition to track improvements to the circuit technologies used in the Myrinet switches the clock rate latency and bandwidth have been parameterized Users can change the parameters in order to meet their simulation needs The remainder of this user manual is organized as follows In Section 2 the formats of the system configuration files are explained through a series of examples Section 3 describes the interface between the Myrinet simulation package with PAINT so that users can integrate their Myrinet network model into their PAINT architecture model 2 Configuration Files Users of this package must provide three configuration files e a system parameter file that describes the performance parameters of the switches and links in the system as well as a small number of global parameters e a network topology file that describes how the switches in the system are interconnected i e what ports are connected to what other ports and e anetwork routing file the describes how to route from every processor to every other processor The names of these parameter files can be specified in the PAINT command line using the k t and r flags For example sim n 16 s 0x1800000 k sfile t tfile r rfile barnes lt input indicates that the system parameter file is called sfile the topology file is call
Download Pdf Manuals
Related Search