RU Builder User Manual
Contents
1. If this is the case then there is no guarantee that the EVM will be able to buffer BU requests for event ids or that the RUs will be able to buffer BU requests for event data e The configuration parameters of a RU builder application must be set before it is configured DAG CERN PH CMD 2005 Page 34 of 342. 8 BU FU interface sequence diagram A FU requests a BU to allocate it one or more events step 1 In response the BU asks the FU to take the requested event data as a set of event data blocks step 2 When a FU has finished processing one or more events it tells the BU to discard them step 3 Fi The BU FU interface of this version of the RU builder does not support partial events Partial events may be supported in a future version DAQ CERN PH CMD 2005 Page 17 of 34 4 2O interface RU Builder User Manual Date 5 2 2005 The I20_BU_ALLOCATE MESSAGE FRAME C structure and its companion BU_ALLOCATE C structure are as follows typedef struct _BU_ALLOCATE U32 fuTransactionIid U32 fset BU_ALLOCATE PBU_ ALLOCATE typedef struct _I20 BU ALLOCATE MESSAGE FRAME I20_PRIVATE_MESSAGE FRAME PvtMessageFrame U32 n BU_ALLOCATE allocate 1 I20 BU_ALLOCATE MESSAGE FRAME PI20 BU ALLOCATE MESSAGE FRAME The FU must fill n allocate The n field specifies the number of events the FU is requesting The allocate field is an array of FU transaction ids and fragment sets For each event a FU requests the FU fills in the fuTransactionId field and the fset field of a BU_ALLOCATE C structure and puts itin the allocate array The fuTransactionlId field is a transaction id that a FU can use to match its requests with the events it receives A BU treats the fuTransactionlId field as being opaque in other words It is not interpret
3. FIFOsofaRU The EVM sendsa RU an event id trigger event number pair when it asks the RU to readout the corresponding event s data step 1 In parallel the RUI informs the RU of event data that is ready to be processed step 2 A RU places each super fragment for which it has received a pair into the fragment lookup table step 3 BUs ask RUsto send them the super fragments of the events they are building step 4 A RU services a BU request by retrieving the super fragment from its fragment lookup table and asking the BU to cache the super fragment step 5 Each RU has a worker thread that executes the behavior of that RU All of the internal FIFOs of a RU that is to say the pairFIFO blockFIFO and requestFIFOS are used by the peer transport thread s to store incoming messages for the worker thread to process blockFIFO EVM Readout pairFIFO E Fragment request Iros fe Cache Figure 4 RU FIFOs DAG CERN PH CMD 2005 Page 12 of 34 4 2O interface RU Builder User Manual Date 5 2 2005 4 12O interface All the 120 messages of the EVB including the internal and external messages of the RU builder are defined in the package TriDAS dagq interface The 120 function codes of all the RU builder 20 messages are given in the file TriDAS dagq interface shared include i2o0XFunctionCodes h The C structures that define the 120 messages are in the file TriDAS dag interface evb include i20EVBMsgs h The I2O interfa
4. TriDAS dagq evb examples rui 1lib linux x86 1ibRUI so TriDAS dagq evb examples ta 1l1ib linux x86 1ibTA so TriDAS dagq evb rubuildertester 1l1ib linux x86 libRUBuilderTester so Enter the following to build these libraries cd XDAQ_ ROOT dag evb make LORU builder self test This section explains how to perform the self test of the RU builder This test helps to determine whether or not the RU builder has been successfully installed The self test consists of the RU BuilderTester application plus one EVM one RU and one BU all running on the same XDAQ executive The EVM is told to generate dummy triggers the RU is told to generate dummy super fragments and the BU is told to drop the events it builds The step by step instructions to run the self test are Step 1 O btain and build the RU builder See section 9 for instructions Step 2 Open a terminal and run a XDAQ executive by typing xdaq exe h HOST p PORT Where HosT is the hostname of the computer on which you are running the executive and PORT is the TCP IP port you wish the executive to listen on Step 3 Open another terminal and create the XML configuration file for the self test by typing DAG CERN PH CMD 2005 Page 29 of 34 10 RU builder self test RU Builder User Manual Date 5 2 2005 cd XDAQ_ ROOT dag evb xml produceSelfTestXml pl HOST PORT XDAQ_ ROOT 1x1SingleXDAQ template xml gt 1x1SingleXDAQ xml Where HOST and PORT are the same as those passed to the
5. are enabled Thirdly all RU builder applications throw away their internal data and any incoming I20 message frames when they are halted Configuration parameters are read and acted The application can only participate in event upon when the application is configured building when it is in the Enabled state The application releases all incoming 120 message The contents of all internal data frames when it is in the Halted state structures are discarded when the application is halted Figure 9 FSTN ofa RU builder application DAQ CERN PH CMD 2005 Page 21 of 34 5 Application state machines RU Builder User Manual Date 5 2 2005 5 2 BU EVM and RU finite state machines The FSTNs specific to each type of RU builder application are shown in figure 10 RU Halted Configure Enable Enable Send request to Send initial EVM for initial Credits Eo TA set of event LOIS Figure 10 BU EVM and RU FSTNs Enabled DAG CERN PH CMD 2005 Page 22 of 34 6 Starting the RU builder RU Builder User Manual Date 5 2 2005 6 Starting the RU builder The RU builder is part of a larger system called the event builder EVB Besides run control the RU builder communicates with a TA one or more RUIs and one of more FUs The RU builder cannot be started at any arbitrary moment in time its start up must be synchronized with that of the TA RUIs and FUs The RU builder and the EVB components it interacts with are XDAQ applications
6. as many super fragments as there are event ids in the RU builder blockFIFOCapacity nbEvtIdsInBuilder x size of a super fragment block size 4096 x 16K 4KB 16384 The parameters of the form Packing should not normally be modified as they have only been tested with the default value of 8 However the user may modify them if they are experiencing performance problems with the RU builder DAQ CERN PH CMD 2005 Page 27 of 34 9 How to obtain and build the RU builder RU Builder User Manual Date 5 2 2005 9 How to obtain and build the RU builder This chapter is divided into two sections The first explains how to obtain the RU builder source code and the second explains how to build it Both sections assume the following e The user has already installed version 3 1 of the XDAQ core framework e The user has defined the shell environment variable XDAQ_ ROOT to point to the root of their XDAQ installation in other words their TriDAS directory e The shell of the user istsch e The user accesses the CMS CVS server as an anonymous read only user 9 1 Checking out the source code from CVS The RU builder source code is stored in the CMS CVS server For more information about this server please look at the webpage http cmsdoc cern ch cmsoo projects cvs server html You must login to the CMS CVS server if you wish to use it To do so enter the following setenv CVSROOT pserver anonymous cmscvs cern ch cvs_server repositories Tr
7. corresponding event data is given to the RUs DAG CERN PH CMD 2005 Page 8 of 34 2 RU builder overview RU Builder User Manual Date 5 2 2005 2 2 What the EVB does For each event the EVB e Reads out the trigger data This trigger data will become the first super fragment of the event e Reads out the fragments of the event from the detector front end drivers FED 5 e Builds the fragments into RU super fragments using the FED builder e Builds the whole event using the RU builder The whole event is the trigger super fragment plus the set of RU super fragments e Decides whether or not the event is interesting for physics using the filter units FU s e Sends the event to permanent storage if it is interesting for physics or discards it if it is not 2 3 The RU builder applications The RU builder consists of a single event manager EVM one or more readout units RUS and one or more builder units BU s The EVM is responsible for controlling the flow of data through the RU builder The RUs are responsible for buffering super fragments until they are requested by the BUs The BUS are responsible for building and buffering events until they are requested by the filter units FUs The trigger adapter TA readout unit inputs RU Is and filter units FU s are external to the RU builder The TA is responsible for interfacing the DAQ trigger to the EVM The RU Is are responsible for pushing super fragment data from the FED builde
8. has 64MB of physical memory for caching super fragments e Aneventis 1MB e An event is made up of 64 super fragments 1 per RU of equal size therefore the size of a super fragment is 16KB e The block size size of an 120 message frame used to transport event data is 4KB e The RUIs only give as many events to the RU s as the TA gives triggers to the EVM e The maximum number of FUs per BU is 64 e A FU will never have more than 1024 outstanding requests for events e Fast control messages are sent if they are older than 1 second e The packing factor of fast control messages is 8 The need to know the total number of event ids in the RU builder is common to all three types of RU builder applications e The total number of event ids in the RU builderis a function of RU memory Assuming each RU has 64MB of memory for buffering super fragments and that the size of an event is 1 MB BU EVM amp RU nbEvtIdsInBuilder sum of the memory of all RUs size of an event 64 x 64MB 1MB 4096 The BUs and EVM send fast control messages e Fast control messages are sent if they are older than 1 second BU amp EVM ageMessages true BU amp EVM msgAgeLimitDtMSec 1000 It has been assumed that all events have the fixed size of IMB If the RU builder is to build events of varying sizes then the appropriate safety factor needs to be taken into account when calculating the number of event ids in the RU builder DAG CERN PH CMD 2005
9. nbSuperFragmentInEvent 1 The blockNb field is numbered from to nbBlocksInSuperFragment 1 The buResourceld field is an opaque handle thata FU should use to identify events resources to be discarded The fuTransactionlId field is the FU transaction id of the FU request that caused the BU to reply with the current event DAG CERN PH CMD 2005 Page 19 of 34 4 2O interface RU Builder User Manual Date 5 2 2005 The 120 BU DISCARD C structure is as follows typedef struct _I20 BU DISCARD MESSAGE FRAME I20_PRIVATE_MESSAGE_ FRAME PvtMessageFrame U32 n U32 buResourceld 1 I20 BU DISCARD MESSAGE FRAME PI20 BU DISCARD MESSAGE FRAME The FU must fill n buResourceld The n field specifies the number of events resources to be discarded The buResourceld field is an array of the ids of the BU resources to be discarded DAG CERN PH CMD 2005 Page 20 of 34 5 Application state machines RU Builder User Manual Date 5 2 2005 5 Application state machines 5 1 Commonalities of the application finite state machines The finite state machines of the BU s EVM and RUs have commonalities Figure 9 shows the finite state transition network FSTN which all three types of application follow There are three common behaviors Firstly all RU builder applications read and act upon configuration parameters when they receive a Configure SOAP message Secondly all RU builder applications only participate in event building when they
10. see something like RUBuilderTester start est Figure 15 RUBuilderTester web page for self test Step 9 Start the self test by clicking the startTest button You should now see something similar to EVM N events BUO NIB s 317817 495 6 79 Figure 16 Web page of the self test running The web page will refresh every 5 seconds Wait at least 10 seconds to see some events being built DAG CERN PH CMD 2005 Page 33 of 34 11 Configuration guidelines RU Builder User Manual Date 5 2 2005 11 Configuration guidelines This chapter summarizes and highlights the most important points with regards to configuring the RU builder The default values of the RU builder control parameters have been chosen for the majority of use cases The user should rarely need to diverge from these values The RU builder is dependent on the instance numbers of the BUs EVM FUs RUs and TA e RUs must be assigned instance numbers from 0 to the number of RUs 1 e BUs must be assigned instance number from 0 to the number of BUs 1 e The EVM must be assigned instance number 0 e The TA must be assigned instance number 0 The RU builder has the following configuration restrictions e A single BU can service a maximum of 64 FUs e The sum of the maximum number of event ids each BU can have at any moment in time BU maxEvtsUnderConstruction must not exceed the total number of event ids in the RU builder BU EVM RU nbEvtIdsInBuilder
11. Application Display Modules Debug gt O AP Control Cluster Corntiguration Data ML file Browse Configure Cluster L Submit Authors Johannes Gutleber and Luciano Orsini Web site xdaq web cern ch Copyright 2004 CERN European Organization for Nuclear Research Figure 13 Executive configuration form for self test Step 7 Enter the location of the XML configuration file that you made in step X and click the Submit button there is no need to check the Configure Cluster check box it can be ignored for this test Please note that your web browser must be able to read the configuration file in order for it to be able to upload the file to the XDAQ executive DAG CERN PH CMD 2005 Page 32 of 34 10 RU builder self test RU Builder User Manual Date 5 2 2005 You should now see something similar to XD AQ Configure Executive Result View Applications Zo e Cluster Upload Application Display Modules Debug SCL AP Control Cluster File Uploaded via Configure page Name XMLC ontigurationFile Filetype texte Filename Cc test ls 1 Single XDA xml Size 1864 Configured local executive Application Table Application Instance Id Context Properties a amamaa E Figure 14 Executive configuration results for self test Step 8 Access the web page of the RU BuilderTester application by clicking the RU BuilderTester link You should now
12. Page 26 of 34 8 Exported configuration parameters RU Builder User Manual Date 5 2 2005 The default values of the BU control parameters were calculated as follows e To prevent a BU from monopolizing event ids each BU has a maximum number of event ids it Can acquire at any moment in time Assuming all BUs are equal each BU is allowed to acquire BU maxEvtsUnderConstruction nbEvtIdsInBuilder number of BUs 4096 64 64 e The blockFIFO of a BU see figure 2 in section 3 2 is responsible for buffering incoming event data In the worst case this FIFO would have to buffer the blocks of all outstanding requests for event data BU blockFIFOCapacity maxEvtIdsUnderConstruction x size of an event block size 64x1MB 4KB 64 x256 16384 e A BU hasa single FIFO called discardFIFO for FU discard messages and one FIFO per FU called requestFIFO for FU request messages Knowing that a BU can service a maximum of 64 FUs and assuming that a single FU will never have more than 1024 outstanding requests for events BU discardFIFOCapacity 1024 x maximum number of FUs 1024 x 64 65536 BU requestFIFOCapacity 1024 The default values of the RU exported parameters were calculated as follows e The blockFIFO of the RU is responsible for buffering incoming super fragment data Assuming a RUI only gives as many super fragments to a RU as the TA gives triggers to the EVM then in the worst case the blockFIFO must hold the blocks of
13. The Compact Muon Solenoid Experiment TriDAS Trigger and Data Acquisition RU Builder User Manual Version 3 0 5 2 2005 Version 3 0 Date 5 2 2005 Authors S Murray Cl identifier EVB D 18306 CERNY Fermilab Revision History Finalization of version 1 0 J Gutleber July 11 2003 July 31 2003 the RU builder November 1 2004 Updated for version 2 1 of the RU builder S Murray May 2 2005 Updated for version 3 0 of the RU builder S Murray May 24 2004 Updated for version 2 0 of the EVB now referred to as S Murray 0 Cl Record CI identifier Identifier EVB_D_ EVB_D_18306 A ee wie aero eee the RU builder for an integrator who will put the builder into a full DAQ system Components None E cmsnfs1 export project xdaq doc library EVB_D_18306 RUB_V3_0 doc 10 11 Table of Contents Introduction 1 1 Document purpose and scope 1 2 Intended readership 1 3 system requirements and dependencies 1 4 References 1 5 Definitions Acronyms and Abbreviations RU builder overview 2 1 How the RU builder fits within the EVB 2 2 What the EVB does 2 3 The RU builder applications RU builder application FIFOs 3 1 BU FIFOs 3 2 EVM FIFOs 3 3 RU FIFOs I20 interface 4 1 TA EVM interface 4 2 RU RUI interface 4 3 BU FU interface Application state machines 5 1 Commonalities of the application finite state machines 5 2 BU EVM and RU finite state machines Starting th
14. XDAQ executive in step 2 Step 4 Open a web browser and enter the following URL http HOST PORT Where HOST and PORT are the same as those passed to the XDAQ executive in step 2 You should now see the HyperDAQ page of the XDAQ executive you started in step 2 It should look similar to the following screenshot DAQ HyperDAQ Control Panel Executive FPeerlransponHTIP PeerlransportFifo ARelay HyperDAR Authors Johannes Gutleber and Luciano Orsini Web site xdaq web cern ch Copyright 2004 CERN European Organization tor Nuclear Research Figure 11 HyperDAQ web page for self test DAG CERN PH CMD 2005 Page 30 of 34 10 RU builder self test RU Builder User Manual Date 5 2 2005 Step 5 Go to the control panel by clicking on the Control Panel link You web browser should now display something similar to DAQ HyperDAQ View Applications Configure Cluster Upload Application Debug SOAP Control Cluster Display Modules Exit Process Authors Johannes sutleber and Luciano Orsini Web site xdaq web cern ch Copyright 2004 CERN European Organization for Nuclear Research Figure 12 HyperDAQ Control Panel for self test DAG CERN PH CMD 2005 Page 31 of 34 10 RU builder self test RU Builder User Manual Date 5 2 2005 Step 6 Click on the Configure Cluster link Your browser should now look similar to this DAQ Configure Executive View Applications Contgure Cluster Upload
15. ages DAQ CERN PH CMD 2005 Page 24 of 34 8 Exported configuration parameters User Manual S Exported configuration parameters RU Builder Date 5 2 2005 Configuration parameters need to be set before an application is sent a Configure SOAP message Table 1 lists the exported control parameters of each type of RU builder application The type and default value of each parameter is given BU EVMERU ageMessages boot e BU EVM GRU exitonFal o O boo e D Bu blockFOCapaciy unsoed ong 16387 BU discardFIFO Capaciy unsigned long 65536 ray 120 Evit ALLOCATE CLEAR Pakina T unsoed on O E BU f maxvisUnderConstruction unsonediong 67 BU 120 RU SEND Packng unsignedlong 8 D sendCreditsW ithD ispatchFrame bool _ false 120 RU READOUT Packing unsigned long 8 I20_TA_CREDIT_Packing unsigned long 8 fblockFIFO Capacity unsigned long 16384 Table 1 Exported configuration parameters The default values are set when the RU builder application is instantiated The default values have been chosen with the goal of covering the majority of use cases for the RU builder A user should rarely need to diverge from these default values DAQ CERN PH CMD 2005 Page 25 of 34 8 Exported configuration parameters RU Builder User Manual Date 5 2 2005 The following assumptions were made when calculating the default values of the RU builder s configuration parameters e A RU builderis composed of 64 BU s and 64 RUs e A RU
16. and as such depend on one or more peer transports to communicate with each other These peer transport must be up and running before these applications try to communicate with each other To start the RU builder and the components it interacts with run control should do the following in order 1 Start the necessary peer transports so that the RU builder and its surrounding applications can communicate 2 Start the TA so that is can service credits from the EVM 3 Start the RU builder so that it can receive super fragments from the RU Is 4 Start the RUIs and FUs so they can start pushing in super fragments and extracting events respectively The RU builder is a distributed application whose nodes BU s EVM and RU s need to be started up in a specific order To put the RU builder into the state where it will build events run control should do the following in order Send Configure to all of the RU builder applications Send Enable to all of the RUs Send Enable to the EVM Send Enable to the BUs a NY A Unlike version 2 x of the RU builder the state changes of version 3 0 are synchronous There is no need to poll the state of an application after a request to change state The RUs have to be enabled first because they have to be ready to receive event number event id pairs from the EVM The EVM can start sending these pairs immediately after it has been enable The EVM has to be enabled before the BUs so that it is ready
17. ce of the RU builder is subject to change The description of the interface provided by this document cannot be relied upon to be valid beyond this release Figure 5 shows the external I20 interfaces of the RU builder the TA EVM interface the RUI RU interface and the BU FU interface This chapter is divided into three sections one for each interface RUI RU interface RU builder TA EVM interface TA gt BU FU interface Figure 5 Extemal 120 interfaces of the RU builder DAG CERN PH CMD 2005 Page 13 of 34 4 2O interface RU Builder User Manual Date 5 2 2005 4 1 TA EVM interface The TA EVM interface specifies how e The EVM gives the TA trigger credits e The TA gives the EVM trigger data Figure 6 is a Sequence diagram describing the protocol between the EVM and the TA TA EVM Function code I20 TA CREDIT Step ae C structure I20_TA_CREDIT_MESSAGE_FRAME Function code I20 EVM TRIGGER C structure I20 EVENT DATA BLOCK MESSAGE FRAME n Figure 6 TA EVM interface sequence diagram The EVM communicates with the TA using a credit based mechanism The EVM tells the TA the current Capacity of the RU builder by sending the TA a trigger credit count step 1 One trigger credit represents the RU builder s ability to build one event The TA should only send the EVM trigger data for as many events as the EVM has given the TA credits step 2 The TA is responsible for getting receiving trigger data from t
18. e 5 2 2005 1 4 References 1 The CMS collaboration The Trigger and Data Acquisition project Volume Il Data Acquisition amp High Level Trigger CERN LHCC 2002 26 ISBN 92 9083 111 4 1 5 Definitions Acronyms and Abbreviations BU Builder Unit 120 Intelligent Input O utput CVS Concurrent Versioning System RU Readout U nit DAQ Data Acquisition system RUI Readout U nit Input EVB Event builder TA Trigger Adapter EVM Event M anager TDR Technical Design Report FED Front End Driver TriDAS Trigger and Data Acquisition FSTN Finite State Transition N etwork XDAQ Cross platform data acquisition toolkit FU Filter U nit DAQ CERN PH CMD 2005 Page 7 of 34 2 RU builder overview RU Builder User Manual Date 5 2 2005 2 RU builder overview 2 1 How the RU builder fits within the EVB The RU builder is a component of a larger system called the event builder EVB The EVB is a distributed application that reads out event fragments from one set of nodes and assembles them into entire events in another set of nodes Figure 1 shows the applications of the RU builder and how they interact with the rest of the EVB e FED builder Super fragments Trigger credits Trigger data Allocates Discards Events FU 0 FU q Permanent storage Figure 1 RU builder applications and how they interact with the rest of the EVB The external interfaces of the RU builder assume that triggers are given to the EVM in the same order as their
19. e RU builder Stopping the RU builder Exported configuration parameters How to obtain and build the RU builder 9 1 Checking out the source code from CVS 9 2 Building the application libraries RU builder self test Configuration guidelines Oo N NOOO O 24 25 28 28 29 29 34 List of Figures Figure RU builder applications and how they interact with the rest of the EVB 00 8 UNS BU PIERO ita lances ea cheese ape ee nineteen nna ET 10 Fiere SEVM FIFS eroian dhcteustcbacenten E E T O O E 11 Ficure4 RUFIFO crecian ra a e TCA E E inte OT E 12 Figure 5 External 20 interfaces of the RU builder cccennnteeteeeeeeseeeeeeeeeeeeeees 13 Figure 6 TA EVM interface sequence diagram sssssssssescccccccccccccceeeeeessssssssssseecsesseees 14 Fip re 7 RU RUT mterf ace Sequence Cia er ain ccie E E E 16 Figure 8 BU FU interface sequence diagram cccccsssessseccccccceeeeeseeeeceeeeeeaaeeenseeceeeeseeeaeseess 17 Figure 9 FSTN of a RU builder application cece cccesesseeeeeeecececceeeeeaaaaeaesssseeseeeeeeeeeeeees 21 Pioure 10 BU EVM amd RUFSTN errori on E E T ORE 22 Figure LL Hyper DAO web pace Tor Sell eSt asnes E E 30 Figure 12 HyperDAQ Control Panel for Self test ee eeccccccccccccccccccceeeessssssssssseesecesseees 31 Figure 13 Executive configuration form for self test cccccecccccccccccccceeeeeeeessseeeeeeeeeeeeeeeeees 32 Figure 14 Executive configuration
20. ed A BU will send back a copy of the fuTransactionId field in each of the I20_EVENT_ DATA BLOCK _MESSAGE_FRAMES that make up the requested event The fset field isa fragment set identifier Fragment sets are a way to describe partial events The fset field is ignored by the BU in this version of the RU builder because this version does not support partial events DAG CERN PH CMD 2005 Page 18 of 34 4 2O interface RU Builder User Manual Date 5 2 2005 The 120 EVENT DATA BLOCK MESSAGE FRAME C structure is as follows typedef struct I20 EVENT_DATA_BLOCK_MESSAGE FRAME I20_PRIVATE_MESSAGE FRAME PvtMessageFrame U32 eventNumber U32 nbBlocksInSuperFragment U32 blockNb U32 eventIid U32 buResourceld U32 fuTransactionIid U32 nbSuperFragmentsInEvent U32 superFragmentNb U32 padding I20_EVENT DATA BLOCK MESSAGE FRAME PI20 EVENT DATA BLOCK MESSAGE FRAME The FU should only read nbSuperFragmentsInEvent superFragmentNb nbBlocksInSuperFragment blockNb buResourcelId fuTransactionIid An event is composed of 120 _ EVENT_DATA BLOCK _FRAMES The nbSuperFragmentsInEvent superFragmentNb nbBlocksInSuperFragment blockNb fields are used to identify the position of an event data block within an event An event is composed of one trigger super fragment plus N RU super fragments where N is the number of RUs Therefore the nbSuperFragmentsInEvent field is set to the number of RUs plus 1 The superFragmentNb field is numbered from 0 to
21. face sequence diagram A super fragment is composed of one or more I20 EVENT DATA BLOCK MESSAGE FRAMES The I20 EVENT DATA BLOCK _MESSAGE_FRAME C structure is as follows typedef struct I20 EVENT_DATA_BLOCK_MESSAGE_ FRAME I20_ PRIVATE_MESSAGE_ FRAME PvtMessageFrame U32 eventNumber U32 nbBlocksInSuperFragment U32 blockNb U32 eventIid U32 buResourceld U32 fuTransactionIid U32 nbSuperFragmentsInEvent U32 superFragmentNb U32 padding I20 EVENT DATA BLOCK MESSAGE FRAME PI20 EVENT DATA BLOCK MESSAGE FRAME The RUI must fill eventNumber nbBlocksInSuperFragment blockNb The nbBlocksInSuperFragment field gives the number of blocks the super fragment is composed of The blockNb field indicates the block s position within the super fragment Blocks are numbered from 0 to nbBlocksInSuperFragment 1 DAG CERN PH CMD 2005 Page 16 of 34 4 2O interface RU Builder User Manual Date 5 2 2005 4 3 BU FU interface The BU FU interface specifies how e A FU requests events from a BU e ABU sends an event to a FU e A FU tells a BU to discard an event Figure 8 is a sequence diagram describing the protocol between a BU and a FU FU BU Step 1 Function code I20 BU ALLOCATE C structure I20 BU ALLOCATE MESSAGE FRAME Function code I20 FU_TAKE C structure I20 EVENT DATA BLOCK MESSAGE FRAME Step 3 sega nn Ca I20 BU DISCARD C structure I20 BU DISCARD MESSAGE FRAME Figure
22. he trigger and for providing backpressure to the trigger as necessary The 120 TA CREDIT MESSAGE FRAME C structure is as follows typedef struct _I20 TA CREDIT MESSAGE FRAME I20_PRIVATE_ MESSAGE FRAME PvtMessageFrame U32 nbCredits I20_TA_CREDIT_MESSAGE_ FRAME PI20 TA CREDIT MESSAGE FRAME The EVM must fill nbcredits DAG CERN PH CMD 2005 Page 14 of 34 4 2O interface RU Builder User Manual Date 5 2 2005 The 120 EVENT DATA BLOCK MESSAGE FRAME C structure is as follows typedef struct I20 EVENT_DATA_BLOCK_MESSAGE FRAME I20_PRIVATE_MESSAGE FRAME PvtMessageFrame U32 eventNumber U32 nbBlocksInSuperFragment U32 blockNb U32 eventIid U32 buResourceld U32 fuTransactionIid U32 nbSuperFragmentsInEvent U32 superFragmentNb U32 padding I20_EVENT DATA BLOCK MESSAGE FRAME PI20 EVENT DATA BLOCK MESSAGE FRAME The TA must fill eventNumber nbBlocksInSuperFragment blockNb Version 3 0 of the RU builder only supports single block trigger data Therefore the TA must set nbBlocksInSuperFragment to and blockNb to 0 DAG CERN PH CMD 2005 Page 15 of 34 4 2O interface RU Builder User Manual Date 5 2 2005 4 2 RU RUI interface The RU RUI interface specifies how aRU I passes super fragments to a RU Figure 7 is a sequence diagram describing the protocol between the RUI and the RU RUI RU Function code I20 RU DATA READY C structure I20 EVENT DATA BLOCK MESSAGE FRAME og Figure 7 RU RUI inter
23. iDAS cvs login You will be prompted for a password CVS password Please enter 98passwd The RU builder source code needs to be placed in the directory XDAQ ROOT dag evb Enter the following commands to put it there using the CMS CVS server cd XDAQ ROOT cvs export r EVB_S_18305_ V3_0 TriDAS daqgq evb The RU builder source code depends on two 120 interface files in the directory SXDAQ ROOT dagq interface Enter the following commands to obtain these files from the CMS CVS server cd XDAQ ROOT cvs export r EVB_S 18305 V3_0 TriDAS dag interface evb include i20EVBMsgs h cvs export r EVB_S_ 18305 _V3_0 TriDAS dag interface shared include i2o0XFunctionCodes h DAG CERN PH CMD 2005 Page 28 of 34 10 RU builder self test RU Builder User Manual Date 5 2 2005 9 2 Building the application libraries The RU builder is composed of seven XDAQ application libraries There is one library for each of the RU builder applications libBU so libEVM so and libRU so Plus there is one library for each example application libFU so libRUI so and libTA so Finally there is one library for the RU BuilderTester application As its name suggests this application performs the self test feature of the RU builder The locations of the libraries are as follows TriDAS daq evb bu 1ib linux x86 1ibBU so TriDAS dagq evb evm 1ib linux x86 1ibEVM so TriDAS dagq evb ru 1ib linux x86 1ibRU so TriDAS dagq evb examples fu 1ib linux x86 1ibFU so
24. r into the RUs The FUs are responsible for selecting interesting events for permanent storage DAQ CERN PH CMD 2005 Page 9 of 34 3 RU builder application FIFOs RU Builder User Manual Date 5 2 2005 3 RU builder application FIFOs The RU builder applications use FIFOs to keep track of requests trigger data and event data Knowledge of these FIFOs is required in order to correctly configure the RU builder This chapter is divided into three sections one for the BU one for the EVM and one for the RU Each section gives a brief description of the application s behavior and how its FIFOs are used 3 1 BU FIFOs A BU is responsible for building events An event is composed of one trigger super fragment and N RU super fragments where N is the number of RUs To understand the internal FIFOs of a BU itis first necessary to know its dynamic behavior Figure 2 shows the internal FIFOs of aBU With free capacity available aBU requests the EVM to allocate it an event step 1 The EVM confirms the allocation by sending the BU the event id and trigger data of an event step 2 This trigger data is the first super fragment of the event The BU now requests the RUs to send it the rest of the event s super fragments step 3 The BU builds the super fragments it receives from the RUs step 4 into a whole event within its resource table step 5 FUs can ask a BU to allocate them events step 6 A BU services a FU request by asking the FU to take a
25. results for self test cccecccccccccceceeeeeeeeessseeeseeeeeeeeeeeeeees 33 Figure 15 RUBuilderTester web page for Self test onrcnericiiiinoies i a T 33 Feur 16 Web page Of he SeH Test TIMMINS desiree E teats 33 List of Tables Table 1 Exported configuration parameter ccccccccccsssssssseccceeeeceeaseseeeceeeeeeeeaaaeseeeeeeeeeeaaas 25 1 Introduction RU Builder User Manual Date 5 2 2005 I Introduction The RU builder is a distributed XDAQ application that is part of a larger system called the event builder EVB The CMS data acquisition group is presently developing the EVB as described in the TriiDAS TDR 1 This document explains how to obtain build and configure version 3 0 of the RU builder Version 3 0 of the RU builder provides the following e Runs with XDAQ 3 1 e Synchronous state changes e Simpler 120 interface e Example applications FU RUI an TA e Self test feature implemented with the RU BuilderTester application 1 1 Document purpose and scope The goal of this document is to enable the reader to integrate the RU builder into a running system composed of the RU builder itself a trigger source one or more event data sources one or more data sinks and some form of run control The RU builder cannot run without the components just listed This document describes how to obtain build and configure the RU builder This document does not describe the other components of a running system
26. such as run control software or how to setup a trigger or event data source It is also not the purpose of this document to describe the internal workings of the RU builder Developers are referred to the source code for such information The code has been structured and commented so that it can be easily read and understood It is recommended to use Doxygen to generate documentation from the code as compatible comment tags have been used If the reader is not familiar with Doxygen then they are referred to its website htto www doxygen org AY It must be emphasized that the RU builder is still under development and subject to change No description of the RU builder given in this document can be relied upon to be valid beyond this release 1 2 Intended readership This document is intended for a system integrator someone that needs to integrate the RU builder into a data acquisition system DAQ Itis assumed that the DAQ system is based on the XDAQ framework If the reader is not familiar with this framework then they are referred to the XDAQ website http xdaq web cern ch xdaq 1 3 System requirements and dependencies Version 3 0 of the RU builder only supports the Linux operating system running on an Intel x86 processor The code was tested using version 3 2 3 of gcc This version of the RU builder is dependent on version 3 1 of the XDAQ core framework DAQ CERN PH CMD 2005 Page 6 of 34 1 Introduction RU Builder User Manual Dat
27. to service their requests for eventids BUs can start requesting event ids as soon as they are enabled Enabling the EVM causes it to send an initial trigger credit count to the TA The number of initial credits is equal to the total number of event ids in the RU builder As soon as trigger data arrives at the EVM from the TA the EVM sends event number event id pairs to the RUs As explained in the previous paragraph this is why the RUs have to be enabled before the EVM Enabling a BU causes it to send its Initial request for event ids to the EVM The number of initial event ids requested is equal to the maximum number of event ids the BU is allowed to acquire at any single moment in time DAG CERN PH CMD 2005 Page 23 of 34 7 Stopping the RU builder RU Builder User Manual Date 5 2 2005 7 Stopping the RU builder The current version of RU builder foresees two ways of stopping the RU builder e Stop the trigger and event data entering the RU builder e Haltall of the RU builder application When stopping the trigger and event data entering the RU builder it is useful to know when the RU builder has finished building the events for which it has received triggers and event data This can be found out by reading the following exported parameter of the EVM EVM ruBuilderIsEmpty Halting a RU builder application causes it to discard destroy all the data in its internal data structures and to release all incoming I20 mess
28. whole event step 7 When a FU has finished with an event it tells the BU to discard it step 8 Each BU has its own worker thread that executes the behavior of that BU The eventldFIFO blockFIFO requestFIFOs and discardFIFO are used by the peer transport thread s to store incoming messages ready for the worker thread to process them The fullResourceFIFO IS manipulated solely by the worker thread It is used to store which events were built in which order This enables a BU to service a FU request with the next event that was built lonk re le 1 Confirm TILL Resource Allocate new and or clear previous l fullResourceFIFO discardFIFO Ja CELLA MAOS FU Allocate discard FU take Figure 2BU FIFOs DAG CERN PH CMD 2005 Page 10 of 34 3 RU builder application FIFOs RU Builder User Manual Date 5 2 2005 3 2 EVM FIFOs The EVM is responsible for controlling the flow of event data through the RU builder To understand the internal FIFOs of the EVM it Is first necessary to know its dynamic behavior Figure 3 shows the internal FIFOs of the EVM The EVM tells the TA the capacity of the RU builder by sending it trigger credits step 1 One trigger credit represents the ability to build one event Given a credit the TA sends the EVM the trigger data of an event step 2 The EVM pairs the trigger data with a free event id step 3 The EVM also requests the RUs to readout the event s data step 4 A BU with the abilit
29. y to build an event will ask the EVM to allocate it an event step 5 Within such a request a BU will normally give back the id of an event to be cleared For each cleared event id the EVM sends a trigger credit to the TA and makes the id a free event id step 6 The EVM confirms the allocation of an event by sending the requesting BU the event id and trigger data of the allocated event step 7 The EVM has a worker thread that executes the behavior of the EVM The triggerFIFO clearedEventIdFIFO and requestFIFO are used by the peer transport thread s to store incoming messages ready for the worker thread to process them The pairFIFO and freeEventIdFIFO are manipulated solely by the worker thread The pairFIFO keeps track of the event id trigger data pairs that have yet to be sent to requesting BUs The freeEventIdFIFO stores the ids of free events for which trigger credits have been sent to the TA RU Readout EVM pairFIFO creLggerct Iro a HHH Trigger TA Credit clearedEventIdFIFO freekventIdFIFO Allocate and or clear request Iro L BU Confirm Step 7 Figure 3 EVM FIFOs DAG CERN PH CMD 2005 Page 11 of 34 3 RU builder application FIFOs RU Builder User Manual Date 5 2 2005 3 3 RU FIFOs A RU is responsible for buffering super fragments until they are request by the BUs To understand the internal FIFOs of a RU itis first necessary to know its dynamic behavior Figure 4 shows the Internal
Download Pdf Manuals
Related Search