RU Builder User Manual
Contents
1. The Compact Muon Solenoid Experiment TriDAS Trigger and Data Acquisition Version Date Authors Cl identifier RU Builder User Manual Version 3 2 7 4 2005 3 2 7 4 2005 S Murray EVB_D_ 18306 Fermilab Revision History Version Description July 31 2003 3 inaizaton ofverson 1 Finalization of version 1 0 o of version 1 0 J Gutleber Gutleber May 24 2004 2 0 Updated for version 2 0 of the EVB now referred to as ema Murray the RU builder November 1 2004 Updated for version 2 1 of the RU builder May 2 2005 Updated for version 3 0 of the RU builder July 4 2005 Updated for version 3 2 of the RU builder CI Record Description Cl Identifier EVB_D_18306 Description Describes the RU builder for an integrator who will put the builder into a full DAQ system Dependencies Related Version 3 1 of the XDAQ core framework Point of Contact S Murray Steven Murray cerm ch Comments None Physical Location http smurray home cern ch smurray RUB_V3_2 doc 10 11 Table of Contents Introduction 1 1 Document purpose and scope 1 2 Intended readership 1 3 System requirements and dependencies 1 4 References 1 5 Definitions Acronyms and Abbreviations RU builder overview 2 1 How the RU builder fits within the EVB 2 2 What the EVB does 2 3 The RU builder applications RU builder application FIFOs 3 1 BU FIFOs 3 2 EVM FIFOs 3 3 RU FIFOs 120 interface 4 1 TA EVM interfa2. I20_PRIVATE_MESSAGE FRAME PvtMessageFrame U32 n U32 buResourceld 1 I20_BU_DISCARD_ MESSAGE _ FRAME PI20 BU _DISCARD_MESSAGE_ FRAME The FU must fill n buResourceld The n field specifies the number of events resources to be discarded The buResourcefld field is an array of the ids of the BU resources to be discarded DAQ CERN PH CMD 2005 Page 21 of 33 5 Application state machines RU Builder User Manual Date 7 4 2005 5 Application state machines 5 1 Commonalities of the application finite state machines The finite state machines of the BU s EVM and RUs have commonalities Figure 9 shows the finite state transition network FSTN which all three types of application follow There are three common behaviors Firstly al RU builder applications read and act upon configuration parameters when they receive a Configure SOAP message Secondly all RU builder applications only participate in event building when they are enabled Thirdly all RU builder applications throw away their internal data and any incoming 120 message frames when they are halted Configuration parameters are read and acted The application can only participate in event upon when the application is configured building when itis in the Enabled state The application releases all incoming 120 message The contents of all internal data frames when it is in the Halted state structures are discarded when the application is halted Figure 9 FSTN ofa RU bu
3. 2005 The I20_BU_ALLOCATE_MESSAGE_FRAME C structure and its companion BU_ALLOCATE C structure are as follows typedef struct _BU_ ALLOCATE U32 fuTransactionId U32 fset BU_ALLOCATE PBU_ALLOCATE typedef struct _I20 BU_ALLOCATE_MESSAGE FRAME I20_PRIVATE_MESSAGE FRAME PvtMessageFrame U32 n BU_ALLOCATE allocate l1 I20_BU_ALLOCATE_MESSAGE FRAME PI20_BU_ALLOCATE MESSAGE FRAME The FU must fill n allocate The n field specifies the number of events the FU is requesting The allocate field is an array of FU transaction ids and fragment sets For each event a FU requests the FU fills in the fuTransactionId field and the fset field of a BU_ALLOCATE C structure and puts itin the allocate array The fuTransactionlId field is a transaction id that a FU can use to match its requests with the events it receives A BU treats the fuTransactionId field as being opaque in other words it is not interpreted A BU will send back a copy of the fuTransactionId field in each of the I20_EVENT_DATA_BLOCK_MESSAGE_FRAMES that make up the requested event The fset field isa fragment set identifier Fragment sets are a way to describe partial events The fset field is ignored by the BU in this version of the RU builder because this version does not support partial events DAQ CERN PH CMD 2005 Page 19 of 33 4 120 interface RU Builder User Manual Date 7 4 2005 The I20_ EVENT DATA BLOCK MESSAGE FRAME C structure is as follows typ
4. User Manual Date 7 4 2005 11 Configuration guidelines This chapter summarizes and highlights the most important points with regards to configuring the RU builder The default values of the RU builder control parameters have been chosen for the majority of use cases The user should rarely need to diverge from these values The RU builder is dependent on the instance numbers of the BUs EVM FUs RUs and TA e RUs must be assigned instance numbers from 0 to the number of RUs 1 e BUs must be assigned instance number from 0 to the number of BUs 1 e The EVM must be assigned instance number 0 e The TA must be assigned instance number 0 The RU builder has the following configuration restrictions e A single BU can service a maximum of 64 FUs e The sum of the maximum number of event ids each BU can have at any moment in time BU maxEvtsUnderConstruction must not exceed the total number of event ids in the RU builder BU EVM RU nbEvtIdsInBuilder If this is the case then there is no guarantee that the EVM will be able to buffer BU requests for event ids or that the RUs will be able to buffer BU requests for event data e The configuration parameters of a RU builder application must be set before it is configured DAQ CERN PH CMD 2005 Page 33 of 33
5. not the event is interesting for physics using the filter units FU s e Sends the event to permanent storage if it is interesting for physics or discards it if it is not 2 3 The RU builder applications The RU builder consists of a single event manager EVM one or more readout units RUS and one or more builder units BUs The EVM is responsible for controlling the flow of data through the RU builder The RUs are responsible for buffering super fragments until they are requested by the BUs The BUs are responsible for building and buffering events until they are requested by the filter units FU s The trigger adapter TA readout unit inputs RU Is and filter units FU s are external to the RU builder The TA is responsible for interfacing the DAQ trigger to the EVM The RUIs are responsible for pushing super fragment data from the FED builder into the RUs The FUs are responsible for selecting interesting events for permanent storage DAQ CERN PH CMD 2005 Page 10 of 33 3 RU builder application FIFOs RU Builder User Manual Date 7 4 2005 3 RU builder application FIFOs The RU builder applications use FIFOs to keep track of requests trigger data and event data Knowledge of these FIFOs is required in order to correctly configure the RU builder This chapter is divided into three sections one for the BU one for the EVM and one for the RU Each section gives a brief description of the application s behavior and how its FIFOs ar
6. of the XDAQ executive you started in step 3 It should look similar to the following screenshot XDAQ HyperDAQ http localhost 65432 um xdaq application lid 3 82S Eg Control Panel Executive PeerTransportHTTP PeerTransportFifa HyperDAQ XRelay um xdaq application lid 0 urn xdaq application lid 1 urn xdaq application lid 8 urn xdaq application lid 3 urn xdaq application lid 4 Oo o wa E Li XMem RUBuilderTester EVM RU BU um xdaq application lid 5 urn xdaq application lid 1 urm xdaq application lid 12 urm xdaq application lid 1 lt urn xdaq application lid 15 Figure 11 HyperDAQ web page for self test DAQ CERN PH CMD 2005 Page 31 of 33 10 RU builder self test RU Builder User Manual Date 7 4 2005 Step 5 Go to the default web page of the RU BuilderTester application by clicking on the RU BuilderTester link Your web browser should now display something similar to Taat E start stop E eventNb 0 state Halted deltaT 1 009878e 00 deltaN 0 deltaSumOfSquares 0 000000e 00 deltaSumOfSizes 0 throughput 0 MB s average DIV BY 0 rate 0 KHz rms DIV BY 0 RUO state Halted Figure 12 RUBuilderTester web page Step 6 Start the self test by clicking the start button Clicking the refresh button of your browser should show events being built For example the eventN b variable of the EVM 0 should increment DAQ CERN PH CMD 2005 Page 32 of 33 11 Configuration guidelines RU Builder
7. to be placed in the directory XDAQ_ROOT daq evb Enter the following commands to put it there using the CMS CVS server cd XDAQ_ ROOT cvs export r EVB_S_18305_V3_2 TriDAS daq evb The RU builder source code depends on two 120 interface files in the directory XDAQ_ROOT dag interface Enter the following commands to obtain these files from the CMS CVS server cd XDAQ ROOT cvs export r EVB_S 18305 _V3_2 TriDAS daq interface evb include i20EVBMsgs h cvs export r EVB_S_18305_V3_2 TriDAS daq interface shared include i2oxXFunctionCodes h DAQ CERN PH CMD 2005 Page 29 of 33 10 RU builder self test RU Builder User Manual Date 7 4 2005 9 2 Building the application libraries The RU builder is composed of seven XDAQ application libraries There is one library for each of the RU builder applications libBU so libEVM so and libRU so Plus there is one library for each example application libFU so libRUI so and libTA so Finally there is one library for the RU BuilderTester application As its name suggests this application performs the self test feature of the RU builder The locations of the libraries are as follows TriDAS daq evb bu 1lib linux x86 1ibBU so TriDAS daq evb evm 1ib linux x86 1ibEVM so TriDAS daq evb ru lib linux x86 1ibRU so TriDAS daq evb examples fu lib linux x86 1libFU so TriDAS daq evb examples rui lib linux x86 1ibRUI so TriDAS daq evb examples ta lib linux x86 1ibTA so TriDAS daq evb rubuildertester li
8. E The RUI must fill eventNumber nbBlocksInSuperFragment blockNb The nbBlocksInSuperFragment field gives the number of blocks the super fragment is composed of The blockNb field indicates the block s position within the super fragment Blocks are numbered from 0 to nbBlocksInSuperFragment 1 DAQ CERN PH CMD 2005 Page 17 of 33 4 120 interface RU Builder User Manual Date 7 4 2005 4 3 BU FU interface The BU FU interface specifies how e A FU requests events from a BU e ABU sends an event to a FU e A FU tellsa BU to discard an event Figure 8 is a sequence diagram describing the protocol between a BU and a FU FU BU Step 1 io Function code I20 BU_ALLOCATE C structure 120 _BU_ALLOCATE MESSAGE FRAME EE eee ooo Function code I20 FU_TAKE C structure I20 EVENT DATA BLOCK MESSAGE FRAME Step 3 Function code I20_BU_DISCARD C structure I20 BU DISCARD MESSAGE FRAME Figure 8 BU FU interface sequence diagram A FU requests a BU to allocate it one or more events step 1 In response the BU asks the FU to take the requested event data as a set of event data blocks step 2 When a FU has finished processing one or more events it tells the BU to discard them step 3 The BU FU interface of this version of the RU builder does not support partial events Partial events may be supported in a future version DAQ CERN PH CMD 2005 Page 18 of 33 4 120 interface RU Builder User Manual Date 7 4
9. GE_FRAME I20_PRIVATE_MESSAGE FRAME PvtMessageFrame U32 eventNumber U32 nbBlocksInSuperFragment U32 blockNb U32 eventId U32 buResourceld U32 fuTransactionId U32 nbSuperFragmentsInEvent U32 superFragmentNb U32 padding I20_EVENT_DATA BLOCK_MESSAGE FRAME PI20_ EVENT_DATA BLOCK_MESSAGE FRAME The TA must fill eventNumber nbBlocksInSuperFragment blockNb Version 3 0 of the RU builder only supports single block trigger data Therefore the TA must set nbBlocksInSuperFragment to and blockNb to 0 DAQ CERN PH CMD 2005 Page 16 of 33 4 120 interface RU Builder User Manual Date 7 4 2005 4 2 RU RUI interface The RU RUI interface specifies how a RUI passes super fragments to a RU Figure 7 is a sequence diagram describing the protocol between the RUI and the RU RUI RU Function code I20 RU _DATA READY C structure I20 EVENT DATA BLOCK MESSAGE FRAME i Figure 7 RU RUI interface sequence diagram A super fragment is composed d one or more I20_EVENT_DATA_BLOCK_MESSAGE_ FRAMES The I20_EVENT_DATA_BLOCK_MESSAGE_FRAME C structure is as follows typedef struct I20 EVENT _DATA_BLOCK_MESSAGE_FRAME I20_PRIVATE_MESSAGE_ FRAME PvtMessageFrame U32 eventNumber U32 nbBlocksInSuperFragment U32 blockNb U32 eventId U32 buResourceld U32 fuTransactionId U32 nbSuperFragmentsInEvent U32 superFragmentNb U32 padding I20_EVENT_DATA BLOCK_MESSAGE FRAME PI20 EVENT_DATA BLOCK MESSAGE FRAM
10. O I Confirm LAIT Resource Allocate new and or clear previous i fullResourceFIFO discardFIFO i TLL Step 6 FU ya Allocate Step 8 discard FU take Figure 2 BU FIFOs DAQ CERN PH CMD 2005 Page 11 of 33 3 RU builder application FIFOs RU Builder User Manual Date 7 4 2005 3 2 EVM FIFOs The EVM is responsible for controlling the flow of event data through the RU builder To understand the internal FIFOs of the EVM itis first necessary to know its dynamic behavior Figure 3 shows the internal FIFOs of the EVM The EVM tells the TA the capacity of the RU builder by sending it trigger credits step 1 One trigger credit represents the ability to build one event Given a credit the TA sends the EVM the trigger data of an event step 2 The EVM pairs the trigger data with a free event id step 3 The EVM also requests the RU s to readout the event s data step 4 A BU with the ability to build an event will ask the EVM to allocate it an event step 5 Within such a request a BU will normally give back the id of an event to be cleared For each cleared event id the EVM sends a trigger credit to the TA and makes the id a free event id step 6 The EVM confirms the allocation of an event by sending the requesting BU the event id and trigger data of the allocated event step 7 The EVM has a worker thread that executes the behavior of the EVM The triggerFIFo clearedEventIdFIFO and requestFIFO are used by the peer transport threa
11. RU of equal size therefore the size of a super fragment is 16KB e The block size size of anlI20 message frame used to transport event data is 4KB e The RUIs only give as many events to the RU s as the TA gives triggers to the EVM e The maximum number of FUs per BU is 64 e A FU will never have more than 1024 outstanding requests for events e Fast control messages are sent if they are older than 1 second e The packing factor of fast control messages is 8 The need to know the total number of event ids in the RU builder is common to all three types of RU builder applications e The total number of event idsin the RU builderis a function of RU memory Assuming each RU has 64MB of memory for buffering super fragments and that the size of an event is 1 MB BU EVM amp RU nbEvtIdsInBuilder sum of the memory of all RUs size of an event 64 x 64MB 1MB 4096 The BUs and EVM send fast control messages e Fast control messages are sent if they are older than 1 second BU amp EVM ageMessages true BU amp EVM msgAgeLimitDtMSec 1000 It has been assumed that all events have the fixed size of IMB If the RU builder is to build events of varying sizes then the appropriate safety factor needs to be taken into account when calculating the number of event ids in the RU builder DAQ CERN PH CMD 2005 Page 27 of 33 8 Exported configuration parameters RU Builder User Manual Date 7 4 2005 The default values of the BU contr
12. RU Builder User Manual Date 7 4 2005 4 1 TA EVM interface The TA EVM interface specifies how e The EVM gives the TA trigger credits e The TA gives the EVM trigger data Figure 6 is a sequence diagram describing the protocol between the EVM and the TA TA EVM Function code I20 TA CREDIT Step 1 C structure I20 TA CREDIT MESSAGE FRAME aeee Step B Function code I2O_ EVM_TRIGGER C structure I20_EVENT_DATA BLOCK MESSAGE FRAME gt Figure 6 TA EVM interface sequence diagram The EVM communicates with the TA using a credit based mechanism The EVM tells the TA the current capacity of the RU builder by sending the TA a trigger credit count step 1 One trigger credit represents the RU builder s ability to build one event The TA should only send the EVM trigger data for as many events as the EVM has given the TA credits step 2 The TA is responsible for getting receiving trigger data from the trigger and for providing backpressure to the trigger as necessary The I20_TA CREDIT MESSAGE_FRAME C structure is as follows typedef struct _I20 TA CREDIT MESSAGE FRAME I20_PRIVATE_MESSAGE_FRAME PvtMessageFrame U32 nbCredits I20_TA_CREDIT_ MESSAGE FRAME PI20_ TA CREDIT _MESSAGE_FRAME The EVM must fill nbcredits DAQ CERN PH CMD 2005 Page 15 of 33 4 120 interface RU Builder User Manual Date 7 4 2005 The I20_ EVENT DATA BLOCK MESSAGE FRAME C structure is as follows typedef struct I20 EVENT _DATA_BLOCK_MESSA
13. anak 31 Figure 12 RUBuilderTester web page aiis sseiasssiiessdccsasdavensccesnecaedaanepaatasevsaeeeanaecevesebentenenbaceese 32 List of Tables Table 1 Exported configuration parameters eeesesseseeseessrsrestessrserssresseserssresseseresresseseeesees 26 1 Introduction RU Builder User Manual Date 7 4 2005 1 Introduction The RU builder is a distributed XDAQ application that is part of a larger system called the event builder EVB The CMS data acquisition group is presently developing the EVB as described in the TriDAS TDR 1 This document explains how to obtain build and configure version 3 2 of the RU builder Features introduced with version 3 2 of the RU builder e Example TA and FU use FRL and FED headers trailers within trigger messages e Loggers are named using the class names and instance numbers of their respective applications e Exported parameters are divided into 4 groups o Standard configuration o Debug configuration o Standard monitoring o Debug monitoring e BU EVM and RU applications calculate statistics about data throughput based around Le 2 rms X n Each application exports the following parameters with respect to the above equation o deltaT o deltaN o deltaSumO fSquares o deltaSumO fSizes e BU EVM and RU applications publish monitoring information into info spaces specifically created for use within the future XDAQ monitoring framework e Each application now ha
14. b linux x86 libRUBuilderTester so Enter the following to build these libraries cd XDAQ_ ROOT daq evb make TORU builder self test This section explains how to perform the self test of the RU builder This test helps determine whether or not the RU builder has been successfully installed The self test consists of the RU BuilderTester application plus one EVM one RU and one BU all running on the same XDAQ executive The EVM is told to generate dummy triggers the RU is told to generate dummy super fragments and the BU is told to drop the events it builds The step by step instructions to run the self test are Step 1 Obtain and build the RU builder See section 9 for instructions Step 2 Open a terminal and create the XML configuration file for the self test by typing cd XDAQ_ROOT daq evb xml produceSelfTestXml pl HOST PORT XDAQ ROOT 1x1SingleXDAQ template xml gt 1x1SingleXDAQ xml Where HOST and PORT indicate the port on which the executive shall service HTTP requests DAQ CERN PH CMD 2005 Page 30 of 33 10 RU builder self test RU Builder User Manual Date 7 4 2005 Step 3 Run the XDAQ executive by typing XDAQ_ROOT daq xdaq bin linux x86 xdaq sh h HOST p PORT c 1x1SingleXDAQ xml Where HOST and PORT are the same as those given in step 2 Step 4 Open a web browser and enter the following URL http HOST PORT Where HOST and PORT are the same as those of steps 2 and 3 You should now see the HyperDAQ page
15. ce 4 2 RU RUI interface 4 3 BU FU interface Application state machines 5 1 Commonalities of the application finite state machines 5 2 BU EVM and RU finite state machines Starting the RU builder Stopping the RU builder Exported configuration parameters How to obtain and build the RU builder 9 1 Checking out the source code from CVS 9 2 Building the application libraries RU builder self test Configuration guidelines 25 26 29 29 30 30 33 List of Figures Figure 1 RU builder applications and how they interact with the rest of the EVB 9 Figure 2 BU FIFOS ecient ea a E ee idee Seeds enone ace eaten ad Ua 11 Figure 3 EVM PIPOS chico 5 ied eed ndite ie tiene addi at eco ee eal tet 12 Figure 4 RU FIPOS vcctor nti ieee dan Gane eliean dna 13 Figure 5 External 20 interfaces of the RU builder ec eeeeeseeeneeceeeeseeesneecnaeenseensees 14 Figure 6 TA EVM interface sequence diagram eeeceeescecssececesececeeeeecseeeeesneeeesneeeeseeeenaeeees 15 Figure 7 RU RUI interface sequence diagram eee eesscecsscecseececseececseeeeeseeeesseeeeneeeenaeeees 17 Figure 8 BU FU interface sequence diagram eeececeeccecsscecseccecseceeceeeeecseeeeseeesseeeenaeeess 18 Figure 9 FSTN of a RU builder application eee eeeccecseccecesececeeececseeeecseeeeseeeeneeeenaeeees 22 Figure 10 BU EVM and RU PS TNS naene a eae E eae 23 Figure 11 HyperDAQ web page for self test visG sare te dieiel oie es eal
16. d s to store incoming messages ready for the worker thread to process them The pairFIFO and freeEventIdFIFO are manipulated solely by the worker thread The pairFIFO keeps track of the event id trigger data pairs that have yet to be sent to requesting BUs The freeEvent IdFIFO stores the ids of free events for which trigger credits have been sent to the TA RU Readout p triggerFIFO pairFIFO Trigger TA Credit clearedEventIdFIFO freeEventIdFIFO Step 5 reques tHIEO Figure 3 EVM FIFOs DAQ CERN PH CMD 2005 Page 12 of 33 3 RU builder application FIFOs RU Builder User Manual Date 7 4 2005 3 3 RU FIFOs A RU is responsible for buffering super fragments until they are request by the BUs To understand the internal FIFOs of a RU itis first necessary to know its dynamic behavior Figure 4 shows the internal FIFOs of aRU The EVM sends a RU an event id trigger event number pair when it asks the RU to readout the corresponding event s data step 1 In parallel the RUI informs the RU of event data that is ready to be processed step 2 A RU places each super fragment for which it has received a pair into the fragment lookup table step 3 BUs ask RUs to send them the super fragments of the events they are building step 4 A RU services a BU request by retrieving the super fragment from its fragment lookup table and asking the BU to cache the super fragment step 5 Each RU has a worker t
17. e used 3 1 BU FIFOs A BU is responsible for building events An event is composed of one trigger super fragment and N RU super fragments where N is the number of RUs To understand the internal FIFOs of a BU itis first necessary to know its dynamic behavior Figure 2 shows the internal FIFOs of a BU With free capacity available aBU requests the EVM to allocate it an event step 1 The EVM confirms the allocation by sending the BU the event id and trigger data of an event step 2 This trigger data is the first super fragment of the event The BU now requests the RU s to send it the rest of the event s super fragments step 3 The BU builds the super fragments it receives from the RU s step 4 into a whole event within its resource table step 5 FU s can ask a BU to allocate them events step 6 A BU services a FU request by asking the FU to take a whole event step 7 When a FU has finished with an event it tells the BU to discard it step 8 Each BU has its own worker thread that executes the behavior of that BU The eventldFIFO blockFIFO requestFIFOs and discardFIFO are used by the peer transport thread s to store incoming messages ready for the worker thread to process them The fullResourceFIFO is manipulated solely by the worker thread It is used to store which events were built in which order This enables a BU to service a FU request with the next event that was built Step 3 Jeenal RU cacr E blockFIFO EVM event IdFIF
18. edef struct I20 EVENT _DATA_BLOCK_MESSAGE_FRAME I20_PRIVATE_MESSAGE FRAME PvtMessageFrame U32 eventNumber U32 nbBlocksInSuperFragment U32 blockNb U32 eventId U32 buResourceld U32 fuTransactionId U32 nbSuperFragmentsInEvent U32 superFragmentNb U32 padding I20_EVENT_DATA BLOCK_MESSAGE FRAME PI20_ EVENT_DATA BLOCK MESSAGE FRAME The FU should only read nbSuperFragmentsInEvent superFragmentNb nbBlocksInSuperFragment blockNb buResourcelId fuTransactionIid An event is composed of 120_EVENT_DATA_BLOCK_FRAMES The nbSuperFragmentsInEvent superFragmentNb nbBlocksInSuperFragment blockNb fields are used to identify the position of an event data block within an event An event is composed of one trigger super fragment plus N RU super fragments where N is the number of RUs Therefore the nbSuperFragmentsInEvent field is set to the number of RUs plus 1 The superFragmentNb field is numbered from o to nbSuperFragmentInEvent 1 The blockNb field is numbered from to nbBlocksInSuperFragment 1 The buResourceld field is an opaque handle that a FU should use to identify events resources to be discarded The fuTransactiontId field is the FU transaction id of the FU request that caused the BU to reply with the current event DAQ CERN PH CMD 2005 Page 20 of 33 4 120 interface RU Builder User Manual Date 7 4 2005 The I20_BU_DISCARD C structure is as follows typedef struct _I20 BU_DISCARD_ MESSAGE FRAME
19. ent block size 4096 x 16K 4KB 16384 The parameters of the form Packing should not normally be modified as they have only been tested with the default value of 8 However the user may modify them if they are experiencing performance problems with the RU builder DAQ CERN PH CMD 2005 Page 28 of 33 9 How to obtain and build the RU builder RU Builder User Manual Date 7 4 2005 9 How to obtain and build the RU builder This chapter is divided into two sections The first explains how to obtain the RU builder source code and the second explains how to build it Both sections assume the following e The user has already installed version 3 1 of the XDAQ core framework e The user has defined the shell environment variable XDAQ_RooT to point to the root of their XDAQ installation in other words their TriDAS directory e The shell of the user is tsch e The user accesses the CMS CVS server as an anonymous read only user 9 1 Checking out the source code from CVS The RU builder source code is stored in the CMS CVS server For more information about this server please look at the webpage http cmsdoc cern ch cmsoo projects cvs_server html You must login to the CMS CVS server if you wish to use it To do so enter the following setenv CVSROOT pserver anonymous cmscvs cern ch cvs_server repositories TriDAS cvs login You will be prompted for a password CVS password Please enter 98passwd The RU builder source code needs
20. geW essages P boot tue msgA geLimitD tM Sec exitOnFal o o fase blockFIFO Capacity unsigned long 16384 discardFIFO Capacity unsigned long 65536 BU EVM amp RU BU EVM amp RU BU EVM amp RU BU EVM amp RU I20_EVM_ALLO CATE CLEAR Packing unsignedlong 8 maxEvtsU nderConstruction unsigned long 64 requestFIFO Capacity 120_RU_SEND_ Packing unsigned long s 8 sendCreditsW ithDispatchFrame bool 20_RU_READOUT Packing unsigned long 8 20_TA_CREDIT_Packing unsigned long 8 fblockFIFO Capacity unsigned long 16384 Table 1 Exported configuration parameters BU bllockFIFOCapacity BU diiscardFiFOCapacity BU 120_EVM_ALLOCATE CLEAR Packing BU maxEvtsUnderConstruction BU requestFIFO Capacity BU 20 RULSEND Packing s The default values are set when the RU builder application is instantiated The default values have been chosen with the goal of covering the majority of use cases for the RU builder A user should rarely need to diverge from these default values DAQ CERN PH CMD 2005 Page 26 of 33 8 Exported configuration parameters RU Builder User Manual Date 7 4 2005 The following assumptions were made when calculating the default values of the RU builder s configuration parameters e A RU builder is composed of 64 BU s and 64 RUs e A RU has 64MB of physical memory for caching super fragments e Aneventis 1MB e An event is made up of 64 super fragments 1 per
21. hread that executes the behavior of that RU All of the internal FIFOs of a RU that is to say the pairFIFO blockFIFOand requestFIFOs are used by the peer transport thread s to store incoming messages for the worker thread to process RU EVM i pairFIFO Readout Fragment lookup educe TETOS ga Mi Cache Figure 4 RU FIFOs DAQ CERN PH CMD 2005 Page 13 of 33 4 120 interface RU Builder User Manual Date 7 4 2005 4 2O interface All the 120 messages of the EVB including the internal and external messages of the RU builder are defined in the package TriDAS dag interface The I20 function codes of all the RU builderl20 messages are given in the file TriDAS daq interface shared include i2oxXFunctionCodes h The C structures that define the 120 messages are in the file TriDAS daq interface evb include i20EVBMsgs h The 120 interface of the RU builder is subject to change The description of the interface provided by this document cannot be relied upon to be valid beyond this release Figure 5 shows the external 120 interfaces of the RU builder the TA EVM interface the RUI RU interface and the BU FU interface This chapter is divided into three sections one for each interface RUI RU interface RU builder wa Nal FU 0 FU q TA EVM interface ta gt BU FU interface Figure 5 Extemal 120 interfaces of the RU builder DAQ CERN PH CMD 2005 Page 14 of 33 4 120 interface
22. ice credits from the EVM 3 Start the RU builder so that it can receive super fragments from the RU Is 4 Start the RUIs and FUs so they can start pushing in super fragments and extracting events respectively The RU builder is a distributed application whose nodes BU s EVM and RUs need to be started up ina specific order To put the RU builder into the state where it will build events run control should do the following in order Send Configure to all of the RU builder applications Send Enable to all of the RUs Send Enable to the EVM Send Enable to the BUs PUNG Unlike version 2 x of the RU builder the state changes of version 3 0 are synchronous There is no need to poll the state of an application after a request to change state The RUs have to be enabled first because they have to be ready to receive event number event id pairs from the EVM The EVM can start sending these pairs immediately after it has been enable The EVM has to be enabled before the BU s so that it is ready to service their requests for event ids BUs can start requesting event ids as soon as they are enabled Enabling the EVM causes it to send an initial trigger credit count to the TA The number of initial credits is equal to the total number of event ids in the RU builder As soon as trigger data arrives at the EVM from the TA the EVM sends event number event id pairs to the RUs As explained in the previous paragraph this is why
23. ilder application DAQ CERN PH CMD 2005 Page 22 of 33 5 Application state machines RU Builder User Manual Date 7 4 2005 5 2 BU EVM and RU finite state machines The FSTN s specific to each type of RU builder application are shown in figure 10 RU Halted Halted Halted Configure Configure Configure Enable Enable Send request to Send initial EVM for initial credits to TA set of event ids Enabled Bnabied Figure 10 BU EVM and RU F STNs m DAQ CERN PH CMD 2005 Page 23 of 33 6 Starting the RU builder RU Builder User Manual Date 7 4 2005 6 Starting the RU builder The RU builder is part of a larger system called the event builder EVB Besides run control the RU builder communicates with a TA one or more RUIs and one of more FUs The RU builder cannot be started at any arbitrary moment in time its start up must be synchronized with that of the TA RUIs and FUs The RU builder and the EVB components it interacts with are XDAQ applications and as such depend on one or more peer transports to communicate with each other These peer transport must be up and running before these applications try to communicate with each other To start the RU builder and the components it interacts with run control should do the following in order 1 Start the necessary peer transports so that the RU builder and its surrounding applications can communicate 2 Start the TA so thatis can serv
24. lied upon to be valid beyond this release DAQ CERN PH CMD 2005 Page 6 of 33 1 Introduction RU Builder User Manual Date 7 4 2005 1 2 Intended readership This document is intended for a system integrator someone that needs to integrate the RU builderinto a data acquisition system DAQ It is assumed that the DAQ system is based on the XDAQ framework If the reader is not familiar with this framework then they are referred to the XDAQ website http xdag web cern ch xdaq 1 3 System requirements and dependencies Version 3 2 of the RU builder only supports the Linux operating system running on an Intel x86 processor The code was tested using version 3 2 3 of gcc This version of the RU builder is dependent on version 3 1 of the XDAQ core framework DAQ CERN PH CMD 2005 Page 7 of 33 1 Introduction RU Builder User Manual Date 7 4 2005 1 4 References 1 The CMS collaboration The Trigger and Data Acquisition project Volume II Data Acquisition amp High Level Trigger CERN LHCC 2002 26 ISBN 92 9083 111 4 1 5 Definitions Acronyms and Abbreviations BU Builder Unit 120 Intelligent Input O utput Cvs Concurrent Versioning System RU Readout U nit DAQ Data Acquisition system RUI Readout U nit Input EVB Event builder TA Trigger Adapter EVM Event M anager TDR Technical Design Report FED Front End Driver TriDAS Trigger and Data Acquisition FSTN Finite State Transition Network XDAQ Cross platform data acquisition toolki
25. ol parameters were calculated as follows e To prevent a BU from monopolizing event ids each BU has a maximum number of event ids it can acquire at any moment in time Assuming all BUs are equal each BU is allowed to acquire BU maxEvtsUnderConstruction nbEvtIdsInBuilder number of BUs 4096 64 64 e The blockFIFO of a BU see figure 2 in section 3 2 is responsible for buffering incoming event data In the worst case this FIFO would have to buffer the blocks of all outstanding requests for event data BU blockFIFOCapacity maxEvtIdsUnderConstruction x size of an event block size 64x1MB 4KB 64 x256 16384 e ABU hasa single FIFO called discardFIFO for FU discard messages and one FIFO per FU called requestFIFO for FU request messages Knowing that a BU can service a maximum of 64 FUs and assuming that a single FU will never have more than 1024 outstanding requests for events BU discardFIFOCapacity 1024 x maximum number of FUs 1024 x 64 65536 BU requestFIFOCapacity 1024 The default values of the RU exported parameters were calculated as follows e The blockFIFO of the RU is responsible for buffering incoming super fragment data Assuming a RUI only gives as many super fragments to a RU as the TA gives triggers to the EVM then in the worst case the blockFIFO must hold the blocks of as many super fragments as there are event ids in the RU builder blockFIFOCapacity nbEvtIdsInBuilder x size of a super fragm
26. s an I20 exception handler that logs asynchronous exceptions raised by peer transports when dealing with RU builder 120 messages e The RUBuilderTester gathers and displaysmore monitoring information e The web interface of each application has been improved this is subjective 1 1 Document purpose and scope The goal of this document is to enable the reader to integrate the RU builder into a running system composed of the RU builder itself a trigger source one or more event data sources one or more data sinks and some form of run control The RU builder cannot run without the components just listed This document describes how to obtain build and configure the RU builder This document does not describe the other components of a running system such as run control software or how to setup a trigger or event data source It is also not the purpose of this document to describe the internal workings of the RU builder Developers are referred to the source code for such information The code has been structured and commented so that it can be easily read and understood It is recommended to use Doxygen to generate documentation from the code as compatible comment tags have been used If the reader is not familiar with Doxygen then they are referred to its website htto www doxygen org It must be emphasized that the RU builder is still under development and subject to change No description of the RU builder given in this document can be re
27. t FU Filter U nit DAQ CERN PH CMD 2005 Page 8 of 33 2 RU builder overview RU Builder User Manual Date 7 4 2005 2 RU builder overview 2 1 How the RU builder fits within the EVB The RU builder is a component of a larger system called the event builder EVB The EVB isa distributed application that reads out event fragments from one set of nodes and assembles them into entire events in another set of nodes Figure 1 shows the applications of the RU builder and how they interact with the rest of the EVB FED builder SUPO r Fragments Trigger credits Trigger data Allocates Discards Event s Permanent storage Figure 1 RU builder applications and how they interact with the rest of the EVB The external interfaces of the RU builder assume that triggers are given to the EVM in the same order as their corresponding event data is given to the RUs DAQ CERN PH CMD 2005 Page 9 of 33 2 RU builder overview RU Builder User Manual Date 7 4 2005 2 2 What the EVB does For each event the EVB e Reads out the trigger data This trigger data will become the first super fragment of the event e Reads out the fragments of the event from the detector front end drivers FEDs e Builds the fragments into RU super fragments using the FED builder e Builds the whole event using the RU builder The whole event is the trigger super fragment plus the set of RU super fragments e Decides whether or
28. the RUs have to be enabled before the EVM Enabling a BU causes it to send its initial request for event ids to the EVM The number of initial event ids requested is equal to the maximum number of event ids the BU is allowed to acquire at any single moment in time DAQ CERN PH CMD 2005 Page 24 of 33 7 Stopping the RU builder RU Builder User Manual Date 7 4 2005 7 Stopping the RU builder The current version of RU builder foresees two ways of stopping the RU builder e Stop the trigger and event data entering the RU builder e Halt all of the RU builder application When stopping the trigger and event data entering the RU builder it is useful to know when the RU builder has finished building the events for which it has received triggers and event data This can be found out by reading the following exported parameter of the EVM EVM ruBuilderIsEmpty Halting a RU builder application causes it to discard destroy all the data in its internal data structures and to release all incoming 120 messages DAQ CERN PH CMD 2005 Page 25 of 33 8 Exported configuration parameters User Manual S Exported configuration parameters RU Builder Date 7 4 2005 Configuration parameters need to be set before an application is sent a Configure SOAP message Table 1 lists the exported control parameters of each type of RU builder application The type and default value of each parameter is given PARAMETER NAME nbEvtldsInBuilder a
Download Pdf Manuals
Related Search