RU Builder User Manual
Contents
1. 1 RU builder applications and how they interact with the rest of the EVB The external interfaces of the RU builder assume that triggers are given to the EVM in the same order as their corresponding event data is given to the Rus DAG CERN PH CMD 2006 Page 8 of 34 2 RU builder overview RU Builder User Manual 2 2 What the EVB does For each event the EVB e Reads out the trigger data This trigger data will become the first super fragment of the event e Reads out the fragments of the event from the detector front end drivers FED s e Builds the fragments into RU super fragments using the FED builder e Builds the whole event using the RU builder The whole event is the trigger super fragment plus the set of RU super fragments e Decides whether or not the event is interesting for physics using the filter units Fus e Sends the event to permanent storage if it is interesting for physics or discards it if it is not 2 3 The RU builder applications The RU builder consists of a single event manager EVM one or more readout units Rus and one or more builder units Bus The EVM is responsible for controlling the flow of data through the RU builder The Rus are responsible for buffering super fragments until they are requested by the Bus The Bus are responsible for building and buffering events until they are requested by the filter units Fus The trigger adapter TA readout unit inputs RU Is and filter units Fus are ext2. Assuming each RU has 64MB of memory for buffering super fragments and that the size of an event is 1 MB BU EVM amp RU nbEvtIdsInBuilder sum of the memory of all RUs size of an event 64 x 64MB 1MB 4096 The BUs and EVM send fast control messages e Fast control messages are sent if they are older than 1 second BU amp EVM ageMessages true BU amp EVM msgAgeLimitDtMSec 1000 It has been assumed that all events have the fixed size of IMB If the RU builder is to build events of varying sizes then the appropriate safety factor needs to be taken into account when calculating the number of event ids in the RU builder DAG CERN PH CMD 2006 Page 26 of 34 8 Exported configuration parameters RU Builder User Manual The default values of the BU control parameters were calculated as follows e To prevent a BU from monopolizing event ids each BU has a maximum number of event ids it Can acquire at any moment in time Assuming all BUs are equal each BU is allowed to acquire BU maxEvtsUnderConstruction nbEvtIdsInBuilder number of BUs 4096 64 64 e The blockFIFO of a BU see figure 2 in section 3 2 is responsible for buffering incoming event data In the worst case this FIFO would have to buffer the blocks of all outstanding requests for event data BU blockFIFOCapacity maxEvtIdsUnderConstruction x size of an event block size 64x1MB 4KB 64 x256 16384 e A BU hasa single FIFO called discardF
3. The module tag for libxdaq2rc so must appear within the context tag before any of the module tags libBU so ibEVM so or libRU so DAG CERN PH CMD 2006 Page 30 of 34 11 RU builder self test RU Builder User Manual LIRU builder self test This section explains how to perform the self test of the RU builder This test helps determine whether or not the RU builder has been successfully installed The self test consists of the RU BuilderTester application plus one EVM one RU and one BU all running on the same XDAQ executive The EVM is told to generate dummy triggers the RU is told to generate dummy super fragments and the BU is told to drop the events it builds The step by step instructions to run the self test are Step 1 Obtain and build the RU builder See section 9 for instructions Step 2 Open a terminal and create the XML configuration file for the self test by typing cd XDAQ_ ROOT dagq evb xml produceSelfTestXml pl HOST PORT XDAQ_ ROOT 1x1SingleXDAQ template xml gt 1x1SingleXDAQ xml W here HOST and PORT indicate the port on which the executive shall service HTTP requests Step 3 Run the XDAQ executive by typing SXDAQ_ ROOT dag xdagq bin linux x86 xdaq sh h HOST p PORT c 1x1SinglexXDAQ xml W here HOST and PORT are the same as those given in step 2 DAG CERN PH CMD 2006 Page 31 of 34 11 RU builder self test RU Builder User Manual Step 4 Open a web browser and enter the following URL http
4. a running system composed of the RU builder itself a trigger source one or more event data sources one or more data sinks and some form of run control The RU builder cannot run without the components just listed This document describes how to obtain build and configure the RU builder This document does not describe the other components of a running system such as run control software or how to setup a trigger or event data source It is also not the purpose of this document to describe the internal workings of the RU builder Developers are referred to the source code for such information The code has been structured and commented so that it can be easily read and understood It is recommended to use Doxygen to generate documentation from the code as compatible comment tags have been used If the reader is not familiar with Doxygen then they are referred to its website htto www doxygen org It must be emphasized that the RU builder is still under development and subject to change No description of the RU builder given in this document can be relied upon to be valid beyond this release 1 2 Intended readership This document is intended for a system integrator someone that needs to integrate the RU builder into a data acquisition system DAQ It is assumed that the DAQ system is based on the XDAQ framework If the reader is not familiar with this framework then they are refered to the XDAQ website http xdaq web cern c
5. event data entering the RU builder e Haltall of the RU builder applications When stopping the trigger and event data entering the RU builder it is useful to know when the RU builder has finished building the events for which it has received triggers and event data This can be found out by reading the following exported parameter of the EVM EVM ruBuilderIsEmpty Halting a RU builder application causes it to discard destroy all the data in its intemal data structures and to release all incoming I20 messages DAQ CERN PH CMD 2006 Page 24 of 34 8 Exported configuration parameters User Manual S Exported configuration parameters RU Builder Configuration parameters need to be set before an application is sent a Configure SOAP message Table 1 lists the exported control parameters of each type of RU builder application The type and default value of each parameter is given PARAMETER NAME nbEvtldsInBuilder ageM essages boor tue msgA geLimitD tM Sec exitO nFail rbot false blockFIFO Capacity unsigned long 16384 BU EVM amp RU BU EVM amp RU BU EVM amp RU BU EVM amp RU gJ C gJ C gJ discardFIFO Capacity 20_EVM_ALLOCATE_CLEAR Packing unsigned long 8 maxEvtsU nderConstruction unsigned long 64 requestFIFO Capacity 20_RU_SEND_Packing unsigned long 8 sendCreditsW ithD ispatchFrame bool false riggerFIFO Capacity 20_RU_READOUT_ Packing unsigned long 8 20_TA_CREDIT Packing unsigned
6. given the TA credits step 2 The TA is responsible for getting receiving trigger data from the trigger and for providing backpressure to the trigger as necessary The 120 TA CREDIT MESSAGE FRAME C structure is as follows typedef struct _I20 TA CREDIT MESSAGE FRAME I20_PRIVATE_ MESSAGE FRAME PvtMessageFrame U32 nbCredits I20_TA_CREDIT_MESSAGE_ FRAME PI20 TA CREDIT MESSAGE FRAME The EVM must fill nbcredits DAG CERN PH CMD 2006 Page 14 of 34 4 2O interface RU Builder User Manual The 120 EVENT DATA BLOCK MESSAGE FRAME C structure is as follows typedef struct I20 EVENT_DATA_BLOCK_MESSAGE FRAME I20_PRIVATE_MESSAGE FRAME PvtMessageFrame U32 eventNumber U32 nbBlocksInSuperFragment U32 blockNb U32 eventIid U32 buResourceld U32 fuTransactionIid U32 nbSuperFragmentsInEvent U32 superFragmentNb U32 padding I20_EVENT DATA BLOCK MESSAGE FRAME PI20 EVENT DATA BLOCK MESSAGE FRAME The TA must fill eventNumber nbBlocksInSuperFragment blockNb Version 3 0 of the RU builder only supports single block trigger data Therefore the TA must set nbBlocksInSuperFragment to 1 and blockNb to 0 DAG CERN PH CMD 2006 Page 15 of 34 4 2O interface RU Builder User Manual 4 2 RU RUI interface The RU RUI interface specifies how a RUI passes super fragments to a RU Figure 7 is a sequence diagram describing the protocol between the RUI and the RU RUI RU Function code I20 RU DATA READY C structure
7. long 8 blockFIFO Capacity unsigned long 16384 Table 1 Exported configuration parameters gJ C CU JJ C C EVM EVM EVM EVM The default values are set when the RU builder application is instantiated The default values have been chosen with the goal of covering the majority of use cases for the RU builder A user should rarely need to diverge from these default values DAQ CERN PH CMD 2006 Page 25 of 34 8 Exported configuration parameters RU Builder User Manual The following assumptions were made when calculating the default values of the RU builder s configuration parameters e A RU builder is composed of 64 BUs and 64 RUSs e A RU has 64MB of physical memory for caching super fragments e Aneventis 1MB e An event is made up of 64 super fragments 1 per RU of equal size therefore the size of a super fragment is 16KB e The block size size of an 120 message frame used to transport event data is 4KB e The RUIs only give as many events to the RU s as the TA gives triggers to the EVM e The maximum number of FUs per BU is 64 e A FU will never have more than 1024 outstanding requests for events e Fast control messages are sent if they are older than 1 second e The packing factor of fast control messages is 8 The need to know the total number of event ids in the RU builder is common to all three types of RU builder applications e The total number of event ids in the RU builderis a function of RU memory
8. not exceed the total number of event ids in the RU builder BU EVM RU nbEvtIdsInBuilder If this is the case then there is no guarantee that the EVM will be able to buffer BU requests for event ids or that the RUs will be able to buffer BU requests for event data The configuration parameters of a RU builder application must be set before it is configured DAG CERN PH CMD 2006 Page 34 of 34
9. ERN PH CMD 2006 Page 28 of 34 9 How to obtain and build the RU builder RU Builder User Manual 9 2 RU builder application and support libraries The RU builder is composed of seven application libraries TriDAS daq evb bu 1ib linux x86 1ibBU so TriDAS daq evb evm 1l1ib linux x86 1ibEVM so TriDAS dagq evb ru lib linux x86 1ibRU so TriDAS daq evb examples fu 1ib linux x86 1libFU so TriDAS dagq evb examples rui lib linux x86 1ibRUI so TriDAS dagq evb examples ta 1lib linux x86 1ibTA so TriDAS dagq evb rubuildertester 1l1ib linux x86 libRUBuilderTester so These libraries use the xdaq2rc library to send messages from XDAQ to run control TriDAS RunControl tools xdagq2rc lib linux x86 1ibRUBuilderTester so In other words 1ibBU so depends on libxdaq2rc so 1ibEVM so depends on libxdaq2rc so 1ibRU so depends on libxdaq2rc so See chapter 10 for how these dependencies effect the XDAQ configuration of a RU builder 9 3 Building the RU builder application and support libraries Enter the following to build the xdaq2rc library and the RU builder application libraries cd XDAQ ROOT RunControl tools xdaq2rc make cd XDAQ_ ROOT dag evb make DAG CERN PH CMD 2006 Page 29 of 34 10 Example configuration file RU Builder User Manual 10Example configuration file The following example configuration file shows how to configure a 1x1 RU builder to run on a single XDAQ executive lt xc Partition xmlns soapenc http schemas xmlsoap org
10. HOST PORT Where HOST and PORT are the same as those of steps 2 and 3 You should now see the HyperDAQ page of the XDAQ executive you started in step 3 It should look similar to the following screenshot O XAQ HyperDAQ httplocalhost cerm ch 65432 urmnxdag application senice hypendag Control Panel Cluster Explore FeerTranspontHTI Executive Relay PeerlranspartFil urmnicxdag application lid 1 urn sdaqg application sermic sa akedubeep plication senvic sametdag application lid s HyperlAc RUBuilderlester Ewi RL BLU urnixdag application senic satypdetageplication lid 12 urn sdag application lid 1 urn sdaqg application lid 1 urn sdag application lid 15 Figure 11 HyperDAQ web page for self test Step 5 Go to the default web page of the RU BuilderTester application by clicking on the RU BuilderTester link Your web browser should now display something similar to 5 RUBuilderTester0 Version EVB_S_18305_V3_9 l l TEE 0 Number of rubuilder exceptions 0 Number of state change notifications U EBS Control HyoerDAG event O Figure 12 RUBuilderT ester web page DAG CERN PH CMD 2006 Page 32 of 34 11 RU builder self test RU Builder User Manual Step 6 Go to the control web page by clicking on the Ctrl icon You should now see something like RUBuilderTesterO Version EVB_S_18305_V3_9 Ea Figure 13 RUBuilderTester control web page Step 7 Start the self test by clicking the
11. I20 EVENT DATA BLOCK MESSAGE FRAME og Figure 7 RU RUI interface sequence diagram A super fragment is composed of one or more I20 EVENT DATA BLOCK MESSAGE FRAMES The I20 EVENT DATA BLOCK _MESSAGE_FRAME C structure is as follows typedef struct I20 EVENT_DATA_BLOCK_MESSAGE_ FRAME I20_ PRIVATE_MESSAGE_ FRAME PvtMessageFrame U32 eventNumber U32 nbBlocksInSuperFragment U32 blockNb U32 eventIid U32 buResourceld U32 fuTransactionIid U32 nbSuperFragmentsInEvent U32 superFragmentNb U32 padding I20 EVENT DATA BLOCK MESSAGE FRAME PI20 EVENT DATA BLOCK MESSAGE FRAME The RUI must fill eventNumber nbBlocksInSuperFragment blockNb The nbBlocksInSuperFragment field gives the number of blocks the super fragment is composed of The blockNb field indicates the block s position within the super fragment Blocks are numbered from 0 to nbBlocksInSuperFragment 1 DAG CERN PH CMD 2006 Page 16 of 34 4 2O interface RU Builder User Manual 4 3 BU FU interface The BU FU interface specifies how e A FU requests events from a BU e ABU sends an event to a FU e A FU tells a BU to discard an event Figure 8 is a sequence diagram describing the protocol between a BU and a FU FU BU Step 1 Function code I20 BU ALLOCATE C structure I20 BU ALLOCATE MESSAGE FRAME Function code I20 FU_TAKE C structure I20 EVENT DATA BLOCK MESSAGE FRAME Step 3 sega nn Ca I20 BU DISCARD C
12. IFO for FU discard messages and one FIFO per FU called requestFIFO for FU request messages Knowing that a BU can service a maximum of 64 FUs and assuming that a single FU will never have more than 1024 outstanding requests for events BU discardFIFOCapacity 1024 x maximum number of FUs 1024 x 64 65536 BU requestFIFOCapacity 1024 The default values of the RU exported parameters were calculated as follows e The blockFIFO of the RU is responsible for buffering incoming super fragment data Assuming a RUI only gives as many super fragments to a RU as the TA gives triggers to the EVM then in the worst case the blockFIFO must hold the blocks of as many super fragments as there are event ids in the RU builder blockFIFOCapacity nbEvtIdsInBuilder x size of a super fragment block size 4096 x 16K 4KB 16384 The parameters of the form Packing should not normally be modified as they have only been tested with the default value of 8 However the user may modify them if they are experiencing performance problems with the RU builder DAQ CERN PH CMD 2006 Page 27 of 34 9 How to obtain and build the RU builder RU Builder User Manual 9 How to obtain and build the RU builder This chapter is divided into two sections The first explains how to obtain the RU builder source code and the second explains how to build it Both sections assume the following e The user has already installed the XDAQ core framework e The user has d
13. The Compact Muon Solenoid Experiment TriDAS Trigger and Data Acquisition RU Builder User Manual Version 3 9 May 8 2006 Authors S Murray Cl identifier EVB D 18306 CERNY Fermilab Revision History July 11 2003 July 31 2003 May 24 2004 Finalization of version 1 0 J Gutleber Updated for version 2 0 of the EVB now referred to as S Murray the RU builder a a 1 0 1 1 0 2 0 2 1 3 0 3 2 3 3 3 4 3 5 3 6 Wa 3 8 3 9 Cl Record C cc Cl Identifier EVB_D_18306 Description Describes the RU builder for an integrator who will put the builder into a full DAQ system Point of Contact S Murray Steven Murray cern ch Physical Location http smurray home cern ch smurray RUB_V3_9 doc 10 11 12 Table of Contents Introduction 1 1 Document purpose and scope 1 2 Intended readership 1 3 system requirements and dependencies 1 4 References 1 5 Definitions Acronyms and Abbreviations RU builder overview 2 1 How the RU builder fits within the EVB 2 2 What the EVB does 2 3 The RU builder applications RU builder application FIFOs 3 1 BU FIFOs 3 2 EVM FIFOs 3 3 RU FIFOs I20 interface 4 1 TA EVM interface 4 2 RU RUI interface 4 3 BU FU interface Application state machines 5 1 Commonalities of the application finite state machines 5 2 BU EVM and RU finite state machines Starting the RU builder Stopping the RU builder Exported configuration paramete
14. bered from O to nbSuperFragmentInEvent 1 The blockNb field is numbered from to nbBlocksInSuperFragment 1 The buResourceld field is an opaque handle thata FU should use to identify events resources to be discarded The fuTransactionlId field is the FU transaction id of the FU request that caused the BU to reply with the current event DAG CERN PH CMD 2006 Page 19 of 34 4 2O interface RU Builder User Manual The 120 BU DISCARD C structure is as follows typedef struct _I20 BU DISCARD MESSAGE FRAME I20_PRIVATE_MESSAGE_ FRAME PvtMessageFrame U32 n U32 buResourceld 1 I20 BU DISCARD MESSAGE FRAME PI20 BU DISCARD MESSAGE FRAME The FU must fill n buResourceld The n field specifies the number of events resources to be discarded The buResourceld field is an array of the ids of the BU resources to be discarded DAG CERN PH CMD 2006 Page 20 of 34 5 Application state machines RU Builder User Manual 5 Application state machines 5 1 Commonalities of the application finite state machines The finite state machines of the Bus EVM and Rus have commonalities Figure 9 shows the finite state transition network FSTN which all three types of application follow There are four common behaviors Firstly all RU builder applications read and act upon configuration parameters when they receive a Configure SOAP message Secondly all RU builder applications only participate in event building when they are enabled Th
15. efined the shell environment variable XDAQ ROOT to point to the root of their XDAQ installation in other words their TriDAS directory e The shell of the user istsch e The user accesses the CMS CVS server as an anonymous read only user 9 1 Checking out the source code from CVS The RU builder source code is stored in the CMS CVS server For more information about this server please look at the webpage http cmsdoc cern ch cmsoo projects cvs server html You must login to the CMS CVS server if you wish to use it To do so enter the following setenv CVSROOT pserver anonymous cmscvs cern ch cvs_server repositories TriDAS cvs login You will be prompted for a password CVS password Please enter 98passwd The RU builder source code needs to be placed in the directory KDAQ_ ROOT daqg evb Enter the following commands to put it there using the CMS CVS server cd XDAQ ROOT cvs export r EVB_S_18305_ V3_9 TriDAS daqgq evb The RU builder source code depends on the interface directory KDAQ_ROOT dag interface Enter the following commands to obtain this directory from the CMS CVS server cd XDAQ ROOT cvs export r EVB_S_18305 V3_9 TriDAS dag interface The RU builder source code depends on the xdaq2rc library directory XDAQ ROOT RunControl tools xdaq2rc Enter the following commands to obtain this directory from the CMS CVS server cd XDAQ ROOT cvs export r EVB_S_18305 V3_9 TriDAS RunControl tools xdaq2re DAG C
16. end Enable to all of the Rus Send Enable to the EVM Send Enable to the Bus a NY A Unlike version 2 x of the RU builder the state changes of this version are synchronous There is no need to poll the state of an application after a request to change state The Rus have to be enabled first because they have to be ready to receive event number event id pairs from the EVM The EVM can start sending these pairs immediately after it has been enable The EVM has to be enabled before the Bus so that it is ready to service their requests for event ids Bus can start requesting event ids as soon as they are enabled Enabling the EVM causes it to send an initial trigger credit count to the TA The number of initial credits is equal to the total number of event ids in the RU builder As soon as trigger data arrives at the EVM from the TA the EVM sends event number event id pairs to the Rus As explained in the previous paragraph this is why the Rus have to be enabled before the EVM Enabling a BU causes it to send its initial request for event ids to the EVM The number of initial event ids requested is equal to the maximum number of event ids the BU is allowed to acquire at any single moment in time DAG CERN PH CMD 2006 Page 23 of 34 7 Stopping the RU builder RU Builder User Manual 7 Stopping the RU builder The current version of RU builder foresees two ways of stopping the RU builder e Stop the trigger and
17. ernal to the RU builder The TA is responsible for interfacing the DAQ trigger to the EVM The RU s are responsible for pushing super fragment data from the FED builder into the Rus The Fus are responsible for selecting interesting events for permanent storage DAQ CERN PH CMD 2006 Page 9 of 34 3 RU builder application FIFOs RU Builder User Manual 3 RU builder application FIFOs The RU builder applications use FIFOs to keep track of requests trigger data and event data Knowledge of these FIFOs is required in order to correctly configure the RU builder This chapter is divided into three sections one for the BU one for the EVM and one for the RU Each section gives a brief description of the application s behavior and how its FIFOs are used 3 1 BU FIFOs A BU is responsible for building events An event is composed of one trigger super fragment and N RU super fragments where N is the number of Rus To understand the internal FIFOs of a BU it is first necessary to know its dynamic behavior Figure 2 shows the internal FIFOs of aBU With free capacity available aBU requests the EVM to allocate it an event step 1 The EVM confirms the allocation by sending the BU the event id and trigger data of an event step 2 This trigger data is the first super fragment of the event The BU now requests the Rus to send it the rest of the event s super fragments step 3 The BU builds the super fragments it receives from the Rus step 4 into a w
18. gure 10 BU EVM and RU FSTNs DAG CERN PH CMD 2006 Page 22 of 34 6 Starting the RU builder RU Builder User Manual 6 Starting the RU builder The RU builder is part of a larger system called the event builder EVB Besides run control the RU builder communicates with a TA one or more RUIs and one of more Fus The RU builder cannot be started at any arbitrary moment in time its start up must be synchronized with that of the TA RUIs and Fus The RU builder and the EVB components it interacts with are XDAQ applications and as such depend on one or more peer transports to communicate with each other These peer transport must be up and running before these applications try to communicate with each other To start the RU builder and the components it interacts with run control should do the following in order 1 Start the necessary peer transports so that the RU builder and its surrounding applications can communicate 2 Start the TA so that is can service credits from the EVM 3 Start the RU builder so that it can receive super fragments from the RU Is 4 Start the RUIs and Fus so they can start pushing in super fragments and extracting events respectively The RU builder is a distributed application whose nodes Bus EVM and Rus need to be started up in a specific order To put the RU builder into the state where it will build events run control should do the following in order Send Configure to all of the RU builder applications S
19. h xdaq DAQ CERN PH CMD 2006 Page 6 of 34 1 Introduction RU Builder User Manual 1 3 System requirements and dependencies Version 3 9 of the RU builder depends on version 3 5 of the XDAQ core framework 1 4 References 1 The CMS collaboration The Trigger and Data Acquisition project Volume II Data Acquisition amp High Level Trigger CERN LHCC 2002 26 ISBN 92 9083 111 4 1 5 Definitions Acronyms and Abbreviations BU Builder Unit 120 Intelligent Input O utput CVS Concurrent Versioning System RU Readout U nit DAQ Data Acquisition system RUI Readout U nit Input EVB Event builder TA Trigger Adapter EVM Event M anager TDR Technical Design Report FED Front End Driver TriDAS Trigger and Data Acquisition FSTN Finite State Transition N etwork XDAQ Cross platform data acquisition toolkit FU Filter U nit DAQ CERN PH CMD 2006 Page 7 of 34 2 RU builder overview RU Builder User Manual 2 RU builder overview 2 1 How the RU builder fits within the EVB The RU builder is a component of a larger system called the event builder EVB The EVB is a distributed application that reads out event fragments from one set of nodes and assembles them into entire events in another set of nodes Figure 1 shows the applications of the RU builder and how they interact with the rest of the EVB e FED builder Super fragments Trigger credits Trigger data Allocates Discards Events FU 0 FU q Permanent storage Figure
20. he 120 messages are in the file TriDAS dag interface evb include i20EVBMsgs h F The I2O interface of the RU builder is subject to change The description of the interface provided by this document cannot be relied upon to be valid beyond this release Figure 5 shows the external 120 interfaces of the RU builder the TA EVM interface the RUI RU interface and the BU FU interface This chapter is divided into three sections one for each interface RUI RU interface EVM RU builder TA EVM interface _ _ BU FU interface Figure 5 Extemal 120 interfaces of the RU builder DAG CERN PH CMD 2006 Page 13 of 34 4 2O interface RU Builder User Manual 4 1 TA EVM interface The TA EVM interface specifies how e The EVM gives the TA trigger credits e The TA gives the EVM trigger data Figure 6 is a Sequence diagram describing the protocol between the EVM and the TA TA EVM Function code I20 TA CREDIT Step ae C structure I20_TA_CREDIT_MESSAGE_FRAME Function code I20 EVM TRIGGER C structure I20 EVENT DATA BLOCK MESSAGE FRAME ee Figure 6 TA EVM interface sequence diagram The EVM communicates with the TA using a credit based mechanism The EVM tells the TA the current Capacity of the RU builder by sending the TA a trigger credit count step 1 One trigger credit represents the RU builder s ability to build one event The TA should only send the EVM trigger data for as many events as the EVM has
21. he trigger data with a free event id step 3 The EVM also requests the Rus to readout the event s data step 4 A BU with the ability to build an event will ask the EVM to allocate it an event step 5 Within such a request a BU will normally give back the id of an event to be cleared For each cleared event id the EVM sends a trigger credit to the TA and makes the id a free event id step 6 The EVM confirms the allocation of an event by sending the requesting BU the event id and trigger data of the allocated event step 7 The EVM has a worker thread that executes the behavior of the EVM The triggerFIFO clearedEventIdFIFO and requestFIFO are used by the peer transport thread s to store incoming messages ready for the worker thread to process them The pairFIFO and freeEventIdFIFO are manipulated solely by the worker thread The pairFIFO keeps track of the event id trigger data pairs that have yet to be sent to requesting Bus The reeEventIdFIFO stores the Ids of free events for which trigger credits have been sent to the TA RU Readout EVM creLggerct Iro mm i HHH Trigger TA Credit clearedEventIdFIFO freekventIdFIFO Allocate and or clear request Iro L BU Confirm Step 7 Figure 3 EVM FIFOs DAG CERN PH CMD 2006 Page 11 of 34 3 RU builder application FIFOs RU Builder User Manual 3 3 RU FIFOs A RU is responsible for buffering super fragments until they are request by the Bus To under
22. hole event within its resource table step 5 Fus can ask a BU to allocate them events step 6 A BU services a FU request by asking the FU to take a whole event step 7 When a FU has finished with an event it tells the BU to discard it step 8 Each BU has its own worker thread that executes the behavior of that BU The eventldFIFO blockFIFO requestFIFOs and discardFIFO are used by the peer transport thread s to store incoming messages ready for the worker thread to process them The fullResourceFIFO IS manipulated solely by the worker thread It is used to store which events were built in which order This enables a BU to service a FU request with the next event that was built lonk re le 1 Confirm TILL Resource Allocate new and or clear previous l fullResourceFIFO discardFIFO Ja CELE FU Allocate discard FU take Figure 2 BU FIFOs DAG CERN PH CMD 2006 Page 10 of 34 3 RU builder application FIFOs RU Builder User Manual 3 2 EVM FIFOs The EVM is responsible for controlling the flow of event data through the RU builder To understand the internal FIFOs of the EVM it Is first necessary to know its dynamic behavior Figure 3 shows the internal FIFOs of the EVM The EVM tells the TA the capacity of the RU builder by sending it trigger credits step 1 One trigger credit represents the ability to build one event Given a credit the TA sends the EVM the trigger data of an event step 2 The EVM pairs t
23. in other words It is not interpreted A BU will send back a copy of the fuTransactionId field in each of the I20_EVENT_ DATA BLOCK _MESSAGE_FRAMES that make up the requested event The fset field isa fragment set identifier Fragment sets are a way to describe partial events The fset field is ignored by the BU in this version of the RU builder because this version does not support partial events DAG CERN PH CMD 2006 Page 18 of 34 4 2O interface RU Builder User Manual The 120 EVENT DATA BLOCK MESSAGE FRAME C structure is as follows typedef struct I20 EVENT_DATA_BLOCK_MESSAGE FRAME I20_PRIVATE_MESSAGE FRAME PvtMessageFrame U32 eventNumber U32 nbBlocksInSuperFragment U32 blockNb U32 eventIid U32 buResourceld U32 fuTransactionIid U32 nbSuperFragmentsInEvent U32 superFragmentNb U32 padding I20_EVENT DATA BLOCK MESSAGE FRAME PI20 EVENT DATA BLOCK MESSAGE FRAME The FU should only read nbSuperFragmentsInEvent superFragmentNb nbBlocksInSuperFragment blockNb buResourcelId fuTransactionIid An event is composed of 120 EVENT DATA BLOCK FRAMES The nbSuperFragmentsInEvent superFragmentNb nbBlocksInSuperFragment blockNb fields are used to identify the position of an event data block within an event An event is composed of one trigger super fragment plus N RU super fragments where N is the number of Rus Therefore the nbSuperFragmentsInEvent field is set to the number of Rus plus 1 The superFragmentNb field is num
24. irdly all RU builder applications throw away their internal data and any incoming I20 message frames when they are halted Fourthly all RU builder applications go to the Failed state when they encounter a fatal error A RU builder application cannot be recovered from the Failed state Configuration parameters are read and acted The application can only participate in event upon when the application is configured building when it is in the Enabled state Fatal error The contents of all internal data structures are discarded when the application is halted The application releases all incoming 120 message The application cannot be recovered frames when it is in the Halted state from the Failed state Figure 9 FSTN ofa RU builder application DAQ CERN PH CMD 2006 Page 21 of 34 5 Application state machines RU Builder User Manual 5 2 BU EVM and RU finite state machines The FSTNs specific to each type of RU builder application are shown in figure 10 Halted Halted Configure Configure Enable Enable send request to Send initial S Failed l Failed EVM for initial Credits to TA set of event Enabled Enabled _ Batted alted Configure Ready Enable Enabled Mismatch tolerating true TimedOut tolerating true MismatchDetected TimedOut Tolerating Tolerating Mismatch tolerating false TimedOut tolerating false MismatchDetected TimedOut BackPressuring BackPressuring Fi
25. rs How to obtain and build the RU builder 9 1 Checking out the source code from CVS 9 2 RU builder application and support libraries 9 3 Building the RU builder application and support libraries Example configuration file RU builder self test Configuration guidelines Oo N NNOO O 24 25 28 28 29 30 31 34 List of Figures Figure 1 RU builder applications and how they interact with the rest of the EVB 00 0 8 Fee BU wl Uk dormer teeter ter nen E Tp Sener Re eer ee Ree eee ee 10 Prove 5 EV MiP OS ereina E E E E E A OE EEEO 11 Firu ARU FIFO ereciaieiin nea a TAO E E A CO O 12 Figure 5 External 2O interfaces of the RU builder cccennneteteeeeeseeseeeeeeeeeeeees 13 Figure 6 TA EVM interface sequence iagram sssssssssesccccccccccccceceeeeessssssssssseesesesseees 14 Pioure 7 RU RUT mterf ace S quence Cia er aint csin eri e E 16 Figure 8 BU FU interface sequence diagram ccccccssssssseccccccceeeeesseccceceeseaeeseeecceeesseeaeesees 17 Figure 9 FSTN ofa RU builder applications sssciccncnicccniniinnin 21 Fiure 10 BU EVM and RU FSIN secsi on E E T 22 Ficore 11 HyperDAO webpage dor Sell St asr E E 32 Figure 12 RU Butlderl ester Web pase cnomcrreneiinn a a E eee 32 Figure 13 RUBuilderTester control Web page sirenes E E 33 Pioure 4 Running RUDU incest eacteioves tans conven O 33 List of Tables Table 1 Exported configuration parameters ccccccccccsssssssseecceeeceaeeee
26. soap encoding xmlns xc http xdaq web cern ch xdaq xsd 2004 XMLConfiguration 30 xmlns xsi http www w3 org 2001 XMLSchema instance gt lt i20 protocol xmlns i2o0 http xdaq web cern ch xdaq xsd 2004 I20Configuration 30 gt lt i2o target class EVM instance 0 tid 23 gt lt i2o target class RU instance 0 tid 24 gt lt i2o0 target class BU instance 0 tid 25 gt lt i20 protocol gt lt xc Context url http localhost 65432 gt lt xc Module gt XDAQ_ ROOT RunControl tools xdaq2rc lib linux x86 libxdaq2rc so lt xc Module gt lt xc Application class RUBuilderTester id 12 instance 0 network local gt lt xc Module gt XDAQ_ROOT daq evb rubuildertester 1lib linux x86 libRUBuilderTester so lt xc Module gt lt xc Application class EVM id 13 instance 0 network local gt lt xc Module gt XDAQ_ROOT daq evb evm 1ib linux x86 1ibEVM so lt xc Module gt lt xc Application class RU id 14 instance 0 network local gt lt xc Module gt XDAQ_ROOT daq evb ru 1lib linux x86 1ibRU so lt xc Module gt lt xc Application class BU id 15 instance 0 network local gt lt xc Module gt XDAQ ROOT dag evb bu 1ib linux x86 1ibBU so lt xc Module gt lt xc Context gt lt xc Partition gt N Two import rules for the module ta 1 The module tag for libxdaq2rc so must appear once within each context tag where there is a module tag for libBU so 4bDEVM so or libRU so 2
27. sseecceeeeeeaaeeseeeeeeeeeeaaas 25 1 Introduction RU Builder User Manual I Introduction The RU builder is a distributed XDAQ application that is part of a larger system called the event builder EVB The CMS data acquisition group is presently developing the EVB as described in the TriiDAS TDR 1 This document explains how to obtain build and configure version 3 9 of the RU builder The new features introduced by this release are e The BU EVM and RU now use the xdaq2rc library to send asynchronous state change messages to run control e The finite state machine FSM of the RU has been expanded to indicate whether or not the RU Is tolerating or back pressuring when it detects an event number mismatch or a time out in the event data stream e The RU is now decoupled from BU instance numbers to facilitate the prototyping of multiple slices e Bug fix The BU and EVM now correctly resynchronize with the RU mask when they are configured e The HTTP content type of the BU EVM RU and example applications is now explicitly set This release was driven by the needs of central DAQ However other users will be affected by the use of the xdaq2rc library and by the changes made to the FSM of the RU The use of the xdaq2rc library is documented in chapters 9 and 10 The new finite state machine of the RU is documented in chapter 5 1 1 Document purpose and scope The goal of this document is to enable the reader to integrate the RU builder into
28. stand the internal FIFOs of a RU itis first necessary to know its dynamic behavior Figure 4 shows the internal FIFOsofa RU The EVM sendsaRU an event id trigger event number pair when it asks the RU to readout the corresponding event s data step 1 In parallel the RUI informs the RU of event data that is ready to be processed step 2 A RU places each super fragment for which it has received a pair into the fragment lookup table step 3 Bus ask Rus to send them the super fragments of the events they are building step 4 A RU services a BU request by retrieving the super fragment from its fragment lookup table and asking the BU to cache the super fragment step 5 Each RU has a worker thread that executes the behavior of that RU All of the internal FIFOs of a RU that is to say the pairFIFO blockFIFOand requestFIFOS are used by the peer transport thread s to store incoming messages for the worker thread to process blockFIFO EVM Readout pairFIFO E Fragment request Iros fe Cache Figure 4 RU FIFOs DAG CERN PH CMD 2006 Page 12 of 34 4 2O interface RU Builder User Manual 4 1I2O interface All the 120 messages of the EVB including the internal and external messages of the RU builder are defined in the package TriDAS dagq interface The 120 function codes of all the RU builder 20 messages are given in the file TriDAS dagq interface shared include i2oXFunctionCodes h The C structures that define t
29. start button The button should now be labeled stop Step 8 Return to the main web page of the RU builder tester by clicking on the RUB Test icon Clicking the refresh button of your browser should now show events being built For example the eventN b variable of the EVM O should increment gt RUBuilderTester0 Version EVB_S_18305_V3_9 reS i ene pees UO Number of rubuilder exceptions 0 Number of state change notifications 0 FUS Control HyoerDaAg evento 58112 Figure 14 Running RU builder DAG CERN PH CMD 2006 Page 33 of 34 12 Configuration guidelines RU Builder User Manual 12 Configuration guidelines This chapter summarizes and highlights the most important points with regards to configuring the RU builder The default values of the RU builder control parameters have been chosen for the majority of use cases The user should rarely need to diverge from these values The RU builder is dependent on the instance numbers of the BUs EVM FUs RUs and TA RU s must be assigned instance numbers from 0 to the number of RUs 1 BU s must be assigned instance number from 0 to the number of BUs 1 The EVM must be assigned instance number 0 The TA must be assigned instance number 0 builder has the following configuration restrictions A single BU can service a maximum of 64 FUs The sum of the maximum number of event ids each BU can have at any moment in time BU maxEvtsUnderConstruction must
30. structure I20 BU DISCARD MESSAGE FRAME Figure 8 BU FU interface sequence diagram A FU requests a BU to allocate it one or more events step 1 In response the BU asks the FU to take the requested event data as a set of event data blocks step 2 When a FU has finished processing one or more events it tells the BU to discard them step 3 Fi The BU FU interface of this version of the RU builder does not support partial events A Partial events may be supported in a future version DAQ CERN PH CMD 2006 Page 17 of 34 4 2O interface RU Builder User Manual The I20_BU_ALLOCATE MESSAGE FRAME C structure and its companion BU_ALLOCATE C structure are as follows typedef struct _BU_ALLOCATE U32 fuTransactionIid U32 fset BU_ALLOCATE PBU_ ALLOCATE typedef struct _I20 BU ALLOCATE MESSAGE FRAME I20_PRIVATE_MESSAGE FRAME PvtMessageFrame U32 n BU_ALLOCATE allocate 1 I20 BU_ALLOCATE MESSAGE FRAME PI20 BU ALLOCATE MESSAGE FRAME The FU must fill n allocate The n field specifies the number of events the FU is requesting The allocate field is an array of FU transaction ids and fragment sets For each event a FU requests the FU fills in the fuTransactionId field and the fset field of a BU_ALLOCATE C structure and puts itin the allocate array The fuTransactionlId field is a transaction id that a FU can use to match its requests with the events it receives A BU treats the fuTransactionId field as being opaque
Download Pdf Manuals
Related Search