Home

DABC manual

1. lt Port name Input0 gt lt InputQueueSize value 5 gt lt MbsServerKind value Transport gt lt MbsServerName value 1xi009 gt lt MbsServerPort value 6000 gt lt Port gt lt Port name Inputl gt lt InputQueueSize value 5 gt lt MbsServerKind value Transport gt lt MbsServerName value 1xi010 gt lt MbsServerPort value 6000 gt lt Port gt lt Port name Input2 gt lt InputQueueSize value 5 gt lt MbsServerKind value Transport gt lt MbsServerName value 1xi011 gt lt MbsServerPort value 6000 gt lt Port gt lt Port name FileOutput gt lt OutputQueueSize value 5 gt lt MbsFileName value combiner lmd gt lt MbsFileSizeLimit value 128 gt lt Port gt lt Port name ServerOutput gt lt OutputQueueSize value 5 gt lt MbsServerKind value Stream gt 102 DABC Programmer Manual Example MBS lt Port gt lt Module gt lt Context gt lt dabc gt 14 9 MBS upgrade for DABC This section is rather for the MBS programmer than for application programmers To have minimal changes we use standard collector and transport Two changes 14 9 1 Increased buffer size support This is done in a completely compatible way For the following see data structures 14 9 4 3 page 103 and 14 9 4 2 page 103 The only problem is the 16 bit i_used field in the old buffer header s_bufhe structure new iUsed ke
2. uint32_t iTimeSpecNanoSec compatible with s_bufhe 2 32bit struct timespec uint32_t iEndian compatible with s_bufhe free 0 uint32_t iWrittenEndian LMD__ENDIAN_BIG LMD__ENDIAN_ LITTLE LMD__ENDIAN_ UNKNOWN uint32_t iUsedWords total words following header free 2 uint32_t iFree3 free 3 sMbsFileHeader 14 9 4 4 Data element structures e Time stamp typedef struct uint32_t iMaxWords uint32_t iType uint32_t iTimeSpecSec uint32_t iTimeSpecNanoSec sMbsTimeStamp e Common data item header typedef struct uint32_t iWords following data words uint32_t iType compatible with s_vel0_1 low type 10 high subtype sMbsHeader e MBS event header type 10 1 typedef struct uint32_t iWords data words 4 uint32_t iType compatible with s_vel0_1 low type 10 high subtype uint32_t iTrigger uint32_t iEventNumber sMbsEventHeader e MBS subevent header typedef struct uint32_t iWords data words 2 uint32_t iType compatible with s_ves10_1 low type 10 high subtype uint32_t iSubeventID 2 low bytes procid next byte subcrate high byte control sMbsSubeventHeader 14 9 4 5 Some fixed numbers define LMD YPE_FILE_HEADER_101_1 0x00010065 define LMD YPE_EVENT_HEADER_10_1 0x0001000a define LMD YPE_FILE_INDEX_101_2 0x00020065 define LMD_ TYPE BUFFER_HEADER_10_1 0x0001000a define LMD YPE_BUFFER_HEADER_100_1 0x00010064 define LMD YPE_TIME_STAMP_11_1 0x
3. uint32_t iMaxWords compatible with s_bufhe total buffer siz header uint32_t iType compatible with s_bufhe low type 100 high subtype uint32_t iUsed not used for iMaxWords gt MAX__DLEN 16360 low 16bits only uint32_t iBuffer compatible with s_bufhe uint32_t iElements compatible with s_bufhe uint32_t iTemp Used volatile uint32_t iTimeSpecSec compatible with s_bufhe 2x32bit struct timespec uint32_t iTimeSpecNanoSec compatible with s_bufhe 2 32bit struct timespec uint32_t iEndian compatible with s_bufhe free 0 uint32_t iWrittenEndian LMD__ENDIAN_BIG LMD__ENDIAN_LITTLE LMD__ENDIAN_UNKNOWN uint32_t iUsedWords total words without header free 2 uint32_t iFree3 free 3 sMbsBufferHeader 14 9 4 3 File header File header maps s_bufhe some fields used in different way typedef struct uint32_t iMaxWords Size of largest element in file uint32_t iType compatible with s_bufhe low type 100 high subtype Imdoff_t iTableOffset optional offset to element index table in file uint32_t iElements Number of elements in file uint32_t ioffsetSize Offset size 4 or 8 bytes uint32_t iTimeSpecSec compatible with s_bufhe 2x32bit struct timespec 104 DABC Programmer Manual Example MBS
4. 10 3 Control system plug in o o DAT CEROS o eoe ai ert ee ne 1032 Dialer u ca By A a a e a a a 19321 Virtual methods lt s 2 24 464 66548 4 e 10 3 2 2 Baseclass methods 004 10 3 3 Default implementation for DIM 10 3 3 1 dimc Manager CC 10 3 3 2 dime Registry 2 2 C ee 10 3 3 3 dimc Server oes ea si a CC CC 10 3 3 4 dimc ServiceEntry CC 22 10 3 3 5 dimc ParameterInfo o o o o o o coo 11 DABC Programmer Manual Services 11 1 Memory management 0 0 11 1 1 Zero copy Approach o oo v e be nenn 112 Memory pool area nr BUA BUSES ties wad er a ILIA Poin eg seo acts A HIS Duler oote s sk aw eae en S a Mil ANCHO lt 2 eared bark SOR a DG de wa Gees 11 2 Threads organization o e 1121 Werking loop s ca 0434 cias e ba 11 22 Sockets handling x ca dea es A 12 2 Die mie o ana ee 11 3 Command execution u a us Her ewe es nenn 1131 Conan gt oc cr 22 oa aoe Go Kon aan 1132 Commandresener os e cossa esa sen ee 1133 Commandclient o ia sc so cs se urn 12 DABC Programmer Manual Plugins 64 64 65 65 65 66 66 66 66 66 68 68 69 69 70 70 70 71 71 71 72 72 73 74 74 75 75 75 75 76 76 77 78 81 Vili CONTENTS 12 1 Introduction 12 2 Modules 12 2 1 12 22 122 3 12 2 4 12 2 5 12 2 6 12 2 7 12 3 Device and transport 12 3 1 12 3 2 12 33
5. 1824 DABC GUIusage fDIM 2 2 2 2 e Lars eee 136 18 3 GUI global layout osmosis SS oe Ee nr a ee we 137 183 1 Prompter panels occ saosi oa Ba ea na van ee aes 137 1842 APIS PADE S oe 2 a ee RE Ro a ai here ee A oe 137 184 Gl Paiels e coi eh ee aE A ba AA EES ee o 137 184 1 ABC launch patel oi to 2er are nn ee 137 184 2 MBS launch panel o oo oos 24 42 02 2 0 2 un au na ee 137 18 4 3 Combined DABC and MBS launch panel 2 2 2 2 2 eee 137 18 44 Parametertable 2 020 4064 bb 42 be eae edb Od ba bw a es 138 184 5 Parameter selection panel e o co coc oneco tem ean enwa ek e e ee 138 1846 Command panel lt sa nde ea sa an a an en 138 184 7 Montenne panels es ocea sea e N ee eat 138 LSAT Meler oo ciora coa ee en a a ad 138 PATa SRE o o ae nern ach ar a ee a E a ee en Dee ee a 138 BRA BOM oaa es SA a A we ee a a Se o 138 BRAS IO o e ee ee a ia Ae Boe ee o ie a 138 SATS SOO 2G a ee Eb ee ES Cae ee 139 184 8 Logzing window o ee a ee er ee Es 139 145 GUL saverrestoce setups ck ca Re en en a rear 139 el Record anne 2 nee anna 139 1852 Parameter filter o oaei 2 Ae ea a Shh ee ea eS 139 Pes WEONES TANDIL inc wis ee a er gee A ee 140 18 54 DABC launch panel values 2 2 2 eee 140 1855 MBS launch panel values oos oe sos uosa 22 a ee ee 140 13 6 DIM update mechanism o o ee u wen sea ba Hee eee eet 141 186 1 xDimBrowser ee ee ee 141 18 6 2 Getting parameters and commands 0 00200
6. for unsigned n 0 n lt NumOutputs n Output n gt Send buf gt MakeReference dabc Buffer Release buf The dabc Buffer object has a 32 bit type identifier which can be set with method SetTypeld and can be retrieved with GetTypeld Its purpose is to identify the type of the buffer content The value of this identifier is application specific for instance the MBS plugin defines its own type which is then used by the transports to distinguish if the buffer contains an MBS event format Each dabc Buffer can be supplied with an additional header This is piece of memory which is allocated and managed by the pool separately from the main payload memory and in generally should be smaller than the payload memory The idea of the buffer header is to add user specific information to an already existing Buffer without changing the contained payload data and even without touching the Buffer identifier The header size can be set by SetHeaderSize method the pointer to the header can be obtained by GetHeader method The main difference between header memory and payload memory concerns the behaviour when the Buffer is send via a zero copy network transport implementation like InfiniBand verbs in contrast to the payload data which will be transferred directly from the Buffer memory by DMA the header contents will be explicitely copied first 11 1 4 Pointer Class dabc Pointer provides a virtual contiguous access to s
7. Chapter 15 DABC Programmer Manual Example Bnet programmer prog exa bnet tex 15 1 Overview Complex experiments feature a lot of front end systems running in parallel These take data and mark them with trigger information or just with time stamps To completely analyze such data all portions belonging to the same event or time stamp must be combined in one processing unit Such task is usually called event building To support event building functionality in DABC a special sub framework called BNET Building NETwork was introduced Its main purpose is to simplify the experiment specific implementation of such event building distributed over several network nodes A typical event building network contains several readout nodes each connected to several data sources A readout node reads data from its data sources and combines together data parts which logically belong together this is called subevent building In case of a triggered system it combines together data with the same trigger number in case of time stamped data it combines together data which belongs to the same time interval Because there are several readout nodes building a complete event requires to bring together all data of the same trigger number or time interval resp into the same builder node Typically the system has not only a single builder node but several of them so full connectivity between all readouts and all builder nodes is necessary Once
8. 10 3 3 2 dimc Registry The main component of the dimc Manager that offers service methods really implementing the manager interface It registers all parameters commands and subscriptions and it defines the allowed access methods for the DIM server itself 1 The DIM server is instantiated in the constructor as dimc Server singleton Methods StartDIMServer and StopDIMServer actually initiate and terminate the service 2 Naming of nodes and services Method GetNodeName int num of dimc Manager uses CreateDIMPre fix num of dimc Registry This evaluates the unique name for node number num from the dabc Configuration object It consists of a global prefix DABC the configuration NodeName and the ContextName property of the node id all separated by forward slashes The node name is also taken as prefix for the helper methods BuildDIMName ReduceDIMName resp that transform local DABC parameter and command names into unique DIM names and back resp Moreover methods CreateFullParameterName ParseFullParameterName resp define how the local parameter name itself is composed decomposed resp from the names of its parent module and its internal variable name They utilize corresponding static methods of class dimc nameParser in a thread safe way 3 Parameter export dimc Manager ParameterEvent uses methods RegisterParameter and Unregister Parameter resp to declare undeclare resp a corresp
9. e dabc LocalTransport and dabc LocalDevice for memory transport within same process e dabc SocketTransport and dabc SocketDevice for tcp ip sockets e verbs Transport and verbs Device for InfiniBand verbs connection e pci Transport and pci BoardDevice for DMA I O from PCI or PCIe boards dabc Manager Is manager of everything in DABC There is the only instance of manager in the process scope availible via dabc mgr or dabc Manager Instance functions It combines different roles 1 It is a manager of all dabc Basic objects in the process scope Objects e g modules devices parameters are kept in a folder structure and can be identified by full path name 2 It defines the interface to the controls system state machine remote command communication pa rameter export this is to be implemented in a subclass The manager handles the command and parameter flow locally and remotely commands submitted to the local manager are directed to the command receiver where they shall be executed If any parameter is changed this is recognized by the manager and optionally forwarded to the associated controls system Current implementations of manager are e dabc Manager provides base manager functionality can only be used for single node application without any controlling possibilities e dabc StandaloneManager provides simple socket controls connection between several node in multi node cluster cannot be used with GUI 60 DA
10. lt dabc version 1 gt lt Variables gt lt node0 value 1xi008 gt lt nodel value 1xi009 gt lt node2 value 1xi010 gt lt node3 value 1xi011 gt lt node4 value 1xi012 gt lt custport value 16015 gt lt Variables gt lt dabc gt Here nodeO specifies the node where the cluster controller will run nodel and node3 are used as readout nodes node2 and node4 as builder nodes On all four worker nodes one MBS generator application will be started To run the application just type run sh SetupBnetMbs xml Chapter 16 DABC Programmer Manual Example ROC programmer prog exa roc tex 16 1 Overview The CBM ReadOut Controller ROC is an FPGA based board to configure and read out the nXYTER chip 5 and to transport the acquired data over Ethernet to aPC The software package ROClib provides the basic functionality to work with such ROC To support the usage of ROC in DABC the following classes were implemented roc Device device class wrapper for the SysCoreController class of ROClib roc Transport corresponding transport with access to the functionality of SysCoreBoard class of ROClib roc CombinerModule module to combine data from several ROCs into a single output port roc CalibrationModule module to calibrate the time scale in ROC data roc ReadoutApplication application to perform readout from ROC boards roc Factory factory class to organize these plugins 16 2 Device and
11. The following example configuration file SDABCSYS applications mbs Combiner xml shows how to configure one combiner module reading from two MBS transport servers A simple setup looks like this lt xml version 1 0 gt lt dabc version 1 gt lt Context host localhost name MbsEb gt lt Run gt lt lib value libDabcMbs so gt lt func value StartMbsCombiner gt lt Run gt lt Module name Combiner gt lt NumInputs value 2 gt lt DoFile value false gt lt DoServer value true gt lt BufferSize value 16384 gt lt Port name Input0 gt lt MbsServerKind value Transport gt lt MbsServerName value X86 xx gt lt MbsServerPort value 6000 gt lt Port gt lt Port name Inputl gt lt MbsServerKind value Transport gt lt MbsServerName value X86 yy gt lt MbsServerPort value 6000 gt lt Port gt lt Port name FileOutput gt lt OutputQueueSize value 5 gt lt MbsFileName value combiner lmd gt lt MbsFileSizeLimit value 128 gt lt Port gt lt Port name ServerOutput gt lt MbsServerKind value Stream gt lt Port gt lt Module gt lt Context gt lt dabc gt We have one node Context with a simple run function StartMbsCombiner that uses a single Module to do the event combination from two input Ports The node names and other parameters of the external MBS connec tions are specified in the MbsServerName prope
12. The most important classes of the DABC core system are described in the following dabc Basic The base class for all objects to be kept in DABC collections e g dabec Folder dabc Command Represents a command object A command is identified by its name which it keeps as text string Additionally acommand object may contain any number of arguments integer double text These can be set and requested at the command by their names The available arguments of a special command may be exported to the control system as dabc CommandDefinition objects A command is sent from a dabc CommandClient object to a dabc CommandReceiver object that executes it in its scope The result of the command execution may be returned as a reply event to the command client The manager is the standard command client that distributes the commands to the command receivers i e module manager or device See chapter 11 3 for more details on the command mechanisms 58 DABC Programmer Manual Overview bnet SenderModule inheritance ernennen association Figure 9 3 Simplified UML diagram of the most important DABC classes for active components Framework base classes are coloured in green Some implementation examples are shown with other corresponding colours sockets verbs Bnet PCI and MBS See text for details dabc Parameter Parameter object that may be monitored or changed from control system An
13. lt Run gt lt Application class bnet Cluster gt lt NetDevice value dabc SocketDevice gt lt Application gt lt Context gt lt Context host 1xi009 name Worker1 42 gt lt Context host 1xi010 name Worker2 42 gt lt Context host 1xi011 name Worker3 42 gt lt Context host 1xi012 name Worker4 42 gt lt Defaults gt lt Context name x gt lt Run gt lt logfile value test DABCNODEID log gt lt loglevel value 1 gt lt lib value libDabcBnet so gt lt Run gt lt Context gt lt Context name Workerx x gt lt Run gt lt lib value DABCSYS applications bnet test libBnetTest so gt 45 46 DABC User Manual DABC Application Bnet lt Run gt lt Application class bnet TestWorker gt lt IsGenerator value true gt lt IsSender value true gt lt IsReceiver value true gt lt NumReadouts value 4 gt lt Application gt lt Context gt lt Defaults gt lt dabc gt The setup of such BNET contains several lt Context gt nodes Generally the BNET has two types of nodes e One Controller node that has a master controller functionality implemented in the lt Application gt of class bnet Cluster The controller node must be specified at the DABC GUI setup to receive the direct cluster control commands e g state machine transitions commands In the DABC BNET framework the controller also keeps a general parameter lt NetD
14. next event from the sockets Handling these events allows to send and receive of data via sockets in a non blocking manner i e one can run several socket operations in parallel with one thread At the same time dabc SocketThread class allows to run normal jobs implemented with base class dabe WorkingProcessor So within a SocketThread one can mix socket processors like some Transports with normal processors like modules A similar approach was used to support the InfiniBand verbs APT in DABC 11 2 3 Mutex usage All methods of dabc WorkingThread and dabc WorkingProcessor are thread safe except for those started with underscore _ symbol So user code could avoid mutexes completely But if data is shared between Proces sors which run in different threads one should use mutexes though Here it is recommended to work with a 76 DABC Programmer Manual Services dabc LockGuard This class takes care that the mutex will be unlocked automatically whenever the current func tion scope is left For instance if one has global static variable associated with a mutex one should implement a thread safe setter method like this include dabc threads h int GlobalVariable 0 dabc Mutex GlobalMutex void SetGlobalVariable int newvalue dabc LockGuard guard GlobalMutex GlobalVariable newvalue 11 3 Command execution The idea of command execution is to invoke user specific code from any part of the s
15. xPanelParameter extending JPanel Is rebuilt from scratch by xDesktop whenever the DIM service list has been updated The panel gets the list of parameters xDimParameter from the DIM browser xDimBrowser It builds a table from all visible parameters It creates a list of command descriptors xXmlParser 18 4 5 Parameter selection panel xPanelSelect extending xPanelPrompt This form can be used to specify various filters on parameter attributes Parameters matching the filters are shown in a separate frame Values are updated on DIM update and can be modified interactively 18 4 6 Command panel xPanelCommand extending JPanel Is rebuilt from scratch by xDesktop whenever the DIM service list has been updated This panel is split into a right and a left part On the left there is the command tree on the right the argument prompter panel for the currently selected command The panel gets the list of commands xDimCommand from the DIM browser xDimBrowser The list of command descriptors xXmlParser is copied in xDesktop from xPanelParameter to xPanelCommand and the xXmlParser objects are added to the xDimCommand objects they belong to 18 4 7 Monitoring panels These panels are very similar to xPanelGraphics but have additional functionality TODO In the future xPanel Graphics should be extended to provide all that functionality or at least serves as base class xPanelMeter JPanel for rate meters xMeter xPanelState JPanel
16. 12 3 4 12 35 12 3 6 12 5 Factories 13 DABC Programmer Manual Setup 13 1 Parameter class 13 2 Use parameter for control 13 3 Example of parameters usage 13 4 Configuration parameters 13 5 Usage of commands for configuration 14 DABC Programmer Manual Example MBS 14 1 Overview 14 2 Event iterators 14 3 File VO 14 4 Socket classes Parameters and configurations Commands processing ModuleSync ModuleAsync Special modules Transport heh Brian ee DES ee eS oe he a Local transport Network transport Data transport Input output objects 12 4 The DABC application 14 5 Server transport 14 6 Client transport 14 7 Event generator 14 8 MBS event building 14 9 MBS upgrade for DABC CONTENTS 1X 14 9 1 Increased buffer size support os oec Common 1492 Variable sized Dullers ur caw ee ee hae sa eh 1493 New LND tle OL 0 a A we ee ee 14 9 4 MBS data structures 2 0 ee 14 9 4 1 Connect to MBS transport 2 2 Hmm onen 14942 Bullet besdir cocos ae a ES RR eee SE a 141943 Ple header oo eb eh ea ewe EASE EG EAH as 14 9 4 4 Data element structures lt lt nenn a a 14 945 Someixeimimber sso sose coa e a ne eee a 1495 MBS update for DIM control oo ce 2 4 20 8 as tses adanada 140 3 1 New or ROOMS TER ee a ee au len a 11952 TS N 5 ee ee en 149 53 MBS lauliche oscila a 14 9 5 4 MBS DIM commands and parameters o o e 14 9 5 5 DIM control modes 2 2200 lt l
17. 6006 gt lt Port gt lt Module gt lt Context gt lt dabc gt Example file DABCSYS applications mbs GeneratorTest xml demonstrates the usage of a gener ator module 14 8 MBS event building 101 14 8 MBS event building Class mbs CombinerModule allows to combine events from several running MBS systems It has following parameters Name Type Dfit Description BufferSize int 16384 buffer size of output data Numinputs int 2 number of mbs data sources DoFile bool false create LMD file store for combined events DoServer bool false create MBS server to provide data further The module implements two optional output ports for file storage port name FileOutput and for providing data further over an MBS server port name ServerOutput Function StartMbsCombiner initializes the combiner module and starts data taking The following example configuration file SDABCSYS applications mbs Combiner xml shows how to configure a combiner module reading from three MBS transport servers lt xml version 1 0 gt lt dabc version 1 gt lt Context host localhost name Worker gt lt Run gt lt lib value libDabcMbs so gt lt func value StartMbsCombiner gt lt logfile value combiner log gt lt Run gt lt Module name Combiner gt lt NumInputs value 3 gt lt DoFile value false gt lt DoServer value true gt lt BufferSize value 16384 gt
18. DABC This is also a good example of the dabc SocketProcessor classes mbs ServerTransport is based on the generic class dabe Transport and uses internally two kinds of sockets one socket for the connection handling and another I O socket for sending data The server transport has following parameters Name Type Dfit Description MbsServerKind str Transport kind of mbs server Transport or Stream MbsServerPort int 6000 server port number for socket connection These parameters can be set in the XML configuration file like this lt Module name Generator gt lt Port name Output gt lt MbsServerKind value Transport gt lt MbsServerPort value 16020 gt lt Port gt lt Module gt To create such transport and connect it to the module s output port the following code should be executed dabc mgr gt CreateTransport Generator Output mbs typeServerTransport GeneratorThrd Another possibility to specify these parameters consists in the command dabc CmdCreate Transport which may wrap such values dabc Command cmd new dabc CmdCreateTransport Generator Output mbs typeServerTransport MbsTransThrd cmd gt SetStr mbs xmlServerKind mbs ServerKindToStr mbs StreamServer cmd gt SetInt mbs xmlServerPort mbs DefaultServerPort mbs StreamServer 5 dabc mgr gt Execute cmd 14 6 Client transport Class mbs ClientTransport allows to connect DA
19. Is invoked by the framework when any Parameter is created argument value event parCreated 0 changed event parModified 1 or destroyed event parDestroy 2 resp Pointer par should be used to access parameter name and value for ex port to the control system void CommandRegistration dabc Module m dabc CommandDefinition def bool reg Is invoked by the framework when any module exports argument reg true or unexports argument reg false a command definition object to or from the control system resp This allows to invoke such commands via the controls connection from a remote node The command definition object def contains a description of possible com mand parameters pointer m should be used to access the owning module and get its name This information may be used to represent the command within the controls implementation bool Subscribe dabc Parameter par int remnode const char remname This method shall link the value of a local parameter par to a remote parameter of name remname that exists on node number remnode of the DAQ cluster Control system implementation may use a publisher subscriber mechanism here to update the local subscription whenever the remote parameter changes its value The actual update handler must call method InvokeChange const char val of the local dabc Parameter par then The new value val is passed to the parameter which will change itself appropriately This decouples the paramet
20. MSG DataRateKb KB s y Meter Statistics Trend Columns gt Limits Defaults Be 0 100 0 setAutoscale setLog Limits Clear Colors gt Large Figure 4 10 Steering menus 4 2 GUI Panels 29 On the left side in Fig 4 10 page 28 the Settings menu is shown It affects all items in the panel One can Zoom toggle between large and normal view change the number of columns change the display mode toggle Autoscale and set limits applied to all meters Besides that each individual item can be adjusted by right mouse button The context menu is shown on the right Allchanges done individually are changing the defaults The global changes can be overwritten by these defaults All settings are saved with the setup and restored on GUI startup see 4 3 page 30 4 2 8 3 Histograms Histogram panels are handled in pretty much the same way as the rate meters All histograms are a Histograms Layout Figure 4 11 Histograms displayed in the histogram panel Fig 4 11 page 29 Histograms can have arbitrary size set in Layout menu 4 2 8 4 Information ALLL Figure 4 12 Info Information records mainly display one line of text with a color see Fig 4 12 page 29 4 2 8 5 Logging window Fig 4 13 page 30 show the logging window 30 DABC User Manual GUI Logging DABC TxgO gsi de 1969 Controller 41 statusMessage State mac
21. all corresponding subevents have been delivered to the same builder node the complete event may be build and eventually stored on disk or tape For such use case the BNET framework defines a programming interface to implement the functional units 1 e applications and modules and it already provides several important components BNET also defines the topology of these functional units which can be customized up to a definite level 15 2 Controller application The event building task is usually distributed over several nodes which must be controlled and synchronized Therefore in BNET all nodes are classified by their functionality in two kinds controller node and worker nodes Workers perform all data transport and run the sub event building code The controller configures and steers all workers The controller node is implemented as bnet ClusterApplication class Via the control system interface the cluster controller distributes commands from the operator GUI to all workers It observes the state of all workers and may reconfigure them automatically if errors are detected 113 114 DABC Programmer Manual Example Bnet The functionality of bnet ClusterApplication is based on state machine logic of DABC All actions are performed during the execution of a state changing command implemented in virtual method DoStateTransition A state transition on the cluster controller node means that the appropriate state transition is perf
22. application Execution of these commands is independent of the state machine transitions 2 3 3 Configuration and monitoring The configuration is done using parameter objects On application startup time the configuration system may set the parameters from a configuration file e g XML configuration files During the application lifetime the control system may change values of the parameters by command However since the set up is changed on DoConfigure time only it may be forbidden to change true configuration parameters except when the application is Halted Otherwise there would be the possibility of a mismatch between the monitored parameter values and the really running set up However the control system may change local parameter objects by command in any state to modify minor system properties independent of the configuration set up e g switching on debug output change details of processing parameters The current parameters may be stored back to the XML file Apart from the configuration the control system may use local parameter objects for monitoring the components When monitored parameters change the control system is updated by interface methods of the manager and may refresh the GUI representation Programmer Manual Chapter 13 page 91 will explain the usage of parameters for configuration in detail 2 4 Package and library organisation The complete system consists of several packages 2 4 1 Core system The
23. for states xState xPanelInfo JPanel for infos xInfo xPanelHisto JPanel for histograms xHisto The monitoring panels contain special graphics objects 18 4 7 1 xMeter Displays a changing value between limits as rate meter bar histogram or trend With the right mouse a context menu is popped up where one can switch between these modes One also can change the limits autoscale mode limits are adjusted dynamically and the color 18 4 7 2 xRate Displays a changing value between limits as bar Very compact with full name 18 4 7 3 xState Displays a severity as colored box together with a brief text line 18 4 7 4 xHisto Displays a histogram 18 5 GUI save restore setups 139 18 4 7 5 xInfo Displays a colored text line 18 4 8 Logging window xPanelLogger extending JPanel Central window to write messages 18 5 GUI save restore setups There are several setups which can be stored in XML files and are retrieved when the xGUI is started again DABC_ CONTROL DABC Values of DABC control panel Saved by button in panel Default DabcControl xml Filename in panel itself DABC_ CONTROL_MBS Values of MBS control panel Saved by button in panel Default MosControl xml Filename in panel itself DABC_RECORD_ATTRIBUTES Attributes of records Saved by main save button Default Records xml DABC_ PARAMETER _ FILTER Values of parameter filter panel Saved by main save button Default Selection xml DABC
24. lt NumEventsCombine value 1 gt lt Application gt lt Context gt lt Context host readl name Read1 gt lt Application class bnet MbsWorker gt lt NumReadouts value 1 gt lt InputO0Cfg value mbs1 gt lt Application gt lt Context gt lt Context host read2 name Read2 gt lt Application class bnet MbsWorker gt lt NumReadouts value 1 gt 6 2 MBS and DABC with Bnet 43 lt Input0Cfg value mbs2 gt lt Application gt lt Context gt lt Context host eb1 name Build1 gt lt Context host eb2 name Build2 gt lt Defaults gt lt Context name x gt lt Run gt lt logfile value Context log gt lt loglevel value 1 gt lt cpuinfo value 1 gt lt Run gt lt Module name x gt lt Ratemeter name Datax lower 0 upper 20 gt lt Ratemeter name Event lower 0 upper 20000 gt lt Module gt lt Context gt lt Context name Readx gt lt Run gt lt lib value libDabcBnet so gt lt lib value libDabcMbs so gt lt lib value libBnetMbs so gt lt Run gt lt Application class bnet MbsWorker gt lt IsSender value true gt lt ReadoutBuffer value bufsize gt lt Application gt lt Module name Combiner gt lt Port name Inputx gt lt MbsServerPort value custport gt lt InputQueueLength value 20 gt lt Port
25. see section 5 1 2 page 31 But most of the buttons are very common From left to right they startup a system configure it start data taking pause data taking stop tasks shut down At the very left we see a save button at the right a shell execution button Values are read from file DabcControl xml default may be saved restored to from other file see 4 3 page 30 lt xml version 1 0 encoding utf 8 gt lt DabcLaunch gt lt DabcMaster prompt DABC Master value node xxx de gt lt DabcName prompt DABC Name value Controller 41 gt lt DabcUserPath prompt DABC user path value myWorkDir gt lt DabcSystemPath prompt DABC system path value dabc gt 24 DABC User Manual GUI DabcController Login Name server xg0523 gsi de User name goofy Password RET Master node xg0523 gsi de Master name Controller 41 Servers 5 System path misc goofy sniff dabc User path misc goofy dabc work Setup file SetupMbs xml Script ps Control file DabcControl xml CUMA E Figure 4 3 DABC controller panel lt DabcSetup prompt DABC setup file value SetupDabc xml gt lt DabcScript prompt DABC Script value ps gt lt DabcServers prompt Number of needed DIM servers value 5 gt lt DabcLaunch gt DabcMaster Node where the master controller shall be started Can be one of the worker nodes DabcName
26. size of buffer used to read data from ROCs default 16384 e NumOutputs number of outputs default 2 As output MBS events are provided Each MBS event contains ROC messages between two sync markers For each ROC a separate MBS subevent is allocated field Subcrate of the subevent header contains the ROC id 16 4 Calibration module Class roc CalibrationModule performs the calibration of the time scale for all ROCs and merges all messages into a single data stream As output an MBS event with a single subevent is produced The module has following configuration parameters e NumRocs number of ROC boards which should be provided in MBS event default 2 e BufferSize size of buffer used to produce output data default 16384 e NumOutputs number of outputs default 2 16 5 Readout application The main aim of roc ReadoutApplication class is to configure and run the application which combines the data readouts from several ROCs It can store the data into a 1md file and it may provide MBS stream or transport servers for online monitoring e g with a remote Go4 analysis It has following configuration parameters NumRocs number of ROC boards Roclp0 Roclp1 Roclp2 addresses IP or nodname of ROC boards DoCalibr defines calibration mode see further BufferSize size of buffer NumBuffers number of buffers MbsServerKind kind of MBS server None Stream Transport RawFile name of 1md file to s
27. 22 42 2 ea nn Be RE a he are 53 9 2 8 DI ll hae ee ira bbe Bae A a be ar he ae 53 929 APOC oo ola ro a de eee Pa ae ee eee Ae ee 53 93 Controls and configuration oo ec ca cac eati rn na na 53 93 Finite state MACIAS o za za ba Qe a ee ee GREE Sw cle ale Ee 53 O32 COMMONS o o Ge a RG RO a a We Re ee SR Sree 55 9 3 3 Parameters for configuration and monitoring ooo e 55 94 Package and library organisation ooo ee a 55 9 4 1 KERN HEN ooe a hoa eee ee o 55 9 4 2 Control and configuration system oo 55 9 4 3 Plugin packages oe cde re er er 56 Oeil Boet packase 3 4 ua a wa Bann 56 DAS Transport packages lt a cuadra Brass Wah ae eae Gad aaa ie 56 9 4 4 Application packages 2 2 ee o eA a eee ee ee de 57 9 4 5 Distribution contents o so eo ca RR Re ae an na na 57 OS Man a ee ee Sa das a ea dee eo GaSe oa cle dale dae 57 431 KERNE o AA A o A U E a 57 95 2 BNET classes eo a a Car ee A eee we ee 60 10 DABC Programmer Manual Manager 63 101 Tumoducton o kek eke ae ah eo ee eek ae e 63 10 2 PEA work Wenge oe g Gade ee a Sea wd es ee See Rel a A N 63 102 1 General object management 0 24 zu Kaas an an dad Eee HA Ee HS 63 CONTENTS vii 10 22 Factory methods a ec oc cu he eee tw edad 10 2 3 Module manipulation 1024 Thread management o 0 5 isc ss 10 2 5 Command submission o 10 2 6 Memory pool management 10 2 7 Miscellaneous methods
28. Chapter 3 page 13 describes how to install the DABC packages on any linux machine and how to set up the working environment Additionally some typical use cases and their configuration files are shown The following chapters then give more detailed explanations how to operate in different modes with the DABC Java GUI Chapter 4 page 21 covers the general functionality of the GUI which is common for most applications Especially this is mostly sufficient to control a DAQ cluster purely with one or several DABC nodes Chapter 5 page 31 describes the DABC GUI in a mode to control a pure MBSdata acquisition system without a native DABC node The application use case for a mixed DAQ cluster both with DABC and MBS nodes is treated in Chapter 6 page 39 Chapter 7 page 45 describes the use case of a DABC builder network BNET both with and without using MBS Finally Chapter 8 page 47 describes the use case of ROC front ends However the scope of the DABC User Manual does not contain detailed descriptions of the DABC framework architecture the software mechanisms and the example programs These subjects are treated thouroughly in the DABC Programmer Manual 1 2 Release Notes 1 2 1 Version 1 0 01 10 March 2009 1 Add IP multicast support in SocketTransport 2 Add IB multicast support in verbs Transport DABC User Manual Overview gt vo So Un Possibility to add user defined parameters directly in xml file
29. DIM record and command descriptors is implemented with Java It may be extendable with user defined components Application packages some example applications such as o Simple MBS event building o Bnet with switched MBS event building o Bnet with random generated events Chapter 3 DABC User Manual Setup user user setup tex 3 1 Installing DABC When working at the GSI linux cluster the DABC framework is already installed and will be maintained by people of the gsi EE department Here DABC needs just to be activated from any GSI shell by typing dabclogin dot space In this case please skip this installation section and proceed with following section 3 2 page 14 describing the set up of the user environment However if working on a separate DAQ cluster outside GSI it is mandatory to install the DABC software from scratch Hence the DABC distribution is available for download at http dabc gsi de It is provided as a compressed tarball of sources dabc_vn m ss tar gz where n m and ss are version numbers The following steps describe the recommended installation procedure 1 Unpack this DABC distribution at an appropriate installation directory e g cd opt dabc tar zxvf dabc_v1 0 00 tar gz This will extract the archive into a subdirectory which is labelled with the current version number like opt dabc dabc_v1 0 00 This becomes the future DABC system directory 2 Prepare the DABC environment login script A template for
30. Device is typically created in the user application by 12 3 Device and transport 87 dabc mgr gt CreateDevice roc Device ROC Later one can find this device with dabc Manager FindDevice method Each dabc Device implementation should define the virtual method CreateTransport such that an appropriate Transport instance is created and connected to the specified Port This factory method is invoked by the framework when the device is connected to a module port This is usually specified in the user application by calls of dabc mgr gt CreateTransport or dabc mgr gt ConnectPorts resp Similar to the Module functionality the Device class may export configuration Parameters It may also define Commands which are handled by extending the virtual method ExecuteCommand with a device specific imple mentation 12 3 3 Local transport dabc LocalTransport implements the connection between two local ports i e the ports are on the same node with a common memory address space It organizes a queue which is shared between both connected ports and performs the movement of dabc Buffer pointer through this queue If corresponding modules run in the same thread LocalTransport works without any mutex locking To manage the LocalTransport the dabc Manager always has instance of dabc LocalDevice class It can be accessed via dabc mgr gt FindLocalDevice call To connect two local ports one shoul
31. EvSizeTrendB 128 File filling status in percent typically only on one node transport fx FileFilled 100 StreamsFull 100 TriggerRate 10000 Trigger rates for the individual triggers 01 15 x Trigger0lRate 10000 States 5 3 Working directories 37 Delayed or immediate event building i x BuildingMode Current eventbuilding running or suspended EventBuilding Shows spill signal Ex SpillOn Shows if file is open typically only on one node transport FileOpen Show trigger master TriggerMode F User integers from daqst 00 15 can be set by f_ut_set_daqst_user index value Ex UserVal_00 fx TriggerCount Trigger counts for the individual triggers 01 15 Fx Trigger0lCount t Histograms Fx TrigCountHis fx TrigRateHis F Node XXX uppercase XXX EventRate 10000 FXXX DataRateKb 16000 FXXX FileOpen XXX FileFilled 100 XXX SpillOn XXX EventTrend 10000 XXX DataTrendKb 16000 FXXX TriggerMode 38 DABC User Manual MBS GUI Chapter 6 DABC User Manual DABC Application MBS user user app mbs tex 6 1 MBS event building with DABC In this case one DABC node reads data from several MBS nodes via Transport socket connections and combines them into one MBS output event To run MBS front ends with DABC node
32. Failure This state is reached when there is an error in a state transition function Note that a run error during the Running state would not lead to Failure but rather to stop the run in a usual way to Ready The state transitions between the 5 generic states correspond to commands of the control system for each node application DoConfigure between Halted and Configured The application plug in creates application specific devices modules and memory pools Application typically establishes all local port connections DoEnable between Configured and Ready The application plug in may establish the necessary con nections between remote ports The framework checks if all required connections are ready DoStart between Ready and Running The framework automatically starts all modules transport and device actions DoStop between Running and Ready The framework automaticall stops all modules transport and device actions i e the code is suspended to wait at the next appropriate waiting point e g begin of MainLoop wait for a requested resource Note queued buffers are not flushed or discarded on Stop DoHalt switches states Ready Running Configured or Failure to Halted The framework automati cally deletes all registered objects transport device module in the correct order 10 DABC User Manual Introduction 2 3 2 Commands The control system may send user defined commands to any component module device
33. IsName Output DOUT1 Output flag changed to s DBOOL GetParBool Output For performance reasons one should avoid to use parameter getter setter methods like GetParBool or SetParInt inside a loop being executed many times The main purpose of a parameter object is to provide a connection to the control and configuration system In other situations simple class members should be used 13 4 Configuration parameters Another use case of parameters consists in the object configuration When one creates an object like a module or a device it is often necessary to deliver one or several configuration values to the constructor e g the required 13 5 Usage of commands for configuration 93 number of input ports or a server socket port number For such situation configuration parameter are defined These parameters should be created and set in the object constructor with following methods only GetCfgStr string GetCfgDouble double GetCfgInt integer GetCfgBool boolean All these methods have following arguments the parameter name a default value optional and a pointer to a Command object optional Let s add one configuration parameter to our module constructor UserModule const charx name dabc Command cmd 0 dabc ModuleAsync name cmd CreateParBool Output true CreateParlInt Counter 0 double period GetCfgDouble Period 1 0 cmd CreateTimer Timer period false Here
34. The arguments are passed as string A command structure could look like password char 14 argument char string Format C The argument string has the same XML as the command description Thus the same parser can be used to encode decode the description parameter and the command An alternate format is the MBS style format argument value where boolean arguments are given by argument if argument is true 18 2 2 9 Setting parameters If a parameter should be changable from the xGUI there must be a command for that A fixed command SetPa rameter must be defined on the server for that Argument is a string of form name value In the parameter table of the xGUI one field can be provided to enter a new value and the command SetParameter is used to set the new value 18 2 3 Application servers Any application which can implement DIM services can be controlled by the generic xGUIif it follows the protocol described above The first application was DABC the second one MBS 18 2 4 DABC GUI usage of DIM The service names follow a structured syntax as described above The name fields are used to build trees for com mands Using the DIM quality longword delivered by the server together with each update simple aggregated data services records are defined Currently the records STATE RATE HISTOGRAM COMMANDDESC and INFO are used When the xGUI receives the first update of a service immediately after subscribing it can determine the r
35. be applied without the DABC data flow engine The only feature which is not supported by dabe DataInput is the CallBack mode In addition methods dabc DataInput Read_Init and dabc DataOutput Write_Init can be implemented to get configuration parameters from the port object to which the i o object is assigned to A typical use case of input output objects is the file I O For instance 1md file handling is implemented using these classes To instantiate such classes user should inplement factories methods CreateDataInput and CreateDataOutput see 12 5 12 4 The DABC Application The specific application controlling code is defined in the dabc Application On startup time the dabc Application is instantiated by means of a factory method CreateApplication As argument the factories get the application class name provided from the configuration file Thus to use his her application implementation the user must provide a dabc Factory that defines such method The manager has exactly one application object the name of this object is always App The application singleton can be accessed from everywhere via dabc mgr gt GetApp call The application may register parameters that define the application s configuration These parameters can be set at runtime from the configuration file or by controls system The application class implements the user specific actions during the state machine transitions The appli
36. by a dedicated working thread The thread executes a method MainLoop with arbitrary code which may block the thread In blocking calls of the framework resource or port wait optionally command callbacks may be executed implicitly non strictly blocking mode In the strictly blocking mode the blocking calls do nothing but wait A timeout may be set for all blocking calls this can optionally throw an exception when the time is up On timeout with exception either the MainLoop is left and the exception is then handled in the framework thread or the MainLoop itself catches and handles the exception On state machine commands e g Halt or Suspend see section 9 3 1 the blocking calls are also left by exception thus putting the mainloop thread into a stopped state 51 52 DABC Programmer Manual Overview 9 2 1 2 Class dabc ModuleAsync Several asynchronous modules may be run by a shared working thread The thread processes an event queue and executes appropriate callback functions of the module that is the receiver of the event Events are fired for data input or output command execution and if a requested resource e g memory buffer is available The callback functions must never block the working thread Instead the callback must return if further processing requires to wait for arequested resource Thus each callback function must check the available resources explicitly whenever it is entered 9 2 2 Commands A module
37. column indicate if the parameter is shown in some graphics panel It can be removed from or added to this panel by the buttons The table can be ordered by columns click on column header The column width can be adjusted and is saved restored by main save button see 4 3 page 30 4 2 GUI Panels 27 ES ParameterSelect ParameterList Parameter selection Full name Date Selected 15 Node X86 7 DABC X86 7 MSG DataRateKb 6553 0 Application MSG DABC X86 7 MSG DataTrendKb 6553 0 Name DABC X86 7 MSG EventRate 3192 0 Check full DABC X86 7 MSG EventTrend 3192 0 Check nodes DABC X86 7 MSG FileFilled 0 0 ene anne DABC X86 7 MSG StreamRateKb 0 0 NE DABC X86 7 MSG StreamTrendKb 0 0 DABC X86 7 MSG StreamsFull 0 0 DABC X86G 4 MSG DataRateKb 14924 0 DABC X86G 4 MSG DataTrendKb 14924 0 DABC X86G 4 MSG EventRate 7295 0 Infos DABC X86G 4 MSG EventTrend 7295 0 A E AR AG R Kh AN Below only Rates States Figure 4 7 Parameter selection panel and selected parameter list 4 2 7 1 Parameter selection To get a more selective view on the parameters one can specify filters in the panel shown at the left side of Fig 4 7 page 27 Text substrings for each of the four name fields can be specified as well as a selection of record types Values can be saved see 4 3 page 30 With the check boxes the filter function for each of these can de activated The p
38. const char name const char required_class 0 Access to a WorkingThread by name The required_class string may be specified to check if the working thread implementation matches the client intentions Returns 0 if thread object does not exist or if it does not fullfill required_class Application GetApp Access to the unique Application Object of this node 10 2 2 Factory methods Since all DABC objects are provided by dabc Factory plug ins the application programmer needs to invoke corresponding factory methods to instantiate them However the factories themselves should not be accessed by the user code although the Manager offers a getter method see section 10 2 1 Instead creation and registration of the key objects like Module or Device is done transparently by the Manager within specific creation methods These will scan over all existing factories whether the corresponding factory method can provide an object of the requested class name In this case the object is created kept in the object manager and may be addressed by its full name later bool CreateModule const char classname const char modulename const char thrdname 0 Instantiate a Module of class classname with the object name modulename Optionally the name of the working thread thrdname may be specified that shall run this module If a thread of this name is already exisiting it will be also applied for the new module otherwise a new thread of t
39. e by such rule we set the output queue length for all modules on all nodes This allows to configure a big multi node cluster with a compact XML file Another possibility to set default value for some parameters create parameter with the same name in parent object Here word create is crutial one should use CreateParInt method in module constructor itis not enough just put additional tag in xml file For instance one can create parameter MbsServerPort in generator module and than MBS server transport created for output port will use that value for as default server port number 3 4 Installation of additional plug ins Apart from the DABC base package there may be additional plug in packages for specific use cases Generally these plug in packages may consist of a plugins part and an applications part The plugins part offers a library containing new components like Devices Transports or Modules The appli cations part mostly contains the XML setup files to use these new components in the DABC runtime environment however it may contain an additonal library defining the DABC Application class As an example we may consider a plug in package for reading out data from specific PCIe hardware like the Active Buffer Board ABB 4 This package is separately available for download at http dabc gsi de and described in detail in chapter 17 page 125 of the DABC programmer s manual There are principally two different ways to instal
40. file has no effect 94 DABC Programmer Manual Setup Chapter 14 DABC Programmer Manual Example MBS programmer prog exa mbs tex 14 1 Overview MBS Multi Branch System is the standard DAQ system of GSI Support of MBS in DABC includes several components type definitions for different MBS structures iterator classes for reading creating MBS event subevent data support of new Imd file format mbs ClientTransport for connecting to MBS servers mbs ServerTransport to emulate running MBS servers mbs CombinerModule for performing local mbs events building mbs GeneratorModule for generating random mbs events This plugin is part of the standard DABC distribution All sources can be found in the SDABCSYS plugin mbs directory All these sources are compiled into library 1ibDabcMbs so which is located in DABCSYS lib 14 2 Event iterators The MBS system has native event and subevent formats To access such event data several structures are intro duced in mbs LmdTypeDefs hand mbs MbsTypeDefs h In the first file structure mbs Header is defined which is just a container for arbitrary raw data Such container is used to store data to or to read data from 1md files resp In file mos MbsTypeDefs h the following structures are defined e mbs EventHeader MBS event header of 10 1 type e mbs SubeventHeader MBS subevent header of 10 1 type DABC operates with buffers of type mbt_ MbsEvents that contain several su
41. gt lt Module gt lt Context gt lt Context name Buildx gt lt Run gt lt lib value libDabcBnet so gt lt lib value libDabcMbs so gt lt lib value libBnetMbs so gt lt Run gt lt Application class bnet MbsWorker gt lt IsReceiver value true gt lt IsFilter value false gt lt EventBuffer value bufsize gt lt Application gt lt Context gt lt Defaults gt lt dabc gt With the same setup of the two MBS nodes as before we can run this example In the DABC control panel we only have to change the number of DABC servers 5 and the name of the setup file 44 DABC User Manual DABC Application MBS Chapter 7 DABC User Manual DABC Application Bnet user user app bnet tex 7 1 DABC eventbuilder network BNET The full functionality of DABC is shown in the case that the DAQ uses an event building network BNET transferring the partial data from n readout nodes to m event building nodes such that each event builder can work on the full detector data This scenario is discussed in detail in chapter 15 page 113 of the DABC programmer s manual Appropriate configuration files can be found at DABCSYS applications bnet test directory An example setup file SetupBnet xml may look like this lt xml version 1 0 gt lt dabc version 1 gt lt Context host localhost name Controller 41 gt lt Run gt lt runfunc value RunTestBnet gt
42. i e the generic PCIe 1 It implements the Transport factory method CreateTransport This will create a pci Transport and assign a dedicated working thread for each transport The DABC framework will use this method to establish the data connection of a Port with the PCI device 2 It defines a plug in point for an abstract board component The device functionalities may require driver implementations that are more board specific Because of this the mprace library provides base class Board with some virtual methods to work on the driver This is applied here as handle to the actual Board implementation e g a mprace ABB that must be instantiated in the constructor of the subclass Note that all functionalites require a real Board implementation thus it is not possible to instantiate a mere pci BoardDevice without subclassing it 125 126 DABC Programmer Manual Example PCI 3 It adds Device specific commands CommandSetPCIReadRegion and CommandSetPCIWriteRegion that define the regions in the PCI address space for reading or writing data resp Method ExecuteCommand Command is extended to handle such commands 4 It manages the scatter gather mapping of userspace dabc Buffer objects for the DMA engine These are taken from a regular DABC memory pool and are each mapped to a mprace DMA Buffer representation The DMA mapping is done in method MapDMA Buffer which gets the reference to the dabc Memory Pool that is used for the pe
43. is a test button button Tool tip whatever it does this check addCheckBox Data server on off check this graphlcon xSet getIcon icons windowgreen png addButton Display Display info graphlcon this state new xState ServerState xState XSIZE xState YSIZE stapan new xPanelGraphics new Dimension 160 50 1 one column of states metpan new xPanelGraphics new Dimension 410 14 1 one columns of meters franame new String MyGraphics fralayout xSet getLayout franame if fralayout null fralayout xSet createLayout franame new Point 400 400 new Dimension 100 75 1 true frame new xInternalCompound franame graphlIcon 0 fralayout xSet blueD Here we have the callback function for the interactive elements the text prompt the button the checker and the icon private void print String s System out println s public void actionPerformed ActionEvent e String cmd e getActionCommand if prompt equals cmd print cmd prompt getText check isSelected 148 DABC Programmer Manual GUI else if button equals cmd print cmd prompt getText check isSelected else if check equals cmd print Data server check isSelected if check isSelected if param null param setParameter 0 state redraw 0 Green Active true else if param null param setParamete
44. methods are provided Method ReadPCIStart dabc Buffer may start the asynchronous filling of one mapped Buffer from the configured PCI board addresses It should not wait for the completion of the data transfer but return immea diately without blocking after triggering the DMA In contrast to this ReadPCIComplete dabc Buffer must wait until the DMA transfer into the specified Buffer is completely finished So the pci Transport will initiate DMA by ReadPCIStart and check for DMA completion by ReadPCIComplete A subclass of pci BoardDevice may re implement these methods with board specific functionalities 6 Writing data to the board Method WritePCI dabc Buffer implements writing data from a DABC buffer to the PCI address space using the BAR the PCI start address and the write size as specified before These write parameters may be either set by method SetWriteBuffer unsigned int bar unsigned int address unsigned int length or by submitting the corresponding command CommandSetPCIWriteRegion to the pci Device If DMA mode defined in the constructor is not enabled this will just use PIO to transfer the specified dabc Buffer to the PCI addresses If DMA mode is enabled it will perform DMA from the user space dabc Buffer this must be taken from a memory pool that was mapped before by means of MapDMABuffer This call will initiate the DMA transfer and block until it is complete Currently there is no asynchronous implementatio
45. object dabc CmdCreateMemoryPool where all possible settings can be done via following static methods AddMem add configuration for specified buffer size AddRef add configuration for number of references and header sizes AddCfg set generic configuration like cleanup timeout or size limit For instance one can do the following bool UserApplication CreateAppModules dabc Command cmd new dabc CmdCreateMemoryPool WorkPool dabc CmdCreateMemoryPool AddMem cmd 8192 100 8K bufs dabc CmdCreateMemoryPool AddMem cmd 2048 500 2K bus dabc CmdCreateMemoryPool AddRef cmd 2048 64 refs with 64 B headers 11 2 Threads organization 75 dabc CmdCreateMemoryPool AddCfg cmd true set fixed layout dabc mgr gt Execute cmd All parameters configured for the command can be set up in the configuration file In this case one should just call dabc mgr gt CreateMemoryPool WorkPool 11 2 Threads organization Class dabc WorkingThread organizes a working loop and performs execution of runnable jobs represented by dabc WorkingProcessor class 11 2 1 Working loop The implementation of dabc WorkingThread is based on the pthreads library The main task of dabc WorkingThread is to wait for events using pthread_cond_wait function and then exe cute the event callback in the corresponding WorkingProcessor This functionality is implemented in dabc WorkingThrea
46. objects of xGUI all have implemented this interface Example xMeter xState xHisto Dimension getDimension int getID String getName JPanel getPanel Point getPosition void setActionListener ActionListener a void setID int id Set internal ID o void setSizeXY Sets the preferred size of item to internal vale o void setSizeXY Dimension d Sets the preferred size of item to specified dimension gt DE e 0 00 0 00 Example public void setActionListener ActionListener a action a public JPanel getPanel return this public String getName return sHead public void setID int i iID i public int getID return 11D public Point getPosition return new Point getX getY public Dimension getDimension return new Dimension ix iy public void setSizeXY setPreferredSize new Dimension ix iy public void setSizeXY Dimension dd setPreferredSize dd 18 7 4 Example Example of a minimalistic application panel Full running code in MiniPanel That is how the class must look ES MyPanel gt MyPanel My Command Defaultvalue This is a test button Data server on off _ Figure 18 1 Mini panel like public class MiniPanel extends xPanelPrompt implements xiUserPanel ActionListener The constructor must not have arguments Icon name and tooltip have to be passed by getter function to the caller the GUI desk
47. of DABC servers is one 6 1 3 1 Combined DABC and MBS controller buttons a Save panel settings see 4 3 page 30 BE Execute script dabcstartup sc at DABC master node Starts DIM servers Execute scriptprmstartup sc at MBS master node Starts prompter dispatchers and message loggers Waits for all components Sum of DIM servers are running A progress panel pops up during that time see 4 2 3 page 25 If all components are up trigger the main Update O Configure Execute user s MBS startup procedure in prompter Waits for all MBS Dag_rate tasks are run ning Executes state transition command Configure on DABC master node and wait for the transition All plug in components are created Then execute Enable If all components are up trigger the main Update Oo Start MBS acquisition wait for all acquisition states Running then execute DABC Start command All components go into running state Running oO Pause acquisition Execute MBS stop acquisition wait for all acquisition states Stopped Execute DABC Stop command All components go into standby state Ready oO Halt acquisition Executes DABC Halt command This closes all plug ins States go into Halted Execute 42 DABC User Manual DABC Application MBS user s MBS shutdown procedure in prompter Prompter dispatcher and message loggers should still be running Next must be shut down or configure After two seconds trigger the main Update tke Shut down all Execute EXIT comm
48. of name modulename for processing De pending on the Module type synchronous or asynchronous see section 9 2 1 this will start execution of the MainLoop or activate processing of the queued events belonging to this module resp void StopModule const char modulename Disables processing for the module of name modulename bool StartAllModules int appid 0 Enables processing for all modules with apllication identifier number ap pid The optional identifier may be set in the Module definition to select different kinds of modules here By default this method will start all exisiting modules on this node Returns true of false depending on success bool StopAllModules int appid 0 Disables processing for all modules with application identifier number ap pid The optional identifier may be set in the Module definition to select different kinds of modules here By default this method will stop all exisiting modules on this node Returns true of false depending on success bool DeleteModule const char modulename Deletes the module of name modulename Returns true of false depending on the deletion success 10 2 Framework interface 65 bool IsModuleRunning const char modulename Method returns true if module of name modulename is run ning i e its processing is enabled If module does not exist or is not active false is returned bool IsAnyModuleRunning Method returns false if no exisiting module is running an
49. of the configuration set up e g switching on debug output change details of processing parameters The current parameters may be stored back to the XML file Apart from the configuration the control system may use local parameter objects for Monitoring the components When monitoring parameters change the control system is updated by interface methods of the manager and may refresh the GUI representation Chapter 13 will explain the usage of parameters for configuration in detail 9 4 Package and library organisation The complete system consists of different packages Each package is represented by a subproject of the source code with own namespace There may be one or more shared libraries for each package Main packages are as follows 9 4 1 Core system The Core system package uses namespace dabc It defines all base classes and interfaces and implements basic functionalities for object organization memory management thread control and event communication Section 9 5 1 gives a brief overview of the Core system classes 9 4 2 Control and configuration system Depends on the Core system Defines functionality of state machine command transport parameter monitoring and modification Implements the connection of configuration parameters with a database i e a file in the trivial case Interface to the Core system is implemented by subclass of dabc Manager Note that default implementations of state machine and a configuratio
50. procedure typically startup scom located on user path MbsShutdown The user specific MBS shutdown command procedure typically shut down scom located on user path MbsCommand With RET an MBS command in executed on current node The shell script button executes this string as rsh command on master node MbsServers Number of nodes plus prompter This information is minimum for the GUI to know when all MBS nodes are up The GUI waits until this number of DIM servers is up and running That file can be created from within the GUI in the MBS controller panel Enter all values necessary and store them 5 1 2 1 MBS controller buttons a Save panel settings see 4 3 page 30 ES Execute script prmstartup sc at master node Starts prompter dispatchers and message loggers and waits until they are up Trigger the main Update A progress panel pops up during that time see 4 2 3 page 25 Execute script dimstartup sc at master node Starts dispatcher and message logger for single node MBS Trigger the main Update 8 Configure Execute user s MBS startup procedure in prompter dispatcher Wait for all m_daq_rate tasks are running Trigger the main Update Start acquisition Execute Start acquisition Wait for all acquisition states go into Running O Pause acquisition Execute Stop acquisition Wait for all acquisition states go into Stopped 5 1 MBS event building 33 oO Halt acquisition Execute user s MBS sh
51. programmer prog setup tex 13 1 Parameter class Configuration and status information of objects can be represented by the Parameter class Any object derived from WorkingProcessor class e g Application Device Module and Port can have a list of parameters assigned to it There are a number of WorkingProcessor methods to create parameter objects of different kinds and access their values These are shown in the following table Type Class Create Getter Setter string StrParameter CreateParStr GetParStr SetParStr double DoubleParameter CreateParDouble GetParDouble SetParDouble int IntParameter CreateParlnt GetParlnt SetParInt bool StrParameter CreateParBool GetParBool SetParBool The CreatePar methods will internally create a new Parameter of the specified name if it does not exist before For any type of parameter the GetParStr and SetParStr methods can be used which will deliver the parameter value as text string expression As one can see to represent a boolean value a string parameter is used If text of string is true in lower case the boolean value is recognized as true otherwise as false It is recommended to use these WorkingProcessor methods to create parameters and access their values but one can also use FindPar method to find any parameter object and use its methods directly 13 2 Use parameter for control One advantage of the DABC parameter objects is tha
52. saved to other file see 4 3 page 30 The file MbsControl xml can be created easily in the GUI itself by filling the input fields of the control panel and save lt xml version 1 0 encoding utf 8 gt bsLaunch gt bsMaster prompt MBS Master value node xx gt bsUserPath prompt MBS User path value myMbsDir gt bsSystemPath prompt MBS system path value mbs v51 gt bsStartup prompt MBS startup value startup scom gt bsShutdown prompt MBS shutdown value shutdown scom gt bsCommand prompt Script command value whatever command gt lt lt lt lt lt lt lt lt MbsServers prompt Number of needed DIM servers value 3 gt 31 32 DABC User Manual MBS GUI T MbsController OK MBS shut down Name server x90523 gsi de User name goofy Password RET Master node x86g 4 Servers 3 System path daq usr goofy mbswark v5 1 User path 50 x86 newmbs Startup startup scom Shutdown shutdown scom Command sho task Launch file MbsControl xml BER O000 OY Figure 5 1 MBS controller lt MbsLaunch gt MbsMaster Lynx node where the MBS prompter is started MbsUserPath MBS user working directory The GUI need not to have access to that filesystem MbsSystemPath Path on Lynx where the MBS is installed GUI needs no access to this path MbsStartup The user specific MBS startup command
53. several similar parameters for instance the same queue length for all output ports in the module One possible way is to use syntax as described above The disadvantage of such approach is that one must expand the XML file to set each queue length explicitely from the appropriate variable so in case of a big number of ports the file will be very long and confusing to the user Another possibility to set several parameters at once consists in wildcard rules using or symbols These can be defined in a lt Defaults gt node lt xml version 1 0 gt lt dabc version 1 gt lt Variables gt lt myname value Generator gt lt myport value 6010 gt lt Variables gt lt Context name Mgr myname gt lt Run gt lt lib value libDabcMbs so gt lt func value InitMbsGenerator gt lt Run gt lt Module name myname gt lt SubeventSize value 32 gt lt Port name Output gt lt MbsServerPort value myport gt lt Port gt lt Module gt lt Context gt lt Defaults gt lt Module name x gt 18 DABC User Manual Setup lt Port name 0utputx gt lt OutputQueueSize value 5 gt lt Port gt lt Module gt lt Defaults gt lt dabc gt In this example for all ports which names begin with the string Output and which belong to any module the output queue length will be 5 A wildcard rule of this form will be applied for all contexts of the configuration file i
54. so gt lt lib value DABCSYS lib libDabcAbb so gt lt lib value libBnetTest so gt lt Run gt lt Application class bnet TestWorker gt lt NumReadouts value 1 gt lt Input0Cfg value ABB gt 132 DABC Programmer Manual Example PCI lt Application gt lt Devices gt lt Device name ABBDevice gt lt ABB_BoardNumber value 0 gt lt ABB_ReadoutBAR value 1 gt lt ABB_ReadoutAddress value 8192 gt lt ABB_ReadoutLength value 16384 gt lt Device gt lt Devices gt lt Context gt Note that the ABB plugin library 1ibDabcAbb so must be loaded to instantiate the abb Factory and apply its classes on the node Chapter 18 DABC Programmer Manual GUI programmer prog gui tex 18 1 GUI Guide lines The DABC GUl is written in Java In the following we refer to it as a whole as xGUI It uses the DIM Java package to register the DIM services provided by the DABC DIM servers It is generic in that it builds most of the panels from the services available Thus it can control and monitor any system running DIM servers conforming to rules described in the following According the description above it does the following Get list of commands and parameters and create objects for each Put parameters in a table Put commands in a command tree e e e e Create graphics panels for rate meters states histograms and infos 18 2 DIM Usage DIM is a light weight communication
55. subevent header is done by iter rawdata method Finally FinishSubevent and FinishEvent will complete the subevent or event definition resp 14 3 File VO MBS uses the 1md List Mode Data file format Class mbs LmdFile provides a C interface for reading and writing such files To use mbs LmdFile as input or output transport of the module classes mbs LmdInput and mbs LmdOutput were developed resp In general to provide user specific input output capability over a port one should implement the complete dabc Transport interface which includes event handling queue organization and a complex initialization sequence All this is nec essary for cases like socket or InfiniBand transports but too complicated for simple cases as file I O Therefore a special kind of transport dabc DatalOTransport was developed which handles most of such complex tasks and reduces the implementation effort to the relatively simple dabe Datalnput and dabc DataOutput interfaces Class mbs LmdOutput inherits dabc DataOutput and allows to save MBS events contained in dabc Buffer objects into an x Ima file In addition to mbs LmdFile functionality it allows to create multiple files when a file size limit is exceeded The class has following parameters e MbsFileName name of Imd file including Imd extension 14 4 Socket classes 97 e MbsFileSizeLimit size limit in Mb of single file 0 no limit Class mbs LmdInput inh
56. this script can be found at scripts dabclogin sh e Edit the DABCSYS environment according to your local installation directory This is done in the following lines export DABCSYS opt dabc dabc_1_0 00 e Specify correct location of your JAVA installation This is done in the lines shown here an example make sure to get the path where the include directory is located export JAVA_HOME usr lib jvm e Copy the script to a location in your global PATH for later login e g usr bin Alterna tively you may set an alias to the full pathname of dabclogin sh in your shell profile 3 Execute the just modified login script in your shell to set the environment dabclogin sh This will set the environment for the compilation 4 Change to the DABC installation directory and start the build cd DABCSYS make This will compile the DABC framework and install a suitable version of DIM in a subdirectory of SDABCSYS dim 13 14 DABC User Manual Setup After succesful compilation the DABC framework installation is complete and can be used from any shell after invoking dabclogin sh The next sections 3 2 page 14 and 3 3 page 15 will describe further steps to set up the DABC working environment for each user 3 2 Set up the DABC environment Once the general DABC framework is installed on a system still each user must activate the environ ment and do further preparations to work with it 1 Execute the DABC login script in a linux sh
57. transport Supports formats and readout of GSP s standard DAQ system MBS Multi Branch System May also write data into MBS listmode format and may emulate MBS socket data servers Addition ally MBS systems can be controlled by the DABC GUI Chapter 2 DABC User Manual Introduction user user introduction tex 2 1 About DABC The Data Acquisition Backbone Core DABC is a Data Acquisition DAQ framework with modular com ponents for dataflow on multiple nodes It provides a C runtime environment with all basic services such as threads and event handling memory management command execution configuration logging and error handling User written DAQ applications can be run within this environment by means of a plug in mechanism DABC contains a sub framework with additional interfaces to set up distributed event builder networks As transport layers for such networks tcp ip and InfiniBand verbs are supported DABC supports by default the data formats and readout connections of GSI s standard DAQ system MBS Multi Branch System It may also write data files with the MBS x 1md format and it may emulate MBS data server sockets such as stream or transport servers The DABC control system features a finite state machine logic and parameters for monitoring and con figuration The current implementation is based on the DIM protocol 3 other implementations could replace this one A generic Java GUI is provided to operate this
58. with dabc di_ Repeat so the thread can continue its event loop and handle other workers For data output the user should just implement virtual method WriteBuffer In some cases user may redefine ProcessPoolChanged which is called when memory pool changes its layout new buffers were allocated or released It may be required for DMA operations where each buffer from a memory pool should be initialised once before it can be used for data transport It is not always necessary to create a user specific Device for a user written DataTransport since the standard LocalDevice can be used if it is only required as owner for the transport objects In this case the factory method Create Transport should be provided already in the user Factory see section 12 5 However some user implementations of DataTransport may require services of a corresponding Device though In this case the user should implement a Device that provides the factory method CreateTransport see section 12 3 2 This can instantiate the DataTransport with a back reference to the responsible Device 12 4 The DABC application 89 12 3 6 Input output objects Besides the Transports DABC provides an interface for implementing a simple input output by means of base classes dabc DataInput and dabc DataOutput The interface is similar to that of dabc Data Transport but these classes are not depending on any other components threads devices etc and therefore can
59. 0 100 100 columns 0 show false gt lt MbsLauncher shape 50 14 404 272 columns 0 show false gt lt DabcMbsLauncher shape 0 0 430 424 columns 0 show false gt lt ParameterSelect shape 300 0 271 326 columns 0 show true gt lt ParameterList shape 13 364 810 426 columns 1 show true gt lt windowLayout gt lt TableLayout gt lt Parameter width 74 74 74 74 74 74 74 74 gt lt TableLayout gt lt Layout gt 18 5 4 DABC launch panel values File DabcLaunch xml lt xml version 1 0 encoding utf 8 gt DabcLaunch gt DabcMaster prompt DABC Master value node xxxx de gt DabcName prompt DABC Name value Controller 41 gt DabcUserPath prompt DABC user path value myWorkDir gt DabcSystemPath prompt DABC system path value dabc gt DabcSetup prompt DABC setup file value SetupDabc xml gt DabcScript prompt DABC Script value ps gt DabcServers prompt Number of needed DIM servers value 5 gt lt DabcLaunch gt lt lt lt lt lt lt lt lt 18 5 5 MBS launch panel values File MosLaunch xml lt xml version 1 0 encoding utf 8 gt bsLaunch gt bsMaster prompt MBS Master value node xx gt bsUserPath prompt MBS User path value myMbsDir gt bsSystemPath prompt MBS system path value mbs v51 gt bsScript prompt MBS Script value script remote_exe sc gt bsCommand prompt Script command valu
60. 0001000b define LMD__INDEX 1 define LMD__OVERWRITE 1 define LMD__LARGE_FILE 1 define LMD__BUFFER 1 define LMD__NO_INDEX 0 define LMD__NO_OVERWRITE 0 define LMD__NO_LARGE_FILE 0 define LMD__NO_BUFFER 0 define LMD__NO_VERBOSE 0 define LMD__VERBOSE 1 define LMD__ENDIAN_BIG 2 define LMD__ENDIAN_LITTLE 1 14 9 MBS upgrade for DABC 105 define LMD__ENDIAN_UNKNOWN 0 14 9 5 MBS update for DIM control 14 9 5 1 New or modified files New files f_dim_server c f_dim_server h all DIM functions dimstartup sc dimshutdown sc for single node prmstartup sc prmshutdown sc dimremote_exe sc for multi node propagate DIM DNS NODE m_launch c fork programs to appear without path in ps output m_cmd2xml c Generate xml command description file from output of show command Modified alias com add launch alias m_prompt c m_dispatch c m_msg_log c f_ifa c f_ifa h f_mg_msg_output c f_mg_msg_thread c in mbs v51 has argument which is not specified in call f_pr_reset c kill processes in defined order f_stccomm c Socket created in stc_createserver will be shut down and closed by stc_close m_wait_for c Dont wait for zombies remote_exe sc use m_launch to run program Makefile modules using DIM must include DIM path all programs must link DIM library 14 9 5 2 f stccomm On the server side stc_createserver fills structure s_tcpcomm with socket number s_tcpcomm sock_rw stc
61. 25 17 22 AIN 126 17 3 Active Buffer Board implementation o o eee 127 1731 SEEN a osa aaa aa Ad a ale ine 127 17 32 abboReadoutNiddple oon ee nennen a a 128 17 33 abboWitterModule cs 2 4 6 eae ci a E a de eee ee 128 17294 MEER 6 2a ha ace do Parka bared Bae PRE oe bared dark eS 128 174 Simple read and write tests s coco e eee E 128 174 1 DMA Read from the board 2 2 2 2 0 2 002 eee ee ee eee 129 17 4 2 DMA Write tothe board 0 CC none 130 17 4 3 Simultaneous DMA Read and Write 0 0 0 200000 0 eee eee 130 17 5 Active Buffer Board with Bnet application o o 131 18 DABC Programmer Manual GUI 133 18 1 KAT Cre les ee kee en A te eh di ewe ee a een 133 182 DIN WAR Soh od eat a es Gad See Soe a a 133 18 2 1 DABC DIM naming conventions 2 020 eee eee 133 18 22 DABLE DIM records o cia ee ES ale a a hae Wk a eR ear ae cn So 134 18221 Record Dah Plain o 20 404 race 045 eee eS en aa 134 18 2 2 2 Record ID 1 Generic self describing gt o o 135 182253 Beco ID FRE a Ban Ba A e a da 135 19 224 Recor Der Rate od a ee ra a hen 135 18223 Record ID 4 Histosrai o cc ea be 2 ua ne a Ba a ea 135 18226 Record IDS 1 Inte ce 0 8 aa Rw Aw ee ew Gee ne F 135 18 2 2 7 Record ID 9 Command descriptor o o e e 135 19222 Mo A 136 18229 Bi Pam u a A ee A 136 18 223 Applietensewens osos rra a aaa aa a a Be 136 CONTENTS xi
62. 7 5 Active Buffer Board with Bnet application The DAQ builder network Bnet example as described in section 15 8 may optionally utilize the ABB as input for the Readout module This is provided in class bnet Test WorkerApplication which implements the bnet WorkerApplication The Bnet factory method bool bnet TestWorkerApplication CreateReadout const char portname int portnumber will instantiate an abb Device if the configuration parameter for the portnumber p InputpCfg as delivered by ReadoutPar p is set to ABB This abb Device is connected directly to the input port of the standard Bnet combiner module as specified by the portname argument of the method if ReadoutPar portnumber ABB const char abbdevname ABBDevice fABBActive dabc mgr gt CreateDevice abb Device abbdevname res dabc mgr gt CreateTransport portname abbdevname if res EOUT Cannot create ABB transport Note that the abb ReadoutModule is not used here this is applied for the simple examples only see section 17 4 Any other value of ReadoutPar p will apply the TestGeneratorModule of the standard Bnet example The parameters for the ABB can be set in the XML configuration file using the names as defined in abb Factory h This may look as follows lt Context host node01 name Worker2 42 gt lt Run gt lt lib value DABCSYS lib libpcidriver so gt lt lib value S DABCSYS lib libmprace
63. A AA de 36 HS Working direcciones qes dira Rae aR Da ee eR eee Bde 36 5 31 MBS configuration of DIM 36 6 DABC User Manual DABC Application MBS 39 6 1 MBS event building with DABC 2 CL none 39 6 1 1 MES SWP ceras a A a ee ba ta dhe aod a 39 6 1 2 DABC SER 0 our ee ea GS A E E e E aA a a G E A 39 6 1 3 Combined DABC and MBS control panel oaa a 41 6 1 3 1 Combined DABC and MBS controller buttons 41 6 2 MBS and DABC with Bnet e 42 7 DABC User Manual DABC Application Bnet 45 Td DABC eventbuilder network BNET 2 2222 Emo nennen 45 7 2 DABC eventbuilder network BNET with MBS o 46 8 DABC User Manual DABC Application ROC 47 BRIL DABC as MES data server oo esce er ri 47 Ba BOSSE WUNDE ooe oea ca ba OHA de ee de e a ran aaa Ee OS ee 48 vi CONTENTS II Programmer Manual 49 9 DABC Programmer Manual Overview 51 A a oo 4 a oda ee ed waren wha GA ba ded Barack He 51 9 2 Role and functionality of the objects o o e 51 92 1 Modules 5 e RARE RHA dL ee eek 51 9 2 1 1 Class dabc ModuleSyne 2 aaa 51 9 2 1 2 Class dabe ModuleAsync 1 1 52 92 2 FOND 646 483 rs ba ha ee do ede bade PEGS oe Re Carga es 52 9 2 3 a oi Oo hs ee ve SO ee Oe ee Be eel eae oo Se ee 52 9 2 4 Manager ooe e ko A AN 52 9 23 Memory and butlers o ei ana ae Ae ee ee 52 9 2 6 A wh aan aes se Ap a digs E a Te eh Be gas A a ee 53 9 2 7 VAIS PORE 2
64. A unique name inside DABC of the system DabcUserPath User working directory The GUI does not need to have access to the filesystem DabcSystemPath Path where the DABC is installed DabcSetup Setup file name DabcScript Command to be executed in an ssh at the master node DabcServers Number of workers and controllers This information is minimum for the GUI to know when all DABC nodes are up The GUI waits until this number of DIM servers is up and running Note that this number must be consistent with the DABC setup file used The name server name is translated from shell environment variable DIM_DNS_ NODE the user name from shell environment variable USER Password can be chosen when the first remote shell script is executed which itself is protected by user password All following commands then need this password 4 2 2 1 DABC controller buttons E Save panel settings to the file Control file If you choose a name different from the default you must set a shell variable to it to get the values from that file see 4 3 page 30 BE Startup all tasks Executes a DABC script dabcstartup sc via ssh on the master node under user name Then it waits until the number of DIM servers expected are announced A progress panel pops up during that time see 4 2 3 page 25 When the servers are up the main GUI Update is triggered building all panels from scratch according the parameters offered by the servers Configure Executes state transi
65. BC Programmer Manual Overview e dimc Manager Provides DIM 3 as transport layer for controlling commands Additionally pa rameters may be registered and updated automatically as DIM services There is a general purpose Java GUI for this implementation 3 It provides interfaces for user specific plug ins that define the actual set up several dabc Factory objects to create objects and one dabc Application object to define the state machine transition actions dabc Factory Factory plug in for creation of applications modules devices transports and threads dabc Application Defines user actions on transitions of the finite state machine of the manager Good place for export of application wide configuration parameters May define additional commands 9 5 2 BNET classes The classes of the Bnet package providing functionalities of the event builder network IbsWorkerApplication Pit inheritance el bnet MbsFilterModule SA association Figure 9 4 Simplified UML diagram of the Bnet classes green DABC base classes dark green Bnet interface classes magenta Bnet implementation classes yellow MBS implementation example See text for details bnet ClusterApplication Subclass of dabc Application to run on the cluster controller node of the build
66. BC with MBS At the moment the MBS transport and stream servers are supported Client transport has following parameters Name Type Dfit Description MbsServerKind str Transport kind of mbs server Transport or Stream MbsServerName str localhost host name where mbs server runs MbsServerPort int 6000 server port number for socket connection 14 7 Event generator 99 To create client connection the following piece of code should be used dabc Command cmd new dabc CmdCreateTransport Combiner Inputo mbs typeClientTransport MbsTransThrd cmd gt Set Str mbs xmlServerKind mbs ServerKindToStr mbs StreamServer cmd gt Set Str mbs xmlServerName 1xi010 gsi de cmd gt Set Int mbs xmlServerPort mbs DefaultServerPort mbs StreamServer 5 dabc mgr gt Execute cmd 14 7 Event generator Class mbs GeneratorModule is an example of a simple module which just fills buffers with random MBS events and provides them to the output port Schematically the implementation of this module looks like this include dabc ModuleAsync h class GeneratorModule public dabc ModuleAsync protected dabc PoolHandle fPool dabc BufferSize t fBufferSize public GeneratorModule const char name dabc Command cmd 0 dabc ModuleAsync name cmd fBufferSize GetCfgInt dabc xmlBufferSize 16384 cmd fPool CreatePoolHandle Pool fBufferSize 10 Create
67. Command completion is sent back Prompter may terminate start again and connect The master logger starts a server in a thread waiting on port 6005 for connections of message logger slaves For each slave a new thread is started waiting for messages of the slave These threads are protected by mutex Only one thread can write into logfile However slaves do not write logfile If prompter was started from remote master message logger starts server on port 6007 waiting for remote message client to connect If remote node is specified argv TCP Slave DIM remote none 0 0 task 0 0 master 1 0 pth_server pth_links dim 0 0 msg status pth_dim_serv masterdim 1 0 msg status pth_dim_serv pth_server pth_links masternode 1 1 connect masternode masterdim any 1 0 msg status pth_dim_serv pth_server pth_links masternode slavedim 1 1 status pth_dim_serv connect masternode master remotenode 1 0 msg status pth_dim_serv pth_rem_serv pth_server pth_links Table 14 2 MBS Message logger modes pth_rem_serv thread waits for connection of a message client After connection global 1_tcp_chan_rem is set and _mg_msg_output send messages to that socket As TCP master the pth_server thread waits for connections of message slaves After connection starts pth_links thread to read messages and process in f_mg_msg_output mutex locked As TCP slave connect to lt masternode gt set global 1_tc
68. CommandClient with true as constructor argument at the end a list of all commands will be available via ReplyedCmds method dabc CommandClient cli true bool res cli WaitCommands DOUT1 One has u commands in replyed queue cli ReplyedCmds Size One more use case of the command client interface is the dabc CommandsSet This class inherits from dabc CommandClientBase the abstract base class for all commands clients It useful if execution of a master command should cause the execution of several other commands For instance when execution of a command in one module should be distributed to two other modules one should do int UserModule ExecuteCommand dabc Command cmd if cmd gt IsName MasterCommand 11 3 Command execution 79 dabc CommandsSetx set new dabc CommandsSet cmd dabc Modulex ml dabc mgr gt FindModule Modulel m1 gt Submit set gt Assign new dabc Command UserCommand1 dabc Modulex m2 dabc mgr gt FindModule Module2 m2 gt Submit set gt Assign new dabc Command UserCommand2 dabc CommandsSet Completed set 10 return dabc cmd_postponed return dabc ModuleAsync ExecuteCommand cmd Here one creates a CommandsSet for a master command and submits two slave commands via the command client argument to two other modules Method dabc CommandsSet Completed is used to inform th
69. Core system package defines all base classes and interfaces and implements basic functionalities for object organization memory management thread control and event communication 2 4 2 Control and configuration system Depends on the Core system Defines functionality of state machine command transport parameter monitoring and modification Implements the connection of configuration parameters with a database i e a file in the trivial case Interface to the Core system is implemented by subclass of Manager Note that default implementations of state machine and a configuration file parser are already provided by the Core system 2 4 3 Plug in packages Plug in packages may provide special implementations of the core interface classes Device Transport Module or Application Usually these classes are made available to the system by means of a corresponding Factory that is automatically registered in the Manager when loading the plug in library 2 4 Package and library organisation 11 User Application User Plug ins 01 U09 Cc y O O c EN le gt j e SE Figure 2 3 Schematic view of the distributed DABC components coloured and user specific extensions white When installed centrally the Plugin packages are kept in subfolders of the SDABCSYS plugins directory Alternatively the Plugin packages may be installed in a user directory and linked against the Core system insta
70. DABC setup files can be found in the sections 6 1 page 39 7 1 page 45 and 7 2 page 46 of this manual 3 3 2 Basic syntax A DABC configuration file should always contain lt dabc gt as root node Inside the lt dabc gt node one or several lt Context gt nodes should exists Each lt Context gt node represents the application context which runs as independent executable Optionally the lt dabc gt node can have lt Variables gt and lt Defaults gt nodes which are described further in the following sections 3 3 5 page 16 and 3 3 6 page 17 3 3 3 Context A lt Context gt node can have two optional attributes host host name where executable should run default is localhost name application manager default is the host name Inside a lt Context gt node configuration parameters for modules devices memory pools are contained In the example file one sees several parameters for the output port of the generator module 16 DABC User Manual Setup 3 3 4 Run arguments Usually a lt Context gt node has a lt Run gt subnode where the user may define different parameters relevant for running the DABC executable lib name of a library which should be loaded Several libraries can be specified func name of a function which should be called to create modules This is an alternative to instantiating a subclass of dabc Application compare section 12 4 page 89 runfunc function name to run some sequence of operation
71. Data Acquisition Backbone Core User Manual Programmer Manual J Adamczewski Musch S Linev H G Essel GSI Darmstadt Experiment Electronics Department Produced August 17 2009 Revisions Document Date Editor Revision Comment DABC man 2009 03 10 Hans G Essel 1 0 1 First release il Contents I User Manual 1 1 DABC User Manual Overview 3 1 1 Outline Of this manual E a a a a a E a G 3 12 Release NOGE soui een re ne en 3 1 2 1 Version 1 0 01 10 March 2009 ooo ra dasu a 3 1 2 2 Version 1 0 00 26 February 2009 2 2 0 200 Swen 4 2 DABC User Manual Introduction 5 21 ADM DSB a e ee at re de a 5 Se a oe ee e i ee ee dee BE 5 2 2 1 Dialer o ar A eS A a ee ed ea SS 5 Ze Synchronous Mod l s sos 2 nash oy ar Baas Sake Hae eo ead er A 6 22 12 Asyochronousnisdile orcas otras o 584 ea Ba ba ba eos a 6 2 2 2 A ie ee eee OE ad De i OD EE a ee ee ee ee 6 2 23 PALAIS a ee ee Ek ie S 7 2 2 4 NAT ee a A A 7 2 Memory and butlers o oa We ed OE Oe ee ee 7 2 20 e tk eG ee a ne de ee eae 7 2 2 7 Transparent we at 8 2 2 8 DREIER 2 heli aea a he hed Cee ee babel ew eke HE ae ole oO 8 2 2 9 ag lt i 00 dba alae a Ba elk Med bale ae oko 8 23 Controlsand configuration s s e soe cse nso eca ee ee e ee 8 2 3 1 Pinte state machine ee ee 8 P as C OOE a ee ee ee ee ee 10 2edd Configuration and momitormg lt s o gt ea ee a a a a 10 2 4 Package and library organisat
72. EventTrend Events s as trend MSG EvSizeRateB Event size sample in bytes MSG EvSizeTrendB Event size sample in bytes MSG StreamRateKb Stream server Kbyte s MSG StreamTrendKb Stream server Kbyte s as trend MSG FileFilled File filled in percent MSG StreamsFull Number of full streams in percent MSG TriggerRate Trigger s of readout tasks MSG TriggernnRate nn 01 15 Trigger s type nn of readout tasks 5 2 3 MBS histograms Shown in histo window MSG TrigCountHis Histogram with 16 channels for counts of trigger types 0 total as seen by the readout task MSG TrigRateHis Histogram with 16 channels for count rates of trigger types 0 total as seen by the readout task 5 2 4 MBS infos Shown in info window MSG eFile Name of file MSG ePerform Events MBytes Events s and MBytes s MSG eSetup Name of setup file loaded PRM Current Current command execution node master node only PRM NodeList List of nodes master node only 5 2 5 MBS tasks Task list is shown in info window name slightly different Dispatch Msg_Log Read_Meb Collector Transport Event_Serv Util Read_Cam Esone_Serv Stream_Serv Histogram Prompt Rate SMI Sender Receiver Asynch_Receiver Rising Time_Order Vme_Serv 5 2 6 MBS text MSG GuiNode Node where GUI runs MSG Date Date as written in file header MSG Run Run ID as written in file header MSG Experiment Experiment as written in file header MSG User Lynx user name as written in file header 36 DABC User
73. EventsCombine 1 cmd For the moment bnet MbsCombinerModule skips an event if it is not present on all local data inputs Class bnet MbsBuilderModule builds MBS events from the buffers as delivered by the receiver module It also takes application parameter CfgEventsCombine to tell how many real MBS events are contained in the input buffers Application class bnet Mbs WorkerApplication implements several methods to correctly instantiate combiner and builder modules It also implements virtual method CreateReadout where the input transport for the combiner module is created In case of MBS there are three possibilities 1 when IsGenerator true module mbs GeneratorModule connected as data input 2 when the appropriate readout parameter like InputOCfg for the first input is a filename with Imd suffix the specified file will be used as data input 3 otherwise the value of readout parameter like InputOCfg will be used as MBS server name for connecting of mbs ClientTransport to the appropriate data input In virtual method CreateStorage an output 1md file will be created if parameter StoragePar value is not empty Example file SDABCSYS applications bnet mbs SetupBnetMbs xml shows the configuration for an MBS event building with 2 readout nodes connected with 2 generators each and 2 event builder nodes This configuration file can be customised via lt Variables gt definitions in the beginning lt xml version 1 0 gt
74. Get reference to list of parameters from browser 2 Set in all parameters the table index to 1 infoHandlers will no longer update table fields 3 Scan through all parameters and check if any quality is still 1 which would mean that the type is undefined That is repeated two times with 2 seconds delay to give the DIM servers the chance to update all parameters If still any quality is 1 this is an error Restore record attributes of meters and histograms from XML file cleanup graphics panels Create new table Add parameters to table by calling function xDimParameter addRow This function also creates graphi cal presentations of the parameters e g xMeter and add them to the appropriate graphics panels e g IANA 142 DABC Programmer Manual GUI xPanelMeter if needed Builds list of command descriptors xXmlParser 9 Add table to its panel 10 updateAll graphics panels co 18 6 2 2 xPanelCommand Extends JPanel It has references to the browser It owns the command tree JTree In the constructor the following steps are performed 1 Get reference to list of commands from browser 2 Create from that list a command tree to be shown on left side in window 3 Create arguments panel for the right side When a command is selected and an XML descriptor is available the arguments are shown as prompter panel 4 Call back functions for command execution Function setCommandDescriptors is called from
75. It is allowed to define a variable as a combination of text with another variable but neither arithmetic nor string operations are supported Using variables one can modify the example in the following way lt xml version 1 0 gt lt dabc version 1 gt lt Variables gt lt myname value Generator gt lt myport value 6010 gt lt Variables gt lt Context name Mgr myname gt lt Run gt lt lib value libDabcMbs so gt lt func value InitMbsGenerator gt lt Run gt lt Module name myname gt 3 3 DABC setup file 17 lt SubeventSize value 32 gt lt Port name Output gt lt OutputQueueSize value 5 gt lt MbsServerPort value myport gt lt Port gt lt Module gt lt Context gt lt dabc gt Here context name and module name are set viamyname variable and mbs server socket port is set via myport variable There are several variables which are predefined by the configuration system e DABCSYS top directory of DABC installation DABCUSERDIR user specified directory DABCWORKDIR current working directory DABCNUMNODES number of lt Context gt nodes in configuration files DABCNODEID sequence number of current lt Context gt node in configuration file Any shell environment variable is also available as variable in the configuration file to set parameter values 3 3 6 Default values There are situations when one needs to set the same value to
76. Manual MBS GUI MSG Platform CPU platform 5 2 7 MBS numbers MSG BufferSize MSG Buffers collected so far MSG Events collected so far MSG FileMbytes written in file MSG FlushTime MSG MBytes collected so far MSG StreamKeep MSG StreamMbytes MSG StreamScale MSG StreamSync MSG UserVal_nn nn 00 15 These values can be set in the user readout function MSG TriggernnCount nn 01 15 Trigger counts type nn of readout tasks 5 3 Working directories 5 3 1 MBS configuration of DIM Optional text file dimsetup in the MBS working directory specifies which rate meters histograms or states shall appear in the GUI Upper limits of the rate meters can be specified This file can be copied from MBSROOT set dimsetup Only the parameters which are in this file are optional Note that a file name of an open Imd file is only displayed when either FileOpen or FileFilled is selected for this node This file controls the rate meter and state appearance File name must be dimsetup and in the MBS working directory The value numbers are the maximum values for rate meters Colons only if value is specified Node names must be uppercase wildcards all All nodes Rates x EventRate 10000 fx EventTrend 10000 DataRateKb 16000 DataTrendKb 16000 StreamRateKb 16000 StreamTrendKb 16000 EvSizeRateB T28 fx
77. NE 1 cmd gt Set Int ABB_COMPAR_OLENGTH 10 cmd gt SetStr ABB_COMPAR_POOL ABB standalone pool cmd gt SetStr ABB PAR DEVICE devname c_str res manager Execute cmd DOUT1 Create ABB writer module s DBOOL res Again the parameter names are expressd by common string definitions as set in abb Factory h 4 The transport connection between the output port of the writer module and the abb Device is established by a direct method call of the manager res manager CreateTransport ABB Sender Output devname c_str DOUT1 Connected module to ABB device s DBOOL res 5 The writer module s processing is started with a manager method manager StartModule ABB Sender The main process waits 5 seconds while the DABC threads and the board DMA perform the data transfer The module is stopped again then After the module has stopped its internal dabc Ratemeter will print some average data rate values to the terminal Finally the manager is cleaning up all objects and the program terminates cmd gt SetInt cmd gt SetInt T anana 17 4 3 Simultaneous DMA Read and Write The example code abb_test cxx shows in a simple main function how to utilize the abb classes to write data from the PC to the ABB in one DMA channel and simultaneously read data back from the board with another DMA channel It applies the abb Device both with a abb WriterModule and a abb ReadoutMod
78. Output Output fPool 5 virtual void ProcessOutputEvent dabc Portx port dabc Bufferx buf fPool gt TakeBuffer fBufferSize FillRandomBuffer buf port gt Send buf y In the module constructor a pool handle is created declaring a required memory pool with 10 buffers of defined size fBufferSize The buffer size is taken from a configuration parameter with name dabc xmlBufferSize this string constant is predefined as BufferSize When the output port is created this pool handle and a default queue size is specified The only virtual method implemented for generator module is ProcessOutputEvent This function is called every time when a free buffer slot appears in the port output queue Thus when the module starts processing this call will be immediately executed N times size of output queue here 5 because there are N empty entries in the queue The only action here is to take a new buffer from the memory pool fill it with random events and send it to output port The actual mbs GeneratorModule is part of the 1ibDabcMbs so library and has following parameters 100 DABC Programmer Manual Example MBS Name Type Dfit Description NumSubevents int 2 number of subevents in generated event FirstProcld int 0 value of procid field of first subevent SubeventSize int 32 size of rawdata in subevent Go4Random bool true is raw data filled with random value BufferSize int 16384 server port number fo
79. Q application in different execution states This may be a subclass of specific existing application Additionally they may provide experiment specific Module classes When installed centrally the Application packages are kept in subfolders of the SDABCSYS applications directory Alternatively an Application package may be installed in a user directory and linked against the Core system installation and the required Plugin packages 2 4 5 Distribution contents The DABC distribution contains the following packages Core system This is plain C code and independent of any external framework Bnet plugin Depends on the core system only Transport plugins Network transport for tcp ip sockets and InfiniBand verbs Additionally transports for GSI Multi Branch System MBS connections socket filesystem is provided Optionally exam ple transport packages may be installed that illustrate the readout of a PCIe board or data taking via UDP from an external readout controller ROC board Control and configuration system The general implementation is depending on the DIM framework only DIM is used as main transport layer for commands and parameter monitoring On top of DIM a generic record format for parameters is defined Each registered command exports a self describing command descriptor parameter as DIM service Configuration parameters are set from XML setup files and are available as DIM services GUI A generic controls GUI using the
80. RETURN executes the command Only the MBS commands of the running tasks are shown Fig 5 3 page 34 shows that only dispatcher and prompter are up and therefore only their commands are seen Fig 5 4 page 34 shows in addition the commands of util and transport after configuration 34 DABC User Manual MBS GUI teo Dispatch Msg_log 4 Prompt Msg_log Dispatch a i E E E fa E m u iz f Commands Command ConnectDispatcher scope gt CI MbsDispatch nodel C OPT o CI MbsPrompt all II Figure 5 3 Info and command panel vee 7 Dispatch Msg_log Util Rirec Transport Daq_rate 4 Prompt Msg_log Dispatch Util Rirec Transport Daq_rate i tet pel bel fel Commands Command CloseFile scope MBS gt C MbsDispatch o 3 MbsPrompt tape L OPT gt 7 MbsRirec gt C MbsTransport gt 3 Mbsutil Figure 5 4 Info and command panel 5 2 MBS DIM parameters 5 2 1 MBS states Acquisition State Running Stopped BuildingMode State Delayed Immediate EventBuilding State Working Suspended FileOpen State File open File closed RunMode State DABC connected MBS to DABC I Transport client MBS standalone SpillOn State Spill ON Spill OFF 5 2 MBS DIM parameters 35 TriggerMode State Master Slave 5 2 2 MBS rates MSG DataRateKb KByte s MSG DataTrendKb KBytes s as trend MSG EventRate Events s MSG
81. _GUI LAYOUT Layout of frames Saved by main save button Default Layout xml 18 5 1 Record attributes File Records xml lt xml version 1 0 encoding utf 8 gt lt Record gt lt Meter name DABC X86 7 MSG DataRateKb visible true mode 0 auto false log false low 00000000 0 up 00016000 0 color Red gt lt Record gt 18 5 2 Parameter filter File Selection xml lt xml version 1 0 encoding utf 8 gt lt Selection gt lt Full contains Date filter false gt lt Node contains X86 7 filter false gt lt Application contains MSG filter false gt lt Name contains x filter false gt lt Records Only true Rates true States false Infos false gt lt Selection gt 140 DABC Programmer Manual GUI 18 5 3 Windows layout File Layout xml lt xml version 1 0 encoding utf 8 gt lt Layout gt lt WindowLayout gt lt Main shape 357 53 857 953 columns 0 show true gt lt Command shape 0 230 650 200 columns 0 show false gt lt Parameter shape 20 259 578 386 columns 0 show false gt lt Logger shape 0 650 680 150 columns 0 show false gt lt Meter shape 463 13 413 236 columns 4 show false gt lt State shape 85 504 313 206 columns 2 show false gt lt Info shape 521 482 613 217 columns 1 show false gt lt Histogram shape 124 508 613 206 columns 3 show false gt lt DabcLauncher shape 0
82. _acceptclient returnes socket number of connection This socket must be closed by stc_discclient socket stc_close shuts down and closes socket s_tcpcomm socket not s_tcpcomm sock_rw Therefore a server side st c_close did not close the server socket This has been changed in that s_t cpcomm socket is now equal s_tcpcomm sock_rw When a server called st c_close no more accept is possible as opposed to the current behavior On the client side a stc_connect server returns the socket as well as setting s_tcpcomm socket Therefore in this case stc_close was always shutting down and closing the socket whereas stc_discclient socket closes the socket 14 9 5 3 MBS launcher launch lt program gt lt program path gt lt program args gt If no program path is given MBSROOT bin directory is assumed Note that in the launched program environment PWD is the path from where m_launch was called whereas the current path is the one of the program Therefore program must chdir getenv PWD to work on the expected directory 14 9 5 4 MBS DIM commands and parameters The parameters and commands follow the DABC naming conventions The DIM command MbsCommand is generic The argument string is any MBS command All scom files are provided as DIM commands New 106 DABC Programmer Manual Example MBS program cmd2xml generates an xml file with the DABC formatted description of all MBS commands 14 9 5 5 DIM control modes MBS can be con
83. a central singleton to access framework functionalities the dabc Manager is also the interface base class for the control and configuration system that is applied with DABC 10 3 2 1 Virtual methods The dabc Manager defines several virtual methods concerning the finite state machine the registration and sub scription of parameters the command communication in between nodes and the management of a DAQ cluster resp These methods have to be implemented for differrent kinds of control systems in an appropriate subclass and are described as follows 10 3 Control system plug in 67 Manager const char managername bool usecurrentprocess Configuration cfg The constructor of the sub class The recommended parameters are passed from the manager factory see section 10 3 1 to the base class constructor such as the object name of the manager optionally a flag indicating to use either the main process or another thread for manager command execution and an optional configuration object cfg 1 The constructor should initialize the control system implementation 2 If the default state machine module of the DABC core is used the constructor should invoke method InitSMmodule Otherwise the constructor must initialize an external state machine of the control system following the state and transition names defined as static constants in dabc Manager h 3 The constructor must call method init to initialize the base functionalities a
84. a configuration to convert raw data into calibrated data without running any real DAQ lt xml version 1 0 gt lt dabc version 1 gt lt Context name Calibr gt 16 8 Running the ROC application 123 lt Run gt lt lib value libDabcMbs so gt lt lib value libDabcKnut so gt lt logfile value Calibr log gt lt Run gt lt Application class roc Readout gt lt DoCalibr value 2 gt NumRocs value 3 gt BufferSize value 65536 gt NumBuffers value 100 gt RawFile value d cbm06 cbmdata SEP08 raw run028 run028 1md gt bsServerKind value Stream gt CalibrFile value testcal lmd gt bsFileSizeLimit value 110 gt lt Application gt lt Context gt lt dabc gt AN AAN A N AA Here the raw data is read from the files matching the name wildcard pattern as defined in the lt RawFile gt tag Note that this example will read subsequently all data of run 028 which possibly was saved into several files with subsequent numbers appended to their names due to the lt MbsFileSizeLimit gt mechanism as described above The calibrated data is written as usual to the output file as specified in the lt CalibrFile gt tag 124 DABC Programmer Manual Example ROC Chapter 17 DABC Programmer Manual Example PCI programmer prog exa pci tex 17 1 Overview Reading data streams from a PCI board into the PC is a common use case for
85. also sets the transport synchronous mode In this mode transport processes streams only if a client is connected 14 9 3 New LMD file format With DABC as event builder for MBS there is no need to write files in MBS This gives more freedom to design a new file format This format will be written by DABC and read by fLmd functions get event The format is quite simpler than the old one because it has no buffer structure causing so much complications by event spanning The data elements itself mainly the events remain unchanged A file has a file header as before but with a fixed size part and a variable part size iUsedWords Behind the header follow data elements with sMbsHeader headers length type subtype allowing to identify and process or skip them Elements must be sized in 4 byte units Besides event data time stamps may be inserted from the original MBS formatted buffers to preserve this time information Writing reading such a file is very straight forward The file header contains the number of data elements iElements and the maximum size of elements iMaxWords This information is collected throughout the file writing and written on close into the file header The file header is an sMbsFileHeader structure The file size is no longer restricted to 2GB Optionally an element index is written at the end of the file This allows for random access of elements in the file through this index table The table itself has 32 bit value
86. ameter to the new value by means of an InvokeChange call Chapter 11 DABC Programmer Manual Services programmer prog services tex 11 1 Memory management TakeBuffer qe ass yo Penden ee 1 I I I I I 1 I I a ay E MakeReference Figure 11 1 Schema of DABC MemoryPool with blocks and subblocks Buffers with segments and Pointer object A PoolHandle is used to access the pool from within a Module See text for details 11 1 1 Zero copy approach The DABC framework is based on a dataflow concept Data buffers are flowing through many components like Modules Transports and Devices If it was required to copy the data content in each step of such transfer chain 71 72 DABC Programmer Manual Services this would reduce performance drastically Therefore DABC has a central memory management that provides global memory Buffers from a Memory Pool All components use just references to this memory these can be passed further without copying the content This technique is called zero copy approach and is fully supported by DABC 11 1 2 Memory pool The memory in dabc MemoryPool is organized in big blocks of contiguous virtual memory Each block is divided into memory pieces of the same size the subblocks the size of each subblock is defined as a power of 2 e g 4096 bytes The MemoryPool can have several memory blocks with different subblock sizes Usually a Mem
87. and on all DABC nodes Execute script prmshutdown sc at MBS master node After two seconds trigger the main Update 6 MBS Show acquisition Output in log panel M A Shell script for MBS master node D A Shell script for DABC master node 6 2 MBS and DABC with Bnet The following example configuration file SDABCSYS applications bnet mbs SetupBnetMbs xml shows how to configure two DABC nodes reading from two MBS transport servers and two event builder nodes Another node is used as controller The example setup file shows two techniques first the use of XML variables which are set at the beginning and can then be referenced second the specification of default values for parameters of contexts or modules lt xml version 1 0 gt lt dabc version 1 gt lt Enter the values for specific setup gt lt Variables gt lt ctrl value 1xg0523 gt lt mbs1 value r3g 100 gt lt mbs2 value r3g 101 gt lt readl value 1x1001 gt lt read2 value 1x1002 gt lt ebl value 1x1003 gt lt eb2 value 1x1004 gt lt bufsize value 65536 gt lt custport value 6000 gt lt Variables gt lt Context host ctrl name Controller gt lt Run gt lt lib value DABCSYS lib libDabcBnet so gt lt runfunc value RunTestBnet gt lt Run gt lt Application class bnet Cluster gt lt NetDevice value dabc SocketDevice gt lt CtrlBuffer value 2048 gt lt TransportBuffer value bufsize gt
88. arameter list at the right window in Fig 4 7 page 27 shows only the parameters matching all filters If the data field is white the parameter can be changed This cannot be done in place because the param eter might be updated in the mean time Instead press RETURN in the field A prompter will pop up to enter the value 4 2 8 Monitoring panels As already mentioned the DABC provides definitions of special purpose DIM parameters These Records can be recognized by the GUI and are handled in appropriate way Currently there are States Rates Histograms O O O o Infos 4 2 8 1 States States are records having a number for severity 0 to 4 a color and a brief state description see Fig 4 8 page 28 Of cause the states of the DABC state machine are shown as states Application plug ins may use this kind of records also for other information 28 DABC User Manual GUI 10 Readout E Ixi011 Readout E Ixi014 Eventbuilder Ix Controller Figure 4 8 States RateMeters iii Settings 7 MSG TrendKb 65G 4 M5G EventRate 16000 Figure 4 9 Rates 4 2 8 2 Rate meters All rate meters are displayed in the meter panel Fig 4 9 page 28 Meters can be removed in the parameter table See Fig 4 6 page 26 with the Show buttons like the other graphical parameters Saving the setup the visibility will be preserved RateMeters oo amp Settings RateMeters o a bd Settings 86 7 MSG 86 7 MSG X86 7
89. areas are kept in dabc MemoryPool objects In general case dabc Buffer contains a list of references to scattered memory fragments from memory pool Typically a buffer references exactly one segment Buffer may have an empty list of references In addition the buffer can be supplied with a custom header The auxiliary class dabc Pointer offers methods to transparently treat the scattered fragments from the user point of view concept of virtual contiguous buffer Moreover the user may also get direct access to each of the fragments The buffers are provided by one or several memory pools which preallocate reasonable memory from the operating system A memory pool may keep several sets each set for a different configurable memory size A modules communicates with a memory pool via a dabc PoolHandle object A new buffer may be requested from a memory pool by size Depending on the module type and mode this request may either block until an appropriate buffer is available or it may return an error value if it can not be fulfilled The delivered buffer has at least the requested size but may be larger A buffer as delivered by the memory pool is contiguos 9 3 Controls and configuration 53 Several buffers may refer to the same fragment of memory Therefore the memory as owned by the memory pool has a reference counter which is incremented for each buffer that refers to any of the contained fragments When a user frees a buffer objec
90. as argument dabc_run Readout xml This executable will load the specified libraries create the application configure it and switch the system in the Running state Part II Programmer Manual 49 Chapter 9 DABC Programmer Manual Overview programmer prog overview tex 9 1 Introduction The DABC Programmer Manual describes the aspects of the Data Acquisition Backbone Core framework that are necessary for programming user extensions To begin with this overview chapter explains the software objects and their collaboration the intended mechanisms for controls and configuration the dependencies of packages and libraries and gives a short reference of the most important classes The following chapters contain full explanations of the DABC interface and service classes describe the set up with parameters and give a reference of the Java GUI plug in possibilities Finally some implementation examples are treated in detail to illustrate these issues the adaption of the GSI legacy DAQ system MBS within DABC the application of a distributed event builder network Bnet the data import via UDP from a readout controller board ROC and the use of a PCI express board ABB 9 2 Role and functionality of the objects 9 2 1 Modules All processing code runs in module objects There are two general types of modules the dabc ModuleSync and the dabc ModuleAsync 9 2 1 1 Class dabc ModuleSync Each synchronous module is executed
91. atches and handles the exception On state machine commands e g Halt or Sus pend see Programmer Manual section 9 3 1 the blocking calls are also left by exception thus putting the mainloop thread into a stopped state 2 2 1 2 Asynchronous module Several asynchronous modules may be run by a shared working thread The thread processes an event queue and executes appropriate callback functions of the module that is the receiver of the event Events are fired for data input or output command execution and if a requested resource e g memory buffer is available The callback functions must never block the working thread Instead the callback must return if further processing requires to wait for a requested resource Therefore each callback function must check the available resources explicitly whenever it is entered 2 2 2 Commands A module may register Command objects and may define command actions by overwriting a virtual command callback method ExecuteCommand 2 2 Introduction 7 2 2 3 Parameters A module may register Parameter objects Parameters are accessible by name their values can be monitored and optionally changed by the controls system Initial parameter values can be set from XML configuration files 2 2 4 Manager The modules are organized and controlled by one manager object which is persistent independent of the application s state The manager is an object manager that owns and keeps all registered basic o
92. be re implemented in the user subclass if one wants to treat all events centrally As arguments one gets the pointer to the relevant component port timer and a number describing the event type dabc evntInput dabc evntOutput Class dabc ModuleAsync has no methods which can block Nevertheless the user should avoid any kind of polling loops waiting for some other resource buffer output queue and so on the callbacks should return as soon as possible In such situation processing must be continued in another callback that is invoked when the required resource is available This might require an own bookkeeping of such situations kind of state transition logic Let s consider as an example the same repeater module but implemented as asynchronous module include dabc ModuleAsync h include dabc Port h class RepeaterAsync public dabc ModuleAsync unsigned Cnt public RepeaterAsync const char name dabc ModuleAsync name CreatePoolHandle Pool 2048 1 Createlnput Input Pool 5 12 2 Modules 85 CreateOutput OutputO0 CreateOutput Outputl fCnt 0 virtual void ProcessInput while Input gt CanRecv dabc Buffer buf Output fCnt virtual le dabc Buffer buf Output fCnt whi The constructor of this module has absol fCnt member to count direction for output void ProcessOutput Input gt CanR
93. bjects into a folder structure Moreover the manager defines the interface to the control system This covers registering sending and receiving of commands registering updating unregistering of parameters error logging and global error handling The manager receives and dispatches commands to the destination modules where they are queued and eventually executed by the modules threads see Programmer Manual section 9 2 1 The manager has an independent manager thread used for manager commands execution parameters timeout processing and so on 2 2 5 Memory and buffers Data in memory is referred by Buffer objects Allocated memory areas are kept in MemoryPool objects In general case a buffer contains a list of references to scattered memory fragments from memory pool Typically a buffer references exactly one segment Buffers may have an empty list of references In addition buffers can be supplied with a custom headers The buffers are provided by one or several memory pools which preallocate reasonable memory from the operating system A memory pool may keep several sets each set for a different configurable memory size A new buffer may be requested from a memory pool by size Depending on the module type and mode this request may either block until an appropriate buffer is available or it may return an error value if it can not be fulfilled The delivered buffer has at least the requested size but may be larger A buffer as deli
94. bsequent MBS events there is no buffer header in front here To iterate over all events in such buffer and to access them class mbs ReadIterator is provided as defined inmbs Iterator h This is done in the following way include mbs Iterator h void Print dabc Bufferx buf mbs ReadIterator iter buf 95 96 DABC Programmer Manual Example MBS whil iter NextEvent DOUT1 Event Su size u iter evnt gt EventNumber iter evnt gt FullSize while iter NextSubEvent DOUT1 Subevent crate Su procid Su size Su iter subevnt gt iSubcrate iter subevnt gt iProcld iter subevnt gt FullSize Another class mbs Writelterator fills MBS events into dabc Buffer This is illustrated by the following code include mbs Iterator h void Fill dabc Buffer buf mbs Writelterator iter buf unsigned evntid 0 whil iter NewEvent evntid for unsigned subcnt 0 subcnt lt 3 subcnt if iter NewSubevent 28 0 subcnt return ill raw data iter rawdata here memset iter rawdata 0 28 iter FinishSubEvent 28 if iter FinishEvent return Method NewEvent will put new event header at the current iterator position in the buffer In a similar way method NewSubevent will put a subevent header with the specified arguments there Access to the data pointer after the last event or
95. called for each parameter update o void releaseDimServices All local references to commands or parameters must be cleared 18 7 1 2 Interface xiUserCommand o boolean getArgumentStyleXml String scope String command Return true if command shall be composed as XML string false if MBS style string Scope is specified in the XML command descriptor command is the full command name 18 7 1 3 Interface xiUserInfoHandler o void infoHandler xiDimParameter p int handlerID An object implementing this interface can be added to each parameter as call back handler This is done by the browser function setInfoHandler see below Function infoHandler is then called in the callback of the parameter 144 DABC Programmer Manual GUI o String getName Called by xDimParameter to get a uniquie name of this handler Must return a name of the handler to distinguish from other handlers 18 7 2 Java Interfaces provided by GUI 18 7 2 1 Interface xiDesktop o void addFrame JInternalFrame f Adds a frame to desktop if a frame with same title does not exist void addFrame JInternalFrame frame boolean manage Adds a frame to desktop if a frame with same title does not exist boolean findFrame String title Checks if a frame exists on the desktop void removeFrame String title Remove dispose a frame from the desktop and list of managed frames void setFrameSelected String title boolean select Switch a frames selectio
96. cation has virtual method DoStateTransition which is called from the state machine during state change As argument name of state transition command is delivered There are the following state machine commands dabc Manager stcmdDoConfigure creates all necessary application components devices modules memory pools dabc Manager stcmdDoEnable connects local and or remote nodes together if necessary dabc Manager stcmdDoStart starts execution of user modules dabc Manager stcmdDoStop stop execution of user modules dabc Manager stcmdDoHalt destroy all components created during configure dabc Manager stcmdDoError react on error which happened during other commands Class dabc Application already has default implementation for DoStateTransition method where some virtual methods are called CreateAppModules creates all necessary application components ConnectAppModules activity to connect with remote nodes or IsAppModulesConnected check if connection is already performed BeforeAppModulesStarted optional activity before modules are started AfterAppModulesStopped optional activity after modules are stopped BeforeAppModulesDestroyed optional call before modules are destroyed Actually for a single node application it is enough to implement CreateAppModules since all other methods have meaningfull implementation for that case In simplest case one can just implememnt C function which is call
97. ce Note queued buffers are not flushed or discarded on Stop DoHalt switches states Ready Running Configured or Failure to Halted The framework automatically deletes all registered objects transport device module in the correct order However the user may explicitly specify on creation time that an object shall be persistent e g a device may be kept until the end of the process once it had been created 9 3 2 Commands The control system may send user defined commands to each component module device application Execu tion of these commands is independent of the state machine transitions 9 3 3 Parameters for configuration and monitoring The Configuration is done using parameter objects The manager provides an interface to register parameters to the configuration control system On application startup time the configuration system may set the parameters from a configuration file e g XML configuration files During the application lifetime the control system may change values of the parameters by command However since the set up is changed on DoConfigure time only it may be forbidden to change true configuration parameters except when the application is Halted Otherwise there would be the possibility of a mismatch between the monitored parameter values and the really running set up However the control system may change local parameter objects by command in any state to modify minor system properties independent
98. ch user see section 3 3 page 15 For the ABB example this is found at SDABCSYS applications bnet test SetupBnetIB ABB xm 3 4 2 Plug in packages in user directory This is the case when DABC is installed centrally at the fileserver of an institute and several experimental groups shall use different plug ins It is also the recommended way if several users want to modify the source code of a plug in library independently without affecting the general installation The new plug in package should be installed in a user directory then with the following steps 1 2 Download the plug in package tarball e g abbl tar gz Create a directory to contain your additional DABC plugin packages mkdir HOME mydabcpackages Call the dabclogin sh script of the DABC installation see section user env Copy the downloaded tarball to the SDABCSYS directory and unpack it there cp abbl tar gz SHOME mydabcpackages cd SHOME mydabcpackages tar zxvf abbl tar gz This will extract the new components into the appropriate plugins and applications folders below the working directory To build the plugins part change to the appropriate package plugin directory and invoke the local Makefile e g for the ABB example cd SHOME mydabcpackages plugins abb make This will create the corresponding plug in library in a subfolder denoted by the computer architec ture e g SHOME mydabcpackages plugins abb x86_64 lib libDabcAbb
99. check if execution of module code shall be continued This method will also execute the queued commands if synchronous command execution was specified before by method SetSyncCommands By default a command can be executed in any place of the code Let s consider a simple example of a module which has one input and two output ports and delivers buffers from input to one or another output sequentially Implementation of such class will look like include dabc ModuleSync h class RepeaterSync public dabc ModuleSync public RepeaterSync const char name dabc ModuleSync name CreatePoolHandle Pool 2048 1 Createlnput Input Pool 5 CreateOutput Output0 Pool 5 CreateOutput Outputl Pool 5 virtual void MainLoop unsigned cnt 0 84 DABC Programmer Manual Plugins while ModuleWorking dabc Buffer x buf Recv Input if cnt 2 0 Send Output 0 buf else Send Output 1 buf y In constructor one sees creation of pool handle and input and output ports Method MainLoop has a simple while loop that receives a buffer from the input and then sends it alternatingly to the first or the second output 12 2 6 ModuleAsync In contrast to data processing in dabc ModuleSync main loop class dabc ModuleAsync provides a number of callbacks routines which are executed only if dedicated DABC events occur For instance when any input port gets n
100. cuted in the appropriate thread context With method Execute of class dabc CommandReceiver one can execute a command directly in the receiving object Here one can specify a dabc Command object as argument or just a command name if the command has no arguments Method Execute will block until the command is executed this is called the synchronous mode of command execution Optionally one can set a timeout how long the calling thread will wait until the command is executed Method Execute can only check if a command is executed succesfully or not as it has a boolean return value There are advanced methods Executelnt and ExecuteStr which return the result of a command execution as integer or string value resp They will deliver the final value of the command parameter which is specified by name in the second function argument For example the result of command UserGetCommand execution from the previous example one can obtain like this 78 DABC Programmer Manual Services dabc Modulex m dabc mgr gt FindModule Modulel int res m gt Executelnt UserGetCommand UserRes There is an other way to execute a command submit the Command with Submit method see also section 10 2 5 In this case the command will be executed asynchronous to the calling thread therefore one cannot get any direct information about the result of command execution from the return value of Submit 11 3 3 Command clie
101. d Main Usually events are produced by calling dabc WorkingProcessor FireEvent method this method can be invoked from any thread All events are queued and a pthread condition is fired in this case The thread waiting for this condition is woken up and the next event from the queue will be delivered to the WorkingProcessor by calling virtual method dabc WorkingProcessor ProcessEvent Here any user specific code can be implemented Another task of dabc WorkingThread consists in timeout handling Some WorkingProcessors may require to be invoked not only by events but also after specified time intervals Method dabc WorkingProcessor ActivateTimeout requests the thread to execute dabc WorkingProcessor ProcessTimeout after the specified time interval This virtual method may also be implemented by the user 11 2 2 Sockets handling The POSIX sockets library provides the handling of all socket operations in an event like manner using the select function Such approach was used in dabc SocketThread and dabc SocketProcessor classes to handle several sockets in parallel from a single thread With each dabc SocketProcessor a socket descriptor is associated which can deliver events like can read next portion of data from socket sending over socket will not block socket is broken and so on The main loop of dabc SocketThread is modified such that instead of waiting for the pthread condition the thread waits for the
102. d call dabc mgr gt ConnectPorts Modulel Output Module2 Input 12 3 4 Network transport This is a kind of Transport which is used to connect Ports on different nodes Abstract base class dabc NetworkTransport introduces such kind of functionality this transport is locally connected to one port only and all buffer transfer is done via network connections with the remote node For the moment DABC has two implementations of network transports for socket and InfiniBand verbs To use NetworkTransport on the nodes one should follow a two step strategy At the first step on all nodes the necessary Devices and modules should be created dabc mgr gt CreateDevice dabc typeSocketDevice UserDev dabc mgr gt CreateModule UserModule MyModule Then during the second step on the master node where dabc mgr gt IsMainManager is true see section 10 3 2 page 66 one should call dabc mgr gt ConnectPorts Node0 MyModule Input Nodel MyModule Output UserDev Such call starts an elaborated sequence at first a server socket will be opened by Device UserDev on node Node0 then Device UserDev on Nodel will try to connect to that server socket finally on both nodes appropriate NetworkTransports will be created using these negotiated sockets and connected to the ports My Module Input and MyModule Output resp 88 DABC Programmer Manual Plugins Exactly for th
103. d to its receiver if such is specified as command parameter or the Manager working thread itself will execute the command Thus method does not block and returns true if it accepts the command for execution otherwise false Manager commands queue can also be used to submit command not only direct for manager but also for any other object on local or remote node For that one from several following functions SetCmdReceiver should be used Command SetCmdReceiver Command cmd const char itemname Command SetCmdReceiver Command cmd Basic rev Command SetCmdReceiver Command cmd const char nodename const char itemname Command SetCmdReceiver Command cmd int nodeid const char itemname Set receiver attribute of com mand cmd The itemname is item name of the local node like Module1 One can also specify nodename or nodeid of the node to which command should be submitted Returns pointer on the command itself One can use SetCmdReceiver to submit command like dabc mgr gt Submit dabc SetCmdReceiver cmd Modulel dabc mgr gt Submit dabc SetCmdReceiver cmd Nodel Modulel dabc mgr gt Submit cli Assign dabc SetCmdReceiver cmd 0 Modulel 10 2 6 Memory pool management bool CreateMemoryPool const char poolname unsigned buffersize unsigned numbuffers unsigned numincrement unsigned headersize unsigned numsegments Instantiates adabc MemoryPool of name poolname
104. data acquisiton systems In DABC one can implement access to such boards by means of special Device and Transport classes that com municate with the appropriate linux device driver The Device represents the board and may do the hardware set up at Configure time using dedicated dabc Parameters The Transport may fill its data buffers via board DMA and pass the Buffers to the connected readout Module This example treats the Active Buffer Board ABB 4 a PCI express PCIe board with a Virtex 4 FPGA and optical connectors to receive data from the experiment frontend hardware It is developed for the CBM experiment 2 by the Institut f Technische Informatik at Mannheim University The board developers deliver a kernel module as linux device driver and the mprace C library to work with the board from user space Since this driver software may also be applied for other PCIe boards the corresponding DABC classes pci Device and pci Transport are rather generic using namespace pci The special properties of the ABB board are then implemented in a pci BoardDevice subclass and in further classes with namespace abb Besides some simple test executables that read from and write to the ABB ona single machine there is an example of a bnet WorkerApplication that applies the ABB classes for the readout module 17 2 PCI Device and Transport 17 2 1 pci BoardDevice Subclass of dabc Device Adapter to the the mprace Board functionality
105. defines the so called all to all topology i e each node communicates with all other nodes including itself Parameter NumReadouts 4 means that there are 4 inputs on each combiner resulting in 16 data sources for the complete system To run this example one should specify correct host names for all contexts and startit with run sh SetupBnet xml command This can be used as template for developing a user specific application One can change functionality of combiner and builder modules and provide a real readout instead of the generator module 15 9 BNET for MBS application This is a ready ro use implementation of distributed event building for MBS The source code can be found in DABCSYS plugins bnet mbs directory It contains following classes bnet MbsCombinerModule bnet MbsBuilderModule bnet Mbs WorkerApplication bnet MbsFactory Class bnet MbsCombinerModule combines together events with the same event identifier from all inputs In the cluster application parameter NumEventsCombine defines how many events should be bundled together in one 118 DABC Programmer Manual Example Bnet buffer It is crucial that transport buffer size is big enough for such number of subevents During initialisation cluster parameter NumEventsCombine is copied to each worker parameter CfgEventsCombine which is finally used in bnet MbsCombinerModule bnet MbsCombinerModule MbsCombinerModule fCfgEventsCombine GetCfgInt Cfg
106. der 777777 m CAL ADaramatore g State o RX Layout Acquisition E 4 RunMode E IxgO u Ixi013 Node lIxi014 startTime RunMode 4 Acquisition Ki Controller Ixi010 Readout yentbuilder E 1x1014 Eventbuilder Parameter Type Curr int 123 Nxi014 workerTid int 42 X86 7 X86 7 X86 7 DataRateKb DataTrendKb EventRate EventTrend rate 1596 rate rate rate RateMeters Settings E E E E E E m E E E E E E El Logger Logging X86G 4 X86G 4 X86G 4 X860 4 gt X86G 4 X86G 4 util X86G 4 util xi0 7 io X86 7 util X86G 4 util X86G 4 util f util 1xi010 R util Dz util Name of output device util Tape label Dismounted file CLOSED util 0 000 MB written to tape 0 000 to file HS O ntbuil Ixi014 EventRate d ilde DataRa ZEZ2 2222222 a Figure 4 2 More typical full screen view GUI itself can be closed and reopened any time 4 2 1 Main DABC GUI buttons E Quit GUL Will prompt RET will quit The DABC will continue to run The GUI may be started 4 2 GUI Panels 23 anywhere again In case you saved the layout recommended see 4 3 page 30 and you start the GUI from the same directory it will look pretty much the same as you
107. device If the dabc Transport implementation shall be able to block e g on socket receive there can be only one trans port for this thread A dabc Device instance usually represents an I O component e g network card there may be more than one dabc Device instances of the same type in an application scope The device objects are owned by the manager singleton transport objects are owned and managed by their corresponding device A device is persistent independent of the connection state of the transport In contrast a transport is created during connect or open and deleted during disconnect or close respectively A device may register parameters and define commands This is the same functionality as available for modules 9 2 9 Application The dabc Application is a singleton object that represents the running application of the DAQ node i e one per system process It provides the main configuration parameters and defines the runtime actions in the different con trol system states see section 9 3 1 In contrast to the dabc Manager implementation that defines a framework control system e g DIM EPICS the subclass of dabc Application defines the experiment specific behaviour of the DAQ 9 3 Controls and configuration 9 3 1 Finite state machine The running state of the DAQ system is ruled by a Finite State Machine 6 on each node of the cluster The manager provides an interface to switch the application sta
108. dispatcher goes into pause loop needed with non threaded DIM version for keep alive signals When message logger is started with argv 1 dim it creates the DIM parameters and starts a thread to update these every second One DIM parameter is used for the messages and updated when a message arrives Messages from local tasks are received in a thread f_mg_msg_thread and either sent to master message logger when this one is slave or processed by _mg_msg_output This function updates the DIM message parameter when in DIM mode sends message to connected remote message client or prints it if not and writes log file 14 9 5 7 Multi node mode In multi node mode the MBS nodes are controlled through one master node where a prompter is running The prompter can be started and stopped interactively or from a remote node by script There are two new scripts to start and shutdown a multi node MBS from a remote node GUD 14 9 MBS upgrade for DABC 107 prmstartup scandprmshutdown sc These are called by rsh from the GUI node Arguments are the path of MBSROOT and the user working path For starting the DIM server also the DIM name server node is passed prmstartup sc SMBSROOT SPWD SDIM_DNS_NODE SREMOTE_NODE launches m_prompt dim r lt remotenode gt after waiting for all 60xx sockets closed prmshutdown sc SMBSROOT SPWD callsm_remote reset 1 task m_prompt Withm_wait_for task m prompt it waits for prom
109. e whatever command gt bsServers prompt Number of needed DIM servers value 3 gt MbsLaunch gt lt lt lt lt lt lt lt lt 18 6 DIM update mechanism 141 18 6 DIM update mechanism To get informed when a DIM parameter has been updated a DIM client has to register to it In a Java DIM client this is done by instantiating a subclass of DimInfo In xGUI this is xDimParameter implementing callback function infoHandler After registration the callback function is called once immediately In infoHandler one can use getter functions to get the quality the format string and the value s 18 6 1 xDimBrowser The central object handling the available lists of DIM parameters and commands is the xDimBrowser It provides the functions xDimBrowser Constructor Arguments references to the graphics panels xPanelMeter xPanelState xPan ellnfo and xPanelHisto There are protected functions to get then the references to these panels protected initServices String wildcard Get list of available services from DIM name server DIM DNS_ NODE Create vectors of alphabetically ordered parameters xDim Parameter and commands xDimCommand and their interfaces respectively The references of the graphics panels are passed to the parameter objects addInfoHandler xiDimParameter p xiUserInfoHandler ih Interface function to add an additional info handler to a parameter The infoHandler function of this handl
110. e a subevent CreateBuilder create a module which combines N subevents to a complete event CreateFilter optional filter module to filter out events CreateReadout creates a readout transport connected to a data source CreateStorage creates a storage transport to store data on disk tape The user must define his her own application class which inherits from bnet WorkerApplication implementing these virtual methods Class bnet WorkerApplication has the following public configuration parameters Name Type Dfit Description IsGenerator bool false use generators instead of data sources IsSender bool false 1s sender module is created readout functionality IsReceiver bool false 1s receiver module is created event builder functionality IsFilter bool false 1s filter module is created event builder should be true NumReadouts int 1 number of data inputs InpitOCfg str string parameter to configure input 0 user specific Inpit1Cfg str string parameter to configure input 1 and so on user specific StoragePar str string parameter to configure storage user specific ReadoutBuffer int 2048 buffer size used for readout ReadoutPoolSize int 4MB size of memory pool for readout TransportPoolSize int 16MB size of memory pool for data transport EventBuffer int 32768 buffer size used for event building EventPoolSize int 4MB size of memory pool for event building There are also number of private parameters which are not se
111. e framework that all commands are submitted and should be ready within 10 seconds Return argument dabc cmd postponed indicates that the master command may not be ready when ExecuteCommand is returned Therefore dabc CommandsSet will take care about the correct reply of the master command either when all slaves are ready or when the master command timeout has expired 80 DABC Programmer Manual Services Chapter 12 DABC Programmer Manual Plugins programmer prog plugin tex 12 1 Introduction A multi purpose DAQ system like DABC requires to develop user specific code and adopt this into the general framework A common object oriented technique to realize such extensibility consists in the definition of base classes as interfaces for dedicated purposes The programmer may implement subclasses for these interfaces as Plug Ins with the extended functionality that matches the data format hardware or other boundary conditions of the data taking experiment Moreover the DABC core itself applies such powerful plug in mechanism to provide generic services in a flexible and maintainable manner This chapter gives a brief description of all interface classes for the data acquisition processing itself This covers the processing Modules the Transport and Device objects that move data between the DAQ components and the Application that is responsible for the node set up and run control A Factory pattern is used to introduce new classes
112. e if p getFull indexOf Setup_File gt 0 param vipar get i E 18 7 Application specific GUI plug in 149 end list of parameters stapan addGraphics state false stapan updateAll metpan updateAll if frame null frame rebuild stapan metpan All references or allocated objects from setDimServices we have to free in releaseDimServices public void releaseDimServices metpan cleanup stapan cleanup param null We provide a little extra class implementing xiUserHandler function infoHandler Each parameter we want to monitor gets its own handler instance which has direct access to our graphics panels private class myInfoHandler implements xiUserInfoHandler private myParameter String Name xMeter Meter xState State name new String Name store meter Meter store state State store public String getName return name public void infoHandler xiDimParameter P if meter null meter redraw P getMeter getValue true true if state null state redraw P getState getSeverity P getState getColor P getStat getValue true 18 7 5 Store restore layout It is absolutely necessary to save and restore window layouts to be able to see the GUI after restart as before This is done through xLayout objects which are managed centrally They keep information about frame position size visibili
113. e may be installed in a user direc tory and linked against the Core system installation and the required Plugin packages 9 4 5 Distribution contents The DABC distribution contains the following packages Core system This is plain C code and independent of any external framework Bnet plugin Depends on the core system only Transport plugins Network transport for tcp ip sockets and InfiniBand verbs Additionally transports for GSI Multi Branch System MBS connections socket filesystem is provided Optionally example transport packages may be installed that illustrate the readout of a PCIe board or data taking via UDP from an external readout controller ROC board Control and configuration system The general implementation is depending on the DIM framework only DIM is used as main transport layer for commands and parameter monitoring On top of DIM a generic record format for parameters is defined Each registered command exports a self describing command descriptor parameter as DIM service Configuration parameters are set from XML setup files and are available as DIM services GUI A generic controls GUI using the DIM record and command descriptors is implemented with Java It may be extendable with user defined components Application packages some example applications such as o Simple MBS event building o Bnet with switched MBS event building o Bnet with random generated events 9 5 Main Classes 9 5 1 Core system
114. eCommand dabc Command cmd if cmd gt IsName UserCommand int v cmd gt GetInt UserArg 0 DOUT1 Execute UserCommand with argument d v return dabc cmd_true else if cmd gt IsName UserGetCommand DOUT1 Execute UserGetCommand without arguments cmd gt SetInt UserRes fCounter return dabc cmd_true return dabc ModuleAsync ExecuteCommand cmd Method ExecuteCommand should analyse the command name and perform command specific actions It should return dabc cmd _ true if the command has been executed succesfully or dabc cmd _ false otherwise The default implementation of dabe CommandReceiver methods performs the command execution in the calling thread However most command actions may access resources which are also used by another working thread as signed to the CommandReceiver object In this case all command execution code had to protect these resources by mutex locks see section 11 2 3 which would decrease performance Because of this class dabc WorkingProcessor inherits from dabc CommandReceiver and implements several virtual methods like IsExecutionThread Sub mit which are necessary to deliver and execute a command in the thread context of the assigned WorkingTh read The user must not reimplement these methods again in the derived classes In the DABC subclasses of dabc WorkingProcessor like dabc Module dabc Application the custom commands will be exe
115. ead name cmd gt SetInt ABB COMPAR _BUFSIZE readsize cmd gt Set Int ABB_COMPAR_STALONE 1 cmd gt Set Int ABB_COMPAR_QLENGTH 10 cmd gt Set Str ABB_COMPAR_POOL ABB standalone pool res manager Execute cmd DOUT1 Create ABB readout module s DBOOL res Again the parameter names e g ABB_COMPAR_OLENGTH use common string definitions as set in abb Factory h such as the size of the memory pool buffers ABB_COMPAR_BUFSIZE which is set to the required DMA transfer sizereadsize the standalone run mode of the module ABB_COMPAR_STALONE the port queue length ABB_COMPAR_QLENGTH the name of the module s memory pool ABB_COMPAR_POOL which also 4 The transport connection between the input port of the readout module and the abb Device is established by a direct method call of the manager res manager CreateTransport ABB Readout Input devname c_str DOUT1 Connected module to ABB device s DBOOL res The manager will find the ABB device instance by the string devname and use its factory method Create Transport to instantiate a pei Transport that will be connected to the port of name ABB_Readout Input 5 The readout module processing is started by name with a manager method 130 DABC Programmer Manual Example PCI manager StartModule ABB_Readout DOUT1 Started readout module Then the main process wait
116. ead_Size Read_Start Starts reading of buffer Should return dabc di_Ok normal case call of Read_Complete will follow dabc di_ Error error skip buffer starts again from Read_Size dabc di_ CallBack asynchronous readout user should call Read_CallBack If di CallBack returned processing of this transport is suspended until user calls Read_CallBack method providing the result of reading di_Ok or di_ Error This mode is only possible if the device driver has its own thread or DMA engine resp that can perform the readout and then can call DABC methods The big advantage of such mode the data transport thread is not blocked by waiting for a result from the device therefore several DataTransports can share the same thread Read_Complete Finish reading of the buffer Can return dabc di_Ok normal buffer will be delivered to port dabc di_ Error error close transport dabc di EndOfStream end of stream normal close of the transport dabc di_SkipBuffer normal but buffer will not be delivered to the port dabc di_ Repeat not ready call again as soon as possible dabc di_ Repeat Timeout not ready call again after timeout In the simple case actual reading of data is directly performed in this method Otherwise one may wait here until another thread or a DMA transfer initiated before by Read_Start has filled the buffer In this case one should be carefull and not block thread forever it is better to return
117. ecord type and handle the record in an appropriate way The COMMANDDESC record is an XML string describing a command The name of a descriptor record must be the name of the command it describes followed by an underscore 18 3 GUI global layout 137 18 3 GUI global layout The top window of the xGUI is a JFrame Inside that is a JPanel which contains on top a JToolBar all the main buttons in the middle a JDesktopPane main viewing area and at the bottom a JTextArea One line text for server list All other windows are inside added to the desktop as JInternalFrames Typically such a frame contains again a JPanel Inside that panel various different layouts can be used like JSplitPane or a Jtree in a JScrollPane In fact xInternalFrame a subclass of JInternalFrame is used It can contain exactly one panel has a mechanism to store and restore its size and position and implenents the callback functions for resizing and closing Inside the internal frames two types of panels are often used prompter panels and graphics panels 18 3 1 Prompter panels Prompter panels can be implemented subclassing class xPanelPrompt Example DABC launch panel The layout is in rows A row can be a prompter line JLabel label and JTextField input field a text button JButton or a JLabel label and JCheckBox At the bottom there is a JToolBar where buttons with icons can be placed The prompter class must implement the ActionListener ie provide the actio
118. ecution will thus happen in re implemented method ExecuteCommand of dimc Manager Thus the command action runs independent of the DIM command handler thread 5 Parameter subscription Method Subscribe Unsubscribe resp of dimc Manager are forwarded to SubscribeParameter UnsubscribeParameter resp of dimc Registry These implement it by means of the DimService update mechanism Subscriptions are kept as vector of dimc DimParameterInfo objects see section 10 3 3 5 6 Remote command execution Method SendOverCommandChannel of dimc Manager is forwarded to SendDimCommand of dimc Registry The streamed dabc Command is wrapped as text argument into the DIM ManagerCommand and send to the destination by node name via DimClient sendCommand 70 DABC Programmer Manual Manager 10 3 3 3 dimc Server Subclass of DIM class DimServer implementing command handler error handler and exit handlers for client and server exit events 1 Because most DIM server actions are invoked by static methods of DimServer it is reasonable to have only one instance of dimc Server thus this class is designed as singleton pattern Access and initial creation is provided by method Instance A safe cleanup is granted by Delete ctors and dtors are private and cannot be invoked directly 2 The dimc Registry is set as owner of dimc Server by means of a back pointer All handler methods of the DimServer are implemented as forward cal
119. ecv Pool Pool Event dabc Portx port o 5 amp amp Output fCnt Input gt Recv 2 gt CanSend 2 gt Send buf Event dabc Portx port o 5 2 gt CanSend amp amp Output fCnt Input gt Recv 2 gt Send buf utely the same components as in previous example One should add of next buffer Value of fCnt in some sense defines current state of the module Instead of the MainLoop one can see two virtual methods for input and output event processing In each methods one sees same code with while loop inside In the loop one checks that input and current output are ready and retransmit buffer When any port input or output has no more possibility to transmit data method will be returned One needs a while loop here because not every input event and not every output events leads to buffer transports If input queue is empty CanRecv returns false or output queue is full CanSend returns false one cannot transfer a buffer from input to output thus the callback must be returned But when the event processing routine is called the next time one should tranfer several buffers at once Since methods Send and Recv cannot block such while loop will not block either But in any case one should avoid such wrong code virtual void ProcessInput Event dabc Portx port this kind of waiting is WRONG o 5 2 while Outp
120. ed instead CreateAppModules method of application name of with function should be specified in configu ration file in Run func node For special DAQ topologies e g Bnet the framework offers implementations of the dabc Application containing the generic functionality e g bnet WorkerApplication bnet ClusterApplication In this case the user specific parts are implemented by subclassing and implementing additional virtual methods e g CreateReadout 90 DABC Programmer Manual Plugins 12 5 Factories The creation of the application specific objects is done by dabc Factory subclasses The user must define a dabc Factory subclass to add own classes to the system The user factory should already be instantiated as global stack object in its class implementation code this will create the factory immediately after the user library has been loaded On creation time a factory is registered automatically to the dabc Manager instance The user factory may implement such methods CreateModule Instantiate a dabc Module of specified class CreateDevice Instantiate a dabc Device of specified class CreateThread Instantiate a dabc WorkingThread of specified class CreateApplication Instantiate a dabc Application of specified class CreateTransport Instantiate a dabc Transport of specified class This method is used when transport does not requires specific device functionality like dabc DataTransp
121. egmented data which is referenced by a dabc Buffer object Using dabc Pointer one should not care how many segments are referenced by the Buffer and how big they are One can use following methods Pointer or reset initialize or reset the pointer as a reference of a dabc Buffer of another dabc Pointer or just of a simple memory region ptr or operator the current memory pointer rawsize size of contiguous system memory from current pointer position fullsize size of full memory from current pointer position shift shift pointer copyfrom set pointed memory content from a dabc Pointer or just from a memory region copyto copy pointed memory content into specified memory region Example of pointer usage include dabc Buffer h include dabc Pointer h void UserModule ProcessOutputEvent dabc Portx port if Input 0 gt CanRecv return dabc Buffer buf Input 0 gt Recv dabc Pointer ptr buf uint32_t v 0 while ptr fullsize gt 0 ptr copyfrom amp v sizeof v ptr shift sizeof v Vr Output 0 gt Send buf 74 DABC Programmer Manual Services 11 1 5 Buffer guard Class dabc BufferGuard is the equivalent of a LockGuard for threads see section 11 2 3 preventing memory leaks due to unreleased Buffers It should be used to automatically release a Buffer whenever the function scope is left both by returning regularly and by throwing an exception One should e
122. ell to set the environment At GSI linux installation this is done by dabclogin For the user installation as described in above section 3 1 page 13 by default the script is named dabclogin sh The login script will already enable the DABC framework for compilation of user written compo nents Additionally the general executable dabc_run now provides the DABC runtime environ ment and may be started directly for simple batch mode applications on a single node However further preparations are necessary if DABC shall be used with DIM control system and GUI 2 Open a dedicated shell on the machine that shall provide the DIM name server e g ssh nsnode cluster domain export DIM_DNS_NODE nsnode cluster domain dabclogin sh dimDns amp dimDid amp to launch the DIM name server This is done once at the beginning of the DAQ setup usually the DIM name server needs not to be shut down when DABC applications terminate The DID is useful for inspecting DIM services 3 Set the DIM name server environment variable in any DABC working shell e g the shell that will start the dabc gui later dabclogin sh export DIM_DNS_NODE nsnode cluster domain 4 Now the DABC GUI can be started in such prepared shell by typing dabc or mbs for a plain MBS gui resp See below in gui section To operate a DABC application one should create a dedicated working directory to keep all relevant files e Setup files for DABC XML e Log fil
123. en by control system and cannot be configured via XML file 15 4 Combiner module 115 Name Type Dfit Description CfgNodelD int node id starts from 1 for workers CfgNumNodes int number of nodes in configuration CfgSendMask str string in form of xxox defines which nodes are sender x or not o CfgRecvMask str string in form of xxox defines which nodes are sender x or not o CfgClusterMgr str name of cluster controller node CfgNetDevice str name of configured network device same as cluster param NetDevice CfgEventsCombine int number of events combined together same as cluster param NumEventsCombine CfgReadoutPool str name of memory pool used for readout ReadoutPool or TransportPool CfgConnected bool true when local configuration of application completed These parameters are set during initialization phase Some of them like CfgEventsCombine should be used by modules for it s configuration If required the user subclass of bnet WorkerApplication may define additional configuration parameters 15 4 Combiner module The combiner module merges together several data sources and produces subevent packets Here a subevent means that data from all sources which belong to the same event or time frame resp are put into the same dabc Buffer object This buffer object should have a header with a unique identifier of type bnet Event Id this is a 64 bit unsigned integer dabc Buffer buf fPool gt Ta
124. eping the number of 16 bit data words behind buffer header The other 16 bits are used for event spanning With a new rule we store this number in 32 bit field 1_free 2 now iUsedWords Only if old 1_dlen now iMaxWords is less equal MAX__DLEN defined as 32K sizeof bufhe 2 this number is also stored in i_used iUsed as before Modifications have to be made in all MBS modules accessing i_used Modules outside MBS can be modified on demand to support large buffers Current buffers still can be handled without change When MBS writes large buffer files only the used part of the file header is written Number of 16 bit words behind buffer header structure is stored in filhe_used Event API _evt is updated to handle large buffers on input Note by setup the number of buffers per stream can be set to one This suppresses event spanning Large buffers can be used by standard MBS 14 9 2 Variable sized buffers As a second step variable sized buffers are implemented They get a new type 100 The allocated buffers are still fixed length as before However the MBS transport would send only the used part of the buffers to clients Processing these buffers a module must first read the header then get the used size from iUsedWords old 1_free 2 and read the rest Modules outside MBS must be modified to process such buffers In MBS after stream buffers are created buffer types are set to 100 by a new command enable dabc in transport This command
125. er network 1 It implements the master state machine of the Bnet The controlling GUI usually sends state machine commmands to the controller node only the Bnet cluster application works as a command fan out and state observer of all worker nodes 9 5 Main Classes 61 2 It controls the traffic scheduling of the data packets between the worker nodes by means of a data flow controller class bnet GlobalDFCModule This controller module communicates with the Bnet sender modules on each worker to let them send their packets synchronized with all other workers 3 It may handle failures on the worker nodes automatically e g by reconfiguring the data scheduling paths between the workers bnet WorkerApplication Subclass of dabc Application to run on the worker nodes of the builder network 1 Implements the local state machine callbacks for each worker with respect to the Bnet functionality 2 It registers parameters to configure the node in the Bnet and methods to set and check these parame ters 3 Defines factory methods CreateReadout CreateCombiner CreateBuilder CreateFilter Cre ateStorage to be implemented in user specific subclass These methods are used in the worker state machine of the Bnet framework bnet GeneratorModule Subclass of dabc ModuleSync Framework class to fill a buffer from the assigned memory pool with generated i e simulated data 1 Method GeneratePacket buffer is to be implemen
126. er change from the invoking control system callback in a thread safe manner bool Unsubscribe dabc Parameter par The subscription of a local parameter par to a remote paramter by a formerly called Subscribe is removed from the control system bool IsMainManager Should return true if this node is the single master controller node of the DAQ cluster This node will define the master state machine that rules the states of all other nodes Otherwise returns false this node is a simple worker node The node properties should be taken from the configuration bool HasClusterInfo Returns true if this node has complete information of the DAQ cluster int NumNodes Returns the number of all DAQ nodes in the cluster This may be taken from a configuration database e g an XML file but may also test the real number of running nodes each time it s called 68 DABC Programmer Manual Manager int Nodeld const Returns the unique id number of this node in the DAQ cluster This should be taken from the cluster configuration bool IsNodeActive int num Returns true if DAQ cluster node of id number num is currently active otherwise false This may allow to check on runtime if some of the configured nodes are not available and should be excluded from the DAQ setup const char GetNodeName int num For each DAQ cluster node of id number num this method must define a unique name representation The name should represent the node i
127. er is called at the end of the infoHandler function of xDimParameter removelnfoHandler xiDimParameter p xiUserInfoHandler ih Interface function to remove an info handler added before protected Vector lt xDimParameter gt getParameterList protected Vector lt xDimCommand gt getCommandList Vector lt xiDimParameter gt getParameters From outside one gets only references to the interfaces Vector lt xiDimCommand gt getCommands From outside one gets only references to the interfaces protected releaseServices boolean cleanup Removes all external handlers of the parameters Sets all parameters to inactive This means that in the infoHandlers no more graphical activity is performed If cleanup is true all parameters release their service and are set to inactive Then the parameter vector is cleared Then the command vector is cleared Note that the objects themselfes are removed only by next garbage collection protected enableServices All parameters are set to active 18 6 2 Getting parameters and commands Once the parameter and command objects have been created by the browser it is up to the xPanelParameter and xPanelCommand object respectively to manage them These two objects are created new each time an update occurs 18 6 2 1 xPanelParameter Extends JPanel It has references to the browser and all graphics panels It owns the parameter table JTable In the constructor the following steps are performed 1
128. erits dabc DataInput and allows to read MBS events from 1md file s and to provide them over input ports into a module It has following parameters e MbsFileName name of Imd file multicast symbols and are supported e BufferSize buffer size to read data CreateDatalnput and CreateDataOutput methods were implemented in mbs Factory class such that a user can instantiate these classes via the DABC plugin mechanism Here is an example how the output file for a generator module can be configured dabc mgr gt CreateModule mbs GeneratorModule Generator dabc Command cmd new dabc CmdCreateTransport Generator Output mbs typeLmdOutput cmd gt SetStr mbs xmlFileName output lmd cmd gt SetInt mbs xmlSizeLimit 100 dabc mgr gt Execute cmd At first the module is created then the type of output transport and its parameters are set via command Another example shows how several input files can be configured for a combiner module dabc Command cmd new dabc CmdCreateModule mbs CombinerModule Combiner cmd gt SetInt dabc xmlNumInputs 3 dabc mgr gt Execute cmd for unisgned n 0 n n lt 3 cmd new dabc CmdCreateTransport FORMAT Combiner Input u n mbs typeLmdInput cmd gt Set Str mbs xmlFileName FORMAT input su_x x 1md n dabc mgr gt Execute cmd In this example one creates a module with 3 i
129. erred to m DABC eventbuilder nodes Example file SDABCSYS applications bnet mbs SetupBnetMbs xml shows the configuration for an MBS event building with 2 DABC readout nodes connected with 2 MBS nodes each simulated by DABC generator modules here and 2 DABC event builder nodes A detailled description of this setup is given in section 15 9 page 117 of the DABC programmer manual The usage of such configuration is similar to the BNET example as described above in section 7 1 page 45 The list of lt Context gt nodes or the corresponding lt Variables gt resp must be edited for the actual node names Additionally the names of the MBS nodes for readout should be specified Then the BNET setup may be launched and controlled by the DABC GUL Chapter 8 DABC User Manual Application ROC user user app roc tex 8 1 DABCas MBS data server The use case here is that a single DABC node should provide data in the MBS event format on a server socket to be used by external analysis and monitoring programs like Go4 1 The event data can be simulated by a generator module A practical case is to read data from any front ends and format it like MBS events This method is used by the ROC readout For the random event generator such set up looks like this lt xml version 1 0 gt lt dabc version 1 gt lt Context host 1xi009 name Server gt lt Run gt lt lib value libDabcMbs so gt lt func value InitMbsGenerator gt lt R
130. es Section 9 5 2 gives a brief overview of the Bnet package classes chapter 15 describes an example using the Bnet plugins 9 4 3 2 Transport packages Depend on the Core system and may depend on external libraries or hardware drivers Implement dabc Device and dabc Transport classes for specific data transfer mechanism e g verbs or tcp ip socket May also implement dabc Device and dabc Transport classes for special data input or output Each transport package provides a 9 5 Main Classes 57 factory to create a specific device by class name However the most common transport implementations are put directly to the Core system e g local memory or socket transport the corresponding factory is part of the Core system then 9 4 4 Application packages They depend on the Core system and may depend on several transport packages on the Bnet package or other plugin packages They may also depend on other application packages Application packages provide the actual implementation of the core interface class dabc Application that defines the set up and behaviour of the DAQ application in different execution states This may be a subclass of specific existing application e g subclass of bnet WorkerApplication Additionally they may provide experiment specific dabc Module classes When installed centrally the Application packages are kept in subfolders of the SDABCSYS applications directory Alternatively an Application packag
131. es text The following section 3 3 page 15 gives a general description of the setup file syntax The GUI may run on a machine with no access to the DABC working directory e g a windows PC Therefore the GUI setup files may use a different working directory containing e Data files for startup panels XML e Configuration files for GUI XML These configuration files for the GUI are described in more detail in Chapter 4 page 21 Of course both setups for the DABC application and the GUI can be put into one working directory if the GUI has access to it 3 3 DABC setup file 15 3 3 DABC setup file The setup file is an XML file in a DABC specific format which contains values for some or all configu ration parameters of the system 3 3 1 Setup file example Let s consider this simple but functional configuration file lt xml version 1 0 gt lt dabc version 1 gt lt Context host localhost name Generator gt lt Run gt lt lib value libDabcMbs so gt lt func value InitMbsGenerator gt lt Run gt lt Module name Generator gt lt Port name 0utput gt lt OutputQueueSize value 5 gt lt MbsServerPort value 6000 gt lt Port gt lt Module gt lt Context gt lt dabc gt This is an example XML file for an MBS generator which produces MBS events and provides them to an MBS transport server This use case is described further in section 8 1 page 47 Other examples of
132. evice gt for the data connection device of the entire DAQ cluster this can be dabc SocketDevice for tcp ip or verbs Device for an InfiniBand cluster e Several Worker nodes of an experiment specific lt Application gt They may be configured for different jobs in the BNET this example provides an Application class bnet TestWorker with some boolean parameters to define the functionality Note the usage of wildcards in the lt Context gt names to define properties that should be valid for all nodes matching the pattern e g the libraries to load or the common application setup for all worker nodes Here there are 4 workers which all produce random event data enabled in lt IsGenerator gt and all send their data to all others enabled in lt IsSender gt In parallel they all receive data from the other workers to build the complete event enabled in lt IsReceiver gt Such BNET setup is best started by means of the DABC GUI The name of the controller lt Context gt node and the setup file name must be specified in the control panel of the GUI see section 4 2 2 page 23 Then all nodes can be started just by the Launch button ES The configuration and run control of the nodes is done by the state machine buttons of the control panel 7 2 DABC eventbuilder network BNET with MBS A more realistic example of a BNET uses data which is read from n external MBS nodes each connected to one DABC readout node and transf
133. ew buffer virtual method ProcessInputEvent will be called User should reimplement this method to react on the event One should use dabc ModuleAsync for situations when simple main loop approach is not possible for instance one cannot decide on which input next data is exepected Also the main advantage of such approach is that the thread is not blocked and several dabe ModuleAsync modules can run within same working thread At the same time using such programming technique may require additional bookkeeping as it is not allowed to block the callback routine while waiting for some resource to be available Class dabc ModuleSync provides number of methods for handling different events Method Description ProcessInputEvent new buffer in input queue it can be read with port gt Recv ProcessOutputEvent new space in output queue is available one can use port gt Send ProcessConnectEvent port is connected to transport ProcessDisconnectEvent port was disconnected from transport ProcessPoolEvent requested buffer can be read with handle gt TakeRequestedBuffer ProcessTimerEvent timer has fired an event By reimplementing some of these methods one can react on corresponding events Actually all events are dispatched to the methods mentioned above by method ProcessUserEvent This method is called by the working thread whenever any event for this module shall be processed However this virtual method may also directly
134. f gt GetDataSize if fxBoard gt fillData charx buf gt GetDataLocation fullsz return dabc DataInput di_SkipBuffer if fullsz 0 return dabc DataInput di_SkipBuffer buf gt SetTypeld roc rbt_RawRocData buf gt SetDataSize fullsz return dabc Datalnput di_Ok The return value dabc Datalnput di_ CallBack of function Read_Start indicates that processing of this trans port should be suspended because the requested amount of data is not ready yet When all this data has been received by the ROClib it will invoke method SysCoreControl DataCallBack which is reimplemented in sub class roc Device simply forwarding to the following method of roc Transport void roc Transport CompleteBufferReading unsigned res Read_Complete fCurrentBuf Read_CallBack res As the required amount of data is ready now one only retrieves it to the current buffer with the same Read_Complete method and reactivates the processing of this transport instance by calling Read_CallBack 16 3 Combiner module 121 16 3 Combiner module Class roc CombinerModule combines data from several ROC boards in one MBS event It also performs sorting of data according the timestamp resolves the last epoch bits and fixes several coding errors class SysCoreSorter is used for this The module has following configuration parameters e NumRocs number of ROC boards connected to combiner default 1 e BufferSize
135. first argument is the name of a command parameter and the second is the new parameter value of the corresponding type In all getter methods the first argument is again the parameter name and the second is an optional default parameter value This default value is returned if a parameter of that name is not contained in the Command To instantiate a command one should do dabc Command cmd new dabc Command UserCommand cmd gt SetInt UserArg 5 11 3 Command execution 77 Usually the name of a Command defines the action which will be performed There are several subclasses of dabc Command for instance in file dabc Manager h but these subclasses are only used to set the command name and command specific parameters There is no sense to define some extra methods in the subclass since dabc Command is designed as a mere container for parameters With method ConvertToString one can convert a Command and all contained parameters in a plain string Method ReadFromString is used to reconstruct the Command object from a string This feature is useful to transfer a Command over a network connection or store it to a file 11 3 2 Command receiver Class dabc CommandReceiver provides the interface for all classes which should execute a Command The main place for user code is virtual method ExecuteCommand which gets a Command object as argument A typical implementation of this method looks like int UserModule Execut
136. g and visualization of such records 18 2 2 1 Record ID 0 Plain Scalar data item of atomic type 18 2 DIM Usage 135 18 2 2 2 Record ID 1 Generic self describing For these one would need one structure per number of arguments Therefore the generic type would be rather realized by a more flexible text format like XML This means the DIM service has a string as argument which must be parsed to get the values XML schema char similar to command descriptor Format C 18 2 2 3 Record ID 2 State severity int O Success 1 warning 2 error 3 fatal color char Red Green Blue Cyan Magenta Yellow state char name of state Format L 1 C 16 C 16 18 2 2 4 Record ID 3 Rate value float displaymode int arc bar statistics trend lower limit float upper limit float lower alarm float upper alarm float color char Red Green Blue Cyan Magenta Yellow alarm color char Red Green Blue Cyan Magenta Yellow units char Format F 1 L 1 F 1 F 1 F 1 F 1 C 16 C 16 C 18 2 2 5 Record ID 4 Histogram Structure must be allocated including the data field witch may be integer or double channels int lower limit float upper limit float axis lettering char content lettering char color char White Red Green Blue Cyan Magenta Yellow first data channel int Format L 1 F 1 F 1 C 32 C 32 C 16 I or D 18 2 2 6 Record ID 10 Info verbose int 0 Plain text 1 Node text color char Red Green Bl
137. gger Command thread Command F loop command queue to pid Task Dispatcher u dabcstop Hold acquisition a disconn Shut down all e exitall Exit all E fileclose Close filesave Save dal histowin Histogram panel 6 info Show acquisition infowin Info panel El logwin Log panel O mbsconfig Configure mbsicon MBS launcher O mbsstart Start acquisition 0 mbsstop Stop acquisition Ao meterwin Rate meter panel E paramwin Parameter table Figure 14 2 Multi node MBS controlled by TCP 110 DABC Programmer Manual Example MBS message queue to 2000 Prompter Dispatcher Master Logger command queue to pid TCP 6004 message queue to 2000 Logger TCP 6005 Dispatcher TCP 6004 command queue to pid Figure 14 3 Multi node MBS controlled by DIM E rshmbx MBS remote shell script T savewin Save settings g statewin State panel usergraphics Graphics panel usericon Parameter selection panel ES usericonblue User panel E usericonred User panel ESl H usericongreen User panel usericonyellow User panel user Windows windowclose Windows close Ej windowblue Graphics window Ej windowred Graphics window 14 10 List of icons 111 CIJ E windowgreen Graphics window 112 DABC Programmer Manual Example MBS
138. he name will be created If thrdname is not defined DABC will use module name for it Returns true or false depending on the instantiation success bool CreateDevice const char classname const char devname Instantiate a Device of class classname with the object name devname Returns true or false depending on the instantiation success bool CreateTransport const char portname const char transportkind const char thrdname 0 Instanti ate a Transport of the specified kind transportkind e g mbs ServerTransport and connect it to the port of full name portname e g Readout Inputl Kind can specify device name which than create appro priate transport instance Optionally the name of the working thread thrdname may be specified that shall run this transport If a thread of this name is already exisiting it will be also applied for the new transport otherwise a new thread of the name will be created If thrdname is not defined DABC will use a new thread automatically with an internal name Returns true or false depending on the instantiation success bool CreateApplication const char classname 0 const char appthrd 0 Instantiate the Application of class classname Optionally the name appthrd of the main application thread may be specified Used in the main function of the DABC runtime executable on inititialization time 10 2 3 Module manipulation void StartModule const char modulename Enables the module
139. hine reached current state Configu a DABC Ixg0523 gsi de 1969 Controller 41 DoEnable il DABC IxIO11 g5i de 1970 Readout 42 statusMessage State machine reached current state Ready DABC IxIO10 g5i de 1970 Readout 42 statusMessage State machine reached current state Ready DABC Ixi013 gsi de 197 1 Eventbuilder 42 statusMessage State machine reached current state Ready DABC IxiO14 g5i de 197 1 Eventbuilder 42 statusMessage State machine reached current state Ready DABC Ixg0523 gsi de 1969 Controller 41 statusMessage State machine reached current state Ready DABC IlxgO0523 gsi de 1969 Controller 41 statusMessage Setting dim service Setup_File to mySetup DABC IxgO0523 gsi de 1969 Controller 41 Setup_File mySetup 4 Figure 4 13 Logging 4 3 GUI save restore setups There are several setups which can be stored in XML files and are retrieved when the xGUI is started again The file names can be specified by shell variables DABC CONTROL_DABC Values of DABC control panel Saved by button in panel Default DabcControl xml Filename in panel itself DABC CONTROL_ MBS Values of MBS control panel Saved by button in panel Default MosControl xml Filename in panel itself DABC_RECORD_ ATTRIBUTES Attributes of records Saved by main save button Default Records xml DABC PARAMETER _ FILTER Values of parameter filter panel Saved by main save button Default Selection xml DABC_GUI_LAYOUT Layo
140. i Transport This is required at Device initialization time the mapping must be re freshed on the fly if the memory pool changes though Method GetDMA Buffer dabc Buffer will deliver for each dabc Buffer of the mapped memory pool the corresponding mprace DMA Buffer object to be used in the underlying mprace library These are asso ciated by the dabc Buffer id number which defines the index in the std vector keeping the mprace DMABuffer handles Method DoDeviceCleanup is implemented for a proper cleanup of the mapped DMA buffers when the Device is removed by the framework 5 Reading data from the board Method ReadPCI dabc Buffer implements reading one buffer from PCI using the BAR the PCI start address and the read size as specified before These read parameters may be either set by method SetReadBuffer unsigned int bar unsigned int address unsigned int length or by submitting the corresponding command CommandSetPCIReadRegion to the pci Device If DMA mode defined in the constructor is not enabled this will just use PIO to fill the specified dabc Buffer from the PCI address range If DMA mode is enabled it will perform DMA into the user space dabc Buffer this must be taken from a memory pool that was mapped before by means of MapDMABuffer This is a synchronous call that will initiate the DMA transfer and block until it is complete For asynchronous DMA double buffering of dabc DataTransport following virtual
141. icated control panel is provided by the MBS application This panel is described in the MBS section 6 1 3 page 41 4 2 6 Command panel The control system of DABC and or the application specific plug ins can define commands These commands are encoded as DIM services including a full description of arguments Therefore the GUI can build up at runtime a command tree and provide the proper forms for each command Commands are executed in all components of DABC The DABC naming convention for commands and parameters defines four main name fields separated by slashes 1 DIM server name space example DABC 2 Node example 1xg0523 3 Application example Controller 41 4 Name example doEnable 26 DABC User Manual GUI 1 Commands 3 DoConfigure 9 EI DoEnable EI Controller 41 1x90523 95i de 1969 co 9 EI Eventbuilder 42 1xi013 gsi de 1971 Ixi014 gsi de 1971 amp 3 Readout 42 Figure 4 5 Command panel Example DABC Ixg0523 Controller 41 doEnable Fig 4 5 page 26 shows on the left side the com mand tree The tree is built from name application nodes Double click or RETURN on a treenode executes the command on all treenodes below A click on a command opens at the right side the ar gument panel Entering argument values and RETURN executes the command In the example shown in the figure double click on doEnable would execute that command on three nodes Double click on Eventb
142. in Context User section If Context Run copycfg true config file will be copied to working directory of specified node useful for cluster without common file system Implement all to all and multicast tests in net test application Bugfix several minor errors in Verbs plugin Bugfix suppress output of scripts running from ssh caused problems with GUID Bugfix GUI Register DIM service after full instantiation of parameter object Bugfix GUI Histogram drawer had uninitialized field 1 2 2 Version 1 0 00 26 February 2009 These are the features of the first official release 1 2 A Data Acquisition framework in C language for linux platforms with modular components for dataflow on multiple nodes Runtime environment with basic services for threads event handling memory management com mand execution configuration logging error handling Plug in mechanism for user defined DAQ applications Plug in mechanism for a control system Features a finite state machine logic and parameters for monitoring and configuration The default implementation is based on the DIM protocol http dim web cern ch dim Java GUI to operate the standard DIM control system of DABC MBS Fully generic evaluating DABC process variables but extendable by user written components Contains a sub framework to set up distributed event builder networks BNET Supports TCP IP and InfiniBand verbs networks for data
143. ion e 10 2 4 1 COTE SYSICM coc a ee ee 10 2 4 2 Control and configuration system ee 10 111 iv CONTENTS 2 4 3 Plug in packages coord aa eee EA ae eae 10 243 1 Wet packae6 4 2 4a ka an a Daa a eRe wee a edb a eed BS 11 2432 Transport packages acco va ia ee ea we 11 2 4 4 Application packages 2 0 ua nun en ee a a en 12 2 4 5 Tistnib kon contents gt ae a vate ba eG Rae EER ad OR ea He ea He 12 3 DABC User Manual Setup 13 al Unetellinge DADC a ee 13 32 Set pthe DABE environment e co ea be au Sa ea ea 14 33 DABC mpe o oo ke A Oe ei ee ae ee 15 dl SCLUp Me EXA oee ee ee eg ee 15 ae BASIC ANAL o iati eae dk a eek Ad Aa eb oe ws add ald ed wed 15 ee CODER ee che enue node cack ce we dk a ae che Mw a Ge a wears wand 15 3 3 4 Kun PIE a ee ara 16 uae Vorab aaa ace ee ahd bbe Bae PE Re eee be ee barks oe 16 3 3 6 Tretia VANS a ee a e ee ee Pa ee eee Ae eM 17 3 4 Installation of additional plug ins o o e e 18 3 4 1 Add plug in packages to SDABCSYS 0 coo oonoooooso ss ss 18 3 4 2 Plug in packages in User directory gt su nun an anne ee 19 4 DABC User Manual GUI 21 4 CeCe MES o es ok a ed we bee hae ae i i a ed ble owed 21 42 SAUD Pele n doe en o wk Sw Ge add we eae RA 21 4 2 1 Mian DABE GUL BUCKS oie a doe Se a a A a 22 4 2 2 DABC control patel 2 2 ga ara ba ea Ba edGae Sada vena ed Pa 23 4221 DABC controller buttons s so es cem sa
144. is kind of actions the DABC state machine has two transition commands DoConfigure and DoEn able first command used to create necessary components and second to connect them together Accordingly class dabc Application has two methods CreateAppModules and ConnectAppModules see 12 4 12 3 5 Data transport In general to implement a user specific transport one should subclass from dabc Transport But this requires a deeper knowledge about the DABC mechanisms how threads are working how one should organize input output queues how the transport should request data from a memory pool and which initialization commands are used by the framework To simplify transport development and provide all basic services class dabc DataTransport was introduced For a data input the user should implement the following virtual methods Read_Size Should return the required buffer size to read next portion of data from the data source For instance many file formats have a header before each portion of data describing the payload size that follows This method then should be used to read such header Method can also return following values dabc di_EndOfStream end of stream normal close of the input dabc di Repeat nothing to read now call again as soon as possible dabc di RepeatTimeout nothing to read now try again after timeout dabc di_ Error error close transport Read_Timeout Defines timeout in seconds for operation like R
145. it 65 Subscribe 67 Unsubscribe 67 abb Device 127 abb Factory 128 abb ReadoutModule 128 abb WriterModule 128 DMA 126 127 pei BoardDevice 125 pei Transport 126 PCI express 125 TODO Adjust old mbs bnet configurator scripts for new xml format 46 dabcsetupfiles 40 Mbs BNET example with real mbs nodes instead generators 46 xPanelGraphics 138
146. ize of the segment Usually a dabc Buffer contains just one segment which fully covers a complete subblock of the MemoryPool for instance when a new Buffer is requested with method MemoryPool TakeBuffer Methods NumSegments and Segment unsigned provide access to the list of segments One can also directly access the pointer and the size of each segment via methods GetDataLocation unsigned and GetDataSize unsigned respectively For instance filling a complete Buffer with zeros will look like this include dabc Buffer h void UserModule ProcessOutputEvent dabc Portx port dabc Buffer buf Pool gt TakeBuffer 2048 memset buf gt GetDataLocation 0 buf gt GetDataSize port gt Send buf It is also possible to create a Buffer object that reference the same memory of another Buffer by means of method Buffer MakeReference This will deliver the pointer to a new dabc Buffer instance with the same list of segments as the original instance It will also increment the reference counter for all used subblocks in the MemoryPool This method should be used e g to send the same data over several Ports one just makes as many reference Buffers as required and sends them to all destinations independently without copying the data For instance a simplified version of dabc Module SendToAllOutputs will look like void dabc Module SendToAllOutputs dabc Bufferx buf 11 1 Memory management 73
147. keBuffer bufsize buf gt SetHeaderSize sizeof bnet EventId x bnet Eventldx buf gt GetHeader evid The subevent identifier number should be subsequently increasing without a gap When no data for the current identifier is available an empty buffer with no data and correct header must be delivered to the output Class bnet CombinerModule provides the prototype of a combiner module It uses the following single parameter Name Type Dfit Description NumReadouts int 1 number of data inputs Actually parameter NumReadouts may not be defined in the configuration of the module itself Since class bnet WorkerApplication already has a parameter of such name its value will be directly used for the module configuration When implementing an experiment specific combiner class one should either derive it from bnet CombinerModule class or start from scratch by subclassing dabc ModuleSync or dabc ModuleAsync One may add more experiment specific parameters to the module 15 5 Network topology The connection topology of the event building network is defined by parameters IsSender and IsReceiver of bnet WorkerApplication These parameters configure the roles of each worker node e collector of data from data source s and sender to event builder e receiever of data from collectors and builder of complete events e both functions at the same application It is required that at least one of both para
148. l such separate plug in packages Either within the general DABCSYS directory as part of the central DABC installation as described in following section 3 4 1 page 18 Or at an independent location in a user directory as described in section 3 4 2 page 19 3 4 1 Add plug in packages to DABCSYS This is the recommended way to install a plug in package if this package should be provided for all users ofthe DABC installation A typical scenario would be that an experimental group owns dedicated DAQ machines with system manager priviliges In this case the plugin package may be installed under the same account as the central DABC installation probably but not necessarily even the root account The new plug in package should be directly installed in the DABCSYS directory then with the following steps 1 Download the plug in package tarball e g abbl tar gz 2 Call the dabclogin sh script of the DABC installation see section user env 3 Copy the downloaded tarball to the SDABCSYS directory and unpack it there cp abbl tar gz SDABCSYS cd SDABCSYS tar zxvf abbl tar gz This will extract the new components into the appropriate plugins and applications folders below SDABCSYS 3 4 Installation of additional plug ins 19 4 5 Build the new components with the top Makefile of SDABCSYS make To work with the new components the configuration script s of the applications part should be copied to the personal workspace of ea
149. l visibility level for parameters aka debug level This global level can be changed by WorkingProcessor SetGlobalParsVisibility static function or in configura tion file value Context Run parslevel Normally module parameters has visibility level 1 module items port pool handle parameters 3 configuration parameters 5 Thus to see all parameters in control system one should set parslevel 5 13 3 Example of parameters usage Let s consider an example of a module which uses parameters class UserModule public dabc ModuleAsync public UserModule const char name dabc Command cmd 0 dabc ModuleAsync name cmd CreateParBool Output true CreateParInt Counter 0 CreateTimer Timer 1 0 false virtual void ProcessTimerEvent dabc Timerx SetParlnt Counter GetParlnt Counter 1 if GetParBool Output DOUT1 Counter d GetParlnt Counter y In the module constructor two parameters are created boolean and integer and a timer with 1 s period When the module is started the value of integer parameter Counter will be changed every second If boolean parameter Output is set to true the counter value will be displayed on debug output Using a control system the value of the boolean parameter can be changed To detect and react on such change one should implement following method virtual void ParameterChanged dabc Parameterx par if par gt
150. lass 62 DABC Programmer Manual Overview bnet FilterModule Subclass of dabc ModuleSync Framework prototype class to filter out the incoming physics events according to the experiment s trigger conditions 1 Has one input port to get buffers with already tagged physics events from the preceding Bnet builder module 2 Has one output port to connect a user defined output or storage module resp 3 Method TestBuffer buffer is to be implemented in user defined subclass e g bnet MbsFilterModule and is called in module s MainLoop for each incoming buffer Method should return true ifthe event is good for further processing 4 Forwards good buffers to the output port and discards others Chapter 10 DABC Programmer Manual Manager programmer prog manager tex 10 1 Introduction The dabc Manager is the central singleton object of the DABC framework It combines a number of different roles such as objects manager memory pools manager threads manager commands dispatcher run control state manager plug in manager for factories and application implementation of control and configuration system OOF 0 OF 0 CO O Although these functionalities internally could as well be treated in separate classes dabc Manager class defines the common application programmer s interface to access most of these features Since the manager is a singleton these methods are available everywhere in the user code b
151. lass dabe DataTransport implicitly provides a double buffering mechanism here unsigned Read_Complete dabc Buffer buf Will wait until filling the buffer buf from a DMA read operation is completed The DMA either must have been started asynchronously by a previous Read_Start call or it must be started synchronously here This method is used by the base class for synchronization between transport thread and DMA engine of the PCI board The functionality is forwarded to ReadPCIComplete of pci BoardDevice bool WriteBuffer dabc Buffer buf Write content of buf to the PCI region as set for the BoardDevice with SetWriteBuffer or CommandSetPCIWriteRegion resp This is a pure synchronous method i e it will start the DMA transfer and return no sooner than it s completed The functionality is forwarded to WritePCI of pci BoardDevice void Process PoolChanged dabc MemoryPool pool Is called by the framework whenever the memory pool associated with the transport instance changes e g at transport connection time pool expansion etc It calls MapDMABuffers of pci BoardDevice to rebuild the scatter gather mappings for each buffer of the pool 17 3 Active Buffer Board implementation 17 3 1 abb Device This subclass of pci BoardDevice adds some functionality that is rather specific to the ABB hardware and the test environment 1 The constructor instantiates the mprace Board component for the ABB functionalities Additi
152. le bnet FilterModule In this case the user specific parts like data formats are implemented by subclassing these special module classes 1 Instead of implementing MainLoop or ProcessUserEvent resp other virtual methods e g DoBuildE vent TestBuffer may be implemented that are implicitly called by the superclass MainLoop or by the appropriate event callbacks resp 2 The special base classes may provide additional methods to be used for data processing 12 3 Device and transport All data transport functionality is implemented by subclassing dabc Device and dabc Transport base classes 12 3 1 Transport Actual transport of Buffers from to a Port is done by a dabc Transport implementation During connection time each module port gets the pointer to a transport object which provides a number of methods for buffer transfer As the Transport object typically runs in another thread than the module the transmission of a buffer does not happen immediately when calling dabc Port Send or dabc Port Recv methods but the buffer is at first kept in a queue which must be provided by the Transport implementation 12 3 2 Device Class dabc Device usually but not always represents some physical device like a network or a PCIe card and has the role of a management unit for the Transports which belong to that device The Device is always the owner of its Transport objects i e it creates keeps and deletes them A
153. le ROC unsigned roc Transport Read_Size return fBufferSize When the system has delivered a buffer of the requested size function Read_Start is called to start reading of that buffer from the data source unsigned roc Transport Read_Start dabc Buffer buf int req fxBoard gt requestData fReqNumMsgs if req 2 return dabc Datalnput di_CallBack if req 1 return dabc Datalnput di_Ok return dabc Datalnput di_Error The SysCoreBoard accessed by pointer xBoard keeps internally own buffers of the received UDP messages The call SysCoreBoard requestData informs by return value either that the required number of messages is already received or that the caller should wait i e poll this method here until it returns that all data is ready However waiting would mean that the working thread is blocked and could not run the other transport instances Therefore another approach is used the ROClib will call back the virtual method SysCoreControl DataCallBack when the required amount of data is there This method is implemented for roc Transport to complete filling the current dabc Buffer If data already exists in the internal buffers of SysCoreBoard the value dabc Datalnput di_Ok is returned then the DataTransport framework will immediately call Read_Complete which finally fills the DABC buffer unsigned roc Transport Read_Complete dabc Bufferx buf unsigned fullsz bu
154. left it w Test shell script a Save settings window layout record attributes command arguments parameter selection filters Details see 4 3 page 30 Note that the content of the control panels must be saved by similar buttons in these panels T Open DABC MBS control panel see 6 1 3 page 41 Open DABC control panel see 4 2 2 page 23 Open MBS control panel see 5 1 2 page 31 8 Refresh All parameters and commands are removed Rebuild DIM service list from DIM name server Parameters and Commands are sorted alphabetically by name All panels are updated In normal operation there is no need to refresh manually Ooo ri Ep Open command panel 4 2 6 page 25 E Open parameter table 4 2 7 page 26 Open parameter selection panel 4 2 7 1 page 27 Open rate meter panel 4 2 8 page 27 Ma Open histogram panel 4 2 8 page 27 3 Open state panel 4 2 8 page 27 Open info panel 4 2 8 page 27 Open log panel 4 2 8 page 27 ES Eventually one might see additional icons from application panels this one is only an example The three control panels DABC MBS combined DABC and MBS are used depending on the ap plication to be controlled Eventually an application provides additional specific control panels 4 2 2 DABC control panel The standard DABC control panel is shown in 4 3 page 24 As mentioned already some applications may provide their own control panels like the MBS applications
155. libDabcMbs so gt lt lib value libDabcKnut so gt lt logfile value Readout log gt lt Run gt lt Application class roc Readout gt lt DoCalibr value 0 gt lt NumRocs value 3 gt RocIp0 value cbmtest01 gt RocIpl value cbmtest02 gt RocIp2 value cbmtest04 gt BufferSize value 65536 gt NumBuffers value 100 gt TransportWindow value 30 gt RawFile value run090 1md gt bsServerKind value Stream gt bsFileSizeLimit value 110 gt lt Application gt lt Context gt lt dabc gt A Here the parameters are defined for the lt Application gt instance roc Readout that controls the readout of 3 ROC nodes via UDP and combines the data into one MBS event by means of some internal Modules Hence there is no simple run function as before the DABC runtime environment will call appropriate methods of the Application to configure and run the set up Note that in this case the MBS data is not only provided to a stream server as defined in lt MbsServerKind gt but is also written to a x 1md list mode data file which can be specified in application parameter lt RawFile gt Both single node examples above do not require to be launched from the DABC GUI although this is possible and may be useful to monitor the data rates and actual parameters They can be started directly from a shell by calling the standard dabc_run executable with the configuration file name
156. llation 2 4 3 1 Bnet package This package depends on the Core system and implements modules to cover a generic event builder network It defines interfaces virtual methods of the special Bnet modules to implement user specific code in subclasses The Bnet package provides a factory to create specific Bnet modules by class name It also provides application classes to define generic functionalities for worker nodes and controller nodes These may be used as base classes in further Application packages 2 4 3 2 Transport packages Depend on the Core system and may depend on external libraries or hardware drivers Implement Device and Transport classes for specific data transfer mechanism e g verbs or tcp ip socket May also implement Device and Transport classes for special data input or output Each transport package provides a factory to create a specific device by class name However the most common transport implementations are put directly to the Core system e g local memory or socket transport the corresponding factory is part of the Core system then 12 DABC User Manual Introduction 2 4 4 Application packages They depend on the Core system and may depend on several transport packages on the Bnet package or other plugin packages They may also depend on other application packages Application packages provide the actual implementation of the core interface class Application that defines the set up and behaviour of the DA
157. ls to corresponding methods of the dimc Registry and treated there such as o commandHandler to HandleDIMCommand errorHandler to OnErrorDIMServer clientExitHandler to OnExitDIMClient exitHandler to OnExitDIMServer O O O 10 3 3 4 dimc ServiceEntry This is a container to keep the DimService together with the corresponding dabc Parameter object and some extra properties The dimc ServiceEntry objects are managed by the dimc Registry and applied for the RegisterPa rameter method 1 For std string parameters an internal char array is used as buffer which is actually exported as DIM service 2 Method UpdateBuffer updates the DIM service it optionally may copy the parameter contents to the buffer before 3 Method SetValue sets the dabc Parameter to a new value as defined by a string expression 10 3 3 5 dimc ParameterInfo A subclass of DIM class DimStampedInfo which subscribes to be informed if a remote DIM service changes its value The dimc ParameterInfo objects are managed by the dimc Registry and applied for the Subscribe method 1 The dimc ParameterInfo has a reference to a local dabc Parameter object that shall be updated if the subscribed service changes 2 Depending on the subscribing dabc Parameter type integer double string the constructor will instan tiate an appropriate DimStampedInfo type 3 Method infoHandler of DimStampedInfo is implemented to update the par
158. lt Context host 1xi008 name Controller 41 gt lt Run gt 15 9 BNET for MBS application 117 lt runfunc value RunTestBnet gt lt Run gt lt Application class bnet Cluster gt lt NetDevice value dabc SocketDevice gt lt Application gt lt Context gt lt Context host 1xi009 name Worker1 42 gt lt Context host 1xi010 name Worker2 42 gt lt Context host 1xi011 name Worker3 42 gt lt Context host 1xi012 name Worker4 42 gt lt Defaults gt lt Context name x gt lt Run gt lt logfile value test DABCNODEID log gt lt loglevel value 1 gt lt lib value libDabcBnet so gt lt Run gt lt Context gt lt Context name Workerx gt lt Run gt lt lib value S DABCSYS applications bnet test libBnetTest so gt lt Run gt lt Application class bnet TestWorker gt lt IsGenerator value true gt lt IsSender value true gt lt IsReceiver value true gt lt NumReadouts value 4 gt lt Application gt lt Context gt lt Defaults gt lt dabc gt Here one can see cluster controller apllication in the beginning configured to use dabc SocketDevice for work ers connections And there are four workers with the same configurations parameters which can be found in lt Defaults gt section In section lt Context name Worker gt one sees that IsGenerator IsSender and IsReceiver parameters are all set to true This
159. may register dabe Command objects in the constructor and may define command actions by overwriting a virtual command callback method ExecuteCommand 9 2 3 Parameters A module may register dabc Parameter objects Parameters are accessible by name their values can be monitored and optionally changed by the controls system Initial parameter values can be set from xml configuration files 9 2 4 Manager The modules are organized and controlled by one manager object of class dabc Manager this singleton instance is persistent independent of the application s state One can always access the manager via dabc mgr function The manager is an object manager that owns and keeps all registered basic objects into a folder structure Moreover the manager defines the interface to the control system This covers registering sending and receiving of commands registering updating unregistering of parameters error logging and global error handling The virtual interface methods must be implemented in subclass of dabc Manager that knows the specific controls framework The manager receives and dispatches commands to the destination modules where they are queued and eventually executed by the modules threads see section 9 2 1 The manager has an independent manager thread used for manager commands execution parameters timeout processing and so on 9 2 5 Memory and buffers Data in memory is referred by dabc Buffer objects Allocated memory
160. meters has a true value During configuration the cluster controller establishes the connections between the workers such that each sender module is connected with all receiver 116 DABC Programmer Manual Example Bnet modules This guarantees that each receiever node gets data from all sources necessary to perform the full event building The two classes bnet SenderModule and bnet ReceiverModule implement the functionality of data sender and data receiver respectively These classes are instantiated by bnet WorkerApplication and should not be modified by the user The subevents buffers as produced by the combiner module are delivered to the sender module Based on the event identifier the buffer is send to that specific receiver where the event with such id will be build For now a simple round robin schedule is used by BNET but in next DABC versions one or several other data transfer schedules will be implemented One idea of the BNET framework is that such improvements are possible without changing the user application code 15 6 Event builder module The task of the receiver module is to collect all buffers of the same event identifier and deliver them at once to the event builder module To define an experiment specific builder module one can either derive it from bnet BuilderModule class or implement it from scratch by subclassing dabc ModuleSync or dabc ModuleAsync The event builder module has one input and one out
161. n a human readable way e g by means of URL and a functional node description dagqO1 gsi de readout It should match the description in the cluster configuration Note This name must match the local name of the manager object on each node bool SendOverCommandChannel const char managername const char cmddata This method sends a dabc Command as a streamed text representation cmddata to a remote DAQ cluster node of name managername The managername argument must match one of the names defined in GetN odeName int num The implementation should use transport mechanisms of the control system to transfer the command string to the remote site e g native control commands that wrap cmddata The receiver of such commands on the target node should call base class method RecvOverCommandChannel const char cmddata to forward the command representation to the core system which will reconstruct and execute the dabc Command object bool CanSendCmdToManager const char mgrname Returns true if it is possible to send a remote command to the manager on DAQ cluster node of name mgrname otherwise false The node name argument must match one of the names defined in GetNodeName int num This method may implement to forbid the sending of commands on some nodes int ExecuteCommand dabc Command cmd This method executes synchronously any DABC command that 1s submitted to this manager itself It will run in the scope of the manager thread dependi
162. n file parser are already provided by the Core system 56 DABC Programmer Manual Overview User Application User Plug ins Cc O 0 O o 9 Q 5 o e 7 D O 5 Figure 9 2 Schematic view of the distributed DABC components coloured and user specific extensions white 9 4 3 Plugin packages Plugin packages may provide special implementations of the core interface classes dabc Device dabc Transport dabc Module or dabc Application Usually these classes are made available to the system by means of a corresponding dabc Factory that is automatically registered in the dabc Manager when loading the plugin library When installed centrally the Plugin packages are kept in subfolders of the DABCSYS plugins directory Alternatively the Plugin packages may be installed in a user directory and linked against the Core system instal lation 9 4 3 1 Bnet package This package uses namespace bnet It depends on the Core system and implements modules to cover a generic event builder network It defines interfaces virtual methods of the special Bnet modules to implement user specific code in subclasses The Bnet package provides a factory to create specific Bnet modules by class name It also provides application classes to define generic functionalities for worker nodes bnet WorkerApplication and controller nodes bnet ClusterApplication These may be used as base classes in further Application packag
163. n for data output to PCI since this is a rare use case for a DAQ system 17 2 2 pci Transport This class handles the connection between the Port of a module and the PCI device It is created in CreateTrans port of pci BoardDevice when the user application calls the corresponding Manager method with the names of the port and the device instances e g dabc mgr gt CreateTransport ReadoutModule Input AbbDevice3 It extends the base class dabc DataTransport which already provides generic Buffer queues with a data back pressure mechanism both for input and output direction Because this class is also a WorkingProcessor each pci Transport object has a dedicated thread that runs the IO actions The following virtual methods of dabc DataTransport were implemented unsigned Read_Size Returns the size in byte of the next buffer that is to be read from board Uses the current readout length as set for the pci BoardDevice with SetReadBuffer or CommandSetPCIReadRegion resp unsigned Read_Start dabc Buffer buf This initiates the reading into buffer buf and returns without waiting 17 3 Active Buffer Board implementation 127 for completion The functionality is forwarded to ReadPCIStart of pci BoardDevice When Read_Start returns the transport thread can already push the previously filled DMA buffer to the connected Port which may wake up the waiting thread of its Module for further processing Thus base c
164. n state setSelected void toFront String title Set frames to front 18 7 2 2 Interface xiDimBrowser o Vector lt xiDimParameter gt getParameters Typically called in setDimServices to get list of available parameters Only selected parameters may be regis tered to Vector lt xiDimCommand gt getCommands Typically called in setDimServices to get list of available commands void setInfoHandler xiDimParameter p xiUserInfoHandler h Typically called in application function setDimServices to register a call back handler mostly this to a parameter void removelnfoHandler xiDimParameter p xiUserInfoHandler h Typically called in application function releaseDimServices to remove a call back handler of a parameter void sleep int s 18 7 2 3 Interface xiDimCommand o o void exec String command xiParser getParserInfo 18 7 2 4 Interface xiDimParameter 0 0 10 20 70 O10 0 OO double getDoubleValue float getFloatValue int getIntValue long getLongValue String getValue xRecordMeter getMeter xRecordState getState xRecordInfo getInfo xiParser getParserInfo boolean parameterActive boolean setParameter String value Builds and executes a DIM command SetParameter name vale where name is the name part of the full DIM name string 18 7 Application specific GUI plug in 145 18 7 2 5 Interface xiParser String getDns String getNode String getN
165. nPerformed function which is the central call back function for all elements 18 3 2 Graphics panels Graphics panels are provided by class xPanelGraphics The layout is as a matrix with columns and rows All items to be added must be JPanels and implement xiPanelltem see below The items are added line by line The number of items per line columns is a parameter All items must have the same size Currently no menu bar is supported 18 4 GUI Panels Brief description of panels implemented in the xGUI 18 4 1 DABC launch panel xPanelDabc extending xPanelPrompt Form to enter all information needed to startup DABC tasks and buttons to execute standard commands The values of the form internally stored in xFormDabc extending of xForm can be saved to an XML file and are restored from it File name is either DabcLaunch xml or translation of DABC_LAUNCH_DABC respectively 18 4 2 MBS launch panel xPanelMbs extending xPanelPrompt Form to enter all information needed to startup MBS tasks and buttons to execute standard commands The values of the form internally stored in xFormMbs extending of xForm can be saved to an XML file and are restored from it File name is either MosLaunch xml or translation of DABC_LAUNCH_ MBS respectively 18 4 3 Combined DABC and MBS launch panel xPanelDabcMbs extending xPanelPrompt It is a combination of both DABC and MBS launch panel 138 DABC Programmer Manual GUI 18 4 4 Parameter table
166. nd parameters This should be done after the control system is ready for handling parameters and commands and after the optional InitSMmodule call Manager The destructor of the subclass It should cleanup and remove the control system implementation It must call method destroy at the end bool InvokeStateTransition const char state_transition_name Command cmd This should initiate the state transition for the given state_transition_name This must be an asynchronous function that does not block the calling thread possibliy the main manager thread if the state transition is triggered by a command from a remote master state machine node Thus the actual state transition should be performed in a dedicated state machine thread calling the synchronous method DoStateTransition const char of the base class see section 10 3 2 2 Synchronization of the state with the invoking client is done by the passed command object reference cmd This should be used as handle in the static call dabc Command Reply cmd true when the state transition is completed or dabc Command Reply cmd false when the transition has been failed resp Note that base class dabc Manager already implements this method for the DABC default state machine module which is activated in the manager constructor with InitSMmodule It needs a re implementation only if an external state machine shall be used void ParameterEvent dabc Parameter par int event
167. ng dabc CommandDefinition object In this case the command and its arguments are known remotely and can be invoked from a controls GUI UserModule UserModule const char name dabc ModuleSync name dabc CommandDefinitionx def NewCmdDef UserPrint def gt AddArgument Level dabc argInt false optional argument def gt Register true 12 2 5 ModuleSync Data processing functionality in a most intuitive way can be implemented by subclassing the dabc ModuleSync base class which defines the interface for a synchronous module that is allowed to block its dedicated execution thread This class provides a number of methods which will block until the expected action can be performed Method Description Recv Receive buffer from specified input port Send Send buffers over output port RecvFromAny Receive buffer from any of specified port WaitInput Waits until required number of buffers is queued in input port TakeBuffer Get buffer of specified size from memory pool WaitConnect Waits until port is connected In all these methods a timeout value as last argument can be specified Method SetTmoutExcept defines if a dabc TimeoutException exception is thrown when the timeout is expired By default these blocking methods just return false in case of timeout Data processing should be implemented in MainLoop method It usually contains a while loop where Module Working method is used to
168. ng on constructor argument usecurrentprocess this is either the main process thread or a dedicated manager thread It may be re implemented to add new commands required for the controls implementation The DABC mech anism of methods SubmitCommand and ExecuteCommand may allow to decouple control system call backs from their execution thread 10 3 2 2 Baseclass methods In addition to the virtual methods to be implemented in the manager subclass there is a number of dabc Manager base class methods that should be called from the control system to perform actions of the framework bool DoStateTransition const char state_transition_cmd Performs the state machine transition of name state_transition_cmd This method is synchronous and returns no sooner than the transition actions are completed true or an error is detected false Note that the real transition actions are still user defined in methods of the dabc Application implementation bool IsStateTransitionAllowed const char state_transition_cmd bool errout Checks if state transition of name state_transition_cmd is allowed for the default state machine implementation which should be reproduced exactly by any external SM implementation and returns true or false resp Argument errout may specify if error messages shall be printed to stdout void RecvOverCommandChannel const char cmddata Receives a DABC command as text stream cmddata from a remote node Usually this func
169. nputs and connects each input port with an x 1md file transport 144 Socket classes All communication with the MBS nodes is performed via tcp sockets The DABC base package 1ibDabcBase so implements a number of classes for general socket handling The main idea of these classes is to handle socket operations creation connection sending receiving and error handling by means of event processing callbacks Class dabc SocketThread organises the event loop that handles such event signals produced by a socket Each system socket is assigned to an instance of dabc SocketProcessor The actual socket event processing is then done in virtual methods of class dabc SocketProcessor which has several subclasses for different kinds of sockets e dabc SocketServerProcessor server socket for connection e dabc SocketClientProcessor client socket for connection e dabc SocketIOProcessor send recv handling One can use a dabc SocketThread together with other kind of processors like module classes but not vice versa one cannot use socket processors inside other thread types Therefore it is possible to run module with all socket 98 DABC Programmer Manual Example MBS transports in one single thread if the socket thread for such module is created in advance see MBS generator example in section 14 7 14 5 Server transport Class mbs ServerTransport provides the functionalities of an MBS transport server or and MBS stream server in
170. nt To really work with asynchronous command execution one should be able to analyse the result of such commands though This can be done with class dabc CommandClient Before submitted for execution commands should be assigned to a dabc CommandClient object In this case the CommandClient will get a callback from the Command when execution is done and can react on this callback One can assign more than one Command to a CommandClient A first use case for the CommandClient if one needs to execute many commands at once Using Execute method all commands can be executed sequentially only By means of the CommandClient however one can submit many commands first and then wait for all of them to be executed If the associated CommandReceivers run with different threads the commands will be executed in parallel For instance dabc CommandClient cli for unsigned n 0 n lt 10 n dabc Modulex m dabc mgr gt FindModule FORMAT Module u n dabc Command cmd new dabc Command UserCommand cli Assign cmd m gt Submit cmd bool res cli WaitCommands This example submits 10 commands into 10 different modules and waits at one place until all commands are executed Another use case for the CommandClient it keeps the Command object after execution and can analyse the contained result values For instance all 10 commands from previous example may return several values each If one instantiates the
171. odeNane String getNodelD String getApplicationFull String getApplication String getApplicationName String getApplicationID String getName String getNameSpace String getItens String getFull String getFull boolean build String getCommand String getCommand boolean build int getType nt getState nt getVisibility nt getMode nt getQuality nt getNofTypes nt getTypeSizes String getTypeList String getFormat boolean isNotSpecified boolean isSuccess boolean isInformation boolean isWarning boolean isError boolean isFatal boolean isAtomic boolean isGeneric boolean isState boolean isInfo boolean isRate boolean isHistogram boolean isCommandDescriptor boolean isHidden boolean isVisible boolean isMonitor boolean isChangable boolean isImportant boolean isLogging boolean isArray boolean isFloat boolean isDouble boolean isInt boolean isLong boolean isChar boolean isStruct H H H H H H H 10 Or 0 10 10 O O O 000 0 e O 07 0 0 0 O a a a e 0 1030 OO Or Or 0 O_O 007 0 Or 0 00 8 0 07 0 00 146 DABC Programmer Manual GUI 18 7 3 Other interfaces 18 7 3 1 Interface xiPanelltem Interface to be implemented for objects to be placed onto xPanelGraphics The elementary graphics
172. of the GUI described here may be extended by application specific parts How to add such extensions is described in the programmer s manual Typically they are started as prompter panels via buttons in the main GUI menu The standard part builds a set of panels windows according the parameters the DIM servers offer Only services from one single DIM name server node name specified as shell variable DIM_DNS_ NODE defining a name space can be processed See 5 3 1 page 36 for preparations The GUI needs no file access to the DABC working directory However user must have ssh or rsh access to the DABC or MBS master node Currently the GUI must run under the same account as the DABC In monitoring mode no commands the GUI may run under different account Master node must have remote access to all worker nodes The user s ssh settings must enable remote access without prompts The layout of the GUI can be adjusted to individual needs It is strongly recommended to save these settings to see the same layout after a restart of the GUI The GUI can be restarted any time DABC and MBS systems continue without GUI 4 2 GUI Panels DABC Controls and Monitoring Breast ot Figure 4 1 Main toolbar buttons Fig 4 1 page 21 shows the main menu of DABC minimal view The GUI as it comes up is divided 21 2 DABC User Manual GUI in three major parts one sees on top a toolbar with icon buttons Most of these open other windo
173. onally a DMA engine component mprace DMAEngineWG is applied for all DMA specific actions 2 It implements the actual asynchronous DMA by overriding methods ReadPCIStart and ReadPCICom plete The base class pci BoardDevice can provide synchronous DMA only because the generic mprace Board interface does not cover asynchronous features These are handled by the DMAEngineWG component 3 The constructor uses several configuration parameters unsigned int devicenum GetCfgInt ABB_PAR_BOARDNUM 0 cmd unsigned int bar GetCfgInt ABB_PAR_BAR 1 cmd unsigned int addr GetCfgInt ABB_PAR_ADDRESS 0x8000 gt gt 2 cmd unsigned int size GetCfgInt ABB_PAR_LENGTH 8192 cmd The parameter names are handled by string definitions in abb Factory h define ABB PAR _BOARDNUM ABB BoardNumber define ABB PAR BAR ABB ReadoutBAR define ABB PAR ADDRESS ABB ReadoutAddress define ABB PAR LENGTH ABB ReadoutLength The GetCfgInt will look for a parameter of the specified name already existing in the system e g if the Application object has defined such If not a dabc Parameter of that name will be created and exported to the control system If the configuration file specifies a value for this parameter it will be set otherwise the default value second argument of GetCfgInt is set If the constructor gets a command object cmd as argument containing a parameter of the specified name this command s paramete
174. onding DIM service Here the dimc Registry keeps auxiliary objects of class dimc ServiceEntry that link the DimService with the dabc Parameter see section 10 3 3 4 On parameter change method ParameterUpdated will initiate an update of the corresponding DIM service 4 Control system commands Method DefineDIMCommand const char name creates and registers sim ple char array DimCommand objects that may be executed on this node The dimc Registry constructor defines commands for all state machine transitions such as Configure Enable Halt Start Stop Addi tionally there are DIM commands for shutting down the node setting a parameter value and wrapping a DABC command as string representation ManagerCommand for SendOverCommandChannel see section 10 3 2 resp Moreover a DABC module may register a dabc Command as new control system command on the fly In this case dimc Manager method CommandRegistration will use RegisterModuleCommand of dimc Registry This will both define aDimCommand and publish a corresponding command descriptor as DIM service to announce the command structure to the generic Java GUI Method UnRegisterModuleCom mand may remove command and descriptor service again When the DIM server receives a remote command method HandleDIMCommand checks if this com mand is registered then OnDIMCommand will transform the DimCommand into a dabc Command and Submit this to the Manager The actual command ex
175. one Remote transport connections may be not all fully connected since some connections require active negotiations between different nodes Thus the final connecting is done between Configured and Ready Ready The application is fully configured but not running modules are stopped Running The application is fully configured and running Failure This state is reached when there is an error in a state transition function Note that a run error during the Running state would not lead to Failure but rather to stop the run in a usual way to Ready The state transitions between the 5 generic states correspond to commands of the control system for each node application DoConfigure between Halted and Configured The application plug in creates application specific devices modules and memory pools Application typically establishes all local port connections DoEnable between Configured and Ready The application plug in may establish the necessary connections between remote ports The framework checks if all required connections are ready DoStart between Ready and Running The framework automatically starts all modules transport and device actions DoStop between Running and Ready The framework automaticall stops all modules transport and device 9 4 Package and library organisation 55 actions i e the code is suspended to wait at the next appropriate waiting point e g begin of MainLoop wait for a requested resour
176. orar eae ana an 24 4 2 3 Action In progress o ca a Re ee ea a nn 25 4 2 4 MBS Control panel cis ca 2a ba aa a he PREG we dal ee dae 25 4 2 5 Combined DABC and MBS control panel 2 2 2 o eee ee 25 4 2 6 Command panel ocios cer Ad a hs 25 4 2 7 Prater DIE eo doa Gace a er a Bares de 26 42 7 1 Parameter selection lt lt u 0 0 ee eee ee ee ee ee aw ee 27 4 2 8 Monitoring panels o es eb ee ee ee a E 27 ASL SAE en a a e y a Gee Ge BOS eke re bu 27 282 AI ewe er A a E N 28 428 3 MEOS od 4 ace da Gace een BERKS a Bed Marea Bo 29 CONTENTS y 4 294 Tnformaton locucion be ee a en aa 29 4253 Logging widow a e eaeoe ee 29 4 3 GULsaveirestore Beinps cosacos da OG ee an Herne Lhe da 30 5 DABC User Manual MBS GUI 31 Sol MBSovent bulding essa en aa 31 oy MBS SCIP 6 4 ostia baw Gee Ao de oe de OR ee ara Bee Sede we de 31 5 1 2 MBS control panel oo so ccome aa Ban o 31 5 1 2 1 MBS controller b ttons os o 2 0 ecoa man ee So nn 32 1 3 MES command panel o 6p ab a A a ee e 33 52 MBS DIM parameters oo co co ur ac ana en ra aa a ee weet 34 S201 MS Sk a ea RY Rh eed OW Lee PAR ew Ree A ee Ri 34 322 US TBS be cee en ek hen OR Ee a ek ee a Re a a d A 35 5 2 3 MBS histograms oa ca aca Ee Awa ee OH ba ee dee ea eke hous 35 Saw DABS ANOS 2 6 fled ee a ae a ea E ay de A a al ia 35 5 2 3 TUS TBR fe ee Gn ee Oe Be ede ee de ae a ee ee EG 35 320 AI bank COWES Cede eae ee edb aie ede ke 35 oa MBES DUMPS 22 ka eek Paw eA a OA
177. orkingProcessor Represents a runnable job Each working processor is assigned to one working thread instance this thread can serve several working processors in parallel In a special mode a processor can also run its explicit main loop dabc WorkingProcessor is a subclass of dabe CommandReceiver i e a working processor may receive and execute commands in its scope dabc Module A processing unit for one step of the dataflow Is subclass of dabe WorkingProcessor i e the module may be run by an own dedicated thread or a working thread may execute several modules that are assigned to it A module has ports as connectors for the incoming and outgoing data flow dabc ModuleSync Is subclass of dabc Module defines interface for a synchronous module that is allowed to block User must implement virtual method MainLoop that uses a dedicated working thread to run Method TakeBuffer provides blocking access to a memory pool Blocking methods dabc ModuleSync Send and dabc ModuleSync Receive are used from the MainLoop code to send or receive buffers over or from a ports dabc ModuleAsync Subclass of dabc Module defines interface for an asynchronous module that must never block the execution Several dabc ModuleAsync objects may be assigned to one working thread User must either re implement virtual method ProcessUserEvent wich is called whenever any event for this module i e this working processor is processed by the
178. ormed on all worker nodes Technically speaking a state machine command which is executed on the cluster controller is only completed if the state transition commands on all workers are successfully completed This is implemented by means of class dabc CommandsSet see method StartClusterSMCommand for details Class bnet ClusterApplication has following configuration parameters Name Type Dfit Description NetDevice str dabc SocketDevice device class for network connections NumEventsCombine int 1 number of events time frames combined together TransportBuffer int 8192 size of buffer used for data transport cluster wide Class bnet ClusterApplication is fully functional and can be used as is for a real cluster set up 15 3 Worker application The basic functionality of a worker is implemented in bnet WorkerApplication class Its main purpose is to instantiate configure connect and run all working modules triggered by commands from the cluster controller Main functionality of bnet WorkerApplication is implemented in virtual method CreateAppModules which is called during transition from Halted to Configured state In this method all local modules are instantiated and configured Some of these modules depend on the actual experiment therefore class bnet WorkerApplication provides a number of virtual methods to create experiment specific components CreateCombiner create a module combining several data sources to produc
179. ort Typically transport objects created by the dabc Device methods CreateDatalnput Instantiate a dabc DataInput of specified type Initialisation of object will be done by Read_Init call CreateDataOutput Instantiate a dabc DataOutput of specified type Initialisation of object will be done by Write_Init call Since all factories are registered and kept in the global DABC manager all methods mentioned here have equiva lent methods in class dabc Manager The manager simply iterates over all factories and executes the appropriate factory method until an object of the requested class is created For instance to create a module one should do dabc mgr gt CreateModule mbs GeneratorModule Generator Invocation of these methods in manager is implemented via corresponding commands for instance CmdCre ateModule for module creation These command classes should be used directly if one wants to deliver extra configuration parameters to the object s constructor most factories methods gets this command as optional argu ment For instance dabc Command cmd new dabc CmdCreateModule mbs GeneratorModule Generator cmd gt Set Int cmd gt Set Int dabc mgr NumSubevents 5 SubeventSize 64 gt Execute cmd The DABC framework provides several factories for predefined implementations e g bnet SenderModule verbs Device Chapter 13 DABC Programmer Manual Setup
180. oryPool 7 52 dabc Module 59 dabe ModuleAsync 52 59 84 dabc ModuleSync 51 59 83 dabc Parameter 7 52 58 dabc Pointer 52 dabc PoolHandle 52 dabc Port 7 53 59 dabc Transport 8 53 59 dabc WorkingProcessor 59 dabc WorkingThread 58 DABC DIM naming conventions 133 Environment set up 14 Installation 13 Plug in installation 18 Setup file 15 DIM Conventions 133 Introduction 133 DIM Control classes dimc Manager 69 dimc ParameterInfo 70 dimc Registry 69 dimc Server 70 dimc ServiceEntry 70 Finite state machine states 8 54 transition commands 9 54 Manager interface 153 CanSendCmdToManager 68 CleanupManager 66 CommandRegistration 67 ConnectPorts 65 CreateApplication 64 CreateDevice 64 CreateMemoryPool 65 CreateModule 64 Create Transport 64 DeleteModule 64 DeletePool 66 DestroyObject 66 DoStateTransition 68 ExecuteCommand 68 FindPool 66 GetNodeName 68 HasClusterInfo 67 InvokeStateTransition 67 IsAnyModuleRunning 65 IsMainManager 67 IsModuleRunning 65 IsNodeActive 68 IsStateTransitionAllowed 68 MakeThreadFor 65 MakeThreadForModule 65 Nodeld 68 NumNodes 67 ParameterEvent 67 Print 66 RecvOverCommandChannel 68 SendOverCommandChannel 68 SetCmdReceiver 65 StartAllModules 64 StartModule 64 StopAllModules 64 154 INDEX PCI StopModule 64 Subm
181. oryPool has a fixed structure the memory is allocated once and will not change during the com plete run This is the preferrable mode of operation because any memory allocation may lead to an undefined execution time or could even cause an error if the system has too few resources Nevertheless one can configure a MemoryPool to be extendable the MemoryPool will allocate new blocks if it has no more memory available to provide a requested Buffer Each subblock of the Memory Pool has a 32 bit reference counter which counts how many references to this mem ory region are in use by the Buffers This is necessary for book keeping of available memory since several Buffer objects can refer to the same subblock The user can request a new Buffer from the MemoryPool with method TakeBuffer This method returns a dabc Buffer instance with an internal reference to a formerly unused subblock of the appropriate size The refer ence counter of this subblock is incremented then To release a Buffer one should call static method dabc Buffer Release This will delete the Buffer object and decrement again the subblock reference counter 11 1 3 Buffer In the general case dabc Buffer contains a list of segments gather list Each segment represented by class dabc MemSegment refers to a different part of a subblock in the MemoryPool compare section 11 1 2 The dabc MemSegment contains a unique buffer id the pointer to the segment begin and the s
182. p_chan In DIM mode create services and start pth_dim_serv to update every second In all cases _ mg_msg_thread is called where in slave mode messages are sent to the master otherwise processed by _mg_msg_output 108 DABC Programmer Manual Example MBS 14 9 5 8 MBS controlled by DIM Graphics on Eigene Dateien experiments mbs Remote status client Dispatcher TTY message queue to 2000 command queue to pid Figure 14 1 Single node MBS controlled by DIM 14 10 List of icons y browser Browser Ooo E comicon Command conndsp Connect single MBS a connprm Connect MBS prompter ae control Test shell script ap controlmbs DABC shell script aD controldabc MBS shell script 8 dabcconfig Configure dabcicon DABC launcher Hr dabembsicon DABC MBS launcher 0 dabcstart Start acquisition 14 10 List of icons 109 TCP 6006 loop Command Prompter TCP 6004 TCP 6004 Message server message queue TCP 6007 thread to 2000 Main loop Message thread Command z thread o 4 Message client a 2 threads o Task O Master Logger Command loop command queue to pid Dispatcher message queue Message to 2000 thread Main loop Lo
183. pee eee 141 18 6 2 1 xPanelParameter o onc oroo cea Ea a e ae a E a i e e i 141 18 6 2 2 xPanelCommand o o o o 142 1856 3 Startup SEQUENCE eso aoci as a ee ee 142 ROA Update Segut lt cp ew ee a Ree ead Rees ERS a A A 142 18 7 Application specific GUI plug in 2 2 2 2 0 00 000000000084 143 18 7 1 Java Interfaces to be implemented by application 2 22 nn nennen 143 Tey A Loterlace Her Panel eo 2 2 A Oe ee 143 18 7 1 2 Interface xiUser Command a ee 143 18 7 1 3 Interface xiUserInfoHandler 2 Comm en 143 18 7 2 Java Interfaces provided byGUl on une 144 18 7 2 1 Interface xiDesktop o o 144 18 7 2 2 InterfacexiDimBrowser 1 en 144 18 7 2 3 Interface xiDimCommand 1 0 0 a 144 18 7 2 4 Interface xiDimParameter 1 0 Cm 144 18 2 29 literate MPaMEP oo iia ea eae ee ee eed es 145 18 7 3 Otherinteriaces ocioteca an ren ee he eS 146 18 7 3 1 Interface xiPanelltem 2 CC mn 146 1824 Exampl 2242er 24 is lan ded eae hee eee bw ea eS 146 187 3 Storefrestore layout oo o ssaa senro cauere a es 149 References 151 Index 153 Part I User Manual Chapter 1 DABC User Manual Overview user user overview tex 1 1 Outline of this manual This DABC User Manual contains all information that is necessary to install and use the DABC frame work Chapter 2 page 5 should be useful to understand the most commonly used terms of DABC
184. plicationnamespace Japplicationname applicationID TYPE module name Example DABC 1x05 Control RunState We recommend to forbid spaces in any name fields Dots should not be used except in names last field The generic xGUI can handle only services from one server name space defined by DIM_DNS_ NODE For DABC and MBS this servernamespace is set to DABC 18 2 2 DABCDIM records For generic GUls we need something similar to the EPICS records This means to define structures which can be identified How shall they be indentified One possibility would be to prefix a type to the parameter name i e rate DataRate Another to use the quality longword This longword can be set by the server One could mask the bytes of this longword for different information mode MSB visibility type status LSB mode not used visibility Bit wise can be ORed HIDDEN all zero VISIBLE 1 appears in parameter table MONITOR 2 in table graphics shown automatically if type is STATE RATE or HISTOGRAM CHANGABLE 4 in table can be modified IMPORTANT 8 in table also if GUI has a minimal view HISTOGRAM ODULE PORT DEVICE QUEUE COMMANDDESC 9 INFO 10 OANA O BFWN EF exclusive N PEC 0 S ESS INFORMATION W O H un U 52 D DI Z H z Q ll dina Then we could provide at the client side objects for handlin
185. pping the module in AfterModuleStop Alternatively by means of method CreateRateParameter it also defines a rate parameter DMAWriter that is linked to the input port Output and may export the current data rate to the control system 17 3 4 abb Factory A subclass of dabc Factory to plug in the ABB classes 1 Implements CreateDevice for the abb Device The third argument of this factory method is a dabc Command that may contain optional setup parameters of the device 2 Implements CreateModule for the abb ReadoutModule and the abb WriterModule Third argument of this factory method is a dabc Command containing optional setup parameters of the module 3 The factory is created automatically as static singleton instance on loading the 1ibDabcAbb so 17 4 Simple read and write tests The functionality of the ABB can be tested with several simple executables which are provided in the test subfolder of the abb plugin package 17 4 Simple read and write tests 129 17 4 1 DMA Read from the board The example code abb_test_read cxx shows in a simple main function how to utilize the abb classes for a plain readout with DMA 1 It applies the dabc StandaloneManager as most simple Manager implementation int nodeid 0 this node id int numnodes 1 number of nodes in cluster dabc StandaloneManager manager 0 nodeid numnodes 2 The abb Device is created by means of a command CmdCreateDevice which is pas
186. primitives like rate meters Optional xiUserCommand provides a function to be called in the xGUI xPanelCommand when a command shall be executed This function steers if the command arguments have to be encoded in XML style or argument list style There is for convenience another subclass of xInternalFrame and JInternalFrame for easy formatting from one to four panels JPanel or xPanelGraphics inside xInternalCompound Examples of such application panel can be found on directory application 18 7 1 Java Interfaces to be implemented by application 18 7 1 1 Interface xiUserPanel o abstract void init xiDesktop d ActionListener a Called by xGUI after instantiation The desktop can be used to add frames see below o String getHeader Must return a header name text after instantiation o String getToolTip Must return a tooltip text after instantiation o Imagelcon getIcon Must return an icon after instantiation o xLayout checkLayout Must return the panel layout after initialization o xiUserCommand getUserCommand Must return an object implementing xiUserCommand or null See below o void setDimServices xiDimBrowser b Called by xGUI whenever the DIM services had been changed The browser provides the command and param eter list see below One can select and store references to commands or parameters A xiUserInfoHandler object can be registered for each selected parameter Then the infoHandler method of this object is
187. protocoll based on publish subscribe mechanism Servers publish named services commands or parameters to a DIM name server Clients can subscribe such services by name They then get the values of the services subscribed from the server providing it Whenever a server updates a service all subscribed clients get the new value Clients can also execute commands on the server side DIM provides the possibility to specify parameters and command arguments as primitives I or L X C F D or structures The structures are described in a format string which can be retrieved by the clients for parameters and commands and servers for commands TS lis les Thus a client can generically access parameter structures but without semantical interpretation In addition to the data and format string one longword called quality is sent 18 2 1 DABC DIM naming conventions When the number and kind of services of DIM servers often change it would be very convenient if a generic GUI would show all available services without further programming It would be also very nice if standard graphical elements would be used to display certain parameters like rate meters If we have many services it would be convenient to have a naming convention which allows to build tree structures on the GUI Naming conventions for generic xGUI line breaks for better reading 133 134 DABC Programmer Manual GUI servernamespac nodename nodeID ap
188. pter to be stopped then calls m_remote reset all nodes from node_list txt For the message logger modes see table 14 2 page 107 When prompter is started with dim it starts the master message logger with argv 1 masterdim otherwise master When prompter is started with the r lt remote argument gt the remote node name is passed as argv 2 to the message logger All dispatchers are started in the MBS prompter by f_ifa_init NodeList DimNameServer function It calls per node from NodeList f_ifa_remote which starts the dispatcher in function f_ifa_rsh_proc_start by script dimremote_exe sc DIM mode or remote_exe sc normal mode When prompter is started with dim then the DIM command server is started Otherwise it starts TCP server on port 6006 if started with the r lt remotenode gt option waiting for connection of GUI client and reading commands or reading commands from terminal 2 dimremote_exe sc SMBSROOT SPWD SDIM_DNS_NODE SPROMPTER_NOD launches m_dispatch dim lt prompternode gt The dispatchers start their message logger with the same argument prompter node and optional argv 2 slavedim On the prompter node however the message logger should already run started by prompter Because started with argv 2 lt prompternode gt the dispatchers then start a TCP server waiting on port 6004 for connection of prompter When prompter connects dispatcher sends process id Then waits for commands
189. put Over the input port it gets N buffers with subevents for the same event identifier Over the output port it should deliver one buffer with build events When the user inherits his her builder module from bnet BuilderModule it is enough to implement the virtual DoBuildE vent method which gets as argument a list of N buffers with subevents The format of the output buffer is completely user defined It is allowed to fill several events into the same output buffer if necessary 15 7 Filter module This is an optional component of BNET if build events shall be filtered before they are stored To implement such filter one can derive it from bnet FilterModule and reimplement virtual method TestBuffer As an alternative filtering can be implemented directly in the event builder module 15 8 BNET test application This application may test different asspects of a BNET without the necessity to have real data sources The complete source code and configuration examples can be found in SDABCSYS applications bnet test directory The example contains following classes bnet TestGeneratorModule bnet TestCombinerModule bnet TestBuilderModule bnet TestFilterModule bnet TestWorkerApplication bnet TestFactory There are several examples of configuration files For instance the configuration of 4 worker nodes with sender and receiver functionality each is shown in SetupBnet xml lt xml version 1 0 gt lt dabc version 1 gt
190. r 1 state redraw 0 Gray Dead true else if Display equals cmd if desk findFrame franame frame new xInternalCompound franame graphicon 0 fralayout xSet blueD frame rebuild stapan metpan desk addFrame frame My Graphics sition RunMode 4 Acquisition 4 RunMode Figure 18 2 Ministates With the checker we toggle the xState state ServerState in screen shot The xiDimParameter param to be toggled we will find in the next To get access to DIM parameters we must implement setDimServices We suggest that there is a parameter Setup_ File which has a string value The myInfoHandler class is described next public void setDimServices xiDimBrowser browser Vector lt xiDimParameter gt vipar browser getParameters for int i 0 i lt vipar size itt xiParser p vipar get i getParserInfo String pname new String p getNode p getName if p isRate xMeter meter new xMeter xMeter ARC pname 0 0 10 0 xMeter XSIZE xMeter YSIZE xSet blueL meter setLettering p getNode p getName vipar get i getMeter getUnits metpan addGraphics meter false browser addInfoHandler vipar get i new myInfoHandler pname meter null else if p isState xState state new xState pname xState XSIZE xState YSIZ stapan addGraphics state false browser addInfoHandler vipar get i new myInfoHandler pname null state els
191. r socket connection Function InitMbsGenerator can be used to instantiates the generator module It also demonstrates how a thread of type dabc SocketThread can be created and used by both a module and a transport object extern C void InitMbsGenerator dabce mgr gt CreateThread GenerThrd dabc typeSocketThread dabc mgr gt CreateModule mbs GeneratorModule Generator GenerThrd dabc mgr gt CreateTransport Generator Output mbs typeServerTransport GenerThrd To run the generator module with all default parameters this simple XML file is sufficient lt xml version 1 0 gt lt dabc version 1 gt lt Context host 1xi009 name Server gt lt Run gt lt lib value libDabcMbs so gt lt func value InitMbsGenerator gt lt Run gt lt Context gt lt dabc gt Besides one may specify all module and transport parameters explicitely here lt xml version 1 0 gt lt dabc version 1 gt lt Context host 1xi009 name Server gt lt Run gt lt lib value libDabcMbs so gt lt func value InitMbsGenerator gt lt Run gt lt Module name Generator gt lt NumSubevents value 3 gt lt FirstProcId value 77 gt lt SubeventSize value 128 gt lt Go4Random value false gt lt BufferSize value 16384 gt lt Port name Output gt lt OutputQueueSize value 5 gt lt MbsServerKind value Stream gt lt MbsServerPort value
192. r value will override all other values for this parameter defined elsewhere in the system The user may pass such a cmd to the abb Device either as third argument of the manager factory method CreateDevice or by means of a dabc CmdCreateDevice object which is invoked by Execute of the manager This is useful if the device is to be tested without any configuration or control system as shown in the examples of section 17 4 4 It provides pseudo event data for the Bnet test example in the received DMA buffers Method ReadPCI is extended to copy an event header of the Bnet format i e incrementing event count and unique id into each 128 DABC Programmer Manual Example PCI output buffer after the base class ReadPCI is complete This workaround is necessary since the ABB data itself does not contain any information in the test setup Additionally method DoDeviceCleanup will reset the event counters at the end of each DAQ run 17 3 2 abb ReadoutModule Subclass of dabc ModuleAsync generic implementation of a readout module to use the BoardDevice 1 It creates the memory pool which is used for DMA buffers in the pci BoardDevice this pool is propagated to the device via the pci Transport when module is connected since device will use the pool associated with the connection port 2 Module runs either in standalone mode one input port no output for testing or in regular mode one input port one output port 3 ProcessUse
193. rEvent defines the module action for any DABC events e g input port has new buffer In standalone mode the received buffer is just released In regular mode buffer is send to the output port 4 It has a dabc Ratemeter object which is updated for each packet arriving in ProcessUserEvent The aver age data throughput rate is then printed out to the terminal on stopping the module in AfterModuleStop Alternatively by means of method CreateRateParameter it also defines a rate parameter DMAReadout that is linked to the input port Input and may export the current data rate to the control system 17 3 3 abb WriterModule Subclass of dabe ModuleSync generic implementation of a writer module to use the BoardDevice 1 Creates the memory pool which is used for DMA buffers in the pci BoardDevice this pool is propagated to the device via the pci Transport when module is connected since device will use the pool associated with the connection port 2 Module runs either in standalone mode one output port no input for testing or in regular mode one input port one output port 3 MainLoop defines the module action In standalone mode a new buffer is taken from the memory pool and send to the output port In regular mode the send buffer is taken from the input port 4 It has a dabc Ratemeter object which is updated for each packet arriving in MainLoop The average data throughput rate is then printed out to the terminal on sto
194. rbs to an input port of the corresponding Bnet receiver node bnet ReceiverModule Subclass of dabc ModuleAsync Receives the data frames from the Bnet sender mod ules and sorts together packets belonging to the same events or time frames resp 1 It has one input port for each sender node in the Bnet The data frames are buffered in the Bnet receiver module until the corresponding frames of all senders have been received then received frames are send sequentially to the output port 2 It has exactly one output port This is connected to the bnet BuilderModule implementation that performs the actual event building task bnet BuilderModule Subclass of dabc ModuleSync Framework prototype class to select and build a physics event from the data frames of all Bnet senders as received by the receiver module 1 It has one input port connected to the Bnet receiver module The data frame buffers of all Bnet senders are transferred serially over this port and are then kept as an internal std vector in the Bnet builder module 2 Method DoBuildEvent is to be implemented in user defined subclass e g bnet Mbs BuilderModule and is called in module s MainLoop when a set of corresponding buffers is complete 3 It provides one output port that may connect to a Bnet filter module or a user defined output or storage module resp 4 The user has to implement the sending of the tagged events to the output port explicitly in his subc
195. rs and removes all application handlers Discard the parameter and command panel and call Java garbage collector At this point no more references to parameters or commands should exist and all objects can be removed Call xDimBrowser initServices Create xPanelParameter Create xPanelCommand Call setDimServices of all application panels Pass xiUserCommand object from first application panel object to xPanelCommand Call xDimBrowser enableServices Call xPanelCommand setCommandDescriptors 10 Update the internal frames of parameters and commands ZA DO up w O 00 18 7 Application specific GUI plug in 143 18 7 Application specific GUI plug in Besides the generic part of the xGUIit might be useful to have application specific panels as well integrated in the generic xGUI This is done by implementing subclasses of xPanelPrompt The class name only one can be passed as argument to the java command starting the xGUI or by setting variable DABC_ APPLICATION PANELS being a comma separated list of class names Variable is ignored if class name is given as argument The classes must implement some interfaces xiUserPanel needed by xGUI xiUserInfoHandler needed to register to DIM services This could be a separate class xiUserCommand optional to specify command formats One can connect call back functions to parameters get a list of available commands create his own panels for dis play using the graphical
196. rs are used in module for configuration controlling and monitoring More information about parameters handling see in chapter 13 12 2 4 Commands processing There is the possibility in DABC to execute user defined commands in a module context Virtual method Ex ecuteCommand is called every time when a command is submitted to the module The command is always executed in the module thread disregarding from which thread the command was submitted Therefore it is not necessary to protect command execution code against module function code by means of thread locks Most actions in DABC are performed with help of commands Here is an example how command execution can look like int UserModule ExecuteCommand dabc Command cmd if cmd gt IsName UserPrint DOUT1 Printout from UserModule return dabc cmd_true return dabc ModuleSync ExecuteCommand cmd This is invoked somewhere in the code of another component dabc Modulex m dabc mgr gt FindModule MyModule dabc Commandx cmd new dabc Command UserPrint m gt Execute cmd again but in short form m gt Execute UserPrint 12 2 Modules 83 After command execution has finished method Execute returns true or false depending on the success The dabc Command object is deleted automatically after execution In the module constructor one can register a command for the control system by means of a correspondi
197. rties of these ports Of course the MBS setup must match these definitions There are two output Ports in parallel here A FileOutput that writes into a x 1md file as specified in the property MbsFileName and a ServerOutput that offers a standard MBS stream server for a monitoring program A full description is in Programmer Manual section 14 8 page 101 Now we can use the combined controller panel to startup MBS and DABC 6 1 MBS event building with DABC 41 gt DabcMbsController Login Name server User name Password RET MBS master node 1x90523 g5i de goofy R3g 7 MES servers 3 DABC master node Ixi009 DABC master name DABC servers MBS system path MbsEb 1 mbs w51 MBS user path dabc mbs2 MBS Startup startup scom MBS Shutdown MBS command DABC system path DABC user path shutdown scom ps fusrflocal dabc workspace dabc dabc work mbs2 DABC setup file Combiner xml DABC script MBS control file ps MbsControl xml DABC control file DabcControl xml BEOSOOGEFO TV Figure 6 1 Combined DABC and MBS controller 6 1 3 Combined DABC and MBS control panel This panel shown in Fig 6 1 page 41 is simply a superposition of the single ones Here the Context name of the DABC node and the DABC setup file name must be specified Number
198. s start stop reconfigure over application Useful for batch mode port ssh port number of remote host user account name to be used for ssh login without password should be possible init init script which should be called before dabc application starts test test script which is called when test sequence is run by run sh script timeout ssh timeout debugger argument to run with a debugger Value should be like gdb x run txt args where file run txt should contain commands r bt q workdir directory where DABC executable should start debuglevel level of debug output on console default 1 logfile filename for log output default none loglevel level of log output to file default 2 DIM_DNS_NODE node name of DIM dns server used by DIM controls implementation DIM_DNS_PORT port number of DIM dns server used by DIM controls implementation cpuinfo instantiate dabc CpulnfoModule to show CPU and memory usage information Value must be gt 0 If 0 only two parameters are created if 15 several ratemeters will be created parslevel level of pars visibility for control system default 1 3 3 5 Variables In the root node lt dabc gt one can insert a lt Variables gt node which may contain definitions of one or several variables Once defined such variables can be used in any place of the configuration file to set parameter values In this case the syntax to set a parameter is lt ParameterName value VariableName gt
199. s an interface to switch the application state by the external control system This may be done by calling state change methods of the manager or by submitting state change commands to the manager from GUI Some of the application states may be propagated to the active components modules device objects e g the Running or Ready state which correspond to the activity of the thread Other states like Halted or Failure do not match a component state e g in Halted state all modules are deleted and thus do not have an internal state The granularity of the control system state machine is not finer than the node application There are 5 generic states to treat all set ups 2 3 Controls and configuration 9 DoError DoHalt x DoEnable DoStart DoConfigure DoHalt DoHalt Figure 2 2 The finite state machine as defined by the manager Halted The application is not configured and not running There are no modules transports and devices existing Configured The application is mostly configured but not running Modules and devices are created Local port connections are done Remote transport connections may be not all fully connected since some connections require active negotiations between different nodes Thus the final con necting is done between Configured and Ready Ready The application is fully configured but not running modules are stopped Running The application is fully configured and running
200. s as event builders some modifications of the MBS setup files must be done For the DABC side setup files must be provided 6 1 1 MBS setup When we want to use DABC nodes as event builders we need a different setup on the MBS side We assume that we have more than one MBS node Such a multi node system is controlled by an MBS prompter running on one node e The setup has to be changed such that all nodes run as if they are stand alone this is done typically by setting COL MODE to 0 in the usf setup file That means that each node must run the Readout Collector Transport Daq_ rate chain The DABC event builders connect to the transports e The MBS buffer size should be set to the stream size and the number of buffers per stream must be set to one 6 1 2 DABC setup On the DABC user working directory we need configuration files Summary of parameters MbsFileName File name for list mode data file LMD Overwritten by command MbsFileSizeLimit File closes when size is reached and new file opens BufferSize Should match MBS buffer size MbsServerKind Transport Stream MbsServerPort Port number transport 6000 MbsServerName MBS node of transport NumInputs Number of MBS channels for one combiner DoFile Provide output file DoServer Provide server 39 40 DABC User Manual DABC Application MBS These parameters are used to configure an optional event generator NumSubevents FirstProcId SubeventSize Go4Random
201. s for 5 seconds while the DABC threads and the board DMA performs the data transfer The module is stopped again then sleep 5 manager StopModule ABB_Readout DOUT1 Stopped readout module After the module has stopped its internal dabc Ratemeter will print some average data rate values to the terminal Finally all objects are destroyed and the manager is cleaning up the process before the program ends manager CleanupManager 17 4 2 DMA Write to the board The example code abb_test_write cxx shows in a simple main function how to utilize the abb classes to write data from the PC to the ABB with DMA The code is very similar to the read example as described in the above section 17 4 1 1 It applies the dabc StandaloneManager as most simple Manager implementation 2 The abb Device is created by means of a command CmdCreateDevice which is passed to the manager The command contains the initial parameters for the device The DMA transfer size readsize may be set by the executables s first command line parameter see section 17 4 1 for code example 3 It creates a abb WriterModule by means of a command CmdCreateModule which is passed to the man ager The command contains the initial parameters for the module which are then evaluated in method CreateModule of abb Factory cmd new dabc CmdCreateModule abb WriterModule ABB Sender WriterThread ABB_COMPAR_BUFSIZE readsize ABB_COMPAR_STALO
202. s for the element offsets in 32 bit It can therefore address offsets up to 16GB in the file If larger files are needed the table can be created with 64 bit values giving unlimited addressing 14 9 MBS upgrade for DABC 103 Note this file format needs the rewind file function because the file header must be rewritten to store iMaxWords iElements and optionally the offset of the index table This function is currently not implemented in the RFIO package but will be done 14 9 4 MBS data structures All structures are defined independent on endianess When bytes must be swapped always 4 bytes are swapped Fields 8 bytes long must be handled separately Smaller items must be accessed by mask amp shift This makes code independent of endian 14 9 4 1 Connect to MBS transport Structure used to talk between client and transport server Client connects to server MBS and reads this structure first Structure maps the sMosTransport Info info buffer typedef struct uint32_t iEndian byte order Set to 1 by sender uint32_t iMaxBytes maximum buffer size uint32_t iBuffers buffers per stream should be 1 for DABC mode uint32_t iStreams number of streams 0 for DABC mode sMbsTransportInfo 14 9 4 2 Buffer header Buffer header maps s_bufhe some fields used in different way The main difference is the usage of iUsedWords for the data length typedef struct
203. sed to the manager The command wraps also some initial parameters for the device which are then evaluated in method CreateDe vice of abb Factory define READADDRESS 0x8000 gt gt 2 define READSIZE 16x1024 std string devname ABB dabc Command dcom new dabc CmdCreateDevice abb Device devname c_str arguments class name device name set additional parameters for abb device here dcom gt SetInt ABB _PAR_BOARDNUM BOARD_NUM dcom gt SetInt ABB_PAR_BAR 1 dcom gt SetInt ABB _PAR_ADDRESS READADDRESS dcom gt SetInt ABB PAR LENGTH readsize res manager Execute dcom DOUT1 CreateDevice s DBOOL res Here the parameter names e g ABB_PAR_ADDRESS use the string definitions as setin abb Factory h The parameter values are defined locally e g READADDRESS however the DMA transfer size readsize may be set by the executables s first command line parameter Boolean variable res contains the result of the command execution true or false which is printed as debut output to the terminal with the DOUT1 macro 3 It creates a abb ReadoutModule by means of a command CmdCreateModule which is passed to the man ager The command wraps also some initial parameters for the module which are then evaluated in method CreateModule of abb Factory cmd new dabc CmdCreateModule abb ReadoutModule ABB Readout ReadoutThread arguments class name module name thr
204. so For some plug ins there may be also small test executables with different Makefiles in subfolder test These can be optionally build and executed independent of the DABC runtime environ ment The DABC working directory for the new plug in will be located in subfolder applications plugin name For the ABB example the application will set up a builder network with optional Active Buffer Board readouts so this is at SHOME mydabcpackages applications bnet test As in this example there may be an additional library to be build containing the actual Application class This is done by invoking the Makefile within the directory cd SHOME mydabcpackages applications bnet test make Here the application library is produced directly on top of the working directory SHOME mydabcpackages applications bnet test libBnetTest so The actual locations of the newly build libraries plugins and optionally applications part has to be edited in the lt lib gt tag of the corresponding DABC setup file here Set upBnet IB ABB xm1 The default set up examples in the plug in packages assume that the library is located at SDABCSYS 1lib as it is in the alternative installation case as described in section 3 4 1 page 18 20 DABC User Manual Setup Chapter 4 DABC User Manual GUI user user gui tex 4 1 GUI Guide lines The current DABC GUL is written in Java using the DIM software as communication layer The standard part
205. standard DIM control system This GUI may also control MBS systems which support the DIM communication It is extendable by user written components 2 2 Introduction The the following sections we give a short introduction to the main components and terms of DABC Figure 2 1 should be helpful 2 2 1 Modules All processing code runs in module objects There are two general types of modules synchronous and asynchronus A synchronous module may block for longer time waiting for data and must therefore run in its own computing thread Asynchronous modules must never block Therefore several of them may run as a chain in one single thread 6 DABC User Manual Introduction Object manager DABC Module DABC Module process Central data manager Memory pools Bufferqueue Bufferqueue Nemo Figure 2 1 Components and data flow 2 2 1 1 Synchronous module Each synchronous module is executed by a dedicated working thread The thread executes a method MainLoop with arbitrary code which may block the thread In blocking calls of the framework re source or port wait optionally command callbacks may be executed implicitly A timeout may be set for all blocking calls this can optionally throw an exception when the time is up On timeout with ex ception either the MainLoop is left and the exception is then handled in the framework thread or the MainLoop itself c
206. t lt lt ce nie nee 1 030 Simmel node mode rr co sacara aaa ea 149357 Mult node mle rear nennen 14 95 8 MBS controlled by DIM 22 bw sh ner ren 14 109 Listof icons ee nc kd ee bac aa E a a 15 DABC Programmer Manual Example Bnet 15 1 Overview 15 2 Controller applicaen aida eh a ee ae 15 3 Worker application o cscs e 2400 ge ek BG ae 15 4 Combiner modile cis a el A TA a 15 3 Network topoly ses be eR ee ER ee EES bee Sede we do 136 Event builder mod le oo 2 00 Re na ne en 15 Wiltermodile 2 2 26 0484 040 ae VS ESA ba a eee RS 15 BNET test application 2 eo cage 2 2 sun ae RR A da dda d oye 15 9 BNET for MBS application os c eos ow a ceu a a a a a a a a 16 DABC Programmer Manual Example ROC 16 1 Overview 10 2 Device andiranspornt o ces aeaee a tn te R 16 3 Combiner module p zo cc ce 164 Calibration module o lt 22 es cs 688 arsenate ma Era Ca ER aka 102 102 102 103 103 103 103 104 104 105 105 105 105 105 106 106 106 108 108 113 113 113 114 115 115 116 116 116 117 X CONTENTS 16 5 Readoutapplication lt es se scoa cu raa eee ehren 121 199 Faser AE 122 16 7 Saumeandcompihhen 2 ee sesi e s a aa T 122 16 8 Running the ROC application o o ca eeoa ea c eaw na a a nn 122 17 DABC Programmer Manual Example PCI 125 VED GENEN o o eo eo ee O wh gw ee eh 125 172 PCI Device and Transport o See en a a a 125 172 1 peace ecards Oe Be Parke bs 1
207. t the reference counters of the referred memory blocks are decremented If a reference counter becomes zero the memory is marked as free in the memory pool 9 2 6 Ports Buffers are entering and leaving a module through dabc Port objects Each port has a buffer queue of configurable length A module may have several input output or bidirectional ports The ports are owned by the module Depending on the module type there are different possibilities to work with the ports in the processing functions of the module These are described in section 12 2 5 for dabc ModuleSync and section 12 2 6 for dabe ModuleAsync respectively 9 2 7 Transport Outside the modules the ports are connected to dabc Transport objects On each node a transport may either transfer buffers between the ports of different modules local data transport or it may connect the module port to a data source or sink e g file i o network connection hardware readout In the latter case it is also possible to connect ports of two modules on different nodes by means of a transport instance of the same kind on each node e g InfiniBand verbs transport connecting a sender module on node A with a receiver module on node B via a verbs device connection 9 2 8 Device A transport belongs to a dabe Device object of a corresponding type that manages it Such a device may have one or several transports The threads that run the transport functionality are created by the
208. t parameter values can be observed and changed by a control system When a parameter value is changed in the program by a SetPar method the control system is informed and represents such change in an appropriate GUI element On the other hand if the user modifies a parameter value in the GUI the value of the parameter object will be changed and the corresponding parent object Module Device gets a callback via virtual method ParameterChanged By implementing a suitable reaction in this call one could reconfigure or adjust the running program on the fly A parameter object may be fixed via Parameter SetFixed method This disables possibility to change the parameter value both from the program and the control configuration system side Only when the fixed flag is reset to false the parameter can be modified again 91 92 DABC Programmer Manual Setup Not all parameters objects should be visible to the control system Each parameter has a visibility flag which is assigned to the parameter instance when it is created Only when Parameter IsVisible returns true parameter will be known visible to the control system Even if parameter is seen from control system it only can be changed from control system when flag Parameter IsChangable returns true Default flags values for newly created parameters can be set in WorkingProcessor SetParDfits function For visibility user should specify level which is compared with globa
209. te by the external control system This may be done by 54 DABC Programmer Manual Overview calling state change methods of the manager or by submitting state change commands to the manager The finite state machine itself is not necessarily part of the manager but may be provided by an external control system In this case the manager defines the states but does not check if a state transition is allowed However the DABC core system offers a native state machine to be used in the controls implementation it can be activated in the constructor of the dabc Manager subclass by method InitSM Some of the application states may be propagated to the active components modules device objects e g the Running or Ready state which correspond to the activity of the thread Other states like Halted or Failure do not match a component state e g in Halted state all modules are deleted and thus do not have an internal state The granularity of the control system state machine is not finer than the node application DoError Q DoHalt x DoConfigure eo 0 DoHalt DoHalt Figure 9 1 The finite state machine as defined by the manager There are 5 generic states to treat all set ups Halted The application is not configured and not running There are no modules transports and devices existing Configured The application is mostly configured but not running Modules and devices are created Local port connections are d
210. ted in application defined subclass e g bnet MbsGeneratorModule and is called frequently in module s MainLoop 2 Each filled buffer is forwarded to the single output port of the module bnet CombinerModule Subclass of dabc ModuleSync Framework prototype class to format inputs from several readouts to one data frame e g combine an event from subevent readouts on that node 1 It provides memory pools handles and one input port for each readout connection either bnet GeneratorModule or connection to a readout transport 2 Creates output port for combined subevents 3 The formatting functionality is to be implemented in method MainLoop of user defined subclass e g bnet MbsCombinerModule bnet SenderModule Subclass of dabc ModuleAsync Responsible for sending the subevents data frames to the receiver nodes according to the network traffic schedule as set by the Bnet cluster plugin 1 It has one input port that gets the event packets or time sorted frames from the preceding Bnet combiner module The input data frames are buffered in the Bnet sender module and analyzed which frame is to be sent to what receiver node This can be done in a non synchronized round robin fashion or time synchronized after a global traffic schedule as evaluated by the Bnet cluster plugin 2 Each receiver node is represented by one output port of the Bnet sender module that is connected via a network transport tcp socket InfiniBand ve
211. the period of the timer is set via configuration parameter Period How will its value be defined First of all it will be checked if a parameter of that name exists in command cmd If not the appropriate entry will be searched in the DABC setup file as discussed in Section 3 3 page 15 of the DABC user manual If the configuration file also does not contain such parameter the specified default value 1 0 will be used 13 5 Usage of commands for configuration Let s consider the possibility to configure a module by means of the Command class Here the use case is that an object like a module should be created with fixed parameters ignoring the values specified in the configuration file In our example one can modify InitMbsGenerator function in the following way extern C void InitMbsGenerator dabc Command cmd new dabc CmdCreateModule mbs GeneratorModule Generator cmd gt SetInt SubeventSize 128 if dabc mgr gt Execute cmd EOUT Cannot create generator module exit 1 Here one adds an additional parameter of name SubeventSize to the CmdCreateModule object which will set the MBS subevent size to 128 The generator module constructor will get the parameter value via method GetCfgInt as described in section 13 4 Since the parameters of the passed cmd object will override all other settings here the value of the corresponding lt SubeventSize gt entry in the configuration
212. this case the library is build into a subdirectory named like SARCH 1ib where SARCH is the current CPU architecture for instance 1686 16 8 Running the ROC application To run the readout application an approprite XML configuration file is required There are two examples of configuration files in SDABCSYS applications roc File Readout xml configures the readout from 3 ROCs lt xml version 1 0 gt lt dabc version 1 gt lt Context name Readout gt lt Run gt lt lib value libDabcMbs so gt lt lib value libDabcKnut so gt lt logfile value Readout log gt lt Run gt lt Application class roc Readout gt lt DoCalibr value 0 gt lt NumRocs value 3 gt RocIp0 value cbmtest01 gt RocIpl value cbmtest02 gt RocIp2 value cbmtest04 gt BufferSize value 65536 gt NumBuffers value 100 gt TransportWindow value 30 gt RawFile value run090 1md gt bsServerKind value Stream gt lt MbsFileSizeLimit value 110 gt lt Application gt lt Context gt lt dabc gt Because this is a single node application it can be started directly from a shell by calling the standard dabc_run executable with the configuration file name as argument dabc_run Readout xml This executable will load the specified libraries create the application configure it and switch the system in the Running state File Calibr xml shows the special case of
213. thods for working with ports kind Create Count Access Search input CreateInput name NumInputs Input unsigned InputNumber output CreateOutput name NumOutputs Output unsigned OutputNumber inp out CreateIOPort name NumIOPorts IOPort unsigned IOPortNumber A port usually should be created in the module constructor As first argument in the creation methods a unique port name should be specified As second argument the pool handle should be specified this defines the memory pool where necessary memory can be fetched for the transports associated with the port The length of input or and output queue defines how many buffers can be kept in corresponding queue One also can specify the size of user header which is expected to be transported over the port it is important for further transport configurations Any kind of port can be found by name with FindPort method But this is not the fastest way to work with ports because string search is not very efficient One better should use in code methods like NumInputs and Input unsigned for input ports where the port id number i e the sequence number of port creation is used Class dabc Port provides methods Send and Recv to send or receive buffers While these are non blocking methods one should use CanSend and CanRecv methods before one can call transfer operations 12 2 3 Parameters and configurations Paramete
214. tion command Configure on master node and waits for the transition All plug in components are created Then execute Enable Waits until all workers go into Ready state Now the DABC is ready to run Triggers the main GUI Update gt Start acquisition Executes Start command All components go into running state Running 4 2 GUI Panels 25 u Pause acquisition Executes Stop command All components go into standby state Ready Halt acquisition Executes Halt command This closes all plug ins States go into Halted Next must be shut down or configure ts Exit all processes by EXIT commands After 2 seconds trigger the main GUI Update a Shut down all processes on all nodes by script This is the hard shut down aD ssh shell script execution on master node 4 2 3 Action in progress ya Work in progress please wait a a Bd u Current action Figure 4 4 Launching progress When starting up configure or shut down the GUI has to wait until the front ends have completed the action During that time a progress window similar to the one shown in Fig 4 4 page 25 pops up Please wait until the popup disappears 4 2 4 MBS control panel To control and monitor a stand alone MBS system a dedicated control panel is provided by the MBS application This panel is described in the MBS section 5 1 2 page 31 4 2 5 Combined DABC and MBS control panel To control and monitor MBS front ends with DABC event builders a ded
215. tion should be called in a receiving callback of the control system communication layer passing the received command representation to the core system Here the command object is unstreamed again forwarded to its receiver and executed This is the pendant to virtual method SendOverCommandChannel which should implement the sending of a streamed command from the core to a remote manager by transport mechanisms of the control system 10 3 3 Default implementation for DIM The DABC default controls and configuration system is based on the DIM library 3 and is marked by namespace dimc for DIM Control The main classes are described in the following 10 3 Control system plug in 69 10 3 3 1 dimc Manager Implements the control system interface of dabc Manager as described above 1 It uses the default state machine module of the DABC core system This is activated in the constructor by calling InitSMmodule Thus virtual method InvokeStateTransition is not re implemented here 2 It exports a dedicated dabc StatusParameter that is synchronized with the value of the core state machine in ParameterEvent This parameter is required to display the state of the node on the generic Java GUI 3 It applies the generic dabc Configuration for setting up the node properties The standard executable dabc_run will create this object from parsing an XML file 4 The other interface functionalities use one component of class dimc Registry
216. to the framework and let them be available by name at runtime 12 2 Modules DABC provides dabc Module class which plays role of data processing entity in framework In this class necessary components like pool handles ports parameters timers are organised Class dabc Module has two subclasses dabc ModuleSync and dabc ModuleAsync which provides two different paradigms of data pro cessing within explicit main loop and via event processing respectively Before we discuss these two kinds of modules let s consider components which can be used with both types of the module 12 2 1 Pool handles Class dabc PoolHandle should be used in any module to communicate with dabc MemoryPool By creating a pool handle with method CreatePoolHandle the module declares that it wants to use buffers from the memory pool as specified by name More than one pool handles can be used in one module A pool handle can be accessed with method dabc Module FindPool via name or with method dabc Module Pool via handle number started from 0 If a pool of the given name does not exist it will be created automatically at the time of the first request Buffer size and the number of buffers which are specified in the CreatePoolHandle call play a role in this case only 81 82 DABC Programmer Manual Plugins 12 2 2 Ports Class dabc Port is the only legal way to transport buffers from to the module Class dabc Module provides following me
217. top Layout is mandatory Declarations have been masked out in the code snippets There are some 18 7 Application specific GUI plug in 147 icons one could use for the prompter panels rn P ee y usericonblue l usericonred l usericongreen public MiniPanel super MyPanel menulcon xSet getIcon icons usericongreen png name new String MyPanel tooltip new String Launch my panel layout xSet getLayout name if layout null layout xSet createLayout name new Point 100 200 new Dimension 100 75 1 true The simple functions to be implemented for the interface xiUserPanel we do not provide a command formatting function are public String getToolTip return tooltip public String getHeader return name public Imagelcon getIcon return menulcon public xLayout checkLayout return layout public xiUserCommand getUserCommand return null The init is called once after constructor Here we have to setup all panels We have in the main panel three lines one text prompt a text button and a check box At the bottom we have one icon button which would open the display frame There are some icons one could use for that _t CO it Ej windowblue H windowred Ej windowgreen public void init xiDesktop desktop ActionListener al desk desktop save prompt addPrompt My Command Defaultvalue prompt 20 this addTextButton This
218. tore raw combined data after CombinerModule CalibrFile name of 1md file to store calibrated data after CalibrationModule MbsFileSizeLimit maximum size of each file in Mb If the written data would exceed this size a new output file is automatically created with a sequence number appended to the file name Three calibration modes are supported e DoCalibr 0 Only the CombinerModule is instantiated which produces a kind of ROC raw data e DoCalibr 1 Both CombinerModule and CalibrationModule are instantiated e DoCalibr 2 Only the CalibrationModule is instantiated This is used to convert raw data read from Imd files into the calibrated format In all modes output in form of raw or and calibrated data can be stored in 1md file s defined by RawFile and CalibdFile parameters respectively The last mode is a special case since RawFile does not specify the output but the input file for the calibration module 122 DABC Programmer Manual Example ROC 16 6 Factory Factory class roc Factory implements several methods to create the ROC specific application device and mod ules as described in section 12 5 16 7 Source and compilation The source code of all classes can be found in SDABCSYS plugins roc directory Compiled library 1ibDabcKnut so is in directory SDABCSYS 1ib If one needs to modify some code in this library one should copy the sources to a user directory and call make in this directory In
219. transport The ROC device class roc Device inherits from two classes dabc Device and SysCoreControl where SysCore Control provides simultaneous access to several ROC boards Usually the instance of a device class corresponds to one physical device or board but here the device object is rather used as central collection of SysCoreBoard objects and as thread provider Each instance of roc Transport has a pointer to a SysCoreBoard object which handles data taking from a specific ROC The implementation of roc Transport is based on the dabc DataTransport class see section 12 3 5 which runs as WorkingProcessor with an asynchronous event handling mechanism so it does not require an explicit thread This feature allows to process several instances of such transports in the same thread In the ROC case all roc Transport instances use the thread of roc Device Let s have a look how roc Transport is working When the connected module starts method StartTransport is called which will invoke SysCoreBoard startDaq to start data taking After that the buffer filling loop consists in subsequent calls of Read_Size Read_Start and Read_Complete functions implementing the interface of dabc DataTransport see section 12 3 5 Method Read_Size defines the size of the next buffer required for data reading In case of roc Transport this size is fixed and is taken from a configuration parameter 119 120 DABC Programmer Manual Examp
220. trolled through DIM in two modes single and multimode In single mode the dispatcher is a DIM command server the message logger a DIM parameter server for messages and status information selected from DAQ status In multinode mode the prompter is the DIM command server and the message loggers are status servers The master message logger also is the DIM message server The table 14 1 page 106 shows an overview of the different operation modes Mode Interactive DIM GUI Remote GUI Single Dispatcher TTY DIM command Logger TTY file DIM status message file Multi Dispatcher TCP TCP TCP Prompter TTY DIM command TCP MasterLogger TTY file file file Msg server DIM status message Msg server TCP inputs TCP inputs TCP inputs SlaveLogger TCP TCP DIM status TCP Table 14 1 MBS operation modes 14 9 5 6 Single node mode There are two new scripts to start and shutdown a single node MBS dimstartup scand dimshutdown sc These are called by rsh from the GUI node Arguments are the path of MBSROOT and the user working path For starting the DIM server also the DIM name server node is passed dimstartup sc SMBSROOT SPWD SDIM_DNS_NODE launches m_dispatch dim after waiting for all 60xx sockets closed dimshutdown sc SMBSROOT PWD calls m_remote reset 1 When dispatcher is started with dim message logger is started with argv 1 dim otherwise task Then the DIM commands are defined and
221. ty and the number of columns in graphics panels All existing layouts are stored with the save setup button and restored on startup 150 DABC Programmer Manual GUI References J rn Adamczewski Musch Hans Georg Essel and Sergei Linev The go4 system homepage http go4 gsi de CBM collaboration Cbm experiment Technical status report Technical report GSI January 2005 Clara Gaspar Dim distributed information management system http dim web cern ch dim 2008 Andreas Kugel Wenxue Gao and Guillermo Marcus The active buffer board online documentation http cbm wiki gsi de cgi bin view DAQ ActiveBufferBoardV 1 2008 Walter FJ M ller The n xyter starter kit http cbm wiki gsi de cgi bin view NX YTER NXYTER StarterKit 2009 The Wikipedia Finite state machine http en wikipedia org wiki State_machine 2009 151 152 REFERENCES Index Active Buffer Board 127 Active Buffer Board DMA read 129 DMA read and write 130 DMA write 130 overview 125 with Bnet 131 Bnet classes bnet BuilderModule 61 bnet ClusterApplication 60 bnet CombinerModule 61 bnet FilterModule 62 bnet GeneratorModule 61 bnet ReceiverModule 61 bnet SenderModule 61 bnet WorkerApplication 61 Conventions DIM service names 133 Core classes dabc Application 8 53 60 dabc Basic 57 dabc Buffer 7 52 dabc Command 6 52 57 dabc Device 8 53 59 dabc Factory 60 dabc Manager 52 59 63 dabc Mem
222. ue Cyan Magenta Yellow text char line of text Format L 1 C 16 C 128 18 2 2 7 Record ID 9 Command descriptor This is an invisible parameter describing a command argument list The service name must be correlated with the command name e g by trailing underscore description char XMI string describing arguments 136 DABC Programmer Manual GUI Format C The descriptor string could be XML specifying the argument name type required and description Question if default value should be given here for optional arguments Example lt xml version 1 0 encoding utf 8 gt lt command name coml scope public content default gt lt argument name argl type F value 1 0 required req gt lt argument name arg2 type I value 2 required opt gt lt argument name arg3 type C value def3 required req gt lt argument name arg4 type boolean value required opt gt lt command gt The command definition can be used by the xGUI to build input panels for commands The scope can be used to classify commands content should be set to default if argument values are default values if argument values have been changed 18 2 2 8 Commands Commands have one string argument only This leaves the arguments to semantic definitions in string format To implement a minimal security the first 14 characters of the argument string should be an encrypted password 13 characters by crypt plus space
223. uilder would execute only on two nodes 4 2 7 Parameter table DABC parameters are DIM services as the commands The naming convention is the same The server providing parameters can be make them no visible and un changable DABC defines some special parameter types having a data structure and a specific interpretation like a rate parameter having a value limits a color and a graphic presentation A rate parameter is assumed to be changed and updated regularly The GUI displayes these special parameters in dedicated panels Parameters are used in all components of DABC The central place for all parameters in the GUI is the parameter table as shwon EJ Parameters SAS BEER Node Application Parameter Current Set value Ixi010 1970 Readout 42 CtriPoolsize BnetPlugin 2097152 Ixi010 1970 Readout 42 DABCYersion DABC C Ixi010 1970 Readout 42 DataRate Sender 0 0 Ixi010 1970 Readout 42 EventBuffer BnetPlugin i 524288 Ixi010 1970 Readout 42 EventPoolSize BnetPlugin i 4194304 Ixi010 1970 Readout 42 EventRate Combiner 0 0 Ixi010 1970 Readout 42 InfoMessage State ma RINOONOO Figure 4 6 Parameter table in Fig 4 6 page 26 The parameter table holds all parameters which are marked by the provider to be visible The parameter values can be changed in the Set value column if no minus sign is there in which case the provider does not grant modification The buttons in the Show
224. ule that run in different threads So the code is a merger of the above examples 17 4 1 and 17 4 2 17 5 Active Buffer Board with Bnet application 131 It applies the dabc Standalone Manager as most simple Manager implementation The abb Device is created by means of a command CmdCreateDevice which is passed to the manager The command contains the initial parameters for the device The DMA transfer size readsize same for both directions may be set by the executables s first command line parameter see section 17 4 1 for code example It creates a abb ReadoutModule by means of a command CmdCreateModule which is passed to the man ager see section 17 4 1 for code example 4 It creates a abb WriterModule by means of a command CmdCreateModule which is passed to the manager see section 17 4 2 for code example The transport connections of the abb Device both with the input port of the reader module and the output port of the writer module are established by invoking method CreateTransport of the manager see sections 17 4 1 and 17 4 2 for comments on the code Both modules are started with manager StartModule The main process sleeps for 60 seconds during the DMA transfer then it stops both modules again After the modules have stopped their internal dabc Ratemeter instances will print some average data rate values to the terminal Finally the manager is cleaned up and the program ends Ne W n fon 1
225. un gt lt Module name Generator gt lt NumSubevents value 3 gt lt FirstProcId value 77 gt lt SubeventSize value 128 gt lt Go4Random value false gt lt BufferSize value 16384 gt lt Port name Output gt lt OutputQueueSize value 5 gt lt MbsServerKind value Stream gt lt MbsServerPort value 6006 gt lt Port gt lt Module gt lt Context gt lt dabc gt There is only one Context node specified by the nodename with one simple C function InitMbsGenerator to run and with one Module that produces the event data as specified in its parameters The data server is specified by parameters of the Output Port The tag MbsServerKind can be Stream or Transport to emulate either variant of the standard MBS server sockets A complete description of this example can be found in Programmer Manual section 14 7 page 99 The setup files for standard MBS use cases can be found in directory SDABCSYS applications mbs 47 48 DABC User Manual DABC Application ROC 8 2 ROC event building A more practical use case is to prepare data as MBS events that was read by DABC from external front end hardware This is shown with the setup file for the readout controller ROC example see the full description of this example in Programmer Manual chapter 16 1 page 119 lt xml version 1 0 gt lt dabc version 1 gt lt Context name Readout gt lt Run gt lt lib value
226. ut fCnt dabc Bufferx buf Output fCnt lt 5 gt CanSend usleep 10 Input gt Recv 2 gt Send buf Here the while loop can wait for an infinite time until the output port will accept a new buffer and during this time the complete thread will be blocked As both processing methods are the same instead virtual void ProcessUserl while dabc Buffer buf Output fCnt 5 Event dabc Moduleltemx Input gt CanRecv in the example one can implement central ProcessUserEvent method uint16_t o 5 2 gt CanSend amp amp Output fCnt Input gt Recv 2 gt Send buf 86 DABC Programmer Manual Plugins To introduce time dependent actions in dabc ModuleAsync one should use timers Timer objects can be cre ated with method CreateTimer It delivers a timer event with specified intervals which can be processed in ProcessTimerEvent method One can modify the previous example to display the number of transported buffers every 5 seconds RepeaterAsync const char name dabc ModuleAsync name CreateTimer Timerl 5 virtual void ProcessUserEvent dabc Moduleltemx item uint16_t evnt if evnt dabc evntTimeout DOUT1 Buffers count d fCnt 12 2 7 Special modules For special set ups e g Bnet the framework provides dabc Module subclasses with generic functionality e g bnet BuilderModu
227. ut of frames Saved by main save button Default Layout xml Chapter 5 DABC User Manual MBS GUI user user gui mbs tex 5 1 MBS event building 5 1 1 MBS setup Any MBS system can be controlled by the DABC GUL It can run in two operation modes with MBS event builder or DABC event builder see 6 1 page 39 The first case means a standard MBS system To control a standard MBS nothing has to be done by the user on the MBS side The node running the GUI must get granted rsh access at least to the MBS node where the prompter shall run Note however that in the user s MBS startup file typically startup scom the m_daq_rate task must be started as last task this is probably the case already This task calculates the rates The GUI waits for this task after execution of the startup file Because MBS has no states there is no other way to know when the startup has finished Of cause the MBS itself must have been built with the DIM option since version v5 1 Central log file is written as usual Optionally one can provide a text file with specifications which parameters shall be published by DIM see 5 3 1 page 36 For the standard MBS control one needs no DABC installation The GUI jar file is sufficient DIM must be installed See installation guide on the download page 5 1 2 MBS control panel Fig 5 1 page 32 shows the panel to be used to control a standard MBS The values are restored from file MbsControl xml default may be
228. utdown procedure in prompter Prompter dispatcher and message loggers should still be running ts Shut down all Execute script prmshut down sc at master node After 2 seconds trigger the main Update 6 Show acquisition Output in log panel g Shell script executes command on master node 5 1 3 MBS command panel 7 Commands EI MbsTransport Command OpenFile scope MBS o CloserFile name C OPT gt CI ConnectRfio tape L OPT o EJ DisableTcp sizemb L OPT gt C DisconnectRfio number L OPT 3 DismountTape first L OPT y Gad EnableDabc inhead C OPT amp E EnableTcp outhead C OPT gt InitTape gt EI MountTape auto gt C OpenFile prompt o 4 SetFileheader edit gt 3 ShowFile disk o C ShowTape rfio _ o 3 TypeEvent multi J gt EI Typestream x gt 9 MbsUtil first gt 3 MbsYme_serv Figure 5 2 Command panel Fig 5 2 page 33 shows on the left side the command tree Double click or RETURN on a command executes the command The top tree level is the executing MBS task below that are the commands and the master node prompter node is the only node below each command However command is sent to the prompter node but executed on the current node which is displayed in the info panel see Fig 5 4 page 34 Click on a command opens at the right side the argument panel Entering argument values and
229. veral transports The threads that run the transport functionality are created by the device If the Transport implementation shall be able to block e g on socket receive there can be only one transport for this thread A device object usually represents an I O component e g network card There may be several device objects of the same type in an application scope The device objects are owned by the manager singleton transport objects are owned and managed by their corresponding device A device is persistent independent of the connection state of the transport In contrast a transport is created during connect or open and deleted during disconnect or close respectively A device may register parameters and define commands This is the same functionality as available for modules 2 2 9 Application The Application is a singleton object that represents the running application of the DAQ node i e one per system process It provides the main configuration parameters and defines the runtime actions for the different control system states see Programmer Manual section 9 3 1 In contrast to the Manager implementation that defines a framework control system e g DIM EPICS the Application defines the experiment specific behaviour of the DAQ 2 3 Controls and configuration 2 3 1 Finite state machine The running state of the DAQ system is ruled by a Finite State Machine 6 on each node of the cluster The manager provide
230. vered by the memory pool is contiguous Several buffers may refer to the same fragment of memory Therefore the memory as owned by the memory pool has a reference counter which is incremented for each buffer that refers to any of the contained fragments When a consumer frees a buffer object the reference counters of the referred memory blocks are decremented If a reference counter becomes zero the memory is marked as free in the memory pool 2 2 6 Ports Buffers are entering and leaving a module through Port objects Each port has a buffer queue of config urable length A module may have several input output or bidirectional ports The ports are owned by the module 8 DABC User Manual Introduction 2 2 7 Transport Outside the modules the ports are connected to Transport objects On each node a transport may either transfer buffers between the ports of different modules local data transport without copy or it may connect the module port to a data source or sink e g file i o network connection hardware readout In the latter case it is also possible to connect ports of two modules on different nodes by means of a transport instance of the same kind on each node e g InfiniBand verbs transport connecting a sender module on node A with a receiver module on node B via a verbs device connection 2 2 8 Device A transport belongs to a Device object of a corresponding type that manages it Such a device may have one or se
231. with numbuffers buffers of size buffersize If a pool of this name already exists it will be extended The numincrement value specifies with how many buffers at once the memory pool can op tionally be extended on the fly Optional arguments headersize and numsegments may define the buffer header size and the partition of the buffer segments resp The MemoryPool mechanisms are discussed in detail in section 11 1 Method returns true or false depending on success 66 DABC Programmer Manual Manager MemoryPool FindPool const char name Access to memory pool by name name Returns 0 if not found bool DeletePool const char name Delete memory pool of name name Returns true or false depending on success 10 2 7 Miscellaneous methods bool CleanupManager int appid 0 Safely deletes all modules memory pools and devices with specified application id appid The default id 0 effects on all user components In the end all unused threads are also destroyed virtual void Destroy Object Basic obj Deletes the referenced object obj in manager thread Useful as safe replacement for call delete this void Print Displays list of running threads and modules on stdout 10 3 Control system plug in For the common DABC usage the provided standard control and configuration system featuring DIM protocol 3 XML setup files and a generic Java GUI will probably be sufficient However if e g an experiment control system is already e
232. working thread Or the user may implement callbacks for special events e g ProcessInputEvent ProcessOutputEvent ProcessPoolEvent that are invoked when the corresponding event is processed by the working thread The events are dispatched to these callbacks by the ProcessUserEvent default implementation then There are no blocking function available in dabc ModuleAsync but the user must avoid any polling loops waiting for resources event processing function must be returned as soon as possible dabc Port A connection interface between module and transport From inside the module scope only the ports are visible to send or receive buffers by reference Data connections between modules i e transports between the ports of the modules are set up by the application using methods of dabc Manager which specify the full module port names For ports on different nodes commands to establish a connection may be send remotely via controls layer e g DIM and handled by the manager of each node dabc Transport A producing or consuming entity for buffers which it delivers to or receives from resp a Module via the Port interface As an example dabc NetworkTransport implements the transport between modules on different nodes dabc Device Device class used for creation and configuration of transport objects Is a subclass of dabc WorkingProcessor The dabc Transport and dabc Device base classes have various implementa tions
233. ws The dark line at the bottom shows a list of active DIM servers The other windows are placed in the white middle pane The functions of the buttons and the invoked panels is described in the next sections Depending on the application some buttons may be not seen additional ones may show up If one does not work with MBS plug ins the control panels for MBS are of cause not useful Fig 4 2 page 22 shows a more typical view of a running DABC In general all panels including the DABC Controls Ore 50 E35 BE RAW 3 SE A DabcMbsLauncher OK DABC ready Name server x90523 gsi de EI DoConfigure User name goofy gt 3 Controller 41 Password RET 9 EI Eventbuilder 42 T Commands Command DoConfigure scope Com MBS master node MBS servers DABC master node DABC master name DABC servers MBS system path MBS user path MBS script MBS command DABC system path DABC user path DABC setup file DABC script MBS launch file DABC launch file x869 4 3 Ixg0523 gsi de Controller 41 5 daq usr goofy mbswork v5 1 Ixi013 gsi de 1971 1xi014 gsi de 1971 ES Readout 42 DoEnable DoError DoHalt DoStart v5 0 x86 newmbs script remote_exe sc m_rising v50 x86 newmbs x86g 4 x86 7 imisc goofy snifffdabc miscigoofy dabc work SetupMbs xml ps MbsLaunch xml DabcLaunch xml O0B O 000502 Dostop
234. xDesktop to build the command descriptor list Function setUserCommand is called from xDesktop to specify axiUserCommand object which provides a function getArgumentStyleXml which is used to determine how the command string has to be formatted either like the command XML description or like the MBS style 18 6 3 Startup sequence The build up sequence during the GUI start is done in the xDesktop Sequence on startup Create application panels and graphics panels Create browser xDimBrowser and call its initServices Create prompter panels Create xPanelParameter Call browser enableServices function Now all parameters DIM clients should already operate Create xPanelCommand and call its setCommandDescriptors The descriptors are provided as parameters The descriptor list is generated by xPanelParameter 7 Call init and setDimServices of all application panels Pass xiUserCommand object from first application panel object to xPanelCommand 8 Create the internal frames to display all panels which shall be visible Du Un 18 6 4 Update sequence The update sequence is either triggered by a menu button interactively or invoked in callback functions of prompter panels after changes of the DIM services The update is done in actionPerformed of xDesktop command Update Sequence on update 1 Call releaseDimServices of all application and prompter panels 2 Call xDimBrowser releaseServices which deactivates all paramete
235. xisting and the data acquisition shall be handled with the same means it might be necessary to adjust DABC to another controls and configuration framework Moreover future developments may replace the current standard control system by a more powerful or a more convenient one Because of this the connection between the DABC core system and the control system implementation was designed with a clear plug in interface Again the dabc Manager class plays here a key role This section covers all methods and mechanisms for the control system plug in As an example part 10 3 3 describes in detail the standard implementation as delivered with the DABC distribution 10 3 1 Factory A new control system plug in is added into DABC by means of a dabc Factory subclass that defines the method bool CreateManagerInstance const char kind dabc Configuration cfg This method should create the appro priate dabc Manager instance and return true if the name kind as specified by the runtime environment matches the implementation The default DABC runtime executable will also pass a configuration object cfg read from an XML file which may be passed to the constructor of the Manager As it s mandatory for other DABC factories the dabc Factory for the manager must be instantiated as global object in the code that implements it This assures that the factory exists in the system on loading the corresponding library 10 3 2 Manager Besides its role as
236. xplicitly take out the Buffer from the guard with BufferGuard Take to avoid such automatic release in a normal situation A typical usage of dabc BufferGuard is shown here dabc BufferGuard buf pool gt TakeBuffer 2048 port gt Send buf Take Class dabc ModuleSync provides several methods to work directly with dabc BufferGuard this allows to correctly release a Buffer in case of any exception which otherwise may not be handled correctly by the user 11 1 6 Allocation There are several methods how a MemoryPool can be created e Automatically when the user tries to access it via a PoolHandle the first time e using dabc Manager CreateMemoryPool method e using dabc CmdCreatePool command Automatic creation is useful for simple applications with a few modules In this case the parameters specified by the PoolHandle size and number of buffers are used But in many situations it is good to create a memory pool explicitly setting all its parameters directly or from a configuration file Typically the memory pool is created by the user s Application class in method CreateApp Modules called by state change command DoConfigure In simple case bool UserApplication CreateAppModules dabc mgr gt CreateMemoryPool WorkPool 8192 100 One can call Create MemoryPool method several times to create memory blocks for different buffer sizes As alternative one can create and configure a command
237. y dabc WorkingProcessor implementation may register its own parameters Parameter can be used for con figuration of object at creation time via configuration file monitoring of object properties in GUI or ma nipulating of object properties at runtime changing parameter values via controlling interface Currently supported parameter types are dabc IntParameter simple integer value dabc DoubleParameter simple double value dabc StrParameter simple string value dabc StateParameter contains state record e g current state of the finite state machine and associated colour for gui representation e dabc InfoParameter contains info record e g system message and associated properties for gui representation e dabc RateParameter contains data rate record and associated properties for GUI representation May be updated in predefined time intervals e dabc HistogramParameter contains histogram record and associated properties for GUI representa tion dabc WorkingThread An object of this class represents a system thread The working thread may execute one or several jobs each job is defined by an instance of dabc WorkingProcessor The working thread waits on an event queue by means of pthread condition until an event for any associated working processor is 9 5 Main Classes 59 received then the corresponding event action is executed by calling ProcessEvent of the corresponding working processor dabc W
238. y means of the static handle dabc mgr gt The following section 10 2 describes such interface methods to be used by the programmer of the Module Trans port Device and Application classes In contrast to this section 10 3 gives a guide how to re implement the Manager class itself for a different control and configuration system This should be seldomly necessary for the common DAQ designer but is added here as a reference and as useful insight into the DABC mechanisms 10 2 Framework interface 10 2 1 General object management All objects are organized in a folder structure and can be accessed by the full path name However for most purposes it is recommended to rather use higher level Manager methods to cause some action e g StartModule than to work directly with the primitive objects Module FindModule const char name Access to a Module by name Returns 0 if module does not exist Port FindPort const char name Access to a Port by name Returns 0 if port does not exist Device FindDevice const char name Access to a Device by name Returns 0 if device does not exist Device FindLocalDevice Shortcut to get the local device that is responsible for basic transport mechanisms like transport of buffers through the local memory 63 64 DABC Programmer Manual Manager Factory FindFactory const char name Access to a Factory by name Returns 0 if factory does not exist Working Thread FindThread
239. ymore Otherwise returns true bool ConnectPorts const char portIname const char port2name const char devname 0 Connects module Port of full name portIname with another module Port of full name port2name A full port name consists of the module name and a local port name separated by forward slash e g Read out3 Output CombinerModule Input2 Optionally the Device name for the connection may be defined with argument devname By default the ports are connected with a FIFO like transport of queued Buffer references in local memory as managed by dabc LocalDevice 10 2 4 Thread management bool Make ThreadForModule Module m const char thrdname 0 Creates a thread for module m and assigns module to this thread If thread name thrdname is not specified module name is used Returns true of false depending on success bool MakeThreadFor WorkingProcessor proc const char thrdname 0 unsigned startmode 0 Creates thread for processor proc and assigns processor to this thread If thread name thrdname is not specified a default name is used Value of startmode specifies initial run state of the thread currently thread is started if startmode gt 0 10 2 5 Command submission bool Submit Command cmd This method generally submits a command cmd for execution The command is put in the queue of its command receiver working thread and is then asynchronously executed there The Manager will either forward the comman
240. ystem There are several reasons to prefer a command interface over direct calls of class methods e The execution of a command is performed not in the context of the calling thread but in the thread to which the command receiver object is assigned This allows to avoid unnecessary mutex locking e The execution of a command can be performed synchronous or asynchronous to the calling thread so one can easily specify a timeout for the command execution e A command can be submitted to any object in the system including objects on remote nodes e The code that invokes the command execution does not strongly depend on the code that executes the command the invoking client library must know a command base class and some common parameter names but not the implementation of the execution itself This allows to decouple the required libraries on different nodes e A command object can contain an arbitrary number of argument values and can also be used to return any number of result values 11 3 1 Command class Class dabc Command is a container for argument and result values The name of the command is the main identifier for the command action which is executed in the CommandReceiver object There are a number of methods to set get command parameters Type Getter Setter string GetStrO SetStrO int GetInt SetInt unsigned int GetUIntO SetUlnt bool GetBool SetBool double GetDouble SetDobleQ In all setter methods the

DABC manual

Contents

Download Pdf Manuals

Related Search

Related Contents