RD51 SCALABLE READOUT SYSTEM - Indico
Contents
1. Fig 1 Top left Photo of the first RD51 hybrid Mai 2010 with a wire bonded rad hard APV25 chip with 128 channels The bonding pitch is 88 micron on 4 row pads Standard RD51 hybrids have a 130 pin connector for MPGD s detectors a mini HDMI connector for readout and an optional flat cable connector for extension to a 2na hybrid Local LDO s and PLLs provide the voltages and clocks required by the chip Top right Photo of the bonding area between 128 pads of the APV chip and the micro etched bonding pads of the APV carrier board Bottom left The hybrid contains ESD diodes of less than 1pF capacitance and is to be grounded via lateral MMCX connectors for the chamber ground serving also as a snap in fixation of the hybrid The hybrid can be configured for AC or DC coupling and as a master or slave device for daisy chained readout of 2 hybrids via 1 HDMI cable Bottom right Photo of the APV hybrid mounted on a GEM chamber The lateral MMCX connector brings the chamber ground to the hybrid First data taken with this hybrid are shown above Below Two APV hybrids master and slave version can be read out via a single HDMI cable The two hybrids get connected via a short extension flat cable 6 Augut 11 2010 ike RD51 SCALABLE READOUT SYSTEM APV hybrid details Hybrid extension cable 2 hybrids 1 HDMI cable RD51 standard connectors chamber ground and screwless fixation Jack on hybrid Samtec MMCX P P H ST 2 5 M2. Pixel Area um x um w v MEDIPIX 2 256x256 55x55 100e ph counting MEDIPIX 3 256x256 55x55 72e phrcounting timd Augut 11 2010 ieee RD51 SCALABLE READOUT SYSTEM The Scalable Readout System SRS implements the above requirements in the following practical framework e test systems need a few channels chips that can be connected to a turnkey and low cost readout system with a standard Online system e large system like an LHC experiment use the same readout backends but with a much larger number of frontend chips and frontend cards e typically detectors need some special features for this SRS allows for adapter cards to be integrated into SRS with application specific logic The first SRS system components consists of the following choices from the top of the SRS block diagram below A Common system components of SRS e The DATE Online system of the ALICE LHC experiment stable and user friendly its use by RD51 is based on a MoU with the ALICE DATE team e 10 Gigabit Ethernet as commercial very high performance standard for readout links based on copper or fibre e The Scalable Readout Unit SRU as a 40 fold DTC link concentrator between the frontend system and the 10Gigabit port of DATE not needed for small systems e Programmable Front End Cards FECs that fit in a cheap mechanical framework 6U x 220 Euro card with PCle connectors to interface adapter cards e DTC link cables CAT6 for transferring Da
3. for analogue chip links Augut11 2010 kiipi RD51 SCALABLE READOUT SYSTEM Both FEC and ADapte cards fit in the same Eurocrate slot with front panels on both sides The interface chosen for SRS are three PCle connectors These provide power from the FEC card high speed I O and 12C control for the adapter card resources N WA U 9 ri A iat N gt UOTZRIOgeT IO LXGN STI Otoz OUNI TTA PRO 334 PI J032 749040 PUJ UOJ4 m HOT sall z la ae EH Z ies E a L a _ m a F _ rer Kk C D aad ime PS t ER sti KK PA 7 ee Z Pain TI os w gt m m ci iil K gt a ew t gt zzo S save gen 3 S L m S rin 12 58 m m e e EF cw gee ee 92 ri as 419 othe army TI 10 Augut 11 2010 kiipi RD51 SCALABLE READOUT SYSTEM First tests with FEC and ADC Adapter for APV hybrid El Bus Plot DEV 1 MyDevice1 XC5VLX50T UNIT 0 MyiLAO ILA Pe RRR REED APR OO OEE EEE 7 CT Plot 5 data vs time data vs data Display Bus Selection 4 lill gt MiniMax 1000 S 1200 1400 1600 1800 2000 Y age Y 2829 First digitized APV signals test pulse mode recorded by the FEC card via the HDMI link and ADC adapter card ABC of the adapter cards for the SRS system OO 6 L 11
4. information you consider to be wrong please send e Sharepoint Services mail to the website administrators Equally if you wish to include another front end chip in the database or improve the database in any way we welcome your feedback Links S a APV25 Official Website Beetle Chip Official Website Thank you for your collaboration The site administrators r Linear Chips N Add new link New gt Actions Settings gt Chip Name Experiment w Detectors X channels vA Announcements APV25 CMS silicon microstrip detectors CMS 128 GASTONE 17 02 2010 04 11PM by Sorin Martoiu AFTER T2K TPC TPC 72 A new chip has been added GASTONE a 64 channel binary FE chip for Cylindrical GEM MSGCROC DETNI Position Sensitive Microstip Gas Chambers 32 detectors KLOE experiment for Thermal Neutrons DETNI srs Chip Matrix has been 02 10 2009 12 37 PM Beetle LHCb Si strip MAPMT 128 lt published VFAT TOTEM 128 z by Sorin Martoiu NINO ALICE TOF 8 s S Add new announcement CARIOCA LHCb muon MWPC 8 MiMac Mlcro tpc Matrix of Chambers GEMs 16 Team Discussion Micromegas PASA ALTRO ALICE TPC 16 man cnr mn P aboia 4 P Vii There are no items to show in this view of the lt gt H Team Discussion discussion board To create a new item dick Add new discussion below For assistance with Access Web Datasheet see Help S Add new discussion Pixel Chips New Actions Y Settings Chip Name Modes
5. Augut 11 2010 kilo RD51 SCALABLE READOUT SYSTEM Fig 4 The FEC card can be interfaced to 3adapter card formfactors A B C and via 3 type of connectors connectors PCleX1 PCleX8 PCleX16 The relative sizes and combinations of A B and C cards relative to the FEC card are shown Users who want to design their own A B or C cards please contact us to receive the exact 3D outlines and connector positions to fit with the FEC card For the pinout and interface specifications we will prepare a document ad interim our reference designs FEC card and G card can be used Below View of the straddle mount PCle connectors that connect the A B C cards to the EFC card Gel VAJA AJA garmig 41 wn FEC Frontend Card h MAETI F V A 4 il i SHINI gt ANU ANNUAL oe MTT MT LAS Fig 5 Photo of the 1s FEC card by UPV Valencia with SFP connector for Gigabit Ethernet two RJ45 connectors for DTC links and or daisy chaining power connectors for LV and HV and the 3 straddle mount PCle connectors for A B or C size adapter cards The functionality of the FEC is based on 12 Augut 11 2010 kiili RD51 SCALABLE READOUT SYSTEM firmware for a Xilinx Virtex 5T FPGA with added DDR2 memory buffer First FEC cards have been produced and will be tested with readout firmware via Gigabit Ethernet and DATE in July August 2010 at CERN BELOW Connectivity details of the EFC card A user manual will be made avai
6. LRT H PES RD51 SCALABLE READOUT SYSTEM What is SRS short introduction status and outlook 11 August 2010 The RD51 Working Group 5 activities on common electronics for a multichannel readout system started its design phase in 2009 with the compilation of a chip knowledge base as a common base for electronics requirements for RD51 users The large variety in readout requirements signal polarity trigger concept timing resolution radiation tolerance analogue versus digital choice of readout bus number of channels power and packaging bandwidth data formats online software does not allow for a simple common solution unless one designs a system with the following general properties a common chip link interface for different readout chips on detector resident hybrids b scalability from a small to a large system based on a common readout backend with link interfaces for specific fontends integration of commercial standards for a minimum of custom hardware modules between the chip frontend and the online system d default availability of a very robust and supported data acquisition package e flexibility to implement different readout architectures and trigger schemes CHIP Matrix https espace cern ch rd51 wg5 chipmatrix default aspx Disclaimer 67 CERN This chipmatrix has been compiled using information available in the literature or on the internet talks JA posters etc If you wish to comment or correct any
7. OHM 0 6 GHz win Plug on chamber Samtec MMCX J P H ST Chip link between hybrid and FEC card N Augut 11 2010 kiipi RD51 SCALABLE READOUT SYSTEM FEC card Micro Mega Chamber sa TN gtd mendi HDMI cable HOM 4 34 Cable Fig 2 In this example frontend hybrids are connected via HDMI link cables to a SRS link adapter over up to 20 m Shown is an originally planned small A sized ADC card For better integration density however WG5 preferred to design an ADC card of size C full size like FEC card which can receive 8 HDMI cables 16 hybrids Bottom HDMI connector on hybrid and adapter and the transmission issues for the adapter card For more info see the Freiburg talk of Sorin Martoiu Output Pre Emphasis TMDS ChannelO 7 a TMDS Channel 2 1 away Hot Plug Detect no pre P pe j a with pre TMDS Channel1 T TMDS Clock Adapter card ADC for 16 analogue Chips Augut 11 2010 kiipi RD51 SCALABLE READOUT SYSTEM w 35 oe A d o niin huin ADC C Card Overview 2 Octal ADCs with LVDS serial outputs TI ADS5281 a 16 Analog Inputs a 40MHz sampling Support for a 8 MASTER hybrids a 16 MASTER SLAVE hybrids a Power distribution a Power Temp monitoring Forward compatibility for future hybrids Beetle Fig 3 TOP Photo of the ADC adapter card 12 cards produced in July 2010 Bottom Slide on adapter
8. VA H B AE AE 3536 ewe ne e i Aa AVA WAC WW A LRU A A AA AAA Fig 7 Top Design view of the new Scalable Readout Unit SRU The SRU is needed for large systems with more than a dozen FEC cards and gt 16 k Channels The co design at CERN and CCNU Wuhan for a new SRU integrates new 40nm Virtex 6 technology with four 5 Gigabit Ethernet output ports and 40 DTC inputs A DDR3 memory buffer extends the SRU s event buffer storage capacity to several Gigabytes A general purpose NIM and LVDS interface provides the user with all options for building a trigger and busy logic with external devices A TTC chip interface has been added for LHC clock and trigger applications July 2010 New SRU s are under design revision and expected by 10 2010 Bottom left SRU Front panel with 40 DTC links to the FEC cards Bottom right Back view of the SRU with 4 SFP ports that can be used either with 10 Gigabit Ethernet copper or fibre links Below top The SRU has a fast user defined pulse interface for 100 OHM LVDS or 50 OHM NIM signals These signals are connected to the FPGA and can therefore be injected to the DTC links for examples as readout trigger for the frontend chip Inversely triggers generated by the Frontend electronics can be sent out as common trigger Other applications are trigger BUSY generation or event counting 15 Augut 11 2010 KAPON RD51 SCALABLE READOUT SYSTEM LVDS NIM interface on SRU Ll TT va ka a Laa aa
9. a ja Sco t LE LE ee J cy EBV Dy cy gt anar III 16 Augut 11 2010 li RD51 SCALABLE READOUT SYSTEM Europa chassis SRS AandB cards 41 6 mm SCEM 06 61 62 143 0 C cards only 155 6 mm SCEM 06 61 62 143 1 17 Augut 11 2010 KPN RD51 SCALABLE READOUT SYSTEM Photo of a FEC and ADC adapter card in a Eurochassis slot 18 Augut 11 2010 kiipi RD51 SCALABLE READOUT SYSTEM Fig 8 Top Part references for the 6Ux220 Euro crate used for SRS electronics Bottom Left View of a SRS Euro chassis with 1 FEC card inserted from the back side and full sized C card adapter inserted from the front side Bottom Right View of an SRS Euro chassis with 1 FEC inserted from the front side and two half sized A and B cards inserted from the back side A support bar with card guides is mounted in order to give mechanical support to the A and B cards Crates with partial support bars can combine C card sections and A B side sections ATX 500 Watt supply 7 24 Pin ATX connector 2 5 mm2 Cables to SRS electronics m Special voltage via AC adapter Metallic Box 186x118x56 mm3 Fig 9 TOP ATX power adapter for SRS sytems The LV power for the SRS systems SRU FEC card are provided via standard ATX power units with additional options for user provided power The power adapter filter has a variety of user configurable options and is fused on each LV cable by Mini car fuses A
10. e industry HDMI cable and connector standard for the first hybrids The HDMI cables provide very high bandwidth over distance as well as power at low cost Further link standards may be implemented later if required Chip hybrid A readout chip resides on a micro etched hybrid PCB that gets plugged on the detector It connected via a chip link to an adapter of the FEC crate This feature allows changing or mixing the readout chip whilst maintaining the SRS backend different firmware may be required The first chip chosen is the 128 channel analogue APV25 chip The APV hybrid has a HDMI link interface and can be extended to a dual hybrid with 256 channels First SRS Data with APV hybrid on a GEM GEM APV25 Hybrid GEM APV25 Hybrid Cosmic Particles 2reliminaA Cosmic Particles reliminan LAST 96 astreusen L aaun 2010 kiipi RD51 SCALABLE READOUT SYSTEM Mai 2010 LEFT Offline reconstructed data from APV hybrid on a GEM with cosmics APV25 chip Hybrid gt HMDI link gt ADC12bit 40 MHz gt Virtex 5 FPGA gt Root analysis Semi gaussian pulse shape of cosmic particles and their cluster size RIGHT Amplitude distribution and cluster size distribution More details see Sorin Martoiu s talk Illustrated details and status of SRS this appendix will be regularly updated Frontend Hybrid APV25 EDMS Reference EDA 02075 V2 USALTTTTTTETTTTTTITITTATTTTT LLL LULL ELSE rrerrrrrrrrirrry ei oi 11 tte Mn a Mim 4 TT
11. ected to the online DATE computer The SFP standard for 10 Gigabit on the SRU allows using either cables up to 7 m or fibres length gt 100 m depends on fibre choice The SFP plug on the FEC cards can be equipped with cable adapter for using standard CAT6 network cables Scalable Readout Unit When the system is to be increased to more than a dozen FEC cards a Scalable Readout Unit SRU with 40 DTC link ports has to be added in order to allow connecting up 4 Augut 11 2010 kiipi RD51 SCALABLE READOUT SYSTEM to 40 FECsto the Gigabit output network between the SRU and the online computer The SRU has 4 SFP ports of 5 Gigabit s each A user defined clock and trigger interface based on NIM and LVDS signals allows to adapt SRS to different trigger concepts For LHC applications the SRU has a TTCrx chip to pick up the LHC clock and triggers LO L1 L2a L2r via the fibre network Larger systems or systems with aggregate readout bandwidth above the capability of a single SRU require several SRUs outputs to be interconnected via an optical switch in front of the Online PC farm Trigger architectures A priory SRS does not take assumptions about a specific trigger architecture The DTC links can carry trigger level signals up to a chip or to a FEC card orit can return trigger signals to the SRU where the desired trigger concept can be implemented in firmware The SRU apart from its TTC interface has both NIM and LVDS I O allowing to
12. gue chips with 40 MHz 12 bit ADC HDMI based chip link up to 20 m Hybrids with APV25 tested on a GEM for readout of Cosmics TL LL LLL Augut 11 2010 kiipi RD51 SCALABLE READOUT SYSTEM RD51 scalable architecture DATE Root based offline Analysis Testsystems LHC machine Clock 10 GEASE SR up 200m Mukim da Shar 10 Gee ee lt OBE coppe f Control TIC PC Trigger only for multi SRU ehee switch Single mode ber y GB ethemet MM fiber or copp Specific a DETECTOR Logical overview over the scalable SRS architecture Footnotes 1 It is understood that users who design their own SRS system or adapter logic make the production files and firmware available to RD51 2 Thank you for further help to commissioning the first small SRS systems on 2010 testbeams Augut 11 2010 kiipi RD51 SCALABLE READOUT SYSTEM physical overview SRS of RD51 T oh pe ec ptT iA afi Scalable system with DATE Hybrid adapter amp FEC portable DAQ Chip Hybrids user specific front end electronics with discharge protection Software packages The default Online system for SRS is DATE that needs to be installed on a Server PC running Scientific Linux 4 Users may also install the Root Data Analysis Framework and optionally Amore for data guality monitoring Data Formats The eguipment type used by DATE is UDP allowing to read CDH formatted even
13. interface with external trigger clock and busy logic DTC links A DTC link is a shielded CAT6 cable with an SRS specific protocol that carries Data Trigger and Control information between the SRU and the FEC cards FEC Crate Up to 14 FEC cards fit in one 6Ux220 Euro chassis each FEC card being interfaced to Adapter cards in the same slot The Chassis can be mechanically configured to carry a mix of A B or C type adapter cards Adapter cards Adapter cards can be built in 3 different form factors A B C and primarily interface the chip link to the FEC card The programmable interface is implemented via PCle connectors that provide very high speed links programmable I Os 12C controls and HV LV Power The chip link adapter analogue or digital versions connects N chips with the one FEC Extension adapters can de designed for detector specific purposes example HV bias control for APD s or SIPM s Depending on the real estate and I O requirement there are 3 different adapter sizes A B or C The first chip adapter designed by RD51 is for analogue chips i e APV25 Beetle etc Up to 8 HDMI chip links can connect up to 16 chips on one ADC adapter Digital chips like VFAT require a conceptually simpler chip adapter which can be designed on request from a team Further adapters are currently under design by the NEXT and ATLAS MAMMA collaborations Chip links Amongst the many possibilities for implementing a chip link SRS has chosen th
14. lable Jose F Toledo of by UPV Valencia PCleX1 PCle x8 PCIeX16 ALILLITIIIS LATTTTTTTT TA L h 1454 LV connector PHOENIX MSTB 8pin 5 08 RJ45 NIM 50 OHM HV connector differential LVDS NIM 50 OHM CTB9550 2 connector itrerential coax Fig 6 13 DTC links buses are obsolete parallel transfers higher bandwidth amp trigger rates point point controlledimpedance very high bitrate amp lowest error rate overcomes bus length limitto SRU single point failure not fatal less pin wear out more cost effective Naa RDS1 SCALABLE READOUT SYSTEM DTC link protocol proposed by Fan Zhang CCNU Wuhan Coded tigger and Control for dynamic actions CAT6 cable Example 80 MHz clock Input streamfrom 40FEC s 50 Mbit s 3 2 Gbit s to DATE ao SE buffered terminated LVDS 4x twisted pairs signalling over 15 25m CATS CATG shielded cable dy 5 Ghat rR rE to Online fidh of DTC_TRIG pulse number of Tae ax DTC DATA Serial data or ACK to SRU S 2 Level 0 trigger 4 Levwel 1 trigger amp Level 2 trigger Readout command 8 Control operation 289 Error w DTC_TRIG Serial data or ACK to FEE pr RETURN Coded status to SRU Ready or Error Note fordisabling a FEE card SRU stops CLK for its link 14 Augut 11 2010 Miil RD51 SCALABLE READOUT SYSTEM Scalable readout Unit SRU o c VA VV AVA
15. ll ATX power lines are passed though EMI filters and SRS Voltages 4 2 V and 1 8 Volt are provided For user defined power a AC adapter can be added to the ATX power box Note High voltage up to 500V get directly connected to the 2 pin connector on the FEC front panel BOTTOM The ATX power pack and the ATX Adapter filter can get attached to the crate in the way shown below 19 Augut 11 2010 ike RD51 SCALABLE READOUT SYSTEM DATE performance with Gigabit Ethernet UDP equipment LOG name hozi Number of equp menis n Mumberof ingge re 3238620 O00000 G Ume ni Tagger aie see4000 Average Tagger ale 4805074 Number of Sub we nie 3938690 TE SUD eveni mle et Sub eveniz ecomed 3238618 0 Sub eveni ecomed ale 5283 Byles infected 2694129629136 Bye injected aie 423 248 MBS i Byles ecomed 259413141736 Byde ecomed mie 423 194 MBs 20 Augut11 2010 kiipi RD51 SCALABLE READOUT SYSTEM 500 packet size 16000 bytes 400 300 S200 KA Q O throughput O 0 50000 100000 150000 22 by 250000 300000 350000 payload size bytes Fig 10 DATE scalability test status July 2010 Filippo Costa test with 1 UDP equipment the readout throughput of 3 6 Gbit s is reached above 80 kbyte event payload using 16 kbyte UDP packets For example Event size 80 kB Trigger rate 5 kHz and 420 Mbyte s throughput error free over 50 Million events 21
16. t data via one or more Gigabit Ethernet ports of a PC The sub event hardware format is derived from the RHIC data format and remains invisible to DATE Here we need a decoder software that allows like the RHIG decoder to identify the type of data and decode them according to this embedded information Controls For Front end Control and Configuration the FeC2 software of the DATE team can run on the same or another PC as DATE FeC2 was written originally for the ALICE DDL link to work over IP ports to allow for selective broadcasting Scripts can be used as self contained seguences for initialization of runs The addressing conventions between the user and the firmware are to be defined Run control The Data acquisition is deemed to be started ina READY state before the trigger is enabled and the trigger should be disabled before the run is closed This can be automatized like in ALICE via the ECS system however initial small systems are deemed to take are about the sequence by the shifter Readout Links SRS readout links are Ethernet cables or fibres which are directly plugged to the Gbit Ethernet port of the online computer On the electronics side these can either be connected directly to the SFP port of the FEC cards small system or large systems to one of the four 5 Gbit s SFP ports of the Scalable Readout Unit SRU In practise a small system does not need the SRU since the SFP ports of a few FEC s can be directly conn
17. ta Trigger and Control between the FECs and the SRU e Auser programmable trigger and clock interface based on LVDS and NIM logic on both SRU and FEC cards e TTC fibre interface on the SRU for trigger and timing distribution in LHC experiments B Shareable components for classes of chips analogue digital etc e SRS adapter cards to interface different types of chips analogue digital mixed to the FEC card e Chip readout links cables fibres for distances of tens of meters between the FEC crate and the hybrids on the detector C Detector specific components e Chip hybrids with standard connectors example HDMI on the detector and on the readout side allowing the user to choose the most suited readout chip e Application specific adapter cards in A B or C format for user defined purposes like diode bias control etc By today 14 month after approval in the RD51 CRETA meeting Working Group 5 succeeded with a strong support from users teams and the management to build integrate the following first basic SRS components for the prototype system implementation of first SRS systems DATE Online system ported to Gigabit Ethernet readout UDP equipment type Scalable Readout Unit built and revised with 40 DTC links TTC chip 10 GB Ethernet DTC links Data Trigger and Control tested on Alice Calorimeter FEC cards common for all SRS systems with direct Gigabit Ethernet link ADC Adapter card for up to 16 analo
Download Pdf Manuals
Related Search