ALMA-05.11.00.00-70.35.25.00-A-ICD
Contents
1.2. A05 ee Correct reference to RD 01 was RD 07 A06 Correct outdated figure A07 2013 8 9 Updated to current document number format A08 2013 8 19 Several To Be Done Updated CAN protocol related numbers for LTA and SCC protocols Updated protocol description of both SCC commands A09 2013 8 22 Fix typo on p 20 PSN is 8 Bytes not 4 Bytes DOWNLOAD_VDIF_STATUS to DOWNLOAD_VDIF_HEADER 2 places pf Fix typo in 3 2 2 2 2 Changed A10 2014 2 28 Clarify the start up procedure to show how the commands must be synchronized to external timing This is included in the SET_PIC_CTRL protocol description 3 Change the statistics readout from 2 bytes to 3 bytes per state Add a bit in status word 52 to indicate whether the ROACH is on or off ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 3 of 44 All 2014 3 19 Assign Struct ype to DOWNLOAD_VDIF_HEADER and SET_PIC_CONTROL Assign function codes to APPLY_VDIF_HEADER Add the time to completion of the DOWNLOAD_VDIF_HEADER and APPLY_VDIF_HEADER commands Changed title page prepared by add Alejandro Saez released by replace Mark McKinnon with TBD Specify zero as the structure type for DOWNLOAD_VDIF_HEADER Specify 20503 as the Message RCA for GET_DOWNLOAD_VDIF_HEADER_STATUS Specify the meaning
3. Data interface Four 1O0GigE connections from data source Maximum input data rate 16Gbps maximum 4Gbps per 10GigE port Data recording media 32 disks configured in four removable 8 disk modules 3 5 SATA M amp C interface Gigabit Ethernet M amp C protocol VSI S alternate XML interface planned Figure 3 9 Photo of one of four recorders required for the APP Data capacity When populated with 32 2TB standard SATA disks an aggregate 16Gbps input data stream can be recorded continuously for 8 hours before disk modules are filled 3 4 2 Interface to Computing Connection to Ethernet required Each recorder requires a 1GbE network connection and address and uses one server port for the application No CAN bus connection is required The recorder must be able to connect to a server running the NTP protocol Software requirements During development the recorders use Red Hat Enterprise Linux MIT has a license but as these get transferred to ALMA then ALMA shall ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 44 of 44 supply licenses from Red Hat The protocol details of the interface with Computing are fully described in AD 02 The control code is an APP deliverable and thus the ALMA ICT is not responsible for developing this interface
4. Bit 2 Bit 3 3 Bit 4 Bit 5 Bit 6 Data Frame within second bits 2 to 0 Bits 8 7 when 00 gt normal data from summer when 01 gt incrementing count from PIC counter when 10 gt pseudo random data from PIC pseudo random data generator when 11 gt always zero others Bits 31 24 FPGA personality version number Bits 27 to 0 GPS offset from PIC 1PPS Bits 27 to 0 Maser offset from PIC 1PPS Bits 27 to 0 TE offset from PIC 1PPS measured at seconds 0 6 12 4 Bits 31 to Ox3000 Bits 15 to 6 FPGA_temperature Temp oC FPGA_temperature 503 975 1024 273 15 Bits 5 to 0 spare In the VDIF frame these words have values of 0x0000_0007 Ox0000_0006 and 0x0000_0005 respectively NO OO 2 2 0 1 3 1 3 OO OO 4 y ON 0x7 to 0x5 7to5 Table 3 Definition of status words delivered in Word 5 of the VDIF frame as a function of Data Frame Number within second bits 2 to 0 In some cases fewer than 32 channels will need to be recorded In these cases the frame rate and or length are adjusted to meet the VDIF requirements Table 3 1 below summarizes the frame rate and length for all APP cases ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 28 of 44 peril Num _ Header Data
5. micro seconds BQ oe fa foose 12500000 m6 ff t000 16 62500 00 Bo 4 A 4 100064 15625 00 Qo a 625 ao 12500 00 a 625 160 6250 00 Table 3 4 Summary of the frame timing and contents as a function of the number of IF channels The C167 depends on the CCC to supply all bits of the header except e The Data Frame within second the FPGA can supply this because we always start at a l second boundary where this field must be zero e The least significant nibble of the Magic Alignment word see entry for Word 4 above e The Status Word As a result the CCC supplies the following lsb on the left PSN O psn_lower 32b PSN 1 psn_upper 32b words 0 sec_from_ref_epoch apply TE 30b 0 1b 0 invalidity 1b words 1 0 24b ref_epoch semester since 2000 6b 0 2b words 2 frame_bytes 1004 or 629 24b n_chan_log2 5b 0 3b words 3 station_id 16b thread_id 10b 1 1b 0 5b words 4 0OxA5EA50 upper magic word 24b 0x02 EDV 8b words 5 0 32b words 6 psn_lower 32b words 7 psn_upper 32b ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 29 of 44 J 2 2 2 2 GET _DOWNLOAD_VDIF_HEADER_STATUS RCA 0x20503 RCA 2 05 03 Description Status
6. Byte 0 is the LS byte corresponding to CAIs 7 down to 0 and so forth up through Byte 7 for CAIs 63 down to 56 A one in a bit position specifies that the CAI is included in the sum A zero specifies that it is not Bit 7 of byte 7 should always be zero since this antenna input is used by the phasing system to input the sum into the correlator matrix A 1 in this bit position will be flagged as a warning in the status message A zero in this position will always be downloaded to hardware and returned as status by the firmware in the LTA This command does not require an apply The mask is changed right after the command is received 3 2 2 1 2 SET_ANALOG SUM_MODE Function Code 11 Type Broadcast Data_msg_ID 8 Func code 11 Description Used to select between normal and test data Typical Interval Before the start of data taking Data in Data Messages 2 bytes This protocol directs the analog sum logic to provide either normal data or one of four possible test data streams ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 16 of 44 Fixed length message payloads 8 bytes Message Broadcast Setup Header Payload Message Message Bytes Transmit using Transmit using Message ID 5 Message ID 8 0 spare 0 spare 0 spare 0 spare 0 spare
7. Introduction The specification for the 64 Antenna Correlator RD 10 includes a requirement that the correlator provide hooks for VLBI To take advantage of these hooks various modifications to the correlator are required Hardware modifications to the correlator are documented separately in RD 07 This section details the new CAN protocols required to implement the APP Existing Correlator CAN protocols are documented in RD 04 3 2 2 Interface to Computing RD 04 includes a section section 16 in version D describing the interface between the Correlator and Computing Correlator System The introductory part of this section provides a general overview of the CAN interface between computing and correlator This is not repeated here Also certain existing CAN protocols which are indispensable to implementing the phasing interface are not repeated here The following sections describe only new capabilities required for implementing phasing Recall that a new type of card the PIC is added to the correlator Two such cards are required per quadrant The PIC card shall also respond to all Protocols That Apply to All types of Nodes section 16 3 in RD 11 3 2 2 1 Protocols Specific to LTA Nodes Since the actual summing of signals from antennas takes place in the correlator cards which are controlled by the LTAs some new protocols are required to control and monitor this functionality A short summary of the new protocols
8. PO and P1 as shown in Figure 3 3 below We expect that for our application all four sets will be identical The capability of making them different 1s provided ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 18 of 44 however since is it fairly simple to do so The protocol to deliver the 8 to 2 bit mapping information thus is applicable to one or more configurations as specified by a mask variable in the headers The interpretation of the mask bits is as shown in Figure 3 4 below NEW TO PINS POSUMA 1 0 PIC 2K Add 64 SCALING POSUMB 1 0 5 CT 4 LVDS a POSUMC 1 0 cavers POSUMD 1 0 P1SUMA 1 0 CO PIC 2K Add 64 SCALING P1SUMB 1 0 O CT 4 LVDS aa P1SUMC 1 0 eae P1SUMD 1 0 Figure 3 3 Block diagram showing the scaling of the sum and routing of 2 bit results to the PIC and CI Antennas Antennas Apply to SUMA Apply to SUMB Apply to SUMC Apply to SUMC Apply to Pol 0 Apply to Pol 1 Figure 3 4 Sketch showing the mapping of the mask bits NB There is no basis at present for thinking we need a separate map for the A and B channels i e the PIC should produce exactly what is fed back to the correlator or for a separate map for ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Docu
9. Second Photonic Reference Distribution Rack Photonic Reference Rack Radio Frequency Radio Frequency Interference Sub Array Switch Rack Timing Event a 48 msec timing tick which is the heartbeat of the ALMA control system Temperature Sensor Module ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 11 of 44 2 4 Definitions None so far 3 Interfaces to Various Subsystems This section includes the detailed interfaces to the various subsystems comprising the ALMA Phasing System A section is dedicated to each subsystem 3 1 Hydrogen Maser 3 1 1 Introduction VLBI observations require extremely good phase stability because the phase stability between geographically separated telescopes is required Allan deviation of 2 e 15 at 1000 seconds This is a contrast with connected interferometers like ALMA where frequency references for all antennas are derived from a single reference The frequency reference provided with the original ALMA array a rubidium standard while adequate for connected element interferometry is not adequate for VLBI at millimeter wavelengths Thus a hydrogen maser is provided as a deliverable of the Phasing Project In particular the hydrogen maser provided 1s Manufacturer T4 Science SA Model Number iMaser 3000 The
10. The following table identifies the input source selection options Data 1 _ Sum Output Type Antenna Sum normal data Static logic level 0 Pseudo random data not sync d to tic 2 Pseudo random data sync d to 16 msec tic This type of flexibility on output data is similar to that provided in other parts of the correlator It has proven useful in measuring the reliability of the links between subsystems as well as aiding with troubleshooting In this instance it will make it possible to evaluate the reliability of the link to the PIC cards and downstream link This command does not require an apply The switch is changed right after the command is received 3 2 2 1 3 SET_CIC_SUM_INPUT_SWITCH Type Broadcast Data_msg_ID 8 Func code 12 Description Provides a bit to set the state of the CIC input switch Typical Interval Before the start of data taking Data in Data Messages 2 bytes ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 17 of 44 There is an electronic switch in the Correlator Interface Cards which allows for selection of two possible inputs for the 64 antenna CAI 63 The first is the normal data from CAI 63 The second is the phased sum data from the correlator card The hardware default after a system reset 1s normal
11. interfaces of this subsystem to computing are detailed in this section 3 1 2 Background The ALMA Central Local Oscillator was installed in 2009 and expanded in 2011 It has been in use since that time for ALMA Early Science The ALMA Phasing Project APP is an external ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft a Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 12 of 44 international development effort that will enable ALMA to participate in ultra high resolution VLBI and high frequency phased array science AD 01 The following block diagram illustrates the interface that is the subject of this document This diagram is consistent with Figure 2 1 of AD 01 but adds detail to the CLOA elements H Maser GPS CRD CRG CVR etc In the first figure the ALMA CLOA is shown before the addition of elements required for the ALMA Phasing Project 1 PPS 5MHz l m Maser 10 MHz 1 PF 1 PPS P 48 msec i 10 MHz SAS LLC Local Oscillator Subarray mapped to Antennas Photonic Distribution E 1 ed Antenna 1 badd Antenna 3 CLOA a Antenna 66 Figure 3 1 ALMA CLOA shaded red box with interface to Correlator and Antennas indicated Before ALMA Phasing Project elements are added The changes necessary to accommodate
12. ole a RunTG intemal FPGA signal O In response te the APPLY command the c16 uP brings RunTG low and then high Fi jwhether or not it was low or high before the APPLY This causes the FPGA ta 125 MHz clk 4 TE QCC synchronize its internal 1PPS to the next ans asing edge of the TE from the GCC i TE PIC TE_PIC goes high tor 1 clock E ns after rising edge of TE QC is detected and every 48 Piles ns thereafter PPS PIC PPS PIC goes high for 1 clack A L 000000008 ns after rising ed of TE OCC is detected and every 1 000000000 ns thereafter Figure 3 7 Timing diagram which provides details in synchronizing the internal 1PPS to the external timing reference ATACAMA LARGE MILLIMETER ARRA TE_PIC PPS PIC APPLy_VDIF_HEADER From OCC RunFm internal FPGA signal ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 36 of 44 Synchronizing to the seconds from reference epoch and packet serial number 48 ms OO Bn n a g ons This TE Is aligned with the PIC 1 PPS Big 4 keiss aa Zoom in on this signal edge in TE cycle This causes the data_formatter to wait for the next PPS_PIC_A
13. required includes e Provide a mask to specify which antennas are summed e Provide the capability to substitute test data for normal data in the sum e Provide the capability to control a switch which in turns provides the capability to inject the sum into the correlator matrix in place of the 64 antenna e Provide data for mapping the sum from 8 bits to 2 bits e Provide a means of monitoring the status of the above commands 3 2 2 1 1 DOWNLOAD_ANTENNA_SUM_MASK Function Code 2 Type 11 Type Broadcast Data_msg_ID 8 Func code 2 Structure type 11 Description Provides a mask to select which antennas to sum Typical Interval Before the start of data taking ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 15 of 44 Data in Data Messages 2 bytes Fixed length message payloads 8 bytes Message Broadcast Setup Header First amp Only Data Payload Message Message Message Bytes Transmit using Transmit using Transmit using Message ID 5 Message ID 8 Message ID 9 Data 0 Data_msg_ID 8 Mask Byte 0 Data 2 Targetmask 7 0 LS block size bytes 8 l Data 3 Targetmask 15 8 MS block size bytes O l Datal4 Targetmask 23 16 O sparey d Data 5 Targetmask 31 24 O sparey d Dataf6 Targetmask 39 32 O sparey o o lo
14. resulting from DOWNLOAD_VDIF_HEADER commands Typical Interval Rare Data 2 bytes Reply Message Contents Message RCA 0x20503 A new DOWNLOAD_VDIF_HEADER command should not be sent until the previous one has been processed so CCC should check this status prior to issuing anew DOWNLOAD_VDIF_HEADER command Possible VDIF status values are Bit 0 not done 0 done Other bits Not used ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 30 of 44 42 225 APPLY VDIF HEADER command Function Code 3 Type Broadcast Data_msg_ID 8 Func code 3 Description Used to apply the frame information Typical Interval Before the start of data taking Data in Data Messages 2 bytes Message Broadcast Setup Header Payload Message Message Bytes Transmit using Transmit using Message ID 5 Message ID 8 Data 0 Data_msg_ID 8 Data 1 0 spare LS byte of events 0 spare This command causes the transmitted VDIF header to be applied at the rising edge of the next PIC 1PPS Since the data is already downloaded in hardware the apply is instantaneous Assuming that the CCC transmits the data right after the TE that immediately precedes the PIC IPPS the apply delay is 48 msec The first two data bytes reserve storage for a TE count which will NOT be imp
15. the ALMA Phasing Project are shown next ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft b w Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 13 of 44 1 PPS 48 msec 125 MHz SAS LLC Local Oscillator Subarray mapped to Antennas Photonic Distribution Antenna 1 Antenna 2 Antenna 3 CLOA Antenna 66 Figure 3 2 ALMA CLOA shaded red box with interface to Correlator and Antennas indicated with ALMA Phasing Project elements added The added elements are the following Hydrogen Maser H Maser H Maser Rack HMR Change of 1 PPS from MFS to H Maser Change of 5 MHz now coming from the H Maser to the MFS Addition of 1 PPS from H Maser to Correlator CORR 3 1 1 H Maser Interface to Computing The maser includes a software interface which is documented in RD 01 To monitor the health of the maser the ALMA Monitor and Control System shall use this interface to monitor and log the parameters listed in Table 2 of this manual This interface is fully described in AD 02 3 2 64 Antenna Correlator Upgrades ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 14 of 44 3 2 1
16. 1 APPLY_TFB_PHASES Type Broadcast Data_msg_ID 8 Func code 12 0xC Description Used to apply the frame information Typical Interval Before the start of data taking Data in Data Messages 2 bytes Message Broadcast Setup Header Payload Message Message Bytes Transmit using Transmit using Message ID 5 Message ID 8 Data 0 Data_msg_ID 8 Func_code 12 OxC ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 40 of 44 Data 7 Target mask 47 40 0 spare This command causes the transmitted TFB phases to be applied at the rising edge of the next TE The first two data bytes reserve storage for a TE count which will NOT be implemented initially The TE count would allow the phases to be applied after some number of TEs This type of countdown is implemented in a few other Correlator CAN commands but 1n practice has never been used The next byte configNum specifies the one applicable configuration number 0 to 15 A configuration mask is not supplied since the CCC will always deal with updating phases one configuration at atime This also simplifies the software The command should be transmitted by the early in the TE cycle before the TE at which the apply is to occur be GET _APPLY_TFB_PHASES_ STATUS RCA 0x2050A RCA 2 050A Desc
17. 6 10 1 e Provide time information by executing the following two CAN commands i DOWNLOAD_VDIF_HEADER ii GET_DOWNLOAD_VDIF_HEADER_STATUS f Apply the time information seconds from reference epoch and packet serial number during the TE that immediately precedes the TE when data taking is to start Data taking must start at second 0 or 6 or 12 See the timing diagram Figure 3 8 for additional details This is done with the following two CAN commands i APPLY_VDIF_HEADER ii GET_APPLY_VDIF_HEADER_STATUS g Statistics should be measured at appropriate times as dictated by the astronomical observation and operational concerns keeping in mind the wise use of the CAN bus bandwidth Statistics measurement will begin when a control command with valid entries is received ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 35 of 44 ATACAMA LARGE MILLIMETER ARRA Synchronizing to Site 1PP5 AB rms oo __ TE from QCC om y d y q hms This TE ee Is aligned with the site 1 PP5 __ f j l PRS ALMA site _ q ims Note TE is exactly aligned with Command _ __ gt 200us site 1 PP4 at seconds 6 12 Zoom in on this signal edge APPLY_TIME_DATE a TEeyde gt F CAN Command
18. Atacama Large Millimeter Array Interface Control Document Between ALMA Phasing Project and ALMA Computing ALMA 95 11 00 00 70 35 25 00 A ICD 2014 10 07 Prepared By Organization Role Date and Signature R Lacasse NRAO Geoff Crew Haystack Observatory M Honma NAOJ Approved By Organization Role Date and Signature Richard Lacasse APP SE Lead Engineer Nick Whyborn ALMA Array Lead Engineer Jorge Ibsen ICT Manager Released By Organization Role Date and Signature Michael Hecht APP Project Manager ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 2 of 44 Change Record Version Date Affected Change Reason Initiation Remarks Request o 2013 3 18 Incorporate comments by Geoff Crew and Alejandro Saez 2013 4 1 Several None Change ANALOG_SUM_MODE protocol to use a 16 msec sync instead of a 1 msec sync for PRN generator Add a requirement to the recorder s interface with computing that it be able to connect to a server running the NTP protocol Delete GET_VDIF_HEADER protocol Update GET_PIC_STATUS protocol Update per comments from Geoff Crew per comments Update per comments from Geoff Crew Geoff Crew ny 2013 4 22 File name None Correct doc number in file name Update VDIF frame illustration Fig 3 6
19. In the GET_PIC_STATUS protocol add bit 1 to the ENV6O status byte Update the DOWNLOAD_VDIF_HEADER protocol to show what bits are sent by the CCC and the requirements for the least significant nibble of the Magic Word Change the meaning of the bits in Word 5 to reflect current status and future goals Add a status bit to indicate FPGA write error to the protocol GET_APPLY_VDIF_HEADER_STATUS Update Figure 3 6 Change the suggested startup procedure for the PIC Clarify a typo concerning ROACH_write_error in GET_APPLY_VDIF_STATUS Add GrsFm to Control Byte 1 Update GET_PIC_STATUS protocol to show the availability of the sampled 1 PPS counter Modify figure 3 5 to show different binary codes for data to Station Interface Card and Phasing Interface Card Update DOWNLOAD_VDIF_HEADER protocol to show details of the status words provided by the PIC in the VDIF data frames Incorporate Geoff s comments to the draft of the above Change the recommended PIC startup procedure step 4d In the CAN protocol GET_PIC_Status added bits 2 3 and 4 to PIC status byte 53 In SET_PIC_CONTROL add bit 2 in control word 1 to enable re seeding of the PRN test data generator A21 2014 6 12 A22 2014 7 17 A23 2014 8 1 A24 2014 9 10 A25 2014 09 10 Alejandro Regular ICD number assigned Caceres ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasin
20. _APPLY_TFB_PHASES_ STATUS get waiting for a response GET_APPLY_TFB_PHASES_ STATUS get waiting for a response GET_APPLY_TFB_PHASES_STATUS get success for a response GET_APPLY_TFB_PHASES STATUS get failure for a response 3 2 3 Other Interfaces to PIC ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 42 of 44 In addition to the CAN interface the PIC cards have several other interfaces available These may be useful for engineering purposes None of them require software from the Computing IPT However they are Ethernet network interfaces so ALMA Computing needs to be aware that there may be a need to place them on the ALMA network or a private network connected to the Engineering Port for use by engineers for troubleshooting and maintenance 3 3 Optical Fiber Link System Interface The purpose of this sub system 1s to transmit the antenna sum data from the AOS to the OSF while using minimal fiber resources The eight 10 GbE data streams are wavelength division multiplexed onto one fiber at the AOS This data is transmitted to the OSF where it is de multiplexed and routed to the appropriate recorder sub system The optical fiber link system a pair of a transmitter and a receiver is fully symmetric and the two devices are inter cha
21. _Data 0 Func_code 3 STATIO STATIN S _Data 1 O sparey STATH STATIN _Data 2 LS block size bytes STAT 2 STATIN O _Data 3 MS block size bytes STAT 3 _ STATIN S Data 4 LS block offset bytes STATIA STATIN A _Data 5 MS block offset bytes STATIS STATIN 3 _Data 6 O spare STATI6 STATIN 2 Datal7 O sparey STATI STATIN This protocol returns PIC status bits The meaning of the status bits is detailed below The Block Size and Block Offset in the control message are provided by the CCC to indicate which part of the table it wants in the reply For example block size of 8 and offset of zero returns only the PSN Block size of 8 and offset of 52 returns the PRN error count and environmental monitors Thus for testing debugging access to any part of the table is possible In operations the VOM will normally request the beginning of the table to verify timing probably 56 bytes during setup it will probably request state statistics once and retrieve the full table sometime later The table entries are aligned on 4 byte boundaries so that parsing of this information is easy for the likely use cases Status Bits Name Function Note Byte Num 0 3 All PSN_ PSN LSW captured at last TE o Z oOo o Oo o o ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Projec
22. a data recording system This equipment is more fully described elsewhere AD 01 Some of these subsystems may have several interfaces to the ALMA Observatory e g hardware and software This document covers only the interfaces between new APP equipment and Computing Software to software interfaces are covered in design documents AD 02 RD 06 Access to the new hardware in normal operations is made via the VLBI Observing Mode VOM and is documented in its design document AD 02 The ALMA Phasing Project is identified in the following document AD 01 2 Applicable Document Reference Documents Acronyms and Definitions Applicable documents are necessary for the understanding of this document In some cases they provide additional requirements which are to be incorporated into the ICD Reference documents are supplemental and simply provide further reference for various topics In most cases the acronyms used in this document are consistent with ALMA defined acronyms however additional acronyms have also been listed which are outside the scope of ALMA definitions No distinction is made between these two uses 2 1 Applicable Documents The following documents of the exact issue shown form part of this document In the event of conflict between the documents listed here and this document this document shall take precedence ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Dr
23. aft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 9 of 44 AD 01 APP Project Plan Release 1 3 ALMA Phasing Project Project Plan 1 3 11 Oct 12 AD 02 ALMA Phasing Project Update to Corr Control Design AE MAA O OEE AD 03 ALMA Environmental Specification ALMA 80 05 02 00 001 B SPE AD 04 Seismic Design Specifications for ALMA AOS and SYSE 80 10 00 00 002 B REP ALMA OSF Latest Prod Assurance Requirements APP Update to Corr Control Design APP Update to TelCal Design AD 08 XW 100_protocolrev2 o S o Mark6_command_set Release 1 1 a Table 2 1 Applicable Documents for this ICD 2 2 Reference Documents iMaser 3000 Installation Operation amp RD 01 Maintenance User Manual Issue 1 7 28 T4S MAN 0012 available from www T4Science com May 2010 iMaser 3000 Specifications http www t4science com documents iMaser_Clock_ ee Spec pdf RD 03 64 Antenna Correlator ALMA 60 00 00 00 001 B SPE Specifications and Requirements RD 04 Interface Control Document Between 64 ALMA 60 00 00 00 70 40 00 00 D ICD Antenna Correlator And Correlator Computing System RD 05 VLBI Data Interchange Format VDIP http www vlbi org vdit docs V DIF 20specification Specification 20Release 201 0 20ratified pdf RD 06 APP Update to TelCal Design ALMA 05 11 62 01 001 A DSN RD 07 nterface Control Document between ALMA ALMA 05 11 10 49 60 00 00 00 A ICD P
24. alue that the CCC downloads for this bit is logically ORed by the microprocessor with some MS M SD CH STL SCH locally measured status bits The following conditions can cause NVLD 1 in the transmitted data frame 1 NVLD 1 from the CCC 2 The frame clock on the FPGA 1s out of lock ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete Page 34 of 44 And ALMA Computing with the Maser 125 MHz The FPGA reports an over temperature error this will eventually result in the ROACH being powered off 1 This will take approximately 30 seconds 2 Statistics measurement will begin when a control command with valid entries in both the STL and SCH fields is received even if an acquisition is already in progress 3 The suggested start up procedure for the PIC is a Power on the ROACH 0x01 to control byte 0 and wait 40 seconds to complete b Align the FPGA s internal IPPS signal with the external world by sending an APPLY_TIME_ DATE CAN command in the TE immediately before second 0 or second 6 or second 12 See the timing diagram Figure 3 7 for additional details c DoaGET_TIME_ DATE command to check that the time was properly set d Wait afew TEs and then do a GET_PIC_STATUS command Check the sampled 1PPS counter Bytes 56 to 59 The value should be TEs
25. ator Upgrades eessesssseoeeesssssssecceesssssssseseeessssssssceressssssseeeeeos 13 aa MT A erer EEEE NEIE EE AEE ESEN 14 Lea merice o C C onno eira 14 ie Ober meraces tO PI serrr ieisnr iran n Ei E Kai 4 3 picai Fiber Link System Mern CE eee Ea 42 Lal abric toC Comp E sn EE RNE 42 3 4 VLBI Recorder Interface noonnsesesssooeeressssssseerresssssssectresssssssecereressssseserersssssseees 42 Jad pupyte Denci PO enep nne E E EE rS 42 JA merae OL ompa sicer E Sie 43 ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 7 of 44 List of Figures Figure 3 1 ALMA CLOA shaded red box with interface to Correlator and Antennas indicated Before ALMA Phasing Project elements are added ccc0008 12 Figure 3 2 ALMA CLOA shaded red box with interface to Correlator and Antennas indicated with ALMA Phasing Project elements added ccccccsssseeeeeeeeeeeeees 13 Figure 3 3 Block diagram showing the scaling of the sum and routing of 2 bit results to the PIC EC Eo a A A E saaeeaue heer yneasn E A 18 Figure 3 4 Sketch showing the mapping of the mask bits 0ssssesesseeeeeesessssssssssssssssseeerreeeeees 18 Figure 3 5 Illustration of the effect of the threshold to the mapping of memory addresses corres
26. cessor memory Spare Statistics 32 channels x 4 states channel x 3bytes state ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 39 of 44 1 The time to acquire these will vary with the statistics integration time STL and with the number of channels requested 1 2 4 8 16 or 32 Sixteen bits for statistics should be sufficient since even with a l msec acquisition window counts occupy about 16 bits with about 8 of those being noise 2 To avoid ambiguities in the header timing the PIC status request should be sent shortly after a TE so that the result is received before the next TE According to Table 3 1 the packet duration is ALWAYS an integral number of TEs 300 to 6000 and this header refers to the first one of the group le the first datum is at the time indicated in the header 3 2 2 3 Protocols Specific to SCC Nodes The phase of the digital local oscillators DLOs in the TFBs is key to phasing ALMA Phase updates to these DLOs need to be updated periodically based on a fairly complex measurement process A command already exists to download phase commands to the DLOs However the current command to apply the phases has undesirable side effects for ALMA Phasing A new command without these side effects is described below 3 2 2 3
27. data so that this protocol needs to be used only in phasing mode Fixed length message payloads 8 bytes Message Broadcast Setup Header Payload Message Message Bytes Transmit using Transmit using Message ID 5 Message ID 8 Data 0 Data_msg_ ID 8 Func_code 12 Switch Position Bit 0 normal data 1 phased sum data 0 spare This command does not require an apply The switch is changed right after the command is received 3 2 2 1 4 DOWNLOAD_SUM_SCALING DATA Summing the 2 bit data from the ALMA antennas in the observing Array will result in a sum which has at most 256 possible values To be fed back to the correlator or delivered to the PIC this sum must be again reduced to a 2 bit quantity with proper statistics 1 e one of 4 states representing 3 1 1 or 3 with Gaussian distribution The mapping of 256 to 4 values is a function of the number of antennas N and can be provided to the LTA logic equivalently as a lookup table or as the 3 thresholds dividing the 4 output states and the limiting valid values Even more simply one can assume zero offset and provide a single number the positive threshold and assume the negative threshold is symmetrical Initially we plan to implement the simplest choice Details for this choice are presented in Section 3 2 2 1 4 2 3 2 2 1 4 1 Background information The summing hardware is capable of providing 4 sets A through D of two bit numbers for each of two polarizations
28. dv to load new In response to the APPLY command the values for SFRE and PSF at the next a t167 uP brings RunFM low and then high PPS PIC Adv p whether or not it was low or high before sy the APPLY SS SS SS SS SS OE ae 1235 MHz clk ET Ly Lf f ns q PPS PIC Adv PRS PIC_Ady goes high ae 1 clock before PPS PIC Bins _ _ PPS_PIC_Ady SFRE PSN SFRE and PSN are updated using data provided by the DOWNLOAD VOIF_HEADER CAN a command and triggered by the APPLY VOIF HEADER CAN command coincident Pa with the rising edge of PIC_PPS a a Figure 3 8 Timing diagram which provides details in synchronizing seconds from reference epoch and packet serial number to the internal timing references ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 37 of 44 3 2 2 2 6 GET PIC Status RCA 0x1000 FC 3 Description Returns the status from the addressed PIC Typical Interval Provided for monitoring and troubleshooting Data variable number of bytes GET the PIC returns the status data Message Control Message Data Last Data Message Payload identical for both the Messages Bytes request and response All but last Message RCA Message RCA Message RCA 0x1001 0x 1000 Ox 1000
29. g Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 5 of 44 F Signature matrix modified according to N Whyborn s Sepulveda comments A26 2014 09 29 A27 2014 10 07 R Lacasse Respond to Nick Whyborn s comments e TBDs removed e Added applicable docs to provide req info e Clarified responsibility for recorder interface Fix a typo in Table 3 ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 6 of 44 Table of Contents A SCE EEEO noes 8 RER no ene eee re EEE Mer ee teen ero ne Sree Me een eC pee E mewn Ronee aren Tw ren tree 8 Lo accra csc E AE A AE 8 2 Applicable Document Reference Documents Acronyms and Definitions 8 2L A i Aa ager nes napa EN E 8 a I arse taht eE N A 9 2 3 Abbreviations and Acronyms ssseeeeessssssseccressssssseccreesssssssecteesssssseccttesssssseeceeessssss 9 MN ce ciprecich sistent econ bts E E EEN 11 3 Interfaces to Various SubsystemS ssssecccccssccecccssecceccsseccccccssececcccsecececcsecccccssseccecssseee 11 IA Hyo ner ASE eE E EE E EE 11 dda T A E EEE E EA E EE EE ENES 1 et TN Ios stun ges ado scape E EE E E 11 3 1 1 H Maser Interface to Computing esssessseoeeessssssssecerrssssssssecreessssssesererssssss 13 3 2 64 Antenna Correl
30. h ____ Pos high th 1j _ 1000 000 ___ Memory contents Memory contents for data to 5 for data to PIC Weight ol 11 3 be g it F f 11 O1 E r dlp o0 3 Figure 3 5 Illustration of the effect of the threshold to the mapping of memory addresses corresponds to sums to memory outputs corresponds to scaled sum This command does not require an apply The switch is changed right after the command is Returns status information from all of the above commands received 3 2 2 1 1 GET_SUM_STATUS RCA 0x1000 FC 14 Description Typical Interval After executing above commands Data 8 bytes GET returns status info ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 21 of 44 Message Control Message Data Last Data Message Payload identical for both the Messages Bytes request and response All but last Message RCA Message RCA Message RCA Ox 1001 Ox 1000 Ox 1000 MODE 0 4 Data 2 LS struct size bytes l Mapping Mask Data 3 MS struct size bytes pos2high_th Data 4 O spar _ pos2high_th Data 5 O sparey Spare Datal6 O sparey pare spare Description of data in the message Mask Byte 0 to 7 The mask transmitted by the previous DOWNLOAD ANTENNA SUM MASK command Sw
31. hasing Project and ALMA Correlator Table 2 2 Reference Documents for this ICD 2 3 Abbreviations and Acronyms AC A D Iternating Current Applicable Document ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 10 of 44 ALMA AOS APP AT E CLOA CAI CIC G CVR CVRR DLO PS HMR ICD PT R z p O pd U LVDS M amp C mie a S ade NRAO Al PAS DU PIC PLOTS PPS PRDR RF RFI SASR E Z lt Atacama Large Millimeter Array radio telescope Array Operations Site ALMA Phasing Project Advanced Technology eXtended standard for Personal Computers Back End Central LO Article Correlator Antenna Input Correlator Interface Card Central Reference Generator LRU Central Variable Reference LRU Central Variable Reference Rack Digital Local Oscillator Global Positioning Service Hydrogen Maser Rack Interface Control Document Integrated Product Team Line Length Corrector Rack Local Oscillator Line Replaceable Unit Low Voltage Differential Signal Monitor and Control Master Frequency Standard LRU Master Laser LRU National Radio Astronomy Observatory Preliminary Acceptance In House Provisional Acceptance On Site Power Distribution Unit Phasing Interface Card Photonic LO Test Stand Pulse Per
32. it is changed to 5 Bit 7 ROACH _write_error indicates an error writing to the FPGA either during the APPLY_TIME DATE or APPLY_VDIF_HEADER commands 0 no write errors This bit is set to zero after transmission to the CCC Other bits Not used ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 32 of 44 3 2 2 2 5 SET_PIC_CONTROL Type Broadcast Data_msg_ID 8 Func code 2 type 1 Description Used to download most control bits required by the PIC Typical Interval Before the start of data taking Data in Data Messages 16 bytes Message Broadcast Setup Header Message on Payload Message on Control Data Channel Bytes Channel Transmit using Transmit using Message ID 5 Message ID 8 Various control bits are required to set the operating mode of the PIC These are provided in the CTRL bytes above The meaning of the bits in the CTRL bytes is detailed in the table below 1 0 em or 11 NOP bit 1 is an enable OO power off ROACH 01 power on ROACH 3 GRS 0 no effect 1 reset ROACH registers Note Low level function used by the microprocessor Not useful from the CCC PA TGS Rising edge start the timing generator clocks at the next Rising edge reload ROACH personality Note this feature is not functional as a 2014 4 21 P
33. itch Position The switch position transmitted by the previous SET_CIC_ SUM_INPUT SWITCH command MappingMask The mask transmitted by the previous DOWNLOAD _ SUM SCALING DATA command pos2high_th The lowest numbered address having a memory contents of 01 Reference Figure 3 5 This verifies that the LTA properly interpreted the DOWNLOAD SUM SCALING DATA command from the CCC pos2high_th The lowest numbered address having a memory contents of 11 Reference Figure 3 5 This verifies that the LTA properly interpreted the DOWNLOAD SUM_ SCALING DATA command from the CCC Scale_err This byte has value 0x01 if there are transitions in the sum scaling table except at pos2high_th pos2high_th and between 0x7F and 00 Reference Figure 3 5 This verifies that the LTA properly interpreted the DOWNLOAD SUM SCALING DATA command from the CCC After this command is executed all status words are set to zero This allows the CCC to tell whether commands are being sent as expected ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 22 of 44 3 2 2 2 Protocols Specific to PIC Nodes Eight copies of a new card the PIC must be added to the correlator to provide a phasing interface Two cards are added to each quadrant one for each polarization The primary function
34. lease use the power bits above to reload the personality ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 33 of 44 1 PPS rising edge Note Low level function used by the microprocessor The CCC starts the timing generator with the APPLY TIME DATE command J F Rising edge start transmitting packets at next 1 PPS Falling edge stop transmitting packets at end of current packet Note Low level function used by the microprocessor The CCC normally starts the formatter with the APPLY_VDIF_HEADER command T R G I 7 6 DOUT 00 packets contain normal data 01 packets contain incrementing word count 10 packets contain 64 bit pseudo random data 00 no input data tests 01 test against pseudo random data pattern 10 measure input statistics ee Re seed option for PRN test data generator 0 PRN test generator free runs 1 PRN test generator re seeds every TE 0 no reset ICH Whi ST gy mean sais Vetoes te 27 4 Number of input channels for which to measure statistics given as log2_nchan Consistent with correlator mode definitions we always start with channel 0 So a 5 means 32 channels a 4 means 16 channels etc NVLD Value to insert into Word 0 bit 31 of the data frame indicating data valid or not Note The v
35. lemented initially The TE count would allow the header to be applied after some number of TEs This type of countdown is implemented in a few other Correlator CAN commands but in practice has never been used The command should be transmitted by the early in the TE cycle before the TE at which the apply is to occur In addition the TE at which the apply is to occur should occur on an even second TE is coincident with 1 PPS at seconds 0 6 12 54 ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 31 of 44 3 2 2 2 4 GET_APPLY_VDIF_HEADER_STATUS RCA 0x20504 RCA 2 05 04 Description Status resulting from APPLY _VDIF_HEADER commands Typical Interval Rare Data 2 bytes Reply Message Contents Message RCA 0x20504 Data 0 oe VDIF status Provides the status for the APPLY _VDIF COMMAND The status word can take on the following values Bits 2 to 0 PICApplyState 6 error FPGA write error PICApplyState 5 idle no download or apply received PICApplyState 4 received a download PICApplyState 3 set state to waiting on 1 msec loop PICApplyState 2 waiting for TE PICApplyState 1 error no download prior to apply PICApplyState 0 apply is complete If PICApplyState 6 then after transmission to the CCC
36. ment Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 19 of 44 PO and P1 the polarization sums should be handled identically Since C and D are not connected the mask would then always be OxC3 3 2 2 1 4 2 Message with one threshold Type Broadcast Data_msg_ID 8 Func code 13 Description Provides threshold and mask for mapping 8 bit data to 2 bits Typical Interval Before the start of data taking Data in Data Messages 3 bytes The protocol we have selected to implement uses one threshold It is described in this section Message Broadcast Setup Header Payload Message Message Bytes Transmit using Transmit using Message ID 5 Message ID 8 Data 0 Data_msg_ID 8 Func_code 13 Data 1 0 spare mask controls applicability as defined above Figure 3 4 pos2high_th the positive threshold See figure below for details At and above this threshold the memory output shall be O1 for data to the Station Interface Card SI and 11 for data to the PIC ALMA Project Interface Control Document Between ALMA Phasing Project And ALMA Computing Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Status Draft Draft Pending Approved Released Superseded Obsolete Page 20 of 44 Memory Address 0111 1111 _ _ Pas2high th __ Posthigh_th 1 0000 0000 _ fh 1117 1111 a Pos high_t
37. ngeable as they are totally in the same design The device has no packet monitoring capability so itis a passive participant in the VLBI phasing system Manufacturer Elex Engineering Model Number XW 100 both the transmitter and the receiver 3 3 1 Interface to Computing The details of this interface are fully described in AD 02 3 4 VLBI Recorder Interface The function of the recorder subsystem is to record on computer disks the antenna sum data that is computed by the correlator and transmitted via the optical fiber link system The disks are then shipped to a correlator facility such as Haystack Observatory to be correlated with data from other VLBI sites 3 4 1 Subsystem Description The recorder selected for the APP is the Mark 6 VLBI Data System This system is a disk based recording system to support capturing digital data from VLBI observations up to 16Gbps sustained data rate to an array of 32 disks Each of the four recorders that will comprise the APP record system has the following properties ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Pd Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 43 of 44 Purpose The Mark 6 VLBI data system is a disk based recording system to support capturing digital data from VLBI observations up to 16Gbps sustained data rate to an array of 32 disks
38. of the bits in VDIFstatus for protocol GET_DOWNLOAD_VDIF_HEADER_STATUS A12 2014 4 2 A13 Specify three as the function code for the APPLY_VDIF_HEADER command Al4 2014 4 21 GET_DOWNLOAD_VDIF_HEADER_STATUS Change RCA from 205xx to 20503 Specify values for the return value APPLY_VDIF_HEADER change function code from x to 3 GET_APPLY_VDIF_HEADER_STATUS Change RCA from 205xx and 20503 to 20504 Add several possible states to the status SET_PIC_CTRL Change Type from X to 1 Clarify the function of LDR GRS TGSD FMS and NVLD control bits GET_PIC_STATUS Change function code from X to 3 and type from Y to 0 E 2014 4 22 P Get_PIC_Status Delete the Type parameter 2014 5 4 DOWNLOAD_VDIF_HEADER Change the number of bytes in data messages from 36 to 40 GET_DOWNLOAD_VDIF_HEADER_STATUS clarify the possible values of the return value Add applicable documents AD 06 and AD 07 ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 4 of 44 A17 2014 5 4 A18 2014 5 6 A19 2014 5 15 A20 2014 6 11 For the DOWNLOAD_VDIF_HEADER protocol change the Data in Data Messages from 36 to 40 Clarify the startup procedure in SET_PIC_CTRL In the SET_PIC_CONTROL protocol e Change the definition of ICH e Change the definition of SCH
39. of the card is to format the sum data received from the correlator cards into a VLBI standard VDIF format A short summary of the new protocols required includes e lt A protocol to download required header information for the VDIF data frames A protocol to apply the above header information A protocol to monitor the time and status in the PIC card A pair of protocols to initiate and monitor a test for data quality into the PIC A protocol to monitor the sum data statistics A protocol to control the PIC card power on off etc ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 23 of 44 3 2 2 2 1 DOWNLOAD_VDIF_HEADER Type Broadcast Data_msg_ID 8 Func code 2 type 0 Description Used to download header for VDIF frame Typical Interval Before the start of data taking Data in Data Messages 40 bytes Message Broadcast Setup Header Message on Payload Message on Control Data Channel Bytes Channel Transmit using Transmit using Message ID 5 Message ID 8 Data 0 Data_msg ID 8 Func_code 2 Word 6 LSB LSB LSB Data 1 _ 0 spare Struct_type 0 Word 6 Data 2 Target mask 7 0 LS struct size bytes Word 6 Data 3 Target mask 15 8 MS struct size bytes Word 6 Data 4 Target mask 23 16 0 block nr 0 PSN Word 7 LSB Word 7 W
40. or all frames These are the same for all observations All pink bits All bits shaded with the color pink in Figure 3 6 are identical to the corresponding bits in the CAN message for all frames These may be set by the observer as desired Word 4 bits0 to 23 A magic sync identifier that in effect identifies 2 subversions of which we use 16 at ALMA 1 For the ALMA PICs the value will be OxASAESX where the least significant nibble is constructed as follows L bit 0 0 X pol PIC 1 Y pol PIC 2 bits 1 and 2 ALMA BL quadrant number minus i e the quadrants are named through 4 so these bits will contain 0 through 3 3 bit 3 1 BL Correlator 0 2 ant correlator 2 For Mark6 testing the format is not finalized However it currently encodes the nominal packet rate and information about test marks made in the data stream 3 For use in other applications 0x000000 through 0x7FFFFF are available ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 25 of 44 Word 5 Words 6 and 7 A repeating series of eight 32 bit words is provided in Word 5 as shown in Table 3 These words will provide the value for the packet serial number of the first transmitted frame Subsequent frames will increment this count by 1 per frame Normally the reco
41. ord 7 Word 7 MSB From the start of transmission by the CCC it takes less than 4 msec for the transmitted data to be downloaded into hardware In the VLBI community a particular standard VDIF is followed for writing data additional info may be found below and in RD 05 APP shall follow this standard Additionally it shall prepend a Packet Serial Number PSN to each packet to enable eVLBI Each Phasing Interface Card PIC will produce a stream of VLBI Data Interchange Format VDIF packets The rules governing the packet stream are rather complex in general See RD 05 for all the details if you like For the ALMA case each PIC receives 2 bit data samples from each of the 32 62 5 MHz slices of the 2 GHz IF from its quadrant of the correlator With a factor of 2 for Nyquist sampling each PIC data stream thus produces 2bits channel x 32 channels sample x 2 x 62 5 Msamples s ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 24 of 44 64 x 125 Mbits second 8000 Mb s 1000 MB s 2 5 B s The VDIF data packets must be a multiple of 8 B in length and further the data bits from each second must be parceled equally into an integral number of packets Since these packets will be transmitted on a physical ethernet layer that protocol impo
42. ponds to sums to memory outputs corresponds to scaled sum cc ceeeeeeeeeeeees 20 Figure 3 6 VDIF frame format for the PIC eee ccceccecceccecceceeeeeaeeeeeseeseeeeeeeeceeeeeeeeeeeeaaaaas 26 Figure 3 7 Timing diagram which provides details in synchronizing the internal 1PPS to the ree ta Ad BT LCCC ee E E S 35 Figure 3 8 Timing diagram which provides details in synchronizing seconds from reference epoch and packet serial number to the internal timing references cccceeeeeeeeeeees 36 Figure 3 9 Photo of one of four recorders required for the APP cccccseeeeceeeeeeeeeeeeeeeeees 43 ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 8 of 44 1 Description 1 1 Purpose This ICD covers the all interfaces between ALMA Computing and the ALMA Phasing Project APP equipment 1 2 Scope The ALMA Phasing Project provides ALMA with the capability of phasing up to 63 Antennas and recording the resulting data for later correlation at another facility e g Haystack Observatory Equipment associated with the project includes a hydrogen maser for VLBI phase stability various upgrades to the 64 Antenna Correlator an optical data transmission system to transmit data over a single fiber from the AOS to OSF and
43. rder does not record the PSN that precedes the VDIF packet proper ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Draft Pending Approved Released Superseded Obsolete Page 26 of 44 Byte 3 Byte 2 Byte 1 Byte 0 31 24 23 16 15 8 7 Packet Serial Number LSW PSN Packet Serial Number MSW Word 0 0 1 0 seconds from Reference Epoch REA Word1 0 0 Data Frame within second Word2 0 0 0 Data Frame Length octets 629 or 1004 Word 3 0 EU 0 Thread ID Station ID Word 4 EDV 0x02 Word OxA5AE5X Word 5 PIC Status Word Word 6 Packet Serial Number LSW Word 7 Packet Serial Number MSW oaee ao oe ow eor a ot ow an Octet 0 Sees eee en emoe e er oe Octets 1 through N 2 c15 14 c13 12 c11 c10 c09 c08 cO7 c06 c05 c04 c03 c02 c01 ctet N 1 a oo a a eon oft ont ot oo a to tw wt at ot on aoe a ee oo cO7 c06 c05 c04 c03 c02 c01 c07 c06 c05 c04 c03 c02 c01 c00 Octet for 16ch Octet for 8ch Figure 3 6 VDIF frame format for the PIC ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete Page 27 of 44 And ALMA Computing Field Definitions Description Bit 0 Bit 1 TE Error when 1
44. ription Status resulting from APPLY _VDIF_HEADER commands Typical Interval Rare Data 2 bytes Message Reply Message Contents Payload Bytes Message RCA 0x2050A Data 0 a ee TFB Phases status Provides the status for the APPLY TFB PHASES command The status word can take on the following values e 0x02 No phases were received failure or none since the last apply do we want this e 0x0l Waiting for TE rising edge pending e 0x00 phases have been applied success ALMA Project Doc ALMA 05 11 00 00 70 35 25 00 A ICD Date 2014 10 07 Interface Control Document Status Draft Between ALMA Phasing Project Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 41 of 44 After transmitting a success indication the state is changed so that another monitor without another apply command would result in the no phases were received monitor word So the normal sequence is as follows APPLY _ TFB PHASES GET_APPLY_TFB_PHASES_STATUS get waiting for a response GET_APPLY_TFB_PHASES_ STATUS get waiting for a response GET_APPLY_TFB_PHASES_STATUS get success for a response APPLY _TFB PHASES GET_APPLY_TFB_PHASES_STATUS get waiting for a response GET_APPLY_TFB_PHASES_ STATUS get waiting for a response GET_APPLY_TFB_PHASES_STATUS get success for a response The sequence that will result in a failure indication 1s as follows APPLY_TFB_PHASES GET
45. ses an additional limit of no more than 9000 B packet including the VDIF header and UDP packet overhead The CASPER Roach design for the Ethernet implementation further restricts the VDIF portion to 8192 Subject to these restrictions the largest packet possible is 8000 B for a data rate of 1000 MB s 8000 B packet 125000 packets s The Packet Serial Number is exactly 8 B that precede the VDIF header The VDIF header is exactly 32 bytes that precede the data These fields and data portion are shown schematically in a cartoon of a complete packet as shown in Figure 3 6 In forming VDIF frames for the APP the values received from the CAN protocol are used as follows PSN Used for the PSN of the first transmitted frame In subsequent frames the PSN increases by 1 per frame Word 0 bits 0 to 29 Used to set the initial value of time in the timing generator in the ROACH FPGA at the rising edge of the TE tic following the APPLY_VDIF_HEADER command After this the time increments based on the 125 MHz correlator clock Word 0 bits 30 Always zero in all frames Word 0 bits 31 Sets the initial value for this bit in the VDIF frame The value of this bit can later be changed via a bit in the CAN command SET_PIC_CONTROL It can also be changed to set to 1 if the hardware detects a problem that corrupts the data All brown bits All bits shaded with the color brown in Figure 3 6 are identical to the corresponding bits in the CAN message f
46. t Draft Pending Approved Released Superseded Obsolete And ALMA Computing Page 38 of 44 8 39 40 43 44 47 48 5 ay 54 5 Nn n A ON WIR n N N OO 60l 6 Po W R A No sij OS Notes s pi pi l seek PERE pb 0 All ENVO Bit 0 0 means ROACH is off 1 means ROACH is on Environmental Monitors spare Sampled PIC 1PPS counter LSB TE Sampled PIC 1PPS counter middle bytes IPTE Sampled PIC 1PPS counter most significant nibble bits 3 to 0 UN UN gt Bit 1 0 means ROACH is not operational 1 means ROACH is operational 1 e ready for commands Bit 2 0 means no temperature warning 1 means temperature warning threshold 45 C has been exceeded Bit 3 0 means no temperature error 1 means temperature error threshold 50 C has been exceeded The ROACH board is powered down by the C167 when this occurs Bit 4 0 means no SEU error means that a CRC error has been detected in the FPGA personality likely caused by a Single Event Upset SEU This serves as a warning that the FPGA may not function correctly but no operational changes are made as a result of this error Bits 5 to 7 spare Statistics status 0 valid 1 in process 2 error invalid request 3 stale when valid statistics have been followed by another SET_PIC_CONTROL command 1 e new Statistics are being calculated and old statistics are in micropro
Download Pdf Manuals
Related Search
ALMA 05.11.00.00 70.35.25.00 A ICD