White Rabbit PTP Core User's Manual
Contents
1. location and add the hdlmake binary location to your PATH to be able to call it from any directory export PATH lt your_hdlmake_location gt PATH Note the hdlmake usage instructions here are based on version 493ce82 When there will be newer commits they can be used but please be aware that its execution parameters may change In that case please refer to hdlmake documentation Having Xilinx ISE software and hdlmake you can clone the main WR PTP Core git repository and start building the FPGA bitstream First please create a local copy of the wr cores in the preferred location on your system This release is marked with wrpc v2 0 tag git clone git ohwr org hdl core lib wr cores git your wrpc location cd Xyour wrpc location git checkout wrpc v2 0 Chapter 2 Building the Core 3 Note alternatively you can get the release sources from the tarball available in the files tab of wr cores http www ohwr org projects wr cores files The subdirectory which contains the main synthesis Manifest py for SPEC board and in which you should perform the whole process is cd lt your_wrpc_location gt syn spec_1_1 wr_core_demo Executing hdlmake without any parameters will fetch other git repositories containing submod ules essential for WRPC and store their local copies to lt your_wrpc_location gt ip_cores After that the actual synthesis is just the matter of executing the command make just as in a regular s2. Rx packet counters lock and calibration status If you have a DIO Mezzanine board placed on your SPEC you can check the synchronization quality by observing the difference between 1 PPS signals from the WR Master and WR Slave White Rabbit PTP Core generates 1 PPS signal to the LEMO connector No 1 on DIO Mez zanine However please remember to use oscilloscope cables having the same length and type with the same delay or take their delay difference into account in your measurements Chapter 5 Questions reporting bugs 8 4 Troubleshooting My computer hangs on loading spec ko driver This will occur when you try to load the spec ko kernel driver while your spec vuart is running and trying to get messages from Virtual UART s registers inside WRPC Please remember to quit spec vuart before reloading the driver I want to synthesize WRPC but hdlmake does nothing just quits without any message Please check if you have the Xilinx ISE related system variables set correctly settings32 sh script provided by Xilinx sets them and make sure you have overwritten the XILINX variable to export XILINX opt Xilinx lt version gt ISE_DS or similar if your installation folder differs from default WR PTP Core seems to work but I observe on my oscilloscope that the offset between 1 PPS signals from WR Master and WR Slave is more than 1 ns Run the t2 t4 phase transition value measurement procedure from the WRPC Shell wrc ptp stop wr
3. configs and you choose them by makeing them by name like this make spec defconfig The most important configuration choice at this point in time is whether to enable Etherbone or not It is disabled by default in spec defconfig and enabled by default in etherbone_ defconfig After the package is configured just run make without parameters to build your binary file make The resulting binary wrc bin can be then used with the loader from spec sw software package to program the LM32 inside the White Rabbit PTP Core Chapter 3 Running and Configuring page 4 3 Running and Configuring 3 1 Downloading firmware to SPEC There is a Software support for the SPEC board project in ohwr org It contains a set of Linux kernel drivers and user space tools written by Alessandro Rubini and Tomasz Wlostowski that are used to communicate with the SPEC board plugged into the PCI Express port of the PC The instructions in this section are based on commit 27b4ad9 of spec sw repository and are limited to absolutely minimum required to load WRPC FPGA and LM32 firmware The full manual for spec sw can be found at http www ohwr org attachments download 1506 spec sw 2012 08 08 pdf If there is a newer version of SPEC software support you would like to use the up to date documentation can always be found in doc subdirectory of spec sw git repository First please clone the git repository of SPEC software support package and build the kernel dr
4. for the channel start SoftPLL for the channel stop SoftPLL for the channel set the dac get dac s value starts GUI WRPC monitor prints one line log message prints log message for each second Esc to exit back to shell prints bitslide value for established WR Link needed by calibration procedure start WR PTP daemon stops WR PTP daemon prints available WR PTP modes sets WRPC to operate as Grandmaster clock requires external 10MHz and 1 PPS reference sets WRPC to operate as Free running Master sets WRPC to operate as Slave node tries to read t2 4 phase transition from EEP ROM if not found runs calibration procedure starts calibration procedure that measures t2 4 phase transition and stores the result to EEPROM prints current time from WRPC prints current time in a raw format seconds nanoseconds sets WRPC time prints the ID transceiver cleans the SFP database stored in FMC EEPROM stores calibration parameters for SFP to the database in FMC EEPROM prints all SFP transceivers stored in database of currently used SFP Appendix A WRPC Shell Commands sfp match init erase init add lt cmd gt init show init boot mac get mac getp mac set lt mac gt mac setp lt mac gt sdb ip get ip set lt ip gt mode requires running sfp detect_ first 10 tries to get calibration parameters from data base for currently used SFP transceiver cleans initializat
5. White Rabbit PTP Core User s Manual December 2012 Building and Running Grzegorz Daniluk CERN BE CO HT Table of Contents Ditto UG OB usse euo ve aay dad EDU kes IE EN E ess Gud can ese Ve eed 1 1 Software and hardware requirements Luuuu 1 1 1 Repositories and Releases 0 000 cece ene eee s e 1 1 2 Hardware needed ere t diei veiw dae ase SAANEEN 1 2 Building the Core uio ous ovd xp VERRX REESAd eraran 2 2 l JHDLbsynthesiS ece RM pe RELC CR RE RU Ge RI TEL UK ERU De ERU RD Ee ERR 2 2 0 LM32 software compilation ssssssslseeeslssees E a Rh he 3 3 Running and GConDe ull esce yere ROO rr C ras 4 3 1 Downloading firmware to SPEC sssssssssessessss ee er 4 3 2 Writing EEPROM and calibration 0 00 cece eee OaE 5 3 3 Running the Cores e ceoten uu Rer Rp wens ee Gales Some EP REP SCENA Ra ndr ERES 6 4 Troubleshooting suolo mh a RR RR Re he 8 5 Questions reporting bugs ssseeseeeeee eese 8 Appendix A WRPC Shell Commands 9 Appendix B WRPC GUI elements 11 Chapter 1 Software and hardware requirements 1 Introduction This is the user manual for the White Rabbit PTP Core developed on ohwr org It describes the building and running process If you don t want to get your hands dirty and prefer to use the binaries available at http www ohwr org projects wr cores files you can skip Chapter 2 Bui
6. age 5 you can restart the WR P TP Core by reprogramming the LM32 software with spec cl tool or by typing the shell command wrc init boot After that you should see the log messages that confirm the init script execution Chapter 3 Running and Configuring 7 Cored executing ptp stop executing sfp detect AXGE 3454 0531 executing sfp match SFP matched dTx 46407 dRx 167843 alpha 73622176 executing calibration Found phase transition in EEPROM 2384ps executing mode slave SPLL_Init running as Slave 1 ref channels 2 out channels Locking PLL executing ptp start wrc SPLL_Init running as Slave 1 ref channels 2 out channels Enabling ptracker channel 0 ere Now you should have the White Rabbit PTP Core running in WR Slave mode The Shell also contains the monitoring function which you can use to check the WR synchronization status wrc gui The information is presented in a clear auto refreshing screen full description about information reported by gui is provided in Appendix B WRPC GUI elements page 11 Sat Jan 3 1976 1 21 46 WR Slave TRACK_PHASE wrul 888468 ps 426517 ps TX 16 ps RX 161610 ps TX 46407 ps RX 172643 ps 35434 ps 507798 ps 8 ps 125 ps 2 ps 6 ps 273 Note the Synchronization status and Timing parameters in gui are available only in WR Slave mode When running as WR Master you would be able to see only the current date and time link status Tx and
7. c calibration force and check if the oscilloscope cables you use have the same delays or take the delay difference into account in your measurements I can see in the WRPC GUI that the servo cannot reach TRACK_PHASE state Please stop the PTP daemon on your SPEC read your SFP s parameters from SFP database you have created in EEPROM and run the t2 t4 phase transition value measurement procedure form WRPC Shell wrc ptp stop wrc sfp detect wrc sfp match wrc calibration force 5 Questions reporting bugs If you have found a bug you have problems with White Rabbit PTP Core or one of the tools used to build and run it you can write to our mailing list white rabbit dev ohwr org Appendix A WRPC Shell Commands Appendix A WRPC Shell Commands pll init mode ref channel align pps pll cl channel pll sps channel lt picoseconds gt pll gps channel pll start channel pll stop channel pll sdac index val pll gdac index gui Stat Stat cont stat bts ptp start ptp stop mode mode grandmaster mode master mode slave calibration calibration force time time raw time set sec lt nsec gt sfp detect sfp erase sfp add lt ID gt lt deltaTx gt lt deltaRx gt lt alpha gt sfp show manually run spll_init function to initialize SoftPll check if SoftPLL is locked for the channel set phase shift for the channel get current and target phase shift
8. d the Xilinx ISE Software with free WebPack license installed on a PC It contains the script settings32 sh and settings32 csh depending on the shell you use that sets up all the system variables required by Xilinx software For default installation path the script is located in opt Xilinx version ISE DS settings32 sh and has to be executed before other tools are used The easiest way to ensure that IS E related variables are set in the shell is to check if XILINX variable contains the path to your ISE installation directory Note current version of hdlmake tool developed at CERN requires modification of XILINX variable after settings32 script execution This provided that the installation path for SE is opt Xilinx lt version gt should look like this export XILINX opt Xilinx version ISE DS Note the Xilinx project file included in the WRPC sources was created with Xilinx ISE 14 1 It is recommended to use the newest available version of ISE software HDL sources for WR PTP Core can be synthesized with nothing more but Xilinx ISE software but using hdlmake tool developed at CERN is much more convenient It creates a synthesis Makefile and ISE project file based on the set of Manifest py files deployed among directories inside wr cores repository First please clone the hdlmake repository from its location given in Section 1 1 Repositories and Releases page 1 git clone git ohwr org misc hdl make git your hdlmake
9. eate a calibration database inside the FMC EEPROM The example below presents the WRPC Shell commands which create an empty SFP database and add two Axcen transceivers with deltaTx deltaRx and alpha parameters associated with them Those SFPs are most widely used in WR development and demonstrations wrc sfp erase wrc sfp add AXGE 1254 0531 46407 167843 73622176 wrc sfp add AXGE 3454 0531 46407 167843 73622176 Chapter 3 Running and Configuring 6 To check the content of the SFP database you can execute the sfp show shell command The calibration procedure of WRPC is limited to absolutely minimum and is fully automatic It measures the t2 t4 phase transition point and stores the value into the FMC EEPROM so that the calibration would not have to be repeated every time the Core starts However it is important to remember that this calibration function should be executed only once but for every new FPGA firmware synthesized form wr cores repository wrc ptp stop wrc sfp detect wrc sfp match wrc calibration force The example above detects the SFP transceiver plugged into the SPEC board tries to read its parameters from our newly created SFP database and forces the calibration to be executed The force argument is required since calibration without any arguments tries first to read the t2 t4 phase transition stored in EEPROM and does not run the procedure if the value was previously stored there That is used to get the measu
10. in the spec sw repository Programming is done with the simple command below sudo tools spec cl lt your_wrpcsw_location gt wrc bin Now you should be able to start the Virtual UART software also a part of spec sw package that will be used to interact with the White Rabbit P TP Core Shell sudo tools spec vuart If you are able to see the WRPC Shell prompt wrczz that means the Core is up and running on your SPEC Congratulations 3 2 Writing EEPROM and calibration By default WRPC starts in WR Slave mode uses the calibration values for Axcen AXGE 3454 0531 SFP and for release FPGA bitstream available in hitp www ohwr org projects wr cores files This might be fine for running White Rabbit PTP Core for the first time and synchronizing it to WR Switch There are however two mechanisms that are useful when playing more with WRPC shell and different settings Note the examples below describe only a subset of WRPC Shell commands required to make a basic configuration and calibration A full description of all supported commands can be found in Appendix A WRPC Shell commands page 9 First before making the configuration changes it is recommended but not obligatory to stop the PTP daemon Then the debug messages from daemon would not show up to the console while you will interact with the shell wrc ptp stop If your SPEC has any Mezzanine board plugged into the FMC connector e g DIO Fine Delay TDC then you can cr
11. interface name from which WR daemon gets synchro nization lt wru1 gt round trip delay in picoseconds delayy y estimated one way master to slave link delay delayy s transmission reception delays of WR Masters hardware Arxm Arxm of WR Slave s transmission reception hardware Arxs Arxs delays WR link asymmetry calculated as delayy y 2 delayms round trip fiber latency Slave to Master offset calculated by PTP daemon of f sets current Slave s clock phase shift value the difference between current and previous estimated one way link delay the state of counter incremented every time the WR servo is updated
12. ion script in FMC EEPROM adds shell command at the end of initializa tion script prints all commands from the script stored in EEPROM executes the script stored in FMC EEPROM the same action is done automatically when WRPC starts after resetting LM32 prints WRPC s MAC address re generates MAC address from 1 wire digital thermometer or EEPROM sets the MAC address of WRPC sets MAC address to the 1 wire EEPROM if available prints devices connected to the Wishbone bus inside WRPC prints the IPv4 address of the WRPC sets the IPv4 address of the WRPC after executing _mode_ command ptp start is required to start WR PTP daemon in new available only with Etherbone support compiled in Appendix B WRPC GUI elements 11 Appendix B WRPC GUI elements TAI Time RX TX mode lt Locked NoLock gt Servo state Phase tracking Synchronization source Round trip time mu Master slave delay Master PHY delays Slave PHY delays Total link asymmetry Cable rtt delay Clock offset Phase setpoint Skew Update counter current state of device s local clock Rx Tx packets counters operation mode of White Rabbit PTP Core lt WR Master WR Slave gt SoftPLL lock state current state of WR servo state machine lt Uninitialized SYNC SEC SYNC NSEC SYNC PHASE TRACK PHASE is phase tracking enabled when WR Slave is synchronized to WR Master ON OFF network
13. iver and user space tools git clone git ohwr org fmc projects spec spec sw git your specsw location cd Xyour specsw location git checkout 27b4ad9 make Then you have to copy the spec top bin to lib firmware fmc changing its name Note the commands below have to be executed with superuser rights Chapter 3 Running and Configuring 5 sudo cp your wrpc location syn spec 1 1 wr core demo spec top bin lib firmware fmc spec demo bin and after that you are ready to load the spec ko driver that configures the Spartan 6 FPGA on SPEC with a given bitstream make sure you are in Xyour specsw location sudo insmod kernel spec ko name demo To check if the FPGA firmware file was found by the driver and correctly loaded to FPGA the dmesg Linux command can be called Among plenty of messages you should be able to find something very similar to 99883 768214 spec probe device 0003 0000 99883 768220 spec probe current 8639 insmod 99883 768248 spec 0000 03 00 0 PCI INT A gt GSI 16 level low gt IRQ 16 99883 768302 spec 0000 03 00 0 irq 49 for MSI MSI X 99883 768971 spec load files 99883 774842 spec load fpga got binary file fmc spec demo bin 1485512 0xi6aac8 bytes 99883 966491 spec load submodule load fmc spec demo 256 If everything went right up to this moment you can write the LM32 binary wrc bin to the SPEC board For this purpose there is a spec cl tool
14. lding the Core page 2 and move forward directly to Chapter 3 Running and Configuring page 4 1 Software and hardware requirements 1 1 Repositories and Releases This manual you are reading is not about an official release but a snapshot of the current master branch The last release we blessed is called 2 0 The code and documentation for the project is distributed in the following places http www ohwr org projects wr cores documents This place hosts the pdf documentation for every official release http www ohwr org projects wr cores files Here we place the tar gz file for every release including the git tree and synthe sized compiled binaries git ohwr org hdl core lib wr cores git Read only repository with the complete HDL design of WRPC git ohwr org hdl core lib wr cores wrpc sw git Read only repository with the WRPC LM32 software incl WR PTP daemon Other tools useful in building and running WRPC can be fetched from the following locations git ohwr org misc hdl make git hdlmake is used in the HDL synthesis process to build the Makefile based on the set of Manifest files http www ohwr org attachments download 1133 1m32 tar xz LM32 toolchain used to compile the WRPC firmware software This specific file is linked from the files tab of the wrpc sw project The repositories containing the WRPC gateware and software wr cores wrpc sw are tagged with wrpc v2 0 tag Other tools used to build the core a
15. nd load it into SPEC board should be used in their newest available versions stored in master branch of an appropriate git repository unless specified otherwise in this document Any official hot fixes if any for this release will live in the branch called wrpc v2 0 fixes in each WRPC repository 1 2 Hardware needed The absolutely minimum hardware you need to build and run the White Rabbit PTP Core is a PC computer with Linux and one Simple PCIe FMC Carrier SPEC http www ohwr org projects spec However it is recommended to use also the DIO FMC card http www ohwr org projects fmc dio 5chttla for storing calibration values and configuration in EEPROM described later To test the White Rabbit synchronization you will also need e second SPEC board with DIO FMC or White Rabbit Switch Chapter 2 Building the Core 2 e pair of SFP transceivers AXCEN AXGE 1254 0531 and AXCEN AXGE 3454 0531 are recommended e a roll of G652 single mode fiber to connect your SPECs or SPEC with WR Switch 2 Building the Core Building the White Rabbit PTP Core is a two step process First you have to synthesize the FPGA firmware gateware This describes the hardware inside FPGA that is later used by LM32 software to perform WR synchronization To perform the steps below you will need a computer running Linux 2 1 HDL synthesis Before running the synthesis process you have to make sure that your environment is set correctly You nee
16. oftware compilation process This takes depending on your computer speed about 15 minutes and if you are lucky i e there were no errors it should create FPGA firmware in two files spec_top bit and spec_top bin The former can be downloaded to FPGA using Xilinx software Impact or Chipscope Pro and Xilinx Platform Cable The latter can be used to program the Spartan 6 chip on SPEC using the kernel driver from spec sw repository usage example in Chapter 3 Running and Configuring page 4 If on the other hand you would like to clean up the repository and rebuild everything from scratch you can use the following commands e make clean removes all synthesis reports and log files e make mrproper removes spec_top bin and spec top bit files e hdlmake clean removes all fetched repositories modules from ip cores subdirectory 2 2 LM32 software compilation To compile the LM32 software for White Rabbit PTP Core you will need to download and unpack the LM32 toolchain from the location mentioned already in Section 1 1 Repositories and Releases page 1 wget http www ohwr org attachments download 1133 1m32 tar xz tar xJf 1m32 tar xz C your 1m32 location Similar as with hdlmake in Chapter 2 Building the Core page 2 you will need to add the LM32 toolchain binaries location to you PATH to be able to call them from any directory export PATH lt your_1m32_location gt 1m32 bin PATH To get the release so
17. red value and pass it to PTP daemon before it starts see init script examples The WR PTP Core s mode of operation WR Master WR Slave can be set using the mode shell command in one of the following two ways wrc mode slave wrc mode master This stops the PTP daemon changes the mode of operation but does not start it back auto matically Therefore after changing it you need to start the daemon manually wrc ptp start One option is to type all those commands to initialize the WRPC software to the required state every time the Core starts However you can also write your own init script to FMC EEPROM and WRPC software will execute it each time it comes back from the reset state this also includes coming back from reset after programming the FPGA and LM32 Building the simple script that reads the detected SFP parameters and t2 t4 phase transition value from EEPROM configures the mode of operation to WR Slave and starts the PTP daemon is presented here wrc init erase wrc init add ptp stop wrc init add sfp detect wrc init add sfp match wrc init add calibration wrc init add mode slave wrc init add ptp start Almost exactly the same one can be used for running SPEC in WR Master mode The only difference would be of course init add mode slave vs init add mode master 3 3 Running the Core Having the SFP database t2 t4 phase transition point and the init script created in Section 3 2 Writing EEPROM and calibration p
18. urces of WRPC software please clone the wrpc sw git repository tagged with wrpc v2 0 tag Otherwise you can use the current master branch with the latest improvements and fixes git clone git ohwr org hdl core lib wr cores wrpc sw git your wrpcsw location cd Xyour wrpcsw location git checkout wrpc v2 0 or git checkout master Note alternatively you can get the release sources from the tarball available in the files tab of the wr cores OHWR project The WRPC software repository contains a ptp noposix that contains the WR PTP software daemon in the form of a git submodule Your fresh local copy cloned from ohwr org has therefore the ptp noposix directory empty To fetch the ptp noposix you have to execute the following git commands Chapter 3 Running and Configuring 4 git submodule init git submodule update First you have to compile the tools provided with WRPC software which are used later during the software compilation make C tools Now you have everything that is needed to build the software for WRPC but you need to make a few configuration choices T he package is using Kconfig as a configuration engine so you may run one of the following commnads the first is text mode the second uses a KDE GUI and the third uses a Gnome GUI make menuconfig make xconfig make gconfig Other Kconfig target applies like config oldconfig and so on A few default known good configurations are found in
Download Pdf Manuals
Related Search