Home

DN9002K10PCIE8T

1. 49 CONFIGURATION 240 49 3 1 CONFIGURATION SECTION FEEDBACK c ccsscssesssesscesccseesecesscseesccesscssecstesenseenes 50 2 22 CONFIGURATION tede ee Ee dedere vester tocado di ar pee 51 3 3 USB AND PCIENTERFACES 52 3 4 COMPACTFLASH INTERFACE s cssccssssscsscessescsseescesecesecseesecesscseesceesscsscesseseenseenes 52 ar o 52 32L2 Hardwarezssi asta NEN TE thes 55 3 5 CONFIGURATION REGISTERS cscccssssscssesssescsseescescesecseeeccesscsseessesscsaeeatesscaeenee 56 3 51 Undocumented controls eee eee 56 316 GBIBMWARE hui i EORR fia i 57 CLOCKNETWORRK 57 TT COR LU EE 57 2 GLOBAL CLOCK 57 45 6 58 43T EU TI Ma nmt E 59 4 4 61 dr qe dant E a CPE 61 D ix 62 44 3 Daughtercard zero delay mode esee 62 444 MA TID aid i tei RH ead o toe asd A eR aso a ie eeu 62 4 JPGIEREEGPOCK
2. ibas seta map 169 4 35 L utei tedio 169 4 3 2 USA Schedule B number based on the 5 169 4 3 3 Export control classification number ECON sese 170 AA 170 Chapter 1 Introduction Congratulations on your purchase of DN9002K10PCIEST logic emulation board If you are unfamiliar with Dini Group products you should read Chapter 2 Quick Start Guide to familiarize yourself with the user interfaces the DN9002K10PCIEST provides Figure 1 DN9002K10PCIEST Heat sinks negligently left uninstalled 4 Manual Contents This manual contains the following chapters 1 1 Introduction Reader s Guide to this manual List of available documentation and resources 1 2 Quick Start Guide Step by step instructions for powering on the DN9002K10PCIE8T loading and communicating with a simple provided FPGA design and using the board s common control features 1 3 Controller Software summary of the functionality of the provided software Implementation details for the remote USB board control functions and instructions for developing your own USB host software DN9002K10PCIES8T User Guide www dinigroup com INTRODUCT
3. 12 Mill Max 999 11 210 10 0 X1 AMP Tyco 2 641260 1 2 B 250 6 1 5 MP1 CCI B 250 6 1 5 M1 3 NONE NONE DN9002K10PCIE8T User Guide Connector description PCle Bracket 1 8V adjust Jumper 8 pin DIP socket 1 8V adjust 1 8V connects to DIMMS Used by the DN9002K10PCIEST Board Stiffeners Board Stiffeners M3 Holes www dinigroup com to stote firmware data Board stiffeners Board stiffeners Mounting Holes 129 HARDWARE 23 3 1 Comments Y3 The pins of Y3 are part of the DN9002K10PCIE8T mechanical drawing in i Mechanical section The two pins grounded can be installed to change the 1 8V power rail the DN9002K10PCIE8T 2 5V 2 These jumpers connect the DIMM interfaces to the 1 8V rail They be removed to change the DIMM voltage M1 3 These are connected to GND 24Mechanical The DN9002K10PCIEST is larger than the PCIe specification allows and is not guaranteed to fit into any ATX case It will certainly fail to fit into a rack mount server enclosure The vertical clearance with the fans installed and the ATX power connector not connector is 30mm Lower profile fans are available 14mm but they may not have enough thermal performance for very power hungry designs 135 213 184 743 213 213 262 743 146 9 138 5 o o e sak R o o o 105 05 pip 7 75 854 9
4. ead s EE 111 JO 3 112 182 112 18 3 iet tice i itte eei terre 113 18 3 1 Conventional Memory map eee 114 19 ETHERNET 115 NE Ss 115 49 71 Electrical iesu ierit testet sait beet ense ceo eo orna reae ia 115 ef 116 19 25 ELS ERE 118 20 RE 119 21 CONNECTORS 4 eene et reas eno eo kenne anat eso eaa ta e een n a aao ad 120 21 17 120 21 22 MAINBUSMICTOR 5 Bets 121 222 POWER 2 422204244 lt 6960 eased 123 2247 vete ett 123 22225 eee tee tte io tette tee cds 124 22 32 POWER teet re exeo res leto Fives a e 124 224 GROUND d esee 124 22 5 POWER CONNECTIONS 124 226 POWER MONITORS 125 2275 HEAT es
5. DiniGroup DN6000K10 FLASH Boot Use the check boxes below to limit or expand the default search which includes local paths and removable media The best driver found will be installed Search removable media floppy CD ROM If your hardware came with an installation CD 1 or floppy disk insert it now Include this location in the search What do you want the wizard to do C In install io Dont search will choose the driver to install Choose this option to select the device driver from a list Windows does not guarantee that the driver you choose will be the best match for your hardware Click Next to continue ea In the window that appears select Install from a list or specific location Select Next Click Include this location in the search and browse to D NUSB Software Applications Vdtiver windows wdm Select Next In the next window select the item in the list Dini Group ASIC Emulator Click FINISH After Windows installs the driver you will be able to see the following device in the ASIC Emulators group the Windows device manager DiniGroup Product FLASH Boot 4 2 Operating the USB Controller program Run the USB controller application found on the product CD in D NUSB Software Applications USBControllerNUSBController exe DN9002K10PCIE8T User Guide www dinigroup com CONTROLLER SOFTWARE Products Controlle
6. ine eee aee 83 7 3 3 DMA Channels and 1 83 7 3 4 Posted 84 PS PMA sc e e Rs BAR 84 73 6 84 VS A oor TT 84 buc cd e olas Rath Dna Nerd anaa A EE 85 7 3 9 5 86 7 4 OTHER PROVIDED DESIGNS FOR 2 222 2 0 02 00000 0 1 0 858 89 89 72522 VIL EU 90 7 4 3 Slowdown PIPE Core eerte then thee tnter tenens 90 7 3 TROUBLESHOOTING ect arido ducat 91 9 UNUSABLE PINS tus E hee Qe eie tide des 92 INTRODUCTION OU 92 9 e veis eti tn 92 C UNES 93 POWERRESET ud 93 10 2 USER RESET 94 TD IVA Ga 94 LT ots sesso asa estas loses iain mE Sd aan 95 11 1 1 Compatible Configuration Devices 95 TAE CMPS COPE aa eta tM AM 95 11 2 FIRMWARE UPDATE HEADER 96 11 3 TROUBLESHOOTING eret ierit tocco Decet etie Ee eit Lea a
7. 0 0 0 0 400 40 en 17 gd NGO SUITS an PPOGA AT atia Dea Re e uot 18 42 2 Set Clock Freguencies 2 spe i Ue 19 4 2 3 Hardware Test DDR2 esset ethernet tennis 19 4 3 GETTING DATA TO AND FROM THE FPGA eene entente ei 20 5 COMMUNICATING OVER THE SERIAL 4 21 6 RUN AETEST 220205 0 21 6 1 1 6 RID naue 21 7 SCAN THE JTAG CHAIN veisssscicicccscccsiscscceccestcocsssevescscsestecstsanesccdsccesccsbeveccsecsseecees 23 AA be en 24 CHAPTER 3 CONTROLLER 25 1 220022200400 ta eno roa eoo oracio fo ose 26 LI lt 5 26 Dad Refresh Button 27 1 12 Disable Enable erre 27 OW a E eite fe 27 Td lt Board eset teen ete MG c e ud 27 L2
8. 5 before etes et be oa 29 INTRODUCTION T8 EE 05 0507 FORE 29 E22 Edi Menit ena 29 1 2 3 FPGA Configuration 29 T24 FPGA Reference DOSIS las 30 VU ENIMS COUP E 30 1 2 6 asi aid nude dedi itaq Dade gua A ERR edet 31 L27 Producton Tests or eco UR had case ie d ente 22 1 2 8 Unsupported 32 13i 32 2 USB 33 2 COMPILING AE TEST ned 33 inva 33 3 PCIEAETEST APPLICATION sssscsecssssssessssssoxssessesesesssssvesonesvesssiasovasensarvnesicensese 33 Jub FUNCTIONAL Y oeste 33 3 2 RUNNING AB VES 34 33 34 3 3 1___ AETest for Windows 34 34 4 ROLLING YOUR OWN 2 35 db nebst a Streets 35 MER ec CU EE 35 5 UPDATING THE FIRMWARE
9. 63 4 6 NON GLOBAL CLOCKS a a 63 461 dicc 63 46 2 Ethernet Clock see tete 64 4639 JDDR2 Clocks eiie e iege beate b us te etras eden 64 46 45 SMASCTOCR Ai ioi impe ended ique RE 64 TEST POINTS 65 5 POWER THRU HOLE 65 29 27 POWER TP et URBI SINE 66 5 3 DIMM POWER ERE n e na Ieee 67 OQ PEST POINT Si eet eat et tots oe Sete ees 68 53 CROCK TEST POINTS etse ipie i iet eril 69 5 6 DIMM SIGNALS onte eret reb dh eere ve 70 USB INTERFACE 70 6 1 CONNECTING TO THE 9002 1 2 2 400 0600000000000000 000208 71 Windows oe iicet eter 71 6 1 2 Windows ViStdiicceeccccccccccsscccsssccesscccsssccsssscesssscssscsssscssssecessecesscesessscessescescesssecs 71 INTRODUCTION SS M PIT EE 71 6 54 EO t a bases 71 6 2 VENDOR REQUESTS a 12 621 VR CLEAR FPGA 72 0 2 2 VRSETUP CONFIG 73 02 5 TEN END CONFIG A Ss otio dien edd te dett md chu 73 6 244 SET EPOIC Read buffer size ceo editt Erb 73 6 2 5 VR MEM MAPPED Configuration Registers
10. eee eese sees tatnen 35 5 1 OBTAINING THE UPDATES dace bed iced 36 5 2 UPDATING THE SPARTAN PROM 2 2 2 2 40 240 00 0 1000000 0000000000000022 36 5 2 71 Using JTAG cable Xinlinx products ce cis ah 36 3 22 Using USBGontroller zs edet reet iratos dud 38 9 2 3 AEtest USB tame fecic 38 5 3 UPDATING EEPROM FIRMWARE NOT RECOMMEND cssscssesssesscssesseessenseenee 39 Sx t ODnvtsBCOnrolleP ss Rt M me tbt te MN UU E NS 40 Dub Using AL POSE USE Ute pec 40 5 4 UPDATING THE MCU FLASH FIRMWARE eee ener enne nhnn trennen 41 USBGONITOLEF S te up Neto Uu ftis ntl din 41 3 4 2 Using olea a hahah ciel pieles Sade 41 5 5 PCIEXPRESS ENDPOINT FIRMWARE 0 42 2224 Using JTAG USB cable Xilinx products aac s sete toot tees 42 2052 Anc SDGONTOHOR S t eroe ande ma 45 JIS Usm APTE e 45 CHAPTER 4 HARDWARE 47 I GENERAL OVERVIEW 47 INTRODUCTION 2 3 4 5 6 T MARKETING 59 47 BEE ae ote eue 48
11. iR le 1 1 225 225 i i i i 3 5 333 556 273 r4 304 4 Mounting holes are all over the place These are grounded DN9002K10PCIEST User Guide www dinigroup com 130 HARDWARE Metal runners are along both edges of the board These are for ground oscilloscope probe ground clips You should also handle DN9002K10PCIEST by its ground bars to help prevent ESD damage to the FPGAs 25Daughtercard Headers The daughter card expansion capability of the DN9002K10PCIE8T is provided by two FCI MEG Atray family connectors Even though it uses the same FCI connector it is NOT compatible with the 300 pin MSA standard DN9002K10PCIE8T User Guide www dinigroup com 131 HARDWARE Each daughtercard connector provides 186 signals plus 4 clocks to its associated FPGA The signals can be used with just about any setting of IOSTANDARD and can be used differentially FPGA A FPGA Virtex 5 Virtex 5 eo e gt lt 5 E 5 49 Es gt 2 Only available on LX330 Daughtercard A Daughtercard B BOT Power Reset Power Reset The daughter card interface includes a 400 pin MEG Array connector made by FCI The daughter card header is arranged into three Banks correlating to the banks of IO on the Virtex 5 Each of these banks connects to one or more Banks on the Virtex 5 FPGA This allows three
12. 162 CHAPTER 6 ORDERING INFORMATION sten eene tene ens tns ense 163 1 SECTION TITLE 20565088 163 2 FPGA 163 2 163 22 PARTS echten ecrire 163 2 2 1 Hardware Errata 163 2 53 SMALDE TFPGAS 164 274 SSPEED GRADES 2 165 ZJ UPORA DE POLICY itii etc pe detect N lebe dba evt 165 8 OPTIONAL 2 gt 555 166 3 1 COMPATIBLE DINI 2 20 0 4 0000 166 Sul oe e eee eee edet 166 3 12 JDaushickcards usas nini vates retarded 167 3 2 COMPATIBLE THIRD PARTY PRODUCTS 0 000 00000000 0 000000 ee tene entes en neon 168 INTRODUCTION 4 COMPLIANCE DAT TA sesesescccccccescsososcscesesesesccccccccsosososceseseseseseccccccesesoscscesesese 169 A GOMPLIANCE nai a 169 LE MEM CC p 169 412 SUG a est aee tuv ieu a eandem etus 169 42 ENVIRONMENTAL editae ecd catio rs audere son 169 Ou icu Cafes i d 169 43
13. ALL FPGAs 250MHz 1 d L The diagram above shows the block diagram representing the resources available on a board installed with two LX110 or LX220 FPGAs One third of each daughter card s pins become unusable and the amount of interconnect between FPGA A and B is reduced from 696 signals to 378 signals The Ethernet PCle and DIMMs ate not affected by FPGA selection Also you should analyze your design to determine if the internal resources available in the LX110 and LX220 are sufficient to meet your needs The FPGA selection guide from Xilinx is printed below DN9002K10PCIEST User Guide www dinigroup com 164 LX110 LX220 LX33 ER Part Number XCBVLX110 XC5VLX220 XC5VLX330 CLB Array Size Row x Column 160 x 54 160 x 108 240 x 108 Slices 2 17 280 34 560 51 840 Logic Cells 3 110 592 221 184 331 776 CLB Flip Flops 69 120 138 240 207 360 Maximum Distributed RAM kbits 1 120 2 280 3 420 Block RAM FIFO w ECC 36kbits each 129 192 288 Total Block RAM kbits 4 608 6 912 10 368 DSP48E Slices 64 128 192 FF1760 42 5 x 42 5 vO 800 800 1200 2 4 Speed Grades The interface performance characterizations included in this manual and in advertisements are valid for all shipped FPGAs regardless of speed grade These numbers are characterizations and not guaranteed under all operational conditions Every shipped board has passed this characterization test under some operational conditions If t
14. esee 73 6 2 6 Vendor REQUESTS Sea een terri Dra an bride 73 6 3 MAIN BUS ACCESSES 73 6 3 1 Important Note about ee ee eese tentent ntn 74 0 902 beate t dest pl ta CE Ta EAE 74 64 FPGA CONFIGURATION dde ide emet pa ee tp Iq 75 Readback das ue deti e EH 75 6 5 HARDWARE 76 6 5 1 GA 76 6 2222 V ACHVID LED 77 05 2 77 0 9 4 E 77 6 TROUBLESHOOTING 77 6 6 1 USB Controller Freezes eese eene nennen nennen enne 77 6 6 2 Main Bus always returns Error Codes esses 78 78 7 1 HOST INTERFACE ELECTRICAL 79 Lol 80 JT et aea 81 7 2 HOST INTERFACE MECHANICAL 81 7 PROVIDED FULL FUNCTION PCI EXPRESS ENDPOINT eese 81 A34 BAR 82 7 3 2 e
15. 3 Reference Design Types The Reference Design in this chapter refers to the FPGA designs located on the user CD at D FPGA_Reference_Designs DN9002K10PCIE8T MainRef D FPGA_Reference_Designs Programming_Files DN9002K1 0PCIE8T MainRef Four other self contained designs are on the CD and described in this manual These four designs are described in their own sections later in this chapter The remaining sections describe the design MainTest MainRef reference design and Dini Group reference design are the same thing The four additional designs are PCle Interface Design Tests the 64 bit interface between FPGA and the LX50T PCIe LVDS Reference Design Characterizes the FPGA interconnect using source synchronous Ethernet Reference Design Tests the Ethernet PHY Other features of the board such as memory sockets and daughtercard headers are tested using the Main Test 3 1 Main Test This reference design is also referred to as SINGLE because it is used to test the FPGA to FPGA interconnect This reference design provides access to the following All FPGA clocks DDR2 memory MainBus for USB and PCI Express RS232 Inch header pins DN9002K10PCIEST User Guide www dinigroup com 153 THE REFERENCE DESIGN 3 2 LVDS This reference design is an implementation of Xilinx App Note 705 It achieves 900 Mt sec per LVDS pair between F
16. An example test output is given below TEMPERATURE ALERT FPGA CURRENT TEMPERATURE 81 DEGREES C THRESHOLD TEMPERATURE 80 DEGREES C DN9002K10PCIE8T User Guide www dinigroup com 97 HARDWARE THE FPGA IS BEING CLEARED IN AN ATTEMPT TO PREVENT HEAT DAMAGE SOFTWARE WILL PREVENT RECONFIGURATION UNTIL THE TEMPERATURE DROPS A FULL DEGREE BELOW THE THRESHOLD TEMPERATURE DRA AAA AAA eoe eek eoe AA AACA AACA AACA AAA ACR RA DRA A AAA AAA AAA AACA AAA AAA AA AA A eoe AACA AACA AAA ACR ete TEMPERATURE ALERT FPGA A CURRENT TEMPERATURE 79 DEGREES C THRESHOLD TEMPERATURE 80 DEGREES C THE FPGA HAS DROPPED BELOW THE ALARM THRESHOLD AND MAY NOW BE RECONFIGURED DRA AAA AAA eek A AA eek AAA AAC AACA AACA AAA ACA AA The FPGA can safely operate as hot as 120 degrees but timing is not guaranteed You can use the temperature setting in the ISE place and route tool to make timing allowances for operating the FPGA out of range If you want to disable the temperature limit on the DN9002K10PCIEST you can do that using a menu option in the configuration RS232 interface 14Encryption Battery The Virtex5 FPGA supports bit stream encryption When using encryption the FPGA must decode the bitstream using a secret key that is stored in a persistent memory in the F
17. CONTROLLER SOFTWARE FPGA The syntax of this file is similar or identical to the syntax of the CompactFlash main txt interface Details are found in the USB Controller manual on the user CD at D USB_Software_Applications USBController doc USBController_Manual pdf Configure via CompactFlash this command causes the FPGAs to configure based on the instructions in the main txt file on the CompactFlash card It will also cause the commands and settings on the main txt file to be re issued Clear All FPGAs this command all FPGAs causing them to lose their configuration Reconfigure All FPGAs this menu command is equivalent to selecting reconfigure FPGA in the context menu of each of the FPGAs Each FPGA is cleared before being configured The last bit file that was loaded via USB for each FPGA is loaded again into the FPGA If an FPGA has not been loaded with a bit file using 2 instance of USB controller it is skipped Reset this command asserts the RESET signal to all FPGAs simultaneously This is the same signal that is asserted when the user hits the Soft Reset User Reset button Its function in the user design is left for the user to define In the reference design it causes a global asynchronous reset This option also causes the SYS_RSTn signal on the daughtercards to be asserted 1 2 4 FPGA Reference Design e Single Ended Interconnect Test slow this menu option will run test single ended
18. There is also a Synthesizer that can generate 100 or 250 MHz for use with RocketIO This synthesizer is described in another section Xilinx does not recommend synthesizing a reference clock frequency for use with PCI Express because it is not a supported use model The order of the lanes is as shown above Oh also none of the lanes have inverted polarity and you are required to support that if you are writing your own PCI Express endpoint We can tun the PCI SIG electrical compliance test for you if you want Vert Cursor PE 120 6ps t 119ps At 239 5ps Horiz Cursor V4 477 6 V4 477 6mV 200ps 150ps 100 50 Ops 50 100ps 150ps AV 955 2mV Figure 22 PCI Express eye diagram Here is the board installed with an passing the PCI Express electrical compliance test 7 1 1 Power The DN9002K10PCIE8T current capacity greatly exceeds the maximum allowed power requirements for a PCle card 35W As a result the external power cable is required for operation regatdless of whether the board is installed into a PCI Express slot The only voltage that is required for operation is 12V All other voltages used on the board regulated from this soutce The DN9002K10PCIEST is designed to operate in hot plug environments however most motherboards are not hot plug capable They do not shut off 12V and 3 3V power signals DN9002K10PCIEST User Guide www dinigroup com 80 HARDWARE when physical
19. Active low signals end in In the provided UCF files the is replaced an N 3 5 Datasheet Library Datasheets for all parts used or interfaced to on the DN9002K10PCIE8T are provided on the user CD In order to successfully use the DN9002K10PCIEST you will have to reference these datasheets interface descriptions given in this user manual typically end with electrical connectivity Especially read the Virtex 5 user guide The copy provided on the user CD 1s only recent as of the DN9002K10PCIES8T product announcement 3 6 Xilinx Virtex 5 is a brand new device and technical questions about getting the FPGA and ISE softwate to behave like you expect should be directed to a Xilinx FAE Also use WebCase http www xilinx com support clearexpress websupport htm AnswerBrowser http www xilinx com xlnx xil ans browset jsp ISE Manual http www xilinx com supporti sw manuals f xilinx82 index htm Virtex 5 Manual s i 1 9002 User Guide www dinigroup com INTRODUCTION 3 7 Dini Group Reference Designs The source code to the reference designs are on the User CD Please copy and use any code you would like The reference designs themselves are not deliverables and as such receive limited support 4 Email and Phone Support Dini Group technical support for products can be reached via email at support dinigroup com Our phone number is USA 858 454 3419 Please do not send
20. signals In this document MB will be used when referring the signals themselves and MainBus when referring to the Dini Group defined 36 signal interface description 18 1 2 Electrical The MB signals fixed at a 2 5V signaling level LVCMOS25 is an appropriate singling standard Due to very heavy capacitive loads on the MB signals you must use drive strength of 24mA to use main bus DCI should not be used because the signals are not impedance controlled Although not required by convention data on the MB signals is synchronous to the MB48 clock In order to use the Main Bus interface to communicate with USB or PCle you must use the MB48 clock This clock runs at a fixed 48MHz DN9002K10PCIEST User Guide www dinigroup com 111 HARDWARE Note that as well as the 36 signals there are also 16 signals in the Selectmap_D 15 0 that connect to all FPGAs that could be used for user data Dini Group does not directly support using these signals If you chose to use these signals note that the FPGA design can interfere with the programming of FPGAs You would have to keep the outputs on these signals tri stated until all FPGA configurations are complete 18 1 3 Timing As described above the MB signals are typically run synchronous to the 483 MHz MB48 bus This is the highest speed that the MB signals are guaranteed to run using a system synchronous clocking method You may be able to achieve perfor
21. tap settings of the IODELAY elements in the DQ IO buffers on the DDR2 interface lower bytes DDR2TAPCNT1 The current tap settings of the IODELAY elements in the DQ IO buffers on the DDR2 interface upper bytes This range of addresses is reserved for manufacturing tests Daughtercards SODIMM_SEL This does nothing on the DN9002K10PCIE8T FAN_TACH The current input value of the fan tachometer 0 or 1 IS_LX_330 Ox1 if the FPGA is LX330 0 0 is it is not SODIMM_RANK Data tead from the SODIMM interface SODIMM_COL SODIMM ROW SODIMM BANK SODIMM CAS COUNTER COUNTER COUNTER COUNTER RCLK COUNTER DN9002K10PCIES8T User Guide www dinigroup com Contains contents of GO counter 4 Contains contents of G1 counter Contains contents of G2 counter Contains contents of CLK48 counter LVDS source synchronous clock counters LVDS design only 155 THE REFERENCE DESIGN 0x08000033 0 0800003 0 08000040 0 08000043 0x08000044 0x08000045 0x08000046 0x08000047 0 0800004 0 0800004 0x0800004D 0x0800004E 0x0800004F 0 0800007 0 0800007 0 0 000000 0x0B0003FF 0x0C000X XO 0x0C000X X4 0x0C000X X8 0x0C000X XC COUNTER Clock counters for in backwards order DDR2 clock EXTCLKO EXTCLK1 SMACLK CLK_FBE CLK_FBB CLK125_ETH CLKP CLK_TPp DDR2TESTTAPCNT Reserved for manufacturing tests DDR2
22. LED_OE LED_OUT DDR2SIZE SODIMM2 HIADDRSIZE SODIMM2 SODIMM2 RANK SODIMM2 COL SODIMM2 ROW SODIMM2 BANK SODIMM2 CAS VRP ALL VRN ALL BLOCKRAM BUS XX OUT BUS XX OE BUS XX IN BUS XX Name REG DEFAULT Controls LED output enables Controls LED output values Controls address mapping order on second DIMM interface FGPA C only Number of unique addresses in HIADDR for second DIMM interface FPGA C only data retrieved from the SODIMM in socket 2 FPGA C only Contains input signals on the pins Contains input values on the pins The contents of an internal FPGA block RAM XX can be 0 21 hex Output status of IOs on bus XX XX can be 0 21 hex OE status of IOs XX can be 0 21 hex The input values A unique name of the bus schematic OxDEAD5566 Any undefined register 5 interconnect Single The single ended interconnect test tests the DC connectivity of FPGA to FPGA interconnect and the signals Presented on the MainBus are registers allowing the interface to control the output value output enable and input value of each FPGA to FPGA interconnect pin Each pin on the FPGAs is pulled high This allows a test program to find single stuck at faults open faults and stuck together faults DN9002K10PCIE8T User Guide www dinigroup com 156 THE REFERENCE DESIGN 5 1 Using the Design The design can be controller over the MainBus The register banks co
23. be toggled by writing to config register OxDF23 When Source of G2 is set to 2 then the source of the G2 clock network becomes FPGA A using the signal DN9002K10PCIE8T User Guide www dinigroup com 54 HARDWARE CLOCK FREQUENCY The MCU will adjust the clock synthesizer producing clock lt clockname gt lt clockname gt to the frequency lt number gt lt number gt MHz Figure 20 Main txt Commands An example main txt file is given below VERBOSE LEVEL O This will prevent the MCU output over RS232 to speed up configuration FPGA A a bit this will load the configuration a bit into FPGA A CLOCK FREQUENCY GO 300MHz MAIN BUS 0x08000000 0x00000001 Writes to a register in FPGA A Even if you are not planning to configure your Virtex 5 FPGAs using a CompactFlash card you may want to leave a CompactFlash card in the socket to automatically program your global clock Clocks may also be programmed using the provided USB application or over the PCIe bus 3 4 2 Hardware The Compact Flash interface is hot swappable An activity LED DS147 located next to the Compact Flash slot indicates activity on this interface Please contact support dinigroup com if you find an incompatible card so that we can add software support for it Also the board only accepts CompactFlash cards formatted in the FAT file system Most new compact flash cards come pre formatted wit
24. or EXTI network into zero delay mode Contact support dinigroup com The disadvantage of this method is that the or network must be used and that the zero delay configuration has to be calculated for you by us The advantage is that the entire system be operated on a single clock domain Zero delay on the DN9002K10PCIEST is allowed by enabling PLL devices zero delay buffers connected to the pins of each daughtercard header allow for a very wide range of clock frequencies sourced from the daughtercatd the PLL bandwidth of these buffers must be manually set This can be done via USB PCIe or Compact Flash The PLL can also be bypassed allowing a global system synchronous clock to be used without configuring this PLL To use this method the user will have to experimentally find the proper clock phase to use on the IO of the daughter card Source Synchronous The daughtercard drives a clock into the CC pins of the daughtercard connector This clock is used to latch IOs This method should be used for frequencies exceeding 150 MHz because the phase tolerance of the Virtex 5 FPGA and the clock buffer devices on the DN9002K10PCIE8T EXTO and signals will prevent a reliable system synchronous design at high speeds 25 2 5 Power and Reset The 3 3V 5 0 and 12V power rails are supplied to the Daughter card headers Each pin on the MEG Array connector is rated to tolerate 1A of current without t
25. vendor request or Bulk transfer All other types of USB transactions not supported or documented with the DN9002K10PCIE8T In general Bulk transfers are used for high bandwidth data and vendor requests are used for all other control functions Bulk Transfer Functions Configure FPGA SelectMap Readback FPGA SelectMap MainBus read MainBus wtite Vendor requests can contain short 512Byte messages in either direction and cause the MCU to execute code In response to most vendor requests the MCU will modify or read values in the Configuration memoty space see next section Since vendor requests can contain only a limited amount of data USB Bulk transfers used to send configuration data to the DN9002K10PCIE8T MCU is too slow to process USB 2 0 data at full speed and so the bulk transfer data is sent to external pins on the Cypress MCU see Cypress datasheet and to the configuration FPGA next section Currently this data 1s only used to configure FPGAs and so the data is sent to the SelectMap pins of the Virtex 5 FPGAs To begin communication with the DN9002K10PCIEST the USB Controller program creates a USB connection object in the host operating system by opening Vendor ID 0x1234 product ID 0x1234 For the purposes of updating the firmware the DN9002K10PCIE8T can come up in EPROM mode where it loads a program capable of connecting over USB to a host downloading firmware and writing it to the MC
26. 5R 5232 FPGA RX 13 pig ET RS232 FPGA RXD R5232 17 85232 MCU TSM 136 01 T DV R2OUT pain 83232 MCU RxD SMS HU PRINT LOUT Gn 3 4 SILK Gor RS232 551 SWOUT 15 80 ax nata yw SIKRS232SHONE_ 24 25V 2 m C406 O uF RS232 1 23 1 2 R5232 CPUMPI EN E VCC 2 5 3 uen 5227 CPUNP2 4 EI 5 6 x SILK C434 RS232 CPUMP2 5 7 e 1 c2 93232 VPUMP 9 10 RS232 Vs M CONFIG ND v TSM 136 01 T DV C278 C441 C442 25V TSOP65P638X110 24N 0 0 tuF 0 1uF R338 47 522 FPGA TX EE R337 47 5232 MCU The configuration section RS232 terminal header labeled MCU above be connected to a computer serial port using the settings 19200 Baud No flow control One stop bits No parity DN9002K10PCIES8T User Guide Www dinigroup com 50 HARDWARE The syntax and content of the output messages changes are not given because they change rapidly This interface is not at all fun to use and is intended mostly for Dini Group to debug hardware or software failures 3 2 FPGA Configuration Normally configuration of the Virtex 5 FPGA occurs over the Virtex 5 SelectMap interface The only configuration method possible on the DN9002K10PCIEST that does not use this interface is JTAG For a description of the SelectMap interface see
27. DCA2P DCA2P 0 2 1 3 4 5 DCA2P20_C DCA2P21 DN9002K10PCIE8T User Guide 266 234 14 118 144 115 24 177 140 85 24 83 176 82 101 45 488 578 556 366 555 629 450 425 674 396 433 783 490 388 328 361 539 783 751 741 376 811 340 253 579 448 661 778 729 966 953 490 576 551 368 553 637 451 409 668 389 434 773 481 388 332 365 534 776 749 739 375 0 0 0 4 0 1 www dinigroup com 139 HARDWARE DCA2P22 DCA2P23 DCA2P24 DCA2P25 DCA2P26 DCA2P27 DCA2P28 DCA2P29 DCA2P30 DCA2P31 DCBON DCBON DCBO DCBO DCBON DCBON DCBO DCBO DCBON DCBON DCBO DCBO DCBON DCBON DCBO DCBO DCBON DCBON DCBO DCBO DCBON DCBON DCBO DCBO DCBON DCBON DCBO DCBO DCBON DCBON DCBO 00 01 02 03 04 05_V 06 07 08 V 09 V 0 SAG RO 4 Cow 2 21 22 23 24 25 26 27 28 29 30 DCBOPO1 DCBOP02 DCBOPO03 DCBOP04 DCBOP05 DCBOP06 DCB0P07 DCBOP08 DCBOP09 DCBOP DCBOP DCBOP DCBOP DCBOP DCBOP DCBOP DCBOP DCBOP DCBOP 0 1 2 326 4 5 7 8 9 DCBOP20_C DCBOP21 DCBOP22 DCBOP23 DCBOP24 DCBOP25 DCBOP26 DN9002K10PCIE8T User Guide 804 337 255 573 456 670 773 719 962 958 269 275 234 211 145 257 294 171 113 230 261 113 143 191 132 138 76 199 129 116 63 246 270 93 37 337 267 226 418 67 137 267
28. and won t tell you why Whatever you do if you love your FPGAs do not disable the CRC Check option in bitgen They should have called this option you want your FPGAs to zot catch fire 15LED Interface This section lists all of the LEDs More detailed explanations of the LED functions may be in the sections describing the board system that contains the LED 15 1 Configuration Section LEDs These LEDs controlled by the configuration section and give the status of the board LED Reference LED Signal Name The LED indicates the following when ON Designator Color DS16 GREEN ON 33V The board is powered on 0566 GREEN SPARTAN DONE Configuration circuit is on Firmware loaded LED Reference LED Signal Name The LED indicates the following when ON Designator Color DS17 GREEN A DONE FPGA A is configured DS20 GREEN BDONE FPGA B is configured DS21 YELLOW USB ACT There is USB activity 0565 YELLOW ACT There is CompactFlash activity 0051 YELLOW PCIe There is PCle activity DS2 RED LOS PCI Express has lost link DS3 GEEN PCle_x1 PCI Express has negotiated a 1x link DN9002K10PCIEST User Guide www dinigroup com 99 HARDWARE DS4 GREEN PCle x4 PCI Express has negotiated a 4x link DS5 GREEN PCle x8 PCI Express has negotiated an 8x link DS14 RED DIMM VOLTAGE The DIMM voltage is not 1 8V DS27 RED PCIe PERSTN The PCI Express Reset signal is asserted 15 2 User LEDs These LEDs conne
29. 26 30 MB30 AD ELECTMAP D5 27 25 6 28 30 29 AD ELEC TMAP 29 2 28 30 30 28 AD ELEC TMAP 03 31 29 30 32 30 27 AD SELECTMAP D2 33 31 32 34 30 26 AD SELECTMAP D 1 35 E 5 36 30 25 SELEC TMAP tr ect ao 55 38 30 24 AD 33 GND Loc 2 7 GND GND GNp 2 4 2 767004 2 CONN MIC TOR38 The clock or trigger signals on this connector 48 MIC E driven at a fixed 48 MHz If you need to use a logic analyzer this is the only available trigger If you use the signals SELECTMAP D 7 0 for any purpose other than configuration care must be taken to prevent the FPGAs from driving these signals before all FPGAs are configured or else risk interfering with the configuration process Some SelectMap control signals are connected to this connector but are not user accessible This connector could potentially be used for configuring daughtercards You would have to contact us for information about that possibility DN9002K10PCIE8T User Guide www dinigroup com 122 HARDWARE 22Power The power used by DN9002K10PCIEST is derived from an external 12V voltage supply The current at these voltages is supplied through the PCI Express power connector 3 NO power is taken from the PCle edge connector Therefore if installed in a PCI Express slot with no power connector the board will not power on 12V GTP Synthesizers Daughter Card Daug
30. 284 241 212 144 254 286 170 106 216 273 115 136 187 133 141 78 204 131 121 67 251 272 90 39 344 269 www dinigroup com 140 HARDWARE DCBOP27 DCBOP28 DCBOP29 DCBOP30 DCB1N00 DCB1NO01 DCB1N02 DCB1N03 DCB1N04 DCB1N05 DCB1N06 DCB1NO07_V DCB1N08 DCB1 DCB1N DCB1N DCB1 DCB1 DCB1N DCB1N DCB1 DCB1 DCB1N DCB1N DCB1N20 DCB1N21 DCB1N22 C DCB1N23 DCB1N24 DCB1N25 DCB1N26 DCB1N27 DCB1N28 DCB1N29 DCB1N30 DCB1P00 DCB1P01 DCB1P02 DCB1P05 DCB1P04 DCB1P05 DCB1P06 DCB1P07 DCB1P08 DCB1P09 DCB1P10 DCBIP11 DCB1P12 DCB1P15 DCB1P14 5 6 7 8 Pes lt lt AUN lt pe 5456054 o DCB1P DCB1P DCB1P DCB1P DCB1P19 DCB1P20 DCB1P21 DCB1P22 C DCB1P25 DCB1P24 DCB1P25 DCB1P26 DCB1P27 DCB1P28 DCB1P29 DCB1P30 DCB2N00 DCB2N01 DN9002K10PCIE8T User Guide 227 410 70 143 92 216 184 2 246 96 75 328 106 114 162 307 149 331 85 141 369 303 95 152 397 102 45 475 111 100 75 228 93 148 105 94 221 186 219 248 100 76 329 114 115 154 290 147 327 69 139 377 301 90 148 388 110 43 486 113 102 74 234 90 151 105 371 166 0 5 www dinigroup com 141 HARDWARE DCB2N02 399 Ox6 DCB2N03 696 0 2 DCB2N04 151 Ox9 DCB2N05_V 522 Ox5 DCB2N06 585 0 4 DCB2N07 319 Ox7 DCB2N08 V 112 OxA DCB2N09_V 405 Ox6 DCB2N10 463 Ox5 DCB2N11 298 Ox8 DCB2N12 V 212 Ox9 D
31. 5 0Gb s Figure 21 PCI Express block diagram Normally a user will place his PCI Express endpoint IP in FPGA and his high density logic in FPGA A A large amount of interconnect is provided between FPGA A and Q to easily keep up with a full speed 8 lane PCI Express endpoint The user can provide his own PCI Express IP he can use the Xilinx PCI Express endpoint hard macto or he can use the free provided full function PCI Express endpoint now with DMATV core 7 1 Host Interface Electrical The PCI Express signals from the host computer are connected directly to the LXT RocketlO IOs As required by PCI express standard the transmit signals from the FPGA are passed through ac coupling capacitors For fun the receive signals from the host are also passed through ac coupling capacitors DN9002K10PCIEST User Guide www dinigroup com 79 HARDWARE The RocketIO requires a reference clock frequency to operate On boards with an LX50T this clock is provided on the MGTREFCLKP_112 pin at 100 MHz As required by Xilinx this frequency is identical to the frequency supplied by the host connector on the PCI Express REFCLK signal On boards with an FX70T the clock frequency is instead 250 MHz exactly 2 5 times the frequency of the REFCLK signal provided by the host connector When creating a core using the Xilinx PCI Express core generator you must tell the wizard program the frequency of this clock and to which pins it connects
32. Cable USB 6 MHz usb hs 2 7 1 Moving On Congratulations You have just programmed the DN9002K10PCIE8T and learned all of the features that you have to know to start your emulation project If you ate new to Xilinx FPGA you might want start by compiling the reference design and adding code to the reference design until you ate comfortable with the design flow You should also use the provided UCF constraint file as a starting point for your UCF file DN9002K10PCIEST User Guide www dinigroup com 24 CONTROLLER SOFTWARE Chapter 3 Controller Software The DN9002K10PCIE8T be hosted from USB or PCIe As an example to hosting using these interfaces the Dini Group provides some controller software that allows configuring FPGAs and changing the board settings For more complex host behavior such as interactively transferring data to and from the board from the host computer you may have to develop yout own host software either USB or PCIe At the end of this chapter there is a programmer s guide to help you interface to the DN9002K10PCIEST This along with the source code of the example software should be able to get you communicating with the DN9002K10PCIEST The software included with the DN9002K10PCIEST is USB Controller A Windows XP only GUI application capable of configuting FPGAs sending data to the user FPGA core via USB changing boatd settings and running hardware tests AETest usb A cross platform Windows
33. DCAOP09 DCAOP DCAOP DCAOP DCAOP DCAOP DCAOP DCAOP DCAOP DCAOP DCAOP19 C3 1 DCAOP20_C DCAOP21 DCAOP22 DCAOP23 DCAOP24 DCAOP25 DCAOP26 DCAOP27 DCAOP28 DCAOP29 DCAOP30 DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DCA DN9002K10PCIE8T User Guide N01 N02 03 04 N05 N06 San lt N Z lt gt BON lt lt Z e CY CY 20 N N ie S N25 N26 27 28 N29 N30 31 01 P02 P03 P04 P05 P06 P07 P08 P09 P10 P11 P12 P13 P14 P15_C 241 259 124 164 185 377 122 112 151 296 87 107 41 322 137 102 299 122 256 161 191 259 206 259 283 152 175 163 122 34 21 303 123 48 100 166 274 231 12 117 150 111 24 179 140 83 23 81 179 80 99 45 205 255 292 148 181 152 119 36 14 304 120 42 93 169 0 8 0 8 0 9 www dinigroup com 138 HARDWARE DCAIP16 DCAIPI17 DCAIP18 DCAIP19 DCA1P20 DCAIP21 DCA1P22_C DCA1P23 DCA1P24 DCA1P25 DCA1P26 DCA1P27 ad aA lt gt pee N DCA2P01 DCA2P02 DCA2P03 DCA2P04 DCA2P05 DCA2P06 DCA2P07 DCA2P08 DCA2P09 DCA2P DCA2P DCA2P DCA2P DCA2P DCA2P DCA2P DCA2P
34. DOS Linux and Solaris command line application capable of configuring FPGAs sending data via USB and changing board settings AETest A cross platform Windows XP Windows98 DOS Linux and Solaris command line program capable of configuring FPGAs and sending data to and from user FPGA cores via PCle These programs and the source code for them can be found on the user CD D PCle_Software_Applications Aetest D USB_Software_Applications AETEST_USB D NUSB Software Applications USBConttollerN Precompiled windows XP binaries for USB Controller and AETest usb and AETest are provided on the user CD as a Microsoft Visual Studio 6 project Visual Studio 6 or later is required to compile these programs All three programs use a driver provided by the Dini Group The PCIe drivers can be found at PCI Software Applications Aetest wdmdrv PCI Software Applications Aetest linuxdtv 2 6 PCI Software Applications VAetestNsolaris driver USB dtivet can be found at USB Software Applications Mdriver DN9002K10PCIEST User Guide www dinigroup com 25 CONTROLLER SOFTWARE 1 USB Controller The USB Controller program is intended to Verify Configuration Status Configure FPGAs over USB Configure FPGAs via CompactFlash card Clear FPGAs Reset FPGAs Set Global clocks frequency Update firmware for MCU and Spartan The following function interface with the Dini Group reference design Read Write to FPGAs see t
35. EPROM Contact us about zero bus latency type parts DNSODM200 RLDRAM Reduced latency DRAM Micron 64 bit wide compatible with the 200 pin SODIMM sockets Small EPROM DNSODM200 MICTOR DNSODM200 QUADMIC Provides 2 ot 4 Mictor 38 connectors Compatible with the DDR2 SODIMM sockets User LEDs Small EPROM DNSODM200_DDR1 DDR1 memory module compatible with the 200 pin SODIMM sockets Comes with 512MB standard Allows use of standard PC2700 modules up to 1GB 175MHz performance DNSODM200_SDR SDR memory module compatible with 200 pin SODIMM sockets Accepts PC133 modules up to 512MB User is required to install a Jumper Comes with 256MB standard 75MHz performance DNSODM200_FLASH Spansion S29WS064 memory x2 each is 4Mx16 bit flash DN9002K10PCIES8T User Guide www dinigroup com 166 ORDERING INFORMATION 16Mb SRAM memoty 512k x 32 Compatible with DDR2 SODIMM sockets 66MHz performance read burst Other SODIMMSs include access to the following interfaces USB 3 3V IO FPGA interconnect 3 1 2 Daughtercards Dini Group daughtercards connect to the MEG Array connector 400 pin using the standard Dini Group interface description DNMEG PCIE 8 lane PCIe express PHY card Host or downstream mode DDR2 memory module Virtex 4 FPGA LX40 LX160 DNMEG_AD_DA High speed Analog Digital daughtercard Virtex 4 FPGA DDR2 memory module 250Msps 12 bit ADC 60dB SNR 10 bits 200 kHz 75MHz DNMEG V5T two versions Xi
36. FPGA interface module does not have this interface exposed at the moment Ask us how to do this 7 3 9 Host Interface Software Example software capable of configuring FPGAs communicating over MainBus and DMA transfers to FPGA A 15 provided AETest You may wish to copy this code and use it as a starting point To communicate with the DN9002K10PCIEST you will need to find the device on the PCIe Bus with VendorID 17DF and DeviceID 1900 The device will register itself with the operating system as Dini Group ASIC Emulator with Virtex 5 PCI Express OS dependant Note that many Dini Group products use this vendor and device ID so differentiating between boards requires you to read at a minimum the board type register and the board serial number register DN9002K10PCIE8T User Guide www dinigroup com 86 HARDWARE 7 3 9 1 Driver The source code for the DN9002K10PCIEST s PCIe driver is provided Windows XP Vista Binaries for 32 bit windows 64 bit windows Itanium and 64 bit windows AMD Pentium provided as a binary Use the windows hardware manager to install these drivers Source is provided but shouldn t be required by most of you Linux Soutce is provided for the linux driver Compilation is probably required Provided binaries are unlikely to work Also source is only tested with the latest version of Linux and may not be compatible with older version To compile you will need the kernel source module instal
37. TENTH INCH DN9002K10PCIEST User Guide www dinigroup com 96 HARDWARE n gt 5 o One the board pin 1 is marked with a big unmistakable white circle dot On the provided cable pin one is marked with a red stripe on the cable Hot plugging this connector is acceptable and encouraged The settings required on the serial port of your computer are dependent on the UART in the FPGA Since the flow control signals on the serial cable are not connected to the FPGA you cannot use hardware handshaking The other port settings parity stop bits speed and data bits are user design dependent 12 1 1 Configuration RS232 second RS232 header P2 is for the configuration circuitry to give feedback to the user It is described in the section Configuration Section 13Temperature Sensors Each FPGA is connected to a temperature monitor This monitor can internally measure the temperature of the FPGA silicon die maximum recommended operating temperature of the FPGA 15 85 degrees The accuracy of the temperature sensor is about 0 to 5 degrees When the configuration circuitry measures the temperature of any FPGA rise above 80 degrees it will immediately un configure the hot FPGA and prevent it from re configuring When the temperature drops below 80 the configuration circuitry will again allow the FPGA to configure When this occurs a message will appear on the CONFIG RS232 port
38. This vendor request clears an FPGA DN9002K10PCIEST User Guide www dinigroup com 72 HARDWARE Direction is OUT Size is 0 Value represents which FPGA should be cleared 0 is FPGA A 1 is FPGA B and so on 6 2 2 VR SETUP CONFIG This vendor request must be called before sending configuration data to an FPGA It tells the DN9002K10PCIE8T which FPGA should receive the next configuration stream sent over USB It also clears that FPGA of its current configuration Direction is OUT Size is 1 In the buffer is a number representing which FPGA should be selected 0 is FPGA A 1 is FPGA B 2 is FPGA C and so on 6 2 3 VR END This vendor request de selects and FPGA so that configuration data sent will go to no FPGA and checks the configuration status of an FPGA 6 2 4 VR SET EP6TC Read buffer size The SetReadBufferSize vendor request must be used before any bulk read bulk transfer This sets the size in bytes of the data that will be requested by the bulk transfer If this vendor request is not sent before the bulk read the behavior is undefined The direction is OUT The size is 0 The value is the number of bytes required for the next bulk transfer 6 2 5 VR MEM MAPPED Configuration Registers Some of the controls the DN9002K10PCIEST do not have their own Vendor Request These functions include setting the clock frequencies In order to accomplish these tasks you must use the Configuration Regis
39. Windows and Linux driver It will save you an approximate man month of work and writing and fully testing a custom implementation of a PCI Express Endpoint Software 4 FPGA PCIE 0 Driver PCIE_IN_D 63 0 Provided PCIE JN FOF L PCIE IIl VALID 2 5 Gbs PCIE IIl CHAH 2 0 User clock Pcie IN_mror2 0 E Embedded PCIE_IN_TC 1 0 PCIE_OUT_ALMOST_FULL PCIE_OUT_D 63 0 PCIE_OUT_SOF PCIE_OUT_EOF PCIE_OUT_VALID 2 4 v I v Express FIFO MAC Xilinx BAR4 250Mhz User clock POE cer eon Block FIFO PCIE_OUT_TCJ1 0 Wrapper BARI PCIE IIl ALMOST FULL BARO DMA MASTER Controller Post Signals DN9002K10PCIEST User Guide www dinigroup com 81 E v I To Configuration FPGA FPGA Config Data Clock Settings MainBus etc Figure 23 Full function design block diagram HARDWARE Access to FPGA A is through an allocation of memory space in the BAR regions of 2 3 4 and 5 BARO is used for control of the DMA engine for MainBus accesses to all FPGAs and for board control and FPGA configuration Two DMA channels allow communication to FPGA using the full PCI Express bus bandwidth The best resource for using this endpoint both from a host software and FPGA implementation standpoint is the document provided at FPGA_Reference_Designs common PCIE_x8_Interface pcie8t_user_interface_manual pdf The
40. You can also use the LXT as an additional FPGA in the case that you are not operating in a PCI Express slot at all 7 4 2 PIPE The PIPE bitfile provides the ability to have a standard 125 MHz 16 bit PIPE interface Like the full function design you are required to use in FPGA a provided interface module This module takes care of translating from the native GTP back end into a standard PIPE interface It also takes care of external bus timing and clocking FPGA PIPE Conversion Module Standard Provided PIPE or PIPE2 Bit file provided PCI Express or 2 0 Core GTP 250 MHz Soft PCI Express 1 1 PCLK 125 Mhz Figure 25 PIPE design block diagram We can also provide 8 bit 250 MHz PIPE or PIPE that takes in an external clock These modifications ate not on the user CD but can be generated to suit your needs on request 7 4 3 Slowdown PIPE Core It can be challenging to place and route a PCI Express MAC in an FPGA which is capable of 8x operation and runs with a 125 MHz or even 250 MHz system clock The PIPE slowdown core reduces the system clock PCLK frequency from full frequency to either 2 4 or 8 times slowet DN9002K10PCIEST User Guide www dinigroup com 90 HARDWARE Your PCI Express MAC at low frequency Standard Intel PIPE PIPE2 when available J TXCLK TXDATA 7 0 PCLK RXCLK RXDATA 7 0 RESET N Choose 2 4 8 8 1 Ln_TxData0 lt 1 10b 8b decode Ln TxData1
41. but all of the logic in the device is cleared Pressing the HARD RESET button S1 located near the power connector can trigger the Power reset This reset cannot be triggered over PCIe or USB It is also triggered with one or voltages on the board fall below or above a certain threshold These thresholds are given below Voltage Min Max 1 0V A 0 94V 14V 1 0V B 0 94V 14V 1 8V 1 67V 3 8V 33V 2 7V 3 8V 5 0V 4 0V 5 6V 12V 2 5V 2 25V 2 7V DN9002K10PCIEST User Guide www dinigroup com 93 HARDWARE When the board comes out of reset the micro controller goes through an initialization process that will cause all current settings to be lost including clock settings Also the configuration circuit will act as if the board has just powered on and read from the main txt file to configure FPGAs When reset is triggered it remains triggered until 55us after all trigger conditions are removed This behavior prevents USB from behaving in such a way to permanently disable USB on the host machine Under some conditions the DN9002K10PCIEST can fail to be responsive after rapidly asserting and de asserting reset or if the board is powered off and back on very quickly This behavior is caused due to a flaw in the micro controller used for the DN9002K10PCIEST configuration circuit This flaw is believed to be mitigated by the reset circuitry on the DN9002K10PCIEST If you experience the behavior please report it
42. clock inputs the Main Bus and an 300 pin expansion header The source code for the Configuration FPGA is provided in D Source Code ConfigFPGA this project can be compiled using Xilinx ISE version 7 11 SP4 or later Your board may have been build using LX80 FF1148 or an LX40 FF 1148 for the configuration FPGA 6 5 4 Power The DN9002K10PCIEST does not draw any power from the USB connector Hot plugging the DN9002K10PCIEST is acceptable 6 6 Troubleshooting If you cannot get USB to communicate with your design over Main Bus please try using the USB Controller software with your design and using the Dini Group reference design with your software This will help determine whether the software or the hardware is causing the error 6 6 1 USB Controller Freezes The Vendor requests the DN9002K10PCIEST are blocking Only one can be completed at a time This includes vendor requests that take a very long time like Configure from CompactFlash 10 seconds During this time USB Controller a single threaded application freezes when any Vendor Request is issued All the time The only way to work around this issue is to create a separate board interaction thread DN9002K10PCIEST User Guide www dinigroup com 77 HARDWARE 6 6 2 Main Bus always returns 0x Error Codes 0xDEADDEAD Main Bus timeout The VALID signal on Main Bus was never asserted See the Main Bus section for details Your FPGA may not be configured 0x12
43. connections are lost Therefore a hot plug extender will be required for hot plug Additionally we don t know how the provided full function PCI Express endpoint now with DMA will behave or how the Xilinx PCI Express endpoint hard macro will behave 7 1 2 PCI X We assume you know the difference between and PCI Express This board is designed to burst into flames when installed in a PCIX slot 7 2 Host Interface Mechanical The form factor of the DN9002K10PCIE8T exceeds the allowable form factor for PCI Express in the vertical direction This means that you will likely have to design the case for your system around the DN9002K10PCIEST Additionally many type computer cases do not fit the DN9002K10PCIEST in the horizontal direction If you are married to your computer case and motherboard you can get one of these http www adexelec com pciexp htm PEX8LX Otherwise just get a case that fits the board 7 3 Provided Full Function PCI Express endpoint Unless you need to prototype and test PCI Express logic we recommend that you just use our provided PCI Express endpoint bit file The provided bit file contains a high speed implementation of the Xilinx PCI Express hard macto adds a high speed DMA engine FPGA initiated posting implements high speed IO between FPGAs A and Q at any frequency allows PCI Express control of board functions such as configuration and clock settings and comes with a working
44. designed to deliver the clock edge to all FPGA synchronized with the CCLK pin on the daughtercard header The daughtercard is expected to distribute clocks on it so that ICs on the DN9002K10PCIEST User Guide www dinigroup com 144 HARDWARE daughtercard receive the clock signal synchronized with the pin on the daughtercard header In this way the host and daughter boards should be able to communicate synchronously with equal large IO periods in each direction There are three methods of communicating FPGA to FPGA across the daughtercard interface Local Synchronous The daughtercard generates a clock and drives it over the GCAp n or GCBp n clock pins to the host board FPGA The daughtercard drives a synchronized clock to the logic on the daughtercard adding 0 5ns delay to account for the trace delay on the DN9002K 10PCIE8T The host FPGA will use a DCM in zero delay mode and the logic on the daughtercard should have a low clock to out and setup times or use a DCM This method has the disadvantage of only allowing the one FPGA attached to the daughtercard to use this frequency To communicate globally across the DN9002K10PCIEST the user would have to pass the data across clock domains Global Synchronous The daughter catd generates a clock and drives it over the GCCp n pins to the DN9002K10PCIEST host board The user will select the daughtercard source for either the or 1 networks as appropriate The user sets the
45. device with VendotID 0x1234 and ProductID 0x1234 The HANDLE object returned by CreateFile is suitable for use with DeviceloConttol 6 1 2 Windows Vista Testing was not complete at print time support dinigroup com 6 1 3 Linux To use USB in Linux use the provided usbdrvlinux c file provided on the user CD in AETest_usb driver Connecting to the device occurs using the driver s usb open function int handle usb_open 0x1234 0x1234 0 6 1 4 Communication The USB interfaces that the DN9002K10PCIE8T presents are separated into two types The Vendor requests and the Bulk Transfers All other types of USB transactions are not supported The vendor requests ate low bandwidth control signals used for controlling the board settings The Bulk Transfers are used for configuring and reading back FPGAs and reading and writing to the main Bus interface DN9002K10PCIEST User Guide www dinigroup com 71 HARDWARE 6 2 Vendor Requests Most of the control functions available over USB are accomplished using a vendor request Programming a USB vendor request is out of the scope of this document but you can copy the code provided in the USB Controller program The following table describes the USB interface presented to the host by the MCU micro controller Vendor Request Name ID Description VR_GET_FLASH REV OxAG Returns a revision code of the firmware VR GET FPGA INFO OxA7 VR_CONFIG Ox
46. different sets of voltage or timing requirements to be met on a single daughter card simultaneously Each Bank on the daughter card is 62 signals Other connections on the daughter card connector system include three dedicated differential clock connections for inputting global clocks from an external source power connections bank VCCO power a buffered power on reset signal 25 1 Daughter Card Physical The connectors used in the expansion system are FC MEG Array 400 pin plug 6mm part 84520 102 This connector is capable of as much as 10Gbs transmission rates using differential signaling All daughter card expansion headers on the DN9002K10PCIEST are located on the bottom side of the PWB This is done to eliminate the need for resolving board to board clearance issues assuming the daughter card uses no large components on the backside DN9002K10PCIEST User Guide www dinigroup com 132 HARDWARE The Plug of the system is located on the DN9002K10PCIEST and the receptacle is located on the expansion board This selection was made to give a greater height selection to the daughter card designer 25 1 4 Daughter Card Locations and Mounting The 400 pin daughtercard header is located on the bottom solder side near the right side of the board Each header on a Dini Group product has four standard position mountain holes The drawing below shows the location of the daughter card header and it s associa
47. each power supply 5 3 DIMM Power As described in the DDR2 Interface section provisions have been made for the use of 2 5V modules in the memory sockets of DN9002K10PCIEST To allow this a jumper point is provided for each DDR2 memory power net DN9002K10PCIE8T User Guide www dinigroup com 67 HARDWARE 1 8V The DN9002K10PCIE8T comes with a jumper installed in each of the 1 8V test points shorting the DIMM power supplies to the 1 8V power supply L8V 25V DIMM 2 TP13 B JP1 1 8V ADJ If you require power on any DIMM other than 1 8V you must remove the installed jumper and instead install a different jumper to connect the DIMM power to 2 5V If you require 3 3V on this power net no jumper point is provided and you will have to run a wire 5 4 GC Test points Each FPGA is connected to a two pin test point for debugging purposes This test point is the same as the ones used for the Power Thru hole test points Each test point is connected to a pin on the FPGA meaning it can be used as a differential clock input to the FPGA connecting a reference voltage to the N pin circular this test point can be used to input a single ended signal DN9002K10PCIEST User Guide www dinigroup com 68 HARDWARE w VIRTEX S XC5VLX330 FFG1760EGU0649 0017211 21 TP20 B Note that the signals connected to either side of these test points ar
48. endpoint that the user should use endpoint 2 Note that an endpoint is bi directional Using the driver that Dini group provides the endpoint and direction fields are stuffed within the same byte write to endpoint 2 this byte should be 0x02 To read it should be 0x08 Some people refer to these as uni directional endpoints 2 and 8 6 3 2 Performance Main Bus over USB runs at a maximum speed of 80Mbs for reads and 32Mbs for writes These numbers assume that the FPGA operates the Main Bus interface with zero wait cycles If the FPGA design has more wait cycles this speeds decreases The approximate speed of Main Bus over USB is given below as a function of Main Bus wait states 0 cycles 80Mbs read 32 write 1 cycle 76Mbs read 331 Mbs write 5 cycles 64Mbs read 29Mbs write DN9002K10PCIEST User Guide www dinigroup com 74 HARDWARE 30 cycles 32Mbsread 16 5 write 100 cycles 13Mbsread 11 write 250 cycles 6Mbs read 5Mbs write 6 4 FPGA Configuration The following procedure is used by software on the host computer to configure an FPGA over USB This procedure is followed by the USBController program and AETest usb program on the user CD 1 USB Software gets a handle to a USB device with VID 0x1234 PID 0x1234 2 USB host software sends vendor request VR SETUP CONFIG 0xB7 see Vendor Requests with 1 byte in the data buffer representing which FPGA to configure A is 0x01 B is 0x02 C is 0x03 3 The con
49. ethernet tti mode ovetview 19 1 MII The 4 bit GMII interface is the only required interface on the PHY device The EEPROM MDIO and other signals are only required if you want to put the PHY into a mode that is not default The SMI MDC signals address is set to 0000 Each Ethernet interface for FPGA D one fot P is on its own SMI interface 19 1 1 Electrical The appropriate electrical standard to use is LVDCI 25 In Gigabit mode default the interface runs at 125MHz DDR The ETH125 Signal should use the SSTL 25 DCI signaling standard DN9002K10PCIEST User Guide www dinigroup com 115 HARDWARE 19 1 2 Timing The board is designed such that when using a DCM in zero delay mode on the clock CLK125_ETH the interface will meet timing clocking all IOs on this clock Alternately you can use the to clock inputs using a BUFIO and clock CLK_ETH_TX on the same clock as the rest of your transmit signals By default the 8601 s internal clock compensation mode is enabled This causes the timing of the device to be based on a clock that is delayed 2ns from the clock on the external TX_CLK and pins This makes synchronous operation of the interface possible Length Matched 500ps FPGA Assume a DCM in system synchronous mode Worst clock to out time of Virtex 5 3 37 with DCM No phase shift Worst setup time 0 097 Worst hold time 0 21 PHY clock measure
50. exe files vb files or zip files containing other zip files as attachments as we will not receive these emails Please include the boatd s serial number in your email This will allow us to reference our records regarding your boatd Before contacting support you should complete the following 1 Follow the debugging steps in the troubleshooting sections at the end of the hardware chapter and in any applicable interface sections 2 Test the applicable interface s using the provided software and bit files to help rule out hardware failures DN9002K10PCIE8T User Guide www dinigroup com Chapter 2 Quick Start Guide The Dini Group DN9002K10PCIEST can be used and controlled using many interfaces In order to learn the use of the most fundamental interfaces of the board FPGA Configuration USB data movement etc please follow the instructions in this quick start guide The guide will also show you how to tun the board s hardware test to vetify board functionality The board has already been tested at the factory 1 Provided Materials Examine the contents of your DN9002K10PCIEST kit It should contain DN9002K10PCIE8T board Compact Flash card containing the FPGA configuration bit files required to run the hardware test USB Compact Flash card reader Cable for RS232 10 pin header to female DB9 PCI Express graphics power adapter cable PSU Starter USB cable black Daughtercard mounting hardware CD ROM con
51. in the above diagram that are underlined are connected to pins on the Virtex 5 FPGA These FPGA pins used to supply a voltage reference used as the threshold voltage for the signals on that bank The use of these pins is only necessary when using threshold standatds such as SSTL DCI is used on all FPGA IO banks connected to a daughter card header The reference resistance is 50 Ohms Each Virtex 5 bank that is connected to a header DCI in enabled 25 2 3 Global clocks The daughter pin out defines 6 clock output pins These clock outputs intended to be used a 3 differential signals LVDS Two clock signals GCA connect to clock inputs on the FPGA These clocks can be used only by the FPGA that is associated with the header The GCC signal driven from each FPGA connects to a global clock buffer and can be used by all of the FPGAs on the DN9002K10PCIES8T EXTO and EXT1 networks Since daughter cards and F share the same clock network only one of these two daughtercards can drive a global clock at one time 25 2 4 Timing and Clocking Signal from the FPGAs to the daughtercard connector are not length matched The maximum trace length on the DN9002K10PCIE8T board for these signals is 800ps Each daughtercatd has a global clock output pair DCCLKCp n This LVDS output is distributed on the DN9002K10PCIEST to all Virtex 5 FPGAs The clock buffer on the host board is
52. list of this board This manual is responsible for providing the information necessary to use these features DN9002K10PCIEST User Guide www dinigroup com 47 HARDWARE 1 1 1 Features Prototyping system with one or two Xilinx Virtex 5 FPGAs XC5VLX110 XC5VLX220 or XC5VLX330 1760 pin package 100 FPGA resources available for user application PCIe core not required for PCIe use Configuration and clocks are controlled off FPGA DDR2 and Ethernet do require FPGA cores Nearly 4M ASIC gates LSI measure with 2 LX330s FPGA to FPGA interconnect is single ended or LVDS up to 900Mbs DDR2 SODIMM slot Connects to FPGA B 64 bit data width 200MHz operation DDR2 modules PC2 3200 PC2 4200 4GB maximum density per SODIMM when available aggregate data transfer rate 32GB s Nine board level global clock networks G0 G1 G2 MB48 0 1 75 Three separate programmable synthesizers configurable via CompactFlash USB PCIe Global clocks networks distributed differentially and balanced Two single step clocks Four external differential clock inputs Iwo daughtercard and two SMA cables Flexible customization via daughter cards Two 400 Meg Array connectors One connected to FPGA A one to FPGA B 93 LVDS pairs clocks or 186 single ended Some Daughtercard signals require LX330 devices 400MHz DDR so 800Mbs on all signal pairs Supported Voltages 1 5V 1 8V 2 5V 3 3V Power
53. logic 3 4 1 Main txt The main txt interface is the primary method you will use to control settings on the DN9002K10PCIEST From this interface you Configure FPGAs Set clock frequencies Write to MainBus Other settings on the DN9002K10PCIEST can also be controlled via the main txt file by accessing the configuration registers using the MEMORY MAPPED command To use the main txt interface create a file called main txt on the root directory of the Compact Flash card Plug the card into the DN9002K10PCIE8T The DN9002K10PCIEST will execute commands contained within this file when the board powers on when the Hard Reset button is pressed or when instructed to do so by the USB interface vendor request main txt file contains a list of commands separated by newline characters A list of valid main txt commands is given below DN9002K10PCIEST User Guide www dinigroup com 52 HARDWARE comment FPGA A filename FPGA B filename FPGA Q filename CLOCK FREQUENCY 0 number MHz CLOCK FREQUENCY 1 number MHz CLOCK FREQUENCY 2 number MHz SOURCE 0 2 SOURCE G1 2 SOURCE G2 2 SANITY CHECK yn VERBOSE LEVEL level MEMORY MAPPED 0x lt SHORTADDR gt 0x lt BYTE gt MAIN BUS 0x lt WORDADDR gt 0x lt WORDDATA gt lt comment gt can be any string of characters except for newline lt filename gt can be the name of a file on the root directory of the CompactFlas
54. may automatically switch device when the current device is not valid Exit Closes the USBController application 1 2 2 Edit Menu The Edit Menu performs the basic edit commands on the command log in the bottom half of the USBController window Copy delete and select all 1 2 3 FPGA Configuration Menu The FPGA Configuration Menu has the following options Refresh Window this menu option is equivalent to hitting the Refresh button in the main window It queries the board and updates the graphic for visual feedback Configure Via USB individual this menu option allows you to configure an FPGA It is equivalent to selecting an FPGA by clicking on it and selecting Configure except that this menu option will display a dialog asking which FPGA to configure Before any FPGA is configured in USB Controller a sanity check is performed This reads the header out of the binary bit file and determines whether the bit file is compatible with the FPGA installed on the DN9002K10PCIEST It will prevent configuration if the sanity check is not passed This check can be disabled from the Settings Info menu Configure via USB using file this command allows the user to configure more than one FPGA over USB ata time To use this option you must create a setup file that contains information on which FPGA s should be configured and what bitfiles should be used for each DN9002K10PCIEST User Guide www dinigroup com 29
55. offsets from the base address of any given FPGA Some registers are not valid for all FPGAs Some addresses are not valid for all of the Dini Group s reference designs Main Test does not have LVDS registers and LVDS test does not have DDR2 registers Some of the address bits are decoded as Don t care bits Therefore accesses to undefined addresses may alter stuff DN9002K10PCIE8T User Guide www dinigroup com 154 THE REFERENCE DESIGN Addtess Range 0x00000000 0x07FFFFFF 0x08000001 0x08000002 0x08000003 0x08000004 0x08000005 0x08000006 0x08000007 0x08000008 0x0800000A 0x080000011 0 080000012 0 080000013 0 080000014 0 08000001 0 08000001 0x08000001D 0x08000001E 0x08000001F 0x08000021 0x08000022 0x08000023 0x08000024 0x08000025 0x08000032 Register Register Name Contents DDR2 The data contained in the DDR2 SODIMM memory DDR2HIADDR The upper bits of DDR2 address MainBus memory space is smaller than most DDRZ SODIMMs IDCODE 0x05000142 DDR2HIADDRSIZE The numbet of valid addresses in DDR2HIADDR INTERCONTYPE An ID code used to identify which design is loaded 0x34561111 Interconnect Single 0x34562222 Interconnect LVDS 0x34563333 Interconnect LVDS reversed 0x34560000 Any Other Design PCle Ethernet etc DDR2SIZE A code to control how DDR2 memory is coded into MainBus memory RWREG Read Write Scratch Register for testing DDR2TAPCNTO The current
56. ovet header Built in FPGA configuration Compact Flash PCIe USB JTAG Configuration Readback supported One tri mode Ethernet interface FPGA B RS232 Logic Analyzer LEDs Support DN9002K10PCIEST User Guide www dinigroup com 48 HARDWARE 2 Virtex 5 The DN9002K10PCIEST allows use of each of the new features of the Virtex 5 FPGA well as exercises all of the external interfaces on the DN9002K10PCIEST the included reference design also exercises all of the new Compared to Virtex 4 Virtex 5 features listed below Greater speed logic and internal routing speed Built in PLLs Example PLL usage found in the DDR2 reference design 1 25 Gbs maximum IO speed LVDS design uses high speed IO 900Mbs currently ODELAY output signal delay elements LVDS design dynamically adjusts IDELAY to account for interfaces on the DN9002K10PCIEST where signals are not externally length matched FPGA interconnect 6 input lookup tables Larger total density parts in terms of total LUT gates More flexible IO 3 Configuration Section Many functions on the DN9002K10PCIEST are done by circuitry on the DN9002K10PCIEST external to the FPGA Collectively these circuits are referred to in this document as the Configuration Section The configuration section takes care of CompactFlash interface USB interface PCIe interface Main Bus interface master Temperature sensing Over under voltage sensing Clock frequency
57. program an FPGA mcs file is used to program a SPI flash You can use the Xilinx program Impact to generate an mcs file from a bit file The SPI flash can also be updated using USB Controller When using this method a hex file is required To generate an mcs file from a bit file in Impact select generate prom file and open the provided bit file It will ask what the target device is and it is an SPI Flash of type A T42DB642D Then double click generate To generate an hex file from an mcs file Use the Xilinx program promgen promgen w p hex r mcsfilename o outputfilename DN9002K10PCIEST User Guide www dinigroup com 43 5 5 2 Using USBController You can either generate hex file from bit file or contact support dinigroup com for new hex file Please plug in USB cable and turn the board on 1 Open USBController ini and add service_mode 1 Save and close the USBContiller ini file 2 Lauch USBController exe the Service menu should be selectable 3 Select Service gt ProgramV5TProm select hex file 4 he status bar will be on the bottom of the window The process takes about 1 2 minutes Please recycle power the board 5 5 3 Using AETest USB You can either generate hex file from bit file or contact support dinigroup com for new hex file Please plug in USB cable and turn the board on 1 Runaeusb wdm exe aeusb linux exe Select option 3 FPGA Configuration Menu 2 Selec
58. sequence required to configure FPGAs over PCI Express is given in the Configuration Section 7 3 9 5 Direct PCle to FPGA DMA Detail about the software required by the host of the DN9002K10PCIEST be found in D FPGA Reference Designs vcommon PCIE x8 Interface Vpcie8t user interface manual pdf This document should be used to design software to access the user design in FPGA A DMA in particular requires accessing the LX50T registers BARO to setup each transaction DN9002K10PCIEST User Guide www dinigroup com 88 HARDWARE Using the device driver provided use the dma_scatter_gather_read and dma_scatter_gather_write functions Performance has been characterized using DN9002K10PCIEST reference design on Windows a MSI 56728 motherboard using the AETest application speeds Read DN9002K10PCIE8T to software I have not yet performed this test Write software to DN9002K10PCIES8T I have not yet performed this test 7 3 9 6 Direct PCle to FPGA A Target access If DMA 15 not required accessing FPGA from the host software is super simple Simply read of write to an address in BAR 1 2 3 4 or 5 In Linux this can be performed by mapping a page of memory in a user mode program to the physical address of a DN9002K10PCIEST bar In Windows driver an IOCTL code is provided that will read and write individual bytes to the DN9002K10PCIEST bar address range a block or memory 7 3 9
59. shot test This is a feature of the windows program USB Controller exe Turn on the board and connect it to a windows computer over USB From the settings info menu select one shot test Enter in one of the text boxes the path to your user CD where the bit files are kept Unselect DDR from the test options so that only interconnect is tested 2 1 3 Testing DDR2 Interfaces Tutn on the board and connect it to a windows machine To test the DDR2 interface s configure an FPGA which has a DDR2 interface with the Main reference design Install DDR2 SODIMM into the socket of the FPGA In USB Controller click the enable USB communication button Then set the global clock networks to the following frequencies 100MHz G1 250MHz G2 200MHz DN9002K10PCIEST User Guide www dinigroup com 152 THE REFERENCE DESIGN The frequency of network G1 determines the DDR2 frequency of operation From the settings info menu select Test DDR In the dialog box select the FPGA which is configured The test will report PASS or FAIL 2 1 4 Testing USB USB can be tested by running the DDR2 test or by configuring FPGAs over USB 2 1 5 Testing Ethernet This test can be performed by the user however bit files are not provided If you suspect a hardware failure you will have to contact technical support 2 1 6 Testing Daughtercard Connectors This test requires a test fixture and cannot be performed by the user
60. standard DDR2 Control signals Address BA SH RASH CASH WEZ should be driven by the SSTL18 I DCI standatd The following signals are exceptions to this requirement On four of the DIMM interfaces external termination resistots ate provided The signals with external termination ate listed below DIMMB A00 DIMMB A01 DIMMB A02 DIMMB A03 DIMMB A04 DIMMB A05 DIMMB A06 DIMMB A07 DIMMB A08 DIMMB A09 DIMMB A10 DIMMB A11 DIMMB A12 DIMMB A13 DIMMB A14 DIMMB A15 DIMMB_CAS DIMMB_CS 0 DIMMB ODTO For signals in this list use the SSTL18 drive standard 16 3 2 Serial Interface The SDA and SCL interfaces are connected to 2 5V LVCMOS buffers External pull ups are provided on these signals The address of all DIMMs the DN9002K10PCIEST is set to Zero 16 3 3 Timing The length matching of the DDR2 interface signals includes all signals except for DIMM_SCL and DIMM_SDA signals Due to the source synchronous clocking techniques used by the DDR2 interface the delay from FPGA to DIMM should not be needed but is provided here anyway DIMMB Length 90mm Delay 510ps DN9002K10PCIEST User Guide www dinigroup com 106 HARDWARE The trace impedance to each of the connectors is controlled to 50 ohms All signals in the interface are ground referenced Note that this is contradictory to the recommendations of the DDR2 SODIMM specification To increase the setup time available for control signals modules may
61. test on selected FPGAs e Read FPGA Clock Frequencies This menu option measures and reads back the frequencies of the eight global clock networks and displays them on the message log Read DDR 12 Data This menu option readback DDR2 information 1 2 5 Main Bus The way that user FPGA designs can communicate over USB is the Main Bus interface The Reference design menu uses the main bus to read and write registers in the reference design to control the board tests These tests can be done by the using these menu options without the user having to understand the Main Bus interface or the main bus memory space and it s mapping to the reference design The Main Bus menu allows direct control of the Main Bus This can be useful if you using your own FPGA core that implements the main bus e Write and Read DWORD this displays a dialog box for writing and reading to the Main Bus address space It includes some debugging features All main bus transactions are of length 4 bytes DWORD DN9002K10PCIEST User Guide www dinigroup com 30 CONTROLLER SOFTWARE Test Address Space This writes and reads random data to the address range specified in a dialog box and prints and error message when the read and write do not match Read Address Space to file this reads data from the main bus at the address specified and writes the data to a binary file specified Data on the main bus is in little endian order The address after ea
62. than 4V According to the Virtex 5 datasheets the maximum applied voltage to any IO signals on the FPGA is VCCO This means you should not try to over drive IOs in an FPGA interface above the interface voltage specified in this manual 2 2 Other Some parts of the board are physically fragile Take extra care when handling the board to avoid touching the daughtercard connectors Leave the covers on the daughtercard connectors whenever they not in use Use mounting hardware to secure daughtercards 2 3 Other warnings The following unknown prohibitions apply DN9002K10PCIES8T User Guide www dinigroup com 10 CONTROLLER SOFTWARE 3 Pre Power on Instructions The image below represents your DN9002K10PCIE8T You will need to know the location of the following parts referenced in this chapter M PCI Express Graphics Power n E Li p 1022220 Figure 3 DN9002K10PCTEST stuff you need to know about to get started The FPGAs on the board are names FPGA FPGA as shown in the above photo The Q is Virtex 5 LX50T To begin working with the DN9002K10PCIEST follow the steps below 3 1 Install Memory The DN9002K10PCIE8T comes packaged without memory installed If you want the Dini Group reference design to test your memory interfaces you must install memory modules in the SODIMM slot on the board The reference design supports DDR2 SODIMM modules in any densities up to 4 GB
63. the RD signal Sometime after this within 256 clock cycles the FPGA should assert DONE for one clock cycle On this cycle the master Spartan will register the data on the AD bus and that will be the read data If DONE is not asserted then a timeout will be recorded and the transaction cancelled is a write transaction DN9002K10PCIEST User Guide www dinigroup com 113 HARDWARE 48 USB_CLK SYS CLK RD Spat re MB WR Spartan 33 DONE es ws wenn ns REESE MBBS ADB1 0 Bi MB 312 ALE Spartan MB B2 0 to 200 Cycles When the Spartan asserts the WR signal the FPGA should register the data on the AD bus Sometime after this the FPGA should assert the DONE signal This will allow the Spartan to begin more transactions The FPGA may delay this for up to 256 clock cycles before a timeout is recorded and the transaction is cancelled Main bus can be controlled from the USB Controller program Read and write single addresses to from files It can also be written from the main txt configuration method The main txt syntax 15 MAIN BUS Ox lt address gt Ox lt data gt Where lt address gt and lt data gt are 8 digit 32 bit hexadecimal numbers 18 3 1 Conventional Memory map By convention FPGAs on the main bus interface are assigned address ranges Assigning address ranges is require
64. the Virtex 5 configuration guide Typically the user will supply a file generated by ISE and put it on a or supply it to software over PCI or USB and the user does not have to understand the SelectMap interface USB CompactFlash and PCIe configuration occur over the SelectMap bus The configuration section makes no modification of bit stream sent to it over PCIe or USB It only copies the data to the SelectMap interface The stream must contain all of the SelectMap commands necessary to configure and startup the FPGA These SelectMap commands created automatically by Xilinx tool bitgen part of ISE Not all of the bitstream generation options available in bitgen are compatible with the DN9002K10PCIEST Currently before configuring the using any method except JTAG the configuration section asserts the PROG signal of the FPGA to clear it For this reason the disable SelectMap option in bitgen has no effect On each FPGA the DONE signal is connected to a blue LED located next to each FPGA This signal gives a quick indication of whether each FPGA is configured or not The data signals D 7 0 are dual purpose signals and can be used as additional interconnect pins after all FPGAs have been configured Care must be taken that the FPGA design does not drive these signals until after all FPGAs have been configured The configuration section will assert the RESET si
65. www dinigroup com INTRODUCTION PClIe_Softwate_Applications Aetest Source and binaries for the provided PCIe hosted controller software Schematics Rev 01N Contains a PDF version of the board schematic Search the PDF using control F Also contains an ASCII netlist of the board USB Software ApplicationsN Contains source and binaries for the provided driver USB hosted controller applications AETEST_USB USBController 3 2 Dinigroup com The most recent versions of the following documents are found on the product web page http dinigroup com DN9002k10PCIe 8T php User s Manual this document Errata USB Controller executable 3 3 Errata and Customer Notifications The Errata sheet available at www dinigroup com lists all cases where the DN9002K10PCIEST is found to have failed to meet advertised specifications or where an error in schematics or documentation is likely to cause a difficult to debug error by the user Customers are not notified when changes are made to other documents including the reference design USB Controller and User Manual These documents change or a weekly basis or faster You may always request a duplicate User CD We will also be happy to provide the latest vetsion of documents via email to customets 3 3 1 Existing Errata At the time of print the following errata exist Differential Polarity On Daughtercards Symptom Problem The signal DCA1N29 connects to pin on the FPGA and t
66. you should modify the board by removing the 4 7uF resistors shown above and replacing them with 0 ohm resistors The maximum recommended swing on the differential inputs is 3 3V DN9002K10PCIEST User Guide www dinigroup com 62 HARDWARE To connect a single ended clock source you can connect to one of the SMA connectors and leave the other unconnected The other side of the signal is properly biased 4 5 PCle Refclock A clock network driven from the FPGA Q is called REFCLK When the Dini Group PCI Express endpoint bitfile is loaded into the FPGA and the board is linked to a motherboard over PCI Express then this network will be driven with a 250MHz clock which is equal to 250 100 times the PCI Express REFCLK in frequency The clock is a differential LVDS signal which should be received on each FPGA with a differential clock input buffer with DIFF TERM set to enable 4 6 Non Global Clocks The following sections describe clocks that not considered global because they do not distribute to both FPGAs on the board These clocks may be used for specific interfaces and details on the clocking required for those interfaces are found in a different section in the hardware chapter 4 6 1 Clock TP Each FPGA is connected to a two pinned test point This test point can be used to input a differential clock from off board Each of these test points has a 100 Ohm jumper installed shorting the negative and posit
67. 0 requirements The transmitted signal is slightly higher amplitude than that allowed by the specification in order to allow mote flexible connection options cabling or adapters without compromising reliability In addition Pre emphasis in the transceivers is set to ultra which is not optimal but will improve reliability in crappy systems If you need to pass PCI Express compliance electrical test with your board please request the PCI Express compliance bit files from support They are identical in function but will pass compliance tests 7 3 7 Timing The provided module for FPGA A takes care of the external interface timing so you can probably skip this section When using the full function PCI Express endpoint a source synchronous communication technique is used between FPGA A and FPGA Q Since the FPGAs both have zero hold time inputs the optimal phase alignment between clock and data is when they perfectly in phase Therefore the clock for FPGA A PCIE_PCLK_A is driven from the IOs of FPGA Q in the exact same manner as the IOs and the clock for FPGA PCIE_PCLK_Q is driven from FPGA A in the exact same manner as the IOs On the board the data and clock lines are all phase matched DN9002K10PCIEST User Guide www dinigroup com 84 HARDWARE FPGA Q FPGAA PCIE BUFIO BUFR BUFG Zero hold time Any Clock Source 60 REF etc User Logic PCIE_OUT Figure 24 FPGA A to Q clocking diagra
68. 0 MHz 4 383 14 122 880000 MHz 4 383 14 124 416000 MHz 5 575 24 133 330000 MHz 0 26665 479 155 520000 MHz 5 575 24 156 256000 MHz 4 9765 374 159 375000 MHz 1 509 11 160 380000 MHz 7 485 24 161 130000 MHz 0 10741 199 161 132800 MHz 4 50353 1874 164 360000 MHz 3 1175 99 166 630000 MHz 0 33325 639 166 667000 20 333333 6399 167 331600 MHz 5 92961 3999 172 640000 MHz 0 2157 39 173 370000 MHz 3 11557 399 176 100000 MHz 3 1173 39 176 840000 MHz 3 8841 299 184 320000 MHz 4 671 24 195 312500 MHz 3 6249 191 311 010000 MHz 3 2961 99 4 4 Ext Clocks There two clock networks the DN9002K10PCIEST that are designed to provide clocks from an external frequency reference and EXT1 Each of these clocks is delivered synchronously to all 2 FPGAs and is suitable for synchronous communication among the FPGAs Oo dk xt dE dk Xt oXk dt db Xt xt dk dt dk HHH db dk Xt db xt HHH Q2 Q9 Q9 Ui Ui Ui Ui Q9 SNN UTE 4 4 1 EXTO This clock can be sources from either the external clock input SMAs connectors or daughtercard A By default 0 is set to be sourced from the daughtercard A The source setting be made from the USB Controller by selecting settings gt DN9002K10PCIES8T clock source To make the setting from the compact flash in the main txt file use MEMORY MAPPED command to write to the EXTO register 0xDF27 or the EXT1 register 0xDF28 The registe
69. 0xB888880808e Euer Found Device vi df 11900 name DN9882K1B8PCIEST VirtexS PCI Express 8 lane Board No SP_INTERFACE_DEVICE_DATA available for this GUID instance 1 Compiled on Sep 25 2008 at 16 62 05 press any key The initial display of AETest shows the results of its scan of the PCle bus If the driver for the DN9002K10PCIEST is not installed then the software will display a message that no device was found 3 3 Compiling AETEST PCle 3 3 1 Compiling AETest for Windows XP AETest for Windows requires visual studio 6 or later to compile Open the provided make file and uncomment the lines DESTOS WIN_WDM Run nmake 3 3 2 Cygwin Nope DN9002K10PCIES8T User Guide www dinigroup com 34 CONTROLLER SOFTWARE 4 Rolling Your Own Software Most customers who need to use USB or PCIe as a data interface to their FPGA designs write their own USB and PCIe controller programs since the USBController and AETest programs do not meet their requirements 4 1 USB The behavior of the DN9002K10PCIEST with regard to the USB interface is given in the Hardware chapter 4 2 PCle The host software requirements to interface with the DN9002K10PCIE8T over PCIe given in the Hardware chapter 5 Updating the Firmware Dini Group may release firmware bug fixes or added features to the DN9002K10PCIEST If a firmware update is released you will need to download this new code to the firmware flash of the DN9002K10PCI
70. 345678 This error code may mean the Enable USB gt FPGA Communication button in USB Controller has not be pressed the Main Bus disable register is set 0xDEAD5566 this error code is returned by the Dini Group reference design when there is a Main Bus read to a register that is not defined Default Main Bus output This code is specific to the reference design 0xDEAD1234 this sentinel value should not be returned 0xABCDABCD this error code is returned when a MainBus register corresponding to a memory is read but the memory is not implemented in the Reference Design 7 PCle interface The DN9002K10PCIE8T can be installed in a PCIe slot 16x or 8x slots are acceptable The board will work in a 1x 2x or 4x slot if you can physically manage to install them there using an adapter such as the ones available from Catalyst enterprises The board can support 2 5Gb PCI Express 1 1 compliant signaling or 2 PCI Express 2 0 compliant signaling at 5 0 Gbs PCI Express interface is provided by FPGA a Xilinx Virtex 5 LXT or FPGA For Gen 2 speeds FX70T part is required DN9002K10PCIEST User Guide www dinigroup com 78 HARDWARE Spartan not available for user FPGA B Virtex 5 Virtex 5 LX110 LX220 or LX110 LX220 or LX330 LX330 FF1760 FF1760 sasn pue uoneanBiyuo2 FPGA Q Virtex 5 LX50T or endpoint Provided 8 Lanes PCI Express 1 1 2 5Gbis PCIExpress 2
71. 350 200 200 for GO G1 and G2 respectively 4 2 3 Run Hardware Test DDR2 First hit the Enable USB gt FPGA communication button This must be done befote the program can interact with the reference design You must also have the reference design loaded and a DDR2 module installed in a memory socket connected to the FPGA using that reference design Also the clock settings must be correct Follow the procedure in the previous section to accomplish this From the FPGA Memory menu select Test DDR A box will appear and ask which FPGA should be tested B is the correct answer The log window will report whether the test passed If it fails it will print a list of addresses and data that failed If you would like to simulate a failure you can repeat this guide with the DDR2 module removed Other tests that could be performed from the USB Controller but aren t part of this DN9002K10PCIEST User Guide www dinigroup com 19 CONTROLLER SOFTWARE quick start are interconnect tests Ethernet tests and something else For information on running these tests see the Software chapter 4 3 Getting data to and from the FPGA The USB Controller program also allows you to easily configure and transfer data to and from the user design on the emulation board This data transfer occurs over the boards MainBus This interface is described in the Hardware chapter Some users may choose not to implement the MainBus inte
72. 4 device select the Spartan Firmware update file provided by Dini Group This file should be named prom flp mcs Hit Open DN9002K10PCIEST User Guide www dinigroup com 36 CONTROLLER SOFTWARE iMPACT Boundary Scan Eile Edit View Operations Output Debug Window Help zg E BBX SRI BO 29 ight click device to select ti SalBoundary Scan pe 3 SlaveS erial Ta SelectMAP malDesktop Configuration 2a Direct SPI Configuration E SystemACE xc3s1000 xc18v04 8 PROM File Formatter bypass prom flp mcs iMPACT Processes Available Operations are perations PROGRESS END End Operation Elapsed time 1 sec BATCH CMD identifyMPM BATCH CMD assignFile p 2 file C DiniWVork dn confi 2 Loading file C DiniWork dn config ConfigFPGA DNSO000k1l 0F done BATCH CMD set ittribute position 2 attr readnextdevi B 12 E Configuration Platform Cable USB 6 MHz usb hs Figure 15 Impact Window To program the prom right click on the prom and select Program from the popup menu In the options dialog that follows the options Erase before programming should be selected and Verify should be selected Press OK The programming process should take about 15 seconds over Xilinx platform cable USB DN9002K10PCIES8T User Guide www dinigroup com 37 CONTROLLER SOFTWARE Power cycle the DN9002K10PCIEST The new firmware is now load
73. 44000 MHz 7 372799 MHz 8 192000 MHz 8 867238 MHz 9 216000 MHz 9 830400 MHz 10 160000 MHz 10 245000 MHz 3 11 059200 MHz 7 11 228000 MHz 5 11 289600 MHz 3 12 288000 MHz 7 14 318181 MHz 3 14 745599 MHz 7 16 384000 MHz 4 16 934400 MHz 5 17 734475 MHz 0 17 900000 MHz 0 18 432000 MHz 7 19 200000 MHz 4 19 440000 MHz 5 19 531250 MHz 1 19 660800 MHz 4 22 118400 MHz 7 24 576000 MHz 7 26 562500 MHz 1 32 768000 MHz 4 33 330000 MHz 7 4041 72667 3157 1377 13857 1377 13857 1377 15791 15791 47487 7909 15791 2303 36307 49867 2303 631 15791 2303 2153 2303 15871 2 23221 2303 5613 3611 2303 2549 2303 383 14111 79 2516 124 74 479 74 479 74 624 624 1874 351 624 124 1790 2462 124 24 624 124 52 124 624 507 799 124 249 124 124 87 124 14 624 190485 3735 6085 2303 383 269 31249 15871 2303 2303 3909 383 605 119 124 14 11 767 624 124 124 95 14 31 ONN 1 10 1 UAW 4041 3927 3157 2755 2131 1377 1065 1377 375 281 211 225 187 107 115 273 89 157 93 53 49 47 61 67 39 47 85 35 33 29 29 31 45 33 23 31 49 49 61 19 17 45 29 29 www dinigroup com 47 HARDWARE 38 880000 MHz 5 1133 49 66 660000 MHz 7 403 19 74 175824 MHz 7 6749 363 76 800000 MHz 4 383 14 77 760000 MHz 5 575 24 98 30400
74. 7 Performance Using the provided Full function PCI Express endpoint now with DMA the following speed measurements wete taken from host to FPGA 510 MB s from FPGA to host 350 MB s Target access from host to FPGA 66 MB s Target access from FPGA to host 4 MB s Main Bus to FPGA from host 11 MB s Main Bus from FPGA to host 2 4 MB s Note 1 Using the large buffers DMA method in the driver This method eliminates driver overhead Note 2 This speed can be increased by 2x using double double word writes Note 3 This speed can be increased to the Target access speed in FIFO mode Note 4 Picasso s full name was Pablito Diego Jose Santiago Francisco de Paula Juan Nepomuceno Crispin Crispiniano de los Remedios Cipriano de la Santisima Trinidad Ruiz Blasco y Picasso Lopez 7 3 9 8 64 bit addressing 64 bit addressing has no effect on operation 7 4 Other Provided Designs for the LXT If you ate not testing PCI Express endpoint logic specifically you most likely want to use the provided full function PCI Express endpoint now with DMA design Otherwise you have the following options 7 4 1 No design You can implement your design directly within the LXT connecting directly to the Xilinx MGT In this case you will have to learn the peculiarities of the MGTs and you will have to DN9002K10PCIEST User Guide www dinigroup com 89 HARDWARE convert the output of the MGT into PIPE fairly easy
75. 7K 4 7K 4 7K The pins used to access the EPROM are given below ETH_IIC_SCL 42 ETH_IIC_SDA 41 21Mictor Connectors There are two Mictor 38 pin connectors on the board for the purpose of using a logic analyzer If you need to use a logic analyzer be sure to consider using an embedded logic analyzer instead like ChipScope or Identify These logic analyzers place and routes within your design and are more flexible than a stand alone analyzer 21 1 FPGA B Mictor J16 is a Mictor connector whose signals all connect directly to FPGA B s IOs This Mictor can be used for a logic analyzer as probe points using a Mictor breakout or to cable to another system DN9002K10PCIEST User Guide www dinigroup com 120 HARDWARE By default the signaling level on the Mictor connector is CMOS 2 5V Removing R134 and installing a zero ohm resistor on R135 can change all the IOs associated with this connector changed to 3 3V CMOS signals instead MICTOR J16 Do Not Connect SN GND 4 MICTOR B32 MICTOR B33 1 8V lt MICTOR B15 CLK 6 MICTOR 31 2S 1 8 MICTOR B14 MICTOR B30 SM MICTOR d MICTOR E29 SM MICTOR B12 i MICTOR B28 MICTOR B11 3 MICTOR B27 MICTOR B10 15 MICTOR E26 MICTOR 17 MICTOR E25 MICTOR 19 MICTOR E24 MICTOR B7 MICTOR B23 BO MICTOR B6 F MICTOR B22 B1 MICTOR MICTOR B21 MICTOR B4 27 MICTOR E20 MICTOR 29 MICTOR 19 MICTOR E2 31 MICTOR B18 MICTOR BI 33 MICTOR B17 MICTOR 0 35 MI
76. 8 Note that on the netlist these signals connect to the FPGA twice once on the DDR2 interface bank 1 8V and once on the global clock input bank 2 5V The 2 5V clock bank connections should be used as inputs and the 1 8V bank signals should be configured as outputs For input signals use the LVDSEXT standard with the DIFF_TERM attribute set to TRUE 4 6 4 SMA Clock A FPGA has a pair of SMA connector connected directly to global clock inputs AM28 28 The bank connected to these signals is 2 5V bank Allowed input standards are LVCMOS25 SSTL25 LVDS and DIFF_SSTL18 DN9002K10PCIE8T User Guide www dinigroup com 64 HARDWARE J1 CONN SMA LIGHTHORSE SASF546 P26 X1 3 2 3 J2 CONN_SMA LIGHTHORSE_SASF546 P26 X1 These connections are DC coupled meaning the user must ensure that the levels received on this input are within the limits of the Virtex 5 device to prevent damage to the part This pair of SMA connectors can also be used as outputs as single ended inputs or for non clock signals 5 Test points This section lists all of the test points the DN9002K10PCIE8T A more detailed description may be found in the section about the system that the test point is part of but all test points are listed here for reference 5 1 Power Thru hole Each power rail requiring more than 100mA on the DN9002K10PCIE8T has a dedicated test point associated with it This test point is a through hole two pin
77. AF Causes MCU to go through configuration sequence Media Card VR_CHECK_FPGA_CONFIG 0 5 Returns a string representing if the selected FPGA is configured VR_SET_EP6TC OxBB Sets the size of the bulk transfer Read buffer You must set this to a value equal to the SIZE field of the USB Bulk transfer VR_SETUP_CONFIG OxB7 This vendor request must be called to select an FPGA for configuration prior to a bulk transfer containing the configuration stream for that FPGA VR_END_CONFIG OxBD This vendor request de selects an FPGA after configuration and returns the configuration status of that FPGA DONE signal VR_MEM MAPPED OxBE This vendor request reads or writes to the address Config Read space of the MCU This vendor request can be Config Write used with the configuration register map above to accomplish any configuration task VR CLEAR FPGA 0x90 Clears the selected FPGA of configuration data VR BOARD VERSION OxB9 Returns a byte representing the type of board Each vendor request has a direction request type request and value size and buffer pointer fields The request type is always TYPE VENDOR The request field is the ID listed in the table above The value and data in the buffer pointer fields are vendor request specific The size field is the number of bytes in the buffer The details of how to implement a vendor request are outside the scope of this manual 6 2 1 VR CLEAR FPGA
78. BAR resources available are given below These cannot be changed through any settings made available to the user 0x0 Ox1ff PCI E FPGA registers rest is Configuration FPGA registers 8MB 1 32 bit BAR for User FPGA 8 MB Bar2 3 64 bit BAR for User FPGA 32MB Bar4 5 64 bit BAR for User FPGA 32MB By default prefetch is turned off on 32 bit BARs it may be on for the 64 bit bars The back end FPGA A interface is fixed at 64 bit In a 32 bit addressing machine it will appear as if BAR2 is configured as a 32 bit bar and BAR3 will not be implemented BAR4 will appear a 32 bit bar and BAR5 will not be implemented 7 3 1 BAR 0 Access The Bar 0 accesses are reserved for board settings FPGAs configuration and Bus communication User mode programs can access these registers to control the board from the PCI Host Some of the useful offsets are given below Byte Size Description 0x000 3 0 Version Contains a version code for the firmware of LXT device Read only 0x008 31 0 ID Always returns 0x4675_6C6C for full function design Read only 0x020 31 0 DMAO Lower 32 bit byte address of physical address where the DMAO Base Address descriptor chain starts This address must have the lower bytes cleared to match the DMAO Address Mask register 0x024 3 0 DMAO Upper 32 bits of Base Address 63 0 to form a 64 bit address Base Address Set to 0 if using 32 bit addressing 0 02 31 0
79. CB2N13 C 236 0 8 DCB2N14 225 Ox9 DCB2N15 276 Ox8 DCB2N16_C 226 Ox9 DCB2N17_C 345 Ox7 DCB2N18 281 Ox8 DCB2N19 245 Ox8 DCB2N20_C 163 Ox9 DCB2N21 269 0 8 DCB2N22 448 Ox6 DCB2N23 262 Ox8 DCB2N24 222 Ox9 DCB2N25 160 Ox9 DCB2N26 114 OxA DCB2N27 350 Ox7 DCB2N28 367 Ox7 DCB2N29 512 Ox5 DCB2N30 142 OxA DCB2P00 378 Ox7 DCB2P01 165 Ox9 DCB2P02 406 Ox6 DCB2P03 708 0 2 DCB2P04 155 Ox9 DCB2P05 524 Ox5 DCB2P06 574 0 4 DCB2P07 312 Ox7 DCB2P08 105 OxA DCB2P09 406 Ox6 DCB2P10 467 Ox5 DCB2P11 303 Ox8 DCB2P12 207 Ox9 DCB2P13_C 238 Ox8 DCB2P14 226 Ox9 DCB2P15 280 Ox8 DCB2P16_C 224 Ox9 DCB2P17_C 341 Ox7 DCB2P18 279 Ox8 DCB2P19 246 Ox8 DCB2P20_C 158 Ox9 DCB2P21 262 Ox8 DCB2P22 441 Ox6 DCB2P23 263 Ox8 DCB2P24 227 Ox8 DCB2P25 161 Ox9 DCB2P26 115 OxA DCB2P27 352 Ox7 DCB2P28 359 Ox7 DCB2P29 501 0 5 DCB2P30 143 OxA 25 2 1 Pin assignments The pin out of the DN9002K10PCIE8T expansion system was designed to reduce cross talk to manageable levels while operating at full speed of the Virtex 5 The ground to signal ratio of the connector is 1 1 General purpose IO is arranged in GSGS pattern to allow high speed single DN9002K10PCIEST User Guide www dinigroup com 142 HARDWARE ended or differential use On the DN9002K10PCIE8T host these signals are routed as loosely coupled differential signals meaning when used differentially they benefit from the noise resistant properties of a differential pair but when used single ended ly
80. CI express 1 1 and 1 0a using the Provided DMA enabled PCI Express core and with the Xilinx PCIe endpoint LogiCORE Additionally the LogiCORE endpoint passes the PCI SIG compliance full test The provided PCI Express DMA enabled core has not been tested at a compliance workshop The FX70T passes the PCI Express electrical compliance test for revision 2 0 EYE WIDTH 149ps TIE JITTER 28 to 28ps TOTAL JITTER BER 77 DIFF PEAK VOLTAGE 112 4 2 Environmental 4 2 1 Temperature The DN9002K10PCIEST is designed to operate within an ambient temperature range of 0 50 degrees C All components used on the DN9002K10PCIEST are guaranteed to operate within a temperature range of 0 80 degrees C measured on the device die 4 3 Export Control 4 3 1 Lead Free The DN9002K10PCIEST meets the requirements of EU Directive 2002 95 EC RoHS Specifically the DN9002K10PCIES8T contains no homogeneous materials that a contains lead Pb in excess of 0 1 weight 1000 ppm b contains mercury Hg in excess of 0 1 weight o 1000 ppm contains hexavalent chromium Cr in excess of 0 1 weight 1000 ppm d contains polybrominated biphenyls PBB or polybrominated dimethyl ethers PBDE in excess of 0 1 weight o 1000 ppm contains cadmium Cd in excess of 0 01 weight 100 ppm No exemptions ate claimed for this product 4 3 2 The USA Schedule B number based on the HTS 8471 60 7080 DN9002K10PCIEST Use
81. CTOR B16 37 DO GND GND GND 2 767004 2 CONN_MICTOR38 Hot plugging a Mictor connector is generally safe When connected to a logic analyzer signals MICTOR32 and MICTOR33 should used as trigger signals Signals connected to the Mictor are 50 ohm DCI and SSTL referenced input can be used on the Mictor interface Note the connector is mixed voltage The odd pins 13 37 follow the daughter card s BO voltage and the even pins 12 38 follow the daughter cards B1 voltage 21 2 MainBus Mictor second Mictor connector on the backside of the board is connected to the MainBus and SelectMap interfaces of the DN9002K10PCIEST DN9002K10PCIEST User Guide www dinigroup com 121 HARDWARE Most of the signals attached to the Mictor are accessible from both FPGAs on the DN9002K10PCIEST Since these signals are heavily loaded this connector is not suitable for high speed signaling CLK 48 MIC 30 MICTOR_CLK_E 30 0 1uF 1 Do Not Connect 2 15 CS amp 30 xe 2 gt 3 beum 4 SSFPGAI4 CS 30 ELEC TMAP 7 5 CLK CLK 6 8 FPGA_RD WR 30 9 7 018 10 FPGA DONE 30 11 0 312 SSFPGA M CCLK 30 ELEC TMAP 13 11 12 14 SSFPGA M PROG 30 ELEC TMAP 15 13 14 746 30_ MB35 DONE 17 15 16 18 30 MB34 RD 19 d 20 30 MB33 WR lt 21 b 2 22 30 32 ALE 23 24 30__MB31_AD ELEC TMAP 25 2
82. DMAO Control 0x030 31 0 DMAO Poll Immediate DN9002K10PCIEST User Guide www dinigroup com 82 HARDWARE 0x040 31 0 DMA1 Base Address 0 04 31 0 DMA1 Control 0x050 31 0 Poll Immediate 0x98 357 Scratch Pad Read Write space for user having fun and exercise Byte imaginations 0 208 6 0 Config Control Selects and FPGA and returns the value of these FPGA s PROG INIT and DONE signals 0x210 31 0 Config Data Sends the given configuration word to the selected FPGA 0x238 FPGA Stuffing 0x240 Main Bus ADDR 0x248 MainBus Write 0 250 MainBus Read 0x258 Config Space Write 7 3 2 BAR 1 5 Access PCI Express reads and writes in the BAR1 BAR5 memory space result in communication to FPGA A over the PCIE_IN and PCIE_OUT signals on FPGA A This should be used in conjunction with the provided PCIe interface module in FPGA See source code here D FPGA_Reference_Designs common PCIE_x8_Interface 7 3 3 DMA Channels 0 and 1 There are two independent DMA controllers that are capable of descriptor chaining in the full function endpoint The register interface is described on the user CD in the documents at D FPGA_Reference_Designs common PCIE_x8_Interface It is best that you read the details there Most users will not need to understand the control of DMA because the driver source code and binary in Windows and Linux is provided and works There two software interfaces to DMA 7 3 8 1 Scatter Gather
83. Dini Group LOGIC Emulation Source User Guide DN9002K10PCIE8T DN9002K10PCIEST User Manual Major Revision 1 Last Update April 13 2009 7469 Draper Avenue La Jolla CA92037 USA Phone 858 454 3419 Fax 858 454 1279 support dinigroup com www dinigroup com DN9002K10PCIE8T User Guide www dini 1 Table of Contents Contents 1 TABLE OF CONTENTS soossesoossesoossessossocsossoossesoosoesoossossossoesossoosssssossesoossossse 3 CHAPTER 1 seen s ens ens tas een eene ense eee ene ene 1 1 MANUAL 252202545 rio reae Pe on Cree no se ep Pe eaae eren eo e peni 1 11 1 1 2 QUICK START aD eu du 1 1 33 CONTROLLER SOFTWARE cesscccecssscccecssccececsenceceessnceceessncececsencececssnceeeessneeeeensnes 1 14 HARDWARE oes eee dees 2 15 THE REFERENCE DESIGN cesscccesssccececssccececseccececssccececssccececsenceceessnceeesssnceeeensnes 2 1 6 ORDERING INFORMATION ccccccecssccccecssccececssccececssccececsscceceessnceceesenceeeessnceeeensncs 2 MEE 6 DA D ELO c 2 2 1 08050000 2 2 2 MANUAL CONTENT cecccccecssccececssccececssccececssccececssccececseccececseccesecsss
84. EST There are three firmware files that Dini Group may release EEPROM FLP ic for EEPROM firmware hex for FLASH and prom flp mcs for Spartan PROM The first firmware update is for EEPROM which stands for Electrically Erasable Programmable Read Only Memory The Firmware Mode is booted from here This firmware is rarely changed Please consult with us before updating this device The second firmware update is Micro Controller MCU software that is stored in a flash memory The User Mode is booted from here This update can be accomplished easily from the USBController or AEtest_USB application The third update that may be required is a Spartan FPGA core update The configuration data for the Spartan FPGA is contained in a Xilinx configuration PROM This update can be accomplished with the Xilinx JTAG programming or iMPACT programs Either the Xilinx Platform cable USB 199 or the Xilinx Parallel cable IV 125 will help the updating faster If you don t have Xilinx cable and wish to update Spartan prom you can use USBController AEtest_USB to program it through USB You have to contact us for xsvf file When updating the firmware the Flash PROM and USBController exe should all is updated simultaneously since Dini Group only verifies this code using corresponding versions of each DN9002K10PCIEST User Guide www dinigroup com 35 CONTROLLER SOFTWARE 5 1 Obtaining the updates The firmware update files are not p
85. INTRODUCTION 11 SYSTEM 6 1 000600 9 2 WARNINGS 5 10 Dede EBSD sects 10 DEMENS dE 10 23 OTHER WARNINGS evene ee dee 10 3 PRE POWER ON 8 tens ens ens ens ens n 11 31 1 3 2 PREPARE CONFIGURATION 6 220 0 0000 1 3 3 INSERT THE COMPACT FLASH CARD INTO THE DN9002K10PCIE8T S COMPACT BEASHSEOT 12 3 4 INSTALL DN9002K 10PCTIEST IN 0 4 0 040000 0 12 EN a G BUR 12 3 5 1 Connect 5232 Cabl ccceccccccccsssccccesssececssnsccecssnsccecesssccecessscceceesscceseessaceceesces 12 3 5 2 Connect USB Cable miea a a a a a aa 13 3 5 3 Connect Power cable eee eee eee a a a 13 3 6 VIEW CONFIGURATION FEEDBACK OVER 5232 13 3 7 CHECK LED STATUS 6 0 20 00000000000000 0100 15 4 RUN USB CONTROLLER esee ee seen eee n eene ene e eee eee ene ene ene ena ens eos ens eos n 16 4 1 DRIVER INSTALLATION cccccccccecssccssscccsscccsssecesssccsssscessecesssscessecessscessescesssvesseees 16 4 2 OPERATING THE USB CONTROLLER
86. ION 1 4 Hardware Detailed description and operating instructions of each individual circuit on the DN9002K10PCIEST A description of each user accessible interface and user features 1 5 The Reference Design Detailed description of the provided DN9002K10PCIEST reference design Implementation details of the reference design interaction with DN9002K10PCIEST hardware features 1 6 Ordering Information Contains a list of the available options and available optional equipment some suggested parts and equipment available from third party vendors Compatibility lists 2 Conventions This document uses the following conventions An example illustrates each convention 2 1 Typographical The following typographical conventions are used in this document Convention Meaning or Use Example Prefix Indicates hexadecimal notation Read from address 0x00110373 returned Letter H Signal is active low INT is active low RSTn is active low 2 2 Manual Content 2 2 1 File names Paths to documents included on the User CD prefixed with D V This refers to your CD drive s root directory when the User CD 15 inserted in your Windows computer Alternately copy the entire contents of the User CD to your hard drive and allow D to refer to this path Due to limitations of the Xilinx ISE software we recommend a path without space characters in it Bad places include C Documents and Settings username Desk
87. JTAG debugging tools on the DN9002K10PCIEST you do not need to configure via JTAG DN9002K10PCIEB8T User Guide www dinigroup com 95 HARDWARE 11 2 Firmware Update Header The firmware update JTAG header 16 should not be used unless you updating the DN9002K10PCIEST firmware This header is used with a Xilinx Platform USB or Parallel IV cable The instructions for updating the firmware in the Controller software chapter 11 3 Troubleshooting If you ate having problems getting JTAG to work try connecting the Xilinx Platform USB cable to the JTAG header and running the Xilinx program Impact Impact will generate a failure log that you can email to support dinigroup com If you have an upgraded board please mention this in your email 12RS232 Interface RS232 access is available to all FPGAs through the header P1 FPGA 5232 connect to this header use the provided 1 header to DB9 cable to connect to a PC s serial port The RS232 transmit and receive signals are connected to each FPGA s pins AM13 TX from FPGA 14 RX to FPGA The TX and RX signals use the RS232 data protocol so the FPGA will have to implement a UART in its logic All FPGA share the same RX and TX signals so only one FPGA should use the interface at a time RS232 requires 12V to 12V signaling level which is not available on VirtexS FPGAs so an external RS232 buffer is used TSM 13601 T DV 1 2 TSM 136 01 T DV
88. N 00 M 000001010 DONE Setting GO N 01 000001100 DONE Setting G1 N 01 M 000001000 DONE dee CONFIGURING FPGA 9 Performing Sanity Check on Bit File BIT FILE ATTRIBUTES FILE NAME FPGA_A BIT FILE SIZE 003A943B bytes PART 4v1x100ff151317 09 38 DATA 2005 07 25 17 09 38 Sanity check passed E WITH CONFIGURATION OF FPGA A seem EK CONFIGURING FPGA Performing Sanity Check on Bit File BIT FILE ATTRIBUTES FILE NAME FPGA_B BIT FILE SIZE 003A943B bytes PART 4v1x100ff151317 05 01 DATA 2005 07 19 17 05 01 Sanity check passed CompactFlash card debugging information This lists the files found on the compact flash card If this list is wrong there is something wrong with CompactFlash MCU reads the contents of the file MAIN TXT and executes each instruction line Here the MCU is setting the clocks according to instructions in MAIN TXT The MCU is configuring FPGA A according to instructions in MAIN TXT Debugging information about the bit file The MCU is configuring FPGA B accotding to instructions in MAIN TXT The MCU is setting the temperature threshold Causes the FPGAs to automatically disable when overheating DN9002K10PCIE8T User Guide www dinigroup com 14 CONTROLLER SOFTWARE E WITH CONFIGURATION OF FPGA B SENSORS AYES BYES FPGA Temperature Alarm Thr
89. N9002K10PCIES8T User Guide www dinigroup com 21 CONTROLLER SOFTWARE If this window says something like GUID not found then the driver is not installed properly Check in the windows device manager and see if a device with VID 0x17DF and PID 0x1900 is there loxi ASIC Emulator PCI Controller Driver v1 Current Device gt DN9 62Ki PCIE8T Virtex5 PCI Express 8 lane Board Compiled on Sep 25 2008 at 16 62 05 Device Selection PCI Menu M gt Memory Menu Read Board Temperatures Read Clock Frequencies A B Read Clock Frequencies Q Set Board Level Clocks Config FPGA 12 Control Menu DMA Test Menu Interconnect Test Menu DaughterCard Test CFPGA A only DaughterCard Test FPGA only gt Production Tests Menu Get Board Serial Number Quit PCI BASE ADDRESS 9 4 800000 1 da g 2 46000000 3 48000000 4 88900000 5 86900000 Please select option Figure 10 main menu This is the menu with some things you can do I highly recommend the fun to use Production test menu You can run the PCIe test This will write stuff to the PCIe reference design that may of may not be loaded into FPGA A and report the results pix ASIC Emulator PCI Controller Driver vi Compiled on Sep 25 2008 at 16 62 06 MainBus Write Dword 2 gt MainBus Read Dword MainBUs Memory Fill MainBus Memory Display BAR Write Dword 6 gt PCI BAR Read PCI BAR Me
90. PGA When the DN9002K10PCIEST is powered off a voltage is supplied to the FPGA by a battery installed in socket X2 X2 is designed to house a CR1220 type lithium coin cell battery Typically these batteries produce 3 0V The socket may also wotk with battery types DB T13 L04 PA These howevet have not been tested Insert the battery positive side up B 12 m 2 4 ER c a DN9002K10PCIEB8T User Guide www dinigroup com 98 HARDWARE The same battery is used for both FPGAs Removing the battery will cause the FPGAs to lose their encryption memories and will have to be re programmed before they can work with encrypted bitfiles again To create encrypted bitfiles turn on the encryption option in bitgen The program will produce an additional output file with an nky extension Use the program impact with a Platform USB JTAG cable plugged into the FPGA JTAG connector on the DN9002K10PCIEST to load this nky file into each FPGA When using a bitfile with encryption enabled the DN9002K10PCIEST will not be able to read the FPGA type out of the bitstream It will therefore prevent your FPGA design from loading into the FPGA To disable this behavior you must disable sanity check Adding the following line to your main txt file can do this Sanity check n Also when using encryption you must be careful to correctly set the startup clock option correctly in bitgen or the FPGA will fail to configure
91. PGAs the maximum speed possible using this method Other methods may improve bandwidth beyond this limit The design provides MainBus registers to allow counting the bit error rate of each bank of 40 interconnect pins 3 3 Single Fast This reference design allows the characterization of FPGA to FPGA interconnect using standard synchronous IO methods between Main Bus registers are provided to allow the monitoring of the BER of each bank of 40 interconnect pins 3 4 V5 Interconnect This reference design might not be provided 3 5 Ethernet This reference design is a hardware test of the Ethernet interface It may not be provided 3 6 Header This reference design is a hardware test of the Header interface It requires a test fixture to work propetly It may not be provided 4 Using the Reference Design 4 1 Reference Design Memory Map Each reference design uses the MainBus interface to supply status and controls The following memory map is used These registers are accessible using the windows USB Controller program using the MainBus menu or from AETEST for PCI Express access All addresses on main bus are 32 bits Each address contains one 32 bit word By convention each FPGA has a fixed memory range FPGA will respond to all MB accesses in the range 0x00000000 FPGA B will respond to accesses from 0x10000000 Ox1FFFFFFF Other addresses are not defined The addresses given below are
92. Runaeusb wdm Select option 3 Firmware Menu 3 Please select option 2 Update Flash from lt firmware gt hex DN9002K10PCIES8T User Guide www dinigroup com 41 CONTROLLER SOFTWARE 4 Enter the full path filename It should firmware hex that we provide you 5 The process will take about 2 minutes When it finishes please hit Hard Reset 53 on the board or recycle power the board so that DN9002K10PCIE8T can boot from User Mode ASIC Emulator EEPROM Boot v5 Display Flash Version Update EEPROM from lt filename gt iic file Update Flash from lt firmware gt hex file Update 15326 Register values Boot From Flash Main Menu 9 gt Quit Please select option 2 Please enter filename C DiniWork dn_conf ig MCU DN 666k1G6 F irmware hex Transfered 133766 byte Transfered 178648 byte Set serial number to 6712686 Updated Flash Press any key to continue Figure 19 aeusb_wdm window You can also run this on the commend line aeusb_wdm_cmd exe FLASH lt filename hex gt aeusb linux cmd exe FLASH lt filename hex gt 5 5 PCI Express Endpoint Firmware Although the provided configuration files for LXT FPGA on your board responsible to the PCI Express endpoint are known to be perfect in every way Dini Group may release updates to add features or fix bugs in the PCI Express endpoint In this case Dini Group will provide a programming hex file to
93. SOp RAS and CAS The test points are labeled in silkscreen The test points near DIMMA implicitly are part of the DIMMA interface and so on 17FPGA Interconnect The point to point interconnect on DN9002K10PCIES8T is designed to operate at the maximum switching frequency possible on the DN9002K10PCIEST The fastest switching standard available on the Virtex 5 FPGA is LVDS Using this standard on the interconnect of a DN9002K10PCIEST we have demonstrated switching frequencies as high as 950Mbs A block diagram of the point to point interconnect is below DN9002K10PCIE8T User Guide www dinigroup com 108 HARDWARE FPGA A o em FPGA B Virtex 5 376 900 MHz Virtex 5 320 900 MHz Available on LX330 only AB10 AB17 The interconnect in the above diagram is confusingly described as sets of two busses Marketing explained why this was but I forget the rationale now DE is the bus between FPGA D and FPGA E It contains 120 signals and 120 signals This means there are 240 total signals between D and E If you use LVDS and pain the and signals you would have 120 LVDS signals between these two FPGAs The above diagram is only valid when the board is install with only LX330 FPGAs the largest available size When any LX220 or LX110 FPGAs are installed the interconnect available between FPGAs drops significantly In the Ordering Information chapter of this manual t
94. Spartan FPGA programming information is made available via JTAG header which can be used with the Xilinx program impact The Dini Group does not recommend doing any sort of development on this FPGA because if you add custom code you will not be able to use firmware updates from Dini Group without merging it with your custom code 4 Clock Network 4 1 1 GC Pins When this manual refers to a clock input of an FPGA it means the GC pin described the Virtex 5 user manual These pins have the capability of driving a DCM PLL or BUFG input with a known accounted for delay within the FPGA 4 2 Global Clocks All of the global clock networks on the DN9002K10PCIEST are LVDS point to point signals The arrival times of the clock edges at each FPGA phase aligned length matched on the PCB within about 100ps These clocks are all suitable for synchronous communication among FPGAs Since LVDS is a very low voltage swing differential signal you cannot receive these signals without using a differential input buffer Single ended inputs will not work An example Verilog implementation of a differential clock input is gtven below Wire aclk_ibufds IBUFGDS GOCLK_IBUFG O g0clk_ibufg I GCLKOp IB GCLK0n always g0clk_ibufg begin Registers end Either in the UCF or using a synthesis directive you should turn the DIFF_TERM attribute of the IBUFGDS to TRUE This is recommended because there are no external t
95. TS aano eratac na ino 152 2 1 1 Testing PCI Express IMOITUCE n dui ond scelte to 152 2 1 2 Testing FPGA to FPGA interconnect eese eerte 152 243 Testing DDR2 Inteyaces xc icis delta ic a reise cie 152 21472 153 289 Testing ECT ICT t et acres osa Dread oo ta ADR 153 2 1 6 Testing Daughtercard Connectors etienne 153 REFERENCE DESIGN TYPES eene eene tne ette etes ens ens ens ense 153 Bile ZAIN GEES Diese ICA NN INI SG SAN c 153 22 ES 154 Ou SINGLEPASTzud ssi eats 154 24 VS INTERCONNECT eite ete eee ete Spe Bo oe 154 JO ETHERNET e Sure 154 trie rene hee Ma M 154 4 USING THE REFERENCE DESIGN ccccsssssssssssssssscsssscsssosssssssscsscssoeses 154 4 1 REFERENCE DESIGN MEMORY 4840000000 00 154 5 INTERCONNECT SINGLE ccccssssocssccssccecsscccscsssccnccsccsscsessoccsscsessonsescsseses 156 S USING THE DESIGN 214 157 5 2 RUNNING THE TEST 157 INTRODUCTION 6 DDR2 INTERFACE oiscicicccicccccccssccscecctideessecsseoscescsencssceseocscesdascesdevacotapedecstscoesevecs
96. The DMA controller is capable of fetching descriptors from the host memory allowing the DMA engine to follow scatter gather chains The driver hooks for this weren t written yet when I wrote this You might have to call for an update 7 3 3 2 Large Buffers In large buffers mode the segment list is fixed and points to a ring of buffers in pre allocated locked driver memory space The user has unsynchronized access functions that allow copying DN9002K10PCIEST User Guide www dinigroup com 83 HARDWARE to and from these fixed buffers The DMA engines loop around the fixed buffers constantly completing the DMA on the buffers The user has access to controls that turn on and off the DMA when not in use Example use of this code is provided in the AETEST program in the file pcie_functions cpp 7 3 4 DMA Posted Mode Posted mode allows the FPGA A to initiate DMA transactions to and from the host memory space This mode is possible using the Dini Group full function DMA endpoint but is not enabled in the user interface module due to lack of interest Contact us to get access to posted mode 7 3 5 DMA Main Bus Main Bus is already pretty fast 100MB s however if you really more over main bus then we can tell you how to do DMA on Main Bus You might have to deal with synchronization issues on your own read write ordering 7 3 6 Electrical The electrical input and output characteristics are based on the PCI Express revision 1 1 and 2
97. This is the default state of the DN9002K10PCIEST when it powers To set the DN9002K10PCIEST to enable a configuration register must be DN9002K10PCIEST User Guide www dinigroup com 112 HARDWARE written This behavior is intended to protect users who do not wish to implement Main Bus interface but who wish to use the MBO MB35 signals for their own purposes 18 3 FPGA Interface All memory mapped transactions in the reference design occur over the MB bus This 36 signal bus connects to all Virtex 5 FPGAs and to the Spartan 3 configuration FPGA The Configuration circuit Spartan 3 is the master of the bus All access to the MB bus reads and writes is initiated by the Spartan 3 FPGA when the reference design is in use 848 USB ELK scx SYS CLK RD Spartan mmm MB 34D WR Spartan MB B3 DONE gah E BBS 0 to 200 Cycles All transfers a synchronous to the USB_CLK or SYS CLK signal This clock is fixed at 48MHz and cannot be changed by the user This clock is LVCMOS single ended When the configuration circuit asserts the ALE signal the slave device on the bus the FPGA is required to register the data on the AD bus This is the main bus address All future transfers over the main bus are said to be at this address until a new address is latched On a later clock cycle the master may assert
98. U flash memory U201 The check the MCU makes on reset to determine which mode it should start in is the firmware update switch S1 4 This EPROM code is stored in the EPROM DIP installed in U203 When the MCU is in this mode it registers itself to the operating system as Vendor ID 0x1234 product ID 0x1233 For DN9002K10PCIEST User Guide www dinigroup com 76 HARDWARE firmware update instructions see USB Software Firmware Update For information about the MCU boot up sequence see Hardware Configuration Circuit MCU The source code for the MCU firmware is provided D Source Code MCU FLASH as a Keil Studios MicroVision 2 11 project file 6 5 2 Activity LED A yellow LED located next to the USB connector flickers when there is USB activity 6 5 3 Configuration FPGA The MCU unit controls all of the configuration circuits on the DN9002K10PCIE8T but it does not have sufficient IO to access all of the configuration signals For IO expansion the MCU s external memory bus is connected to a Virtex 5 LX40 FPGA This FPGA provides a memory mapped interface to all of its This bus is called the Configuration Bus The configuration FPGA is connected to all of the configuration signals of the Virtex 5 FPGAs the temperature sensors status LEDs SmartMedia card CompactFlash card reset buttons Main Bus switches RS232 ports clock synthesizer control signals global clock multiplexer control signals FPGA
99. V power is generated from the 5 0V using a 30A power supply 22 4 Ground All ground OV voltages the DN9002K10PCIEST are shared A monolithic ground design strategy was used The nets GND_SHIELD and GND_ANALOG ate directly connected to the ground plane 22 5 Power Connections The primary sources of power for the DN9002K10PCIEST are the PCI Express graphics power connector From these two sources the DN9002K10PCIEST draws current at 12V all other voltages on the board are generated DN9002K10PCIEST User Guide www dinigroup com 124 HARDWARE This connector will work with a standard ATX power supply Any supply rated above 300W is likely to be suitable for use with the DN9002K10PCIES8T If no 6 pin PCI Express graphics power connector is available you may use an adapter cable provided Most new power supplies now have this connector available Note that only a 6 pin PCI Express graphics cable should be used This is easily confused with the now defunct AUX POWER connector also 6 pin and the 4 6 pin EPS server motherboard connections The connector is keyed so the wrong connectors will have difficulty fitting properly into the board Fittings are supplied such that the board can be powered from the PCI Express slot if this feature is desired however this operation is not recommended because it can easily overload the motherboard 22 6 Power Monitors The DN9002K10PCIE8T monitors the
100. ace The pre compile bitfiles for your board are located at D FPGA_Reference_Designs Programming_files pcie_dma In this design accesses to BAR2 BAR3 BAR4 BAR5 and both DMA channels are mapped to separate block rams in the FPGA Upper bits of the address offset are ignored so the block ram loops around To use this design see the PCIe section of the hardware chapter DN9002K10PCIEST User Guide www dinigroup com 162 Chapter 6 Ordering Information Part Number DN9002K10PCIE8T 1 Section Title Request quotes by emailing sales dinigroup com For technical questions email support dinigroup com 2 FPGA Options Any subset of FPGAs can be installed on the DN9002K10PCIE8T Any unneeded FPGA positions can ship empty to reduce the total price 2 1 FPGA A and B Select an FPGA part to be supplied in each position A B Possible selections are NONE LX110 1 2 3 LX220 1 2 3 LX330 1 2 2 2 CES Parts The DN9002K10PCIE8T may ship with CES engineering sample parts This is often the case early the Xilinx product release cycle If your board will ship with CES parts the quote will state the Xilinx part number of each FPGA on your board indicating a CES revision It is important that the user knows that CES parts may have limitations that are not listed in the Vittex 5 datasheet read about these limitations see the Xilinx website and search for Virtex 5 errata In general it is the responsibility of th
101. ail is also used to mean a power net GND ground grounded GND is a net on the DN9002K10PCIEST to which all voltages are referenced Ground is equivalent Grounded means connected to GND There is a single ground net on the DN9002K10PCIE8T 3 Resources The following electronic resources will help you during development with your board 3 1 User CD The User CD contains all the electronic documents required for you to operate the DN9002K10PCIEST These include schematics the user manual FPGA reference designs and datasheets The directory structure of the CD 15 as follows Config_Section_Code ConfigFPGA MCU Datasheets DNMEG _ Intercon_Daughtercard DNMEG Obsetvation_Daughtercard DNPCIE_CBL_CableAdapteraughtercard Documentation Manual Manual Dini_USB_Spec FPGA_Reference_Designs common DN9002K10PCIE8T Programming _Files pcie opencore DN9002K10PCIE8T User Guide the DN9002K10PCIEST firmware source code these sources are not intended to be used for development datasheet for every part used on the boatd You will need these to interface successfully with resources on the DN9002K10PCIE8T Daughtercard manual Daughtercard manual Daughtercard manual Contains this document and USB controller Specification Contains the source and compiled program ming files for the Dini group s DN9002K10 PClIe8t reference design Also board description files and simulation models
102. ally low frequency clocks 26 5 The signal on my board is going crazy on my oscilloscope Make sure the ground clip is attached to the probe If there is an oscillation on the signal at 60Hz there is a problem with the oscilloscope setup Capture the oscilloscope view and email it to support dinigroup com DN9002K10PCIEST User Guide www dinigroup com 149 Chapter 5 Reference Design This chapter introduces the DN9002K10PCIEST Reference Design including information on what the reference design does how to build it from the source files and how to modify it for another application 1 Purpose The purpose of the reference design includes the following Provide a means to test board hardware for failure Give users an understanding of the code necessary to use each interface provided in hardware Provide a starting point for using a tool design flow 1 1 Interfaces used by reference design The reference design helps users by showing them how using each interface is possible Code is provided as is and is intended as proof of concept on each interface advertised for the DN9002K10PCIEST product The Dini Group warrants only that the DN9002K10PCIE8T hardware is functional and usable The interfaces that the Dini Group design exercises and provides examples for are Access to the DDR2 SDRAM Modules at 250MHz FPGA Configuration interfaces over PCI Express USB JTAG and Compact Flash RS232 Communication FPGA Interconnect at hi
103. alog box check the Interconnect Test box The program will automatically load the bit files set the clocks and run the test 6 DDR2 Interface The DDR2 interface design is an example DDR2 controller running at 250MHz You can use this controller as an example especially for the purpose of required IO logic timing and clocking The controller bandwidth is most of the DDR2 bandwidth possible on the DN9002K10PCIEST DN9002K10PCIEST User Guide www dinigroup com 157 THE REFERENCE DESIGN 6 1 Provided Files The DDR2 reference design is part of the MainRef reference design and the MainRef files should be used 6 2 Using the Design The DDR2 memory interfaces are mapped to the address range OxNXX00000 0 Where the 4 bit represents an FPGA ID as described in the MainBus interface description X are don t care Since the remaining 19 bits are insufficient to address an entire 4GB DRAM there is a register DDR2HIADDR that selects the highest address bits of the DRAM Each address refers to a 32 bit location in the DRAM The lowest bit is not mapped to DRAM address but instead selects between the upper and lower 32 bits of the DRAM data This is necessary because MainBus is a 32 bit interface and DN9002K10PCIE8T DRAM interfaces are 64 bits wide The bank and side controls are also mapped to the DDR2HIADDR register The location of the DDR2HIADDR register is given in the Reference Design Memory M
104. an address register All data transferred to and from the main bus is LSB first The address 0x12345678 should be sent as a bulk transfer of 5 bytes 0x00 0x78 0x56 0x34 0x12 send a datum send the code 0x01 followed by 4 bytes LSB first When the DN9002K10PCIE8T receives data word it sends it onto the main bus interface to the address in the address register It then increments the address register Therefore to send two words over main bus 0x00000001 to address 0x0000001 and 0x00000002 to address 0 00000002 the USB Controller would send the following 15 bytes to USB EP2 0x00 0x01 0x00 0x00 0x00 0x01 0x01 0x00 0x00 0x00 0x01 0x02 0x00 0x00 0x00 Note that the number of bytes sent to EP2 must be divisible by 5 To request a main bus read operation the USB Controller sends a USB bulk write to EP2 to set the address register as described in the above paragraph Then the USB Controller sends a bulk read to EP6 endpoint 6 with the USB bulk request SIZE field set to the number of bytes requested The number requested must be divisible by 4 After the bulk read is complete the address register is incremented by SIZE 4 Read and write transactions use the same Before starting a USB read using a bulk transfer you must tell the DN9002K10PCIE8T how many bytes ate going to be read by using the VR SET EPGTC vendor request described in the Vendor Requests section 6 3 1 Important Note about Endpoints There is only one
105. and source configuration SelectMap configuration interface Blinking red and green LEDs The Configuration Section is built around a Spartan 3 and Cypress microprocessor These ICs ate used by the configuration circuit and are not intended for user design The code running these controlling ICs is collectively referred to as the firmware The code for this firmware is provided for reasons I don t know Customer development efforts on these platforms are not supported If you need special configuration circuit behavior please contact Dini Group and request that we implement it and support it as a standard feature The technical details of the configuration circuit are omitted from this manual since the user should not require it DN9002K10PCIEST User Guide www dinigroup com 49 HARDWARE 3 1 Configuration Section Feedback Duting normal operation and in error situations the configuration section prints messages to the RS232 terminal header The configuration section processes that can be monitored using this header are Temperature sensor FPGA overheat CompactFlash card reading USB configuration PCle configuration Main Bus treads writes Global clock settings RS232 MCU and Spartan 2 13 16 RS232 FPGA TX 21 RS232 FPGA TXD TENTH 7 RS232 MCU TX 8 TIOUT 8 35 55 292 MCU INCH M TBIN 8
106. ap section The clock that this design uses G1 must be set to between 200 and 250MHz 6 3 Running the Test To run the hardware test in the USB Controller application select Settings gt OneShotTest and check the DDR2 box The program will automatically load the bit files set the clocks and run the test reporting any errors 7 Clock Counters Each clock available to the FPGA is connected to a counter register and the value of this register is available on MainBus In this way the user can determine if each clock input is wotking propetly 8 LEDs All of the LEDs connected to an output enable register When the LEDs are not enabled the blink a pattern representing which FPGA the design is for When enabled each LED is controlled by the LED value register DN9002K10PCIEST User Guide www dinigroup com 158 THE REFERENCE DESIGN 9 Simulating the Reference Design The simulation environment the Dini Group uses is ModelSim A ModelSim project file is provided but it may not be compatible with your version of ModelSim When you create a ModelSim project add only the top level design file sim_board v Soutce can be found on the user CD D FPGA_Reference_Designs dn9002k1 0pcie8t MainRef source Also you must add to the project a simulation library Simulation models of all of the primitives used in the reference design are found in the Xilinx ISE install directory in the unisims directory Simulation models are al
107. art gt Programs gt Accessories DN9002K10PCIEST User Guide www dinigroup com 147 HARDWARE gt Communication gt HyperTerminal is a suitable program We use VanDyke software SecuteCRT program because it doesn t suck Hopefully the RS232 configuration status dump will tell you exactly what the problem is In any case the Dini Group will need this capture to diagnose the problem 26 2 The FPGAs won t program First connect the RS232 terminal and follow the instructions in the preceding paragraph Usually when an FPGA fails to program the configuration section will detect the problem and print an error message to this terminal Common problems the configuration section might report The syntax in the main txt file is incorrect The bit file on the CompactFlash card is for the wrong type of FPGA The CompactFlash is not formatted with a file system that the DN9002K10PCIE8T can read If the DN9002K10PCIEST reports about or more FPGAs that DONE did not go high then there is a problem with the bit file The bit file may have been generated using bitgen options that are not compatible with the DN9002K10PCIEST See if the FPGAs will configure using USB PCIe or JTAG When you contact Dini Group for support we will need a capture of the RS232 terminal output 26 3 My design doesn t do anything Make sure that the clock your design uses is running Output the clock to an LED and probe it with an osc
108. atching is done on the PCB for daughter card signals except between two ends of a differential pair However the Virtex 5 is capable of variable delay input or output using the built in IDELAY or ODELAY modules A signal delay report is available here Each delay is relative to the shortest delay and given in picoseconds Signal Name CLK_DCA_ON 1N DCA 1P CLK DCB ON CLK DCB 0P CLK DCB 13 CLK_DCB_1P DCAONO0 DCAONO1 DCA0N02 DCAONO3 DCAON04 5 DCAON06 DCAONO7 DCAON08 V DCAON09 V DCAON10 DCAON DCAON DCAON DCAON DCAON DCAON2 DCAON2 DCAON DCA0N24 DCA0N25 DCAON26 DCAON27 DCAON28 DCAON29 DCAON30 DCAOPO00 DCAOP01 DCAOP02 DCAOP03 DCAOP04 DCAOP05 DCAOP06 DCAOPO07 DCAOP08 Iq 4 N N e O Y o N o Reletive Delay ps 589 588 788 780 411 411 267 267 169 201 179 116 103 254 229 116 106 248 257 128 169 190 370 122 108 154 294 82 104 405 316 142 99 293 132 253 159 187 267 174 198 180 117 107 239 228 115 99 DN9002K10PCIE8T User Guide IO Delay Tap Value Ox4 Ox4 Ox1 Ox1 0 6 Ox6 Ox8 Ox8 Ox9 Ox9 Ox9 OxA Ox8 Ox8 OxA OxA 0 8 0 8 OxA Ox9 Ox9 Ox7 OxA OxA 0 9 0 8 OxA OxA 0 6 0 7 OxA OxA Ox8 0 8 0 9 0 9 0 8 0 9 0 9 0 8 0 8 www dinigroup com 137 HARDWARE
109. be set into T2 mode In the reference design the modules T1 mode Address Control signals FPGA Assume a DCM in system synchronous mode Worst clock to out time of Virtex 5 3 37 with DCM No phase shift Worst setup time 0 097 Wotst hold time 0 21 DIMM setup 600ps hold 600ps DQ signals DIMM DQS must be within 350ps of DQ DM setup 400ps Hold 400ps FPGA IDELAY setup 1 23 hold 2 14 clock to out 5 34 16 4 Compatible Modules The DDR2 interfaces are compatible with standard PC2 2700 or faster memory modules up to a capacity of 4GB The greatest capacity modules available at print time are 2GB The interface has been tested with modules with a CAS latency of 3 The interface 15 characterized to 250MHz although faster designs may be possible Xilinx is advertising a maximum DDR2 interface for the Virtex 5 of 333MHz The DDR2 memory interface can also be used with SRAM Flash and other types of memory modules See the chapter on Ordering Options for a list of compatible memory modules DN9002K10PCIEST User Guide www dinigroup com 107 HARDWARE THE DINI GROUP 1 8V 200 PIN SODIMM The interface implementation on these modules is not provided The customer must design the memory interface including timing and clocking 16 5 Test points Each DDR2 interface exposes five signals as test points located on the bottom of the PCB right under the SODIMM connector These signals are DO
110. be stuck in reset See the Troubleshooting section in the Hardware chapter Also check the red Reset LED As well as providing visual feedback the board graphic can be used to control configuration of the FPGAs To do this right click on an FPGA in the graphic to show a contextual menu with the options Configure Clear and reconfigure 1 LOIG wrt 1 Jac Configure FPGA D via USB Clear FPGA D Reconfigure FPGA D Configure will show an Open dialog for you to select the bit file you wish to use with the FPGA Clear FPGA will clear and reset the FPGA of its current configuration Reconfigure DN9002K10PCIEST User Guide www dinigroup com 28 CONTROLLER SOFTWARE FPGA will configure the FPGA with whatever bit file that bis instantiation of USB Controller used to configure that FPGA last 1 2 Menu Options The following sections describe each menu option and its function 1 2 1 File Menu The File Menu has the following options Open opens file with the selected text editor notepad by default To change the text editor see Settings Info Menu section About Displays USB Controller version number along with other things Switch device Displays a list of all Dini Group USB devices detects and allows the user to switch the current device The USB Controller will behave as if the current device is the only attached Dini Group USB product Under some situations the USB Controller
111. below The observation daughtercard DNMEG400_OBS product conforms to these dimensions DN9002K10PCIEST User Guide www dinigroup com 134 HARDWARE Type 0 1 4 Short View Top Side 400 Pin Receptacle on Back View Top Side 300 Pin Receptacle on Back P N 74390 101 P N 84553 101 5 000 3 250 5 000 i 1 1 950 050 1 950 0 500 The board edge constraints given above allow daughtercard to be installed on all positions of the DN9002K10PCIEST simultaneously 25 1 3 Insertion and removal Due to the small dimensions of the very high speed Meg Array connector system the pins on the plug and receptacle of the Meg Array connectors are very delicate When plugging in a daughter card make sure to align the daughter card first before pressing on the connector Be absolutely certain that both the small and the large keys at the narrow ends of the Meg Array line up BEFORE applying pressure to mate the connectors DN9002K10PCIE8T User Guide www dinigroup com 135 3 250 0 500 HARDWARE Place it down flat then press down gently The following two excerpts are taken from the FCI application guide for the Meg Array series of connectors A part can be started from either end Locate and match the triangle connector s A1 position marking for both the Plug and Receptacle Markings are located on the long side of the housing Rough alignment is required prior to conn
112. ch DWORD is implicitly incremented This behavior can be turned off contact support Write Address space from file this reads data from a file and writes the data to the address on main bus specified The data is written in little endian order The address is implicitly incremented after each DWORD of data This behavior can be changed contact support Send Command File this read data from file name do or txt with following format AD lt hex value gt Address WR lt hex data gt Data to write RD lt hex expected data gt read back data and compare to expected value 1 2 6 Settings Info Menu 1 2 3 4 5 6 Change Text Editor This option changes the behavior or Open in the file menu and is otherwise undocumented FPGA Stuffing information displays a list of the FPGAs on the board and their type and speed grade This information is stored in the firmware flash and is not detected on the fly Turn fans on off This command cannot be used with the DN9002K10PCIE8T Boatd Spattan MCU version this option is used to read the version number of the current board s firmware There are two types of firmware the Flash and the Prom The two types of firmware the reference design and the USB Controller application are only guaranteed to work when using corresponding versions of each If you update one you should update the others Read FPGA temperatures Displays the current temperature of th
113. cted to an FPGA and are controller by the user The meaning of the LED is design dependent Below is the general circuit used to connect user LEDs To turn the LED on drive the signal low To turn off tri state or drive high the signal LED E00 4 LED E01 4 LED E02 4 LED E03 4 LED E04 4 LED E05 4 LED E06 The user LEDs are connected to banks where the daughtercards are connected The Bank Voltage may not match the LED s current source voltage In this case use the drive standard corresponding to the bank and not the LED For example when a LVCMOS25 daughtercard is attached and all other signals on the bank are using the LVCMOS265 standard then use the LVCMOS265 standard for the LED on that bank Do not use DCI on LED signals You control the brightness of LEDs by either using a low drive setting DRIVE 2ma in the ucf file or by making the output bounce rapidly high and low like my special sister who 15 now a ward of the state LED Reference Signal Name The LED indicates the following when ON Designator Color DS28 51 YELLOW LEDA User LEDs Drive with LVCMOS DS52 62 YELLOW LED User LEDs Drive with LVCMOS The number of LEDs available for the user on FPGA A is 24 FPGA B has 11 LEDs User LED s on FPGA and B are numbered 0 to 23 The location of the IOs to use for these LEDs can be found in the provided UCF file or the netlist The name of each LED is labeled in silkscreen next to the LED 15 3 Et
114. ction should be updated when Readback is enabled If this sentence is here it means this section has not been verified yet When the DINI H API is made available this section should be removed completely Readback is performed in the same way that configuration except that the direction of the bulk transfer is READ instead of WRITE The commands required by the SelectMap interface to start a Readback must be sent using the configuration interface For this DN9002K10PCIEST User Guide www dinigroup com 75 HARDWARE reason it is the programmer s responsibility to understand and implement the SelectMap protocol 6 5 USB Hardware The actual hardware associated with performing USB communication with the DN9002K10PCIEST is briefly described here Since the user is not required to understand how to operate the hardware from the FPGA much detail is omitted 6 5 1 Cypress CY7C68013A A Cypress Micro controller MCU with a built in USB controller provides the USB interface of the DN9002K10PCIEST For a low level understanding of the way the DN9002K10PCIE8T communicates over USB you should see the Cypress CY7C680134A datasheet The driver that Dini Group provides is the free Cypress EzUSB driver with customizations to the corresponding file to identify the board as a Dini Group Emulator product As with all USB devices communication with the DN9002K10PCIEST is initiated by the host PC and can be either a USB
115. d at PHY pin clock out 2ns setup 2ns valid 1 2ns 9002 User Guide www dinigroup com 116 HARDWARE 25Mhz 10BaseT 100BaseTX R45 Vitesse 1000BaseT VSCB401 TXD 3 0 CLK_ETH_TX Fg RXD 3 0 n ETH MDC 1Kb Eprom The EEDAT and EECLK signals are intended to connect the PHY to an EPROM that would contain configuration settings for the device LED behavior MII timing Link speed duplex auto negotiation etc Since the MDIO interface is connected to the FPGA it is unlikely you would ever use these signals unless you just like emulating EPROMs on weekends and vacations If you do not implement the MDIO interface then the default settings are used for the device This includes settings that are specified by multi level inputs connected to resistors The CMODE options of the Ethernet PHYs has been set as follows CMODEO 0100 8 25K resistor CMODEI 0000 0 Ohm resistor CMODE2 0001 2 2K resistor CMODE3 0000 0 Ohm resistor This results in the following settings ADDR 00000 MDIO address CLKOUT TRUE Drives the CLK_ETH_125 signal PAUSE 00 I don t know DN9002K10PCIE8T User Guide www dinigroup com 117 HARDWARE DOWNSHIFT FALSE I don t know SPEED 00 Gigabit mode ACTIPHY FALSE SKEW 11 ETH125 clock MAC CALIBRATION MODE 00 PPPppp The LEDs on the RJ45 connector are controlled by the PHY The Amber LED indicates ac
116. d because the FPGA sourced signals DONE need to be driven by only one FPGA at a time The convention that Dini Group uses is to reserve the upper four bits in the address as an FPGA select address The address range hex 0 00000000 OxOFFFFFFF is reserved for FPGA 0x10000000 Ox1FFFFFFF is reserved for FPGA and so on The user need not follow this convention but unless you really need 32 bit addresses we recommend using it Only one FPGA has control of the DONE signal If the last address latched by ALE was not for a given FPGA it should tri state the output Before tri stating any DN9002K10PCIEST User Guide www dinigroup com 114 HARDWARE signal with a pull up or pull down resistor it is good practive to drive the signal to the DC value before tri stating So that simulation will match emulation result 19Ethernet An Ethernet interface is available to FPGA It is provided by a Vitesse VSC8601 tri mode Ethernet PHY The RJ45 connector can be used to connect to a regular 10Base T 100Base TX ot 1000Base T Ethernet network connection 9 1 pg Ie ang The VCS8601 device does not contain an Ethernet MAC The FPGA must implement a complete network stack to make use of the Ethernet connection Sorry I know that s retarded and the DN10 000K10PCI will be better Until then check out OpenCores Tri mode Mac controller http www opencotes otg ptojects cgi web
117. d s firmware The second is connected to the JTAG port of the Virtex 5 FPGAs This interface can be used for configuring the FPGAs or using debugging tools like ChipScope or Identify DN9002K10PCIEST User Guide www dinigroup com 94 HARDWARE 11 1 FPGA JTAG The connector for FPGA is shown below 87832 1420 2mm CON14A Bihassecee Note that the signal TDO on the header and in the schematic refers to the TDO port of the FPGA not the connector The order of the FPGA JTAG chain is FPGA A gt FPGA B gt FPGA There are no other components in the chain If you received your board with fewer than two FPGAs installed then the chain will be shorter The voltage of the JTAG chain is fixed at 2 5V and cannot change Hot plug on this header is allowed The header is a 2mm pin grid dual row with shroud and polarization key 11 1 1 Compatible Configuration Devices The JTAG header 15 designed to work with the Xilinx Parallel IV or Platform USB cable The JT AG chain is tested at manufacture using a Platform USB cable at 12 2 The driver installation process for the Platform USB cable is relatively difficult for a USB device Follow the instructions carefully In order to achieve high speed configuration using a Parallel IV cable you need to enable ECP mode on yout parallel port This is probably a BIOS setting on your computer 11 1 2 ChipScope In order to use
118. de www dinigroup com 40 CONTROLLER SOFTWARE C DiniWork Aetest_USB aetest_usb aeus ASIC Emulator EEPROM Boot v5 Display Flash Version Update EEPROM from lt filename gt iic file Update Flash from lt firmware gt hex file Update 15326 Register values Boot From Flash Main Menu Q gt Quit Please select option 1 Please enter filename C DiniWork dn_conf ig MCU EEPROM EEPROM_FLP iic This process take about 2 minutes please patient is updated Please POWER CYCLE the board or functionality doesn t work property AETest Quitting Press any key to continue Figure 18 aeusb_wdm window 5 Please power cycle the board You can also run this on the commend line aeusb_wdm_cmd exe EEPROM lt filename iic gt 5 4 Updating the MCU Flash firmware To protect against accidental erasure the MCU Flash firmware cannot be updated unless the board is put in Firmware Mode during power on see 5 3 You can either use USBController or AEtest_USB program to update MCU Flash firmware 5 4 1 Using USBController 1 Putthe boatd into Firmware Mode see 5 3 2 Run USBController exe Flash Update dialog will appear please select Yes 3 Please select firmware hex we provide you this file 4 When finish please recycle power the board or hit Hard Reset 53 on the board to boot from User Mode 5 4 2 Using AETest USB 1 Putthe boatd into Firmware Mode See 5 3 2
119. denine 96 12 RS232 INTERFACE DN QUI 96 12 1 1 Configuration RSS 97 13 TEMPERATURE 97 14 ENCRYPTION BATTERY 98 15 CEDINTERFACE 99 15 1 CONFIGURATION SECTION 99 922 100 15 3 ETHERNET LEDS iier tiere iei PAN BLUR NR Fed Un 100 154 POWER 101 13 37 UNUSED e hee e bula 102 ILE DID Lost c 102 POWERS iiio Aion dubie eden 103 TOI 103 16 1 2 Changing ihe DIMM 103 16 2 CLOCKING vsisi 104 16 3 SIGNALING 105 TO X Standards ut iss ns 105 1632 106 od ied MU ied dpa 106 16 4 COMPATIBLE 6 bu 107 10 22 8 108 17 FPGA INTERCONNECT Fox 108 19 MAIN BUS 110 111 INTRODUCTION 18 1 1 AUT OW 111 18122
120. do not interfere with each other excessively A BCD EF GH J K 1 1 2 2 3 3 4 4 5 5 6 5 7 7 Special Signal gt 8 9 9 10 S 10 11 11 12 12 Ti ex 13 14 14 15 15 16 16 17 17 18 g 18 19 19 20 A 20 21 2 AE 21 22 22 23 23 24 ET 24 25 25 26 26 27 2 27 28 28 29 29 30 30 31 31 32 32 33 33 34 34 35 1 35 36 36 37 37 38 38 39 39 40 40 B G J K DN9002K10PCIE8T User Guide www dinigroup com 143 HARDWARE All high speed signals the DN9002K10PCIEST including daughter card signals are routed against a ground potential reference plane When creating a daughter card it is recommended that these signals remain against a ground plane to maintain trace impedance The central columns of the connector pin out use a closely coupled differential pair pin arrangement which is uniformly surrounded by ground pins Below is a graphic representation of the pin assignments for the 400 pin connectors Note that this is a view from the backside of the connector The green boxes represent ground connections Special purpose pins are described below 25 2 2 VREF DCI Some of the signals connected to the daughter card expansion headers are clock capable the inputs on the Virtex 5 FPGA can be used for source synchronous clocking In the schematic and customer netlist on the user CD these pins contain C in the pin name Pins declared
121. e on die FPGA temperature sensors Force Memory Menu display When the Dini Group reference design is not loaded in at least one FPGA the FPGA Reference Design menu is disabled This menu command causes that menu to be displayed in this situation The USB Controller determines if the Dini Group reference DN9002K10PCIEST User Guide www dinigroup com 31 CONTROLLER SOFTWARE design is loaded by reading a memory location on Main Bus and comparing the result to a predetermined value This menu may also be disabled because the USB gt FPGA Communication is disabled 7 Toggle Sanity Check This menu command changes the behavior of configuration where it reads the header in the binary bit file and determines if the file is compatible with the installed FPGA This may be necessary if using bitstream encryption or using a custom bitfile not created by ISE 8 2 bitgen 8 Setup clock frequencies this menu option displays a dialog box allowing the three global clock networks to be configured 9 Global clock muxes setup This menu option display a dialog box allowing to select with output for global clocks and External clocks 1 2 7 Production Tests 1 Test DDR2 this menu option displays a dialog box allowing testing of the DDR2 sockets on the DN9002k10PCIe8T If the Dini Group reference design is not loaded the command will automatically load them into the FPGAs If the clocks not set the command will automatical
122. e programming files from the reference design The batch script will synthesize using XST from the source assigning the correct value to each define switch in the source The Build Utility is found at DN9002K10PCIE8T MainRef buildxst_DN9002K10PCIE8T make bat This batch file can be used to run ISE and bitgen If you do not have cygwin installed you need sed Stream Editor command to run make bat sed can be download from http gnuwin32 sourceforge net packages sed htm please download the setup program and set your PATH environment to executable file under lt SED gt bin folder You may also need to add the Xilinx bin directory to your path so the command par calls the correct program First make ucf file the script can be found at DN9002K10PCIE8T MainRef ucf_DN9002K10PCIE8T make_ucf bat Make sure that when you do type under cygwin gt which map cygdtive c Xilinx 10 1 ISE bin nt map gt which pat cygdtive c Xilinx 10 1 ISE bin nt par There are command line options that cause the script to output the correct reference design Since all the reference designs use the same source files Most commonly you would want to make the single ended or main reference design This includes the DDR2 controller Type gt make bat SINGLE to change the current source compilation type to Single ended Then type gt make bat build all 1x330 to start synthesis place and route and bitfile generation t
123. e shorted together by 5 ohm resistor This connection allows the use of external clock feedback If you need to use these test points as two separate IOs this resistor would have to be removed The reference design uses this connection for external clock feedback The register is calleed TPP 5 5 5 Clock Test points Each of the Global clock networks has a test point These points are not length matched with the global clock network so there may be some phase offset between this point and the FPGA input iu B Ed 0225224 o DN9002K10PCIEB8T User Guide www dinigroup com 69 HARDWARE ec 0 All of test points output LVDS signaling LVDS test points have the signal connected to pin 1 square and connected to pin 2 circular 100 ohm resistor connects the P and side of these clock signals This is excellent for probing with a high impedance probe but not so good for connecting wires You can remove this tesistor if needed 5 6 DIMM Signals Some key signals on each DDR2 interface are connected to test points for debugging The test point pad used is the same as on the Power TP test points The test points are not labeled with their reference designators Instead they are labeled with the signal name 6 USB interface The DN9002K10PCIEST allows the user FPGA to communicate to a host PC over USB The confi
124. e user to determine if the board is suitable for his application prior to ordering a board Details about the interfaces on the board that are not in this manual and characterizations of interfaces if available can be requested 2 2 1 Hardware Errata Details There are no errata for Virtex 5 production non CES parts DN9002K10PCIES8T User Guide www dinigroup com 163 ORDERING INFORMATION 2 3 Small FPGAs The DN9002K10PCIE8T is optimized for two Xilinx Virtex 5 LX330 FPGAs Optionally it can be ordered with LX110 or LX220 FPGAs instead When installed with one ot more LX110 ot LX220 FPGAs the amount of available interconnect is reduced due to some IOs in those devices being no balled DDR2 SODIMM MICTOR 4GB Max 64 64 250MHz i 20 COMPACT Configuration 36 MB 35 0 FLASH NYJ canis 2 1052 8 125 MHz Spartan 3 ente FPGAA FPGA B 400 7 uP Config Virtex 5 450MHz 900Mb s Virtex 5 Control LX110 LX220 or Fi 175 LX110 LX220 or 77 lt LX330 i LX330 FF1760 FF1760 clock config MICTOR Global Clocks LX50T Virtex 5 FF655 PCle Controller 1 8 2 5V 3 3V 1 8 2 5V 3 3V 8 8 Express PCI Express 1 1 2 5 Gb s A ynt or Express 2 0 5 0 Gb s with JTAG oo Daughtercard I 5232 p ALL FPGAs D i 49M MB48CLK
125. ear the contents of this text box 1 1 4 Board Graphic USB Controller s main window shows a graphic representing your DN9002K10PCIE8T The number of FPGAs that are installed on your board should appear in this graphic If one or more DN9002K10PCIEST User Guide www dinigroup com 27 CONTROLLER SOFTWARE FPGAs are configured on the board a blue LED will glow next to the FPGA in this graphic window just like on the actual board hardware itself If the USB Controller could not find a DN9002K10PCIEST connected to any USB port this window will appear USBGontroller The DiNi product was not found Please check the following 1 Your USB cable is firmly plugged into the computer and the board 2 Your board is powered on 3 The device driver for the board is loaded 4 The device is not presently configuring itself From the media card If the board is turned on and plugged in the USB Controller should be able to detect it If it does not try opening the Device manager You can right click on the My computer icon and select Hardware tab and click the Device Manager button This will display a list of the devices connected to your computer If a Dini Group Product FLASH Boot appears in the USB section then USB is working properly on the board but the program is unable to connect to it Select Switch Device from the File menu If the board does not appear in the Hardware manager then the DN9002K10PCIE8T may
126. ector mating as misalignment of gt 0 8mm could damage connector contacts Rough alignment of the connector is achieved through matching the Small alignment slot of the plug housing with the Small alignment key of the receptacle housing and the large alignment slot with the large alignment key Both connector housings have generous lead in around the perimeter and will allow the user to blind mate assemble the connectors Align the two connectors by feel and when the receptacle keys start into the plug slots push down on one end and then move force forward until the receptacle cover flange bottoms on the front face of the plug Like mating a connector pair can be unmated by pulling them straight apart However it requires less effort to un mate if the force is originated from one of the slot key ends of the assembly Reverse procedure from mating Mating or un mating of the connector by rolling in a direction perpendicular to alignment slots keys may cause damage to the terminal contacts and is not recommended DN9002K10PCIEST User Guide www dinigroup com 136 HARDWARE 25 2 Daughter Card Electrical The daughter card pins out and routing were designed to allow use of the Virtex 5 s 1 Gbps general purpose IO All signals on the DN9002K10PCIEST all routed as differential 50 Ohm signal to ground transmission lines Signals can be used as single ended also Proper electrical levels are explained in the VCCO section No length m
127. ed You can close impact and disconnect the Xilinx JTAG cable 5 2 2 Using USBController If you do not have a JTAG cable you will need to use the following instructions to update your Spartan PROM firmware This update is dependent on USBController and Flash firmware version Please double check with us support dinigroup com to make sure that your current version MCU version USBController supports this option and request xsvf file from us 1 Open USBController ini and add the line service_mode 1 You save and close the file 2 Launch USBController go to Service menu and select Program Update Spartan warning message will appear to ensure that you want to update Spartan If you do hit Yes button 3 Open file Dialog will appear Please select the xsvf file that we provide you 4 After selecting file there will be debug level dialog Please select debug level 0 5 The process takes about 10 15 minutes please leave the board and USBController alone The process bar is on the bottom of USBController window 6 When the execution is finished power cycle the boatd 5 2 3 Using AEtest USB If you do not have a JTAG cable you will need to use the following instructions to update your Spartan PROM firmwate This update is depending on AEtest USB and Flash firmware version Please double check with us support dinigroup com to make sure that your current version MCU version AEtest_USB supports
128. ed Eten 126 22 74h 126 22 7 2 Removing 126 22 7 3 Fan Tachometers wiccccccccccsccccsssccsssscesscscesscccsssscsssscessescsssscessssesssscesseccssscsessseees 127 23 127 23 11 FPGA USER INTERFACE 8 0400 000 127 23 1 128 23 2 NON FPGA USER INTERFACE 8 408888 128 23 2Up Ae te eii Me 129 23 3 NOT FOR USE CONNECTORS cc cccsscsssssssesssesscesscecssecseecssscsscsscssscsssssessseaesese 129 23 3 E TE RAN 130 24 MECHANICAL 130 25 DAUGHTERCARD 8 131 25 1 DAUGHTER CARD 60480000000 enne entren tese nn innen 132 25 1 1 Daughter Card Locations and Mounting esee 133 25 1 2 Standard Daughtercard Size ae eet a 134 INTRO DUCTION 25 1 3 Insertion and 1 135 25 2 DAUGHTER CARD ELECTRICAL e
129. ee enne nnne nennen nest nen 137 29 21 Pin assi enmen d c aee rd e ae a ca EU due AU ies 142 25 227 CEP VRE DCT reote ee NIRE 144 25 2 3 Global clocks veecciccccccccccccsccccsssccsssscssscecesscccsssccsssscessescesscsessssesssscessescsssesesssees 144 25 2 4 Timing and Clocking dA an A Re 144 25 25 Power and Reset a o ette teat tee eiie 145 2252007 VCCO Voltar css DER I FUN ODE LO de 146 25 2 7 VCCO Dids PeleFallolsstoo a etc estado ea pecado Sd ue 146 25 3 ROLLING YOUR OWN 147 26 TROUBLESHOOTING eese en seen netus ens ens eene e eee eee ese eene ene ens ens ense 147 26 1 THE BOARD IS 444 147 26 2 THE FPGAS WON T PROGRAM ccccccccssscssscsssesssesscecscecssecssecesscssscsssssesessesesesesess 148 26 3 MY DESIGN DOESN T DO ANYTHING eet enne enne trennen nennen 148 264 THE DEMS WON TEOCK Sis 149 26 5 THE SIGNAL MY BOARD IS GOING CRAZY ON MY 149 CHAPTER 5 REFERENCE DESIGN e eese eese eese enses eene ee e eset en etes e ene oae 151 I 151 1 1 INTERFACES USED BY REFERENCE 4 000 151 2 HARDWARE TES
130. eeeeessneeeeesaes 2 224 2 2 22 Dimensions S ines da 2 223 Part Pin 3 2 2 4 Schematic Clippings RM 3 2 3 TERMINOLOGY ede oto 3 3 RESOURCES coeeectr P 4 4 3 2 2 200046000 0 5 3 3 ERRATA AND CUSTOMER 8 4 0 4 4 0 8 5 33d Aon rd d rre Rm 2 3 4 SCHEMATICS AND NEILIST eese nennen tenetis seen estes seite seen ndn 6 3 4 1 217 6 342 CONVENTIONS scccceensececsensccecesnsececssnsccecssnsececsensccesssnsccecssnacescesnaceess 6 3 5 DATASHEET 20 4 004460000 6 2 0 DEUX NIMIUM DAMM NIE 6 3 7 DINI GROUP REFERENCE 468000 7 4 EMAIL AND PHONE SUPPORT eeeeeeeee esee essen stesse ens tns eos ees esee esee 7 CHAPTER 2 QUICK START GUIDE ssscssscsscsnsscscsosssantecossconsoosesonscasasessenasseasconseneoessios 9 1 PROVIDED MATERIALS eeeeeeee ee e eee eee ene ene ene ens ens eos eos eoa n 9 DN9002K10PCIE8T User Guide www dini
131. ematic drawings ate included in this document to aid quick understanding of the features of the DN9002K10PCIEST These clippings have been modified for clarity and brevity and may be missing signals parts net names and connections Unmodified Schematics are included in the User CD as a PDF Phase refer to this document when designing an interface in the FPGA Use the PDF search feature to search for nets and patts 2 3 Terminology Abbreviations and pronouns are used for some commonly used phrases The user is assumed to know the meaning of the following Spartan Spartan refers to the Spartan 3 FPGA device used by the DN9002K10PCIEST to perform configuration circuit functions It is used interchangeably with configuration circuit DCM DLL PLL Digital Clock Manager or Digitally locked loop This is a clock synthesis module in a Virtex 5 FPGA PLL is Phase locked loop See Xilinx documentation LVDS Low Voltage differential signaling a signaling standard with a 1 2V DC and 300mV AC level in this manual and in advertisements LVDS is often used where Differential Signal should be used instead Net Signal Plane rail DN9002K10PCIEST User Guide www dinigroup com INTRODUCTION A net is an electrically continuous piece of conductor on the PCB before assembly Signal can refer to an electrically continuous conductor on the PCB or to the logical meaning of that net Plane is a net for voltage sources R
132. ermination resistors on the DN9002K10PCIE8T All global clock networks have a differential test point The positive side of the differential signal is connected to pin 1 square and the negative side is connected to pin 2 circular DN9002K10PCIEST User Guide www dinigroup com 57 HARDWARE Global Clocks dock config hasis ICS GCLK1 MHz 8442 Daughtercard EXT1 Daughtercard B BOT 4eMHz MBABCIK LIII A diagram of the global clock network is shown above Each of the eight clock outputs of the clock network is distributed to both FPGAs 4 3 GO G1 G2 Clocks The GO G1 and G2 clocks are the primary clock resource for your FPGA design Each of these clocks can be set to a wide range of frequencies between 0 125 MHz and 550 MHz On the schematic these signals are named CLK_G _ p where is 0 1 or 2 and is the name of the FPGA connected to that signal The possible source of GO and G1 clock is either 15326 synthesizer or a step clock The step clock is driven by the configuration circuit and can be toggled over USB or PCIe by writing to the correct configuration register OxDF23 0 0xDF23 1 Before the Synthesizer or step clock drives the network the correct source setting must be made in the GUI or in the main txt file By default the source is the synthesizer The configuration register that sets the source of the clocks is at location OxDF16 bit 0 corresponds to GO bit 1 cor
133. eshold 80 degrees C Figure 4 RS232 Output Note this won t be the same as the output that you get but you will have general idea 3 7 Check LED status lights The DN9002K10PCIEST has many status LEDs to help the user confirm the status of the configuration process Check the power Failure LEDs to confirm that all voltage rails of the DN9002K10PCIE8T are within tolerance If the voltage of any critical power net on the DN9002K10PCIEST is too high or too low the board will be held in reset and at least one of the red LEDs will light The LEDs are located on the top edge near the left corner of the PCB Each one is labeled with the voltage that it represents Normally all of these LEDs are off If any of these LEDs light there is a power problem with the board and you should contact us First make sure that the output of the power supply is acceptable If the 5V 3 3V or 12V power fail LED is lit you most likely have a problem with the power supply you are connected to and less likely with the DN9002kK10PCIEST Reset LED When the board is in reset for any reason including powet failure or pressing the reset button this LED will light RED The LED is located below the RED voltage LEDs next to the logic reset button Check the Spartan FPGA status LED located near pin 1 of the PCIe edge connector 0566 This LED should remain BLUE as long as the board is powered on except for a quarter second just as the board is powering
134. figuration circuit on receiving this vendor request asserts the PROG signal of the selected FPGA This resets the FPGA and clears any configuration data it may already have This Vendor request also selects the FPGA so that SelectMap bus activity only affects the selected FPGA Bulk transfers initiated after this command to endpoint 2 are interpreted as SelectMap transfers rather than Main Bus transfers See Main Bus access above This will be so until vendor request VR SETUP END 0xBD is called 4 USB host software sends a bulk write USB request to EP2 Each byte of data in the bulk write is sent to the selected FPGA over the SelectMap bus and the FPGA signal CCLK is pulsed once for each byte of data sent Note that the LSBit in the USB transaction is sent to the LSBit in the SelectMap interface so bit swapping as described in the Virtex 5 Configuration Guide is not required A standard bit file from Xilinx bitgen can be transferred in binary over this USB interface to correctly configure an FPGA on the DN9002K10PCIEST Make sure CCLK is selected as the startup clock in the bitgen settings This is the default setting 5 After an FPGA configures the DONE signal will go high lighting the blue LED next to the FPGA labeled 6 The USB Controller sends a vendor request out VR SETUP END 0 This request deselects the FPGA so that further bulk requests are interpreted as Main Bus transactions 6 4 1 Readback This se
135. fore using them to configure each FPGA If the target FPGA annotated in the bit file header is not the same type as the FPGA the MCU detects on the board it will reject the file and flash the error LED Before this command is executed lt yn gt is set to the default value If you want to encrypt of compress your bit files you will need to set lt yn gt to n MAIN BUS Writes data in lt WORDDATA gt to the address on the main bus Ox WORDADDR interface at lt WORDADDR gt This command only makes sense 0 lt gt in the context of the Dini Group reference design unless your design implements a compatible controller on the main bus pins The Specification for this interface is in MainBus section MEMORY MAPPED Writes to a configuration Register This command can be used to O0x lt SHORTADDR gt access features that do not have a main txt command Example lt gt applications include setting clock sources settings the EXT1 clock buffers to zero delay mode or setting the clocks to frequencies lower than 31MHz SOURCE G0 2 The SOURCE instructions cause the global clock networks to SOURCE G1 2 output a clock from an alternate source When source of is set SOURCE G2 2 to 2 then the global clock GO becomes a step clock which can be accessed through config register 0xDF23 When source of G1 is set to 2 the global clock network G1 becomes step clock which
136. gh speed techniques High Speed PCIe mode to FPGA A MainBus interface for USB and PCIe communication Blink LEDs in cool patterns Reset Button New internal Virtex 5 features PLL ODELAY 550MHz clocking and 800Mbs IO Set global clocks Ethernet DN9002K10PCIEST User Guide www dinigroup com 151 THE REFERENCE DESIGN All source code for the reference design is included on the CD and may be used freely in customer development Precompiled bit files for the FPGA types that are installed on your board are provided and can be used to verify board functionality before beginning development A build utility described in the section Compiling The Reference Design can be used to generate new bit files or to generate bit files for less common configurations of the DN9002K10PCIEST 2 Hardware Tests The provided bit files and software is suitable for testing most of the hardware interfaces on your board Some hardware tests require test fixtures and these are not provided 2 1 1 Testing PCI Express interface Install the board into a windows machine in a PCI Express x16 or x8 slot other slots will cause the test to erroneously report a failure Turn on the machine Run the provided executable aetest wdm exe From the main menu select production tests and then pci test The test should report PASS or FAIL 2 1 2 Testing FPGA to FPGA interconnect To test the FPGA interconnect you will need to run the one
137. gnal until this occurs CompactFlash configuration only If you do use the SelectMap data signals as interconnect the provided software USB Controller is not guaranteed to function propetly may interfere with your design When using these signals as interconnect the appropriate drive standard is LVCMOS25 The IO voltage is 2 5V The data signals D 8 15 are not used by default and can be used by the design for interconnect between FPGAs SelectMap Readback is possible on the DN9002K10PCIE8T This be accomplished over PCle or USB The user interface for obtaining this data is not defined If you need this feature contact the Dini Group The JTAG configuration method does not go through the configuration circuit See JTAG interface section for details about this DN9002K10PCIEST User Guide www dinigroup com 51 HARDWARE 3 3 USB and PCle interfaces The USB and PCIe interfaces can be used for both configuration FPGA configuration and clock settings etc or for direct communication with the user design in the FPGA These interfaces are described individually in their own sections in the hardware chapter 3 4 CompactFlash Interface jm Most important settings on the za DN9002K10PCIE8T be controller through the TM Compact Flash interface This interface can also be used to configure FPGAs The CompactFlash interface is not under the direct control of the user but is accessed only by the configuration
138. gt 5 3 Updating EEPROM firmware not recommend To protect against accidental erasure the EEPROM firmware cannot be updated unless the board is put in firmware update mode during power on Find Switch S2 User Reset on the DN9002K10PCIEST Figure 17 Switch 52 DN9002K10PCIEST User Guide www dinigroup com 39 CONTROLLER SOFTWARE Hold down the User reset button while the DN9002K10PCIE8T powers on Or alternately while holding down the User reset switch press the Hard reset button The DN9002K10PCIEST samples the user reset button on power on to enter into firmware update mode 5 3 1 Using USBController 1 Putthe boatd into Firmware Mode see 5 3 2 Open USBController ini and add this line service mode 1 save and close the file 3 Run USBController Update Flash dialog will appear please select NO because we are doing update EEPROM 4 Go to Service menu select Program EEPROM This Process will take about 1 minute Please hit OK 5 Select file EEPROM_FLP tic When USBController completes the update please power cycle power the board 5 3 2 Using AETest_USB 1 Put the board into Firmware Mode See 5 3 2 Run aeusb_wdm aeusb_linux Select option 3 Firmware Menu 3 In EEPROM Boot Menu please select option 1 Update EEPROM from lt filename gt iic file 4 Enter filename full path The process should take about 2 minutes DN9002K10PCIEST User Gui
139. guration circuitry allows this by bridging USB to the Main Bus interface For most users implementing USB communication will be as simple as making a Main Bus controller In the reference design there is an example Main Bus controller See the Main Bus section of this chapter for more information on the Main Bus DN9002K10PCIEST User Guide www dinigroup com 70 HARDWARE USB on the DN9002K10PCIEST also allows control of the configuration circuitry from a host PC This includes configuring FPGAs setting clock frequencies and others This section will describe the software interface required to communicate to the DN9002K10PCIEST In addition to reading this section you may choose to modify the provided software USB Controller and usb source code for these programs is on the user CD These programs collectively implement all of the available controls on the DN9002K10PCIEST 6 1 Connecting to the DN9002K10PCIE8T Depending on the operating system there are different methods of obtaining a software handle to the DN9002K10PCIE8T in order to access it from software 6 1 1 Windows XP What driver is this It s the EzUSB driver HANDLE handle CreateFile NNN A NEzusb 0 GENERIC WRITE FILE SHARE WRITE NULL OPEN EXISTING 0 NULL The EzUsb 0 device name is registered with Windows when installing the EzUSB device driver The ini file provided with the driver causes the driver to be assigned to any USB
140. h Card lt number gt be any positive number in decimal Decimal points allowed lt yn gt can be the letter y or the letter n lt level gt can be 0 1 2 or 3 lt SHORTADDR gt is 2 digit number in hexadecimal 16 bits lt BYTE gt is a 1 digit number in hexadecimal 8 bits lt WORDADDR gt 4 digit 32 bit number in hexadecimal representing a main bus address WORDDATA 4 digit 32 bit number in hexadecimal containing data for a main bus transaction The following table describes the function of each of the available main txt commands Instruction Function comment The configuration circuitry performs no operation and moves to the next command VERBOSE LEVEL This command will set the amount of output that will be produced lt level gt over the RS232 port during configuration When level is set to 0 the port will produce only error output FPGA A lt filename gt The Virtex 5 FPGA A will be configured with the file named by lt filename gt DN9002K10PCIEST User Guide www dinigroup com 53 HARDWARE FPGA B lt filename gt The Virtex 5 FPGA will be configured with the file named by lt filename gt FPGA Q lt filename gt The Virtex 5 FPGA Q will be configured with the file named by lt filename gt SANITY CHECK lt yn gt If lt yn gt is set to y then the MCU will examine the headers in the bit files on the CompactFlash card be
141. h the FAT32 file system In this case the DN9002K10PCIEST will not be able to recognize files on the card DN9002K10PCIEST User Guide www dinigroup com 55 HARDWARE 3 5 Configuration Registers The configuration control on the DN9002K10PCIEST is controlled by setting configuration registers Basically these are just locations in the memory space of the on board micro controller that controls the board s function A full description of the function of this micro controller is omitted but some of the registers in this space are required to be accessed over USB or PCIe to control the board For information on how to access this address space over USB or PCIe see the corresponding section in this chapter REGISTER ADDRESS FUNCTION is for GO is for G1 Bit2 is for G2 Setting a bit to 1 will cause the global clock network G1 or G2 to use its alternate source For GO this is step clock 0 for G1 this is step clock 1 For G2 this is CLOCK SOURCE DF16 FPGA A RESET DF22 Write 0 2 to hold reset 0 0 to release G0 FREQUENCY LO DFCO Sets the divider value of GO G0 FREQUENCY DF30 Sets the 8442 multiplier of GO G1 FREQUENCY LO Sets the 8442 divider value of G1 G1 FREQUENCY HI DF32 Sets the 8442 multiplier of G1 G2 FREQUENCY DF33 Sets the 8442 divider value of G2 G2 DF34 Sets the 8442 multiplier of G2 When high each bit causes the configuration circuit to update the represented cl
142. h to use source synchronous interconnect ignore this reference design with prejudice All FPGA to FPGA interconnect in this design is constantly being driven by one FPGA sending uni directionally a test pattern The receiving FPGA checks the test pattern for correctness against a known pattern The design is intended to characterize the bandwidth of the interconnect between FPGAs Access to test status is provided over the MainBus interface Note that there are two designs ADC and In the design the directions of LVDS connections between FPGAs are uni directional In the all of the signals are in a direction opposite to the ABC design signals 11 1 Provided Files source is located at D FPGA Reference Designs NDN9002K10PCIE8TMMainRef Note that this is the same source as the Main Reference Design compile the design for LVDS define statements in the Verilog code must be added or removed The make bat utility desctibed in the compiling the reference design section automatically adds and removes these directives The pre compiled bitfiles for this design are located at D FPGA Reference Designs VProgramming Files DN9002K10PCIES8 TAL VDsSIntercon V 11 2 Using the Design The design s MainBus interface is undocumented The IOs in the LVDS reference design are clocked using the GO clock A clock setting of 300MHz on GO results data transmission from FPGA to FPGA of 600Mbs pe
143. he clock points are listed in the test point section 23 1 FPGA User Interface Connectors The following connectors are directly connected to the FPGA and the user needs to know the interface requirements in detail All of these connectors should be fully described in the manual section indicated below FPGA Interface Connectors Reference Manufacturer Part Number Connector description FPGA Manual Section J2 J1 Lighthorse LTI SASF546 P26 X1 SMA Jacks differential A Clocks J16 AMP 2 767004 2 Mictor logic analyzer connector Mictor DN9002K10PCIEST User Guide www dinigroup com 127 HARDWARE J17 AMP 2 767004 2 Mictor logic analyzer connector ALL Mictor P5 FCI 84520102LF MEG Array 400 pin plug A Daughtercards P6 FCI 84520102LF MEG Array 400 pin plug B Daughtercards J4 JAE MM50 200B2 1E DDR2 200 pin SODIMM socket B DDR2 2 Samtec TSM 136 01 T DV Dual row 0 1 RS232 header ALL RS232 19 Molex 22 27 2031 3 pin Fan Power A Power 115 Molex 22 27 2031 3 pin Fan Power B Power 52 Omron B3S 1002 Reset Pushbutton ALL Reset 23 1 1 Comments If you have a boatd with fewer than two installed connectors associated with the missing FPGA will be not be installed P2 Connections to this RS232 header are through a 12V buffer 1915 Pin 1 GND Pin 2 5V Pin 3 Tachometer 23 2 Non FPGA User Interface Connectors The following connectors are not directly connected to FPGA IO and therefore the user does not need
144. he build script creates a directory called out and places its output files there After the script completes you will find files for each FPGA that was built fpga bit is the file to be downloaded to the FPGA When using the provided VHDL the genetic definitions are not complete in the Dini Group code Some of the signals that are governed by generics must be defined externally or defined in the first place 10 2 Bitgen Options The Make bat script correctly sets all bitgen options that are compatible with the DN9002K10PCIEST The following options should be used with the DN9002K10PCIEST Options that are not listed here can be selected by the user or left to their default settings Compress OFF Or you can disable sanity check option on board UnusedPin Pullnone DN9002K10PCIEST User Guide www dinigroup com 160 THE REFERENCE DESIGN Persist Yes Only require is Readback is used Encrypt No YES requires that you disable sanity check option on board DonePipe No DriveDone Yes Don t ever disable CRC Check This is the easiest and most certain way to turn your FPGAs into little piles of carbon ash I am pretty sure this option exists to increase sales of replacement FPGAs 11LVDS Reference Design The LVDS Interconnect design is to show the user how to implement source synchronous communication between FPGAs Using this method the advertised 900Mbs system speed can be achieved If you do not wis
145. he chapter Reference Design for a description of the Main bus interface Test DDRs FLASH Reigsters FPGA Interconnect 1 1 Main Window The main USB Controller window has the following components a menu bar a refresh button a Disable USB button and board graphic and a message log Each item in the menu bar is described later in this section Products USRGontroller File Edit FPGA Configuration P n Settings Info Service Refresh Enable USB gt FPGA BOARD DN9002k10PCIe8T USB to FPGA communication is disabled Enable if you want to use reference design features 0x22 Board Serial Number 0708024 Sanity Check passed Configuring FPG via USB please wait File work DN les DN9G02K1 OPCIEST MainTest LX330 pga_ A bit Configured FPGA A via USB Sanity Check passed Configuring FPGA via USB please wait File C work DN bit fi les DN9002K1 1330 _ B bit Configured FPG B via USB Figure 13 Main USBController window DN9002K10PCIE8T User Guide www dinigroup com transferred transferred 26 CONTROLLER SOFTWARE 1 1 1 Refresh Button Se Vil Use File Edit FPGA Configuration Refresh Di The Refresh button updates the board graphic by querying the DN9002K10PCIE8T and reading back its status The USB Controller program does not poll the board and only updates the status when
146. he signals DCA1P29 connects to pin on the FPGA Impact Only customers who ate using one of these three pairs as differential signal are affected Single ended use of these signals is not impacted Solution W ork Around When using daughtercards requiring differential signaling on these pins then the FPGA RTL must invert the transmitted or received logic The differential input buffer should connect so that the connects to a pin on the FPGA and port connects to an N pin on the FPGA DN9002K10PCIEST User Guide www dinigroup com INTRODUCTION 3 4 Schematics and Netlist Unmodified Schematics are included in the User CD as a PDF Use the PDF search feature to search for nets and parts 3 4 1 Netlist In lieu of providing a machine readable version of the schematic the Dini Group provides a text netlist of the board This netlist contains all nets on the board that connect to user IO on any FPGA When interfacing with any device or connector on the DN9002K10PCIE8T you should use either the provided ucf or the netlist to generate the pinout The netlist is located on the user CD at D Schematics Rev_01 DN9002K10PCIE8T_customer_netlist txt 3 4 2 Net name conventions All power nets begin with a symbol or GND All clock signals begin with CLK 4292 ec 0 Two sides of a differential signal differ by one character or n This character is near the end of the net name
147. here are any interfaces where performance is only characterized for specific speed grade parts this is noted in the advertisement and in this document Below is a list of all such interfaces 1 PCle Express Some interfaces may run at increased speeds above and beyond Dini Group s advertised performances when used with 2 or 3 speed grade parts Some performance numbers that are advertised by Xilinx are listed here These characterizations have not been performed on the DN9002K10PCIEST but we have no reason to think the DN9002K10PCIEST is limiting factor on these interfaces FPGA to FPGA interconnect LVDS 1 25 Gbs 625 MHz FPGA to FPGA interconnect single ended 800 Mbps 400 MHz DDR2 Interface 667 Mbps 333 MHz 2 5 Upgrade Policy Upgrading adding FPGAs to a DN9002K10PCIE8T Call for a quote DN9002K10PCIES8T User Guide www dinigroup com 165 ORDERING INFORMATION 3 Optional Equipment The following tools are suggested for use with the Dini Group DN9002K10PCIEST 3 1 Compatible Dini Group products The Dini Group supplies standard daughtercards and memory modules that you can use with the DN9002K10PCIEST 3 1 1 Memories The memory module solutions from Dini Group allow the user to install whichever type of memory his application requires DNSODM200 SRAM Memory module for use in the 200 pin SODIMM sockets Standard memory configuration Two GS8320V32 memories 1M x 32 each Performance up to 175MHz SDR Small
148. here is a block diagram showing the available features on a board loaded with LX220 or smaller FPGAs If the boatd has mixed LX220 LX330 FPGAs then the interconnect available between any two FPGAs is the lesser of the signal counts shown in these diagrams Each FPGA to FPGA interconnect signal is tested at 700Mbs prior to shipping no matter which speed grade is installed on your board Higher speeds are possible given appropriate IO timing methodology and speed grade parts The theoretical limitation imposed by the DN9002K10PCIEST 15 1 1Gbs the limit of the Virtex 5 s internal clock network Dini group has demonstrated speeds up to 0 95Gbs on each pair of interconnect signals Information on how to achieve this interconnect switching speed can be obtained by examining the Xilinx application note XAPP855 Other methods of implanting high bandwidth interconnect are described in 860 The Dini Group reference design uses an older method designed for Virtex 4 In a synchronous system between two FPGAs and a DCM in zero delay mode the following timing is possible Clock to Out 3 37 NS Trace Delay 1 70 NS Rise time adjustment 0 30 NS Clock skew 0 20 NS duty cycle 0 05 NS jitter 0 05 NS DN9002K10PCIEST User Guide www dinigroup com 109 HARDWARE setup time 1 00 NS Min Period 6 67 NS Max Frequency 0 15 GHZ If LVDS is used make sure to assign the DIFF TERM attribute to the IBUFDS in the receiver FPGA As the fre
149. hermal overload Most of the power available to daughter cards through the connector comes from the two 12V pins for a total of 24W Each power rail supplied to the Daughter card is fused with a reset able DN9002K10PCIEST User Guide www dinigroup com 145 HARDWARE switch Daughter cards are required to provide their own power supply bypassing and onrush current limiting 3 3 5 0 12 0V GCAP 104 lt DCO GCAN 104 GCBP 104 Z 8 GC BN lt JDC 0_GCBN 104 n 4 lt pco 85 lt 0 GCCN 85 DCO RSTn DC_RSTn gt gt Section 1 of 5 Clock Power Reset 74LVC1 G07 EN MEG Array 300 P in SOT95P 280 5N The RSTn signal to the daughter card is an open drain buffered copy of the SYS_RST signal It is also asserted when the User Reset is active When RSTn is de asserted the 3 3V 5 0V 12V power rails are guaranteed to be within the DN9002K10PCIE8T tolerance If there are additional power requirements the daughter card is required to ensure these 25 2 6 VCCO Voltage The daughter card is required to provide a voltage on the VCCO pin on the connector This voltage is used on the DN9002K10PCIEST to power the FPGA IOs that are connected with that daughter card In this way the daughter card can control what voltage the interface will use Each bank of the connector BO B1 or B2 uses a separate VCCO pin and ca
150. hernet LEDs These LEDs are controlled by the Ethernet PHYs connected to FPGA B They can also be user controller by setting registers in the serial interface of the PHYs DN9002K10PCIEST User Guide www dinigroup com 100 HARDWARE i iwas pg ASEN T1 and 12 are the RJ45 jacks on the top edge of the board There is a yellow and a green LED embedded in this connector facing the board edge LED Reference LED Signal Name The LED indicates the following when ON Designator Color T1 GREEN LINK1000 Ethernet PHY has established link 1000Base T T1 YELLOW ETH ACT Ethernet PHY has detected activity DS18 GREEN LINK100 Ethernet PHY has established link 100Base 15 4 Power LEDs These LEDs indicate is one or more power supplies fail either outputting a voltage that is too high or too low The voltage that the LED indicates is marked in silkscreen near the LED DN9002K10PCIE8T User Guide www dinigroup com 101 HARDWARE LED Reference LED Signal Name The LED indicates the following when ON Designator Color 056 055 RED POWER_FAIL One of the 1 0 power supplied has failed DS1 2 3 4 RED POWER_FAIL One of the board power supplied has failed DS14 RED RESET The board is in reset 15 5 Unused LEDs These LEDs controlled by the configuration circuitry One GREEN LED is always on One yellow one flickers when something undefined is happening Two RED ones signal which FPGA is undergoing some sort of configuration
151. hter Cards Figure 27 Board power topology diagram The maximum power draws on each of these rails is given below 12V 9A 1 0 15 1 0 15 2 5 20 3 3V 6A 5 0 9 VDIMM_A 2A VDIMM_B 2A 1 2V_S 0 2A 0 9VA 0 2 0 9VB 0 2 22 1 Power 12V The 12V rail is used to generate most other voltages on the board The only places where 12V is used directly are the daughtercards Below is a list of the maximum power draw of each of the 12V loads on the DN9002K10PCIE8T DN9002K10PCIEST User Guide www dinigroup com 123 HARDWARE Rail Max Cutrent Uses 12V current 1 0V 25 Internal FPGA power 2 3A 1 0V_B 25 Internal FPGA power 23 1 8V 2 5 DIMM 0 3A 2 5 DIMM 0 3A 2 5V 9 Spartan 3 1 2V 2 6A FPGA IO FPGA Aux power Daughtetcatds 10W 1 2A TOTAL 9 0A The total possible power requirement of the DN9002K10PCIEST is 9A on 12V 108W typically each FPGA would only use 10W and daughtercatds would use little power 2W Under these conditions the 12V power requirement is only 2 5A 25W Under these conditions use in a server tack would work 22 2 Power 3 3V 3 3V is used by the DN9002K10PCIEST to supply the clock distribution network the configuration logic Micro controller and Spartan 3 FPGA and daughtercard power The maximum power requirement for the DN9002K10PCIEST on 3 3V is 1 3 3V is taken directly from the ATX power supply or from the PCIe slot 22 3 Power 2 5V 2 5
152. ignals are connected to 2 5V clock bank on the FPGA DIMM_SDA and DIMM_SCL should be driven using 525 standard For details on the DIMM C2 signal see the clocking section below The DIMM interfaces are not designed for hot plug 16 1 2 Changing the DIMM voltage If you need to change the voltage of the DIMM interface there is a set of jumper points provided for each interface allowing power to be redirected from a source other than the on board 1 8V power supply When the DN9002K10PCIEST is shipped a jumper is installed connecting the DIMM FPGA Bank power to the 1 8V power rail Next to each of these jumpers is a 2 5V test point suitable for jumper ing to the DIMM power rail if necessary Some Dini Group products DNSODM_SDR DNSODM_DDR1 require this jumper to be installed When installing this jumper remove the 1 8V jumper to prevent shorting 1 8 and 2 5V supplies together DN9002K10PCIEST User Guide www dinigroup com 103 HARDWARE 1 8V 2 5V 2 1 8 B j Em For example to change the DIMMD interface to 2 5V remove the jumper installed in 12 and install a jumper from TP12 2 to TP13 1 16 2 Clocking The data signals the DDR2 interface are clocked source synchronously In order to clock in and out the DQ data signals the DOS signal are used as a clock using the Virtex 5 BUFIO clock driver Details on how to implement a DDR2 controller are in the Xili
153. illoscope Check the pin out in your constraint file Check the PAR report file to make sure that 100 of yout IOBs used have LOC constraints There is never a reason not to constrain an IO Use the PAD report to make sure your constraints were all applied Some situations may cause constraints to be ignored Double check that the connections match between your FPGA pins and the daughtercard pins using the schematic If MainBus interface is not working make sure that none of the other FPGAs are driving those MB pins Make sure that the Unused IOBs option in bitgen is set to Float Check for Timing errors in the timing report DN9002K10PCIEST User Guide www dinigroup com 148 HARDWARE Route the clock signal to a pin and observe it with an oscilloscope 26 4 DCMs won t lock 1 The DCMs are required to be set in a frequency mode compatible with the frequency of the reference clock input Check the following attributes of the DCMs DFS FREQUENCY MODE DFS PERFORMANCE MODE 2 All clock inputs of the DCM are required to be stable for a certain number of microseconds before releasing the DCMs reset signal If you are generating the reference clock from an FPGA or another you will need to build a delayed reset circuit to reset the second 3 Make sure the global clock you are using is being received with an LVDS receiver not a single ended one Make sure the DIFF TERM attribute 1s turned on especi
154. ive signals To input ot output differentially you must remove this resistor 02 3 XCBVLX330FF 1760 GC3 GC 16 2 le coles CC B amp G 2102 ooo OOOO lalo kalo ob APG 0 8 isp G 513 DN GC VRP 3 I UZ OOOO colos L14 0 clc als oo g 79 e e 16 The schematic clipping above shows FPGA B s 8 point but all FPGAs usg the same pinout as A list of all test points on the board can be found the Gest points seco E26 SU DN9002K10PCIEST User Guide www dinigroup com 63 HARDWARE This signal can also be used as an external feedback path for a DCM When connecting the output of a DCM to K14 the DCM FB input can be connected to K15 Using this configuration output flip flops connected to of the DCM will have an effective clock to out time of less than zero 4 6 2 Ethernet Clock The VSC8601 Ethernet PHY device outputs a 125MHz clock The signals in the schematic are CLK125 ETH This signal is LYCMOS25 single ended signals The frequency is fixed Details about appropriate clock methodology for the Ethernet interface is in the Ethernet section 4 6 3 DDR2 Clocks The CK signals in the DDR2 interface are described in the DDR2 interface section OUTPUTS CLK_DIMMB_CK2p AC33 CLK_DIMMB_CK2n AD32 INPUTS CLK_DIMMB_CK2p AM28 CLK_DIMMB_CK2n AN2
155. led on your computer The executable created by the source is a kernel module which is loaded dynamically A kernel module load script is provided DOS Under DOS only direct device access is supported The DOS version of AETest program does not use a dtiver You therefore need to figure out how to configure and access a device on the PCI subsystem 15 not supported Solaris The Solaris driver does not support DMA 7 3 9 2 Configuration Register writes Board settings clocks FPGA temperatures etc can be changed over PCle by accessing the Configuration Register interface A description of the registers in this interface is in the Configuration Section of this chapter Writes To write to a configuration register write to BARO offset 0x258 Send a 32 bit word of data This data is encoded as follows Bits 31 16 Configuration Register address in only addresses 0x DFO0 OxDFFF are valid See the Configuration Register map in the Configuration Section section Bits 15 8 Ignored Bits 7 0 The Data value to write to the register Reads DN9002K10PCIEST User Guide www dinigroup com 87 HARDWARE To read from a configuration register read one byte from PCIe at an address within encoded as follows Bits 31 24 The DN9002K10PCIE8T s BARO Bits 23 16 the lower 8 bits of the address of the configuration register you would like to read The upper 8 bits must be OxDF or the read will not be va
156. lid Bits 15 0 0x0260 7 3 9 3 Bus The Main Bus interface is how you can communicate to all FPGAs on the DN9002K10PCIEST over PCIe not just FPGA A The bandwidth available over the Main Bus is much lower than that of PCIe so performance is not as great using this method For details about the Main Bus see the Main Bus section in this chapter Expected speeds will be 30 to 80 MB sec To write to Main Bus over PCIe write to BARO at the address QLPCI REG MBADDR with the 32 bit value representing the main bus address you would like to write to Then write a second write to address QLPCI REG MBWRDATA with 32 bit data representing the data that you would like to write to main bus After the Spartan 3 has received a write to both the MBADDR and MBWRDATA registers it will write to the main bus interface To read from the Main Bus over PCle first write to BARO address QLPCI REG MBADDR with the 32 bit value representing the main bus address you would like to read from Then read from BARO QLPCI REG MBRDDATA The returned value will be the value read off the main bus at the selected address When an error has occurred No FPGA responded to the read request the read will return the value OxBBBBBBBB If all you get is 0x1234567 this means the main bus is being used by USB at the moment QLPCI REG MBADDR 0x240 QLPCI REG MBCTRL 0x270 QLPCI MBWRDATA 0 248 QLPCI MBRDDATA 0x250 7 3 9 4 FPGA Configuration The
157. linx Virtex 5 LXT FPGA with high speed serial interfaces SMA SATA SFP PCI Express DNMEG INTERCON Connects headers for FPGA A and B together DNMEG OBS Adjustable voltage tenth inch pitch headers User LEDs Iwo Mictor 38 connectors DN9002K10PCIEST User Guide www dinigroup com 167 ORDERING INFORMATION SMA global clock inputs for host board PN DNMEG Obs 412V 45V 33 Power LEDs BalGid Connector 40 pin Header 0 1 pin spacing VO Vohage Control 40 pin 4 Header 3 2 Compatible third party products The following products have been shown to work with the DN9002K10PCIES8T Intel Entry Server board SE7230NH1 E http www intel com design servers boards se7230nh1 e index htm Standard DDR2 modules 256 MB 19 512 MB 15 1GB 25 2GB 79 4GB eventually http www crucial com store listmodule DDRIIl list html Xilinx Platform USB Cable required for JTAG FPGA programming ChipScope Pro HW USB G http nuhotizons com Mictot bteakout MIC 38 BREAKOUT http www emulation com catalog off the shelf solutions mictor DN9002K10PCIE8T User Guide www dinigroup com 168 ORDERING INFORMATION 4 Compliance Data 4 1 Compliance 4 1 1 EMI Since the DN9002K10PCIEST is not intended for production systems it has not passed EMI testing Compliance is only done by special request 4 1 2 PCle SIG The DN9002K10PCIEST passes the electrical compliance test for P
158. location where pin one is the power rail and pin two is a ground connection These test point locations ate suitable for supplying at least 2A regardless of the power requirements or capabilities of the power net DN9002K10PCIEB8T User Guide www dinigroup com 65 HARDWARE Pin one is square Pin two is circular TP14 TP16 TP18 TP23 TP10 9 22 15 0 9V B 1 0V 1 0V B 12V 1 8V 2 5V 1 2V 3 3V 5 0V Power for the 12V 5 0V and 3 3V nets are generated off board These test points are suitable for wiring to if power is needed off board for some reason or maybe you need to bring power in from an external source 5 2 Power TP The following test points are located along the left edge of the board next to an LED associated with that power net These test points are square pads They are not suitable for supplying power to the board or off the board DN9002K10PCIE8T User Guide www dinigroup com 66 HARDWARE Power Fail LEDs TP 14 1 0 DNI COPPERDOT The test point reference designator is not visible on the silkscreen of the DN9002K10PCIEST Instead there is a label indicating which power net the test point is connected to These test points are connected by thin traces that are not capable of conducting more than 100mA of current You should only use these test points for probing For noise measurements it is better to use the test points next to
159. lt m Parallel to serial lt Clock recovery circuit PLL Bit stream 2 5 Gbit s TX_P TX_N REFCLK_P REFCLK_N RX_P RX_N Upstream or Downstream 2 5 Gbs Gen 1 or 5 0 Gbs Gen 2 Figure 26 PIPE Slowdown block diagram Using this core a PCI Express controller can interact with a real full speed link partner and test control paths that a non interactive simulation might never test There is a fee for use of the PIPE slowdown cote 7 5 Troubleshooting In PCI or PCI Express when bus master does not receive a responds for a read request within a certain timeout petiod it will return to the upstream requestor This can happen for various reasons The board has lost its configuration data the PCI configuration space registers are not programmed The FPGA on BAR DN9002K10PCIEST User Guide www dinigroup com 91 HARDWARE 8 Unusable pins 8 1 1 Configuration The following pins All FPGAs are the SelectMap data pins used to configure the FPGAs These pins are connected to both Virtex 5 FPGAs Using these signals for FPGA interconnect is possible but may interfere with the configuration circuitry on the DN9002K10PCIEST 9 System Monitor ADC The new Virtex 5 feature System Monitor allows the FPGA to use some of its IO as analog to digital inputs 43 3 V e D OUT BUSY D DONE DN9002K10PCIE8T User Guide www dinigroup com 92 HARDWARE The voltage measuremen
160. ly set them to settings compatible with the DN9002K10PCIE8T reference design 2 One Shot Test this menu option tests various functions on the board automatically configuring the FPGAs and setting up the clocks as required 1 2 8 Unsupported Features If you somehow discover these features they are not intended for customer use 1 3 INI File Some command considered debugging commands save persistence information in an int file that gets created in the same directory as the USB Controller executable This file should not be generated for most users If it is generated you can safely delete it unless you like it Some of the settings that can be stored in this file are the Text Editor Selection settings the location of path to the reference design programming files for one shot test and enabling the debug menu DN9002K10PCIEST User Guide www dinigroup com 32 CONTROLLER SOFTWARE 2 AETest USB 2 1 Compiling AETest_usb AETest_usb can be compiled using Microsoft Visual Studio 6 or later or on any version of Linux that supports the usbdevfs library A make file is provided but you must un comment one of the following lines to define which operating system you are running In Windows you should run nmake ZDESTOS WIN WDM ZDESTOS LINUX ZDESTOS SOLARIS 2 1 1 Cygwin Nope 56 only 3 PCle AETEST Application AETEST utility program can test and verify the functionality of the DN9002K10PCIE8T Logic Em
161. m Driving clocks from IOs is best accomplished using a ODDR flip flop If you don t know what I am talking about there is a description of exactly what to do elsewhere in this manual 7 3 8 FPGA Interface A Verilog module is provided that correctly implements the interface between the FPGA and the FPGA Q for PCI Express communication The source for this module is provided on the user CD in the following location D FPGA Reference Designsvxcommon PCIE 8 InterfaceN A module contained in the provided source file pcie x8 user interface v is an implementation of the interface that must be included in the A user design The user interface presents 6 separate interface ports Target Write Target Read DMA RO DMA DMA and DMA The Target Write and Target Read interfaces share BAR and address lines as target reads and writes cannot happen simultaneously Each interface has its own enable accept and data ports Read interfaces also have a data valid port The enable signals are held active until the associated accept signal goes active The accept signal for an interface may be tied high if it is guaranteed that transfers for that interface can be accepted every clock cycle if the interface is connected to a block RAM Data valid can be pulsed with the accept signal or any time after this allows reads to be pipelined For the purposes of simulation a model of low synthesi
162. mance from FPGA to FPGA on this bus as high as 75MHz if you adjust input and output clocks and perform a timing analysis Using LVCMOS25 with a drive strength of 24mA you can assume there is 10ns rise time flight time for signals on this bus No length matching is done on the MB signals Virtex 5 clock to out time 3 37ns with DCM Virtex 5 setup time 0 97ns Flight time 10ns includes rise time adjustment for capacitive load Total 14 34ns 69 MHz The MB signals are tested at 485MHz 18 2 Error Codes The Main Bus interface has no way of signaling an error condition on read requests but some errors will result in the same sentinel values being returned Following is a list of these values OxABCDABCD The Main Bus read timed out PCIe only OxDEADDEAD The Main Bus read times out USB only When this condition occurs a register accessible as part of the configuration register space gets incremented In this way it is possible for a Main Bus access program to verify that a MainBus transaction has succeeded OxFFFFFFFF The PCIe bus timed out This is not a value returned by the DN9002K10PCIEST The PCIe request was not returned The LX50T may not be configured correctly OxDEAD5566 This value is returned by the Dini Group reference design as a default value when a read request is to an address that has no registers associated with it OxBABABABA unknown Contact support 0x12345678 The Main Bus is disabled
163. more than 4 GB is not tested Although the DN9002K10PCIESTEST is compatible with any DDR2 SODIMM module support for certain addressing configurations may not have been implemented so if you find your module doesn t work email us the model number The socket DIMMB is connected to FPGA B The socket can accept any capacity DDR2 SODIMM module Note that DDR1 modules will not work in these slots since they are a completely different pin out and voltage level 3 2 Prepare configuration files The DN9002K10PCIEST reads FPGA configuration data from a CompactFlash card program the FPGAs on the DN9002K10PCIE8T FPGA design files with a bit file extension put on the root directory of the CompactFlash card file using the provided USB card reader DN9002K10PCIEST User Guide www dinigroup com 11 CONTROLLER SOFTWARE The DN9002K10PCIE8T ships with a 256MB Compact Flash card preloaded with the Dini Group reference design These bit files can also be found on the User CD You can also compile the reference design source provided on the CD and place the generated bit files on the Compact Flash card Insert the provided Compact Flash card labeled Reference Design into your USB card reader Make sure the contains the files FPGA_A bit if FPGA A installed FPGA_B bit if FPGA B installed main txt The files FPGA_A B bit are files created by the Xilinx program bitgen part of the ISE 9 2 tools The file main txt contai
164. mory Display PCI BAR Memory Range Test Soft Reset lt User Reset Disable PCI Mainbus Communication Test DDR2 on FPGA B Test all DDR2 on FPGA B Test all DDR2 on FPGA B using DMA and Main Bus Test BlockRAM on FPGA A PCI BASE ADDRESS da8 68086 1 da d6 666086 d8 666066 4 66608000 Main Menu Quit Please select option Figure 11 Memory menu The memory menu is also fun as it lets you display PCle memory space directly and also allows access to the MainBus interface that we used a little earlier DN9002K10PCIES8T User Guide www dinigroup com 22 CONTROLLER SOFTWARE PCI E Test LUDS Interconnect Test DIR ABC LUDS Interconnect Test DIR Single Ended Interconnect Test Quit Enter Option Figure 12 Production test menu It is acceptable to operate the DN9002K10PCIES8T from USB and PCIe at the same time The mutual exclusivity of all features is not finalized but it s a safe bet that if you use the MainBus feature from PCIe and USB simultaneously the board will do something other than work properly 7 Scan the JTAG chain If you wish you can program the FPGAs using their interface Connect a Xilinx Platform USB cable into the FPGA JTAG port J7 and open the IMPACT program that is installed with Xilinx ISE 9 1 DN9002K10PCIES8T User Guide www dinigroup com 23 CONTROLLER SOFTWARE When you c
165. n have a different voltage applied to it When designing a daughter card you must determine the current requirements for the DN9002K10PCIEST and supply enough current capacity on these pins The VCCO voltage impressed by the daughter card should be less than 3 75 to prevent damage to the Virtex 5 IOs connected to that daughter card Additionally the voltage applied to the header pins from a daughtercard or external source should be equal to or less than the VCCO voltage of the bank that contains the IO For example a 2 5V daughtercard one that uses 2 5V on each VCCO pin should not drive a 3 3V signal onto the daughtercard pins 25 2 7 VCCO bias generation Since a daughter card will not always be present on a daughter card connector a VCCO bias generator is used on the motherboard for each daughter card bank to keep the VCCO pin on the FPGA within its recommended operating range The VCCO bias generators supply 1 2V to the VCCO pins on the FPGAs and are back biased by the daughter card when it drives the VCCO rails DN9002K10PCIEST User Guide www dinigroup com 146 HARDWARE lt DC0_BO_VCCO 380mA MAX AT 1 22V LT1763C S8 SO1C127P600 8N R467 2 10 0K l 380mA MAX AT 1 22V The output voltage of this regulator can be adjusted 1f needed This will require changing the resistors on the ADJ pin of the regulators The bias regulators can provide up to 1 5A of current Some low speed designs may not need more than this Dini Gr
166. ne clock cycle of latency in board trace delay alone For the maximum bandwidth use single ended signaling at 00MHz For single ended signaling an IOSTANDARD of LVCMOS25 is appropriate Use drive strength of 6mA 8mA 18Main bus Main Bus is the interface that the DN9002K10PCIEST uses to bring USB and PCIe access to both of the Virtex 5 FPGAs If you want to use USB in your design or want PCIe access without implementing PCIe in FPGA then you must implement a Main Bus slave your FPGAs The reference designs include one such controller and you free to use it DN9002K10PCIEST User Guide www dinigroup com 110 HARDWARE Drive strength Please use the highest drive strength IOs available 24mA 18 1 MB Signals The DN9002K10PCIEST in addition to the dense interconnect available between FPGAs point to point topology provides a 36 signal wide MB bus that is connected to both Virtex 5 FPGAs MICTOR MB 35 24 Configuration 35 0 Spartan 3 FPGAA FPGAB Virtex 5 Virtex 5 MB48CLk These signals are reserved 18 1 1 Disambiguation The MainBus has two meanings In this document it usually refers to the interface connecting the FPGAs to USB and PCIe via the configuration circuitry It can also mean the group of 36 signals on the DN9002K10PCIEST that connects both Virtex 5 FPGAs It just so happens that the MainBus interface is implemented using 36 of the
167. nnected to the IO are arranged into busses Each bus has an ID code an OE register bank an ENABLE register bank and an IN register bank The addresses of the IO registers are as follows FpgaNum 4 bit MB_SEL_INTERCON 4 bit busnum 20 bit reg_offset 4 bit FPGA NUM is 0x0 for FPGA A 0 1 for FPGA 0x2 for FPGA MB SEL INTERCON is 0xC busnum is any number but only low values less than LAST ADDR will constrain valid busses reg offset is 0 0 for REG OUT 0x4 for REG OE 0x8 for REG IN and for REG ENABLED To determine which bits if any in bus are valid read the REG ENABLED register The 32 bits returned 41 are a mask for which of the bits in the REG OUT REG OE and REG IN registets are meaningful To get the bus ID of a bus write value 0x1 32 bit to REG ENABLED then read ENABLED then write 0x0 32 bit to ENABLED The value returned will be a coded name for the bus Bits 0 15 are ASCII characters representing FPGA names Bits 16 31 are an arbitrary unique integer distinguishing the bus Connecting busses from two different FPGAs have the same bus ID To cause an FPGA to output signals on a bus write Ox FFFFFFFF on REG OE To set the outputs all to high write OxFFFFFFFF to REG OUT To read the current received value from bus inputs read from REG IN 5 2 Running the Test In the USB Controller program select Settings gt OneShot Test From the di
168. ns instructions for the DN9002K10PCIE8T configuration circuitry including which FPGAs to configure and to which frequency the global clock networks should be automatically adjusted 3 3 Insert the Compact Flash card into the DN9002K10PCIES8T s Compact Flash slot This step involved inserting the CompactFlash card into the DN9002K10PCIEST s CompactFlash slot No further advice is given 3 4 Install DN9002K10PCIES8T in computer If you ate not using the DN9002K10PCIEST in a PCIe Express slot skip this step You may instead choose to host the DN9002K10PCIE8T over USB The DN9002K10PCIE8T is compatible with PCIe Express 1 4 or 8 lane Using the board in a 1x slot will requite an adapter such as those available from Catalyst The rest of this chapter assumes you are installing the board into a Windows XP computer DN9002K10PCIEST and provided software is compatible with other operating systems but there is no corresponding quick start guide 3 5 Cables 3 5 1 Connect RS232 Cable The configuration circuit displays status messages to an RS232 terminal If when something goes wrong with configuration this terminal will output error messages Normally you would only connect this cable when something is not working and you want to debug the problem Use the provided ribbon cable to connect the MCU RS232 port P2 to a computer serial port to view feedback from the configuration circuitry during FPGA configuration R
169. nx application note XAPP858 You can also see the provided DDR2 reference design for example code A basic block diagram of the clocking is given below DN9002K10PCIE8T User Guide www dinigroup com 104 HARDWARE DDR2 SODIMM Module Global Clock G0 Gi G2 REFCLK CLKOUTO CLKFB Note that the DIMM CK2 signal is driven by the FPGA from a 1 8V bank The output should be a DIFF SSTL18 It is received by a global clock pin on the Virtex 4 device To receive the signal use an LVDS EXT input with DIFF TERM attribute set to TRUE The and 2 signals are length matched so this input should be synchronous to the clock input of the DIMM module The DQ and DM signals ate synchronous to the DOS signals in each bank See the DDR2 SODIMM module specification for information on the timing of this interface DOS timing In order to clock the and DM inputs using the DOS signal you must use a BUFIO clock buffer on the DOS signal 16 3 Signaling 16 3 1 Standards DQ and DM signals should use the SSTL18 II drive standard required VREF VRP and VRN connections required for this standard are provided on all DIMM interface banks DN9002K10PCIEST User Guide www dinigroup com 105 HARDWARE DQS signals should use the DIFF_SSTL18_II drive standard External differential termination is provided on these signals at the FPGA DDR2 clock signals should be driven by the DIFF SSTL18 II
170. ock frequency with the UPDATE_CLOCK_FLA current M and values 0x01 is GO 0x02 is G1 0x04 G DF40 is G2 FPGA_COMMUNICATI ON DF39 Disables Main Bus interface TEMP_SENSOR_A DF50 Temperature of FPGA A TEMP_SENSOR_B DF51 Temperature of FPGA B SERIAL_ NUMBER DFFA BOARD_TYPE DFFE 3 5 1 Undocumented controls Most of the accessible registers to control board function used by the AETest_usb and USB Controller programs are not documented in the table above This is because we do not anticipate a need for customer use If there are board features that are accessible through USB Controller or AETEST programs that you feel you need access to in your own PCIe or USB applications contact support dinigroup com and we will provide details on using the interface you require A list of these features is provided below G0 G1 G2 frequencies below 31 MHz Single step clocking on GO G1 G2 clock networks Zeto Delay Daughtercard clock network External clock source selection DN9002K10PCIEST User Guide www dinigroup com 56 HARDWARE Readback of G0 G1 G2 frequency measurements MainBus error counter 3 6 Firmware A Spartan 3 FPGA and a Cypress micro controller control the configuration circuitry The programming data for the FPGA is stored on a flash device and the code for the micro controller is stored on a separate flash device The instructions for updating the firmware are given in the software section The flash that stores the
171. on This LED indicated the configuration and control FPGA is on If this LED is not on it indicates a problem with the board or firmware Check the DONE LEDs of each FPGA When an FPGA is configured a blue LED labeled DONE will glow to the upper left of the FPGA Check the FPGA A user LEDs located just below the FPGA A These LEDs should be active if the Dini Group reference design is correctly loaded Repeat this step for FPGAs B These Yellow LEDs are lined up below the Check the CF activity LED located just below the CompactFlash socket When the board is reading off the CompactFlash during configuration this yellow LED should blink DN9002K10PCIEST User Guide www dinigroup com 15 CONTROLLER SOFTWARE 51 W 1 NEUSS Reset I LED Config active LED should blink when doing configuration Figure 5 LEDs 4 Run USB Controller This section will get you started with USB and show you how to operate the provided software 4 1 Driver Installation When the DN9002K10PCIE8T powers or you connect it to a USB port for the first time the computer will ask you to install a driver DN9002K10PCIEST User Guide www dinigroup com 16 CONTROLLER SOFTWARE Found New Hardware Wizard Found New Hardware Wizard Welcome to the Found New Please choose your search and installation options eS Hardware Wizard This wizard helps you install software for
172. onnect the Platform USB cable for the first time Windows will automatically install a driver three times in a row like a retarded parrot The program scans the chain to auto detect the type and number of FPGAs installed on your board and display them on the screen Right click on an FPGA and select choose configuration Browse to the bit files provided on the user CD For example D FPGA_Reference_Designs Programming_Files DN9002K1 0PCIE8T MainRef LX110 f pga_A bit This JTAG port should also be used for visibility products like Xilinx ChipScope E Boundary Scan Edit View Operations Output Debug Window Help Exe xsx iug 22420 wm ight click device to select operations 8 Scan Rig pe alSlaveSerial 25 SelectMAP 3 Configuration 3 SPI Configuration xc5vix330 xc5vix330 xcSvixSot E PROM File Formatter file file file iMPACT Processes Operations E Boundary Scan INFO iMPACT 1777 Reading C Xilinx virtex5 data xc5vlx330 bsd INFO iMPACT 501 1 Added Device 5 1 330 successfully Manufacturer s ID Xilinx xc5v1x330 Version INFO iMPACT 501 1 Added Device xc5v1x330 successfully PROGRESS END End Operation Elapsed time 1 sec BATCH CMD identifyMPM Transcript B Output Waming Configuration Platform
173. operation and will pause with that indication if thete is an error The primary purpose of these LEDs if for Dini Group to debug it s software so I wouldn t be surprised if this information were outdated already LED Reference LED Signal Name The LED indicates the following when ON Designator Color 0522 25 RED GREEN MCU No meaning DS26 YELLOW HOST ACT No meaning DS70 73 GREEN LED SPARTAN No meaning DS12 RED ERR_TEMP FPGA over temperature DS13 RED ERR_CONF No meaning 16DDR2 There is one DDR2 memory socket interfaces on the DN9002K10PCIEST By convention the name of this interface is DIMMB In this section the interfaces may be called DIMM SODIMM or DDR2 interface interchangeably DN9002K10PCIE8T User Guide www dinigroup com 102 HARDWARE Signal names given in this section and in other documentation ucf files are given in the form DIMMB signal name gt 16 1 Power Each DDR2 SODIMM is capable of drawing 5A of current when in auto precharge mode The DN9002K10PCIEST is capable of providing this amount of current 16 1 1 Interface Voltages The standard DDR2 interface voltage is 1 8V The banks that connect to the DIMM interface are powered by 1 8V and the power pins on the socket is connected to this same power net In a DDR2 interface these signals are driven using the SSTL18 DCI drive standard There are some exceptions listed below DIMM SDA DIMM SCL DIMM CK2 These s
174. osted on the web site In order to obtain them you must request them from support dinigroup com You may be required to perform a firmwate update to your board to receive support and some features When updating firmware you should update in the following order 1 USB Controller exe http www dinigroup com product common USBController zip 2 Configuration FPGA PROM firmware 3 EEPROM option 4 MCU Flash 5 LTX Bitfile hex file 5 2 Updating the Spartan PROM firmware 5 2 1 Using JTAG cable Xinlinx products Connect a Xilinx Platform USB configuration cable to your computer When the cable is working properly but not connected to JTAG chain the LED on the cable turns amber When connected to the DN9002K10PCIEST the LED turns green Connect the cable to the Firmware header 9 Figure 14 Firmware Update Header Power on the DN9002K10PCIE8T When the Platform USB cable is connected to a header the status light turns green Open the Xilinx program Impact usually found at Start gt Programs gt Xilinx gt ISE gt Accessories impact iMPACT Project Dialog will appear please hit Cancel Choose the menu option File gt Initialize Chain Impact should detect 2 devices in the JTAG chain XC381000 and XC18V04 For each item in the chain Impact will direct you to select a programming file for each For the XC3S1000 Press Bypass Impact will then ask for a programming file to program the XC18V0
175. oup recommends placing the IO voltage regulators on the daughtercards because this does not require modification of the DN9002K10PCIE8T 25 3 Rolling your own daughtercard Small quantities of the connectors required for building a daughtercard can be obtained at cost from the Dini Group If you need help designing a daughtercard we will be happy to review your schematic for errors Send it 26Troubleshooting 26 1 The board is dead If the board is not responding at all when connected to a Windows XP computer there is no Dini Emulation Engine in the hardware manager the board may be stuck in reset Check the power failure LEDs If any of them are red then the board is stuck in reset due to a power problem If the failing voltage is 3 3V 5V or 12V then the problem is probably caused by yout power supply Check the voltages of these power rails and make sure they are within at least 5 of their nominal voltages If the power supply was the one supplied by Dini Group make sure that the voltage trim faceplate is connected This faceplate allows trimming the 3 3 5 0 and 12V outputs up and down for performance reasons If the plate is not connected all of the power supply s outputs default to their lowest settings This will cause the Dini Group board to reset due to undet voltage If the board is not in reset the RS232 terminal will be active Connect a computer serial port to this header and open a terminal program on the computer St
176. quency of synchronous communication between FPGAs increases the user must implement more difficult techniques As a general guide these techniques are described below 0 MHz 20 MHz The user should use the Pack the IOBs by using synthesis attributes The output delay for each output and setup time for each input is a known value 100 MHz Use DCMs in each FPGA to eliminate the variation of clock network skew internal to each FPGA The clock must be free running 250 MHz Use DDR clocking and DDR IO buffers 300 Use source synchronous clocking between FPGAs The clock is driven with the data for each bus The receiving FPGA uses the clock signal received on a CC pin to clock the IOs in the bus An IDELAY element on the CC pin input delays the clock with respect to the data by a fixed amount to allow some setup time 550 Use the 5 build in ISERDES and OSERDES modules 600 Use Virtex 5 PLL devices to reduce cycle to cycle jitter on the clocks 700 MHz Individually de skew each bit using IDELAY elements Use a training pattern or hard code the correct delay values for each input 800 MHz Use LVDS signal standard 900 MHz dynamically de skew each bit to account for temperature and voltage variation 1 GHz Highest speed grade parts are required Note that for speeds above 550MHz you must use the ISERDES and OSERDES modules adding latency to your interconnect At speeds greater than 500MHz there is more than o
177. r File Edit FPGA Configuration FPGA sinbus SettingsfInfo Service eference Design 1 Refresh Enable USB gt FPGA Com Clear Log BOARD TYPE DN9002k10PCIe8T USB to FPGA communication disabled Enable if you want to use reference design features MCU FLASH VERSION 1 27 BOARD TYPE DN9002k10PCIe8T SPARTAN CONFIG FPGA VERSION 0x22 Board Serial Number 0708024 Figure 6 USB Controller Window This window will appear showing the current state of the DN9002k10PCIES8T If FPGA configured next to each FPGA a blue light will appear 4 2 1 Configure an FPGA Even though the reference design should already be loaded because you had a Compact Flash card installed when the board powered on let s configure an FPGA over USB Clear an FPGA of its configuration right click on an FPGA and selecting from the popup menu Clear FPGA The blue light above the FPGA on the GUI and on the board should turn off To re configure that FPGA using the USB Controller program right click on the FPGA and select Configure FPGA via USB from the popup menu The program will open a dialog box for you to select the configuration file to use for configuration Browse to the provided user s CD D FPGA_Reference_Designs Programming_Files DN9002K10PCIES8T MainRef LX330 fpga_a bit If you are configuring an LX220 or LX110 device you should select a bit file from the LX220 or LX110 directories instead Failing to select
178. r Guide www dinigroup com 169 ORDERING INFORMATION 4 3 3 Export control classification number ECCN EAR99 4 4 Mission Critical DN9002K10PCIEST and supporting hardware and software are not intended for use on human subjects that you like in life support mission critical systems or aviation DN9002K10PCIEST User Guide www dinigroup com 170
179. r bit map 15 as follows DN9002K10PCIEST User Guide www dinigroup com 61 HARDWARE OxDF28 4 0 23 51 50 PLLSEL CLKSEL Write value 0x02 to select the daughtercard Write value 0x01 to select the FBA clock 4 4 2 EXT1 This clock can be sourced only from daughtercard By default is set to be sourced from the daughtercard B 4 4 3 Daughtercard zero delay mode and EXT1 can be set to zero delay mode where each FPGA is able to receive the clock synchronous to the daughtercard This feature requires configuring the clock distribution network with the frequency of the clock The interface by which the user can do this is not defined Contact support dinigroup com if you require this feature Before you attempt to implement this daughtercard clocking method consider using the DCA and DCB clock pins on each daughtercard to directly clock the daughtercard s associated FPGA rather than to drive the global network 4 4 4 SMA input The 0 clock can be sourced from a pait of SMA inputs 11 J13 These SMAs connectors are designed to connect to a differential clock source ign X J11 CONN SMA LIGHTHORSE SASF546 P26 X1 CLK USERpc CLK USERnc H 5 z J13 CONN_SMA LIGHTHORSE_SASF546 P26 X1 The inputs are AC coupled This limits the minimum possible frequency of the clock input to around 4 kHz If you require an external clock with a frequency lower than this
180. r signal DN9002K10PCIEST User Guide www dinigroup com 161 THE REFERENCE DESIGN The G2 clock is required to be 200MHz or IDELAY will not calibrate correctly and performance will be degraded 11 3 Running the Test In the USB Controller program select Settings gt OneShot Test From the dialog box check the Interconnect Test box The program will automatically load the bit files set the clocks and run the test 12PCle Interface Reference Design The PCle reference design is an example of how to use the provided LX50T_module_interface module provided 12 1 Provided Files 12 2 Using the Design The PCIe reference design maps internal FPGA block rams to BAR 1 through BAR6 of the FPGA s PCIe interface and a separate block ram to the DMA channel of the PCIe interface When the design in loaded the FPGA a host machine can read and write to this memory space to verify the interface is working Only 4KB of memory is mapped to each BAR even though the size of each BAR is larger The block ram memory will wrap 12 3 Running the Test The PCIe Reference Design is an FPGA A only design that implements the pcie_x8_user_interface module described the document D FPGA_Reference_Designs common PCIE_x8_Interface pcie8t_user_interface_manual pdf This design implements a PCle target access and DMA interface to a block ram inside FPGA A source code is located on the CD at D FPGA_Reference_Designs common PCIE_x8_Interf
181. reprogram the LXT FPGA This information is stored in an SPI flash device on the boatd 5 5 1 Using JTAG USB cable Xilinx products To install this updates plug the USB JTAG cable into the header marked FPGA JTAG on the left edge of the board DN9002K10PCIES8T User Guide www dinigroup com 42 CONTROLLER SOFTWARE When you scan the JTAG chain you will see the entire user FPGAs of device type LX110 LX220 or LX330 In addition the last device in the chain will be either a LX50T or FX70T device Right click on this device and choose select configuration file Dini Group will provide a file possible called Q bit Select that thing Then right click on the last device again and select add SPI or BPI prom Then select a firmware file that might be called _ q mcs that Dini Group will provide The program will then for some reason ask for what type of prom you have The correct answer is AT48DB642D Now the picture with the six FPGAs will have a little picture of an SPI prom attached to the last FPGA Right click on this and hit configure Then wait a little while The SPI prom that is connected to the LX50T FPGA is where the LX50T FPGA pets its load file The LX50T FPGA can be programmed directly using a bit file but then it will lose its configuration once the board is reset When you program the SPI flash it will keep its configuration when the board is reset A bit file is used to
182. responds to G1 and bit 2 corresponds to G2 change the source to the stop clock write a 1 to the bit location corresponding to the clock network Then write a 1 to the bit corresponding to the clock network in the update register 40 Writing to this register will cause a glitch in the clock DN9002K10PCIEST User Guide www dinigroup com 58 HARDWARE From the compact flash card source can be set by using the source instruction source 2 sets GO to step clock 0 source G1 2 sets G1 to step clock 1 soutce G2 2 sets G2 to feedback In USB Controller from the settings menu select DN9002K10PCIES8T clock source settings The possible sources of the G2 clock are the synthesizer or FPGA A using the FBA INT signal FPGA drives this 3 3V LVCMOS clock signal From the USB Controller program you can select INT from the settings gt DN9002K10PCIE8T clock source dialog This setting can also be achieved over PCIe or USB using the configuration registers 4 3 1 Clock Synthesizers The GO G1 and G2 clock synthesis source is driven by an 515326 clock synthesizer chip This chip is capable of driving a wide range of output frequencies The configuration register that allows selecting the output frequency supports each multiple of 0 125MHz up to 550MHz If the desired frequency is between one of these steps or in the Khz range then you will have to use a compact flash card to
183. rface and use these signals for general purpose FPGA interconnect To allow this by default the main bus is disabled and the Host interface USB in this case is prevented from operating it To override this setting hit the Enable USB gt FPGA communication button near the top of the window To read data from the FPGA design the Dini Group reference design select from the menu MainBus gt Read In the resulting dialog box enter 080000000 in the Start Address box and 10 in the Size box Press OK and then DONE The result of the read is printed to the USB Controller log window FPGA READ ADDRESS DATA 0 08000000 Oxdead5566 0 08000001 0 00000000 0 08000002 0 05000135 0 08000003 Oxffffffee 0x08000004 0x34561111 0x08000005 0x00000001 0x08000006 0x00000000 0x08000007 0x00000000 Figure 8 USB Controller Log Output The address 0x080000000 is by MainBus convention assigned as part of the space available for implementation by FPGA on the DN9002K10PCIEST If FPGA is not loaded with the Dini Group reference design or a design that implements the MainBus slave then all address reads will return Ox DEADDEAD DN9002K10PCIEST User Guide www dinigroup com 20 CONTROLLER SOFTWARE 5 Communicating over the Serial Port You may want to communicate with your design over the user serial port P1 The MainRef reference design that you already loaded has an asynchronous loop back on
184. rs on it automatically loads Xilinx FPGA design files ending with a bit extension found on the CompactFlash card in the CompactFlash slot into the FPGAs using the main txt file as a guide 3 6 View configuration feedback over RS232 As the DN9002K10PCIE8T powers on your RS232 terminal connected to P2 will display information about the Configuration process If FPGAs ever fail to configure using the Compact Flash card this is the best place to look for help A typical RS232 power on session is given below Rebooting from FLASH please wait This line has to do with the firmware update mode Seting The board is setting the global clock frequencies according to the N 01 M 000001000 li DONE main txt file on the CompactFlash card The messages here Setting G1 mostly only useful to whoever programmed the firmwate N 01 M 000001000 DONE Setting G2 N 01 M 000001000 Prints the FPGAs the configuration circuit thinks you have on your DONE board DN9002K10PCIEST User Guide www dinigroup com 13 CONTROLLER SOFTWARE DN9002K10PCIE8T MCU FLASH BOOT FPGAS STUFFED AB COMPACTFLASH INFO MAKER ID EC DEVICE ID 75 SIZE 32 MB FILES FOUND ON COMPACTFLASH CARD FPGA B BIT A BIT MAIN TXT CONFIGURATION FILES FPGA A BIT FPGA B B BIT OPTIONS Message level set to default 2 Sanity check is set to default ON
185. s 157 6 1 PROVIDED EE EE E EEE EEA 158 6 2 JUSING THE DESIGN ae EE R AEA 158 6 3 RUNNING THE 158 7 lt CEOEK COUNTERS tas 520 ie ee Fons repa ie 158 B A D ro EEA E E E IEEE 158 9 SIMULATING THE REFERENCE DESIGN 159 10 COMPILING THE REFERENCE DESIGN e eeeeee seen een seen s ens ens ena n 159 10 1 1 Xilinx Embedded Development Kit EDK sss 159 10 1 2 onde cal T is coser 159 10 1 3 Build Utility Make bat esee teens 160 10 2 BITGEN OPTIONS 160 11 LVDS REFERENCE DESIQGN e eeeeeee eene tenen etta etes a ens ens ens ens ens eos eoa n 161 11 1 PROVIDED entren tenen tese 161 11 2 USING THE 0 0400 0 0 0 A E A T 161 11 3 RUNNING THE 162 12 PCIE INTERFACE REFERENCE 162 12 1 PROVIDED FILES ciet tit re eerte rre eter e eter er ei nre eres end 162 12 2 JISING THE DESIGN elei ert e 162 12 3 RUNNING THE
186. set the frequency On the provided compact flash card there is a table giving the command to set a clock to any of a large number of intermediate frequencies The main txt syntax is Soutce G1 1 lt a gt lt b gt lt c gt lt d gt lt e gt Where lt a gt lt b gt lt c gt lt d gt and lt e gt are arbitrary parameters given in the table The correct value of the five parameters for select frequencies are given below 0 003000 MHz 7 29393 1599 7 146969 0 005000 MHz 1 960 23 6 96999 0 010000 MHz 1 969 23 6 48499 0 015734 MHz 6 44035 2178 3 44035 0 024000 MHz 5 22453 999 5 22453 0 032000 MHz 3 10825 374 3 21651 H 0 032768 MHz 7 63915 3478 7 13455 0 038400 4 15787 624 4 15787 H 0 044100 MHz 7 139971 7618 7 9997 0 048000 MHz 7 9185 499 7 9185 0 050000 MHz 1 969 23 6 9699 0 060000 MHz 3 5773 199 3 11547 0 075000 MHz 2 10777 349 2 10777 H 0 076810 MHz 5 168385 7498 5 7015 H 0 096000 MHz 5 5613 249 5 5613 0 100000 MHz 1 969 23 6 4849 DN9002K10PCIEB8T User Guide www dinigroup com HARDWARE dk XE XE XE XE XE XE XE XE Xdb Xt db XE XE XE XE Xt db Xt dt db Xt db XE XE db XE Xt db XdE db XE XE Xt dk Xt db db dt db XE dto Xdb xtodt DN9002K10PCIES8T User Guide 0 150000 MHz 0 176400 MHz 0 192000 MHz 0 220000 MHz 0 325000 MHz 0 440000 MHz 0 455000 MHz 0 880000 MHz 1 843199 MHz 2 457600 MHz 3 276800 MHz 3 579545 MHz 3 686399 MHz 4 096000 MHz 4 194304 MHz 4 433617 MHz 4 915200 MHz 6 1
187. so provided of the DN9002K10PCIEST as a whole board along with DDR2 modules headers and the MainBus interface 10Compiling the Reference Design The MainRef reference design for which bit files are included on the user CD and the provided CompactFlash card can be found on the user CD here D FPGA_Reference_Designs common DDR2 controller_ver common DDR2 ddr2_to_mb DN9002K10PCIE8T MainRef source The top module is D FPGA_Reference_Designs DN9002K10PCIES8T MainRef source fpga v This module includes all of the other required sources and expects the directory structure found on the CD 10 1 1 Xilinx Embedded Development Kit EDK The DN9002K10PCIEST does not use the EDK because it has no embedded processor 10 1 2 Xilinx ISE Xilinx ISE version 9 1 service pack 1 or later is required to use the reference designs Earlier versions may work but are not supported If you using a third party synthesis tool you can create a new ISE project file and add the edf as a source For part type select the type of FPGA installed on your board Make sure to add the provided ucf file to the project or the produced place and route will not work Run the map implement and generate steps DN9002K10PCIEST User Guide www dinigroup com 159 THE REFERENCE DESIGN 10 1 3 The Build Utility Make bat If you ate not using a third party synthesis tool then you should use the provided batch script to generate th
188. t 8 Load V5T Prom with filename hex and enter the file name hex 3 process takes about 1 2 minutes DN9002K10PCIEST User Guide www dinigroup com 45 Chapter 4 Hardware 4 General Overview The DN9002K10PCIE8T ASIC emulation platform is optimized for providing the maximum amount of interconnect between the Virtex 5 FPGAs It is the highest density off the shelf development board using the Xilinx Virtex 5 FPGA Below is a block diagram of the DN9002K10PCIE8T A B O T 9 9 Rias MICTOR DRAM SRAM RS232 2 ALL FPGAs FLASH RLDRAM DDR3 DDR2 SODIMM 4GB Max 9 9 9 ALL FPOAs 64 64 250MHz DONPACT Configuration 7HWOSZ 8 1 Spartan 3 FPGAB Virtex 5 LX110 LX220 or LX110 LX220 or 2 Contra ret FF1760 FF1760 Global Clocks ZHMWOSZ IET Virtex 5 FF655 PCle Controller 1 8 2 5V 3 3V 41 8V 2 5 43 3V PCI Express PCI Express 1 1 2 5 Gb s or POU Express KO S U GIb s WIN EXT LVDS when paired but can be run single ended 1 1 baughtereard BB gt gt 1 1 48 2 Ee sous UN nerak DN9002K10PCle 8T iss 8 lane PCle Express ASIC eal Prototyping Board with Virtex 5 FPGAs Block Diagram 1 0 1 1 Marketing The following is the advertised feature
189. taining Virtex 5 Reference Designs User manual PDF Board Schematic PDF USB program usbcontroller exe PCIe program Aetest exe Source code for USB program PCIe program and DN9002K10PCIE8T firmware Board netlist and Virtex 5 simulation model 1 1 System Requirements To compile Verilog designs for Virtex 5 ISE 9 2 may be required To use the provided controller software you need any Windows XP computer with USB 2 0 DN9002K10PCIE8T User Guide www dinigroup com CONTROLLER SOFTWARE Although firmware updates can be completed without a Jtag cable board recovering after failed update required a Jtag cable for ISP If you don t have a Jtag cable you can ship the board back for recovery 2 Warnings 2 1 ESD The DN9002K10PCIEST is sensitive to static electricity so treat the PCB accordingly The target markets for this product are engineers that are familiar with FPGAs and circuit boards Howevet if needed the following web page has an excellent tutorial on the Fundamentals of ESD for those of you who are new to ESD sensitive products http www esda ore esd fundamentals html Figure 2 Bzzzzzzt There are two large grounded metal rails on the DN9002K10PCIEST The user should handle the boatd using these rails as they are much less ESD sensitive than any other point on the board The 400 pin connectors are not 5V tolerant Very few exposed surfaces on the board are tolerant of voltages greater
190. ted mounting holes 2 m N N LO 262 743 140 105 05 225 98 3 5 55 6 273 304 4 This view of the 9002 10 8 daughter card locations is from the top of the looking through to the bottom side The Dini Group standard daughtercard DNMEG_OBS400 is compatible with the DN9002K10PCIE8T DN9002K10PCIEST User Guide www dinigroup com 133 HARDWARE The mounting holes are designed to be used with 14mm M3 standoffs Dini Group has available appropriate mounting hardware on request Standoffs Male to Female Part 1789 Harwin R30 3001402 Mouser 855 R30 3001402 M3 x 14mm HEX 5mmA F Harwin Metric Spacers RoHS Compliant Box 100 Big Round Nuts Part 1787 LMI HN4600300 M3 x 0 5mm Screws Part 1788 MPMS 003 0005 PH Digi key H742 ND SCREW MACHINE METRIC PH M3x5MM With this host plate daughter card arrangement there is a limited Z dimension clearance for backside components on the daughter card This dimension is determined by the daughter card designer s part selection for the MEG Array receptacle X X GND SIG Note that the components on the topside of the daughter DN9002K10PCIE8T face in opposite directions 25 1 2 Standard Daughtercard Size The daughtercard mechanical provisions on the DN9002K10PCIEST are designed to mount a hypothetical daughtercard with the dimensions given
191. ters The full list of registers is in the Configuration Section section To write to a configuration register use the VR MEMORY MAPPED vendor request The direction is OUT The value field is the address you wish to write to example 0xDF39 the disable Main Bus register The size field should be 1 The buffer should contain a single byte containing the byte to be written to the Configuration Register All configuration registers are one byte 6 2 6 Other Vendor Requests Many of the Vendor requests used by the USB Controller program are not documented Dini Group does not support these requests for users If you need a function that you feel is not described here contact support dinigroup com 6 3 Main Bus accesses The USB Controller control the DN9002K10PCIE8T reference design using USB vendor requests and bulk transfers that access the configuration FPGAs registers These registers cause DN9002K10PCIEST User Guide www dinigroup com 73 HARDWARE Main Bus transactions with the user FPGAs The host computer initiates all Main Bus transactions To see a specification of the Main Bus interface see Reference Design To request a Main Bus interface write transaction the USB Controller program sends a USB bulk write to EP2 endpoint 2 The first byte contains a code either 0x00 or 0x01 determining whether the next 4 bytes contain an address or a datum If this byte is a 0x00 the next 4 bytes in the bulk transfer are stored into
192. the correct type of bit file will result in the USB DN9002K10PCIES8T User Guide www dinigroup com 18 CONTROLLER SOFTWARE Controller program to warn you and the FPGA fail to configure The program will report the status of the configuration when it finishes DONE did not go high This refers to the DONE selectMap signal which is asserted by the FPGA when it is properly configured If you ate configuring FPGA or FPGA Q you should select fpga_b bit or fpga q bit instead Should you configure the wrong FPGA with the wrong bitfile the FPGA will succeed to configure but probably won t function properly This is not recommended because it could lead to bus contention and excessive heat generation Done FPGA B cleared successfully FPGA A cleared successfully Doing a sanity check Sanity Check passed Configuring FPGA B via USB please wait File D dn_BitFiles DN9002K10PCIE8T MainRef LX330 fpga_b bit transferred Configured FPGA B via USB Figure 7 USB Controller Log Output The message box below the DN9002K10PCIEST graphic should display some information about the configuration process When the configuration is successful the green LED should re appear next to the FPGA 4 2 2 Set Clock Frequencies To change the clock frequencies of GO G1 or G2 select the Clock settings option from the Settings menu A dialog box appears asking to which frequency you would like to set each clock Enter
193. there is some user command Items that may be updated when the refresh button is hit Type of board connected DN9002K10PCIEST in this case Number of FPGAs installed Whether or not the FPGAs configured blue DONE LED on off Whether the Dini Group reference design is loaded in one or more FPGAs disable enable the FPGA Reference Design menu Check whether USB is enabled 1 1 2 Disable Enable USB Jration FPGA Reference Design Mainbu Disable USB gt FPGA communicate to the FPGA design using USB the Bus interface is used See the hardware chapter for more information on this interface Some users elect not to use the Main Bus for USB communication To allow these users to make use of the signals in the Main Bus for their own purposes the USB Controller is careful not to use the Main Bus unless explicitly given permission by the user The user can give permission to use Main Bus by pressing the Enable USB gt FPGA communication button It can revoke that permission by pressing the Disable USB gt FPGA communication button When the DN9002K10PCIE8T powers on it begins in the disabled state The state is stored on the board so that multiple programs accessing the DN9002K10PCIE8T may prevent each other from using the Main Bus 1 1 3 Log Window This text box prints the result of each user command in USB Controller There is a clear log button to cl
194. this option and request xsvf file from us 1 Runaeusb wdm exe or aeusb linux 2 At the main menu please select option 3 FPGA Configuration Menu 3 In Flash Boot Menu please select option 9 Note the option menu is not displayed for security purpose 4 Please enter the full path filename for the xsvf file 5 Verbose level is O The higher verbose level the slower the program runs DN9002K10PCIEST User Guide www dinigroup com 38 CONTROLLER SOFTWARE C DiniWork Aetest_USB aetest_usb aeusb_wdm exe Display Flash Version Check FPGA configuration status Configure FPGA via smartmedia Configure FPGA individually via USB Configure FPGA from configuration file Set PowerPC RS232 Multiplexing Clear All FPGAs Read PowerPC RS232 Multiplexing Load UST Prom with filename hex Toggle Sanity Check Main Menu 02 Quit Please select option 9 You are about to run command that change Spartan s prom Do you want to continue gt Please enter filename C DiniWork dn_conf ig Conf igFPGA DN 666k16 prom_f lp xsvuf Please enter verbose level 4 gt Om Figure 16 aetest_usb window 6 The progress will start from 0 to 100 This will take long time to complete 10 minutes Please do not disturb the process 7 Power cycle the board when finish You can also use commend line aeusb_wdm_cmd exe XSVF lt filename xsvf gt or aeusb linux cmd exe XSVF lt filename xsvf
195. this port If you want to test this connection connect an RS232 terminal to the header and type stuff The port should echo back that very same stuff 6 Run AETest wdm The program provided to access the DN9002K10PCIE8T over PCIe is called AETest It is located on the user CD D PCle_Software_Applications Aetest aetest aetest_wdm exe If you are running Linux or Solaris you must compile AETest before continuing this quick start guide This involves installing the kernel source packages on the computer then loading a kernel module somehow Details in the Software Chapter The rest of this guide assumes you using Windows XP After you turn your computer on the computer will display a dialog asking for the driver for a Dini Group board with Virtex 5 PCI Express Click Choose a driver to install gt Click Have Disk and browse to D PCle_Software_Applications Aetest wdmdrv drv dndev inf 6 1 1 Use AETest The AETest application should display its main menu bjx Symbolic link is pe iftven_1 df amp dev_1900 subs ys_19001 7df amp rev_01 483022 4e6 38080 04 Be 3t lt f Obida2 6bac 4d1f 9eb80 1daf1b7e7131 Got ConfigFPGR Found Device 174 41900 name DN9882Ki1BPCIEST VirtexS PCI Express 8 lane Board SP_INTERFACE_DEVICE_DATA available for this GUID instance 1 Compiled on Sep 25 2008 at 16 02 05 press any key Figure 9 start up menu D
196. tivity and the Green LED indicates link in gigabit The LED DS64 located next to the RJ45 connector indicates link in 100Mbit mode The 10Mb link LED is not configured Hot plug is acceptable on a 1000Base T connection The Ethernet PHY works with the Xilinx Ethernet IP but only in 10 and 100Mbit modes Gz a VSCB401 LED1 LED1 0826 1 1 23 BEL 01810 The above schematic clipping is useless but looks cool and technological 19 2 JTAG The VSC8601 device is attached to a JTAG chain The schematic clipping showing this connection is given below DN9002K10PCIEST User Guide www dinigroup com 118 HARDWARE R211 R206 R220 4 7K 4 7K 4 7K 3 3V I don t know why you would need access to this It isn t tested or thought about ever This JTAG chain does not connect to the FPGA JTAG chain 20 small EPROM 1K is attached to FPGA These devices intended to store identification data for generating a unique MAC address for the Ethernet interfaces However the EPROM can be used for any user defined purpose requiring static memory intensive tasks like remembering your name and birthday The interface to the EPROM is a standard at 1 8V The address of the devices is binary 1010 000 The maximum clock speed of the IIC interface is 400 kHz DN9002K10PCIEST User Guide www dinigroup com 119 HARDWARE 24C64C SOIC 127P600 8N R8 R7 R6 4
197. to support dinigroup com 10 2 User Reset The USER RESET circuit is intended for use by the user When this reset is asserted the RESET signal from the schematic is asserted to each FPGA After at least 200ns this signal is de asserted simultaneously to each FPGA This signal is connected to a regular user IO on the FPGA so it is up to the FPGA designer to implement reset correctly within his design The User Reset is asserted whenever the User Reset button is pressed This button S2 is located just above the USB connector There is no LED indicating the state of user reset User reset is also asserted when the reset vendor request is sent over USB When User reset is asserted the RSTn signal to each daughtercard is also asserted The rise time of the reset signal is fairly slow 10s of nanoseconds and the delay within the FPGA of the reset signal cause the actual de assertion time of the logic within the FPGA to be uncertain by as many as 20ns the timing of a synchronous reset within a single FPGA is guaranteed This means that if this signal is used to reset circuitry used for inter FPGA communication cate needs to be taken that a synchronous reset is not required for the multiple FPGA system to operate correctly Alternately you design can re generate a synchronous reset and distribute this signal using a MB signal 11JTAG There are two JTAG headers on the DN9002K10PCIEST The first is used only to update the boar
198. to be in an ambient temperature of 35 degrees In a closed computer case the ambient temperature will increase 22 7 1 Fans The fan units attached above the heat sinks are powered by 5V Each fan has its own power connector 22 7 2 Removing Heatsinks The heat sink fan assemblies are attached using a plastic clip There is a thermal interface material between the FPGA and heat sink that is slightly adhesive Forcibly removing the heat sink will not damage the FPGA DN9002K10PCIE8T User Guide www dinigroup com 126 HARDWARE 22 7 3 Fan Tachometers Each FPGA fan has a tachometer connected to it for the detection of fan failure If you intend to use this system in a rack or production system you may want to monitor the fans FAN E TACH 512 as CC 90 2 aaa LIP CC 52 HET peed LIN CC 2 Bane L2P A23 2 PARTO L2N 22 0 L3P A21 2 ae L3N A202 E FCS B2 Er LAN VREF FOE 2 Fen DPFWEB2 ao LN CSO B2 FPGA E 16 072 cx 05 2 76 052 29 5 1760 L7N D42 15 14 v 2 2 5 3 2 The fan tachometer inputs AH16 can be LVCMOS25 The fan will produce 2 rising edges revolution You may need to de bounce the signal if you intend to count the fan frequency 23Connectors This section provides a list of all connectors on the DN9002K10PCIE8T Items considered test points including t
199. to know detailed information about them The interfaces relating to these connectors are functionally described in the manual section indicated The FPGA indicated is used to access the connector s interface but is not directly connected to the connector pins Non User connectors Reference Manufacturer Part Number Connector description FPGA Manual Section J12 13 Lighthorse L TI SASE546 P26 X1 SMA Jack differential ALL Clocks J14 Molex 538 53856 5070 CompactFlash socket CompactFlash DN9002K10PCIE8T User Guide www dinigroup com 128 HARDWARE Molex 67068 8000 S1 Omron B3S 1002 J7 Molex 87832 1420 X2 Keystone 3001 1 Belfuse 0826 1X1T 23 F 16 NONE NONE P4 NONE NONE J10 Molex 87832 1420 P3 Samtec TSM 136 01 T DV 18 1 641737 1 P1 Molex 39 29 9202 23 2 1 Comments USB Type B female USB Hard Reset Pushbutton Reset 14Pos 2MM JTAG header ALL JTAG Coin Batter retainer Encryption RJ45 w LEDs B Ethernet 0 1 pitch mounting positions Ethernet PCIe 64 bit edge connector A PCle 14Pos 2MM header Dual row 0 1 RS232 header RS232 20 pin Vertical ATX Power 4 pin Right angle Hard drive Power If you have a board with fewer than 2 FPGAs installed connectors with missing FPGAs as associated FPGA will also not be installed 23 3 Not For Use Connectors The following connectors are not intended for use by the user Not for use Connectors Reference Manufacturer Part Number Y5 Gompf 93340115
200. top 2 2 2 Physical Dimensions By convention the board is oriented as shown in the above board photo with the top of the board being the edge near the DDR2 SODIMM The right edge is near FPGA The left side is the side with the PCIe bezel side refers to the side of the PWB with FPGAs and fans the back side is the side with the two daughtercard connectors The reference origin of DN9002K10PCIEST User Guide www dinigroup com INTRODUCTION the board is the center of the lower PCIe bezel mounting hole Physical dimensions are given in millimeters 2 2 3 Part Pin Names References to individual part s pin are given in the form lt X gt lt Y gt lt Z gt The lt X gt is one of U for ICs R for resistors C for capacitors P or J for connectors FB or L for inductors TP for test points MH for mounting structures FD for fiducials for sockets DS for displays light emitting diodes F for fuses PSU for power supply modules Q for discrete semiconductors RN for resistor networks G for oscillators X for sockets Y for crystals and the PCle bezel lt Y gt is a number uniquely identifying each part from other parts of the same class lt Z gt is the pin or terminal number or name as defined in the datasheet of the part Datasheets for all standard and optional parts used on the DN9002K10PCIEST are included in the Document library on the user CD 2 2 4 Schematic Clippings Partial sch
201. ts at these inputs are referenced to the voltage on the pin VREFP On the DN9002K10PCIEST this voltage is generated by a high precision external voltage reference IC The primary ADC input is routed to a differential test point There is one test point labeled ADC for each FPGA Some of the auxiliary inputs to the ADC are routed to the Mictor connector on FPGA B This could be used for something I guess 10Reset There are two reset circuits on the DN9002K10PCIES8T One is the power on reset or Hard Reset that holds the board including the configuration circuitry in reset until all power supplies on the boatd are within their tolerances The second reset citcuit is the user reset or Soft reset 10 1 Power Reset The power reset signal holds the configuration circuit including a micro controller and Spartan 3 FPGA in reset It also causes the FPGAs to become un configured and causes the RSTn signal on the daughtercards to be asserted When the board is reset the Hard Reset LED DS85 is lit red It is located about an inch above the USB connector When the board is in reset FPGAs cannot be configured USB does not function the host computer will not be able to communicate with the device PCle cannot access the FPGA or configuration functions the device will still be accessible from PCIe and LX50T registers can still be read and written When in reset the Spartan configuration FPGA remains configured
202. ulation board and provide data transfer to and from the User design All AETEST source code is included on the CD ROM shipped with your DN9002K10PCIE8T Logic Emulation kit AE TEST can be installed on a variety of operating systems including Windows 2000 XP Vista Windows WDM linux 3 1 Functionality All communication to the board using this program is over PCI express In this way the basic functionality of PCI Express is tested The AETEST utility program contains the following tests DMA and BAR accesses over PCI Express When using the full function deisen for LXT DDR2 Memory Test Flash Test Daughter Card Test requires special fixture that we didn t give you AETEST also provides the user with the following abilities Recognize the DN9002K10PCIE8T Display Vendor and Device ID Set PCIe Device and Function Number Display all configured PCIe devices Various loops for PCIe device function and ID numbers DN9002K10PCIEST User Guide www dinigroup com 33 CONTROLLER SOFTWARE Write and Read Configuration DWORD for board settings Access to the Main Bus interface BAR Memory operations Configure Save BARs from to a file Configure FPGAs 3 2 Running AETEST The following images show a terminal session in Windows XP Symbolic link is pe iffven_17df amp dev_1900 amp subs ys_19001 7df amp reu_ 114830224e6 38080 De3l c b da27 6ac7 4d1f 9eb8 1dafib7e7131 Got ConfigFPGR id
203. un a serial terminal program on your PC On Windows you can use HyperTerminal Start Programs Accessories gt Communications HyperTerminal and make sure the computer serial port is configured with the following options DN9002K10PCIEST User Guide www dinigroup com 12 CONTROLLER SOFTWARE Bits per second 19200 Data bits 8 Parity None Stop Bits 1 Flow control None Terminal Emulation VT100 or None if available 3 5 2 Connect USB Cable Use the provided USB cable to connect the DN9002K10PCIEST to a Windows computer Windows XP is recommended If your board is installed in a PCIe slot the USB host is allowed to be the same computer as the non USB host The two computers should be connected to the same power outlet if this is the case 3 5 3 Connect Power cable The power cable connected to 6 is required If you do not plug a cable in here the board will not power on This is true whether or not the board is installed into a PCI Express slot Most new power supplies have a 6 pin PCI Express Graphics power connector If yours does not use the provided adapter cable If you are operating desk top and not in a motherboard then your power supply might not turn on if the 20 or 24 pin motherboard power connector is not connected to anything In this case connect the provided PSU starter to the PSU Power On Turn on the ATX power supply USB Hosting or the computer PCIe hosting When the DN9002K10PCIE8T powe
204. voltage levels on the board to ensure they within tolerance If they fall out of tolerance above or below voltage the board will enter a reset state These tolerance ranges are listed below 10V 0 95 to 1 21 18V 1 65 to 3 00 25V 2 20 2 90 33V 2 89 to 4 00 50V 3 99 to 6 02 The voltage monitors filter the voltage at a frequency of about 1KHz The following voltages are not monitored 1 2 VCCO VCCO VCCO B2 DIMM VTT DIMM_VREF DN9002K10PCIEST User Guide www dinigroup com 125 HARDWARE When a power supply voltage falls out of tolerance the board is put in reset the SYS_RST signal is asserted and SYS_RSTn LED glows and an LED along the right hand side of the board will light to indicate which power rail has failed The voltage levels are measured with a RC filter time constant of around 1KHz This means transient voltage spikes may not trigger a board reset 22 7 Heat The maximum power dissipation supported for each FPGA is 25W Using the provided heat sink and fan assemblies FPGAs will remain under the maximum recommended junction temperature 85 degrees C If your design exceeds this limit you can assume the temperature of the device rises 2 degrees for each watt above this amount your design uses Put this number in the settings of the timing analyzer Power requirements of a design can be estimated using the power estimator tool in ISE 9 1 For this calculation the board is assumed
205. zability of the LXT is provided DN9002K10PCIEST User Guide www dinigroup com 85 HARDWARE 7 3 8 1 LEDs Six LEDs are controlled by the PCI Express FPGA Activity Link1 Link4 Link8 and PERSTn GEN2 and LOS PERSTn directly shows the state of the PCI Express reset signal from the host This is typically only during power on Activity is generated by the PCI Express FPGA whenever a packet is received This signal on certain Intel based hosts may blink constantly because of some mysterious configuration register read that gets generated all the time The Link1 LED will only be active when the PCI Express LED is communicating without error to a link partner with a 1x negotiated lane width The Link4 LED will only be active when the PCI Express LED is communicating without error to a link partner with a 4x negotiated lane width The Link8 LED will only be active when the PCI Express LED is communicating without error to a link partner with a 8x negotiated lane width When the PCI Express LED has negotiated a 2x link both Link 1 and Link8 will light How did you manage to link in 2x mode Send your interesting anecdotes to support dinigroup com The LOS LED will light when there is no recetver detected on lane 0 or when some other thing isn t working Gen 2 will light if the design has linked at 5 0 Gbs 7 3 8 2 FPGA initiated DMA The DMA controller is capable of issuing PCI Express transactions initiated from the FPGA A The

Download Pdf Manuals

Related Search

DN9002K10PCIE8T

DN9002K10PCIE8T

Contents

Download Pdf Manuals

Related Search

Related Contents