Dini Buses User FPGA Design Manual
Contents
1. lt td ma2_tohost 4 User Interface Modules 4 1 IO Modules Interfaces Each connection between the config FPGA and a field FPGA uses some type of physical interface eg NMB DDR PCle SFB etc Each one of those physical interfaces has a related set of IO modules one for the configFPGA and one for the user FPGA A wrapper is provided for the user IO module and the pcie_interface module so that the user can have a single module that translates between the physical signals and the recommended Target DMA interface An example top level design is also provided showing how to connect a user design to the interface module 17 Figure 12 Detail of NMB example design pcie_nmb_user_fpga nmb_user_interface user_design nmb_user_io pcie_interface target_fromhost NMB_FROMHOST ISERDES et dma0_fromhost dma1_fromhost dma2_fromhost e t target_tohost OSERDES Lo dma0_tohost a dma1_tohost t dma2_tohost lt _NMB_TOHOST 4 2 NMB Interface The most common bus architecture for the Marvell based boards uses the NMB bus NMB is a high speed LVDS SERDES point to point bus for communication between the configFPGA and the user FPGAs NMB is not shared with configuration signals NMB uses 10 signals 8 bit data 1 bit control 1 source synchronous clock in each direction with a maxim2. 4 dna_to host almost ful 11 e dma from _host_valid is asserted at start of transfer and remains asserted while transfer data control is active 64 bit address is transferred on first valid cycle dma_from_host_ctrl 5 0 2 b01 dma_from_host_ctrl 4 should be O to indicate write Length data is transferred on second valid cycle dma_from_host_ctrl 5 0 2 b10 Return data will begin some time later there is no latency requirement on reads dma_to_host_valid will be asserted when read return data is valid Figure 6 DMA Read Throttling 4 dma_to_host_ctrl e dma_to_host_valid shoud be deasserted when read return data is invalid return data does not need to be continuous e Ifdma_to_host_almost_full is asserted read return data should not advance and dma_to_host_valid should be deasserted Figure 7 DMA Read Ending e dma_from_host_valid is deasserted after last valid data e dma_to_host_ctrl 3 should be asserted on last valid data 2 2 3 Gotchas e Allowable latencies for throttling are dependent on the depth of the FIFOs used in the design and the actual latencies are dependent on the pipeline length of signals Ideally the response to deasserting from_host_advance or asserting dma_to_host_almost_full would be immediate and would thus require a smaller FIFO depth Longer pipeline logic requires deeper FIFOs dma_from_host_advance needs to be asserted for a DMA transfer to start DMA reads should return
3. e target_write_enable will assert when target_write_data and target_write_be are valid 2 Target Reads target_read_accept must be asserted for the transaction to begin target_address_valid and target_read_enable will pulse when target_address target_read_be target_request_tag and target_read_ctrl are valid e target_read_data_valid should be assered when target_read_data target_read_data_tag and target_read_data_ctrl are valid 3 Gotchas e Target reads must complete within 4K clock cycles The host processor will stall while the read is outstanding and to prevent a permanent system hang the configFPGA will return a timeout value after 4K clock cycles e The exact amount of data requested should be returned Returning too much or too little data will result in unexpected unsupported behavior e The common use for the target interface is a register interface that can always accept accesses it is recommended that target_write_accept and target_read_accept always be asserted e Only one target read is issued at a time and the host processor will hang while it waits for a response it is encouraged to use this interface for low latency accesses e target_request_tag and target_read_ctrl should be returned as target_read_request_tag and target_read_data_ctrl 2 2 DMA Interface Detail 2 2 re 2 DMA Write Beginning Figu 4 1 DMA Writes reset dock dma_from_host_data 0000010000000000 dma_fr
4. Configuration a 13 Figure 9 Non Marvell based Board Confgeurapon 13 Figure 10 Connect Sim Model directly to pcie_interface eee eecsceesseceeneeceeneeceeeeeceeeeecseeeecneeeeeneeeees 16 Figure 11 Connect Sim Model to pcie_interface using intermediary IO modules eee eeeeeeeee 17 Figure 12 Detail of NMB example design ia A A tt E ee 18 Figure 13 NMB B scar a as 19 Eisure E te DDR BIS la ent il es len eds bo e de eh aa see aed 19 Figure 15 SEB Bus a A ee 20 Table 1 Change Log Revision Name Details Brian Poladian Brian Poladian Added detail for target dma accesses Brian Poladian Added TSK_DMA_READ TSK_DMA_WAIT_FINISH usage 1 Overview This document describes the interfaces signals and procedures necessary to understand and simulate the Dinigroup FPGA Reference Design that interfaces to the backend of the Dini Group PCle DMA ConfigFPGA Design There are several different physical bus connections that share this interface module including NMB SFB and DDR PCIe All of these are discussed here This document is not pertinent to the PCIe8T family of boards please see http www dinigroup com product common pcie8t_user_interface_manual pdf if working with a DN9000K 10PCIesT DN9002K 10PCIe8T DN9200K10PCIesT DN7006K10PCIe8T DN7406K10PCIEST or DNMEG_V5T_PCIE The Dinigroup PCle DMA design includes BAR memory access and DMA engines Using various physical bus interfaces the reference des
5. the exact amount of data requested 12 3 ConfigFPGA PCle Interface On Marvell based Dinigroup boards the configFPGA connects via PCIe to the Marvell the host computer is not able to directly see the configFPGA s DMA engines or register space The software driver interface to the configFPGA runs on the Marvell and a separate set of software drivers runs on the host PC to communicate with the Marvell via PCIe USB or Ethernet For more information please see the EMU and DiniCMOS documentation Figure 8 Marvell based Board Configuration figFPGA Marvell PCle x4 SE Ethernet PCle Fingers PCle x4 On non Marvell boards and additionally on Marvell based boards with alternate configuration options the configFPGA is connected directly to the PCIe fingers and its BAR space and DMA engines are directly accessible from the host PC For more information please see the AETest documentation Figure 9 Non Marvell based Board Configuration configFPGA PCle x4 PCle Fingers 3 1 ConfigF PGA Simulation The manner in which the configFPGA connects to the field user FPGAs varies depending on the specific Dinigroup board Each physical interface has a supplied IO module that connects to the configFPGA across a common bus interface The common bus interface is described below 13 Table 4 PCIE tohost fromhost Interface Signal Name Direction Description clk clk 2x Output eS NENE pcie_fromhos
6. Dini Buses User FPGA Design User Manual 5 12 2011 Rev 1 3 Table of Contents A NN 5 2 Verilog User Interface insano sageauaaads ancasusean ea AAR ENEE NEE ES 5 2 1 Target Interface Deal di 10 SE A OOO 10 PAA A KE A E te oh ea 10 21 3 A A NN 10 2 2 DMA Interface Detalla ia aaa ae ees 10 SE DMA Writes ai toa ono otitis 10 22 2 DMA RES ida id 11 Ge O EE 12 3 Contis FPGA POLE ee E 13 Conte REA STA A SAS 13 4a User Interface Modules sman action o dins stand sada save adeda bs babs a Aa ads dada lavsuah dogo aobssdasdea anA 17 4 1 TO Modules Interlac es iS AS A AAA A 17 4 2 IN WEB Interfaces cited OTT te e 18 4 3 BIDI TEE 19 4 4 SEB Ne race IE AE A eege AA Bs Bes EE 20 5 Description Location Of Files viii id eege dd gdSdeEI ANE 20 5 1 Piles O contis FPGA AE SEET 20 5 2 Files forUserEPGA ia a A iia TS 21 A O ON 21 5 22 UNIVE ads 21 A EE 21 J24 PCIE DDR A EE 22 5 3 Sim latne th Det A 22 6 Other EE 23 Table of Tables Fable LEA e o a de 4 Table 2 Backend Clocks Reduir arc WEEN EA WENN da 5 Table User Signals tacna 5 Table 4 PCIE tohost fromhost Interface 14 Table of Figures Figure 1 Example Us r Tra s actions oirlo ia ipod 9 AER eu 10 Fig te 3 DMA Write E EE 11 ere AE Write Ending iii anan lee ii A 11 Figure 5 DMA Read Beginning arco o id E ANNE Endras 11 Figure 6 DMA R ad Throttling tc td ica 12 Figure 7 DMA Read EX A AAA A A A A E A A AE 12 Figure 8 Marvell bas d Board
7. cation of Files All file locations are relative from the FPGA_Reference_Designs folder in the board support package 5 1 Files for configFPGA common pcie pcie_dma pcie_fpga testbench pcie_config_sim_model v Simulation model of the configFPGA design Tasks are used for stimulating the user interfaces see section 5 3 for further description The testbench is configurable to connect to a user specified number of interfaces common pcie pcie_dma pcie_fpga testbench pcie_board v Board level testbench connections between configFPGA appropriate IO modules and the user design common pcie pcie_dma pcie_fpga testbench tb_top v Top level testbench that instantiates the board level testbench and calls the Target DMA tasks common pcie pcie_dma pcie_fpga testbench bar_tests v Example list of Target interface tests to run common pcie pcie_dma pcie_fpga testbench dma_tests v Example list of DMA interface tests to run common NMB nmb_io v 20 NMB IO module connects to nmb_user_io nmb_user_interface modules common SuperFastBus sfb_io v SFB IO module connects to sfb_user_io sfb_user_interface modules common pcie pcie_dma user_fpga pcie_ddr_io v PCIE DDR IO module connects to pcie_ddr_user_io pcie_ddr_user_interface modules 5 2 Files for User FPGA 5 2 1 Common common pcie pcie_dma user_fpga pcie_interface v Translates between pcie tohost fromhost interface and recommended Target DMA interface common pcie pcie_dma user_f
8. est Bits 2 4 7 are reserved When 1 data ctrl is transferred into the module When high user should stop writing data to the module soon Same Same as DMAO but 1 in the name instead of 0 Separate interfaces for each DMA engine Same Same as DMAO but 2 in the name instead of 0 Separate interfaces for each DMA engine Input A parameterizable number of user level interrupts which will be combined into a single interrupt and sent to the PCIe core Interrupts must be asserted until cleared by configFPGA Figure 1 Example User Transactions clk_out user_clk reset_out reset target_address 63 0 target_address_valid target_write_accept target_write_enable target_write_data 63 0 target_write_be 7 0 target_read_accept target_read_enable target_read_data 63 0 target_read_data_valid target_request_tag 3 0 target_read_data_tag 3 0 dma0_from_host_data 63 0 dma0_from_host_ctri 7 0 dma0_from_host_valid dma0_from_host_advance dma0_to_host_data 63 0 dma0_to_host_ctrl 7 0 dma0_to_host_valid dma0_to_host_almost_full BAR Read 00000004 BAR Write 00000000 00000004 DMA 0 To Host Transaction 2 1 2 1 Target Interface Detail 1 Target Writes target_write_accept must be asserted for the transaction to begin target_address_valid will assert when target_address is valid
9. ign connects the user s design in the field FPGA to the ConfigFPGA If you have questions comments concerns please email supportOdinigroup com 2 Verilog User Interface Here we discuss the signals in and out of the pcie_interface module It is recommended that the user instantiate this module to interface with the configFPGA This module will connect to either a PCIe DDR NMB or SFB IO module which will convert the physical interface to the internal interface described here Direction of ports described is in relation to the pcie_interface module Table 2 Backend Clocks Resets Signal Direction Description Name reset Input Main module reset Input from user Allows resetting the module from an external source reset_out Output Reset to user module When de asserted this indicates that the interface is ready to use user_clk Input User clock Clocks logic on the user side of the interface Can be sourced by clk_out Clock sent to user Clocks logic on the physical side of the interface Table 3 User Signals Signal Name Direction Description Target Interface AAA target_address 63 0 Byte address bits 1 0 always 0 target_address_valid Output One clock cycle strobe that indicates the target_address is valid targot_write_ data 63 0 target_write_be 7 0 Byte enables bit O means data 7 0 is valid etc target_write_enable Output Indicates valid write data on this interface target_write_accept Inp
10. nnect directly to the pcie_interface module which will translate the tohost fromhost interface into the recommended user interface with separate ports for target DMAO DMA1 and DMA2 as described in section 2 15 Figure 10 Connect Sim Model directly to pcie_interface pcie_config sim_model pcie_fromhos lt pcie_tohost pcie_interface tb_top target_fromhost ma0_fromhost ma1_fromhost ma2_fromhost t lt target_tohost dma0_tohost a dma1_tohost a dma2_tohost If the user desires to include the physical interface in simulation then the IO modules will be placed user_design between the pcie_config_sim_model and pcie_interface modules The pcie tohost fromhost bus will not be modified by the physical IO modules 16 Figure 11 Connect Sim Model to pcie_interface using intermediary IO modules to_top user_design pcie_config bi sim_model HIH nmb_user_io pcie_interface target_fromhost pcie_fromhost ISERDES NMB_FROMHOST ISERDES Lines fromhostr dma0_fromhost D dma1_fromhost dma2_fromhost gt i target_tohost lt dma0_tohost pcie_tohost OSERDES 4 ams TOHOST OSERDES pcie_tohost lt t dma1_tohost
11. nternal tohost fromhost interface 5 3 Simulating the Design See section 3 for the connections necessary to attach the configFPGA simulation model to the user design After the connections are established the tasks in the configFPGA simulation model are used to initiate transactions to the user design TSK_TARGET_WRITE 32 or 64 bit write to the target interface input 63 0 address address to write to input 31 0 bar BAR to simulate direct PCIe access set to 0x1 if only simulating NMB input 63 0 data data to write input 1 0 dword_enable enable for write data TSK_TARGET_READ 64 bit read from target interface input 63 0 address address to read from input 31 0 bar BAR to simulate direct PCIe access set to 0x1 if only simulating NMB output 63 0 data read data returned TSK_DMA_WRITE Write data from file to DMA interface input 31 0 engine 0x0 0x1 0x2 DMA engine to use input 63 0 address starting address of transfer input 255 0 file_name source file for DMA transfer data TSK_DMA_READ Read data from a DMA interface input 31 0 engine 0x0 0x1 0x2 DMA engine to use input 63 0 address starting address of transfer input 23 0 length length of DMA read num dwords TSK_DMA_WAIT_FINISH Wait for all DMA transfers to finish Note To properly simulate hardware behavior only one DMA read may be issued to a DMA engine at a time TSK_DMA_READ will not wait for all DMA read da
12. om_host_ctrl dma_from_host_valid dma_from_host_advance dma_to_host_data dma_to_host_ctrl dma_to_host_valid dma_to_host_almost_full e dma_from_host_valid is asserted at start of transfer and remains asserted while transfer data control is active 64 bit address is transferred on first valid cycle dma_from_host_ctrl 5 0 2 b01 dma_from_host_ctrl 4 should be O to indicate write Length data is transferred on second valid cycle dma_from_host_ctrl 5 0 2 b10 e Data is transferred from third valid cycle onward Figure 3 DMA Write Throttling 4 reset 4 dock E4 dma_from_host_data E4 dma_from_host_ctrl 4 dma_from_host valid 4 dma_from_host_advance E4 dma_to host data E4 de o host ctrl 4 dma_to_host_valid 4 dma_to host almost ful e dma_from_host_advance allows the write to continue deasserting this signal prevents write data from advancing Figure 4 DMA Write Ending 4 AA E dma_to_host_data E4 dma_to_host o 03 oc eee 7 eee e The write transaction is considered officially over at the stat of the next transaction read or write read pictured here otherwise the write may continue when dma_from_host_valid is reasserted e Note that the last cycle of data may have a different dword enable value pictured above depending on the length of data to be transferred 2 2 2 DMA Reads Figure 5 DMA Read Beginning 0123456789abcdes bp Oc lid 0123456789abcde2 0b RSR 03
13. pga pcie_user_design v BlockRAM attached to Target and DMA interfaces User should use transaction logic from this design common pcie pcie_dma user_fpga pcie_dma_blockram v Condensed version of pcie_user_design that attaches BlockRAM to a single DMA engine common pcie pcie_dma user_fpga pcie_defines v Misc defines used in the pcie_user_design module 5 2 2 NMB common pcie pcie_dma user_fpga pcie_nmb_user_fpga v FPGA level example design that connects the nmb_user_interface and pcie_user_design modules common NMB Mmb_user_interface v Wrapper for the nmb_user_io and pcie_interface modules common NMB nmb_user_io v Translates physical NMB IO signals to internal tohost fromhost interface 5 2 3 SFB common pcie pcie_dma user_fpga pcie_sfb_user_fpga v FPGA level example design that connects the sfb_user_interface and pcie_user_design modules common SuperFastBus sfb_user_interface v Wrapper for the sfb_user_io and pcie_interface modules common SuperFastBus sfb_user_io v Translates physical SFB IO signals to internal tohost fromhost interface 21 5 2 4 PCle DDR common pcie pcie_dma user_fpga pcie_ddr_user_fpga v FPGA level example design that connects the pcie_ddr_user_interface and pcie_user_design modules common pcie pcie_dma user_fpga pcie_ddr_user_interface v Wrapper for the pcie_ddr_user_io and pcie_interface modules common pcie pcie_dma user_fpga pcie_ddr_user_io v Translates physical PCIe DDR IO signals to i
14. sociated with a target read User must save target_read_ctrl 7 0 when target_read_enable is asserted and return it on target_read_data_ctrl 7 0 DMAO Interface CA PE dma0_from_host_data 63 0 Output Address length or data depending on the state of dma0_from_host_ctrl 5 0 dma0_from_host_ctrl 7 0 Output Bit meanings 5 0 indicates type of data 3 2 are dword enables when bit 3 is 1 63 32 is valid when bit 2 is 1 31 0 is valid 4 indicates read request when 1 write packet when 0 dma0_from_host_ valid dma0 from_host_advance dma0_to_host_data 63 0 dma0_to_host_ctrl 7 0 dma0_to_host_valid dma0_to_host_almost full DMA1 Interface Same as DMAO DMA2 Interface Same as DMAO User Interrupts user_interrupts 0 Bits 1 6 7 reserved Decode of 5 0 01 64 bit board dword address 10 24 bits of dword length 23 0 Upper 40 bits are reserved For reads user must return this many dwords of data properly aligned based on bit O of the dword board address For writes this information is provided but informational only 00 Data 11 Reserved Undefined Never Occurs Output Indicates valid data on data and ctrl signals Input This signal high and dma0_from_host_valid high indicates data transfer Data to return to PCI E FPGA Input Bit meanings 1 0 are dword enables when bit 1 is 1 63 32 is valid data when bit 0 is 1 31 0 is valid data 3 Indicates last data for this read requ
15. t_data 63 0 Muxed target DMA address control data pcie_fromhost_transaction_type 3 0 0 Target transaction 1 DMAO transaction 2 DMA transaction 3 DMA2 transaction pcie_fromhost_isaddress 1 0 Output 0 Address start of transaction 1 end of transaction pcie_fromhost_info 1 0 Output 0 Read transaction write_n 1 64 bit target access OR DMA am data 7 data 63 32 valid pcie_fromhost_almost_full 4 0 0 Target interface almost full 1 DMAO interface almost full 2 DMA 1 interface almost full 3 DMA2 interface almost full pcie_tohost_data 63 0 Muxed target DMA data pcie_tohost_transaction_type 3 0 0 Target transaction 1 DMAO transaction 2 DMA transaction 3 DMA2 transaction pcie_tohost_info 4 0 When Target transaction 0 64 bit access 4 1 Target return tag 14 When DMA transaction 0 data 31 0 valid 1 data 63 32 valid 2 DMA demand mode 3 DMA EOF pcie_tohost_valid 1 0 Input 0 data 31 0 valid 1 data 63 32 valid pcie_tohost_almost_full 4 0 Input 0 Target interface almost full 1 DMAO interface almost full 2 DMA 1 interface almost full 3 DMA2 interface almost full This internal interface is subject to change as new features are added and the signal definitions listed here should not be considered static It is NOT recommended to use this interface directly This interface is intended to co
16. ta to return TSK_DMA_WAIT_FINISH needs to be called between successive DMA reads to a given engine 22 TSK_DEMANDMODE_DMA Set up the reference design to initiate DMA transfers input 31 0 engine 0x0 Ox 1 0x2 DMA engine to use input 7 0 dma_interface 0x0 OxFF Upper byte of board address input 63 0 address host address for DMA transfer input 31 0 tag tag ID for DMA transfer input 4 0 first_be first word byte enable input 4 0 last_be last word byte enable input 23 0 size length of DMA transfer input direction_tohost DMA direction 0xO fromhost 0x1 tohost Note TSK_DEMANDMODE_DMA requires the use of the full reference design provided in pcie_user_design v Users implementing their own logic to initiate DMA transfers will need to modify this function accordingly 6 Other Features If you have a new feature request or feel that an essential function is missing please contact support dinigroup com 23
17. um speed of 1 Gbps per signal for a total data throughput of 8Gbps in each direction full duplex Each user FPGA has its own independent NMB bus 18 Figure 13 NMB Bus NMBO configFPGA User FRGRA NMB1 User FPGA B NMBx User FPGA X 4 3 DDR PCle Interface The PCIe DDR bus is a 64 bit data 13 bit control source synchronous clock DDR bus Because of the high pin count there is usually only one of these busses attached to the configFPGA and it is commonly combined with SFB interfaces on Dinigroup boards The PCIe DDR bus will run 100Mhz for a total data throughput of 6 4 Gbps in each direction full duplex Figure 14 PCIe DDR Bus configF PGA PCle DDR User FPGA 19 4 4 SFB Interface The SFB bus is a 8 bit data 4 bit control global clock bus that shares its signals with the SelectMAP configuration bus Because of its low pin count it is ideal to combine with the PCIe DDR bus on Dinigroup boards Multiple FPGAs can be connected to a single SFB bus but communication is half duplex and so only one FPGA may communicate on the bus at a time The SFB bus will run 60MHz for a total data throughput of 480 Mbps Figure 15 SFB Bus User FPGA User FPGA User FPGA F00 F01 FOX SFBO configFPGA SEH GE Ke Ge Se SFB1 5 Description Lo
18. ut Accepts the valid write data Data is allowed to transfer when target_write_accept and target_write_enable are active If connecting to blockRAM this signal can be tied high debug_target_bar 2 0 Output Valid for reads and writes Indicates which bar is being accessed bar number 1 2 or 4 Onehot so 3 b001 means BARI 3 b010 means BAR2 and 3 b100 means BAR4 To treat target_address as a unified 64 bit address space this signals should be used for debug informational purposes only target_read_enable High on a target read request target_request_tag 3 0 Tag associated with this read request target_read_accept Input Acceptance of read request Allows interface to move on before the read data is returned by the user Target_read_enable and target_read_accept being high signals transfer of request If connecting to blockram this signal can be tied high target_read_data 63 0 Read Data to return to configFPGA target_read_be 7 0 Byte enables bit 0 means data 7 0 is requested etc target_read_data_tag 3 0 Input Tag that accompanies this data This value must match the value provided on target_request_tag when target_read_enable was high target_read_data_valid Clock cycle pulse indicating read data is valid target_read_ctrl 7 0 Output Tag associated with a target read Bit 0 indicates both dwords are requested non dword aligned or quadword transfer target_read_data_ctrl 7 0 Input Tag as
Download Pdf Manuals
Related Search