ConnectX®-4 Multi-Host Evaluation Kit User Manual
Contents
1. 90 32 001 1 nennen nnne nene nnne nent Mellanox Technologies 7 Rev 1 0 Assembly 3 3 Assembly Instructions 1 For each multi host configuration option connect PCB000981 server side extender with repeater into the slot available on the server 2 Connect PCB000981 with PCB000980 Multi Host EVB using QSFP cables MT PCI HS QSFP REV AI Connect the server side extender with repeater to the Multi Host EVB according to the colored connectivity scheme shown in Section 3 1 Two Hosts Configura tion and Section 3 2 Four Hosts Configuration 3 The Multi Host EVB can also share power rails from four different servers In order to test this feature connect the 12V and 3 3V power rails from PCBO000981 to 00980 using the power cable 90 32 001 REV AT 4 For comfortable work without overload of power cables it is possible to use the power supply included in the kit Connect the power supply to the electricity plug from one end and connect it to the adapter harness on the other end Then connect the harness to J1024 connector on the EVB 5 When power of the EVB is supplied from only one host the maximum power may exceed the 25W per slot according to the PCIe spec Instead of connecting to only one host it is best to connect at least two power sources 1 e to two hosts for better power distribution between the hosts 8 Mellanox Technologies2. Mellanox TECHNOLOGIES ConnectX 4 Multi Host Evaluation Kit User Manual P N MCX4MHEVK ECAA Rev 1 0 www mellanox com Rev 1 0 THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT S AND OR THE SYSTEM USING IT THEREFORE MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT INDIRECT SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES OF ANY KIND INCLUDING BUT NOT LIMITED TO PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY FROM THE USE OF THE PRODUCT S AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF S
3. ConnectXG 4 Multi Host Evaluation Board EVB User Manual Rev 1 0 4 Driver and Firmware 4 1 Linux Driver For Linux download and install the latest OpenFabrics Enterprise Distribution OFED software package available for ConnectX 4 via the Mellanox web site at http www mellanox com gt Products gt Software gt InfiniBand VPI Drivers gt Linux SW Driver gt Download 4 2 Windows Driver For Windows download and install the latest Mellanox WinOF VPI for Windows software pack age available for ConnectX 4 via the Mellanox web site at http www mellanox com Prod ucts gt Software InfiniBand VPI Drivers gt Windows SW Driver gt Download 4 3 Adapter Firmware In order to receive the latest firmware for this product please contact Mellanox support Mellanox Technologies 9 Rev 1 0 Specifications 5 Specifications 51 Specifications Table 2 MCXAMHEVB ECAT Specifications Table Size 7 5 in x10 5 in 192 58mm x 267mm Physical Connector QSFP28 InfiniBand Ethernet Copper and optical InfiniBand IBTA v1 2 1 Auto Negotiation 1 2 4 SDR 2 5Gb s per lane DDR 5Gb s per lane QDR 10Gb s per lane FDR10 10 3125Gb s per lane FDR 14 0625Gb s per l EDR 25Gb 1 t Protocol Support ane s per lane por Data Rate Up to 100Gb s EDR InfiniBand Up to 100Gb s Ethernet PCI Express Gen3 SERDES 8 0GT s 16 lanes 2 0 and 1 1 compatible Voltage 12V 3 3
4. ConnectX 4 Multi Host Evaluation Board EVB User Manual Rev 1 0 2 Unpacking the EVB Kit Before using your new MCX4MHEVB ECAA unpack the system and check to make sure that all the parts have been sent Check the parts for visible damage that may have occurred during shipping Note If anything is damaged or missing contact your customer representative immediately InfiniBand Ethernet PCIe cables CPU Pu CPU CPU RAM SERVER PCIe Slot SERVER PCIe Slot SERVER PCIe Slot SERVER 21 Kit Components e Multi Host EVB MCX4MHEVB ECAA 4x Server side extender PCI Express PCIe repeater cards 00981 to be con nected to independent servers e 8 QSFP 2m passive cables MT PCI HS QSFP 1 used to connect to the PCI repeater cards e 4x PCIe power cables MT90C32 001_REV A1 to be connected to the power connec tor of the PCIe repeater cards e 1 external 12V power supply MW155RA1251F01 to power the board independently with no need to connect the power from the PCIe repeater cards 1 Harness connecting the power supply and the EVB Mellanox Technologies 5 Rev 1 0 Assembly 3 Assembly 3 1 Two Hosts Configuration In a two hosts configuration Multi Host provides two x8 PCIe lanes per host In order to work in the 2 hosts PCIe 8x per host configuration assemble shunt jumpers accord ing to the below o 2 HOSTs 81 J1
5. with no performance degradation The connection of four fully independent PCIe buses to four hosts lowers total cost of ownership in the data center It reduces CAPEX requirements from four cables four adapters NICs and four switch ports to only one of each Furthermore it reduces OPEX by cutting down on switch port management and overall power consumption Multi Host technology features uncompromising independent host management with full inde pendent NC SI MCTP support to each host and to the NIC IT managers can remotely control the configuration and power state of each host individually such that management of one host does not affect host traffic performance or the management of the other hosts guaranteeing host secu rity and isolation To further lower the total cost of ownership ConnectX 4 supports manage ment of the multiple hosts using a single BMC with independent NC SI MCTP management channels for each of the managed hosts The below figure provides an illustration of the a Multi Host technology that the ConnectX 4 Multi Host Evaluation Board EVB enables The ConnectX 4 adapter connects to the hosts over the PCI Express interface with dedicated PCIe lanes per host Figure 1 Multi Host Support Feature on ConnectX 4 EN Connect X 4 Management E ak aA I aee gums cum LISSE LER LER MR m Eg a H 4 Mellanox Technologies
6. 028 PD BO J1027 PU 4 HOSTs B1 J1028 PU BO J1027 PD 00045 Power supply a z Extender Repeater E 000981 SFGO01350 E MT PCI HS OSFP_REV A1 4 MT PCI H Fi ConnectX 4 2 MT PCHH5 DSFP REV M 4 7 wer Host i 12y 3 3V Ed Extender Repeater MT PCHHS QSFP REV A1 Ho 1 000951 56001350 i MM MT PCI HS QSFP REV A1 Shomron Multi Host EVB 000980 SFG001474 eene MTBOC32 001 REV AT eene nnne 6 Mellanox Technologies ConnectX 4 Multi Host Evaluation Board EVB User Manual Rev 1 0 3 2 Four Hosts Configuration In a four hosts configuration Multi Host provides four x4 PCIe lanes per host In order to work in the 4 hosts PCIe 4x per host configuration assemble shunt jumpers accord ing to the below d 2 0513 81 1028 PD BO J1027 PU 4 M0STs B1 J1028 PU BO J1027 PD MT80032 001 1 MT PCi H5 QSFP REV AT MT PCIHH5 QSFP REV A1 90 32 001 MIT PCI H5 QSFP REV A1 Connectx 4 MT PCI H5 QSFP REVAL 0981 95001350 Shomron Multi Host PCBODOSSO SFGO00147A4 MT PCI HA DSFP REV A1 5 REVAL Il R
7. UCH DAMAGE Mellanox TECHNOLOGIES Mellanox Technologies 350 Oakmead Parkway Suite 100 Sunnyvale CA 94085 U S A www mellanox com Tel 408 970 3400 Fax 408 970 3403 Copyright 2015 Mellanox Technologies All Rights Reserved Mellanox Mellanox logo BridgeX ConnectX Connect IB CoolBox CORE Direct InfiniBridge InfiniHost InfiniScale MetroX MLNX OS TestX PhyX ScalableHPC SwitchX UFM Virtual Protocol Interconnect and Voltaire are registered trademarks of Mellanox Technologies Ltd ExtendX FabricIT HPC X Mellanox CloudX Mellanox Open Ethernet Mellanox PeerDirect Mellanox Virtual Modular Switch MetroDX M Unbreakable Link are trademarks of Mellanox Technologies Ltd All other trademarks are property of their respective owners 2 Mellanox Technologies Document Number MLNX 15 5270 ConnectXG 4 Multi Host Evaluation Board EVB User Manual Rev 1 0 Revision History This document was printed on August 13 2015 Table 1 Revision History Table Date Rev Comments Changes August 2015 1 0 First Release Mellanox Technologies 3 Rev 1 0 Overview 1 Overview Mellanox s ConnectX 9 4 Multi Host technology enables connecting multiple hosts into a single interconnect adapter by separating the ConnectX 4 PCIe interface into multiple and independent PCIe interfaces Each interface is connected to a separate host
8. V Typ Power Passive Cables 23 5W Max Power Passive Cables 25 6W Power and Max power available through QSFP port 3 0W Environmental Temperature Operational 0 to 55 Non operational 0 C to 70 C Humidity 90 relative humidity Air Flow 600LFM Safety IEC EN 60950 1 2006 ETSI EN 300 019 2 2 Regulatory IEC 60068 2 64 29 32 RoHS RoHS R6 a The ConnectX 4 adapters supplement the IBTA auto negotiation specification to get better bit error rates and longer cable reaches This supplemental feature only initiates when connected to another Mellanox InfiniBand product b For both operational and non operational states c Air flow is measured 1 from the heat sink between the heat sink and the cooling air inlet The measured air flow is in the case of using a 3 5W QSFP module In case of lower QSFP module power a lower air flow can be used Please advise Mellanox for more information 10 Mellanox Technologies ConnectX 4 Multi Host Evaluation Board EVB User Manual Rev 1 0 5 2 LED Operation There is a one bicolor LED per port Function Yellow physical e Constant on indicates a good physical link link e Blinking indicates a problem with the physical link If neither LED is lit then the physical link has not been established Green logical data A constant yellow indicates a valid logical data activity link without activity link data transfer A blinking yellow indicates a valid logical
9. link with data transfer If only the green LED is lit and the yellow LED is off then the logical link has not been established e Power good is valid and all power supplies are functioning e Power good has an error Mellanox Technologies 11 Rev 1 0 Specifications 5 3 Board Mechanical Drawing and Dimensions All dimensions are in millimeters All the mechanical tolerances are 0 1 mm 5 3 1 Multi Host Evaluation Board 1 58 10 Jmm 320 NPH x11 MAX COMP HEIGHT ES 192 58 197 90 43 00 NPH x2 MAX COMP HEIGHT 14 4mm 12 Mellanox Technologies Rev 1 0 ConnectXG 4 Multi Host Evaluation Board EVB User Manual 5 3 2 Server Side Extender with Repeater IIT iL LLL ALL th tse aoee Sos ILLI 13 Mellanox Technologies Rev 1 0 Specifications 5 3 3 Component Print Side Placement aoe og No E oF Jaq E 3 b Jh M ron nm II pr vel as da 4 Ja 1 4 14 Mellanox Technologies
Download Pdf Manuals
Related Search