Home

M ellanox Technologies

1. API interface for both Linux and Windows operating systems Dual 4X InfiniBand ports with standard copper connectors each port runs at 10Gb s for MHEA28 XT and at 10Gb s or 20Gb s for MHGA28 XT e Hardware support for up to 16 million OPs EEs and CQs Memory Protection and Translation Tables fully implemented in hardware IB Native layer 4 DMA hardware acceleration e Multicast support Programmable MTU size from 256 to 2K bytes Eight Virtual Lanes supported plus Management Lane Support for InfiniBand transport mechanisms UC UD RC RAW e EEPROM used for storing Vital Product Data VPD 1 2 Key Mellanox Features e Embedded InfiniRISC Processors for Management amp Subnet Management Agent SMA e Integrated Physical Layer SerDes Mellanox Technologies Confidential Rev 1 10 Overview 8 e Integrated GSA General Service Agents e Low Latency Communication Technology e Flexible Completion Mechanism Support Completion Queue Event or Polled operation 1 3 Operating Systems Support MHEA28 XT and MHGA28 XT can operate in Linux and Windows operating system environments See Firmware and Software to obtain device drivers 1 4 Firmware and Software The MHEA28 XT and MHGA28 XT cards come with the latest version of the appropriate firmware available at the time of manufacturing Firmware updates are periodically provided and the most recent firmware can be obtained from the Firmware Downloads pa
2. Lanes for all ports Mellanox Technologies Confidential Rev 1 10 19 MHEA28 XT amp MHGA28 XT InfiniHost III Ex PCI Express MemFree HCA Adapter Card Board User s Manual Revision History Table 1 Revision History Date Rev Comments Changes Added the MHGA28 XT HCA board part number supporting IB DDR operation added its spec July 2005 1 10 ifications in Appendix A Specifications on page 19 and updated the power number for MHEA28 XT Dec 2004 1 00 Created the document Mellanox Technologies Confidential Rev 1 10
3. PCI Express x8 interface hardware transport advanced per queue pair QP QoS services and support for millions of QPs and completion queues CQs The MHEA28 XT MHGA28 XT host chan nel adapter is a PCI Express Revision 1 0a compatible card with dual 4X InfiniBand compliant connectors for copper cables Note The MHEA28 XT HCA card only utilizes media detect circuitry enabling external fiber media adapter modules such as the Emcore QTR3400 allowing for fiber channel communication across up to 300m at SDR only The cards feature MemFree technology which eliminates the requirement for local memory on the PCI Express adapter cards The cards can be inserted into PCI Express x8 or higher slotssof standard server blade server storage and communications platforms to enable InfiniBand system area networks The InfiniHost III Ex core features a full hardware implementation of the InfiniBand architecture memory protection and translation tables as well as hard ware transport This drastically reduces CPU overhead to enable the host processor to spend its cycles on applications and not on communications This advanced third generationdesign achieves industry leading bandwidth performance coupled with low latency for technical compute clusters data centers and storage applications 1 1 Product Features e PCI Express x8 version 1 0a compatible card Two Port InfiniBand Version 1 1 compatible Host Channel Adapter e InfiniBand Compatible Verbs
4. 4X port The physical link green illuminates once VAPI InfiniBand Verbs API is started and a physical connection is made between two nodes The data activity link yellow illuminates once the InfiniBand network is discovered over the physical link The activity link is a steady yellow when it is discovered but no data is being passed The activity link blinks when data is being passed If the LEDs are not active either the physical or the logical or both connections have not been established See Figure 2 I O Panel with Dual Ports and LEDs on page 13 Table 1 LEDs Port 1 LED Name Physical Link Green Logical Link Yellow Port 2 LED Name Physical Link Green Mellanox Technologies Confidential Rev 1 10 MHEA28 XT MHGA28 XT Board 10 Table 1 LEDs Continued Port 1 LED Name Logical Link Yellow 2 1 4 2C Interface A three pin header is provided as the I2C interface Please refer to the Mellanox MST Tools User s Manual see Doc uments List on page 5 for usage 2 2 Power The MHEA28 XT MHGA28 XT board receives 3 3v and 12v power from the PCI Express Edge connector All other required power is generated by on board switch mode regulators 2 3 Memory The MHEA28 XT MHGA28 XT supports multiple memory devices through the PCI Express Flash and 12C inter faces 2 3 1 System Memory The MHEA28 XT MHGA28 XT utilizes the PCI Express in
5. A28 XT Lion Mini DDR Table 4 VPD Format for MHGA28 XT shows Mellanox PCI VPD Vital Product Data format details for the MHGA28 XT HCA card This definition complies with the format defined in the PCI 2 3 Specification Appendix I Table 4 VPD Format for MHGA28 XT Offset Decimal Item Value Format Description 0 Large Resource Type ID String Tag 0x02 0x82 1 Length 0xD 3 Data Lion mini DDR Alphanumeric 16 Large Resource Type VPD R Tag 0x 10 0x90 17 Length Ox4F 19 VPD Keyword PN Numbers Add in Card Part Number 21 Length Ox15 22 Data MHGA28 XT 43 VPD Keyword EC Alphanumeric Engineering Change Level of the card rev 45 Length 0x2 46 Data Al PCB revision 48 VPD Keyword SN Alphanumeric Serial Number 50 Length 0x18 51 Data 00 gt 00XXXX XX 75 VPD Keyword vo Misc Information 77 Length 0x10 78 Data PCI EX x8 94 VPD Keyword RV 96 Length 0x1 97 Data Checksum 98 Large Resource Type VPD W Tag 0x11 0x91 99 Length 0x9A 101 VPD Keyword v EFI Driver version 103 Length 0x6 104 Data N A Number 110 VPD Keyword YA Asset Tag 112 Length 0x20 113 Data N A Alphanumeric N A 145 VPD Keyword RW Remaining read write area 147 Length 0x6b Rev 1 10 Mellanox Technologies Confidential MHEA28 XT amp MHGA28 XT InfiniHost III Ex PCI E
6. III Programmer s Reference Manual A reference describing the interface used by developers to write a driver Document no 2248PM for MemFree Mellanox InfiniHost III devices InfiniHost III Ex MT25208 Hardware Reference Manual Reference for hardware engineers responsible for designing systems and Document no 2246HM boards incorporating InfiniHost III Ex components InfiniBand Administration IBADM Package User s Manual User s Manual describing the utilities included in the IBADM tools pack Document no 2130UM age for system administration of an InfiniBand cluster Mellanox MST User s Manual This manual describes various tools and utilities included in the Mellanox Document no 2125SM Software Tools MST package for accessing burning firmware and trac ing Mellanox silicon devices VAPI HCA Device Driver Release Notes InfiniHost III device driver release notes Version 4 0 3 or later Mellanox Technologies Confidential Rev 1 10 MHEA28 XT amp MHGA28 XT InfiniHost III Ex PCI Express MemFree HCA Adapter Card Board User s Manual 1 Overview The MHEA28 XT and MHGA28 XT are dual port 4X InfiniBand PCI Express x8 Adapter Cards based on the Infini Host III Ex MT25208 third generation InfiniBand Host Channel Adapter HCA Each 4X InfiniBand port on the MHEA28 XT operates at SDR 10Gb s whereas each 4X InfiniBand port on the MHGA28 XT operates at SDR 10Gb s or DDR 20Gb s InfiniHost III Ex features include a
7. Information 3 1 MHEA28 XT Board Diagram Note The MHGA28 XT has exactly the same footprint of MHEA28 XT therefore its diagram is not included 12C Compatible Connector Flash Jumper Figure 1 PCI Express Expansion Board Mellanox Technologies Confidential Rev 1 10 MHEA28 XT amp MHGA28 XT InfiniHost III Ex PCI Express MemFree HCA Adapter Card Board User s Manual Physical Link Green LED Data Activity Link Yellow LED Port 1 Port 2 Figure 2 I O Panel with Dual Ports and LEDs 3 2 Air Flow Requirement Per the PCI specification the MHEA28 XT MHGA28 XT board needs to be provided with 200 LFM at 55C ambi ent temperature Mellanox Technologies Confidential Rev 1 10 Vital Product Data VPD Format 14 4 Vital Product Data VPD Format This chapter includes the following sections VPD for MHEA28 XT Lion Mini on page 14 VPD for MHGA28 XT Lion Mini DDR on page 16 4 1 VPD for MHEA28 XT Lion Mini Table 3 VPD Format for MHEA28 XT shows Mellanox PCI VPD Vital Product Data format details for the MHEA28 XT HCA card This definition complies with the format defined in the PCI 2 3 Specification Appendix I Table 3 VPD Format for MHEA28 XT Offset Decimal Item Value Format Description 0 Large Resource Type ID String Tag 0x02 0x82 1 Length 0x9 3 Data Lion Mini Al
8. Mellanox Technologies MHEA28 XT amp MHGA28 XT Memfree InfiniHost III Ex PCI Express x8 HCA Card Board User s Manual Rev 1 10 A Copyright 2005 Mellanox Technologies Inc All Rights Reserved MHEA28 XT amp MHGA28 XT InfiniHost III Ex PCI Express MemFree HCA Adapter Card Board User s Manual Document Number 2151BM Mellanox Technologies Inc 2900 Stender Way Santa Clara CA 95054 U S A www Mellanox com Tel 408 970 3400 Fax 408 970 3403 Mellanox Technologies Ltd PO Box 586 Hermon Building Yokneam 20692 Israel Tel 972 4 909 7200 Fax 972 4 959 3245 Rev 1 10 Mellanox Technologies Confidential MHEA28 XT amp MHGA28 XT InfiniHost III Ex PCI Express MemFree HCA Adapter Card Board User s Manual Table of Contents Table of Contents 3 About This Manual 5 1 Overview 7 1 1 Product Features 7 1 2 Key Mellanox Features 7 1 3 Operating Systems Support 8 1 4 Firmware and Software 8 1 5 MHEA28 XT MHGA28 XT Contents 8 2 MHEA28 XT MHGA28 XT Board 9 2 1 I O Interfaces 9 2 1 1 InfiniBand Interface 9 2 1 2 PCI Express Interface 9 2 1 2 1 PCI Express Board Features 9 2 1 3 LED Assignment 9 2 1 4 I2C Interface 10 2 2 Power 10 2 3 Memory 10 2 3 1 System Memory 10 2 3 2 Flash ROM 10 2 3 3 EEPROM 10 2 4 Jumper Configuration 11 3 Mechanical and Other Information 12 3 1 MHEA28 XT Board Diagram 12 3 2 Air Flow Requirement 13 4 Vital Product Data VPD Format 14 4 1 VPD for MHEA28 XT Lion M
9. ge reached from www mellanox com The firmware can be upd ted using the ibf wmg tool of the IBADM package see Documents List on page 5 Both low profile HCA cards include verbs interface and basic device drivers for both Windows and Linux operating systems In addition the card includes an internal Subnet Management Agent SMA and General Service Agents eliminating the requirement for an external management agent CPU The HCA is fully compatible with the open source OpenIB software suite The OpenIB software is available through the InfiniBand Gold Distribution which includes software for database clustering high performance computing communications and storage applications This collection consists of driv ers protocols and management applications in a simple to install package To download the package go to www mel lanox com The board is also supported by open source applications and management package Complete initialization diagnostic and management utilities are also provided to facilitate quick system bring up Alternatively it is possible to download the HCA driver and other software from Products HCA Board Products MemFree InfiniHost III Ex PCI Express Adapter Card as well as from Code Releases InfiniHost III Ex on the DDS at www mellanox com 1 5 MHEA28 XT MHGA28 XT Contents e MHEA28 XT MHGA28 XT PCI Express HCA Adapter Card Mellanox Technologies Confidential Rev 1 10 MHEA28 XT amp MHGA28 XT Inf
10. ini 14 4 2 VPDforMHGA28 XT Lion Mini DDR 16 Appendix A Specifications 19 A 1 MHEA28 XT Specifications 19 A2 MHGA28 XT Specifications 19 Revision History 21 Mellanox Technologies Confidential Rev 1 10 MHEA28 XT amp MHGA28 XT InfiniHost III Ex PCI Express MemFree HCA Adapter Card Board User s Manual About This Manual This Board User s Manual describes the following InfiniHost III Ex PCI Express x8 MemFree HCA Cards MHEA28 XT Code Name Lion Mini supporting operation of IB ports at SDR Single Data Rate MHGA28 XT Code Name Lion Mini DDR supporting operation of IB ports at DDR Double Data Rate The manual provides details as to the interfaces of the boards specifications required software and firmware for operating the boards and relevant documentation Intended Audience This manual is intended for the installer and user of the MHEA28 XT or MHGA28 XT HCA cards The manual assumes basic familiarity with the InfiniBand architecture specification Related Documentation Table 1 Documents List MHEA28 XT amp MHGA28 XT InfiniHost III Ex PCI Express Mem This manual Free HCA Adapter Card Board User s Manual Document no 2151BM MHEA28 XT amp MHGA28 XT InfiniHost III Ex PCI Express Mem A quick start guide for helping in quick hardware installation software and Free HCA Card Quick Start Guide firmware installation and usage of the adapter cards Document no 2143UG InfiniHost
11. iniHost III Ex PCI Express MemFree HCA Adapter Card Board User s Manual 2 MHEA28 XT MHGA28 XT Board 2 1 I O Interfaces The MHEA28 XT MHGA28 XT board includes the following interfaces Two 4X InfiniBand Copper Connectors PCI Express x8 Edge Connector e I O Panel LEDs PC compatible connector for debug 2 1 1 InfiniBand Interface The MHEA28 XT MHGA28 XT board provides two 4X InfiniBand v1 1 4X connectors for external InfiniBand cables The MT25208 device is compliant with the IBTA specification 1 1 The MT25208 gt silicon device has two 4X ports A and B which connect to 4X connectors 1 and 2 of the MHEA28 XT MHGA28 XT respectively The stan dard InfiniBand connectors include a media detect circuit that supports external InfiniBand fiber solutions such as the Emcore Model QTR3400 Smart Module 2 1 2 PCI Express Interface The PCI Express interface is version 1 0a compatible The MT25208 can be either a root complex initiating the PCI Express bus operations or a downstream device responding to PCI bus operations The PCI Express bus can connect to either a host CPU in an HCA application or to an I O device such as Gigabit Ethernet when used as a Target Channel Adapter 2 1 2 1 PCI Express Board Features Low profile short PCI Express 8x expansion board approximately 2 5 x 4 6 inches with auto negotiation fea tures 2 1 3 LED Assignment The board has four LEDs located on the I O panel 2 LEDs per
12. phanumeric 12 Large Resource Type VPD R Tag 0x10 0x90 13 Length 0x51 15 VPD Keyword PN Numbers Add in Card Part Number 17 Length 0x4 18 Data MHEL C T 39 VPD Keyword EC Alphanumeric Engineering change level of the card Rev 41 Length 0x4 42 Data A 02 PCB Revision 46 VPD Keyword SN Alphanumeric Serial Number 48 Length 0x18 49 Data MTYYWWFXXXXX 73 VPD Keyword V0 Misc Information 75 Length 0x10 76 Data PCI Ex x8 92 VPD Keyword RV 94 Length 0x1 95 Data Checksum 96 Large Resource Type VPD W Tag 0x11 0x91 97 Length 0x9C Rev 1 10 Mellanox Technologies Confidential MHEA28 XT amp MHGA28 XT InfiniHost III Ex PCI Express MemFree HCA Adapter Card Board User s Manual Table 3 VPD Format for MHEA28 XT Continued Offset Decimal Item Value Format Description 99 VPD Keyword VP EFI Driver Version 101 Length 0x6 102 Data NA Number 108 VPD Keyword YA Asset Tag 110 Length 0x20 111 Data NA Alphanumeric N A 143 VPD Keyword RW Remaining read write area 145 Length 0x6d 146 Data Reserved 0x00 255 Small Resource Type END Tag 0x11 0x78 256 Mellanox Read Only Mask 0x0 0 Numbers 355 Mellanox Read Write Mask 0x1 1 Numbers 511 Mellanox Read Only Mask 0x0 Numbers Mellanox Technologies Confidential Rev 1 10 15 Vital Product Data VPD Format 16 4 2 VPD for MHG
13. terface to store and access IB fabric connection infor mation on the system memory 2 3 2 Flash ROM The MHEA28 XT MHGA28 XT board supports 4MB of Flash ROM space via the CPU interface of the MT25208 InfiniHost III Ex device The details of the Flash supported are 8 bit data 22 bit address e Programming via the CPU bus Intel s TE28F320J3C 110 Flash device used onthe MHEA28 XT MHGA28 XT board has the following character istics 32 Mb Uniform 4MB e 110 ns access time 2 3 3 EEPROM The board incorporates an EEPROM that is accessible through an I2C interface The EEPROM is used for storing the Vital Product Data VPD VPD format adheres to the PCI Local Bus specification rev 2 3 VPD definition EEPROM capacity is 512 bytes Mellanox Technologies Confidential Rev 1 10 MHEA28 XT amp MHGA28 XT InfiniHost III Ex PCI Express MemFree HCA Adapter Card Board User s Manual 11 2 4 Jumper Configuration Table 2 Jumper Configuration Present Shorting Block present Flash not present FLASH Present MEDA Default Ref H Name Description Option G n irili Comments J4 Flash Flash present not present No Shorting Block Default Flash No Shorting Header 1x2 Block Note See Figure 1 PCI Express Expansion Board on page 12 for the Jumper location Mellanox Technologies Confidential Rev 1 10 Mechanical and Other Information 12 3 Mechanical and Other
14. xpress MemFree HCA Adapter Card Board User s Manual Table 4 VPD Format for MHGA28 XT Continued 17 Offset Decimal Item Value Format Description 148 Data Reserved 0x00 255 Small Resource Type END Tag 0x11 0x78 256 Mellanox Read Only Mask 0x0 0 Numbers 357 Mellanox Read Write Mask Ox1 1 Numbers 511 Mellanox Read Only Mask 0x0 Numbers Mellanox Technologies Confidential Rev 1 10 MHEA28 XT amp MHGA28 XT InfiniHost III Ex PCI Express MemFree HCA Adapter Card Board User s Manual Appendix A Specifications A 1 MHEA28 XT Specifications RDMA Support Yes All Ports Physical Power and Environmental Size 117mm x 69mm Voltage 12V 3 3V Air Flow 200LFM 55C Maximum Power 8 2W 10Gb s Connector InfiniBand MicroGigaCN Copper Temperature 0 to 55 Celsius Emcore Fiber Channel Protocol Support Regulatory InfiniBand Auto Negotiation 10Gb s 2 5Gb s QoS 8 InfiniBand Virtual Lanes for all ports A 2 MHGA28 XT Specifications RDMA Support Yes All Ports Physical Power and Environmental Size 117mm x 69mm Voltage 12V 3 3V Air Flow 200LFM 55C Maximum Power 11 4W 20Gb s Connector InfiniBand MicroGigaCN Copper Temperature 0 to 55 Celsius Emcore Fiber Channel Protocol Support Regulatory InfiniBand Auto Negotiation 10Gb s 2 5Gb s or 20Gb s SGb s QoS 8 InfiniBand Virtual

M ellanox Technologies

Contents

Download Pdf Manuals

Related Search

Related Contents