Eight 4X SDR InfiniBand Port Switch Platform User`s Manual
Contents
1. FLEXTRONICS Eight 4X SDR InfiniBand Port Switch Platform User s Manual Part Number F X430066 Rev 1 01 8 Eight 4X SDR InfiniBand Port Switch Platform User s Manual Contents Contents List of Figures Revision History About this Manual Chapter 1 Overview Chapter 2 Installation and Basic Operation 2 1 Switch Platform Hardware Overview 2 1 1 InfiniBand Connectors 2 1 2 InfiniBand Port LEDs 2 1 3 System Status Indicators 2 1 4 I2C compatible Bus Connector 2 2 Switch Platform Installation and Operation 2 2 1 Mechanical Installation 2 2 2 Power Connections and Initial Power On 2 23 InfiniBand Copper Cable Installation 2 2 3 1 Cable Length Support And IB Port Configuration Chapter 3 Management Tools Overview 3 1 Updating Firmware 3 2 IB Administration 3 2 1 IBADM Requirements 3 2 2 How to Get IBADM Appendix A Specification Appendix B Mechanical Drawing Appendix Rack Adapter Installation Instructions N a amp 11 12 12 13 13 13 13 13 14 14 14 15 16 16 16 17 17 19 20 21 Rev 1 01 Al Rev 1 01 Eight 4X SDR InfiniBand Port Switch Platform User s Manual List of Figures Figure 1 Switch Front and Rear Panels Figure 2 Default Configuration Model Example Figure 3 Switch Mounting Ears Screws Figure 4 Rear Panel Side with Ear Screws Removed Figure 5 Mounting Ear Assembled on Rear Panel Side Figure 6 View of Rack Adapter Asse2. dard 3 wire AC power cord including a safety ground and rated for 2A or higher 2 2 3 InfiniBand Copper Cable Installation The switch platform uses industry standard 4X InfiniBand cables which are available from several vendors The stan dard 4X cables support full duplex 10Gb s wire speed for all switch platform ports All cables can be inserted or removed with the unit powered on To insert a cable press the connector onto the port receptacle until the connector is firmly seated The GREEN LED indicator to the left of each port will light when the physical connection is estab lished that is when the unit is powered on and a cable is plugged into the port with a functioning port plugged into the other end of the connector After plugging in a cable lock the connector using the latching mechanism particular to the cable vendor To remove disengage the locks and slowly pull the connector away from the port receptacle Both LED indicators will turn off when the cable is unseated Rev 1 01 Eight 4X SDR InfiniBand Port Switch Platform User s Manual Warning Care should be taken not to impede the air exhaust flow through the ventilation next to the InfiniBand ports Cable lengths should be used which allow for routing horizontally around to the side of the chassis before bending upward or downward in the rack 2 2 3 1 Cable Length Support And IB Port Configuration The switch platform is configured to drive cables up to 20 meters
3. teristics of a fully populated switch Chassis as listed in Table 2 Table 2 Switch Platform Mechanical and Environmental Requirements Worst Case Fully Populated Chassis Rack Height Rack Width Rack Depth Weight Power Ambient Temp 1U 12 4 6 9 7 71b 45W Max 50 C 315mm 175mm 3 5Kg Min 0 C w o Rack Adapter Single 100 240 VAC 50 60Hz Input 19 EIA 310 483mm w Rack Adapter Proper ventilation should also be guaranteed for air intake at the front of the chassis and exhaust at the rear in order to maintain good airflow at ambient temperature Cable routing in particular should not impede the air exhaust from the chassis Note that the switch platform can be either front or rear mounted The notion of front and rear is arbitrary how ever rear is used consistently in this manual to refer to the side of the chassis with the InfiniBand connectors 2 2 2 Power Connections and Initial Power On Warning The switch platform will automatically power on when AC power is applied There is no power switch Make sure the power cable is properly plugged into the system before connecting to power Warning The switch platform must be connected to an earthed mains socket outlet Warning In Norway this system should be connected to the IT power distribution system only The input voltage is auto adjusting for a 100 240 VAC 50 60Hz power connection The power cord should be a stan
4. InfiniBand Connectors All InfiniBand connectivity is via the rear panel Figure 1 shows the eight 4X InfiniBand port connectors Each port also includes power supply functionality to support fiber media adapters 2 1 2 InfiniBand Port LEDs Two IB port LEDs are located to the left of each IB port connector on the rear panel see Figure 1 on page 12 The lower Green LED is the IB port Physical link LED and the upper Yellow LED is the IB port Logical link LED Activity LED Physical Link LED Green indications Steady On The Physical link is established Off Physical link error poor connection quality or no physical connection Activity LED Yellow indications e Steady On The Logical link is up but there is no data transfer Blinking Data is being transferred to from the switch port across the cable wires Off The Logical link is down 2 1 3 System Status Indicators Two system status indicators are located on the left of the rear panel and are labeled Status see Figure 1 on page 12 The following status conditions are possible 1 SYSTEM STATUS OK Green ON Yellow OFF 2 TEMPERATURE ALARM Yellow ON 3 SYSTEM OFF Green OFF Yellow OFF 4 POWER CIRCUIT ERROR Green OFF Yellow OFF with the power cord connected to power 2 1 4 D C compatible Bus Connector The PC compatible Bus connector 15 for factory and development use only The connector is a female 9 pin D Type connector Table 1 shows the pinout fu
5. cs link status and system status for all ports in the switch Checking and updating the firmware Intuitive CLI interface Extensible and customizable The figure below shows the default configuration model for these tools Figure 2 Default Configuration Model Example Switch Admin Components One or more switches HCA Driver IBADM Tools InfiniBand HCA 1 Firmware updates to this Eight 4X IB Port Switch System cannot be performed using ibfwmgr the burning tool of IBADM pack age See Section 3 1 Updating Firmware on page 16 Rev 1 01 Eight 4X SDR InfiniBand Port Switch Platform User s Manual 3 2 1 IBADM Requirements The general requirements for installing the IBADM software are listed below Please see the IBADM release notes for details on platform OS Driver and Subnet Management support 1 Computer Platform with an InfiniBand HCA card installed 2 HCA Driver 3 InfiniBand Compliant Subnet Management The Open Source SM Eponyms is supported 3 2 2 How to Get IBADM Please visit Mellanox Technologies s Documents Distribution System at http docs mellanox com The IBADM package is available for download under Code Releases Tools Note that access requires a customer login account Consult your sales representative for details 1 Any Mellanox Technologies HCA card can be used Consult your sales representative for other possibilities Rev 1 01 Manageme
6. long This configuration allows maximum flexibil ity in building a robust IB cluster The selected configuration and cables should meet the required BER specified in the InfiniBand Architecture Specifi cation Volume 2 release 1 2 Rev 1 01 Management Tools Overview 16 3 Management Tools Overview 3 1 Updating Firmware In order to update switch firmware the MFT tools package is needed by Mellanox Technologies To download this package visit http www mellanox com support switch firmware table php Make sure to also download the User s Manual and Release Notes Specifically the spark tool of the MFT package is required for firmware updates Please see the MFT User s Manual for details The most updated firmware is also available for download from the same web page above Please find the Custom Switch based on Mellanox s MT43132 InfiniScale switch device entry in the firmware table 3 2 IB Administration To monitor status conditions in the switch platform the IBADM tools package 1s needed by Mellanox Technolo gies IBADM enables the system administrator to manage one or more switch platforms from a single remote Infini Band host The features include the following Full In Band Management of Multiple Switch and HCA Systems from single host Simple default configuration to get started quickly Name based subnet browsing and topology verification Event monitoring of port statisti
7. mbled on Rear Panel Sides Figure 7 View of Rack Adapter Assembled on Front Panel Sides Rev 1 01 6 Rev 1 01 Eight 4X SDR InfiniBand Port Switch Platform User s Manual Revision History Table 1 Revision History Table Revision amp Date Description 1 01 March 2006 Added warnings to Section 2 2 2 Power Connections and Initial Power On on page 14 1 00 February 2006 First version Rev 1 01 8 Rev 1 01 Eight 4X SDR InfiniBand Port Switch Platform User s Manual About this Manual This manual provides an overview of the Eight 4X SDR InfiniBand Port Switch system and guidelines for its opera tion Specifically it covers the following product Table 1 Switch Products Covered in this User s Manual Product Number Description F X430066 Eight 4X InfiniBand Port Switch Platform Intended Audience This manual is intended for users and system administrators responsible for installing and setting up the switch plat form listed above The manual assumes familiarity with the InfiniBand architecture specification Rev 1 01 10 Rev 1 01 Eight 4X SDR InfiniBand Port Switch Platform User s Manual 11 1 Overview This User s Manual provides an overview of the Eight 4X InfiniBand Port Switch System based on Mellanox Tech nologies MT43132 InfiniScale switch device The switch platform comes pre in
8. nctions Table 1 I2C compatible Bus Connector Pinout Functions Pin Number Function 1 5 GND 6 SDA 8 SCL 7 9 Not Connected 2 2 Switch Platform Installation and Operation Installation and initialization of the switch platform are straightforward processes requiring attention to the normal mechanical power and thermal precautions for rack mounted equipment The switch platform does not require any programming or configuration to operate as a basic InfiniBand switch and includes all of the necessary functionality to operate with external standard InfiniBand Subnet Management software Rev 1 01 Installation and Basic Operation 14 This section describes the installation process and basic operation of the switch platform 2 2 1 Mechanical Installation The switch platform is packaged in a 1U chassis See Table 2 Switch Platform Mechanical and Environmental Requirements Worst Case Fully Populated Chassis If the switch is to be mounted in a standard 19 rack it will require an adapter Two ear brackets are included but not assembled in the packing box of the switch system which can be assembled on the sides of the chassis acting as the 19 rack adapter and include rack mounting holes which conform to the IEA 310 standard for such a rack See Rack Adapter Installation Instructions on page 21 Note The installer should use a rack cable to support the mechanical and environmental charac
9. nt Tools Overview 18 Rev 1 01 Eight 4X SDR InfiniBand Port Switch Platform User s Manual Appendix A Specification Table 3 Specification Data 19 Physical Power and Environmental Size HxDxW 1U x 6 9 x 12 4 43 6mm x 175mm x 315mm Input Voltage 100 240 VAC 50 60Hz Agents for full InfiniBand In Band Manage ment Size with Rack Adapter 1U x 6 9 x 19 43 6mm x 175mm x 482 6mm Maximum Power 45W Weight 7 71b 3 5kg Ambient Temperature 0 C to 50 C Mounting 19 Rackmount Humidity 10 90 non condensing 10Gb s Connector InfiniBand Altitude Shock Vibration Internal Voltage 5 VDC Protocol Support Regulatory Compliance InfiniBand 10Gb s Safety QoS 8 InfiniBand Virtual Lanes for all ports RDMA Support Yes All Ports EMC Management Performance and Device management Scalability and Performance Reliability Availability and Serviceability Features Switching Performance Addressing Simultaneous wire speed any port to any port 48K Unicast Addresses Max per Subnet 16K Multicast Addresses per Subnet Hot Swappable N 1 Redundant None None Rev 1 01 20 Appendix B Mechanical Drawing 13W lv sar 12170 IW 1003 338NU 0 8 Iuno uw Rev 1 01 Eight 4X SDR InfiniBand Port Switch Platform User s Manual Appendix C Rack Adapter Installation Instruc
10. stalled with all necessary firmware and configuration for standard operation in an InfiniBand fabric running an InfiniBand compliant Subnet Management software in the subnet All that is required for normal operation is to follow the usual precautions for installation and connection to the fabric Once connected the Subnet Management software automatically configures and begins utilizing the switch Basic installation and hardware maintenance is covered in Installation and Basic Operation on page 12 Maintenance and configuration of the switch is done In Band through the InfiniBand fabric using the IBADM tools package This package provides the ability to monitor the temperature voltage port utilization and other status parameters in the switch To upgrade switch firmware the MFT package is required See Management Tools Over view on page 16 Rev 1 01 Installation and Basic Operation 12 2 Installation and Basic Operation 2 1 Switch Platform Hardware Overview Figure 1 shows the front and rear panel views of the Eight 4X Ports Switch System including the IC connector and status LEDs 12C compatible Bus Connector Rear Panel LEDs IB Port 1 IB Port 8 e o e e e e e e 1 IB Logical 20000000060 Link LED e e HL gt 1 IB Physical Oa Oa Link LED Front Panel Figure 1 Switch Front and Rear Panels Rev 1 01 Eight 4X SDR InfiniBand Port Switch Platform User s Manual 2 1 1
11. tions C 1 Rack Adapter Assembly Parts The rack adapter assembly parts are included in the switch packing box and are listed in Table 4 Table 4 Rack Adapter Assembly Parts Mounting Ear 2 See Figure 3 Each mounting ear requires 3 Phillips 100 Flat Head Passivated Stainless Steel screws Two additional pairs of UNC Screw 4 40 4 screws are to be found assembled on the switch chassis sides Figure 3 Switch Mounting Ears Screws C 2 Assembling the Rack Adapter STEP 1 Remove the pair of screws from the two sides of the chassis where the rack adapter is to be installed See Figure 4 on page 22 Note The rack adapter can be installed on the sides close to the front panel or to the rear panel Rev 1 01 22 Figure 4 Rear Panel Side with Ear Screws Removed STEP 2 Assemble the rack kit as shown in Figure 5 Figure 5 Mounting Ear Assembled on Rear Panel Side See also Figure 6 View of Rack Adapter Assembled on Rear Panel Sides and Figure 7 View of Rack Adapter Assembled on Front Panel Sides on page 23 Rev 1 01 Eight 4X SDR InfiniBand Port Switch Platform User s Manual 23 Figure 6 View of Rack Adapter Assembled on Rear Panel Sides Figure 7 View of Rack Adapter Assembled on Front Panel Sides Rev 1 01 24 Rev 1 01
Download Pdf Manuals
Related Search