Home

SX6015 User Manual - Mellanox Technologies

1. 17 Figure 11 Power Side Panel ici art hendes ree ber ER n 17 Figure 12 PS Unit Status LEDS ei rats Re GUERRE MEI EPISC 18 Figure 13 Bad Port LED 2 4 sorde Rer ve re a 18 Figure 14 Port BEDS tno E DEDERE Ba Er is EAM 19 Figure 15 Component Air Flow 2 21 Figure 16 Component Airflow Mismatch Wrong 21 Figure 1 7 Port Numbering seie Pog potency Massenet bed oa legs 22 Figure 18 Interface for Internally Managed Switches 23 Figure 19 Interface for Externally Managed Switches 23 Figure 20 RJ 45 DC Connector 25 Figure 21 Reset Buttons 4 sent id e p ER ERR ME ETT 25 Figure 22 Installing the Power Cord Wire Support 30 Figure 23 Two Power Inlets Electric Caution Notifications 31 Figure 24 Console Port nce ete deat einen PE MPO 33 Figure 25 Fan and PS unit Air Flow Direction 39 Figure 26 Power Supply Unit Extraction 39 Fiur 27 PS Unit Pulled Out nun pen eee Hi r 40 Figure 28 Air Flow Labels u ea Re Pe IER e EYE 41 Figure 29 Fan Module Latches 41 Figure 30 Breakout or Fanout Ca
2. 32 3 9 Rerunning the Wizard 37 3 10 Connecting to the Switch Platform 38 3 11 Starting an SSH Connection to the Switch CLI 38 Chapter 4 Extracting and Inserting FRUs 39 4 1 Extracting and Inserting the Power Supply Unit 39 4 2 Extracting and Inserting the Fan Units 40 Chapters Cable 0 SA e ne VE VERIS RE dents Ro ina ae 43 5 1 SX6018 Splitting the Data Stream Using Breakout Fanout Cables 43 3 2 Cable Installation eese lan raet ua 45 Mellanox Technologies 3 Rev 1 0 Chapter 6 Switch Shut Down Procedure 46 Chapter7 Disassembly of the Switch from the Rack 47 Pr Disposal sn er AUR Sr ORE e DS 47 Chapter 8 Management and Tools Overview 48 8 1 Switch Management 48 8 2 Fabric Inspector Diagnostics 49 8 3 Upgrading Software csse eee anaes che ER RP MEET 49 8 4 Externally Managed Switch 49 Chapter9 Troubleshooting essessessessessesees 51 Appendix A Specification simio heri OE ORO a ao s fe RR D De i RR 54 Ad Approved Gables anne TE Es 57 A 2 EMC
3. 3 Login from a serial terminal program as admin and use admin as password This starts the Mellanox configuration wizard 4 Go through the Mellanox configuration wizard Table 12 shows an example of a wizard ses sion Mellanox Technologies 33 Rev 1 1 Installation Table 12 Configuration Wizard Session IP Configuration by DHCP Wizard Session Display Example Comments Mellanox configuration wizard You must perform this configuration the first time you operate the Do you want to use the wizard for initial configu switch or after resetting the switch Type y and then press ration yes lt Enter gt Step1 Hostname switch 1 If you wish to accept the default hostname then press lt Enter gt Otherwise type a different hostname and press lt Enter gt Step 2 Use DHCP on mgmt0 interface no Perform this step to obtain an IP address for the switch ethO is the management port of the switch If you wish the DHCP server to assign the IP address type yes and press Enter If you type no no DHCP then you will be asked whether you wish to use the zeroconf configuration or not If you enter yes yes Zeroconf the session will continue as shown in Table 13 If you enter no no Zeroconf then you need to enter a static IP and the session will continue as shown in Table 14 Step 3 Enable IPv6 yes yes The management interface will be able to use IPv6 addres
4. 36 Table 15 Port Splitting Options 44 Table 16 SX6015 Specification Data 54 Table 17 SX6018 Specification Data 56 Table 18 InfiniBand QSFP Connector Pinout 60 Table 19 RJ45 CONSOLE Pinout trs ew ee ah ek ah rene DE 62 Table 20 RIAS IDE Pinout sagt pers ER names eal eins 62 Table 21 OPNS for Replacement Parts 63 6 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 0 Revision History Table 1 Revision History Table Date Revision Description March 2013 1 0 Initial release Mellanox Technologies 7 Rev 1 0 About this Manual This manual describes the installation and basic use of the Mellanox switch which is based on the SwitchX9 2 switch device Intended Audience This manual is intended for users and system administrators responsible for installing running and managing switch platforms The manual assumes familiarity with the InfiniB and Architecture Specification Related Documentation Additional Documentation available from Mellanox Table 2 Reference Documents and Web Sites Document Name Description InfiniBand Architecture Specifica The InfiniBand Architecture Specification that is provided
5. 52 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 2 SX PPC M460EX 3 2 0000 dev HA 2013 04 11 14 05 39 ppc 3 U Boot prompt Choice 5 Select the image to boot Mellanox Technologies 53 Table 16 SX6015 Specification Data Rev 1 0 Appendix A Specification Physical Size Size SX6015 1 716 1U Hx 16 85 W x SX6015 1 716 10 H x 16 85 W short 16 8 D Standard x24 75 D 43 6mm X 427 9mm X 428 9mm 43 6mm X 427 5mm X 628 9 mm Weight Center of Short 8 01 kg 1 PS unit Gravity As measured from the Connector 8 75 kg 2 PS units side left side bottom corner Weight Standard Short Center of 8 95 kg 1 PS unit 1PS unit 21 6mm X 193mm X 209 8mm Gravity 9 69 kg 2 PS units 2PS units 2 6mm X 214mm X 218 8mm Standard 1PS unit 21 6mm X 201mm X 337mm 2PS units 21 6mm X 214mm X 348mm Mounting dos 19 Rack mount SerDes Speeds Connector SerDes Speeds 10 20 40 or 56 Gb s per port Types QSFP Connector Types Air Flow Heat dissipa SR 62 CFM tion 576 BTUs hr Heat dissipation Maximum Power and Environmental Input Voltage Input Voltage Management Management 100 240 VAC 50 60Hz CPU NO CPU CPU Externally managed Power Typ Q Typ Consumption PS unit fan is at 80 56Gb s Optical Cables 143 W Passive cables 103 W Power numbers Max O Max 56Gb s Optical Cables 169 W Passive cables 123 W
6. de brancher toutes les alimen tations stabilise es Mellanox Technologies 31 Rev 1 1 Installation 3 5 Battery Replacement Mellanox Technologies does not support battery replacement Customer removal of the switch cover will void the warrantee Only Remove the cover to comply with WEEE directives or to dis assemble for environmentally approved disposal A Customer removal of the switch cover will void the warrantee 3 6 Configuring the Switch the First Time The procedures described in this chapter assume that you have already installed and powered on your switch according to the instructions in this document 3 7 Externally Managed Switches Externally managed switches that is the SX6015 switches do not have a management chip or board within the switch These switches do not get configured On externally managed switches the CONSOLE Ethernet and USB connectors are not found do not work Instead there is an I2C connector The externally managed switches are Plug and Play and all firmware updates should be done in band The I2C connection should only be used if the firmware image was corrupted to the point that the regular firmware tools cannot successfully reburn the correct image You will need to confirm that your switch has the latest firmware For externally managed switches confirm that the latest FW revision is installed on this device Use the CLI to get the cur rently installed revision and then go to Mellanox
7. 1f the LED indicator is OFF then the fan unit is disconnected There are two possible air flows for the fan unit The air flow depends on the switch model See Air Flow on page 20 for an explanation of the model numbers and labels Make sure that the fans have the air flow that matches the model number An air flow A Opposite to the switch design will cause the switch to operate at a higher less than optimal temperature Figure 28 Air Flow Labels HU JE I LLLI ULL T LE LEE EEE LET E MN OME These air flow labels must be the same gt To extract a Fan Unit 1 Using two fingers push both latch releases towards each other simultaneously while pulling the fan module out of the switch As the fan unseats the fan status indicator will turn off 1 Grasping the handle with your right hand push the latch release with your thumb while pull ing the handle outward As the fan unit unseats the fan unit status indicators will turn off 2 Remove the fan unit Figure 29 Fan Module Latches aaa Lil These two latches must be pushed towards each other at the same time while the module is pulled out gt To insert a FAN Unit 1 Make sure the mating connector of the new unit is free of any dirt and or obstacles Mellanox Technologies 41 Rev 1 1 Extracting and Inserting FRUs 2 Insert the fan unit by sliding it into the opening until slight resistance is felt Continue press ing the fan unit until it seat
8. HLTLLLELTELEIT Z 151 IND x IMS FI These air flow labels must be the same Red is connector side inlet to power side outlet LUN 1 oa 7 LL LULU en JE SE ES 12 p7 li III Ini y ILE HEG Blue is power side inlet to connector side outlet Figure 16 shows a mismatch in component airflow Figure 16 Component Airflow Mismatch Wrong LIT Te 7 HIE Inn d o M These air flow labels must be the same Any mismatch of these labels will affect the switch temperature Table 10 shows the icons that indicate airflow using arrows and colors All components need the same colored airflow icons Mellanox Technologies 21 Rev 1 1 Basic Operation Table 10 Air Flow Direction OPN Direction Label Mesamia Description Connector side inlet to power side R outlet Power side inlet to connector side outlet F 2 4 QSFP Cable Power Budget Classification All SwitchX 2 QSFP switches are designed for fiber cables with a max power per module of 2W Typical power per port is 1 5W 2 5 Interfaces 2 5 1 Port Connector Interfaces The connector side of the switch has 18 QSFP ports These are placed in one row The ports are labelled from 1 to 18 Figure 17 Port Numbering 22 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 2 5 2 Management and Firmware Updating Interfaces There are six inte
9. cautions sp du mat riel de bas en haut 10 Fuite gt 3 5mA Leakage gt 3 5mA ATTENTION La connexion la terre des forts courants de fuite est essentielle YN avant le branchement de l alimentation Avant de brancher l appareil la conduite d alimentation les vis de protection la terre du terminal de l appareil doivent tre appliqu es l installation de protection la Terre du b timent 11 Forts Courants de Fuite High Leakage Current 4 Attention Forts courants de fuite Il est essentiel de relier a la terre avant de brancher l alimentation 12 Ajouter une information de connexion la masse Connect a Valid Ground to this Device terre du terminal de l appareil doivent tre appliqu es l installation de protection la Avant de brancher l appareil la conduite d alimentation les vis de protection la Terre du b timent 13 Installation du mat riel LA Ce mat riel ne doit tre install remplac ou entretenu que par du personnel form et qualifi 14 Elimination du mat riel h L limination de ce mat riel doit s effectuer dans le respect de toutes les l gislations et r glementations nationales en vigueur Mellanox Technologies 65 Rev 1 0 15 Codes lectriques locaux et nationaux LA Ce mat riel doit tre install dans le respect des codes lectriques locaux et nationaux 16 Codes d installation L appareil doit tre install selon l ancienne
10. Avertissement Syst me de fusible neutre double p le Veuillez debrancher tous les A cordons d alimentation avant d ouvrir le boitier de ce produit ou de toucher un de ses composants internes 6 Orages dangers lectriques h Pendant un orage il ne faut pas utiliser le mat riel et il ne faut pas brancher ou d brancher les c bles 7 Branchement d branchement des c bles en cuivre Les c bles InfiniBand en cuivre sont lourds et ne sont pas flexibles il faut donc faire LA tr s attention en les branchant et en les d branchant des connecteurs Consultez le fab ricant des c bles pour conna tre les mises en garde et les instructions sp ciales 64 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 0 8 Risque de choc et de danger Risque de choc et de danger LA electriques Le de branchment d une seule alimentation stabilise e ne de branch uniquement qu un module Alimentation Stabilise e Pour isoler completement le module en cause Il faut de brancher toutes les alimentations stabilise es Risk of electric shock and energy hazard LA The PSUs are all independent Disconnect all power supplies to ensure a powered down state inside of the switch platform 9 Montage et entretien sur baie ciales pour s assurer que le syst me reste stable En g n ral il faut remplir la baie avec Lorsque ce produit est mont ou entretenu sur baie il faut prendre des pr
11. Certifications o aan ON vta de 57 A 3 China CCC Warning Statement 58 Appendix B QSFP Interface 60 Appendix C RJ 45 CONSOLE and I2C Interface 62 Appendix D Accessory and Replacement Parts 63 Appendix E Avertissements de s curit d installation French 64 Appendix F Installation Sicherheitshinweise German 68 Appendix G Advertencias de seguridad para la instalaci n Spanish 71 4 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 0 List of Figures Figure 1 Connector Side View of the Switch 10 Figure 2 Pull outt b i 2 by uhr SR EN SER ken ans EE aimer nee le 11 Figure3 QSFP Switch System Power and Connector Side Panels 13 Figure 4 QSFP Switch System Connector Side 13 Figure 5 Externally Managed Switch System Power and Connector Side Panels 14 Figure 6 System Status Indicators 14 Figure 7 System LEDS uns ee A la PME EN Boog e 15 Figure 8 Fan Status LED Connector Side 16 Figure 9 Fan Status LED Power side 16 Figure 10 Power Status LED Connector Side
12. Gable power QSFP Temperature tie MAX 2 0W Operating 0 to 45 Celsius TYP 1 5W Non operating 40 to 70 Celsius Shock and Shock and Humidity Vibration _ Vibration ETSI EN 300 019 2 2 1999 09 Operating 5 95 non condensing Humidity 54 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 0 Protocol Support Speed protocol IB Management QoS Auto Negotiation of 56Gb s Externally managed Management 40Gb s 10Gb s 1Gb s Data Rate Data Rate 56Gb s Regulatory Compliance Safety US Canada cTU Vus EMC USA FCC Class A EU IEC60950 Emissions Canada ICES Class A International CB EU EN55022 Class A Safety Russia GOST R EU ENS55024 Class A EMC Argentina S mark DoC EU EN61000 3 2 Class A Emissions EU EN61000 3 3 Class A Japan VCCI Class A Australia New Zealand AS NZS CISPR 22 class A Environmental Acoustic EU IEC 60068 2 64 Random ISO 7779 Environmental Vibration 71dB A Acoustic EU IEC 60068 2 29 Shocks ETS 300 753 Type I II 7 1 Bel EU IEC 60068 2 32 Fall Test Scalability and Performance Switching Switching Switching Performance Performance Simultaneous wire speed any port Capacity 2 02Tb s Capacity to any port Reliability Availability and Serviceability Features Hot swapability Redundancy Hot Swappable 1 1 Redundant Fan Module Power Supplies Mellanox T
13. IP Configuration Example Mellanox configuration wizard Do you want to use the wizard for initial configuration y Step 1 Hostname switch 112126 Step 2 Use DHCP on mgmt0 interface yes n Step 3 Use zeroconf on mgmt0 interface no Step 4 Primary IP address 192 168 10 4 Mask length may not be zero if address is not zero interface mgmt0 Step 5 Netmask 0 0 0 0 255 255 255 0 Step 6 Default gateway 192 168 10 1 Step 7 Primary DNS server Step 8 Domain name Step 9 Enable IPv6 yes yes Step 10 Enable IPv6 autoconfig SLAAC on mgmt0 interface no no Step 11 Admin password Enter to leave unchanged You have entered the following information Hostname switch 112126 Use DHCP on mgmt0 interface no Use zeroconf on mgmt0 interface no Primary IP address 192 168 10 4 Netmask 255 255 255 0 Default gateway 192 168 10 1 Primary DNS server Domain name 9 Enable IPv6 yes 10 Enable IPv6 autoconfig SLAAC on mgmt interface no 11 Admin password Enter to leave unchanged unchanged Qo MON Unc SAO IE To change an answer enter the step number to return to Otherwise hit lt enter gt to save changes and exit Choice Configuration changes saved To return to the wizard from the CLI enter the configuration jump start command from configure mode Launching CLI switch name gt standalone master gt 5 Before attempting a remote for example SSH connection t
14. Manual Rev 1 1 Table 13 Configuration Wizard Session IP Zeroconf Configuration Wizard Session Display IP Zeroconf Configuration Example Mellanox configuration wizard Do you want to use the wizard for initial configuration y Step 1 Hostname switch 112126 Step 2 Use DHCP on mgmt0 interface no Step 3 Use zeroconf on mgmt0 interface no yes Step 4 Default gateway 192 168 10 1 Step 5 Primary DNS server Step 6 Domain name Step 7 Enable IPv6 yes yes Step 8 Enable IPv6 autoconfig SLAAC on mgmt0 interface no no Step 9 Admin password Enter to leave unchanged You have entered the following information Hostname switch 112126 Use DHCP on mgmt0 interface no Use zeroconf on mgmt0 interface yes Default gateway 192 168 10 1 Primary DNS server Domain name Enable IPv6 yes Enable IPv6 autoconfig SLAAC on mgmt0 interface yes Admin password Enter to leave unchanged unchanged DO ICON ER DE To change an answer enter the step number to return to Otherwise hit lt enter gt to save changes and exit Choice Configuration changes saved To return to the wizard from the CLI enter the configuration jump start command from configure mode Launching CLI lt switch name gt standalone master gt Mellanox Technologies 35 Rev 1 1 Installation Table 14 Configuration Wizard Session Static IP Configuration Wizard Session Display Static
15. Rev 1 1 Basic Operation 2 3 Table 9 Connector Physical and Logical Link Assignments for IB LED Status LED Description Off No power to the port Solid Green Logical link is up Flashing Green Data activity flashing speed is proportional to data transfer speed Solid Orange Physical link is up Flashing Orange A problem with the physical link The LED indicator corresponding to each data port will light orange when the physical connec tion is established that is when the unit is powered on and a cable is plugged into the port with the other end of the connector plugged into a functioning port When a logical connection is made the LED will change to green When data is being transferred the light will blink green Air Flow These switches can come with two air flow patterns The two patterns are Connector side inlet to power side outlet Power side inlet to connector side outlet The air flow is specified in the product model number See Mellanox Part Numbering Legend on page 9 On the PS unit and fan modules the air flow direction can be seen on the power side panel All components need to have the same air flow A mismatch in the air flow will affect the heat dissipation A All PS units must have the same air flow as the fan module 20 Mellanox Technologies SwitchX amp 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 Figure 15 Component Air Flow 7927 il
16. When this product is mounted or serviced in a rack special precautions must be taken A to ensure that the system remains stable In general you should fill the rack with equip ment starting from the bottom to the top 9 Equipment Installation This equipment should be installed replaced or serviced only by trained and qualified personnel 10 Equipment Disposal i Disposal of this equipment should be in accordance to all national laws and regula tions 11 Local and National Electrical Codes This equipment should be installed in compliance with local and national electrical codes 12 Hazardous Radiation Exposure Caution Use of controls or adjustment or performance of procedures other than those specified herein may result in hazardous radiation exposure CLASS 1 LASER PRODUCT and reference to the most recent laser standards IEC 60 825 1 1993 A1 1997 A2 2001 and EN 60825 1 1994 A1 1996 A2 2001 28 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 13 UL Approved AC Power Cords For North American power connection select a power supply cord that is UL Listed A and CSA Certified 3 conductor 18 AWG terminated in a molded on plug cap rated at 125 V 15 A with a minimum length of 1 5m six feet but no longer than 4 5m For European connection select a power supply cord that is internationally harmonized and marked lt HAR gt 3 conductor
17. XH99A Tieseu TISSPON AND dpXL UpXL AND dzXL UZXL AND 61 Mellanox Technologies Rev 1 0 Appendix D RJ 45 CONSOLE and I2C Interface The RJ 45 CONSOLE and I2C interface uses the EIA 568A standard wiring color coding en CONSOLE Table 19 RJ 45 CONSOLE Pinout Connection Signal Pin Color DD TXD RJ 45 RTS 1 G W TO were Pin 1 p RU RMS DIR TXD RJ 45 DTR 2 G pu R45_TXD RXD gt RXD RJ 45 TXD 3 O W GND da GND GND 4 Bl av Ping GND GND 5 BI W Looking into the Socket RXD RJ 45 RXD 6 O GND GND 7 Br W GND GND 8 Br Table 20 RJ 45 I2C Pinout Connection Signal Pins Color PC TXD RJ 45 SDA l G W J34 TXD GND 2 G DE RXD GND 3 O W DD Pn1 RIAS SDA GND GND 4 Bl GD GND o o GND GND 5 BI W RXD GND EXD me RJ45 SL RXD RJ 45 SCL 6 O GND QD GD Ping GND GND 7 Br W GND GND 8 Br Looking into the Socket 62 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 0 Appendix E Accessory and Replacement Parts Table 21 OPNs for Replacement Parts Part Description OPN I2C DB9 or RJ 45 to USB Adapter MTUSB 1 Harness RS232 2M cable DB9 to RJ 45 for managed switches only HAR000028 Fan module with power supply side to connector side air flow MSX
18. by IBTA tion Volume 1 Release 1 2 1 Mellanox SwitchX 2 Switch Prod For possible hardware issues see the switch support product page This ucts Release Notes requires a customer support login Mellanox MLNX OS SwitchX 2 This document contains information regarding configuring and manag Software User Manual ing Mellanox Technologies SwitchX 2 Switch Platforms MLNX OS Software Command Ref Command Reference Guide for MLNX OS listing all of the commands erence Guide available through MLNX OS with explanations and examples MLNX OS Software Configuration Configuration Guide for MLNX OS displaying different configuration Guide scenarios All of these documents can be found on the Mellanox Website They are available either through the product pages or through the support page with a login and password Conventions Throughout this manual the names SX6018 and SX6015 and the term switch are used to describe the 18 port 56Gb s IB switch unless explicitly indicated otherwise The following icons are used throughout this document to indicate information that is important to the user Y This icon makes recommendations to the user This icon indicates information that is helpful to the user 5 8 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 0 This icon indicates a situation that can potentially cause damage to hardware or soft
19. com to check the revision of the latest version For more information on how to get the current firmware version see Section 8 4 1 Current Firm ware Revision For Externally Managed Switches on page 50 3 8 Managed Internally Managed Switches Internally managed switches must be configured before they will work Follow the procedures below to configure the switch gt To configure the switch Externally managed switches do not get configured The CONSOLE Ethernet and A USB connectors are not found on externally managed switches Instead there is an I2C connector 1 Connect the host PC to the Console RJ 45 port of the switch system using the supplied cable The Console ports for systems are shown below as examples 32 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 Figure 24 Console Port EEEE Connect the host PC to this port EZH il i mh LA Make sure to connect to the Console RJ 45 port of the switch and not to the Ethernet MGT port No remote IP connection is available at this stage 2 Configure a serial terminal program for example HyperTerminal minicom or Tera Term on your host PC with the settings described in Table 11 Table 11 Serial Terminal Program Configuration Parameter Setting Baud Rate 9600 Data bits 8 Stop bits 1 Parity None Flow Control None
20. el usuario final deber n suministrar N un confinamiento adecuado para componentes el ctricos y mec nicos y contra incen dio 12 Eliminaci n de equipos La eliminaci n definitiva de este equipo se debe efectuar conforme a todas las leyes y reglamentaciones nacionales 13 C digos el ctricos locales y nacionales LA Este equipo se debe instalar conforme a los c digos el ctricos locales y nacio nales 72 Mellanox Technologies SwitchX amp 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 0 14 Cable de alimentaci n homologado por UL y con certificaci n CSA En conexiones de Am rica del Norte seleccionar un cable de alimentaci n A homologado por UL y con certificaci n CSA de tres conductores 16 AWG termi nado en un enchufe moldeado con capuch n de 125 voltios nominal 13 A con una longitud m nima de 1 5 metros pero no m s de 4 5 metros En conexiones europeas seleccionar un cable de alimentaci n armonizado internacionalmente y marcado lt HAR gt de tres conductores hilo de 1 0 mm2 como m nimo 300 voltios nominal con cobertura protectora aislante de PVC El cable debe tener un enchufe moldeado con capuch n de 250 voltios nominal 10 A 15 A adir conexi n a tierra Antes de conectar el dispositivo a la l nea de alimentaci n los tornillos del terminal de A la puesta a tierra de protecci n del dispositivo se deben conectar a la puesta a tierra de protecci n de la instalac
21. empfohlenen Temperatur von 45 C 113 F betrieben werden Es ist ein Luft strom von 200 LFM bei maximaler Umgebungstemperatur erforderlich Au erdem sollten mindestens 8 cm 3 in Freiraum um die Bel ftungs ffnungen sein um einen einwandfreien Luftstrom zu gew hrleisten 3 Stapeln des Chassis ha Das Chassis sollte nicht auf andere Ger te gestapelt werden Wenn das Chassis herun terf llt kann es zu Verletzungen und Besch digungen an Ger ten f hren 4 Redundanter Stromversorgungsanschluss Elektrische Gefahr A Dieses Produkt verf gt ber eine Abdeckung ber dem Bereich f r die redundante Stromversorgung Betreiben Sie das Produkt nicht wenn diese Abdeckung nicht sicher festsitzt oder entfernt wurde 5 Bei Gewitter Elektrische Gefahr A Arbeiten Sie w hrend eines Gewitters und Blitzschlag nicht am Ger t schlie en Sie keine Kabel an oder ab 6 AnschlieBen Trennen von Kupferkabel A Kupferkabel sind schwer und nicht flexible Deshalb m ssen sie vorsichtig an die Anschliisse angebracht bzw davon getrennt werden Lesen Sie die speziellen Warnun gen und Anleitungen des Kabelherstellers 7 Gafahr des elektrischen Schocks M A Gafahr des elektrischen Schocks Entferrnen des Netzsteckers elnes Netzteils span nungsfrei Um alle Einhieten spannungsfrei zu machen sind die Netzstecker aller Netz teile zu entfernen Risk of electric shock and energy hazard The PSUs are all independent Discon
22. is only guaranteed to work with approved Mellanox ConnectX 9 3 adapters SU EN Mellanox Technologies 11 Rev 1 1 Overview 1 4 Internally Managed vs Externally Managed The following table shows which switches come with a management CPU and which do not All internally managed switches have internal chassis management and can support IB fabric of up to 648 nodes Internally managed switches need an initial configuration before they will start working Externally Managed Unmanaged switches are plug and play out of the box Updating the firm ware stack for the externally managed switches is done in band only You will need to confirm that your switch has the latest firmware For externally managed switches confirm that the latest FW revision is installed on this device Use the CLI to get the currently installed revision and then go to Mellanox com to check the revision of the latest version Externally managed switches can only run in an IB fabric Externally managed switches cannot run Ethernet protocol Table 3 Switch Management Externally Family Internally Management Connections Managed SX6015 Externally Plug and Play Managed All firmware updates should be done in band using Mellanox firmware man Unmanaged agement tools I2C port access using MTUSB 1 device is required for firm ware updates if in band burning is not possible SX6018 Internally RS232 cable DB9 to RJ 45 included in the box to connect t
23. leicht zug ngliche Auflistung Abzweigleitung berstrom Schutzeinrichtung 20 A bewertet werden m ssen in dem Geb ude Verkabelung 19 Zweipolige bzw Neutralleiter Sicherung im Netzteil Achtung 4 Zweipolige bzw Neutralleiter Sicherung im Netzteil Netzstecker ziehen um sicher zustellen da keine Spannung am Ger t anliegt Entfernen Sie alle Netzkabel vor dem ffnen der Abdeckung dieses Produkts oder dem Ber hren der Innenteile 20 Bodily Injury Due to Weight Use enough people to safely lift this product A o o 40 lbs 40 70 lbs 70 121 lbs gt 121 lbs 18 kgs 18 32 kgs 32 55 kgs gt 55 kgs 70 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 0 Appendix H Advertencias de seguridad para la insta laci n Spanish 1 Instrucciones de instalaci n Antes de conectar el equipo a la fuente de alimentaci n leer todas las instrucciones de A instalaci n 2 Lesi n corporal por peso A e o 40 lbs 40 70 Ibs 70 121 lbs gt 121 lbs lt 18 kgs 18 32 kgs 82 55 kgs gt 55 kgs 3 Lesi n corporal por peso This equipment is very heavy and should be moved using a mechanical lift to avoid injuries 4 Sobrecalentamiento No se debe utilizar el equipo en un rea con una temperatura ambiente superior a la A m xima recomendada 45 C Adem s para garantizar una circulaci n de aire adec uada se debe dejar como m nimo un espacio
24. minimum 0 75 mm2 wire rated at 300 V with a PVC insulated jacket The cord must have a molded on plug cap rated 250 V 10 A 14 Do Not Use the Switch as a Shelf or Work Space Caution Slide rail mounted equipment is not to be used as a shelf or a work space 15 WEEE Directive According to the WEEE Directive 2002 96 EC all waste electrical and electronic equipment EEE should be collected separately and not disposed of with regular household waste Dispose of this product and all of its parts in a responsible and environmentally friendly way a The switch platform can be rack mounted and is designed for installation in a standard 19 rack The power side of the switch includes a hot swap power supply module a blank cover for an optional second PS unit for redundancy and a hot swap fan tray There are two possible air flow directions Be sure that the switch air flow direction is compatible with your system rack and PS units The connector side of the switch has the QSFP ports system LEDs and management con nection ports The switch platform contains auto sensing 100 240 VAC connections for all possible PS units The installer should use a rack capable of supporting the mechanical and environmental charac teristics of a fully populated platform The rack mounting holes conform to the EIA 310 standard for 19 inch racks Take pre A cautions to guarantee proper ventilation in order to maintain good airflow at ambient temperat
25. the fans are all up and running 2 Make sure that there is nothing blocking the front or rear of the chassis and that the fan modules and ventilation holes are not blocked especially dust over the holes 3 If you find dust blocking the holes it is recommended to clean the fan unit and remove the dust from the front and rear panels of the switch using a vacuum cleaner Issue 4 The green power LED for the fans does not come on 1 Check that the Power LEDs are on 2 Remove and reinstall the fan unit Make sure the mating connector of the new unit is free of any dirt and or obstacles See Section 6 Switch Shut Down Procedure on page 46 3 Check that the mating connector pins are not bent LA Do not run the switch if the System Status LED for the Fans is Red Issue 5 The link LED for the connector does not come on 1 Check that both ends of the cable are connected 2 Check that the locks on the ends are secured 3 Make sure that the latest firmware version is installed on all of the HCA cards and the switch 4 If media adapters are used check that the all connections are good tight and secure Issue 6 The activity LED does not come on When running IB check that the Subnet Manager has been started Issue 7 The switch is off 1 Unplug the switch 2 Wait 5 minutes 3 Plug in the switch 4 If the switch does not come on check the power supplies Mellanox Technologies 51 Rev 1 1 Troubleshooting 5 If
26. the switch comes on Use the MLNX OS management CLI or Web GUI to determine the cause of the Shutdown 6 Check the temperature 7 Check the Fan status Issue 8 The switch is not working and unresponsive 1 Reset the switch If resetting the switch does not work 1 Unplug the switch 2 Wait 5 minutes 3 Plug in the switch 4 If the switch does not come on check the power supplies 5 If the switch comes on use the MLNX OS management CLI or Web GUI to determine the cause of the shutdown Issue 9 The last software update did not succeed 1 Connect the RS232 connector CONSOLE to a laptop 2 Push the reset button on the switch or management module 3 You will have 5 seconds to stop the U Boot by pressing Control B 4 Choose the image to upload Only use image 1 or image 2 U Boot 2009 01 mlnx1 4 date GP AMCC PowerPC 460EX Rev A at 1000 MHz PLB 200 OPB 100 EBC 100 MHz Security Kasumi support Bootstrap Option H Boot ROM Location I2C Addr 0x52 Internal PCI arbiter disabled 32 kB I Cache 32 kB D Cache Board Mellanox PPC460EX Board FDEF No I2C ready DRAM 2 GB ECC enabled 400 MHz CL3 FLASH 16 MB AND 1024 MiB ACIER Bus Dev VenId DevId Class Int PCIEO link is not up PCIEl successfully set as root complex 01 00 15b3 bd34 0c06 00 et ppc 4xx eth0 ppc 4xx ethl Hit Ctrl B to stop autoboot 0 ellanox MLNX OS Boot Menu 1 SX PPC M460EX 3 2 0000 dev HA 2013 04 10 12 02 49 ppc
27. une fiche moul e d une tension nominale de 125 V 15 A avec une longueur minimale de 1 5 m 6 pieds et d une longueur maxi male de 4 5 m 18 pieds Pour les prises lectriques en Europe choisissez un cordon lectrique harmonis inter nationalement et marqu lt HAR gt a 3 conducteurs d un diam tre de fil minimum de 0 75 mm2 d une tension nominale de 300 V avec une gaine isol e en PVC Le cordon doit avoir une fiche moul e d une tension nominale de 250 V et d une intensit nominale de 10 A 66 Mellanox Technologies SwitchX amp 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 0 21 Ah 40 lbs 40 70 lbs 70 121 lbs gt 121 lbs lt 18 kgs 18 32 kgs 82 55 kgs gt 55 kgs 22 Conform ment la Directive WEEE Conform ment la Directive WEEE 2002 96 EC les d chets d quipements lec triques et lectroniques EEE doivent tre tri s et ne peuvent tre limin s avec les d chets m nagers Veuillez disposer de ce produit et de tous ses composants de mani re responsable et Ez respectueuse de l environnement Mellanox Technologies 67 Rev 1 0 Appendix G Installation Sicherheitshinweise German 1 Installationsanleitungen Ah Lesen Sie alle Installationsanleitungen bevor Sie das Ger t an die Stromversorgung anschlieBen 2 bertemperatur A Dieses Ger t sollte nicht in einem Bereich mit einer Umgebungstemperatur ber der maximal
28. uoo see er tas es 8 Abbreviations s cssc sigs dec cR ERU UR ep UR OE Aa Aas ste Ses AU een ee 9 Mellanox Part Numbering Legend 9 Chapter 1L Overview ceo Suo AN ke 10 Tel Features eeu nente ine ee Pere cles 10 1 2 Serial Number and Product Version Information 10 13 FDR Transmission Rate 11 1 4 Internally Managed vs Externally Managed 12 Chapter 2 Basic Operation ssessessessessessessesses 13 2 1 Switch Platform Hardware Overview 13 2 2 Status LEEDS Senn DA rete E reo Er REM She RCM VAR 13 2 3 Ar FLOW So esi seeds A a 20 2 4 QSFP Cable Power Budget Classification 22 2 5 Interfaces os ue O SUE RSS 22 Chapter 3 Installation oed he se WER ROC Ie MERE ORR ER OOS 27 3 1 Installation Safety Warnings 27 3 2 Package Contents 2424 48 en pesa ee ee a EISE ie es 29 3 3 Grounding the Switch 2 30 3 4 Power Connections and Initial Power On 31 3 5 Battery Replacement 32 3 6 Configuring the Switch the First Time 32 3 7 Externally Managed Switches 32 3 8 Managed Internally Managed Switches
29. you want to rerun the wizard run the following commands sx 43 standalone master gt enable sx 43 standalone master configure terminal sx 43 standalone master config configuration jump start Mellanox Technologies 37 Rev 1 1 Installation 3 10 Connecting to the Switch Platform 3 10 1 Starting a Remote Connection to the Switch There are two ways to access the management CPU e Via the Ethernet connector for remote access SSH Telnet e Web SNMP XML Via KVM connections locally For details on the SNMP commands and MIBs supported refer to the MLNX OS Software Command Reference Guide 3 11 Starting an SSH Connection to the Switch CLI 1 Set up an Ethernet connection between the switch and a local network machine the remote machine henceforth using a standard RJ 45 connector 2 Connect to the remote machine rem_machl is used as an example 3 Start a remote shell to the switch using the following command using switch IP address is the IP address of the switch rem machl gt ssh 1 username switch ip address gt Mellanox MLNX 0S Switch Management Password Last login March 13 2013 from 192 168 10 1 Mellanox Switch sx 43 standalone master gt 4 You can enter any supported command now 38 Mellanox Technologies SwitchX amp 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 4 Extracting and Inserting FRUs Never run the switch for mor
30. 13 GND Ground 32 GND GND 7 Rx3p Receiver Non Inverted Data Out 38 Tx3p Tx4p 14 put 34 ty3n Tx4n 3 RE anp 4 15 Rx3n Receiver Inverted Data Output gt Txip Tx2p 3 16 GND Ground EAT an 17 Rxlp Receiver Non Inverted Data Out put 18 RxIn Receiver Inverted Data Output 19 GND Ground 20 GND Ground 21 Rx2n Receiver Inverted Data Output 3 2 Rx2p Receiver Non Inverted Data Out put 3 23 GND Ground 24 Rx4n Receiver Inverted Data Output 3 25 Rx4p Receiver Non Inverted Data Out put 3 26 GND Ground 27 ModPrsL Module Present 28 IntL Interrupt 29 Vcc Tx 3 3 V Power supply transmitter 30 Vccl 3 3 V Power Supply 31 LPMode Low Power Mode 32 GND Ground Tx3n Transmitter Non Inverted Data 60 Mellanox Technologies Rev 1 0 SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual lews QSFP Connector Male and Female Vi Figure 32 Top 18 35 View into Rear of Connector GND AX n TXtp GND TX3n TX3p GND LPMode Vcc1 VccTx Intl ModPrsL GND RX4p RX4n GND RX2p RX2n xs GND N N S sx ee Ss GND TX2n TX2p GND TX4n TX4p GND ModSelL ResetL LO VccRx co SCL SDA GND a o E RX3p RX3n GND RX1p RX1n GND 8 50 View into Front of Cage 1 1 0 9 8 7 6 5 4 3 2 AND qo diXH AND UEXH dexu AND vas 10s
31. 60 FF Fan module with connector side to power supply side air flow MSX60 FR 300W Power Supply w Power Supply Side to Connector side air flow MSX60 PF 300W Power Supply w Connector side to Power Supply side air flow MSX60 PR Power cord Type C13 C14 ACC000500 Mellanox Technologies 63 Rev 1 0 Appendix F Avertissements de s curit d installation French 1 Instructions d installation h Lisez toutes les instructions d installation avant de brancher le mat riel la source d alimentation lectrique 2 Temp rature excessive Ce mat riel ne doit pas fonctionner dans une zone avec une temp rature ambiante N d passant le maximum recommand de 45 C 113 F Un flux d air de 200LFM cette temperature ambiante maximale est necessaire En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouver tures de ventilation 3 Empilage du ch ssis LA Le ch ssis ne doit pas tre empil sur un autre mat riel Si le ch ssis tombe il peut pro voquer des blessures corporelles et des d gradations de biens 4 Connection d Alimentation electrique excedentaire dangers lectriques Ce produit comporte un couvercle transparent sur l espace pour l alimentation lec LA trique redondante Ne pas faire fonctionner le produit si le couvercle transparent n est pas solidement fix ou s il est enlev 5 Syst me de fusible neutre double p le
32. Each subnet needs one subnet manager to discover activate and manage the subnet Each network requires a Subnet Manager to be running in either the switch itself switch based or on one of the nodes which is connected to the fabric host based Y No more than two subnet managers are recommended for any single fabric 8 2 Fabric Inspector Diagnostics Fabric Inspector is a plug amp play software for fee within MLNX OSQ displaying and filtering all identified systems and nodes within the fabric Fabric Inspector includes a complete set of tools for fabric wide diagnostics to check node node and node switch connectivity and to verify routes within the fabric Advanced filtering allows creating filtering rules on a system wide basis between nodes or port connections based on traffic patterns and user assigned system names GUIDs 8 3 Upgrading Software Software and firmware updates are available from the Mellanox Support website Copy the update to a known location on a Remote server within the user s LAN Use the CLI or the GUI in order to perform software upgrades For further information please refer to the the MLNX OS Software User Manual section Upgrading MLNX OS Software Be sure to read and follow all of the instructions regarding the updating of the software on your switch system 8 4 Externally Managed Switch In order to get information regarding the externally managed switch you need to download the Mellanox MFT tools f
33. MLNX OS management SW indicating to system administration that the switch has crossed the Warning threshold Note that this temperature threshold does not require nor lead to any action by hardware such as switch shutdown 2 Critical 120C When the SwitchX device crosses this temperature the firmware will automatically shut down the device 3 Emergency 130C In case the firmware fails to shut down the SwitchX device upon crossing the Critical thresh old the SwitchX device will auto shutdown upon crossing the Emergency 130C threshold Mellanox Technologies 59 Rev 1 0 Appendix C QSFP Interface Table 18 InfiniBand QSFP Connector Pinout Connector Connecto Pin r Pin Signal Description Number Name 1 GND Ground 2 Tx2n Transmitter Inverted Data Input J3 and J6 3 Tx2p Transmitter Non Inverted Data Input 19 y oO a 4 GND Ground a er Rxin 9 22 a m o 5 Tx4n Transmitter Inverted Data Input 23 GND GND 16 6 Tx4p Transmitter Non Inverted Data ern Rx4n Rx3n 5 nu 25 Rx4p Rx3p 14 7 GND Ground 36 cmo Ei 8 ModSelL Module Select 27 ModPrsL spa 12 9 ResetL Module Reset m sc 10 Vcc Rx 3 3 V Power supply receiver vest Veo Rx 10 11 SCL 2 wire serial interface clock 30 9 nos ste 12 SDA 2 wire serial interface data 3 LPMode ModSelL S
34. Mellanox TECHNOLOGIES SwitchX amp 2 18 Port QSFP FDR InfiniBand System Hardware User Manual P N MSX6018F 1SFS MSX6018T 1SFS MSX6018F 1BRS MSX6018T 1BRS MSX6015F 1SFS MSX6015T 1SFS MSX6015F 1BRS MSX6015T 1BRS Rev 1 0 www mellanox com Rev 1 0 NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO S AND OR THE SYSTEM USING IT THEREFORE MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT INDIRECT SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES OF ANY KIND INCLUDING BUT NOT LIMITED TO PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN A
35. NY WAY FROM THE USE OF THE PRODUCT S AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE Mellanox TECHNOLOGIES Mellanox Technologies Mellanox Technologies Ltd 350 Oakmead Parkway Suite 100 Beit Mellanox Sunnyvale CA 94085 PO Box 586 Yokneam 20692 U S A Israel www mellanox com www mellanox com Tel 408 970 3400 Tel 4972 0 74 723 7200 Fax 408 970 3403 Fax 972 0 4 959 3245 Copyright 2013 Mellanox Technologies All Rights Reserved Mellanox Mellanox logo BridgeX ConnectX CORE Direct InfiniBridge InfiniHost InfiniScale MLNX OS PhyXG SwitchX UFM Virtual Protocol Interconnect and Voltaire are registered trademarks of Mellanox Technologies Ltd Connect IBTM FabricIT Mellanox Open Ethernet M MetroXTM MetroDX ScalableHPC Unbreakable Link are trademarks of Mellanox Technologies Ltd All other trademarks are property of their respective owners 2 Mellanox Technologies Document Number 4161 SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 0 Table of Contents Table oF Contents ccs eco A A EA AA 3 List of FIGURES aa e pe Se A eRe EASE aw Eee 5 Listof Tables suser re CHS RAE EN IA Ee bete ct 6 Revision History sc estu s o Ru AN ee ete We bee ER X ERN About this Manual ae en 8 Intended A dience spade a G kp Ade aa det es 8 Related Documentation 8 CONVENTIONS
36. abel F r Nordamerika Stromanschluss w hlen Sie ein Netzkabel das UL und CSA Certi fied A 3 Leiter 18 AWG mit einem angespritztem Stecker bewertet bei 125 V 15 mit einer Mindestl nge von 1 5 m Six Feet aber nicht mehr als 4 5 m F r die europ ischen Zusammenhang w hlen Sie ein Netzkabel das international harmonisiert und der Aufschrift lt HAR gt 3 Leiter mindestens 0 75 mm2 Draht bewertet mit 300 V mit einem PVC Mantel isoliert Das Kabel muss eine angespritztem Stecker bewertet bei 250 V 10 A Mellanox Technologies 69 Rev 1 0 15 Ableitstrom 3 5mA LEAKAGE gt 3 5mA LA WARNUNG Hohe Ableitstrom Earth Verbindung bevor Sie die Verbindung von wesentlicher Bedeutung werden 16 Add GND Verbindung Informationen Bevor Sie dieses Ger t an das Stromnetz die Schutzerde Terminal Schrauben dieses Hn Ger t muss an den Schutzleiter in der Geb udeinstallation 17 Installation Codes Dieses Ger t muss installiert sein entsprechend auf die neueste Version des Landes LA National Electrical Code F r Nordamerika m ssen in bereinstimmung mit den gel tenden Vorschriften in der US amerikanischen National Electrical Code und dem Canadian Electrical Code 18 Zusammenschaltung von EINHEITEN Kabel f r den Anschluss an das Ger t RS232 und Ethernet Schnittstellen m ssen UL LA zertifiziert Typ DP 1 oder DP 2 Hinweis wenn nicht mit Wohnsitz in LPS Schal tung berstromschutz Eine
37. ble 43 Figure 31 SX6018 Port Splitting Options 43 Figure 32 QSFP Connector Male and Female Views 61 Mellanox Technologies 5 Rev 1 0 List of Tables Table 1 Revision History Table 7 Table 2 Reference Documents and Web Sites 8 Table 3 Switch Management 12 Table 4 System Status LED Assignments 15 Table 5 Fan Status LED Assignments 16 Table 6 PS Unit Status LED Assignments 18 Table 7 Bad Port LED Assignments 19 Table 8 Port Connector Physical and Logical Link Assignments for Ethernet 19 Table 9 Connector Physical and Logical Link Assignments for IB 20 Table 10 Art Plow Directions ou inb Fein Peeters od e deter Voss bd id 22 Table 11 Serial Terminal Program Configuration 33 Table 12 Configuration Wizard Session IP Configuration by DHCP 34 Table 13 Configuration Wizard Session IP Zeroconf Configuration 35 Table 14 Configuration Wizard Session Static IP Configuration
38. ctor side panel All connectors can sup port fiber cables 2 2 Status LEDs 2 2 1 System Status Indicators The System Status Indicators are located to the left of the connectors on the connector side panel and on the power side at the far right Both of these LEDs give identical information Mellanox Technologies 13 Rev 1 1 Basic Operation Figure 5 Externally Managed Switch System Power and Connector Side Panels LLL ET ALI FA HUE ni HUE a LLLLLES FG Power Side Panel These two LEDs indicate the status of the switch as assigned in Table 4 on page 15 14 2A 3A 44 54 64 7A EE Y EP Y 104 114 124 134 144 154 164 174 184 fh TET Connector Side Panel Figure 6 System Status Indicators Both of these LEDs show the switch status Power side Connector side status LED status LED The system status indicators should display as follows when the switch is plugged in within five minutes the STATUS LED should light up green the PS1 LED for the plugged in PS unit should light up green the PS2 LED for the second PS unit should light up green only if a second PS unit is installed in the switch for redundancy and it is connected to a power source e if two PS units are installed and only one PS unit is connected to a power supply the PS2 LED will be red ifonly one PS unit is installed in the switch the PS2 LED will be off the FAN LED should light up green If the FAN LED shows red troubles
39. de 8 cm 3 pulgadas alrededor de las aberturas de ventilaci n 5 Apilamiento del chasis Los chasis no se deben apilar sobre otros equipos La ca da del chasis podr a causar lesiones corporales as como da os al equipo 6 Dos fusibles uno en el polo y otro en el neutro A Dos fusibles uno en el polo y otro en el neutro Quitar los cables de corriente antes de abrir la tapa de este producto o tocar cualquier componente interno Mellanox Technologies 71 Rev 1 0 7 Cuando hay rayos peligro de descarga el ctrica No utilizar el equipo ni conectar o desconectar cables durante per odos de actividad de A rayos 8 Conexi n y desconexi n del cable Copper Dado que los cables de cobre son pesados y no son flexibles su conexi n a los conec A tores y su desconexi n se deben efectuar con mucho cuidado Para ver advertencias o instrucciones especiales consultar al fabricante del cable 9 Montaje y mantenimiento de bastidores Al instalar o realizar el mantenimiento de este aparato en un bastidor es preciso adop A tar precauciones especiales para garantizar que el sistema se mantenga estable En gen eral en un bastidor los equipos se deben instalar comenzando desde abajo hacia arriba 10 Instalaci n de equipos La instalaci n el reemplazo y el mantenimiento de este equipo estar n a cargo nica mente de personal capacitado y competente 11 Asegurar confinamientos adecuados El fabricante del producto final o
40. e button is held down for 15 sec onds the switch is reset and the password is deleted You will then be able to enter without a pass word and make a new password for the user admin In the externally managed switches the reset button resets the CPU of the switch device 26 Mellanox Technologies SwitchX amp 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 3 Installation Installation and initialization of the switch platform are straightforward processes requiring attention to the normal mechanical power and thermal precautions for rack mounted equipment The externally managed switch platform does not require any programming or configura tion to operate as a basic InfiniBand switch and includes all of the necessary functionality to operate with external standard InfiniBand Subnet Management software The managed switch platform requires initial configuration to operate as an InfiniBand switch All internally managed switches come with an internal PPC based management board This board allows for internally managing the switch through a host PC or remotely through the Ether net 3 1 Installation Safety Warnings For Safety Warnings in French see Avertissements de s curit d installation French on page 64 for German see See Installation Sicherheitshinweise German on page 68 and for Spanish see Advertencias de seguridad para la instalaci n Spanish on page 71 1 Installation Instruc
41. e than 5 minutes with an uncovered open PSU or Fan A slot 4 1 Extracting and inserting the Power Supply Unit With both power supplies installed in the redundant configuration either PS unit may be extracted without bringing down the system A Make sure that the PS unit that you are NOT replacing is showing all green for both the PS unit and status indicators Figure 25 Fan and PS unit Air Flow Direction TE eu UNE ll P II Fan Air flow direction PS unit Air flow direction See Section 2 3 on page 20 for more information on air flow direction Power supply units have directional air flows similar to the fan module The Fan mod A ule airflow must coincide with the airflow of all of the PS units If the PS unit airflow direction is different from the fan module airflow direction the switch internal temper ature will be affected Figure 26 Power Supply Unit Extraction Push Here uM HE a Latch release If you are not immediately installing a new PSU you must install a blank so that the air flow for cooling the switch is not compromised Mellanox Technologies 39 Rev 1 1 Extracting and Inserting FRUs Figure 27 PS Unit Pulled Out 4 2 To extract a PS unit 1 Remove the power cord from the power supply unit d Grasping the handle with your right hand push the latch release with your thumb while pull ing the handle outward As the PS unit unseats the PS unit status indicators w
42. e unit Table 7 shows the fan status LED assignment 18 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 Table 7 Bad Port LED Assignments LED Configuration Description Off OK No symbol errors have been received in last few seconds Flashing Orange Error One or possibly more ports have just received a symbol error Possible causes are Badcable Badconnection Bad connector This LED lights up when one or more ports is receiving a symbol error The LED immediately goes off until the next symbol error is received 2 2 3 2 UID LED Switch Identifier The UID LED is a debug feature that lights a blue LED on the switch connector side panel for ease in finding a particular switch 2 2 3 3 Port Connector LED Indicators Above the ports are LEDs one for each port A Table 8 shows the port status according to the LED indication for Ethernet protocol Table 9 shows the port status according to the LED indica tion for InfiniBand protocol Figure 14 Port LEDs UTTTUTUUUUUUUUUUUUV 14 24A 3A 4A 5A 6A Table 8 Port Connector Physical and Logical Link Assignments for Ethernet LED Configuration LED Description Off Physical link is down Default Solid Green Physical link is up with no traffic Flashing Green Physical link is up with traffic Flashing Orange Physical errors Mellanox Technologies 19
43. echnologies 55 Table 17 SX6018 Specification Data Rev 1 0 Physical Size Size SX6018 1 716 1U Hx 16 85 W x SX6018 1 716 10 H x 16 85 W short 16 8 D Standard x24 75 D 43 6mm X 427 9mm X 428 9mm 43 6mm X 427 5mm X 628 9 mm Weight Center of Short 8 01 kg 1 PS unit Gravity As measured from the Connector 8 75 kg 2 PS units side left side bottom corner Weight Standard Short Center of 8 95 kg 1 PS unit 1PS unit 21 6mm X 193mm X 209 8mm Gravity 9 69 kg 2 PS units 2PS units 21 6mm X 214mm X 218 8mm Standard 1PS unit 21 6mm X 201mm X 337mm 2PS units 21 6mm X 214mm X 348mm A Mounting Mounting 19 Rack mount SerDes Speeds Connector SerDes Speeds 10 20 40 or 56 Gb s per port Types QSFP Connector Types Can use QSFP to SFP adapter modules A Air Flow Heat dissipa RI 62 CFM crt 675 BTUs hr Heat dissipation Maximum Power and Environmental Input Voltage Input Voltage Management Management 100 240 VAC 50 60Hz CPU PowerPC 460EX CPU CPU Power Typ Typ Consumption PS unit fan is at 80 56Gb s Optical Cables 165 W Passive cables 126 W Power numbers Max Max 56Gb s Optical Cables 198 W Passive cables 152 W QSFP Temperature Prenat MAX 2 0W Operating 0 to 45 Celsius TYP 1 5W Non operating 40 to 70 Celsius Shock and Shock and Humidity Vibration _ Vibration ETSI EN 300 019 2 2 1999 09 Operating 5 95 non condensing Humidit
44. es access to Flash and EEPROMS This connection is for FAE use only N The externally managed switches are Plug and Play and all firmware updates should be done in band YN This interface is for Debug and Troubleshooting only This interface is for FAEs and advanced users only All firmware updates should be done in band using Mellanox Firmware Management Tools You will need to order an MTUSB 1 USB to I2C adapter to make use of the I2C interface The RC connection provides access to Flash and EEPROMs This connection allows access to the switch for updating firmware Figure 20 RJ 45 12C Connector JUUUUUUUUUUUU SX6015 164A 17A 184A 2 5 3 Reset Button On the connector side panel under the system LEDs is a reset button D RST This reset but ton requires a tool to be pressed DO NOT use a sharp pointed object such as needle or push pin for pressing the Reset button Sharp objects can cause damage use a flat object to push the reset button Figure 21 Reset Button Mellanox A Press the reset button to reset the main and amp management CPUs and to delete the existing pe password PS2 QJ UID RST Mellanox Technologies 25 Rev 1 1 Basic Operation This button resets both the CPU of the switch device and the CPU of the management module It thereby resets all of the ports by bringing them down and powering them up when the button is pushed A quick push of this button performs this reset When th
45. et switches These switches must be configured to 100M 1G auto negotiation The IB Subnet Manager running on the SX6018 switches supports up to 648 nodes If the fabric includes more than 648 nodes you may need to purchase Mellanox s Unified Fabric Manager UFMO software package The managed switch system includes the following software components Embedded OS e Chassis manager and system BIST SNMP agent 3rd party tool integration GUI e Subnet manager e Remote logging e SSH telnet e Secured access in band and out band e Pv4 IPv6 network stack 8 1 Switch Management The chassis management includes the following features Upgrading SwitchX9 2 firmware Upgrading software Monitoring of e AC power to the PSUs DC power out from the PSUs Board temperature Fan module status Chassis failures 48 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 e Querying for switch serial numbers and their revisions software version and SwitchX 2 FW version 8 1 1 InfiniBand Subnet Manager The InfiniBand Subnet Manager SM is a centralized entity running in the switch It discovers and configures all the InfiniBand fabric devices to enable traffic flow between those devices The SM applies network traffic related configurations such as QoS routing partitioning to the fabric devices You can view and configure the Subnet Parameters SM via the CLI WebUI
46. f The system has no power Mellanox Technologies 15 Rev 1 1 Basic Operation 2 2 2 Fan Status Indicators The indicator is located to the left of the connectors on the connector side panel Figure 8 Fan Status LED Connector Side This is a two color LED The LED indicator on the Fan Module is on the right side of the module Figure 9 Fan Status LED Power side This is a two color LED Fan air flow direction indicator Table 5 shows the fan status LED assignment Table 5 Fan Status LED Assignments LED Configuration FAN LED Solid Green OK All fans are up and running Solid Red Error One or more fans is not operating properly The system should be powered down and troubleshoot the fan module Off Off The fan unit is not receiving any power Check that the fan unit is properly and completely inserted 16 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 A All fans must be operating while the power supply is plugged in A If the switch shuts down due to over temperature unplug the switch wait 5 minutes and replug in the switch For more information See Troubleshooting on page 51 2 2 3 Power Supply Status Indicators The indicator is located to the left of the connectors on the connector side panel Figure 10 Power Status LED Connector Side Ad A gt PS1 ef These are two color LED
47. fferent regions of the world is located on the Mellanox Website at http www mellanox com related docs user manuals Regulatory and Compliance Guide pdf EMC statements are also in the Regulatory and Compliance Guide Rev 1 0 A 3 China CCC Warning Statement A China CCC Warning statements EENE TEN AREA English translation disconnect all power sources before service Q For non tropical use DX RIAA TRS e Aral Gin 9 260 eret h od teer ers IC Ded c8 W ete IE Rag sr pec Wheel bibs ga Al Dan hab yungh youq gij diuzgen mbouj dwg diegndat haenx ancienz saw Dan hab yungh youq gij digih haijbaz 2000m haenx ancienz saw h HT as en yos DET y NS pe 2000 IR pe Warning for Class A E Ich AER AERP VRR ERST GE MENT T RER TETATIK English translation of above statement This is a Class A product In a domestic environment this product may cause radio interference in which case the user may be required to take adequate measures 58 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 0 Appendix B Thermal Threshold Definitions There are three thermal threshold definitions for the SwitchX switch device which impact the overall switch system operation state Warning Critical and Emergency 1 Warning 100C On managed systems only When the SwitchX device crosses the 100C threshold a Warning Threshold message will be issued by the
48. ged switches do not have a MAC on the tab 10 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 Figure 2 Pull out tab UTTUUUUUUV SX6018 164 174 184 p uu pm Win 7 NEM P N IMSX6018F 1BFS DLL 1 Rev A1 GUID 0002C9020040DDBO OG III a Made in IL Pull out tab IUUUUUUUULUV SX6015 12C 164 174 184 S N MT1117X00014 KANO Y P N MSX60015F 1BFS A I GUID 0002C9020040DDCO Rev A1 KAN VA Made in IL Pull out tab 1 3 FDR Transmission Rate Mellanox SX60XX switch systems support FDR fourteen data rate an InfiniBand data rate where each lane of a 4X port runs a bit rate of 14 0625Gb s with a 64b 66b encoding resulting in an effective bandwidth of 54 54Gb s The FDR physical layer is an IBTA specified physical layer using advanced block types deskew mechanism and framing rules for supporting higher speeds and increased reliability SX60XX switches also supports FDR10 a non standard InfiniBand data rate where each lane of a 4X port runs a bit rate of 10 3125Gb s with a 64b 66b encoding resulting in an effective band width of 40Gb s FDR10 supports 20 more bandwidth over QDR using the same QSFP cables connectors Both FDR and FDR10 support Forward Error Correction FEC as described in IEEE 802 3ap chapter 74 FDR and FDR10 are only guaranteed to work with approved Mellanox Cables FDRIO
49. hoot the fan module See Troubleshooting on page 51 14 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 As long as there is power to the switch one PS unit is connected and the switch is booted up and running the status LED should be green 4 If the STATUS LED shows red after five minutes unplug the switch and call your Mellanox representative for assistance If the switch shuts down due to over temperature unplug the switch wait 5 minutes and replug in the switch For more information See Troubleshooting on page 51 A A If the PS LEDs are not green this indicates a problem with the power supplies The switch is operational only if at least one of the PS LEDs is green Figure 7 System LEDs Ji VVVVVVVVVVY e Y 24 3A 4A 1 UID RST a 2224444441 Table 4 shows the system status LED assignment Table 4 System Status LED Assignments LED Configuration STATUS System Health LED Solid Green OK The system is up and running Flashing Green The system is booting up This assignment is valid on managed systems only Solid Yellow To be implemented in a future SW release Solid Red Major Error Possible damage can result to the switch Turn off immediately e g bad firmware can t boot overheated If the system is booting up it can take up to 5 minutes for the status led to change to green Off Of
50. i n del edificio Informaci n de conexi n a tierra La instalaci n del edificio deber prover un medio para la conexi n con la puesta a tierra de protecci n y un t cnico de servicio deber conectar permanentemente el equipo a dicho medio de conexi n Un T CNICO DE SERVICIO comprobar si la toma el ctrica de la que se suminis trar corriente al equipo provee una conexi n con la puesta a tierra de protecci n del edificio De no ser as el T CNICO DE SERVICIO se encargar de instalar un CON DUCTOR DE CONEXI N A TIERRA DE PROTECCI N del terminal de puesta a tierra de protecci n separado al conductor de tierra de protecci n del edificio El equipo se instalar en un rea donde haya conexi n equipotencial como por ejemplo un centro de telecomunicaciones o una sala de computadoras dedicada 16 C digos de instalaci n Este dispositivo se debe instalar conforme a la versi n m s reciente de los c digos A el ctricos nacionales del pa s en cuesti n En Am rica del Norte el equipo se debe instalar de acuerdo con las disposiciones vigentes del C digo El ctrico Nacional de los EE UU y del C digo El ctrico de Canad 17 Directiva sobre RAEE Conforme a la Directiva 2002 96 CE sobre RAEE todos los residuos de equipos el c tricos y electr nicos EEE se deben recolectar por separado y no se deben eliminar junto con residuos dom sticos A deshacerse de este producto y de todas sus partes h galo de una mane
51. ill turn off Remove the PS unit To insert a PS unit Make sure the mating connector of the new unit is free of any dirt and or obstacles A Do not attempt to insert a PS unit with a power cord connected to it Insert the PS unit by sliding it into the opening until a slight resistance is felt Continue pressing the PS unit until it seats completely The latch will snap into place confirm ing the proper installation Insert the power cord into the supply connector 5 Insert the other end of the power cord into an outlet of the correct voltage The green PS unit indicator should light If not repeat the whole procedure to extract A the PS unit and re insert it Extracting and Inserting the Fan Units This switch can operate indefinitely with less than a full compliment of fans in the fan modules operating properly so long as the ambient temperature is below 45 Celsius This unit is intended for installation in a Restricted Access Location A restricted A access area can be accessed only through the use of a special tool lock and key or other means of security 40 Mellanox Technologies SwitchX amp 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 Risk of Electric Shock LA With the fan module removed power pins are accessible within the module cavity DO NOT insert tools or body parts into the fan module cavity Operation without a fan unit should not exceed two minutes During fan hot swap
52. ive This button resets both the CPU of the switch device and the CPU of the management module It thereby resets all of the ports by bringing them down and powering them up when the button is pushed A quick push of this button performs this reset When the button is held down for 15 sec onds the switch is reset and the password is deleted You will then be able to enter without a pass word and make a new password for the user admin In the externally managed switches the reset button resets the CPU of the switch device Mellanox Technologies 47 Rev 1 1 Management and Tools Overview 8 Management and Tools Overview Using this switch as an InfiniBand switch you will need the MLNX OS package or Mellanox UFMO to update firmware for this switch The MLNX OS package can be downloaded from http www mellanox com gt Products gt Management Software gt MLNX OS The Mellanox UFM Management software is also an option to more closely manage your fabric switches and nodes The switch can be managed either remotely or Out of Band using MLNX OS The SX6018 switches come standard with a management software module for chassis manage ment called Mellanox Operating System MLNX OS MLNX OS is installed on all SwitchX 2 based managed switch systems MLNX OS includes a CLI WebUI SNMP chas sis management software and IB management software OpenSM The Ethernet ports for remote management connect to Ethern
53. key or This unit is intended for installation in a Restricted Access Location A restricted other means of security Caution After inserting a power cable and confirming the green system status LED light is on make sure that the Fan Status indicator shows green gt If the fan status indicator is not green then unplug the power connection and check that the fan module is inserted properly and that the mating connector of the fan unit is free of any dirt and or obstacles Caution When turning off the switch make sure ALL Connector LEDS are off to ensure a powered down status gt Do not hot swap the power supply if your switch has only one power supply You must A power down the system to replace the power supply unit when there is only one PS unit in the switch Figure 23 Two Power Inlets Electric Caution Notifications CAUTION ACHTUNG ATTENTION Risk of electric shock and energy Gafahr des elektrischen Risque de choc et de danger hazard The two PS units are Schocks Entferrnen des e lectriques Le de branch independent Netzsteckers elnes Netz ment d une seule alimentation teils spannungsfrei Um stabilise e ne de branch Disconnect all power supplies to alle Einhieten spannungs uniquement qu un module ensure a powered down state frei zu machen sind die Alimentation Stabilise e inside of the switch platform Netzstecker aller Netzteile Pour isoler completement le zu entfernen module en cause Il faut
54. nect all power supplies to ensure a powered down state inside of the switch platform 68 Mellanox Technologies SwitchX amp 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 0 8 Rack Montage und Wartung sichtsmaBnahmen zu ergreifen um die Stabilit t des Systems zu gew hrleisten Im j Wenn dieses Produkt in einem Rack montiert oder gewartet wird sind besondere Vor Allgemeinen sollten Sie das Gestell von unten nach oben mit Ger ten f llen 9 Ger teinstallation h Diese Ger t sollte nur von geschultem und qualifiziertem Personal installiert ausge tauscht oder gewartet werden 10 Ger teentsorgung h Die Entsorgung dieses Ger ts sollte unter Beachtung aller nationalen Gesetze Bestim mungen erfolgen 11 Regionale und nationale elektrische Bestimmungen h Dieses Ger t sollte unter Beachtung der regionalen und nationalen elektrischen Bestimmungen installiert werden 12 Richtigen Schutz sicherstellen YN Geeigneter elekrischer mechanischer und Feuerschutz sind vom Hersteller des Endprodukts oder dem Endbenutzer bereitzustellen 13 Strahlenkontakt Achtung Nutzung von Steuerungen oder Einstellungen oder Ausf hrung von Proze duren die hier nicht spezifiziert sind kann zu gef hrlichem Strahlenkontakt f hren Klasse 1 Laserprodukt und Referenzen zu den aktuellsten Lasterstandards ICE 60 825 1 1993 A1 1997 A2 2001 und EN 60825 1 1994 A1 1996 A2 2001 14 UL und CSA Certified Netzk
55. o host PC for ini Managed tial configuration of the switch After initial configuration the switch can be managed through the Ethernet port using a remote connection Gateway functionality is not supported in the externally managed SwitchX 9 switches IN 12 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 2 Basic Operation 2 1 Switch Platform Hardware Overview Figure 3 shows the power side panel view of the switch The figure shows the hot swap power supply locations and the hot swap fan module and an I2C banana connector Figure 3 shows the connector side panel views of the internally managed and externally managed switches The figure shows port configurations for the switch systems Managed systems have 2 Ethernet RJ 45 connectors for management 1 RJ 45 connector for connecting to a host PC 1 USB connector Externally managed switches only have a single I2C RJ 45 connector All switches have various status LEDs for on site switch status information Figure 3 QSFP Switch System Power and Connector Side Panels LU FET PFIN HUE JUN HUE ES Power Side Panel Figure 4 QSFP Switch System Connector Side lA 24A 3A 4A 54 ga JA IM 114 124 13 A 144 154 164 174 2 6 A 74 Bh Es SX6018 Connector Side Panel Meer mer FA GASSEN SX6015 Connector Side Panel All connectivity except for power cords is via the conne
56. o the switch check the mgmt0 interface configuration Specifically verify the existence of an IP address To check the cur rent mgmt0 configuration enter the following commands 36 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 Sx 43 standalone master enable Sx 43 standalone master configure terminal Sx 43 standalone master config show interfaces mgmt0 The following is an example of the output Interface mgmt0 state Admin up yes Link up yes IP address 192 168 10 43 Netmask PND 0919572909 c0 Speed 1000Mb s auto Duplex full auto Interface type ethernet Interface source physical MTU 1500 HW address 00 02 C9 11 2A AE Comment RX bytes 1343502058 TX bytes 313920869 RX packets 17589211 TX packets 992 T RX mcast packets 0 TX discards 0 RX discards 0 TX errors 0 RX errors 0 TX overruns 0 RX overruns 0 TX carrier 0 RX frame 0 IX colliisionss M TX queue len 1000 For managed switches confirm that the latest SW revision is installed on this device A Use the CLI to get the currently installed revision and then go to Mellanox com to check the revision of the latest version 6 Run the command show version 7 Compare the results of this command with the latest version for your switch posted on the support page for your switch This page will need a support login and password 3 9 Rerunning the Wizard If
57. onfiguration Table 15 Port Splitting Options Port csi Els 1 Y 2 3 Y Y 3 Y 6 m 7 Y 8 Y o 9 10 S _ 11 Y 12 S gt 13 Y 2 14 Y E 15 16 Y _ 17 va u 18 A Y 44 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 5 2 Cable Installation P 56Gb s and 40Gb s are only guaranteed to work with approved Mellanox Cables All cables can be inserted or removed with the unit powered on To insert a cable press the con nector into the port receptacle until the connector is firmly seated The LED indicator corre sponding to each data port will light when the physical connection is established that is when the unit is powered on and a cable is plugged into the port with the other end of the connector plugged into a functioning port After plugging in a cable lock the connector using the latching mechanism particular to the cable vendor When a logical connection is made the LED will change to green When data is being transferred the light will blink green To remove disengage the locks and slowly pull the connector away from the port receptacle The LED indicator for that port will turn off when the cable is unseated Care should be taken not to impede the air flow through the ventilation holes next to the connec tor ports Cable lengths should be used which allow f
58. or routing horizontally around to the side of the chassis before bending upward or downward in the rack Mellanox Technologies 45 Rev 1 1 Switch Shut Down Procedure 6 Switch Shut Down Procedure To shut down the switch run the command Reload halt noconfirm The switch cannot be restarted remotely A To restart the switch you must physically go to the switch and unplug and plug in the power cord 46 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 7 Disassembly of the Switch from the Rack gt To disassemble the switch from the rack 1 Unplug and remove all connectors D Unplug all power cords e Remove the ground wire B Unscrew the 2 center bolts from the side of the switch with the bracket A Support the weight of the switch when you remove the screws so that the switch does not fall 5 Slide the switch from the rack 6 Remove the rail slides from the rack 7 Remove the ten caged nuts 7 1 Disposal According to the WEEE Directive 2002 96 EC all waste electrical and electronic equipment EEE should be collected separately and not disposed of with regular household waste Dispose of this product and all of its parts in a responsible and environmentally friendly way EE Follow the instructions found in the Mellanox Web site at mellanox com for proper instructions to disassemble and dispose of the Switch according to the WEEE direct
59. posted on the www mellanox com gt Support gt Firmware Downloads gt SwitchX SwitchX 2 Switch System page 3 If the current version is not the latest version follow the directions in the MFT User manual to burn the new firmware inband 4 I2C Connector The I2C connection provides access to Flash and EEPROMs This connection is for FAE use only LA The externally managed switches are Plug and Play and all firmware updates should be done in band LA This interface is for Debug and Troubleshooting only This interface is for FAEs and advanced users only 50 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 9 Troubleshooting As soon as a switch is plugged in make sure that the green power LEDs on the PS units are on Issue 1 Status LED and or Status Health LED If either of these two LEDs is red unplug the switch and call your Mellanox representative Issue 2 Power supply unit If the LED on the PS unit is not lit or is red check that the power cable is plugged into a work ing outlet 1 Check that the power cable has a voltage within the range of 100 240 volts AC 2 Check that the air flow direction of the PS units are consistent with the Fan module air flow 3 Remove and reinstall the power cable 4 Remove and reinstall the PS unit Issue3 The power LED for the switch shuts off 1 Check that the there is adequate ventilation Are the fan LEDs showing that
60. ra respon re i sable y respetuosa con el medio ambiente Mellanox Technologies 73
61. rfaces to connect to the SX6018 2X 100M IGb Ethernet connectors labeled MGT 1 USB port that is labeled S This interface can be used to update software or firmware connector that is labeled CONSOLE Use this connector to connect to the host PC 1 reset button E e I2C banana connector on the power supply side for FAE use only s Figure 18 Interface for Internally Managed Switches WA SX6018 CONSOLE There is one I2C RJ 45 interface to connect to the SwitchX 2 SX6015 Figure 19 Interface for Externally Managed Switches NECTE MY YN 2 5 2 1 RJ 45 Connector CONSOLE Internally Managed Switches only The port labeled CONSOLE is for a local host connection to the management module This is used the first time the switch is connected A harness is included in the package to connect to a DB9 connection on a host PC Connecting to a local host PC and following the instructions in the Installation Guide Configuring the Switch for the First Time must be done before any remote Mellanox Technologies 23 Rev 1 1 Basic Operation management is available For the Socket pinout see RJ 45 CONSOLE and I2C Interface on page 62 2 5 2 2 RJ 45 Ethernet Connector MGT Internally Managed Switches Only The 2 Ethernet connections labeled MGT provide access for remote management The switch can be connected to any Ethernet port 4 Each Ethernet connector get
62. rom the Mellanox website www mellanox com gt Products gt Software gt Firmware tools Select and download the release that matches your system Follow the instructions in the User Manual http www mellanox com pdf MFT MFT user manual pdf to get the tools All firmware updates should be done in band Go to the Mellanox Website and confirm that the firmware is the latest If not reburn the latest firmware from the download site New firmware Mellanox Technologies 49 Rev 1 1 Management and Tools Overview versions will be posted on the Mellanox firmware download page http www mellanox com gt Support gt Download Firmware Access to this page requires a login and password You will need the Mellanox Firmware Tools package to update firmware for this switch It can also be downloaded from http www mellanox com gt Support gt InfiniBand Software and Drivers You will also need to download and unzip the firmware binary image This is provided in the Mellanox Web site at http www mellanox com gt Support gt Download Firmware and go to the SwitchX 2 Switch systems Click in the Table for the firmware image that you need 8 4 4 Current Firmware Revision For Externally Managed Switches In order to obtain the Firmware version of the externally managed switch Run the following command from a host flint d lid number qq 2 Compare the results of this command with the latest version for your switch
63. s The SX6018 with Virtual Protocol Interconnect VPI supporting InfiniBand and Ethernet con nectivity provide the highest performing and most flexible interconnect solution for PCI Express Gen3 servers VPI simplifies system development by serving multiple fabrics with one hardware design VPI simplifies today network by enabling one platform to run both InfiniBand and Ether net subnets on the same chassis The SX6000 switch family enables efficient computing with features such as static routing adap tive routing and advanced congestion management These features ensure the maximum effec tive fabric bandwidth by eliminating congestion hot spots Figure 1 Connector Side View of the Switch 1 1 Features 18 FDR 56Gb s ports in 1U switch IBTA 1 3 and 1 21 compliant SDR DDR QDR FDR10 FDR link speed Congestion control Adaptive routing e Forward Error Correction FEC Port mirroring e IPoIB Bridging 19 rack mountable chassis 1U with redundant power supplies and Fan units This feature will be available to customers in the near future 1 2 Serial Number and Product Version Information The serial number and GUID for the switches are found on the pull out tab below the USB con nector On managed switches only the MAC for the Management PC is also on this tab This MAC is for the mgmt0 management interface the top Ethernet port The second interface mgmtl has the next consecutive MAC Externally mana
64. s PS2 EH UID RST The switch is available with one factory installed Power Supply Unit A second Power Supply Unit can be added to support hot swap ability and to add redundancy See Accessory and Replacement Parts on page 63 for ordering part numbers Figure 11 Power Side Panel PS unit 1 status LED LI EET 7 Wm HUE Jun Je M Fan Unit Primary Power Supply Unit The primary power supply unit PS1 is located on the right side of the power side panel with PS2 on the left side Each PS unit has a single 2 color LED on the right side of the PS unit that indicates the internal status of the unit Mellanox Technologies 17 Rev 1 1 Basic Operation Figure 12 PS Unit Status LEDs This is a two color LED gt EI Power supply fan air flow p ma direction indicator Table 6 shows the power supply unit status LED assignment Table 6 PS Unit Status LED Assignments LED Color Status Solid Green OK The Power supply is delivering the correct voltage 12VDC Solid Red Error The PS unit is not operational Off Off There is no power to the system neither PS unit is receiving power If one PS unit is showing green and the second PS unit is unplugged it will show a red indication 2 2 3 1 Bad Port LED Figure 13 Bad Port LED This is a one color LED The Bad Port indicator is located on the left side of the connector side panel of th
65. s completely The green fan status indicator should light If not extract the fan unit and reinsert it A After two unsuccessful attempts to install the fan unit power off the switch before attempting any system debug 42 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 5 1 SX6018 Splitting the Data Stream Using Breakout Fanout Cables When the SX6018 switch is configured to run Ethernet protocol specific ports can be split by using a QSFP 1X4 breakout cable to split one 40 GbE port into 4 lanes 4 SFP connectors 10GbE each These 4 lanes then go one lane to each of the 4 SFP connectors Some ports can be split into 2 10 Gb s ports using lanes 1 and 2 only When a QSFP port is split into 2 10Gb s ports then only SFP connectors 1 and 2 are used Connectors 3 and 4 are left unconnected Using Figure 31 and Table 15 with breakout cables you can achieve 56 ports of 10Gb s Figure 30 Breakout or Fanout Cable Figure 31 SX6018 Port Splitting Options WWE This port can be split into 4 10Gb s SFP mn This port can be split into 2 10Gb s SFP The maximum 10 Gb s Ethernet ports configurable with this switch is 56 Mellanox Technologies 43 Rev 1 1 Cabling When using this feature you MUST go into the MLNX OS chassis management system and reconfigure the individual ports to split 4 See the MLNX OS Software Command Reference Guide for instructions on port c
66. s connected to Ethernet switches These switches must be 7 configured to 100M 1G auto negotiation A Use only FCC compliant Ethernet cables The externally managed switches are Plug and Play and all firmware updates should be done in band The I2C connection should only be used if the firmware image was cor rupted to the point that the regular firmware tools cannot successfully reburn the cor rect image gt 2 5 2 3 USB Connector Internally Managed Switches Only There is a single USB connector This connector can be used to install software and or firmware upgrades using a memory device that has a USB connector This connector is USB 2 0 compli ant The USB connector is not found on externally managed switches ae 4 USB 1 0 is not supported 4 Do NOT use excessive force when inserting or extracting the USB from the connector ae 2 5 2 4 12C Connector On Internally Managed Switches There is an I2C connector on the far left of the power side of the switch This interface is for Debug and Troubleshooting only This connector can be used to install firmware upgrades should the firmware image be damaged and cannot be upgraded through a host PC or remotely This interface is for support personnel and advanced users only 24 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 2 5 2 5 12C Connector On Externally Managed Switches Only The I2C RJ 45 connection provid
67. ses Step 4 Enable IPv6 autoconfig SLAAC on This turns on autoconfiguration of the IPv6 addresses This is mgmt0 interface no no unsuitable for DHCPv6 Step 5 Admin password Press Enter to leave To avoid illegal access to the machine please type a password unchanged new password and then press Enter Then confirm the password by re enter Step 6 Confirm admin password ing it new password Note that password characters are not printed You have entered the following information The wizard displays a summary of your choices and then asks you to confirm the choices or to re edit them 1 Hostname switch name 2 Use DHCP on mgmt interface yes 7 Enable IPv6 yes Either press Enter to save changes and exit or enter the config 4 Enable IPv6 autoconfig SLAAC on mgmtO uration step number that you wish to return to interface no 5 Admin password Enter to leave unchanged CHANGED Note To run the command configuration jump start you must be in To change an answer enter the step number to Config mode return to Otherwise hit lt enter gt to save changes and exit Choice lt Enter gt Configuration changes saved To return to the wizard from the CLI enter the configuration jump start command from con figuration mode Launching CLI switch name 34 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User
68. tions A Read all installation instructions before connecting the equipment to the power source 2 Over temperature This equipment should not be operated in an area with an ambient temperature exceed A ing the maximum recommended 45 C 113 F Moreover to guarantee proper air flow allow at least 8cm 3 inches of clearance around the ventilation openings 3 Restricted Access Location This unit is intended for installation in a Restricted Access Location A restricted LA access area can be accessed only through the use of a special tool lock and key or other means of security 4 Stacking the Chassis YN The chassis should not be stacked on any other equipment If the chassis falls it can cause bodily injury and equipment damage 5 Redundant Power Supply Connection Electrical Hazard LA This product includes a blank cover over the space for the redundant power supply Do not operate the product if the blank cover is not securely fastened or if it is removed Mellanox Technologies 27 Rev 1 1 Installation 6 During Lightning Electrical Hazard LA During periods of lightning activity do not work on the equipment or connect or dis connect cables 7 Copper Cable Connecting Disconnecting Copper cables are heavy and not flexible as such they should be carefully attached to A or detached from the connectors Refer to the cable manufacturer for special warnings and instructions 8 Rack Mounting and Servicing
69. ur local or national electrical codes require an external ground to all IT components If so connect a ground wire to one of the casing screws and connect the other end to a valid ground If you choose to not use the ground screw make sure that the rack is properly grounded and that there is a valid ground connection between the chassis of the switch and the rack Test the ground using an Ohm meter Some national and or local codes may require IT components to be bonded and exter nally grounded not including the power cord ground You must follow all national pe and local codes when installing this equipment 30 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 1 3 4 Power Connections and Initial Power On The switch platform ships with one or two power supply units For switches with only one unit installed a second PS unit may be installed at a later time Each supply has a separate AC recep tacle The input voltage is auto adjusting for 100 240VAC 50 60Hz power connections The power cords should be standard 3 wire AC power cords including a safety ground and rated for 15A or higher Caution The switch platform will automatically power on when AC power is applied A There is no power switch Check all boards power supplies and fan tray modules for proper insertion before plugging in a power cable access area can be accessed only through the use of a special tool lock and
70. ure Cable routing in particular should not impede the air exhaust from the chassis 3 2 Package Contents Before you install your new switch unpack the system and check to make sure that all the parts have been sent check this against the parts list in the installation kit The installation kit is shipped separately Check the parts for visible damage that may have occurred during shipping The switch comes packed with the following items Mellanox Technologies 29 Rev 1 1 Installation e 1 X switch 1 X wire support for power cable one for each power cable e X Harness HARO000028 Harness RS232 2M cable DB9 to RJ 45 for SX6018 switches e X Quick Start Guide e 1 X China RoHS statement If anything is damaged or missing contact your customer representative immediately A For customer support go to www mellanox com gt Support gt Customer Support Portal Login 1 Follow the installation instructions that are in the installation kit 2 Ground the switch 3 Install the power plug wire support into each PSU Squeeze the wire support and insert the ends into the holes above and below the socket Figure 22 Installing the Power Cord Wire Support 4 Plug in the power cables 5 Check the Status LEDs and confirm that all of the LEDs show status lights consistent with normal operation 6 You can start connecting all of the cables to the switch 3 3 Grounding the Switch Check to determine if yo
71. version des codes lectriques nationaux du LA pays Pour l Am rique du Nord l quipement doit tre install conform ment aux sp cifications du Code Electrique National Am ricain et du Code Electrique Canadien 17 Interconnexion des unites Les c bles de branchement l unit RS232 et les interfaces Ethernet doivent tre certi YN fi s UL de type DP 1 ou DP 2 Note lorsqu il existe dans un circuit non LPS Protection contre la surintensit Un appareil de protection r pertori facilement accessible contre la surintensit du circuit de branchement et calibr 20A doit tre incorpor dans le c blage lectrique du b timent 18 Exposition au rayonnement grave Mise en garde l utilisation de commandes ou de r glages ou l ex cution de proc dures autres que ce qui est sp cifi dans les pr sentes peut engendrer une exposition au rayonnement grave PRODUIT LASER DE CLASSE 1 et r f rences aux normes laser les plus r centes CEI 60 825 1 1993 A1 1997 A2 2001 et NE 60825 1 1994 A1 1996 A2 2001 19 S assurer que les enceintes sont appropri es 4 Des enceintes lectriques m caniques et incendie adapt es doivent tre fournies par le fabricant du produit final ou par l utilisateur final 20 Cordons lectriques CA homologu s UL Pour les prises lectriques en Am rique du Nord choisissez un cordon lectrique A homologu UL et certifi CSA 3 conducteurs 18 AWG termin par
72. ware LA BEWARE This icon indicates a situation that can potentially cause personal injury or damage to hardware or software Abbreviations FDR Fourteen Data Rate Used to indicate a 4X port running at 56Gb s Mellanox Part Numbering Legend Place Field Decoder M Mellanox Technologies SX System Type SwitchX switch Family P Data Transfer Protocol 6 InfiniBand R Size of box 0 1U 1 1 5U 2 2U FF Management Capabilities 15 18 Ports externally managed 18 18 Ports internally managed 25 36 Ports externally managed 36 36 Ports internally managed C Data Rate F FDR T FDR10 Q QDR DDR Separator P Power Supplies 020 1 1 2 2 M Depth of the Unit S standard depth B short depth Y Air Flow direction Rz Connector side to Power side airflow F Power Side side to Connector side airflow R Chip Generation R SwitchX S SwitchX 2 Mellanox Technologies 9 Rev 1 1 Overview 1 Overview The SX6015 and SX6018 switch systems provides the highest performing fabric solution in a 1RU form factor by delivering 2Tb s of non blocking bandwidth with 170ns port to port latency Built with Mellanox s 6th generation SwitchX 2 InfiniBand switch device these switches pro vides up to eighteen 56Gb s full bi directional bandwidth per port These stand alone switches are an ideal choice for top of rack leaf connectivity or for building small to extremely large sized cluster
73. y 56 Mellanox Technologies SwitchX 2 18 Port QSFP FDR InfiniBand System Hardware User Manual Rev 1 0 A 1 A 2 Mellanox Technologies 57 Protocol Support Speed protocol IB Management QoS Auto Negotiation of 56Gb s Managed using MLNX OS Management 40Gb s 10Gb s 1Gb s Data Rate Data Rate 56Gb s Regulatory Compliance Safety US Canada cTU Vus EMC USA FCC Class A EU IEC60950 Emissions Canada ICES Class A International CB EU EN55022 Class A Safety Russia GOST R EU EN55024 Class A EMC Argentina S mark DoC EU EN61000 3 2 Class A Emissions EU EN61000 3 3 Class A Japan VCCI Class A Australia New Zealand AS NZS CISPR 22 class A Environmental Acoustic EU IEC 60068 2 64 Random ISO 7779 Environmental Vibration 71dB A Acoustic EU IEC 60068 2 29 Shocks ETS 300 753 Type I II 7 1 Bel EU IEC 60068 2 32 Fall Test Scalability and Performance Switching Switching Switching Performance Performance Simultaneous wire speed any port Capacity 2 02Tb s Capacity to any port Reliability Availability and Serviceability Features Hot swapability Redundancy Hot Swappable 1 1 Redundant Fan Module Power Supplies Approved Cables For a list of all approved cables see http www mellanox com related docs user manuals Mellanox approved cables pdf EMC Certifications The list of approved certifications per switch in di

SX6015 User Manual - Mellanox Technologies

Contents

Download Pdf Manuals

Related Search

Related Contents