InfiniScale® IV 8-Port QSFP 40 Gb/s InfiniBand Switch User Manual
Contents
1. Date Revision Description June 2012 Rev 1 4 Fixed Formatting and Typos Oct 2011 Rev 1 3 Removed reference to Installation kit included in package Mar 2011 Rev 1 2 Added note and figure to blank panel installtion step Made changes to the installation process for the two switch frame Made changes to FW burning sections Feb 2011 Rev 1 1 Removed UID LED support Jan 2011 Rev 1 0 Initial Release Mellanox Technologies 7 Rev 1 4 About this Manual This manual describes the installation and basic use of the Mellanox 185022 switch which is based on the InfiniScale IV switch device Intended Audience This manual is intended for users and system administrators responsible for installing and setting up the switch platforms listed above The manual assumes familiarity with the InfiniBand Architecture Specification Related Documentation Additional Documentation available from Mellanox Table 2 Reference Documents Switch Firmware and See Firmware Update http www mellanox com gt Support gt Download Firmware Tools Tools Conventions Throughout this manual the name 185022 and the term switch are used to describe the 8 port QSFP 40Gb s InfiniBand switch unless explicitly indicated otherwise The following icons are used throughout this document to indicate information that is important to the user This symbol signals recommendations to the user This symbol ind2. 9 RSI No connection required 10 VeeR Receiver Ground Common with Transmitter Ground 2 11 VeeR Receiver Ground Common with Transmitter Ground 12 RD Receiver Inverted DATA out AC Coupled 13 RD Receiver Non inverted DATA out AC Coupled 40 Mellanox Technologies InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 Table 10 SFP Pinout Pin Symbol Name Description 14 VeeR Receiver Ground Common with Transmitter Ground 2 15 VccR Receiver Power Supply 16 VccT Transmitter Power Supply 17 VeeT Transmitter Ground Common with Receiver Ground 18 TD Transmitter Non Inverted DATA in AC Coupled 19 TD Transmitter Inverted DATA in AC Coupled 20 VeeT Transmitter Ground Common with Receiver Ground a Circuit ground is internally isolated from chassis ground b Traurr is an open collector drain output which should be pulled up with a 4 7k 10k Ohms resistor on the host board if intended for use Pull up voltage should be between 2 0V to Vcc 0 3V A high output indicates a transmitter fault caused by either the TX bias cur rent or the TX output power exceeding the preset alarm thresholds A low output indicates normal operation In the low state the output is pulled to lt 0 8V Laser output disabled on TDIS gt 2 0V or open enabled on TDIS lt 0 8V d Should be pulled up with 4 7kQ 10kO on host board to a voltage between 2 0V and 3 6V MOD
3. 18 La Directive WEEE Conform ment la Directive WEEE 2002 96 EC les d chets d quipements lec triques et lectroniques EEE doivent tre tri s et ne peuvent tre limin s avec les d chets m nagers Veuillez disposer de ce produit et de tous ses composants de mani re responsable et respectueuse de l environnement Mellanox Technologies 47 Rev 1 4 Anhang F Installation Sicherheitshinweise German 1 Installationsanleitungen Lesen Sie alle Installationsanleitungen bevor Sie das Ger t an die Stromversorgung anschlie en 2 bertemperatur Dieses Ger t sollte nicht in einem Bereich mit einer Umgebungstemperatur ber der maximal empfohlenen Temperatur von 45 C 113 F betrieben werden Es ist ein Luft LA strom von 200 LFM bei maximaler Umgebungstemperatur erforderlich Au erdem sollten mindestens 8 cm 3 in Freiraum um die Bel ftungs ffnungen sein um einen einwandfreien Luftstrom zu gew hrleisten 3 Stapeln des Chassis Das Chassis sollte nicht auf andere Ger te gestapelt werden Wenn das Chassis herun terf llt kann es zu Verletzungen und Besch digungen an Ger ten f hren 4 Redundanter Stromversorgungsanschluss Elektrische Gefahr Stromversorgung Betreiben Sie das Produkt nicht wenn diese Abdeckung nicht sicher Dieses Produkt verf gt ber eine Abdeckung ber dem Bereich f r die redundante A festsitzt oder entfernt wurde 5 Bei Gewitter Elektrische Gefahr Arb
4. 70 121 lbs gt 121 lbs lt 18 kgs 18 32 kgs 82 55 kgs gt 55 kgs 3 Temp rature excessive Ce mat riel ne doit pas fonctionner dans une zone avec une temp rature ambiante d passant le maximum recommand de 45 C 113 F Un flux d air de 200LFM LA cette temp rature ambiante maximale est n cessaire En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouver tures de ventilation 4 Empilage du ch ssis Le ch ssis ne doit pas tre empil sur un autre mat riel Si le ch ssis tombe il peut pro voquer des blessures corporelles et des d gradations de biens 5 Syst me de fusible neutre double p le Avertissement Syst me de fusible neutre double p le Veuillez debrancher tous les cordons d alimentation avant d ouvrir le boitier de ce produit ou de toucher un de ses composants internes 6 Orages dangers lectriques Pendant un orage il ne faut pas utiliser le mat riel et il ne faut pas brancher ou d brancher les c bles Mellanox Technologies 45 Rev 1 4 7 Branchement d branchement des c bles InfiniBand en cuivre Les c bles InfiniBand en cuivre sont lourds et ne sont pas flexibles 11 faut donc faire tres attention en les branchant et en les d branchant des connecteurs Consultez le fab ricant des c bles pour conna tre les mises en garde et les instructions sp ciales gt 8 Montage et entretien sur baie Lo
5. ABS pulls line low to indicate module is plugged in e LOS is open collector output Should be pulled up with 4 7kQ 10kQ on host board to a voltage between 2 0V and 3 6V Logic 0 indicates normal operation logic 1 indicates loss of signal e Mellanox Technologies 41 Rev 1 4 Appendix C QSFP Interface 20 GND GND 12 __ 21 ron Rxin 18 Table 11 InfiniBand QSFP Connector Pinout 22 Rx2p Rxtp 17 23 GND ann 15 __ a Ceco hat 24 15 Pin Pin Name Signal Description Rx4n Rx3n Number map Pep E S 1 GND Ground RI END CIND ae 2 Tx2n Transmitter Inverted Data Input tie u SDA mer 3 Tx2p Transmitter Non Inverted Data Input 29 nt ue 10 4 GND Ground 30 ne A 9 5 Tx4n Transmitter Inverted Data Input 6 Tx4p Transmitter Non Inverted Data Input LPMode ModSelL gt 7 GND Ground DN _ 6 8 ModSelL Module Select 34 Tx3n Tx4n 5 9 ResetL Module Reset 35 GND GND 4 10 Vcc Rx 3 3 V Power supply receiver Txip Tx2p 11 SCL 2 wire serial interface clock 38 8 a an 1 12 SDA 2 wire serial interface data 13 GND Ground 14 Rx3p Receiver Non Inverted Data Output 15 Rx3n Receiver Inverted Data Output 16 GND Ground 17 Rxlp Receiver Non Inverted Data Output 18 RxIn Receiver Inverted Data Output 19 GND Ground 20 GND Ground 21 Rx2n Receiver Inverted Data Output 3 22 Rx2p
6. Receiver Non Inverted Data Output 3 23 GND Ground 24 Rx4n Receiver Inverted Data Output 3 25 Rx4p Receiver Non Inverted Data Output 3 26 GND Ground 27 ModPrsL Module Present 28 IntL Interrupt 29 Vcc Tx 3 3 V Power supply transmitter 30 Vec 1 3 3 V Power Supply 31 LPMode Low Power Mode 32 GND Ground 33 Tx3p Transmitter Non Inverted Data Input 34 Tx3n Transmitter Inverted Data Input 35 GND Ground 36 Txlp Transmitter Non Inverted Data Input 37 Txln Transmitter Inverted Data Input 38 GND Ground 42 Mellanox Technologies J Rev 1 4 InfiniScale IV 8 Port OSFP 40 Gb s InfiniBand Switch User Manual ions QSFP Male and Female Connect Figure 17 Top 18 35 View into Rear of Connector GND a8 AXin a7 E TX1p l36 GND 35 TX3n 34 TX3p 33 GND 82 LPMode 31 Vcc1 80 VecTx l29 IntL og ModPrsL 27 GND 196 RX4p 25 RX4n 24 GND 23 RX2p 22 RX2n 21 x GND po Si N L GND TX2n TX2p GND TX4n TX4p ModSelL ResetL VccRx SCL ZL SDA z e v S 9 Z GND 8 6 0 L LL Er eL GND v RX3p ai RX3n 91 GND L RX1p 8k RX1n 6L GND View into Front of Cage M dixL GND ISIdDON au X199A LIDA 9pOJAld I AND dex UEX_L AND 43 Mellanox Technologies Rev 1 4 Appendix D Replacement Parts Ordering Numbers Table 12 Replacement Parts O
7. cables One cable is an I2C cable with DB9 female connectors on both ends One cable is a I2C cable with DB9 female connector to RJ45 male Use the cable that matches your switch The other is a USB cable with one type A host connection to be connected to the server or PC and a type B connection to be connected to the MTUSB adapter Mellanox Technologies 33 Rev 1 4 Installation Figure 16 I2C Cable Connected to IS5022 155022 With the switch connected to a computer through the MTUSB it is now possible to reprogram the SPI Flash memory 5 5 1 3 Instructions for Reprogramming Through the I2C Port 1 Make sure that the MTUSB I2C cable is connected to the switch and USB cable is connected to a com puter MFT is running on the computer mtusb 1 mst device appears in mst status 2 _ Set the switch I2C switch to allow access by running gt mlxi2c d dev mst mtusb 1 s IS5023 p IS4 3 _ Run the burn command flint d dev mst mtusb 1 i fw 1S4 rel 7_4_0 Q_Al bin qq b 4 _ Power cycle the switch by unplugging and re plugging the power cord to load the new firm ware 5 5 2 Open SM To manage the Mellanox switch system using OFED download Mellanox OpenFabrics from http www mellanox com gt Support gt InfiniBand Software and Drivers Be sure to read and follow all of the instructions regarding the installation and use of these tools 34 Mellanox Technologies InfiniScale amp IV 8 Port QSFP 4
8. can also be used with smaller departmental or back end clustering uses with high performance needs such as storage data base and GPGPU clusters Networks built with 185022 systems can carry converged traffic with the combination of assured bandwidth and granular quality of service 185022 systems provide up to 40Gb s full bidirectional bandwidth per port The switch platform comes pre installed with all necessary firmware and is configured for stan dard operation within an InfiniBand fabric This switch requires an InfiniBand compliant Subnet Manager running from one of the hosts or another managed switch or Fabric Management soft ware running on the switch All that is required for normal operation is to follow the usual precau tions for installation and to connect the switch to the HCAs or other switches within the fabric Once connected the Subnet Management software automatically configures and begins utilizing the switch It is recommended that the Mellanox OpenFabrics software package be installed on all nodes con nected to the 185022 The software package provides a subnet manager and network management tools as well as connectivity software for servers and storage and is available on the Mellanox web site See Chapter 3 for more information Basic installation and hardware maintenance is covered in Hardware Basic Operation and Instal lation on page 16 1 1 Features On board temperature monitoring Reset Push button on f
9. connected to a verified ground 2 _ Screw the two switch mounted rails to the switch using six flat head screws per rail Figure 11 Screw on the Switch Mounted Rails Mellanox Technologies InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 3 Place eight caged nuts into the rack at the level you want to mount the switch Four go on the Leaf side and four go on the rear side Figure 12 Caged Nut Placement Leaf side Rear side 4 Install the two switch frame into the rack This frame goes on the leaf side of the rack Do not tighten the screws into the caged nuts 5 Slide the rail slides into the frame and screw the bolts into the rack caged nuts on the rear side of the rack Do not tighten the screws into the caged nuts Slide the switch into the frame and screw in the capture bolts into the frame Tighten the capture nuts 8 Install the blank panel into the frame opposite the switch or install a second switch The front panel is not symetrical The large radius always goes ot the center of the frame Figure 13 Front Panel Orientation arge radius Small radius Small radius Mellanox Technologies 27 Rev 1 4 Installation 9 Tighten the capture nuts 10 Tighten all eight screws for the caged nuts to 9 2 Nm or 81 5 pound inches 11 Ground the switch 12 Plug in the power cord 13 You can start connecting all of the cables to the switch 5 2 3 1 Insta
10. de descarga el ctrica No utilizar el equipo ni conectar o desconectar cables durante per odos de actividad de rayos 8 Conexi n y desconexi n del cable Copper InfiniBand f Dado que los cables de cobre InfiniBand son pesados y no son flexibles su conexi n a B los conectores y su desconexi n se deben efectuar con mucho cuidado Para ver adver tencias o instrucciones especiales consultar al fabricante del cable 9 Montaje y mantenimiento de bastidores N Al instalar o realizar el mantenimiento de este aparato en un bastidor es preciso adop 4 tar precauciones especiales para garantizar que el sistema se mantenga estable En gen eral en un bastidor los equipos se deben instalar comenzando desde abajo hacia arriba 10 Instalaci n de equipos La instalaci n el reemplazo y el mantenimiento de este equipo estar n a cargo nica mente de personal capacitado y competente 11 Asegurar confinamientos adecuados El fabricante del producto final o el usuario final deber n suministrar LA un confinamiento adecuado para componentes el ctricos y mec nicos y contra incen dio 12 Eliminaci n de equipos La eliminaci n definitiva de este equipo se debe efectuar conforme a todas las leyes y reglamentaciones nacionales 13 C digos el ctricos locales y nacionales LA Este equipo se debe instalar conforme a los c digos el ctricos locales y nacio nales Mellanox Technologies 53 Rev 1 4 14 C
11. not fully grounded Finland Laite on liitett v suojamaadoituskoskettimilla varustettuun pistorasiaan gt Norway Apparatet m tilkoples jordet stikkontakt Unit is intended for connection to IT power systems for Norway only Sweden Apparaten skall anslutas till jordat uttag gt Mellanox Technologies 55
12. or serviced only by trained and qual ified personnel 11 Equipment Disposal 4 Disposal of this equipment should be in accordance to all national laws and regula tions 12 Local and National Electrical Codes This equipment should be installed in compliance with local and national electrical LA codes 13 Installation Codes This device must be installed according to the latest version of the country national electrical codes For North America equipment must be installed in LA accordance to the applicable requirements in the US National Electrical Code and the Canadian Electrical Code Mellanox Technologies 13 Rev 1 4 Installation Safety Warnings 14 Interconnection Of Units Cables for connecting to the unit RS232 and Ethernet Interfaces must be UL L certified type DP 1 or DP 2 Note when residing in non LPS circuit Overcurrent Protection A readily accessible Listed branch circuit overcurrent protective device rated 20 A must be incorporated in the building wiring 15 Suitable Enclosure Suitable electrical mechanical and fire enclosure shall be provided by the L end product manufacturer and or the end user 16 UL Listed and CSA Certified Power Supply Cord For North American power connection select a power supply cord that is UL Listed and CSA Certified 3 conductor 16 AWG terminated with a molded plug rated at LA 125 V 13 A with a minimum length of 1 5m six feet but no longer than 4 5m Fo
13. the light will blink green A Cables in the bottom row should be inserted up side down in relation to the how the cables are inserted in the top row 2 Figure 14 Top and Bottom Ports To remove disengage the locks and slowly pull the connector away from the port receptacle The LED indicator will turn off when the cable is unseated Mellanox Technologies 29 Rev 1 4 Installation Care should be taken not to impede the air exhaust flow through the ventilation holes next to the data connection ports Cable lengths should be used which allow for routing horizontally around to the side of the chassis before bending upward or downward in the rack When inserting cables into the switch always insert the center row cable before the top and bottom row cables 5 2 7 Supported Approved Cables For a list of all approved cables see www mellanox com related docs user_manuals Mellanox_approved_cables pdf 5 3 Disassembly of the Switch from the Rack 5 3 1 Disassembly Procedure Single Switch Centered Installation To disassemble the switch from the rack 1 Unplug and remove all connectors 2 _ Unplug the power cord 3 Remove the ground wire 4 Remove the four M6 screws holding the switch to the rack Support the switch while remov ing the screws or the switch may fall out of the rack Loosen the two thumbscrews holding the switch in place Slide the switch out of the rack au
14. 0 Gb s InfiniBand Switch User Manual Rev 1 4 6 Troubleshooting As soon as a switch is plugged in continuously monitor the status light Make sure that it turns green within 30 seconds of booting up Status LED and Fan LED Ifthe fan LED is red make sure that the ambient temperature is within the range specified in the specification section of the appendix If any ofthe LEDs are red after 30 seconds unplug the switch and call your Mellanox representa tive Power supply unit 1 Check that the power cable has a voltage within the range of 100 240 volts AC 2 Remove and reinstall the power cable The green power LED for the fans does not come on Caution Do not run the switch if the System Status LED for the Fans is red or off The connector LED for the port connector does not come on 1 Check that both ends of the cable are connected 2 Check that the locks on the ends are secured 3 Make sure that the latest firmware version is installed on both the cards and the switch 4 _ If media adapters are used check that the all connections are good tight and secure The connector LED is flashing orange 1 Check that the Subnet Manager has been started 2 _ Check the cable is connected properly at both ends 3 Check that the cable is good 4 Check that the connections are good no cable weight on the connector no twists in the cable at the connector etc 5 Check the port connectors at both ends The swi
15. 0002C 903 GUID 0027051 TELLE LL LULU RTE If the status LED shows red after 30 seconds unplug the switch and call your Mella nox representative for assistance If the fan LED shows red unplug the switch and call your Mellanox representative for assistance If the switch shuts down due to over temperature unplug the switch wait 5 minutes and replug in the switch For more information See Troubleshooting on page 35 AN 4 1 1 2 Status LED The status indicator is located on the left side of the front panel connector side of the unit The following status conditions are possible Table 3 Switch Status LED Configurations LED Configuration LED Description Off No power to the switch Solid Green OK the switch is running All OK Blinking Green Switch is booting up Mellanox Technologies 17 Rev 1 4 Hardware Basic Operation and Installation LED Configuration LED Description Red Error there is a problem with power output from the power supply or there is a thermal shut down 4 1 1 3 Fan Indicators A The fan indicator is located on the left side of the front panel connector side of the unit The fol lowing fan status conditions are possible Table 4 Fan LED Configurations LED Configuration FAN LED Off There is no power to the fans Green OK the fans are running Red Error the fans are not operating properly Replace the sw
16. 4 This switch needs to be supported when it is being disassembled 7 Remove the rails 8 Remove the caged nuts 5 3 2 Disassembly Procedure Side by Side Installation To disassemble the switch from the rack Unplug and remove all connectors Unplug the power cord Remove the ground wire Unscrew the 2 center bolts from the connector side of switch Slide the switch from the rack Remove the rail slides rails from the rack St Ue AA OA Remove the 10 caged nuts 30 Mellanox Technologies InfiniScale IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 5 4 Disposal According to the WEEE Directive 2002 96 EC all waste electrical and electronic eguipment EEE should be collected separately and not disposed of with regular household waste Dispose of this product and all of its parts in a responsible and environmentally friendly way For proper disassembly instructions see the Mellanox website 5 5 Updating Firmware When you buy the switch it comes with the latest firmware burned on the board All firmware updates should be done in band Go to the Mellanox website and confirm that the firmware is the latest If not reburn the latest firmware from the download site New firmware versions will be posted on the Mellanox firmware download page http www mellanox com gt Support gt Download Center You will need the Mellanox Firmware Tools package to update firmware for this switch It can also be d
17. 40 lbs 40 70 lbs 70 121 lbs gt 121 lbs lt 18 kgs 18 32 kgs 32 55 kgs gt 55 kgs P Mellanox Technologies 51 Rev 1 4 Appendix G Advertencias de seguridad para la insta laci n Spanish 1 Instrucciones de instalaci n Antes de conectar el equipo a la fuente de alimentaci n leer todas las instrucciones de Instalaci n 2 Lesi n corporal por peso LA e b 40 lbs 40 70 Ibs 70 121 lbs gt 121 los lt 18 kgs 18 32 kgs 32 55 kgs gt 55 kgs 3 Lesi n corporal por peso Me This equipment is very heavy and should be moved using a mechanical lift to avoid A injuries 4 Sobrecalentamiento No se debe utilizar el eguipo en un rea con una temperatura ambiente superior a la m xima recomendada 45 C Adem s para garantizar una circulaci n de aire adec uada se debe dejar como m nimo un espacio de 8 cm 3 pulgadas alrededor de las aberturas de ventilaci n 5 Apilamiento del chasis Los chasis no se deben apilar sobre otros equipos La ca da del chasis podr a causar lesiones corporales as como da os al equipo 6 Dos fusibles uno en el polo y otro en el neutro VN Dos fusibles uno en el polo y otro en el neutro Quitar los cables de corriente antes de A abrir la tapa de este producto o tocar cualquier componente interno 52 Mellanox Technologies J InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 7 Cuando hay rayos peligro
18. A TIERRA DE PROTECCI N del terminal de puesta a tierra de protecci n separado al conductor de tierra de protecci n del edificio El equipo se instalar en un rea donde haya conexi n equipotencial como por ejemplo un centro de telecomunicaciones o una sala de computadoras dedicada 16 C digos de instalaci n Este dispositivo se debe instalar conforme a la versi n m s reciente de los c digos el ctricos nacionales del pa s en cuesti n En Am rica del Norte el equipo se debe instalar de acuerdo con las disposiciones vigentes del C digo El ctrico Nacional de los EE UU y del C digo El ctrico de Canad 17 Directiva sobre RAEE Conforme a la Directiva 2002 96 CE sobre RAEE todos los residuos de equipos el c tricos y electr nicos EEE se deben recolectar por separado y no se deben eliminar junto con residuos dom sticos A deshacerse de este producto y de todas sus partes h galo de una manera respon a sable y respetuosa con el medio ambiente 18 54 Mellanox Technologies InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 Appendix H Special Regulations Regarding Finland Swe den Denmark and Norway Denmark This unit is class I and must be connected with an AC cord compliant with all national electrical codes in Denmark The AC cord shall have an integral ground wire and can only be plugged into a fully grounded outlet Do not connect this unit to any outlet that is
19. Band Switch User Manual Rev 1 4 List of Figures Figure 1 Management MAC Label 11 Figure 2 Switch System Front Panel 16 Figure 3 Status LEDs 17 Figure 4 Port Numbering 20 Figure 5 Top and Bottom Ports 20 Figure 6 RJ45 I2C Connector 21 Figure 7 Placing the Bumpers 23 Figure 8 Installation Kit for a Single Centered Installation 24 Figure 9 Connect the Mounting Brackets 25 Figure 10 Installation Kit Parts for a Side by Side Installation 26 Figure 11 Screw on the Switch Mounted Rails 26 Figure 12 Caged Nut Placement 27 Figure 13 Front Panel Orientation 27 Figure 14 Top and Bottom Ports 29 Figure 15 MTUSB 1 with Cables 33 Figure 16 I2C Cable Connected to 185022 34 Figure 17 QSFP Male and Female Connections 43 Mellanox Technologies 5 Rev 1 4 List of Tables Table 1 Revision History Table 7 Table 2 Reference Documents 8 Table 3 Switch Status LED Configurations 17 Table 4 Fan LED Configurations 18 Table 5 Bad Port LED Configurations 18 Table 6 Port Connector Physical and Logical Link Indications 19 Table 7 Installation Kit Options 22 Table 8 Installation Kit According to Rack Size 28 Table 9 185022 Specification Data 37 Table 10 SFP Pinout 40 Table 11 InfiniBand QSFP Connector Pinout 42 Table 12 Replacement Parts Ordering Numbers 44 6 Mellanox Technologies InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 Revision History Table 1 Revision History Table
20. DUCT S AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE Mellanox TECHNOLOGIES Mellanox Technologies 350 Oakmead Parkway Suite 100 Sunnyvale CA 94085 U S A www mellanox com Tel 408 970 3400 Fax 408 970 3403 O Copyright 2012 Mellanox BridgeX ConnectX CORE Direct InfiniBridge InfiniHost InfiniScale PhyX SwitchX Virtual Protocol Interconnect and Vol taire are registered trademarks of Mellanox Technologies Ltd FabricIT MLNX OS and Unbreakable Link are trademarks of Mellanox Technologies Ltd All other trademarks are property of their respective owners All other trademarks are property of their respective owners Ethernet Switch Hardware User ManuallnfiniScale IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual 2 Mellanox Technologies Document Number 3373 InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 Contents Contents 3 List of Figures 5 List of Tables 6 Revision History 7 About this Manual 8 Intended Audience 8 Related Documentation 8 Conventions 8 Mellanox Part Numbering Legend 9 Chapter 1 Overview 10 1 1 Features 10 1 2 Serial Number and Product Version Information 10 Chapter 2 Installation Safety Warnings 12 Chapter 3 Externally Managed Switches 15 Chapter 4 Hardware Basic Operation and Installation 16 4 1 Switch Platform Hardware Overview 16 4 1 1 LED Assignments 16 4 1 2 Port Connector LED Assignment 19 4 2 AirFlow 19 4 3 Q
21. Mellanox TECHNOLOGIES InfiniScale IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual P N MIS5022Q 1BRR MIS5022Q 1BFR MIS5022Q 1BFR MIS5022Q 1BRR MIS5022Q 1BFC MIS5022Q 1BRC Rev 1 4 www mellanox com Rev 1 4 NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT S AND OR THE SYSTEM USING IT THEREFORE MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT INDIRECT SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES OF ANY KIND INCLUDING BUT NOT LIMITED TO PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY FROM THE USE OF THE PRO
22. SFP Cable Power Budget Classification 20 4 4 Interfaces 20 4 4 1 Port Connector Interfaces 20 4 5 Management and Firmware Updating Interfaces 20 4 5 1 RJ 45 Connector I2C 20 Chapter 5 Installation 22 5 1 Package Contents 22 5 2 Installing the Switch in the Rack 23 5 2 1 Table Top Installation 23 5 2 2 Installation Procedure for a Single Switch Center of Rack Installation 24 5 2 3 Installation Procedure for a Side by Side Installation 25 5 2 4 Power Connections and Initial Power On 28 5 2 5 Grounding the Switch 29 5 2 6 Cable Installation 29 5 2 7 Supported Approved Cables 30 5 3 Disassembly of the Switch from the Rack 30 5 3 1 Disassembly Procedure Single Switch Centered Installation 30 5 3 2 Disassembly Procedure Side by Side Installation 30 5 4 Disposal 31 5 5 Updating Firmware 31 5 5 1 Current Firmware Revision 31 5 5 2 Open SM 34 Chapter 6 Troubleshooting 35 Appendix A Specification 37 A l Approved Cables 38 A 2 EMC Certifications 38 Appendix B SFP Interface 39 Appendix C QSFP Interface 42 Mellanox Technologies 3 Rev 1 4 Appendix D Replacement Parts Ordering Numbers 44 Appendix E Avertissements de s curit d installation French 45 Appendix F Installation Sicherheitshinweise German 48 Appendix G Advertencias de seguridad para la instalaci n Spanish 52 Appendix H Special Regulations Regarding Finland Sweden Denmark and Norway 55 4 Mellanox Technologies InfiniScale amp IV 8 Port QSFP 40 Gb s Infini
23. Switch User Manual Rev 1 4 Caution When turning off the switch make sure ALL Connector LEDS are off to ensure a powered down status 5 2 5 Grounding the Switch Check to determine if your local or national electrical codes reguire an external ground to all IT components If so connect a ground wire to one of the casing screws and connect the other end to a valid ground If you choose to not use the ground screw make sure that the rack is properly grounded and that there is a valid ground connection between the chassis of the switch and the rack Test the ground using an Ohm meter A nally grounded not including the power cord ground You must follow all national gt Some national and or local codes may require IT components to be bonded and exter and local codes when installing this equipment 5 2 6 Cable Installation All cables can be inserted or removed with the unit powered on To insert a cable press the con nector into the port receptacle until the connector is firmly seated The orange LED indicator cor responding to each port will light when the physical connection is established that is when the unit is powered on and a cable is plugged into the port with the other end of the connector plugged into a functioning port After plugging in a cable lock the connector using the latching mecha nism particular to the cable vendor When a logical connection is made the green light will come on When data is being transferred
24. able de alimentaci n homologado por UL y con certificaci n CSA En conexiones de Am rica del Norte seleccionar un cable de alimentaci n homologado por UL y con certificaci n CSA de tres conductores 16 AWG termi nado en un enchufe moldeado con capuch n de 125 voltios nominal 13 A con una longitud m nima de 1 5 metros pero no m s de 4 5 metros En conexiones europeas seleccionar un cable de alimentaci n armonizado internacionalmente y marcado lt HAR gt de tres conductores hilo de 1 0 mm2 como m nimo 300 voltios nominal con cobertura protectora aislante de PVC El cable debe tener un enchufe moldeado con capuch n de 250 voltios nominal 10 A 15 A adir conexi n a tierra N Antes de conectar el dispositivo a la linea de alimentaci n los tornillos del terminal de la puesta a tierra de protecci n del dispositivo se deben conectar a la puesta a tierra de protecci n de la instalaci n del edificio Informaci n de conexi n a tierra La instalaci n del edificio deber prover un medio para la conexi n con la puesta a tierra de protecci n y un t cnico de servicio deber conectar permanentemente el equipo a dicho medio de conexi n Un T CNICO DE SERVICIO comprobar si la toma el ctrica de la que se suminis trar corriente al equipo provee una conexi n con la puesta a tierra de protecci n del edificio De no ser as el T CNICO DE SERVICIO se encargar de instalar un CON DUCTOR DE CONEXI N
25. ale elektrische Bestimmungen Dieses Ger t sollte unter Beachtung der regionalen und nationalen elektrischen Bestimmungen installiert werden 12 Richtigen Schutz sicherstellen Geeigneter elekrischer mechanischer und Feuerschutz sind vom Hersteller des Endprodukts oder dem Endbenutzer bereitzustellen 13 Strahlenkontakt Klasse 1 Laserprodukt und Referenzen zu den aktuellsten Lasterstandards Achtung Nutzung von Steuerungen oder Einstellungen oder Ausf hrung von Proze duren die hier nicht spezifiziert sind kann zu gef hrlichem Strahlenkontakt f hren lt lt ICE 60 825 1 1993 A1 1997 A2 2001 und EN 60825 1 1994 A1 1996 A2 2001 Mellanox Technologies 49 Rev 1 4 14 UL und CSA Certified Netzkabel AR F r Nordamerika Stromanschluss w hlen Sie ein Netzkabel das UL und CSA Certi fied 3 Leiter 18 AWG mit einem angespritztem Stecker bewertet bei 125 V 15 mit einer Mindestl nge von 1 5 m Six Feet aber nicht mehr als 4 5 m F r die europ ischen Zusammenhang w hlen Sie ein Netzkabel das international harmonisiert und der Aufschrift lt HAR gt 3 Leiter mindestens 0 75 mm2 Draht bewertet mit 300 V mit einem PVC Mantel isoliert Das Kabel muss eine angespritztem Stecker bewertet bei 250 V 10 A 15 Ableitstrom gt 3 5mA LEAKAGE gt 3 5mA WARNUNG Hohe Ableitstrom Earth Verbindung bevor Sie die Verbindung von wesentlicher Bedeutung werden 16 Add GND Verbindung Informa
26. ally managed switches must be managed by management software installed on anode or on another managed switch that can be anywhere in the fabric This can be OpenSM MLNX OS Mellanox FabricIT EFM Mellanox UFM or a third party Subnet Manager Mellanox Technologies 15 Rev 1 4 Hardware Basic Operation and Installation 4 4 1 Hardware Basic Operation and Installation Switch Platform Hardware Overview Figure 2 shows the front view of the switch The figure shows port configurations for the switch systems interface I2C connector and various status LEDs Externally managed switches come with an I2C RJ45 connector Figure 2 Switch System Front Panel All InfiniBand connectivity is via the connector side panel All connectors can support active cables 4 1 1 LED Assignments 4 1 1 1 System Status Indicators The System Status Indicators are located on the left side of the front connector side panel The system status indicators should display as follows When the switch is plugged in within three seconds The Status LED A should light up green OD should i The Fan LED should light up green The Bad Port LED 1 should be off The Unit ID LED should be off 16 Mellanox Technologies InfiniScale IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 Figure 3 Status LEDs Mellanox TFCHNOLOSIFS PIN MISS022Q 1 IN BFR Rev A1 IT HYS S N MT1030X00001 BLU NE E HUE TET UID
27. ation Table 6 Port Connector Physical and Logical Link Indications LED Configuration LED Description Off Physical link down Default Solid Green Physical link up no traffic Flashing Green Physical link up with traffic Flashing Orange Physical errors This LED when flashing orange shows port physical errors Possible causes for this are bad cable bad connection bad connector bad cage 4 2 AirFlow These switches can come with two air flow patterns The two patterns are power side inlet to connector side outlet connector side inlet to power supply side outlet The air flow is specified in the product model number See Mellanox Part Numbering Legend on page 9 Mellanox Technologies 19 Rev 1 4 Hardware Basic Operation and Installation 4 3 4 4 4 4 1 4 5 4 5 1 QSFP Cable Power Budget Classification All 185022 OSFP switches are designed for active cables with a max power per module of 2 0W This is power level 2 according to the QSFP Public Specification Interfaces Port Connector Interfaces The connector side of the switch has 8 QSFP ports These are placed in two rows 4 ports to a row The ports are labelled as shown in Figure 4 The bottom row ports are flipped from the top row See Figure 5 Figure 4 Port Numbering 1 3 5 7 2 4 6 8 Figure 5 Top and Bottom Ports umm Managemen
28. com related docs user manuals Regulatory and Compliance Guide pdf EMC statements are also in the Regulatory and Compliance Guide 38 Mellanox Technologies InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 Appendix B SFP Interface o 11 85 se A 1 55 8 50 gt lt 56 50 gt 14 80 13 60 B 5 B Mellanox Technologies PE MTSFPP SR Module Ma Mellanox Technologies 39 Top 13 70 8 50 H a TT rd m E M e lt lt q 4 4 5s 5 S 0 D v EA 3e 2 5 m 5 2 P Oma a a A u A a a o oo N o o gt o m m E M w gt a o N co o o de D D e mm 3 d o lt es x X 6 6 X o 2 4 m og gt O g i v x e y gt 9 SC 9 o e A Table 10 SFP Pinout Pin Symbol Name Description 1 VeeT Transmitter Ground Common with Receiver Ground 2 TX Fault Transmitter Fault 3 TX Disable Transmitter Disable Laser output disabled on high or open 4 SDA 2 wire Serial Interface Data Line 4 3 SCL 2 wire Serial Interface Clock Line 6 MOD_ABS Module Absent Grounded within the module 4 7 RSO No comnection required 8 RX_LOS Loss of Signal indication Logic 0 indicates normal operation
29. eiten Sie w hrend eines Gewitters und Blitzschlag nicht am Ger t schlie en Sie keine Kabel an oder ab 6 Anschlie en Trennen von InfiniBand Kupferkabel an die Anschl sse angebracht bzw davon getrennt werden Lesen Sie die speziellen InfiniBand Kupferkabel sind schwer und nicht flexible Deshalb m ssen sie vorsichtig A Warnungen und Anleitungen des Kabelherstellers 7 Gafahr des elektrischen Schocks frei Um alle Einhieten spannungsfrei zu machen sind die Netzstecker aller Netzteile zu Gafahr des elektrischen Schocks Entferrnen des Netzsteckers elnes Netzteils spannungs A entfernen 48 Mellanox Technologies InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 Risk of electric shock and energy hazard The PSUs are all independent Disconnect all power supplies to ensure a powered down state inside of the switch plat form 8 Rack Montage und Wartung sichtsma nahmen zu ergreifen um die Stabilit t des Systems zu gew hrleisten Im Wenn dieses Produkt in einem Rack montiert oder gewartet wird sind besondere Vor A Allgemeinen sollten Sie das Gestell von unten nach oben mit Ger ten f llen 9 Ger teinstallation Diese Ger t sollte nur von geschultem und qualifiziertem Personal installiert ausge tauscht oder gewartet werden 10 Ger teentsorgung Die Entsorgung dieses Ger ts sollte unter Beachtung aller nationalen Gesetze Bestim mungen erfolgen 11 Regionale und nation
30. for 19 inch racks Guaran tee proper ventilation by leaving 8cm 3 of space to the front and rear of the switch This will ensure proper airflow through the chassis This is crucial for maintaining good airflow at ambient temperature In particular route cables so that they do not impede the air into or out of the chassis 22 Mellanox Technologies J InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 5 2 Installing the Switch in the Rack This installation kit can only be used to install this switch Tools and Parts Tools required and customer supplied parts Phillips Screwdrivers 1 and 2 Grounding screw ESD Strap Grounding wire sufficient to reach a valid ground ESD mat The following procedure can be done by one person 5 2 1 Table Top Installation 1 Peel and stick the four rubber bumpers into the bottom of the switch Place them in the round circles Figure 7 Placing the Bumpers Bottom of the switch 2 _ Place on a flat surface Make sure the switch sits solid on the surface 3 Connect the power cord Mellanox Technologies 23 Rev 1 4 Installation 4 _ Connect the data transfer cables 5 2 2 Installation Procedure for a Single Switch Center of Rack Installation Installation kit parts for a single switch center of rack installation 2 front mounting brackets 8 flat head screws 4 M6 bolts 4 caged nuts for the rack vertical supports 1 P
31. g Any red status LED is cause for concern and must be dealt with immediately It can take up to 30 seconds to boot up during which time the status LED may indicate red Mellanox Technologies 21 Rev 1 4 Installation 5 5 1 Installation There are three possible installation kits for the MIS5022 switch Make sure that you have the installation kit that works with your rack Installation kits are not included with the switch and should be ordered separately This switch does NOT come with an installation kit There are three optional installa tion kits You must order the kit separately Table 7 Installation Kit Options OPN Description For Rack Depths MIS000811 Centered single switch installation kit 400mm 800mm MIS000830 Side by side installation kit with long rail slides 650mm 800mm MIS000831 Side by side installation kit with short rail slides 400mm 650mm Package Contents The package includes 1 switch box containing 1 power cord type B 6ft US 125V 10A chord 1 installation guide parts list Before you install your new switch unpack the system and check to make sure that all the parts have been sent check this against the parts list Check the parts for visible damage that may have occurred during shipping P If anything is damaged or missing contact your customer representative immediately 5 The rack mounting holes conform to the IEA 310 standard
32. icates information that is helpful to the user This symbol indicates a situation that can potentially cause damage to hardware or software 8 Mellanox Technologies InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 BEWARE This symbol indicates a situation that can potentially cause personal injury or damage to hardware or software Mellanox Part Numbering Legend Place Field Decoder M Mellanox Technologies IS System Type InfiniScale Switch 50 Model Family FF Form factor 22 8 Ports Separator Power supplies 1 1 power supply M _ Depth of the Unit B short depth Air Flow direc R Connector side to rear side airflow tion F Rear side to connector side airflow R RoHS R RoHS6 Mellanox Technologies 9 Rev 1 4 Overview 1 Overview This User Manual provides an overview of the InfiniBand switch product an 8 port switch capa ble of up to 640Gb s of non blocking switching capacity This switch is based on the fourth gener ation InfiniScale InfiniBand switch device The platform delivers industry leading bandwidth latency and scalability in a half width 1U form factor The platform can be tiered to build larger multi stage networks in a variety of topologies such as Fat Tree Mesh 3D Torus etc Mellanox 185022 switch systems are an ideal choice for top of rack connectivity from small to extremely large sized clusters This switch
33. itch LA Fans must be operating while the power supply is plugged in If the switch shuts down due to over temperature unplug the switch wait 5 minutes and replug in the switch For more information See Troubleshooting on page 35 4 1 1 4 Bad Port LED The bad port indicator is located on the left side of the front panel connector side of the unit The following bad port conditions are possible Table 5 Bad Port LED Configurations LED Configuration Description Off OK all ports are up and running Flashing Orange Error one or possibly more ports has just received a symbol error This LED shows symbol errors Possible causes for this are bad cable bad connection bad connector 18 Mellanox Technologies InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 This LED lights up when one or more ports is receiving a symbol error The LED immediately goes off until the next symbol error is received 4 1 1 5 UID LED Switch Identifier The UID LED is a debug feature that will become available to customers in the near future For details please contact Mellanox Technologies support 4 1 1 6 Port Connector LED The port LEDs are assigned to the ports as in figure 4 1 2 Port Connector LED Assignment Above the ports are two LEDs one for the upper port and one for the lower port V The fol lowing table shows the port status according to the LED indic
34. ity 10 90 non condensing Protocol Support Regulatory Compliance Mellanox Technologies 37 Rev 1 4 Table 9 185022 Specification Data Physical Power and Environmental InfiniBand Auto Negotiation of 40Gb s 20Gb s Safety US Canada cTUVus 10Gb s EU IEC60950 OoS 8 InfiniBand Virtual Lanes for all International CB ports Management EMC Emis USA FCC Class A Baseboard Performance and Device sions Canada ICES Class A management Agents for full Infini EU EN55022 Class A Band In Band Management EU EN55024 Class A EU EN61000 3 2 Class A EU EN61000 3 3 Class A Japan VCCI Class A Environmental EU IEC 60068 2 64 Ran dom Vibration EU IEC 60068 2 29 Shocks Type I II EU IEC 60068 2 32 Fall Test RoHS R 6 Acoustic ISO 7779 ETS 300 753 Sound power 70 5 dB A or 7 1 Bel level Mens Reliability Availability and Serviceability Features Switching Perfor Simultaneous wire speed any port to mance any port 48K Unicast Addresses Max per Sub Addressing net 16K Multicast Addresses per Subnet Switching Capacity 640Gb s AA Approved Cables For a list of all approved cables see www mellanox com related docs user manuals Mellanox approved cables pdf A 2 EMC Certifications The list of approved certifications per switch in different regions of the world is located on the Mellanox website at www mellanox
35. l Added 10 in band devices 2 List the discovered MST inband devices run mst status rootemymach gt mst status MST modules MST PCI module loaded MST PCI configuration module loaded Inband devices dev mst CA MT25418 sw005 HCA 1 lid 0x0001 dev mst SW MT47396 lid 0x0011 dev mst SW MT48438 lid 0x0003 Identify the switch you want to update according to its description or its LID In the examples below we will use SW MT48438 1id 0x0003 as our target switch Query the switch to determine the current firmware version root mymach gt flint d dev mst SW MT48438 lid 0x0003 q Image type FS2 FW Version 7 3 0 Device ID 48438 Chip Revision AO Description Node Sys image GUI Ds 0002c902004177d8 0002c902004177db Board ID n a MT 0D00110012 32 Mellanox Technologies J InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 VSD n a PSID MT_0D00110012 Burn the switch flint d dev mst SW_MT48438_lid 0x0003 i fw 1S4 rel 7_4_0 Q_Al bin qq b 5 5 1 2 Reprogramming the Switch Through The I2C Port This interface is for Debug and Troubleshooting by FAEs and advanced users only The I2C connector should only be used when the firmware cannot be updated in band 22 Use the MTUSB 1 adapter available from Mellanox to reprogram the switch firmware using a server or PC running MFT The MTUSB adapter comes with three cables but has only two connections for
36. lace the ESD mat on the floor where you will be working and put on the ESD strap Make sure the ESD strap is touching your skin and that the other end is connected to a verified ground 2 Screw the two front brackets to the front side of the switch using 4 flat head screws for each bracket Figure 8 Installation Kit for a Single Centered Installation 24 Mellanox Technologies J InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 Figure 9 Connect the Mounting Brackets Place four caged nuts into the rack at the U level you want to install the switch While supporting the switch catch and tighten the four bolts into the caged nuts Ground the switch Plug in the power cord au Pw You can start connecting all of the cables to the switch 5 2 3 Installation Procedure for a Side by Side Installation Installation kit parts for a side by side installation 1 Two switch metal frame Two rail slides 4 switch mounted rails 8 pairs of caged nuts and bolts 24 flat head screws blank cover The installation kits come with enough switch mounted rails and flat head screws to install two switches Mellanox Technologies 25 Rev 1 4 2 6 Installation Figure 10 Installation Kit Parts for a Side by Side Installation 1 Place the ESD mat on the floor where you will be working and put on the ESD strap Make sure the ESD strap is touching your skin and that the other end is
37. llation Kits There are two installation kit options One long and one short Both the long and the short installa tion kit can be used with this switch Table 8 Installation Kit According to Rack Size Kit OPN Rack Size MSX60 BKIT 40 60 cm MSX60 SKIT 60 80 cm For racks from 60cm to 80cm deep either use the standard depth switches with the long rail kit or the short switches with the long rail kit 5 2 4 Power Connections and Initial Power On Each PSU has a separate AC receptacle The input voltage is auto adjusting for 90 264 VAC 50 60Hz power connections The power cords should be standard 3 wire AC power cords including a safety ground and rated for 15A or higher Caution The power supplies in this switch are Double Pole Neutral Fusing Caution The switch platform will automatically power on when AC power is applied There is no power switch Check all boards power supplies and fan tray modules for proper insertion before plugging in a power cable yn Caution After inserting a power cable and confirming the green system status LED a n light is on make sure that the Fan Status indicator shows green If the fan status indicator is not green then unplug the power connection and check that the fan module is inserted properly and that the mating connector of the fan unit is free of any dirt and or obstacles 28 Mellanox Technologies J InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand
38. ould not be stacked on any other equipment Ifthe chassis falls it can cause bodily injury and equipment damage gt 4 Double pole Neutral Fusing This system has double pole neutral fusing Remove all power cords before opening the cover of this product or touching any internal parts 5 During Lightning Electrical Hazard During periods of lightning activity do not work on the equipment or connect or dis connect cables 6 Copper Cable Connecting Disconnecting or detached from the connectors Refer to the cable manufacturer for special warnings Copper cables are heavy and not flexible as such they should be carefully attached to A instructions 12 Mellanox Technologies J InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 7 Rack Mounting and Servicing to ensure that the system remains stable In general you should fill the rack with equip When this product is mounted or serviced in a rack special precautions must be taken A ment starting from the bottom to the top 8 High Leakage Current WARNING High leakage current Earth connection essential before connecting sup LA ply 9 Connect a Valid Ground to this Device Before connecting this device to the power line the protective earth terminal screws of this device must be connected to the protective earth in the building installation 10 Equipment Installation This equipment should be installed replaced and
39. ownloaded from http www mellanox com gt Support gt Software and Drivers You will also need to download and unzip the firmware binary image This is provided in the Mel lanox website at http www mellanox com gt Support gt Download Center and go to the InfiniS cale IV Switch systems Click in the table for the firmware image that you need 5 5 1 Current Firmware Revision The user can query for the currently loaded firmware revision by using the command ibstat part of MFT package Other tools like Flint can also be useful 5 5 1 1 Instructions for Reprogramming Over the InfiniBand Network To update a switch device having a specific GUID for example 0x00000006660abcd0 or LID the following are the recommended steps to update the device firmware 1 Make sure all subnet ports are in the active state One way to check this is to run opensm the Subnet Manager root mymach gt etc init d opensmd start opensm start OK Mellanox Technologies 31 Rev 1 4 Installation 2 Make sure the local ports are active by running ibv_devinfo 3 Obtain the device LID 1 Use the mst ib add command The mst ib add runs the ibdiagnet ibdiscover tool to discover the InfiniBand fabric and then lists the discovered IB nodes as an MST device under dev mst directory These devices can be used for access by other MFT tools rootemymach gt mst ib add Discovering the fabric Running ibdiagnet skip al
40. r European connection select a power supply cord that is internationally harmonized and marked lt HAR gt 3 conductor minimum 1 0 mm 2 wire rated at 300 V with a PVC insulated jacket The cord must have a molded plug rated at 250 V 10 A 17 Do Not Use the Switch as a Shelf or Work Space Caution Slide rail mounted equipment is not to be used as a shelf or a work space The rails are not intended for sliding the unit away from the rack It is for permanent instal lation at final resting place only not used for service and maintenance 18 WEEE Directive According to the WEEE Directive 2002 96 EC all waste electrical and electronic equipment EEE should be collected separately and not disposed of with regular household waste Dispose of this product and all of its parts in a responsible and environmentally friendly way 14 Mellanox Technologies InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev14 3 Externally Managed Switches Externally managed unmanaged switches are plug and play out of the box All switches come with the latest firmware FW burned on the Flash Updating the firmware on externally managed switches can be performed in band only When new firmware is available an email notification is sent to registered users with the link to the Mellanox firmware download site The download site has the Mellanox firmware tool package and full instructions for updating firmware Extern
41. rdering Numbers Part Description OPN Centered single switch installation kit MIS000811 Side by side installation kit with long rail slides MIS000830 supports 19 racks with depth of 630mm 800mm Side by side installation kit with short rail slides MIS000831 supports 19 racks with depth of 400mm 550mm RS232 Cable RJ45 to DB9 Harness HAR000028 LC DB or RJ45 to USB Adapter MTUSB 1 Power cord Type C13 C14 ACC000251 250V 10A 1 5M Power cord Type C13 C14 ACC000242 250V 15A 2M RIGHT ANGLE Power cord Type B for USA Canada Mexico Taiwan Type B 6ft US 125V 10A ACC000204 cord Power cord Type B for USA Canada Mexico Taiwan ACC000241 125V15A 4M USA UL Type B RIGHT ANGLE Power cord Type H for Israel ACC000205 Power cord Type E F for Sweden France Germany Netherlands Russia ACC000207 Power cord Type G for UK ACC000208 Power cord Type D for India ACC000209 Power cord Type I for China ACC000210 Power cord Type J for Switzerland ACC000211 Power cord Type B for Japan ACC000212 Power cord Type I for Australia ACC000213 44 Mellanox Technologies InfiniScale IV 8 Port OSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 Appendix E Avertissements de s curit d installation French 1 Instructions d installation Lisez toutes les instructions d installation avant de brancher le mat riel la source d alimentation lectrique lt 40 lbs 40 70 lbs
42. ront panel Front Panel indication LEDs Status LED Red Yellow Green Fans Status LED Red Green UID LED Blue Bad port LED Yellow Green 1 2 Serial Number and Product Version Information The serial number and GUID for the switch are found on the Mellanox logo The serial number and product version information are found on the label seen in the figure below 10 Mellanox Technologies InfiniScale IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 Generic Product label Also on this label is the GUID identifier for the switch P N MIS5022Q 1BFR Rev A1 UE EE DUE AREE NN S N MT1030X00001 r a 1 Mi III TERT Il 0002C903002 JOD LLL il ii MI Figure 1 Management MAC Label Management MAC 0002C902004C HOH Mellanox Technologies 11 Rev 1 4 Installation Safety Warnings 2 Installation Safety Warnings Warnings in French can be found on page 45 Warnings in German can be found on page 48 Warnings in Spanish can be found on page 52 1 Installation Instructions Read all installation instructions before connecting the equipment to the power source gt 2 Over temperature This equipment should not be operated in an area with an ambient temperature exceed ing the maximum recommended 45 C 113 F Moreover to guarantee proper air flow allow at least 8cm 3 inches of clearance around the ventilation openings gt 3 Stacking the Chassis The chassis sh
43. rsque ce produit est mont ou entretenu sur baie il faut prendre des pr cautions sp ciales pour s assurer que le systeme reste stable En g n ral il faut remplir la baie avec du mat riel de bas en haut 9 Forts Courants de Fuite High Leakage Current Attention Forts courants de fuite Il est essentiel de relier a la terre avant de brancher l alimentation 10 Ajouter une information de connexion la masse Connect a Valid Ground to this Device Avant de brancher l appareil la conduite d alimentation les vis de protection la terre du terminal de l appareil doivent tre appliqu es l installation de protection la Terre du b timent 11 Installation du mat riel Ce mat riel ne doit tre install remplac ou entretenu que par du personnel form et qualifi 12 Elimination du mat riel L limination de ce mat riel doit s effectuer dans le respect de toutes les l gislations et r glementations nationales en vigueur 13 Codes lectriques locaux et nationaux Ce mat riel doit tre install dans le respect des codes lectriques locaux et nationaux 46 Mellanox Technologies InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 14 Codes d installation pays Pour l Am rique du Nord l quipement doit tre install conform ment aux L appareil doit tre install selon l ancienne version des codes lectriques nationaux d
44. t and Firmware Updating Interfaces The externally managed switch platform does not require any programming or configuration to operate as a basic InfiniBand switch and includes all of the necessary functionality to operate with external standard InfiniBand Subnet Management software such as OFED FabricIT UFM etc There is one interface to connect to the switch It is an I2C RJ45 interface labelled I2C Use this connector to update firmware as a last resort and for advanced debug P Update firmware inband only gt RJ 45 Connector 12C There is an I2C interface on the front panel 20 Mellanox Technologies J InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 Figure 6 RJ45 I2C Connector EF Y Cm N All firmware updates should be done in band using Mellanox Firmware Management Tools The I2C connector is available only on externally managed switches and is used to update firm ware should in band firmware update not work You will need to order an MTUSB 1 USB to IC adapter to make use of the I2C interface The I2C connection provides access to Flash and EEPROMs This is for advanced users and FAEs only The externally managed switches are plug and play and all firmware updates should be done in band The I2C connection should only be used if the firmware image was cor rupted to the point that the regular firmware tools cannot successfully reburn the cor rect image Warnin
45. tch is off 1 Unplug the switch Wait 5 minutes Plug in the switch DD If the switch does not come on check the power supply Mellanox Technologies 35 Rev 1 4 Troubleshooting 5 Ifthe switch comes on use the management software to determine the cause of the shut down Check the temperature of the switch Check the ambient air temperature to make sure it is not too high 8 Check the Fan status The switch is not working and unresponsive 1 Unplug the switch 2 Wait 5 minutes 3 Plug in the switch 4 _ If the switch does not come on check the power supplies 5 If the switch comes on use the management software to determine the cause of the shut down 6 Reset the switch 36 Mellanox Technologies InfiniScale amp IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev14 Appendix A Specification Table 9 IS5022 Specification Data Physical Power and Environmental Size 1 73 1U H x 7 9 W x 9 8 D Input Voltage 100 240 VAC 50 60Hz 44mm X 200mm X 250mm Power Con Weight 1 92kg 4 23 Ibs sumption Typ Mounting 19 Rack mount 8 port QDR Passive Cables 46W Active Cables 66W SerDes Speeds 10 20 40 Gb s per port Connectors and Power Con Cabling QSFP Cable sumption Max 8 port QDR Optical Cable Passive Cables 55W Max Length Active Cables 75W Port Types QSFP QSFP Max Power per port for active cables 2 0W Temperature 0 to 45 Celsius Humid
46. tionen AR Bevor Sie dieses Ger t an das Stromnetz die Schutzerde Terminal Schrauben dieses Ger t muss an den Schutzleiter in der Geb udeinstallation 17 Installation Codes AR Dieses Ger t muss installiert sein entsprechend auf die neueste Version des Landes National Electrical Code F r Nordamerika m ssen in bereinstimmung mit den gel tenden Vorschriften in der US amerikanischen National Electrical Code und dem Canadian Electrical Code 18 Zusammenschaltung von EINHEITEN Kabel f r den Anschluss an das Ger t RS232 und Ethernet Schnittstellen m ssen UL zertifiziert Typ DP 1 oder DP 2 Hinweis wenn nicht mit Wohnsitz in LPS Schal tung berstromschutz Eine leicht zug ngliche Auflistung Abzweigleitung berstrom Schutzeinrichtung 20 A bewertet werden m ssen in dem Geb ude Verkabelung 19 Zweipolige bzw Neutralleiter Sicherung im Netzteil h Achtung Zweipolige bzw Neutralleiter Sicherung im Netzteil Netzstecker ziehen um sicher zustellen da keine Spannung am Ger t anliegt Entfernen Sie alle Netzkabel vor dem ffnen der Abdeckung dieses Produkts oder dem Ber hren der Innenteile 50 Mellanox Technologies InfiniScale IV 8 Port QSFP 40 Gb s InfiniBand Switch User Manual Rev 1 4 20 Bodily Injury Due to Weight lt 40 lbs 40 70 lbs 70 121 lbs gt 121 lbs lt 18 kgs 18 32 kgs 32 55 kgs gt 55 kgs Use enough people to safely lift this product ih e e e lt
47. u A sp cifications du Code Electrique National Am ricain et du Code Electrique Canadien 15 Interconnexion des unites Les c bles de branchement l unit RS232 et les interfaces Ethernet doivent tre certi fies UL de type DP 1 ou DP 2 Note lorsqu il existe dans un circuit non LPS Protection contre la surintensit Un appareil de protection r pertori facilement accessible contre la surintensit du circuit de branchement et calibr 20A doit tre incorpor dans le c blage lectrique du b timent 16 S assurer que les enceintes sont appropri es Des enceintes lectriques m caniques et incendie adapt es doivent tre fournies par le fabricant du produit final ou par l utilisateur final 17 Cordons lectriques CA homologu s UL Pour les prises lectriques en Am rique du Nord choisissez un cordon lectrique homologu UL et certifi CSA 3 conducteurs 18 AWG termin par une fiche moul e d une tension nominale de 125 V 15 A avec une longueur minimale de 1 5 m 6 pieds et d une longueur maxi male de 4 5 m 18 pieds Pour les prises lectriques en Europe choisissez un cordon lectrique harmonis inter nationalement et marqu lt HAR gt 3 conducteurs d un diam tre de fil minimum de 0 75 mm2 d une tension nominale de 300 V avec une gaine isol e en PVC Le cordon doit avoir une fiche moul e d une tension nominale de 250 V et d une intensit nominale de 10 A
Download Pdf Manuals
Related Search