InfiniScale® IV 8-Port QSFP 40Gb/s InfiniBand Switch User Manual
Contents
1. Part Description OPN Centered single switch installation kit MIS000811 Side by side installation kit with long rail slides MIS000830 supports 19 racks with depth of 630mm 800mm Side by side installation kit with short rail slides MIS000831 supports 19 racks with depth of 400mm 550mm RS232 Cable DB9 to DB9 Harness HAR000040 RS232 Cable RJ45 to DB9 Harness HAR000028 I2C DB9 or RJ45 to USB Adapter MTUSB 1 Power cord Type C13 C14 ACC000251 250V 10A 1 5M Power cord Type C13 C14 ACC000242 250V 15A 2M RIGHT ANGLE Power cord Type B for USA Canada Mexico Taiwan Type B 6ft US 125V 10A ACC000204 chord Power cord Type B for USA Canada Mexico Taiwan ACC000241 125V15A 4M USA UL Type B RIGHT ANGLE Power cord Type H for Israel ACC000205 Power cord Type E F for Sweden France Germany Netherlands Russia ACC000207 Power cord Type G for UK ACC000208 Power cord Type D for India ACC000209 Power cord Type I for China ACC000210 Power cord Type J for Switzerland ACC000211 Power cord Type B for Japan ACC000212 Power cord Type I for Australia ACC000213 Mellanox Technologies 43 Rev 1 2 Appendix E Avertissements de s curit d installation French 1 Instructions d installation Lisez toutes les instructions d installation avant de brancher le mat riel a la source d alimentation lectrique 2 Temp rature excessive Ce mat riel ne doit pas fonctionner2. IS5022O 1BFC IS5022O 1BRC A 1 1 FCC Statements USA Class A Statements 15 19 a 4 This device complies with Part 15 of the FCC Rules Operation is subject to the following two conditions 1 This device may not cause harmful interference and 2 _ This device must accept any interference received including interference that may cause undesired operation 15 21 Statement Warning Changes or modifications to this equipment not expressly approved by the party responsible for compliance Mellanox Technologies could void the user s authority to operate the equipment 15 105 a Statement NOTE This equipment has been tested and found to comply with the limits for a Class A digital device pursuant to Part 15 of the FCC Rules These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial envi ronment This equipment generates uses and can radiate radio frequency energy and if not installed and used in accordance with the instruction manual may cause harmful interference to radio communications Operation of this equipment in a residential area is likely to cause Mellanox Technologies 37 Rev 1 2 harmful interference in which case the user will be required to correct the interference at his own expense Statement 8 15 21 Statement Warning Changes or modifications to this equipment not expressly ap
3. potential bonding exists such as a telecommunication centre or a dedicated computer room 13 Installation codes This device must be installed according to the latest version of the country national electrical codes For North America equipment must be installed in accordance to the applicable requirements in the US National Electrical Code and the Canadian Electri cal Code 14 WEEE Directive According to the WEEE Directive 2002 96 EC all waste electrical and elec tronic equipment EEE should be collected separately and not disposed of with regular household waste Dispose of this product and all of its parts in a responsible and environmentally friendly way Mellanox Technologies 13 Rev 1 2 Externally Managed Switches 3 Externally Managed Switches Externally managed unmanaged switches are plug and play out of the box All switches come with the latest firmware FW burned on the Flash Updating the FW on externally managed switches can be performed in band only When new FW is available an email notification is sent to registered users with the link to the Mellanox FW download site The download site has the Mellanox FW tool package and full instructions for updating FW Externally managed switches must be managed by management software installed on a node or on another managed switch that can be anywhere in the fabric This can be OpenSM Mellanox FabricIT EFM Mellanox UFM or a third party Subnet Manager
4. 14 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 4 Hardware Basic Operation and Installation 4 1 Switch Platform Hardware Overview Figure 2 shows the front view of the switch The figure shows port configurations for the switch systems I2C connector and various status LEDs Externally Managed switches come with an I2C RJ45 connector Figure 2 Switch System Front Panel All InfiniBand connectivity is via the connector side panel All connectors can support active cables 4 1 1 LED Assignments 4 1 1 1 System Status Indicators The System Status Indicators are located on the left side of the front connector side panel The system status indicators should display as follows When the switch is plugged in within three seconds the Status LED A should light up green The Fan LED amp should light up green The Bad Port LED ES should be off The Unit ID LED UID should be off Mellanox Technologies 15 Rev 1 2 Hardware Basic Operation and Installation Figure 3 Status LEDs Mellanox TECHNOLOGIES PIN MISSOZ2Q 1BFR Rev A1 THE HS US MT1020X00001 TL GR 000209030027 tj GUID DD27051C ULB LU DE DU EEE N Mae in IL If the Status LED shows red after 30 seconds unplug the switch and call your Mellanox representative for assistance If the Fan LED shows red unplug the switch and call your Mellanox representative for assistance If the switch shuts down
5. 7 3 0 Device ID 48438 Chip Revision AO Description Node Sys image GUIDs 0002c902004177d8 0002c902004177db Board ID n a MT_0D00110012 VSD n a PSID MT_0D00110012 30 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 Burn the InfiniScale IV switch flint d dev mst SW_MT48438_lid 0x0003 i fw IS4 rel 7_4_0 Q_A1 bin qq b 5 5 1 2 Reprogramming the Switch Through The I2C Port This interface is for Debug and Troubleshooting only This interface is for FAEs and advanced users only 4 The I2C connector should only be used when the firmware cannot be updated in band Use the MTUSB 1 adapter available from Mellanox to reprogram the switch firmware using a server or PC running MFT Figure 15 MTUSB 1 with Cables The MTUSB adapter comes with three cables but has only two connections for cables One cable is an I2C cable with DB9 female connectors on both ends One cable is a I2C cable with DB9 female connector to RJ45 male Use the cable that matches your switch The other is a USB cable with one type A host connection to be connected to the server or PC and a type B connection to be connected to the MTUSB adapter Figure 16 I2C Cable Connected to I55022 155022 Mellanox Technologies 31 Rev 1 2 Installation With the switch connected to a computer through the MTUSB it is now possible to reprogram the SPI Flash memory 5 5 1 3 Instructions for Repr
6. s reciente de los c digos el ctricos nacionales del pa s en cuesti n En Am rica del Norte el equipo se debe instalar de acuerdo con las disposiciones vigentes del C digo El ctrico Nacional de los EE UU y del C digo El ctrico de Canad 14 Directiva sobre RAEE Conforme a la Directiva 2002 96 CE sobre RAEE todos los residuos de equipos el ctricos y electr nicos EEE se deben recolectar por separado y no se deben eliminar junto con residuos dom sticos Al deshacerse de este producto y de todas sus partes h galo de una manera res ponsable y respetuosa con el medio ambiente 50 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 Appendix H Special Regulations Regarding Finland Swe den Denmark and Norway Denmark This unit is class I and must be connected with an AC cord compliant with all national electrical codes in Denmark The AC cord shall have an integral ground wire and can only be plugged into a fully grounded outlet Do not connect this unit to any outlet that is not fully grounded Finland Laite on liitett v suojamaadoituskoskettimilla varustettuun pistora siaan Norway Apparatet m tilkoples jordet stikkontakt Unit is intended for connection to IT power systems for Norway only Sweden Apparaten skall anslutas till jordat uttag gt Mellanox Technologies 51
7. Bad Port LED Configurations LED Configuration Description Off OK All ports are up and running Flashing Orange Error One or possibly more ports has just received a symbol error This LED shows symbol errors Possible causes for this are bad cable bad connection bad connector This LED lights up when one or more ports is receiving a symbol error The LED immediately goes off until the next symbol error is received Mellanox Technologies 17 Rev 1 2 Hardware Basic Operation and Installation 4 1 1 5 UID LED Switch Identifier The UID LED is a debug feature that will become available to customers in the near future For details please contact Mellanox Technologies support 4 1 1 6 Port Connector LED 4 2 4 3 Above the ports are two LEDs one for the upper port A and one for the lower port Y The fol lowing table shows the port status according to the LED indication Table 6 Connector Physical and Logical Link Indications LED Description Off No link Orange Physical Link up Solid Green Logical Link up Flashing Green Data activity flashing speed data transfer speed Flashing Orange A problem with the Physical Link This LED when flashing orange shows port physical errors Possible causes for this are bad cable bad connection bad connector bad cage Air Flow These switches can come with two air flow patterns The two patterns are Rear side i
8. Canada Class A Statement This Class A digital apparatus complies with Canadian ICES 003 Cet appareil num rique de la classe A est conforme a la norme NMB 003 du Canada Class B Statement 38 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 A2 VCCI Statements Japan Class A Statement COB ERULERBFERBEB EHE HERZ VCCI OE CROC VIAZAGRRMHEBCI CORE TREERE CHATS CBR Gees KROTFCEPHVET TOSLLHEABEDEYNENEEHET SEFBKANSZTCEPHV ET Translation This is a Class A product based on the standard of the Voluntary Control Council for Interference by Information Technology Equipment VCCI If this equipment is used in a domestic environment radio interference may occur in which case the user may be required to take corrective actions A 3 MIC Certification Korea Korea s Regulation for Certification of Information and Communication Equipment requires EMC testing and certification for many electronic products Korean EMC certifications are issued by Radio Research Laboratory RRL which is organized under the Ministry of Information and Communications MIC EMC testing includes electromagnetic emissions EMI and susceptibil ity EMS Certified equipment is labeled with the MIC mark and certification number 971111 YF LE AAS SES A 71710 1 Baa EE ASAE oA BH 793417 nel ve BS pol FE THRE A A ee Translation Class A Device This device is registered for EMC requirements for industr
9. dans une zone avec une temp rature ambi ante d passant le maximum recommand de 45 C 113 F Un flux d air de 200LFM cette temp rature ambiante maximale est n cessaire En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouvertures de ventilation gt 3 Empilage du ch ssis Le ch ssis ne doit pas tre empil sur un autre mat riel Si le ch ssis tombe il peut provoquer des blessures corporelles et des d gradations de biens gt 4 Orages dangers lectriques Pendant un orage il ne faut pas utiliser le mat riel et il ne faut pas brancher ou d brancher les c bles gt 5 Branchement d branchement des cables InfiniBand en cuivre Les cables InfiniBand en cuivre sont lourds et ne sont pas flexibles il faut donc faire tr s attention en les branchant et en les d branchant des connecteurs Consultez le fabricant des c bles pour conna tre les mises en garde et les instructions sp ciales gt 6 Montage et entretien sur baie Lorsque ce produit est mont ou entretenu sur baie il faut prendre des pr cau tions sp ciales pour s assurer que le syst me reste stable En g n ral il faut remplir la baie avec du mat riel de bas en haut gt 7 Installation du mat riel Ce mat riel ne doit tre install remplac ou entretenu que par du personnel form et qualifi gt 44 Mellanox Technologies QDR InfiniBand
10. due to over temperature unplug the switch wait 5 minutes and replug in the switch For more information See Troubleshooting on page 33 41 1 2 Status LED The Status indicator is located on the left side of the front panel connector side of the unit The following status conditions are possible Table 3 Switch Status LED Configurations LED Description Off No power to the switch Switch is running Solid Green All OK There is a problem with power output from Red the power supply or there is a thermal shut down 16 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 4 1 1 3 Fan Indicators amp The Fan indicator is located on the left side of the front panel connector side of the unit The fol lowing fan status conditions are possible Table 4 Fan LED Configurations LED Configuration FAN LED Off There is no power to the fans Green OK The fans are running Red Error fans are not operating properly Replace the switch Fans must be operating while the power supply is plugged in If the switch shuts down due to over temperature unplug the switch wait 5 minutes and replug in the switch For more information See Troubleshooting on page 33 4 1 1 4 Bad Port LED The Bad Port indicator is located on the left side of the front panel connector side of the unit The following Bad Port conditions are possible Table 5
11. gt 11 UL Listed and CSA Certified Power Supply Cord For North American power connection select a power supply cord that is UL Listed and CSA Certified 3 conductor 16 AWG terminated with a molded plug rated at 125 V 13 A with a minimum length of 1 5m six feet but no longer than 4 5m gt For European connection select a power supply cord that is internationally harmonized and marked lt HAR gt 3 conductor minimum 1 0 mm 2 wire rated at 300 V with a PVC insulated jacket The cord must have a molded plug rated at 250 V 10 A 12 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 12 Add GND connection information Before connecting this device to the power line the protective earth terminal screws of this device must be connected to the protective earth in the building installation GND Connection Information The building installation shall provide a means for connection to protective earth and the equipment shall to be connected to that means permanently by a service person A SERVICE PERSON shall check whether or not the socket outlet from which the equipment is to be powered provides a connection to the building protective earth If not the SERVICE PERSON shall arrange for the installation of a PROTECTIVE EARTHING CONDUCTOR from the separate protective earthing terminal to the pro tective earth wire in the building The equipment shall be installed in area where equi
12. specified in the Specification section of the Appendix If any of the LEDs are red after 30 seconds unplug the switch and call your Mellanox representa tive Power supply unit 1 Check that the power cable has a voltage within the range of 100 240 volts AC If the status LED is red 2 Remove and reinstall the power cable The green power LED for the fans does not come on Caution Do not run the switch if the System Status LED for the Fans is red or off The connector LED for the InfiniBand connector does not come on 1 Check that both ends of the cable are connected 2 Check that the locks on the ends are secured 3 Make sure that the latest FW version is installed on both the HCA cards and the switch 4 If media adapters are used check that the all connections are good tight and secure The connector LED is flashing orange 1 Check that the Subnet Manager has been started 2 Check the cable is connected properly at borh ends 3 Check the the cable is good 4 Check that the connections are good no cable weight on the connector no twists in the cable at the connector 5 Check the port connectors at both ends The switch is off 1 Unplug the switch 2 Wait 5 minutes 3 Plugin the switch Mellanox Technologies 33 Rev 1 2 Troubleshooting 4 _ If the switch does not come on check the power supply 5 If the switch comes on Use the management software to determine the cause of th
13. ts sollte unter Beachtung aller nationalen Gesetze Bestimmungen erfolgen 9 Regionale und nationale elektrische Bestimmungen Dieses Ger t sollte unter Beachtung der regionalen und nationalen elektrischen Bestimmungen installiert werden Mellanox Technologies 47 Rev 1 2 Appendix G Advertencias de seguridad para la insta laci n Spanish 1 Instrucciones de instalaci n Antes de conectar el equipo a la fuente de alimentaci n leer todas las instrucciones de instalaci n 2 Sobrecalentamiento No se debe utilizar el equipo en un rea con una temperatura ambiente superior a la m xima recomendada 45 C Adem s para garantizar una circulaci n de aire adecuada se debe dejar como m nimo un espacio de 8 cm 3 pulgadas alrededor de las aberturas de ventilaci n gt 3 Apilamiento del chasis Los chasis no se deben apilar sobre otros equipos La ca da del chasis podr a causar lesiones corporales as como da os al equipo gt 4 Cuando hay rayos peligro de descarga el ctrica No utilizar el equipo ni conectar o desconectar cables durante per odos de actividad de rayos gt 5 Conexi n y desconexi n del cable Copper InfiniBand Dado que los cables de cobre InfiniBand son pesados y no son flexibles su conexi n a los conectores y su desconexi n se deben efectuar con mucho cuidado Para ver advertencias o instrucciones especiales consul tar al fabricante del cable gt 6 Montaj
14. AMA Mellanox TECHNOLOGIES InfiniScale IV 8 Port QSFP 40Gb s InfiniBand Switch User Manual P N MIS5022Q 1BRR MIS5022Q 1BFR MIS5024Q 1BRR MIS5024Q 1BFR MIS5022Q 1BFR MIS5022Q 1BRR MIS5022Q 1BFC MIS5022Q 1BRC Rev 1 2 www mellanox com Rev 1 2 NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO S AND OR THE SYSTEM USING IT THEREFORE MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT INDIRECT SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES OF ANY KIND INCLUDING BUT NOT LIMITED TO PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARIS
15. ING IN ANY WAY FROM THE USE OF THE PRODUCT S AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE Mellanox TECHNOLOGIES Mellanox Technologies Mellanox Technologies Ltd 350 Oakmead Parkway Suite 100 PO Box 586 Hermon Building Sunnyvale CA 94085 Yokneam 20692 U S A Israel www mellanox com Tel 972 4 909 7200 Tel 408 970 3400 Fax 972 4 959 3245 Fax 408 970 3403 O Copyright 2011 Mellanox Technologies All rights reserved Mellanox BridgeX ConnectX Virtual Protocol Interconnect InfiniBlast InfiniBridge InfiniHost InfiniRISC InfiniScale and InfiniPCI are registered trademarks of Mellanox Technologies Ltd CORE Direct FabricIT and PhyX are trademarks of Mellanox Technologies Ltd All other trademarks are property of their respective owners All other marks and names mentioned herein may be trademarks of their respective companies QDR InfiniBand Switch Platform User Manual Document Number 3373 2 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 Contents Contents 3 List of Figures 5 List of Tables 6 Revision History 7 About this Manual 8 Intended Audience 8 Related Documentation 8 Conventions 8 Mellanox Part Numbering Legend 9 Chapter 1 Overview 10 1 1 Serial Number and Product Version Information 10 Chapter 2 Installation Safety Warnings 11 Chapter 3 Externally Managed Switches 14 Chapter 4 Hardware Basic Operation and Installation 15 4 1 Switch P
16. Rev 1 2 Hardware Basic Operation and Installation Figure 6 RJ45 I2C Connector A rm N A El a am e M All firmware updates should be done in band using Mellanox Firmware Management Tools The I2C connector is available only on externally managed switches and is used to update firm ware should in band firmware update not work You will need to order an MTUSB 1 USB to I2C adapter to make use of the I2C interface The I2C connection provides access to Flash This is for Advanced users and FAEs only The externally managed switches are Plug and Play and all firmware updates should be done in band The I2C connection should only be used if the FW image was corrupted to the point that the regular FW tools cannot successfully reburn the correct image Warning Any red status LED is cause for concern and must be dealt with imme diately It can take up to 30 seconds to boot up during which time the status LED may indicate red 20 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 5 Installation There are three possible installation kits for the MIS5022 switch Make sure that you have the installation kit that works with your rack Installation kits are not included with the switch and should be ordered separately Table 7 Installation Kit Options OPN Description For Rack Depths MIS000811 Centered single switch installation kit 400mm 800mm MIS000830 Side by sid
17. S Canada cTUVus 10Gb s EU IEC60950 Qos 8 InfiniBand Virtual Lanes for all International CB ports Management EMC Emis USA FCC Class A Baseboard Performance and Device sions Canada ICES Class A management Agents for full EU EN55022 Class A InfiniBand In Band Management EU EN55024 Class A EU EN61000 3 2 Class A EU EN61000 3 3 Class A Japan VCCI Class A Environmental EU IEC 60068 2 64 Ran dom Vibration EU IEC 60068 2 29 Shocks Type I II EU IEC 60068 2 32 Fall Test RoHS R 6 Acoustic ISO 7779 ETS 300 753 Sound power 70 5 dB A or 7 1 Bel level So andEenomaner Reliability Availability and Serviceability Features Switching Perfor Simultaneous wire speed any port to mance any port 48K Unicast Addresses Max per Sub Addressing net 16K Multicast Addresses per Subnet Switching Capacity 640Gb s A 1 EMC Certification Statements Table 9 lists the approved certification status per switch in different regions of the world Table 9 Switch Certification Status FCC EN ICES VCCI Switch P N Class Class Class dadas cTUVus CB USA Europe Canada P IS5022Q 1BRR IS5022O 1BFR 36 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 Table 9 Switch Certification Status FCC EN ICES VCCI Switch P N Class Class Class cTUVus CB USA Europe Canada Japan IS5022O 1BFR IS5022O 1BRR
18. Switch Platform User Manual Rev 1 2 8 Elimination du mat riel L limination de ce mat riel doit s effectuer dans le respect de toutes les l gis lations et r glementations nationales en vigueur 9 Codes lectriques locaux et nationaux Ce mat riel doit tre install dans le respect des codes lectriques locaux et nationaux 10 Cordons lectriques CA homologu s UL Pour les prises lectriques en Am rique du Nord choisissez un cordon lec trique homologu UL et certifi CSA 3 conducteurs 18 AWG termin par une fiche moul e d une tension nom inale de 125 V 15 A avec une longueur minimale de 1 5 m 6 pieds et d une longueur maximale de 4 5 m 18 pieds Pour les prises lectriques en Europe choisissez un cordon lectrique harmon is internationalement et marqu lt HAR gt 3 conducteurs d un diam tre de fil minimum de 0 75 mm2 d une tension nominale de 300 V avec une gaine isol e en PVC Le cordon doit avoir une fiche moul e d une tension nominale de 250 V et d une intensit nominale de 10 A Mellanox Technologies 45 Rev 1 2 Appendix F Installation Sicherheitshinweise German 1 Installationsanleitungen Lesen Sie alle Installationsanleitungen bevor Sie das Ger t an die Stromver sorgung anschlie en 2 bertemperatur Dieses Ger t sollte nicht in einem Bereich mit einer Umgebungstemperatur ber der maximal empfohlenen Temper
19. atur von 45 C 113 F betrieben werden Es ist ein Luftstrom von 200 LFM bei maximaler Umgebungstemper atur erforderlich Au erdem sollten mindestens 8 cm 3 in Freiraum um die Bel ftungs ffnungen sein um einen einwandfreien Luftstrom zu gew hrleisten gt 3 Stapeln des Chassis Das Chassis sollte nicht auf andere Ger te gestapelt werden Wenn das Chassis herunterf llt Kann es zu Verletzungen und Besch digungen an Ger ten f hren gt 4 Bei Gewitter Elektrische Gefahr Arbeiten Sie w hrend eines Gewitters und Blitzschlag nicht am Ger t schlie en Sie keine Kabel an oder ab gt 5 Anschlie en Trennen von InfiniBand Kupferkabel InfiniBand Kupferkabel sind schwer und nicht flexible Deshalb miissen sie vorsichtig an die Anschliisse angebracht bzw davon getrennt werden Lesen Sie die speziellen Warnungen und Anleitungen des Kabelherstellers gt 6 Rack Montage und Wartung Wenn dieses Produkt in einem Rack montiert oder gewartet wird sind beson dere Vorsichtsma nahmen zu ergreifen um die Stabilit t des Systems zu gew hrleisten Im Allgemeinen sollten Sie das Gestell von unten nach oben mit Ger ten fiillen gt 7 Ger teinstallation Diese Ger t sollte nur von geschultem und qualifiziertem Personal installiert ausgetauscht oder gewartet werden gt 46 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 8 Ger teentsorgung Die Entsorgung dieses Ger
20. ble is unseated Care should be taken not to impede the air exhaust flow through the ventilation holes next to the InfiniBand ports Cable lengths should be used which allow for routing horizontally around to the side of the chassis before bending upward or downward in the rack 5 3 Disassembly of the Switch from the Rack 5 3 1 Disassembly Procedure Single Switch Centered Installation To disassemble the switch from the rack 1 Unplug and remove all connectors 2 Unplug the power cord 3 Remove the ground wire 4 Remove the four M6 screws holding the switch to the rack Support the switch while remov ing the screws or the switch may fall out of the rack A Loosen the two thumbscrews holding the switch in place 6 Slide the switch out of the rack This switch needs to be supported when it is being disassembled 7 Remove the rails 8 Remove the caged nuts 28 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 5 3 2 Disassembly Procedure Side by Side Installation To disassemble the switch from the rack Unplug and remove all connectors Unplug the power cord Remove the ground wire Unscrew the 2 center bolts from the connector side of switch Slide the switch from the rack Remove the rail slides rails from the rack SO Re D Remove the 10 caged nuts 5 4 Disposal According to the WEEE Directive 2002 96 EC all waste e
21. cables to the switch 5 2 3 Installation Procedure for a Side by Side Installation Installation kit parts for a side by side installation 1 Two switch metal frame Two Rail Slides e 4 switch mounted rails 8 pairs of caged nuts and bolts e 24 flat head screws 1 blank cover The Installation kits come with enough switch mounted rails and flat head screws to install two switches 24 Mellanox Technologies J QDR InfiniBand Switch Platform User Manual Rev 1 2 Figure 10 Installation Kit Parts for a Side by Side Installation 1 Place the ESD mat on the floor where you will be working and put on the ESD strap Make sure the ESD strap is touching your skin and that the other end is connected to a verified ground 2 _ Screw the two switch mounted rails to the switch using six flat head screws per rail Figure 11 Screw on the Switch Mounted Rails Mellanox Technologies 25 Rev 1 2 Installation 3 Place eight caged nuts into the rack at the level you want to mount the switch Four go on the Leaf side and four go on the rear side Figure 12 Caged Nut Placement e Leaf side Rear side 4 Install the two switch frame into the rack This frame goes on the leaf side of the rack Do not tighten the screws into the caged nuts 5 Slide the rail slides into the frame and screw the bolts into the rack caged nuts on the rear side of the rack Do not tighten the screws into the caged nuts Slide the switch i
22. capa ble of up to 640Gb s of non blocking switching capacity Based on the fourth generation InfiniS cale IV InfiniBand switch device The platform delivers industry leading bandwidth latency and scalability in a half width 1U form factor The platform can be tiered to build larger multi stage networks in a variety of topologies such as Fat Tree Mesh 3D Torus etc All port configurations are covered in this manual which include switches for QSFP InfiniBand ports Mellanox IS5022 switch systems are an ideal choice for smaller departmental or back end cluster ing uses with high performance needs such as storage data base and GPGPU clusters Networks built with IS5022 systems can carry converged traffic with the combination of assured bandwidth and granular quality of service IS5022 systems provide up to 40Gb s full bidirectional bandwidth per port The switch platform comes pre installed with all necessary firmware and is configured for stan dard operation within an InfiniBand fabric This switch requires an InfiniBand compliant Subnet Manager running from one of the hosts or another managed switch All that is required for normal operation is to follow the usual precautions for installation and to connect the switch to the HCAs or other switches within the fabric Once connected the Subnet Management software automati cally configures and begins utilizing the switch It is recommended that the Mellanox OpenFabrics software package be install
23. ceiver Non Inverted Data Output 3 23 GND Ground 24 Rx4n Receiver Inverted Data Output 3 25 Rx4p Receiver Non Inverted Data Output 3 26 GND Ground 27 ModPrsL Module Present 28 IntL Interrupt 29 Vcc Tx 3 3 V Power supply transmitter 30 Vcc 1 3 3 V Power Supply 31 LPMode Low Power Mode 32 GND Ground 33 Tx3p Transmitter Non Inverted Data Input 34 Tx3n Transmitter Inverted Data Input 35 GND Ground 36 Tx1p Transmitter Non Inverted Data Input 37 Txin Transmitter Inverted Data Input 38 GND Ground 40 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 Figure 17 QSFP Male and Female Connections To P 18 35 View into Rear of Connector ye lt E gt o 22 I a S 2 0 y 43 LE Sek LI SES SER 2 xO p oS cs 05 F Or FRA O PISOS 5 a O DUO IT IT 320 o2 lt a zz 9 A 1 D AAO Zz 2 28 ZI BD 0 Z 2 KE 2 033 US3 CFT oe 905 5 SS x r o To SEP 18 35 View into Front of Cage Mellanox Technologies 41 Rev 1 2 Appendix C RJ45 12C Interface The RJ45 12C interface uses the EIA 568A standard wiring color coding Looking into the Socket FC J34 Pin1 RJ45 SDA GND GND GND GND RJ45 SCL GND GND Pin 8 Looking into the Socket 42 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 Appendix D Replacement Parts Ordering Numbers Table 11 Replacement Parts Ordering Numbers
24. e installation kit with long rail slides 650mm 800mm MIS000831 Side by side installation kit with short rail slides 400mm 650mm 5 1 Package Contents The package includes e 1 switch e 1 installation kit 1 box containing 1 power cord Type B 6ft US 125V 10A chord e installation guide e 1 parts list Before you install your new switch unpack the system and check to make sure that all the parts have been sent check this against the parts list Check the parts for visible damage that may have occurred during shipping A If anything is damaged or missing contact your customer representative immediately The rack mounting holes conform to the IEA 310 standard for 19 inch racks Guarantee A proper ventilation by leaving 8cm 3 of space to the front and rear of the switch This i will ensure proper air flow through the chassis This is crucial for maintaining good air flow at ambient temperature In particular route cables such that they do not impede the air into or out of the chassis Mellanox Technologies 21 Rev 1 2 Installation 5 2 Installing the Switch in the Rack This installation kit can only be used to install this switch Tools and Parts Tools required and customer supplied parts Phillips Screwdrivers 1 and 2 e Grounding screw ESD Strap e Grounding wire sufficient to reach a valid ground ESD mat L The following pr
25. e shut down 6 Check the temperature of the switch 7 _ Check the ambient air temperature to make sure it is not too high 8 Check the Fan status The switch is not working and unresponsive 1 Unplug the switch Wait 5 minutes Plug in the switch If the switch does not come on check the power supplies AD If the switch comes on use the management software to determine the cause of the shut down 34 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 Appendix A Specification Table 8 IS5022 Specification Data Physical Power and Environmental Size 1 73 1U H x 7 9 W x 9 8 D 44mm X 200mm X 250mm Weight 1 92kg 4 23 lbs Mounting 19 Rack mount SerDes Speeds 10 20 40 Gb s per port Connectors and Cabling QSFP Cable Optical Cable Max Length Port Types QSFP Input Voltage 100 240 VAC 50 60Hz Power Con sumption Typ 8 port QDR Passive Cables 46W Active Cables 66W Power Con sumption Max 8 port QDR Passive Cables 55W Active Cables 75W QSFP Max Power per port for active cables 2 0W Temperature 0 to 45 Celsius Humidity 10 90 non condensing Protocol Support Regulatory Compliance Mellanox Technologies 35 Rev 1 2 Table 8 IS5022 Specification Data Physical Power and Environmental InfiniBand Auto Negotiation of 40Gb s 20Gb s Safety U
26. e y mantenimiento de bastidores Al instalar o realizar el mantenimiento de este aparato en un bastidor es preciso adoptar precauciones especiales para garantizar que el sistema se mantenga estable En general en un bastidor los equipos se deben insta lar comenzando desde abajo hacia arriba gt 7 Instalaci n de equipos La instalaci n el reemplazo y el mantenimiento de este equipo estar n a cargo nicamente de personal capacitado y competente gt 48 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 8 Asegurar confinamientos adecuados El fabricante del producto final o el usuario final deber n suministrar un confinamiento adecuado para componentes el ctricos y mec nicos y contra incendio 9 Eliminaci n de equipos La eliminaci n definitiva de este equipo se debe efectuar conforme a todas las leyes y reglamentaciones nacionales 10 C digos el ctricos locales y nacionales A Este equipo se debe instalar conforme a los c digos el ctricos locales y nacio nales 11 Cable de alimentaci n homologado por UL y con certificaci n CSA En conexiones de Am rica del Norte seleccionar un cable de alimentaci n homologado por UL y con certificaci n CSA de tres conductores 16 AWG terminado en un enchufe moldeado con capuch n de 125 voltios nominal 13 A con una longitud m nima de 1 5 metros pero no m s de 4 5 metros En conexiones europeas seleccionar un cable de alime
27. ed on all nodes con nected to the 185022 The software package provides a subnet manager and network management tools as well as connectivity software for servers and storage and is available on the Mellanox web site See Chapter 3 for more information Basic installation and hardware maintenance is covered in Hardware Basic Operation and Instal lation on page 15 1 1 Serial Number and Product Version Information The Serial number and GUID for the switch are found on the front panel below the Mellanox logo The serial number and product version information are found on the label seen in the figure below Figure 1 Generic Product label P N MIS5022Q 1BFR Rev A1 AT S N MT1030X00001 l l INN N A a Il D 0002C903002705 Also on this label is the GUID identifier for the switch 10 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 2 Installation Safety Warnings Warnings in French can be found on page 44 Warnings in German can be found on page 46 Warnings in Spanish can be found on page 48 1 Installation Instructions Read all installation instructions before connecting the equipment to the power source 2 Bodily Injury Due to Weight Use enough people to safely lift this product lt 40 lbs 40 70 lbs 70 121 lbs gt 121 lbs lt 18 kgs 18 32 kgs 32 55 kgs gt 55 kgs 3 Over temperature This equipment should not be operated in an area with an ambient temperature
28. exceeding the maximum recommended 45 C 113 F Moreover to guarantee proper air flow allow at least 8cm 3 inches of clearance around the ventilation openings 4 Stacking the Chassis The chassis should not be stacked on any other equipment If the chassis falls it can cause bodily injury and equipment damage 5 During Lightning Electrical Hazard During periods of lightning activity do not work on the equipment or connect or disconnect cables Mellanox Technologies 11 Rev 1 2 Installation Safety Warnings 6 Copper InfiniBand Cable Connecting Disconnecting Copper InfiniBand cables are heavy and not flexible as such they should be carefully attached to or detached from the connectors Refer to the cable manu facturer for special warnings instructions gt 7 Rack Mounting and Servicing When this product is mounted or serviced in a rack special precautions must be taken to ensure that the system remains stable In general you should fill the rack with equipment starting from the bottom to the top gt 8 Equipment Installation This equipment should be installed replaced and or serviced only by trained and qualified personnel A 9 Equipment Disposal Disposal of this equipment should be in accordance to all national laws and reg ulations gt 10 Local and National Electrical Codes This equipment should be installed in compliance with local and national elec trical codes
29. ial use The seller or buyer should be aware of this If this type was sold or purchased by mistake it should be replaced with a residential use type Mellanox Technologies 39 Rev 1 2 Appendix B QSFP Interface 0 GND GND 19 21 non Rxin 18 Table 10 InfiniBand QSFP Connector Pinout 22 Rx2p Rx1p 17 cC 23 GND GND 8 onnector Connector 7 ee 24 45 Pin Pin N Signal Description Rx4n Ran I Number oe 25 Rx3 14 ae 1 GND Ground 2 anp 3 CRD 2 Tx2n Transmitter Inverted Data Input El ModPrsL spa 28 11 3 Tx2p Transmitter Non Inverted Data Input o WN a 4 GND Ground 30 ER un 9 5 Tx4n Transmitter Inverted Data Input Vcc1 ResetL Wa 8 6 Tx4p Transmitter Non Inverted Data Input 1 LPMode ModSelL OND God 32 7 GND ND 5 8 ModSelL Module Select 33 nap Tx4p 34 Tyan Tx4n 5 9 ResetL Module Reset 25 GND GND 4 10 VccRx 3 3 V Power supply receiver al Txip Tx2pl 8 11 SCL 2 wire serial interface clock gt Txin Tx2nl 2 Tr NG a 1 12 SDA 2 wire serial interface data 13 GND Ground 14 Rx3p Receiver Non Inverted Data Output 15 Rx3n Receiver Inverted Data Output 16 GND Ground 17 Rxlp Receiver Non Inverted Data Output 18 Rx1n Receiver Inverted Data Output 19 GND Ground 20 GND Ground 21 Rx2n Receiver Inverted Data Output 3 22 Rx2p Re
30. in the installation kit Caution The switch platform will automatically power on when AC power is applied There is no power switch Caution After inserting a power cable and confirming the green system status LED light is on make sure that the Fan Status indicator shows green If the fan status indicator is not green then unplug the power connection and call your supplier 5 2 6 InfiniBand Cable Installation All cables can be inserted or removed with the unit powered on To insert a cable press the con nector into the port receptacle until the connector is firmly seated The orange LED indicator cor responding to each QSFP port will light when the physical connection is established that is when the unit is powered on and a cable is plugged into the port with the other end of the connector plugged into a functioning port After plugging in a cable lock the connector using the latching Mellanox Technologies 27 Rev 1 2 Installation mechanism particular to the cable vendor When a logical connection is made the green light will come on When data is being transferred the light will blink green Cables in the bottom row should be inserted up side down in relation to the how the cables are inserted in the top row Figure 14 Top and Bottom Ports To remove disengage the locks and slowly pull the connector away from the port receptacle The LED indicator will turn off when the ca
31. is a subset of the Mellanox OFED stack Mellanox Firmware The MFT Mellanox Firmware Tools package is a set of firmware tools Tools MFT The manual supplied with this package provides an overview of the firm User s Manual ware its installation and replacement The MFT can be downloaded with its Document 2329 documentation at http www mellanox com gt Support gt Download Firmware Tools Conventions Throughout this manual the name IS5022 and the term switch are used to describe the 8 port QSFP 40Gb s InfiniBand Switch unless explicitly indicated otherwise 8 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 N This symbol indicates information that is helpful to the user BEWARE This symbol indicates a situation that can potentially cause personal injury or damage to hardware or software Mellanox Part Numbering Legend Place Field Decoder M Mellanox Technologies IS System Type InfiniScale Switch 50 Model Family FF Form factor 22 8 Ports Separator Power supplies 1 1 power supply M Depth of the Unit B short depth Y Air Flow direc R Connector side to rear side airflow tion F Rear side to connector side airflow R RoHS R RoHS6 Mellanox Technologies 9 Rev 1 2 Overview 1 Overview This User Manual provides an overview of the InfiniBand switch product an 8 port switch
32. latform Hardware Overview 15 4 1 1 LED Assignments 15 4 2 AirFlow 18 4 3 OSFP Cable Power Budget Classification 18 4 4 Interfaces 19 4 4 1 Port Connector Interfaces 19 4 5 Management and FW Updating Interfaces 19 4 5 1 RJ 45 Connector I2C 19 Chapter 5 Installation 21 5 1 Package Contents 21 5 2 Installing the Switch in the Rack 22 5 2 1 Table Top Installation 22 5 2 2 Installation Procedure for a Single Switch Center of Rack Installation 23 5 2 3 Installation Procedure for a Side by Side Installation 24 5 2 4 Grounding the Switch 27 5 2 5 Power Connections and Initial Power On 27 5 2 6 InfiniBand Cable Installation 27 5 3 Disassembly of the Switch from the Rack 28 5 3 1 Disassembly Procedure Single Switch Centered Installation 28 5 3 2 Disassembly Procedure Side by Side Installation 29 5 4 Disposal 29 5 5 Updating Firmware 29 5 5 1 Current Firmware Revision 29 5 5 2 How to Get Mellanox Firmware Tools MFT 32 5 5 3 Open SM 32 Chapter 6 Troubleshooting 33 Appendix A Specification 35 A 1 EMC Certification Statements 36 A 2 VCCI Statements Japan 39 A 3 MIC Certification Korea 39 Appendix B QSFP Interface 40 Appendix C RJ45 I2C Interface 42 Mellanox Technologies 3 Rev 1 2 Appendix D Replacement Parts Ordering Numbers 43 Appendix E Avertissements de s curit d installation French 44 Appendix F Installation Sicherheitshinweise German 46 Appendix G Advertencias de seguridad para la instalaci n Spanish 48 Appe
33. lectrical and elec tronic equipment EEE should be collected separately and not disposed of with regular household waste Dispose of this product and all of its parts in a responsible and environmentally friendly way For proper disassembly instructions see the Mellanox website 5 5 Updating Firmware When you buy the switch it comes with the latest firmware burned on the board All firmware updates should be done in band Go to the Mellanox Website and confirm that the firmware is the latest If not reburn the latest firmware from the download site New firmware versions will be posted on the Mellanox firmware download page http www mellanox com gt Support gt Download Center You will need the Mellanox Firmware Tools package to update firmware for this switch It can also be downloaded from http www mellanox com gt Support gt InfiniBand Software and Drivers You will also need to download and unzip the firmware binary image This is provided in the Mel lanox Web site at http www mellanox com gt Support gt Download Center and go to the InfiniS cale IV Switch systems Click in the Table for the firmware image that you need 5 5 1 Current Firmware Revision The user can query for the currently loaded firmware revision by using the command ibstat Mellanox Technologies 29 Rev 1 2 Installation part of mft package Other tools like flint can also be useful 5 5 1 1 Instructions for Reprogramming Ove
34. ndix H Special Regulations Regarding Finland Sweden Denmark and Norway 51 4 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 List of Figures Figure 1 Generic Product label 10 Figure 2 Switch System Front Panel 15 Figure 3 Status LEDs 16 Figure 4 Port Numbering 19 Figure 5 Top and Bottom Ports 19 Figure 6 RJ45 I2C Connector 20 Figure 7 Placing the Bumpers 22 Figure 8 Installation Kit for a Single Centered Installation 23 Figure 9 Connect the Mounting Brackets 24 Figure 10 Installation Kit Parts for a Side by Side Installation 25 Figure 11 Screw on the Switch Mounted Rails 25 Figure 12 Caged Nut Placement 26 Figure 13 Front Panel Orientation 26 Figure 14 Top and Bottom Ports 28 Figure 15 MTUSB 1 with Cables 31 Figure 16 I2C Cable Connected to IS5022 31 Figure 17 QSFP Male and Female Connections 41 Mellanox Technologies 5 Rev 1 2 List of Tables Table 1 Revision History Table 7 Table 2 Reference Documents 8 Table 3 Switch Status LED Configurations 16 Table 4 Fan LED Configurations 17 Table 5 Bad Port LED Configurations 17 Table 6 Connector Physical and Logical Link Indications 18 Table 7 Installation Kit Options 21 Table 8 IS5022 Specification Data 35 Table 9 Switch Certification Status 36 Table 10 InfiniBand QSFP Connector Pinout 40 Table 11 Replacement Parts Ordering Numbers 43 6 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 Revi
35. nlet to connector side outlet e Connector side inlet to rear side outlet The air flow is specified in the product model number See Mellanox Part Numbering Legend on page 9 QSFP Cable Power Budget Classification All 185022 QSFP switches are designed for active cables with a max power per module of 2 0W This is power level 2 according to the QSFP Public Specification 18 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 4 4 Interfaces 4 4 1 Port Connector Interfaces The Connector side of the switch has 8 QSFP ports These are placed in two rows 4 ports to a row The ports are labelled as shown in Figure 4 The bottom row ports are flipped from the top row See Figure 5 Figure 4 Port Numbering 1 3 5 7 2 4 6 8 Figure 5 Top and Bottom Ports _ 57 4 5 Management and FW Updating Interfaces This switch is externally managed There is no CPU in this switch for management purposes The switch can be managed through any fabric management software such as Open SM Mellanox UFM There is one interface to connect to the switch It is an I2C RJ45 interface labelled I2C Use this connector to update firmware as a last resort and for advanced debug Update FW inband only 4 5 1 RJ 45 Connector 12C There is an I2C interface on the front panel Mellanox Technologies 19
36. ntaci n armonizado internacionalmente y marcado lt HAR gt de tres conductores hilo de 1 0 mm2 como m nimo 300 voltios nominal con cobertura protectora aislante de PVC El cable debe tener un enchufe moldeado con capuch n de 250 voltios nomi nal 10 A 12 A adir conexi n a tierra Antes de conectar el dispositivo a la l nea de alimentaci n los tornillos del terminal de la puesta a tierra de protecci n del dispositivo se deben conectar a la puesta a tierra de protecci n de la instalaci n del edificio Informaci n de conexi n a tierra La instalaci n del edificio deber prover un medio para la conexi n con la puesta a tierra de protecci n y un t cnico de servicio deber conectar permanentemente el equipo a dicho medio de conexi n Un T CNICO DE SERVICIO comprobar si la toma el ctrica de la que se suministrar corriente al equipo provee una conexi n con la puesta a tierra de protecci n del edificio De no ser as el T CNICO DE SERVICIO se encar gar de instalar un CONDUCTOR DE CONEXI N A TIERRA DE PROTEC CI N del terminal de puesta a tierra de protecci n separado al conductor de tierra de protecci n del edificio El equipo se instalar en un rea donde haya conexi n equipotencial como por ejemplo un centro de telecomunicaciones o una sala de computadoras dedicada Mellanox Technologies 49 Rev 1 2 13 C digos de instalaci n Este dispositivo se debe instalar conforme a la versi n m
37. nto the frame and screw in the capture bolts into the frame 7 Tighten the capture nuts 8 Install the blank into the frame opposite the switch or install a second switch The front panel is not symetrical The large radius always goes ot the center of the nr frame Figure 13 Front Panel Orientation arge radius Small radius Small radius 26 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 9 Tighten the capture nuts 10 Tighten all eight screws for the caged nuts to 9 2 Nm or 81 5 pound inches 11 Ground the switch 12 Plug in the power cord 13 You can start connecting all of the cables to the switch 5 2 4 Grounding the Switch Check to determine if your local or national electrical codes require an external ground to all IT components If so connect a ground wire to one of the casing screws and connect the other end to a valid ground If you choose to not use the ground screw make sure that the rack is properly grounded and that there is a valid ground connection between the chassis of the switch and the rack Test the ground using an Ohm meter Some national and or local codes may require IT components to be bonded and exter gt nally grounded not including the power cord ground You must follow all national and local codes when installing this equipment 5 2 5 Power Connections and Initial Power On The power cord is included
38. ocedure can be done by one person 5 2 1 Table Top Installation 1 Peel and stick the four rubber bumpers into the bottom of the switch Place them in the round circles Figure 7 Placing the Bumpers Bottom of the switch 2 Place on a flat surface Make sure the switch is sits solid on the surface 3 Connect the power cord 22 Mellanox Technologies J QDR InfiniBand Switch Platform User Manual Rev 1 2 4 Connect the data transfer cables 5 2 2 Installation Procedure for a Single Switch Center of Rack Installation Installation kit parts for a single switch center of rack installation 2 front mounting brackets e 8 flat head screws 4 M6 bolts 4 caged nuts for the rack vertical supports 1 Place the ESD mat on the floor where you will be working and put on the ESD strap Make sure the ESD strap is touching your skin and that the other end is connected to a verified ground 2 _ Screw the two front brackets to the front side of the switch using 4 flat head screws for each bracket Figure 8 Installation Kit for a Single Centered Installation Mellanox Technologies 23 Rev 1 2 Installation Figure 9 Connect the Mounting Brackets Place four caged nuts into the rack at the U level you want to install the switch While supporting the switch catch and tighten the four bolts into the caged nuts Ground the switch Plug in the power cord L D RD You can start connecting all of the
39. ogramming Through the 12C Port 1 Make sure that ethe MTUSB I2C cable is connected to the switch and USB cable is connected to a computer MFT is running on the computer mtusb 1 mst device appears in mst status 2 _ Set the switch I2C switch to allow access by running gt mlxi2c d dev mst mtusb 1 s 155023 p IS4 3 _ Run the burn command flint d dev mst mtusb 1 i fw IS4 rel 7 4 0 Q A1 bin qq b 4 _ Power cycle the switch by unplugging and re plugging the power cord to load the new firm ware 5 5 2 How to Get Mellanox Firmware Tools MFT Mellanox Firmware Tools MFT and documentation are available for download via http www mellanox com gt Support gt Download Firmware Tools The MFT kit includes mlxburn flint spark IBspark debug utilities See Related Documentation on page 8 5 5 3 Open SM To manage the Mellanox switch system using OFED download Mellanox Open Fabrics from http www mellanox com gt Support gt InfiniBand Software and Drivers Be sure to read and follow all of the instructions regarding the installation and use of these tools 32 Mellanox Technologies QDR InfiniBand Switch Platform User Manual Rev 1 2 6 Troubleshooting As soon as a switch is plugged in continuously monitor the status light Make sure that it turns green within 30 seconds of booting up Status LED and Fan LED If the Fan LED is red make sure that the ambient temperature is within the range
40. proved by the party responsible for compliance Mellanox Technologies could void the user s authority to operate the equipment 515 105 Statement NOTE This equipment has been tested and found to comply with the limits for a Class B dig ital device pursuant to part 15 of the FCC Rules These limits are designed to provide reason able protection against harmful interference in a residential installation This equipment generates uses and can radiate radio frequency energy and if not installed and used in accor dance with the instructions may cause harmful interference to radio communications However there is no guarantee that interference will not occur in a particular installation If this equip ment does cause harmful interference to radio or television reception which can be determined by turning the equipment off and on the user is encouraged to try to correct the interference by one or more of the following measures e Reorient or relocate the receiving antenna Increase the separation between the equipment and receiver Connect the equipment into an outlet on a circuit different from that to which the receiver is connected Consult the dealer or an experienced radio TV technician for help A 1 2 EN Statements Europe EN55022 Class A Statement Warning This is a class A product In a domestic environment this product may Cause radio interference in which case the user may be A 1 3 ICES Statements
41. r the InfiniBand Network To update an InfiniScale IV switch device having a specific GUID for example 0x00000006660abcd0 or LID the following are the recommended steps to update the device firmware 1 Make sure all subnet ports are in the active state One way to check this is to run opensm the Subnet Manager rootamymach gt etc init d opensmd start opensm start OK Make sure the local ports are active by running ibv_devinfo Obtain the device LID 1 Use the mst ib add command The mst ib add runs the ibdiagnet ibdiscover tool to discover the InfiniBand fabric and then lists the discovered IB nodes as an mst device under dev mst directory These devices can be used for access by other MFT tools root mymach gt mst ib add I Discovering the fabric Running ibdiagnet skip all I Added 10 in band devices 2 List the discovered mst inband devices run mst status root mymach gt mst status MST modules MST PCI module loaded MST PCI configuration module loaded Inband devices dev mst CA_MT25418_sw005_HCA 1_lid 0x0001 dev mst SW MT47396 lid 0x0011 dev mst SW MT48438 lid 0x0003 Identify the switch you want to update according to its description or its LID in the examples below we will use SW MT48438 lid 0x0003 as our target switch Query the switch to determine the current FW version root mymach gt flint d dev mst SW MT48438 lid 0x0003 q Image type FS2 FW Version
42. sion History Table 1 Revision History Table Date Revision Description Mar 2011 Rev 1 2 Added note and figure to blank panel installtion step Made changes to the installation process for the two switch frame Made changes to FW burning sections Feb 2011 Rev 1 1 Removed UID LED support Jan 2011 Rev 1 0 Initial Release Mellanox Technologies 7 Rev 1 2 About this Manual This manual describes the installation and basic use of the Mellanox 1585022 switch which is based on the InfiniScale IV InfiniBand switch device Intended Audience This manual is intended for users and system administrators responsible for installing and setting up the switch platforms listed above The manual assumes familiarity with the InfiniBand Architecture Specification Related Documentation Additional Documentation available from Mellanox Table 2 Reference Documents Switch Firmware and See Firmware Update http www mellanox com gt Support gt Download Firmware Tools Tools Note that the Switch System described in this manual is based on Mellanox Technologies InfiniScale IV switch device Mellanox OFED See Stack for Linux http www mellanox com gt Support gt InfiniBand Software and Drivers User s Manual Click Mellanox OpenFabrics Enterprise Distribution for Linux MLNX_OFED Select the Linux User s Manual The embedded OS and tools on the CPU in the management module
Download Pdf Manuals
Related Search