Home

216 Port InfiniBand QDR Switch Platform Hardware User Manual

1. MAK ry y Wh 4 y Wh 4 y WO d STATUS LA V W AV EEEE ABAARAARA 2 3 2 1 Status LED Each leaf board has a Status LED on the far left of the leaf Table 4 shows the leaf status accord ing to the LED condition The Leaf Status indicator LED has the following LED assignment Table 4 Leaf Status LED LED Condition LED Description Off No power to the Leaf Solid Green Leaf is up and running Flashing Green Leaf is powering up Solid Orange Non fatal error this leaf needs troubleshooting but does not require chassis shutdown Solid Red Fatal error 2 3 2 2 Leaf Board Port Connector LED Assignment Above the ports are two LEDs one for the upper port and one for the lower port V Each port has a single 2 color LED Table 5 shows the link status according to the LED condition Table 5 Connector Physical and Logical Link Indications LED Condition LED Description Off No power to the port Solid Green Logical link up Flashing Green Data activity flashing speed is proportional to data transfer speed Solid Orange Physical link up Flashing Orange A problem with the physical link The LED indicator corresponding to each data port will light orange when the physical connec tion is established that is when the unit is powered on and a cable is plugged into the port with Mellanox Technologies 23 R
2. 4 There are 2 Ethernet ports 1 for each management module that get connected to Ethernet switches These switches must be configured to 10 100M auto negotiation gt The IS5200 switch comes standard with a management software module for chassis management called FabricIT FabricIT is installed on all management modules for the managed switch sys tems based on InfiniScaleIV amp technology FabricIT includes a CLI WebUI SNMP and chassis management features for software and IB management software OpenSM You can get more information via the Mellanox FabricIT EFM Software WebUI User s Manual or the Mellanox FabricIT EFM Software User Manual The managed switch system includes the following software components Embedded Subnet Manager SM Chassis manager and system BIST SNMP agent 3rd party tool integration GUI Remote logging e SSH telnet Secured access in band and out band IPv4 IPv6 network stack The chassis manager will give the user access to Switch temperatures Power supply voltages Fanunit information Power unit information Flash memory Monitoring of AC power to the PSUs DC power out from the PSUs chassis failures querying for switch serial numbers revisions software version SwitchX FW version Mellanox Technologies 71 Rev 1 8 Switch Management Tools switch temperatures The manager also has the ability to burn new firmware and upgrade software on the sw
3. The cable holders should go to the outside of the vertical supports Ae 42 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 2 Set the bottom of the cable holder at the level of the shelf If the cable holder is not set properly the upper bracket will not line up with the cable D o holder Figure 24 Cable Holder Placement Notch for upper bracket washer Set this even with the top of the shel 3 Screw the cable holder onto the rack using the screws provided 4 Repeat steps 1 3 for the second cable holder Mellanox Technologies 43 Rev 1 8 Installation 3 5 4 Installing the Chassis Figure 25 Chassis Rails and Rail Slides e Rail Slide ka me ANNE 1 Screw the handles onto the chassis Use the 8 allen head screws Two go on the faceplate and two go on the sides of the chassis 2 Decide which side of the rack you want the data connectors 44 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Figure 26 Placement of Chassis in Rack Doors RACK Cable with recommended bending radius Chassis E Spine side Insufficient room for recommended bending radius Doors You will most likely need extra room for cable bending on the connector side of the chassis and should plan to keep the spine side as close to the rack door as possible thus having more room for the
4. 3 4 Management Package Contents When a second management module is ordered the package includes 1 management module ordered by the customer e 1 RJ45 to DB9 harness 3 5 Physical Installation Warning This equipment is very heavy Safety is the first concern Make sure that adequate manpower and proper equipment is used for transporting and moving the LA chassis The fully loaded chassis weighs 120 kg 265 LBS full configuration 48 KG 105 LBS empty configuration 83 94kg 185 LBS shipped configuration The switch platform uses 10U of rack pace in a standard 19 rack 9U for the chassis and 1U for the shelf The switch ships from the factory with mounting holes on the spine side There are upper brackets to connect the leaf side to the rack near the top ofthe chassis The weight of the switch is supported from underneath the unit by the shelf This chassis can be installed in standard 19 racks that have between 65cm and 80cm between the vertical supports of the rack Make sure that a fully populated rack including cables will have sufficient air flow for cooling Choose a rack which is able to support the mechanical and environmental characteris tics of a fully populated switch chassis aa Mellanox Technologies 33 Rev 1 8 Installation Y Mellanox recommends a rack that is at least 80 cm long 3 5 1 ESD Connection Before starting any procedure on the 155200 switch system 1 Put an ESD prevention wrist st
5. EMC Certifications The list of approved certifications per chassis in different regions of the world is located on the Mellanox Website at http www mellanox com related docs user manuals Regulatory and Compliance Guide pdf EMC Statements are also in the Regulatory and Compliance Guide 78 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Appendix B Calculating the Weight of a Customized Chassis The weight of a customized chassis can be calculated for any possible customization as follows Take the weight of a chassis with the following FRUs installed All fans modules All power supplies The weight of the 185200 chassis configured above is 83 94kg To this add the weight of installed FRUs Spines Leafs e Leaf blanks Management modules Fill in the table below to calculate the weight of your system Number of FRUs FRU Type Weight of 1 FRU Total Weight FRU Type Chassis as shipped of Spines 3 77 of leafs 2 76 of leaf blanks 764 of management modules 3 15 Total This total is in kilograms Multiply the total by 2 2 to get the total weight in pounds Mellanox Technologies 79 Rev 1 8 Appendix C Calculating the Power of a Chassis To calculate the power consumption of a chassis take the base power and add the power of the fans spines management modules
6. LPMode ModSelL 13 GND Ground BM a a Rx3p Receiver Non Inverted Data Out 3 Tp Tx4p 14 34 Tx8n Tx4n S put 35 GND GND 4 15 Rx3n Receiver Inverted Data Output Taip Tx2p 2 16 GND Ground un Im 17 Rxlp ame Non Inverted Data Out 18 Rxln Receiver Inverted Data Output 19 GND Ground 20 GND Ground 21 Rx2n Receiver Inverted Data Output 3 2 Rx2p Receiver Non Inverted Data Out put 3 23 GND Ground 24 Rx4n Receiver Inverted Data Output 3 25 Rx4p Receiver Non Inverted Data Out put 3 26 GND Ground 27 ModPrsL Module Present 28 IntL Interrupt 29 Vec Tx 3 3 V Power supply transmitter 30 Vcc 1 3 3 V Power Supply 31 LPMode Low Power Mode 32 GND Ground Tx3n Transmitter Non Inverted Data Mellanox Technologies 81 Rev 1 8 lews QSFP Connector Male and Female Vi Figure 54 Top 18 35 View into Rear of Connector GND AXin Txip GND TX3n TX3p GND LPMode Vcc1 VecTx IntL ModPrsL GND RX4p RX4n GND RX2p RX2n GND Ny N M Eu GND TX2n TX2p GND TX4n TX4p GND ModSelE ResetL VccRx SCL SDA GND 18 35 RX3p RX3n GND RX1p RX1n GND View into Front of Cage aNd HHS diXH UND UEXH dexu UND vas 19S XH99A Tieseu I9SPOWN QN9 dpXL UPXL AND dzXL uZXL AND 4 4 0 9 8 7 6 5 4 3 2 EF Mellanox Technologies 82 QDR InfiniBand Swi
7. A N Dos fusibles uno en el polo y otro en el neutro Quitar los cables de corriente antes de abrir la tapa de este producto o tocar cualquier componente interno Mellanox Technologies 93 Rev 1 8 7 Cuando hay rayos peligro de descarga el ctrica No utilizar el equipo ni conectar o desconectar cables durante per odos de actividad de rayos 8 Conexi n y desconexi n del cable Copper InfiniBand f Dado que los cables de cobre InfiniBand son pesados y no son flexibles su conexi n a los conectores y su desconexi n se deben efectuar con mucho cuidado Para ver adver tencias o instrucciones especiales consultar al fabricante del cable 9 Montaje y mantenimiento de bastidores Al instalar o realizar el mantenimiento de este aparato en un bastidor es preciso adop tar precauciones especiales para garantizar que el sistema se mantenga estable En gen eral en un bastidor los equipos se deben instalar comenzando desde abajo hacia arriba 10 Instalaci n de equipos La instalaci n el reemplazo y el mantenimiento de este equipo estar n a cargo nica mente de personal capacitado y competente 11 Asegurar confinamientos adecuados El fabricante del producto final o el usuario final deber n suministrar LA un confinamiento adecuado para componentes el ctricos y mec nicos y contra incen dio 12 Eliminaci n de equipos La eliminaci n definitiva de este equipo se debe efectuar conforme a todas las ley
8. ante nen da ja ER Re Gam wae eo RENTE 72 Chapter 8 TroubleShooting 73 amp T Power Supply Unit are read pue bad ne den IH Ra 73 82 eat Board mn e sn nait areas eI AR E ER EE T EE 73 8 3 Management Module 74 8 3 1 Yellow Status LED for the Chassis on the Management Module is Lit 74 8 3 2 Yellow LED for the Leaf Fan on the Management Module is Lit 74 8 3 3 Yellow LED for the Spine Fan on the Management Module is Lit 75 84 lt Spitie Boards its issus sr ME entame an Mae OM Dis AL Nate 75 8 5 Fabrici Software een Be em rend as mem dr en a oes 75 Appendix A Specification Data 76 01 EMI Certification are sed kr GAS ESL Dash ern nn 78 02 Approved Cables 40 is a a ll DEIN 78 0 3 EMG Certifications cec 3 ota er gids ER SRE En 78 Appendix B Calculating the Weight of a Customized Chassis 79 Appendix C Calculating the Power of a Chassis 80 Appendix D QSFP Interface 81 Appendix E Replacement Parts Ordering Numbers 83 Appendix F Avertissements de s curit d installation French 85 Appendix G Installation Sicherheitshinweise German 89 Appendix H Advertencias de seguridad para la instalaci n Spanish 93 4 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 List of Figures Figure 1 Generic Product Label 11
9. s Manual 8 Table 2 Reference Documents and Web Sites 9 Table3 Switch Rack Mechanical and Environmental Requirements Fully Populated Chassis 13 Table 4 Leaf Status LED 23 Table 5 Connector Physical and Logical Link Indications 23 Table 6 Spine Status LED 24 Table 7 Spine Fan Status LED 25 Table 8 Spine to Leaf IB Link Status 25 Table 9 LEDs Display for Normal Operation 26 Table 10 LEDs Display for Normal Operation 27 Table 11 Specification Data Sheet 185200 76 Table 12 Power Consumption of Chassis Parts 80 Table 13 InfiniBand QSFP Connector Pinout 81 Table 14 Replacement Parts Ordering Numbers 83 Mellanox Technologies 7 Rev 1 8 Revision History Table 1 Revision History of this User s Manual Revision Date Details 1 8 Sept Added figure 26 2012 Added information regarding inserting the bottom spine first Insert spines from bottom to top Fixed broken links Added step to remove plastic covering Updated power numbers Changed style of specifications data sheet Added lift tool Added Section 5 3 1 7 Feb Modified Section 7 to point to the SW UMs 2012 1 6 Aug Changed name to Hardware User Manual 2011 Fixed parts list for installation kit Fixed leaf and spine shut down commands Fixed Max heat output Feature list modified 1 5 May Removed InfiniBand Fabric Initialization and Management 2011 Updated Fabric Management Software Section 1 4 Nov Added last page wit
10. 7 Replace the spine board 8 5 FabriclT Software For more detailed instructions concerning Mellanox OS software see the Mellanox MLNX OS SwitchX Software WebUI User s Manual or the Mellanox MLNX OS SwitchX Software User Manual Mellanox Technologies 75 Table 11 Specification Data Sheet IS5200 Rev 1 8 Appendix A Specification Data Physical Size Mounting Size Weight 15 75 x 19 x 27 inches 400 mm x 482 6mm x 733 mm The shelf adds 44 5mm to the height 120 kg 265 LBS full configura tion 48 kg 105 LBS empty configu ration Mounting Center of Grav ity Measured from the bottom left 19 Rack mount CoGh 311 8 mm from bottom of chas sis a 83 94 kg 185 LBS shipped con cornerasone CoGw un figuration faces the switch 222 mm from left side surface Gravity poni CoGd 328 14 mm from port end sur face Measured from the bottom left corner as one faces the switch ports Air Flow Max Air Flow Max Air Flow Through leafs 16 59M3 min 585 9 CFM Through spines 9 48 M3 min 334 8 CFM SerDes Speeds Connector SerDes Speeds ne DES 10 20 or 40Gb s per port Types QSFP Sound Level Sound Level Ports Ports 76 db A 216 ports Power and Environmental Input Voltage Input Voltage Management Management 100 240 VAC 50 60Hz CPU PowerPC 460EX Temperature Operating 0 to 45 Celsius Non operating 40 to 70 Celsius
11. 990 1 Ir 4 leaf fan modules 12 leaf modules ez 2 management modules 14 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 1 2 Features List 216 QDR 40Gb s InfiniBand ports in a 9U switch IBTA 1 21 compliant SDR DDR QDR link speed Congestion control l Adaptive routing Port mirroring Chassis High Availability e sMB High Availability 1 Features for a future release Mellanox Technologies 15 Rev 1 8 Hardware 2 Hardware 2 1 Installation Safety Warnings These safety warnings are in English For French Spanish and German see the Appendixes 1 Installation Instructions LA Read all installation instructions before connecting the equipment to the power source 2 Bodily Injury Due to Weight AR Use enough people to safely lift this product lt 40 lbs 40 70 lbs 70 121 lbs gt 121 lbs lt 18 kgs 18 32 kgs 32 55 kgs gt 55 kgs 3 Installation in Restricted Access Location LA This unit is intended for installation in a Restricted Access Location 4 Over temperature ing the maximum recommended 45 C 113 F Moreover to guarantee proper air This equipment should not be operated in an area with an ambient temperature exceed A flow allow at least 8cm 3 inches of clearance around the ventilation openings 5 Stacking the Chassis The chassis should not be stacked on any other equipment If the chassis falls it ca
12. BST PSU SFANS L TI Tm 2 6 1 12C y N This interface is for Debug and Troubleshooting only This interface is for FAEs only 28 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 2 6 2 CONSOLE The CONSOLE port is used during the installation process to configure the chassis for remote management Connect this port to a local host using the harness supplied with the chassis See the Installation Guide for the initial configuration procedure 2 6 3 MGT Management The MGT port is an Ethernet port for remote management Any remote terminal connected to the Ethernet port can then be used to manage the fabric and chassis Each Ethernet connector gets connected to Ethernet switches These switches must be configured to 100M auto negotiation eu Initial configuration must be done on all of the management modules The first man agement module you configure will be the master 2 2 6 4 USB The USB port can be used to upload new SW using any storage device that has a USB connector This interface is USB 2 0 2 6 5 Reset RST The Reset button resets the chassis management module when the button is pushed When the button is held down for 15 seconds the management module is reset and the password deleted DO NOT use a sharp pointed object such as needle or push pin for pressing the Reset button Sharp objects can cause damage use a flat object such as a paper clip Figur
13. Cable power QSFP MAX 2W Humidity 5 95 non condensing Humidity TYP 1 5W Operating 76 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Total Power Consumption Typical QDR Passive cables 1643 W Typical QDR Active cables 2136W including QSFP at 1 5W Max QDR Passive cables 2156W Max QDR Active cables Power Con sumption Leaf Typical Passive cables 70W not including QSFP power Typical Active cables 105W including QSFP at 1 5W Max Passive cables 84W not including QSFP power Max Active cables 124W Pow eena MDS 2715W including QSFP at 2 0W including QSFP at 2W Power Con Max Heat Out sumption Spine Typical 87W include 2 Fans put 9264BTUs hr Max 108W include 2 Fans Power Con sumption for Active cables QSFP 2 0W MAX Power level 2 Protocol Support InfiniBand Management Auto Negotiation of 40Gb s Managed using MLNX OS Speed protocol 20Gb s 10Gb s Baseboard Performance and Management Device management Agents for full InfiniBand In Band Manage ment Data Rate QoS Data Rate QoS 8 InfiniBand Virtual Lanes for all 40Gb s per port ports Regulatory Compliance Safety EMC US Canada cTUVus certificate Emissions CE EMC amp safety report and cer International CB report and tificate certificate USA FCC part 15 class A Russia GOST R certificate Canada ICES 003 class A Australia New
14. EFM is updated and the FW image in the leafs and spines of the chassis is 72 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 8 TroubleShooting 8 1 Power Supply Unit d Ifthe Power supplies cannot supply enough power the management module may shut down the system gt As each PSU is plugged in make sure that the green power LEDs on the PSU comes on If the AC power LED is off Check that the power cable is the correct power cable for your country Check that the power cable is plugged into a working outlet Check that the power cable has a voltage within the range of 100 240 volts AC Remove and reinstall the power cable Check the circuit breakers to be sure that the breaker has not tripped Check that the power cable is good Replace the power cable If the AC power LED is green but the OK power LED is off or the FAIL LED is on Replace the PSU SO DAS cO uA D ES 8 2 Leaf Board The power LED for the Leaf board is off 1 Make sure that all of the PSUs are showing DC OK 2 Uninstall and reinstall the Leaf board 3 When the Yellow LED is on this indicates a fault in the board uninstall and reinstall the Leaf board 4 If uninstalling and reinstalling the Leaf board does not work burn the latest FW on the Leaf board and uninstall and reinstall the Leaf board 5 Replace the Leaf board with a new one d Should any ofthe boards shut down due to over temperat
15. Eti a pede ur QUERER need 59 5 2 1 Extracting a Leaf Board 59 522 inserting a Eeat Board E cese Ne enr a oe nate esta ae etna e etd 60 33 Spine Boards 24 4 0 er E REA le eden edendi Mee d I Sede 61 5 3 1 Extracting a Spine Board 61 5 3 2 Ins rtiriga Spine Board riisus 4 ee amd Ne ee as 63 54 Fan Modules uo a la an 64 5 4 1 Leaf Fan Module ut encomedasrreroiaoda oth sea art en pica 64 5 4 2 4 Spine Fan Modules 65 55 Management Mod le ssy Biu 4 042 tag A E A dern ne 66 5 5 1 Extracting a Management Module 66 5 5 2 Inserting a Management Module 67 5 6 Switch Shut Down Procedures 68 Chapter 6 Disassembly and Disposal 69 6 1 Disassembling the Chassis 69 6 1 1 Removing the Chassis 2 24 ur da does sas ter sarah seine 69 6 1 2 Removing the Bottom Shelf 69 62 Disposals 2 222232 Hite e do a bo 70 Chapter 7 Switch Management Tools 71 7 1 InfiniBand Subnet Manager 72 7 2 Accessing the CPU via the Ethernet Connector 72 73 Uperading Software
16. Fig re 2 License Key Label ran nr ae A Heap 12 Figure 3 Product Label Tab Location 12 Figure 4 Management Module MAC Address Location 12 Figure 5 185200 Switch Views 44 44 444 ce been eben nenne esse 14 Figure 6 Power Supply Unit Status Indications 21 Figure 7 PSU Cover On and Off 22 Figure 8 Leaf Board Led Indicators 23 Fig re 9 Spine status LEDS runter 24 Figure 10 Spine Side Panel Display Status Indications 25 Figure 11 Management Module Status Indications 26 Figure 12 Port Numbering uer mes rte Ruben nat in waren a 27 Figure 13 Top and Bottom Ports 28 Figure 14 Management Module Interfaces 28 Figure 15 Reset Button ci ee N ee 29 Figure 16 Shelf Installation Kit Parts 35 Figure 17 Chassis Installation Parts 36 Figure 18 Shock and Tilt Stickers 37 Figure 19 Inserting the Caged Nuts for the Shelf 39 Figure 20 Connect Rail Slide to Rack Vertical suppo
17. and leafs Table 12 Power Consumption of Chassis Parts Part Typ Power Passive Cables Typ Power Active Cables 2w Per Port IB QDR leaf 82 28W 121 45 Spine 103W Not applicable Fan 117 Not applicable Management Module 20 Not applicable To calculate the chassis power 103W of spines 20W of management modules 14W of fans 82 28W of leafs for passive cables or 121 45W of leafs for active cables 0 85 Where 0 85 is the power supply efficiency 80 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Appendix D QSFP Interface Table 13 InfiniBand QSFP Connector Pinout Connector Connecto Pin r Pin Signal Description Number Name 1 GND Ground 2 Tx2n Transmitter Inverted Data Input J3 and J6 i Tx2p Transmitter Non Inverted Data Input ND GND I 4 GND Ground 21 pron Rxin 18 22 cs EM 17 gt Tx4n Transmitter Inverted Data Input 23 GND GND L 18 6 Tx4p Transmitter Non Inverted Data Rx4n Rx3n 12 _ ape 25 Rx4p Rx3p 14 7 GND Ground 26 GND 1 8 ModSelL Module Select 27 yodPrsL spa 2 9 ResetL Module Reset im sch 10 Vcc Rx 3 3 V Power supply receiver 39 veetx vcc Rx 11 SCL 2 wire serial interface clock Vect ResetL_ 12 SDA 2 wire serial interface data
18. by the end prod uct manufacturer and or the end user Mellanox Technologies 17 Rev 1 8 Hardware 13 Equipment Disposal h Disposal of this equipment should be in accordance to all national laws and regula tions 14 Local and National Electrical Codes This equipment should be installed in compliance with local and national electrical LA codes 15 UL Listed and CSA Certified Power Supply Cord Leakage gt 3 5mA For North American power connection select a power supply cord that is UL Listed and CSA Certified 3 conductor 16 AWG terminated with a molded plug rated at YN 125 V 13 A with a minimum length of 1 5m six feet but no longer than 4 5m For European connection select a power supply cord that is internationally harmonized and marked lt HAR gt 3 conductor minimum 1 0 mm wire rated at 300 V with a PVC insulated jacket The cord must have a molded plug rated at 250 V 10 A 16 High Leakage Current LA Warning High leakage current Earth connection essential before connecting supply 17 Add GND connection information Before connecting this device to the power line the protective earth terminal screws of this device must be connected to the protective earth in the building installation LA GND Connection Information The building installation shall provide a means for a connection to protective earth and the equipment shall be permanently connected to that by a service person A SERV
19. chassis remove the cover to the power supply unit for the non work ing PSU Odd numbered PSUs are on the left side and even numbered PSUs are on the right side facing the spines There are four phillips head screws for the cover plate Figure 41 Power Supply Handle CB ince AC Latch Um DO 4 Grasping the handle with one hand push the black latch release while pulling the handle out ward As the PSU unseats the PSU status indicators will turn off 5 Remove the PSU To insert a PSU 1 Make sure the mating connector of the new unit is free of any dirt and or obstacles 2 Insert the PSU by sliding it into the opening until a slight resistance is felt 3 Continue pressing the PSU until it seats completely The latch will snap into place confirming the proper installation 58 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 4 Insert the power cord into the supply connector on the other side of the chassis 5 Replace the cover over the PSUs The green indicators should light If not extract the PSU and re insert it again 5 2 Leaf Boards Do NOT mix replacement parts based on different generations of chip Do not install InfiniScale IV based replacement parts within a SwitchX based chassis and vice versa All Replacement modules must be consistent with the Chassis family be and switch chip
20. doit tre install dans le respect des codes lectriques locaux et nationaux gt 16 Codes d installation L appareil doit tre install selon l ancienne version des codes lectriques nationaux du pays Pour l Am rique du Nord l quipement doit tre install conform ment aux sp cifications du Code Electrique National Am ricain et du Code Electrique Canadien 17 Interconnexion des unites Les c bles de branchement l unit RS232 et les interfaces Ethernet doivent tre certi fi s UL de type DP 1 ou DP 2 Note lorsqu il existe dans un circuit non LPS Protection contre la surintensit Un appareil de protection r pertori facilement accessible contre la surintensit du circuit de branchement et calibr a 20A doit tre incorpor dans le c blage lectrique du b timent gt 18 Exposition au rayonnement grave Mise en garde l utilisation de commandes ou de r glages ou l ex cution de proc dures autres que ce qui est sp cifi dans les pr sentes peut engendrer une exposition au rayonnement grave PRODUIT LASER DE CLASSE 1 et r f rences aux normes laser les plus r centes CEI 60 825 1 1993 A1 1997 A2 2001 et NE 60825 1 1994 A1 1996 A2 2001 Mellanox Technologies 87 Rev 1 8 19 S assurer que les enceintes sont appropri es Des enceintes lectriques m caniques et incendie adapt es doivent tre fournies par le fabricant du produit final ou par l utilisate
21. highly recommended to have a screw gun or electric screwdriver for this step Unscrew the sides of the crate Remove and put aside the box containing the cables Remove and put aside the box containing the cable management kit Remove and put aside the box containing the installation kit AO Gee US Visually inspect the chassis make sure that there is no visible damage all 6 spines are installed 4PSUs are installed e all 4 fans are installed management module is installed 8 Remove all protective plastic film from all sides and top of the chassis 9 If you are not using a mechanical lift to install the chassis reduce the weight of the chassis by removing all of the spines power supply units management modules and fan units and put aside for reinstallation after the chassis 1s installed in the rack 3 5 2 5 Installing the Shelf 1 Place the ESD mat on the floor where you will be working and put on the ESD strap Make sure the ESD strap is touching your skin and that the other end is connected to a verified ground 2 Clip 4 caged nuts into the holes in the rack yo u will be using to connect the shelf brackets Check that both sides of the shelf are at the same level in the rack 38 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Figure 19 Inserting the Caged Nuts for the Shelf Top of shelf 2 Bottom of shelf TT X Do not use this hole 19 3 Using two o
22. latch will engage Mellanox Technologies 63 Rev 1 8 Insertion and or Hot swap Extraction 5 4 Fan Modules There are four fan modules on the chassis for the leafs They are located on the spine side along the top of the chassis When a fan module is not functioning the status LED on the fan will light up Each spine has an individual fan module that contains two individual fans Should a single fan fail the Fan Status LED on the spine and the S Fan LED on the management module will light indicating the necessity to replace the fan module Air flow through the spines is independent of the air flow through the leafs Figure 48 Fan Locations on the Chassis A Fans AM ES se ee iN TIAE p eel Milos ele e ee e l Fan 4 Fan 1 Fan 3 5 4 1 Leaf Fan Module There are 4 Leaf fan modules found on the Spine side 5 4 1 1 Extracting the Leaf Fan Module 1 Push and hold the blue latch release See Figure 49 2 Slowly pull out the fan module using the handle 64 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Figure 49 Leaf Fan Module Extraction Handle Latch 5 4 1 2 Inserting the Leaf Fan Module 1 Make sure the fan module is oriented correctly top side up Confirm that the location of the connector in the chassis will line up with the connector in the fan module 2 Slowly slide in the new leaf fan module Ifthe fan module stops before it goes in all ofthe w
23. one fail Connecting the PSUs to different AC lines pro vides AC failover protection The system should continue to run and allow a hot swap of a defective PSU Should there not be enough power to keep all of the leafs running FabricIT EFM may power down some leafs If this happens it will be necessary to reboot the chassis once the defective PSU has been replaced 1 Check all FRUs for proper insertion and seating before connecting the AC power cords Boards Power supplies e Leaf fan modules Spine fan modules Insert all leafs that you plan to use in the chassis Insert all spines in the chassis Insert thermal blanks in unused leaf slots to maintain balanced air flow Tighten all leaf and spine mounting screws Connect the power cords to the PSUs 2 3 4 3 6 7 Connect the power cords to grounded electrical outlets With N 1 PSU redundancy do not power up the chassis with less than all PSUs installed 4 1 Power Supply and Spine Board Indicator Status at Power ON It can take up to 5 minutes to boot up the system Turn off the system if any LEDs remain red for more than 5 minutes As the power is turned on you should observe the following conditions for normal operation 1 Power Supply Unit s AC OK and DC OK indicators are ON and FAIL indicators are OFF Mellanox Technologies 55 Rev 1 8 Chassis Power Up 2 There is a green Status LED per spine board per leaf board and per management modu
24. 09 Mellanox Technologies 83 Rev 1 8 Table 14 Replacement Parts Ordering Numbers Continued Part Description OPN Power cord Type I for China ACC000210 Power cord Type J for Switzerland ACC000211 Power cord Type B for Japan ACC000212 Power cord Type I for Australia ACC000213 84 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Appendix F Avertissements de s curit d installation French 1 Instructions d installation Lisez toutes les instructions d installation avant de brancher le mat riel la source d alimentation lectrique 2 Temp rature excessive Ce mat riel ne doit pas fonctionner dans une zone avec une temp rature ambiante d passant le maximum recommand de 45 C 113 F Un flux d air de 200LFM cette temp rature ambiante maximale est n cessaire En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouver tures de ventilation 3 Empilage du ch ssis Le ch ssis ne doit pas tre empil sur un autre mat riel Si le ch ssis tombe il peut pro voquer des blessures corporelles et des d gradations de biens 4 Connection d Alimentation electrique excedentaire dangers lectriques Ce produit comporte un couvercle transparent sur l espace pour l alimentation lec trique redondante Ne pas faire fonctionner le produit si le couvercl
25. 3 4 Spine Side Panel Display LED Indicators The spine side panel display has LEDs that show the chassis condition Figure 10 Spine Side Panel Display Status Indications E A E E g O STATUS PSU SPINE LEAF MNG 1 MNG 2 STATUS FANS FANS STATUS STATUS Mellanox Technologies 25 Rev 1 8 Hardware Table 9 LEDs Display for Normal Operation a I s Normal LED Condition Description State STATUS Off No Power Green Green System is up and running Yellow System warning Attention needed such as overheating Red System not operational Diagnostics fail CPU hang HW fail Overheat critical Blinking green System booting Restore factory defaults in progress PSU STATUS Off No power Green Green Normal operational Red PS fault detected User should check individual power sup plies for fault indications SPINE FANS Off No power to fan Green STATUS Green Nominal operational Red One or more of the spine fans is bad User should check individual spine fan LEDs for fault indications LEAF FANS Off No power to fan Green STATUS Green Nominal operational Red One or more of the leaf fans is bad User should check individual leaf fan LEDs for fault indications MNGI Off Green MASTER no power STATUS this management module is not installed this management module is not the master Green Management module is operating as a master MNG2 Off Off
26. 6 1 Powering Up the Switch Platform Make sure that the power cords are compatible with your outlets Power cords for dif ferent countries can be ordered from Mellanox gt The chassis in N 1 mode must be started with a full complement of PSUs thereafter it can run on one less than the total number of PSUs This final PSU is redundant and allows for hot swap ping a PSU should one fail Connecting the PSUs to different AC lines provides AC failover pro tection When using the 1600W PSUs the minimum complement of power supply units to start the chas sis is and the chassis will continue to run at full capacity with only PSUs Mellanox Technologies 51 Rev 1 8 Installation The system should continue to run and allow a hot swap of a defective PSU Should there not be enough power to keep all of the leafs running FabricIT EFM may power down some leafs If this happens it may be necessary to reboot the chassis once the defective PSU has been replaced Two simple ways to reboot is to use the reboot command in the CLI or reboot through the Web GUI The power system will divide the current consumption by the number of working PSUs Should one of the PSUs fail the total current consumption will then be divided by the remaining work ing PSUs When the failed PSU is hot swapped the new PSU will ramp up and pass its share of current so that the total current is always divided by the number of working PSUs 1 Plug in the power cords to th
27. 9 31 33 35 BB BE RUBER et YYYYYYIIY 090020000 00008 0008000008 If you need to hot swap spine 1 or spine 2 1 Check to see if the spine you need to remove is connected to the master management module See Figure 45 Warning If you have only one management module the chassis will crash 2 Push outward on the latches to unlock the ejectors from the chassis 3 Open the ejectors until they are 45 degrees from the module 4 Pull out on the module using both ejectors 5 3 1 2 Extracting Spine Boards Except 1 or 2 1 Run the shut down command no power enable lt module gt For example to shut down spine 06 run the command below switch 1 Orca648 master config no power enable S06 2 Push outward on the latches to unlock the ejectors from the chassis 3 Open the ejectors until they are 45 degrees from the module 4 Pull out on the module using both ejectors 5 3 2 Inserting a Spine Board 1 Start with the latches fully open that is 45 degrees to the front panel of the leaf 2 Hold the spine board by the sides with both latches fully open 3 Insert the board into its rail The board should slide in without resistance 4 Push the board in until the hooks on the latches are touching the vertical bar 5 Now press both latches inward simultaneously You should feel the latches are hooked into the front panel 6 Press both latches applying some force to close them completely The locking
28. Ds Fan status LED and Spine status LED in the spines are all green A N The maximum number of connections from each leaf is 3If the number of leafs x the maximum number of connections per leaf is less than 36 then some of the leafto spine y connection LEDs may be OFF Figure 36 Spine Module 19 21 2325 27 29 31 33 35 DO0D000000 DOOODODO 000 20 22 24 26 28 30 32 34 36 Status LED Leaf connection LEDs 3 7 InfiniBand QSFP Cable Installation The switch uses industry standard QSFP InfiniBand cables which are available from several ven dors The standard QSFP cables support full 40 40Gb s QDR 20 20Gb s DDR and 10 10Gb s SDR bidirectional wire speed of the switch ports All InfiniBand QSFP connections are made to the leaf boards Each leaf has 18 InfiniBand QSFP connectors in two rows which are numbered 1 18 See Section 2 3 6 for port numbering If maximum cable lengths are exceeded data transfer will be reduced and the bit error rate will increase As All cables can be inserted or removed with the unit powered on To insert a cable press the con nector onto the port receptacle until the connector is firmly seated The orange LED indicator above the port will light when the physical connection is established when both ends of the cable are properly connected to working devices Allow 15 seconds for link to get up To remove dis engage the lock and slowly pull the connector away from the port recept
29. Each management module has a label with its MAC address See below for the location of this label Figure 4 Management Module MAC Address Location CONSOLE MGT gt rst PSU S FANS L FANS MASTER T Tr 30 1 2009028 MAC address location 12 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 1 1 4 Product Physical Specifications and Power Table 3 Switch Rack Mechanical and Environmental Requirements Fully Populated Chassis Rack Rack of leafs Ambient Height Rack Width Weight Max Power 216 x QSFP Ports E Depth A Temp Required of Spines 10 U 19 25 7 12 6 120 kg 265 LBS Typical 2553 59W including Max 45 C 444mm EIA 310 685 mm Full configuration QSFP at 2W Min 10 C including 482 6 mm Typical 2337 59W including bottom 48 kg 105 LBS QSFP at 1W shelf Empty configuration Max 2815 11W including QSFP at 2W 83 94 kg 185 LBS Up to 4 Power Supplies of 100 Shipped configura 240VAC 50 60Hz 6 3 13A tion The switch ships in a minimum base configuration plus additional modules depending on the chosen customer configuration Optional modules included Leaf boards Management modules Figure 5 shows the connector and spine sides fully populated Mellanox Technologies 13 Rev 1 8 Overview Figure 5 185200 Switch Views Leaf Side 4 power supply units 6 spine modules SM ponui y Fan DODODDDDD
30. ICE PERSON shall check whether or not the socket outlet from which the equipment is to be powered provides a connection to the building protective earth If not the SERVICE PERSON shall arrange for the installation of aPROTECTIVE EARTHING CONDUCTOR from the separate protective earthing terminal to the pro tective earth wire in the building The equipment shall be installed in area where equi potential bonding exists such as a telecommunication centre or a dedicated computer room 18 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 18 Installation codes This device must be installed according to the latest version of the country national electrical codes For North America equipment must be installed in accordance to the applicable requirements in the US National Electrical Code and the Canadian Electri cal Code gt 19 Interconnection of units Cables for connecting to the unit RS232 and Ethernet Interfaces must be UL certified type DP 1 or DP 2 Note when residing in non LPS circuit Overcurrent Protection A readily accessible Listed branch circuit overcurrent protec tive device rated 20 A must be incorporated in the building wiring h 20 Hazardous Radiation Exposure Caution Use of controls or adjustment or performance of procedures other than those specified herein may result in hazardous radiation exposure CLASS 1 LASER PRODUCT and reference to the most recent lase
31. MAGE Mellanox TECHNOLOGIES Mellanox Technologies Mellanox Technologies Ltd 350 Oakmead Parkway Suite 100 Beit Mellanox Sunnyvale CA 94085 PO Box 586 Yokneam 20692 U S A Israel www mellanox com www mellanox com Tel 408 970 3400 Tel 972 0 74 723 7200 Fax 408 970 3403 Fax 972 0 4 959 3245 Copyright 2012 Mellanox Technologies All Rights Reserved Mellanox Mellanox logo BridgeX ConnectX CORE Direct InfiniBridge InfiniHost InfiniScale PhyX SwitchX Virtual Protocol Interconnect and Voltaire are registered trademarks of Mellanox Technologies Ltd Connect IB FabricIT MLNX OS ScalableHPC Unbreakable Link UFM and Unified Fabric Manager are trademarks of Mellanox Technologies Ltd All other trademarks are property of their respective owners 2 Mellanox Technologies Document Number 3241 QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Table of Contents Table of Contents 3 List of Figures 5 List of Tables 7 Revision History 8 About this Manual 9 Intended Audience u nn ner es een raa raa dened eee 9 Related Documentation es ut ee ee ern nee UR 9 NA nee E in a nenne 9 Chapter 1 Overview 11 1 1 Product Information 11 1 1 1 Serial Number and Product Version Information 11 11 24 License Key gli dns drame ee ne eee ee sure ONE ne 12 1 43 Manag
32. MASTER no power STATUS this management module is not installed this management module is not the master Green Management module is operating as a master 2 3 5 Management Module LED Indicators Figure 11 Management Module Status Indications MAA A 26 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 The management module LEDs display the switch system operating conditions Table 10 LEDs Display for Normal Operation sas Are Normal LED Condition Description State STATUS Off No Power Green This LED Green System is up and running shows the sta Yellow System warning Attention needed such as overheat tus of the chas ing sis Red System not operational Diagnostics fail CPU hang HW fail Overheat critical Blinking green System booting Restore factory defaults in progress PSU STATUS Off No power Green Green Normal operational Red PS fault detected User should check individual power sup plies for fault indications SPINE FANS Off No power to fan Green STATUS Green Nominal operational Red One or more of the spine fans is bad User should check individual spine fan LEDs for fault indications LEAF FANS Off No power to fan Green STATUS Green Nominal operational Red One or more of the leaf fans is bad User should check individual leaf fan LEDs for fault indications MASTER Off this m
33. Mellanox TECHNOLOGIES 216 Port InfiniBand QDR Switch Platform Hardware User Manual PN MIS5200Q 4DNC Rev 1 8 www mellanox com Rev 1 8 NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO S AND OR THE SYSTEM USING IT THEREFORE MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT INDIRECT SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES OF ANY KIND INCLUDING BUT NOT LIMITED TO PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY FROM THE USE OF THE PRODUCT S AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DA
34. Slide the rail slides to the vertical rails 48 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Figure 31 1S5200 Chassis on the Shelf LI Fan 1 a ele ole cu ole Se Fe Fan 3 an 4 fe 185200 PTS it PS2 1 0 0 it PS4 Mellanox Technologies 49 Rev 1 8 Installation 3 5 5 Figure 33 Move the Rail Slide to the Vertical Support 2 E p al f NS 11 Place the 4 holed spacer on the outside of the vertical support and screw in the 4 screws for each rail slide Use the split lock washers here 12 Remove the two handles from the spine side and save all four handles for future use 13 Ground the switch 14 Replace all of the spines power supply units fan units and management modules removed at the start of the installation procedure 15 Plug in the power cables 16 Check the Status LEDs and confirm that all of the LEDs show status lights consistent with normal operation Warning Any yellow status LEDs is cause for concern and must be dealt with imme diately LA It can take up to 5 minutes to boot up during which time the status LED may indicate flashing green 17 You can start connecting all of the cables to the switch Ground Connections Make sure to connect the ground post to a valid electrical ground Use a g rounding lug and a ground wire of sufficient capacity to safely convey a potential discharge A ground wire
35. Start with the latch fully open that is 45 degrees to the front panel of the leaf Carefully set the leaf board into the side guide rails and slowly slide the board into the chassis Catch the hooks onto the vertical bar and push the latch shut 60 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 5 3 Spine Boards Do NOT mix replacement parts based on different generations of chip Do not install InfiniScale IV based replacement parts within a SwitchX based chassis and vice versa All Replacement modules must be consistent with the Chassis family and switch chip generation When hot swapping any of the units it is necessary to wait 1 minute after removing the defective part before inserting the new part This is necessary so that the manage ment module will start a new cycle checking through the leafs and spines for the FW versions Each spine has a pair of ejectors that lock the board in place and serve as levers for seating or extracting see Figure 46 Management board 1 is connected to spine board 1 and management board 2 is connected to spine board 2 When a slave management board is not installed or not working the spine board connected to the master management board cannot be hot swapped All of the spine boards can be hot swapped when two management boards installed and working When more than one spine slot is empty always insert the lowest spine board first then work your wa
36. Status LED Table 6 shows the spine status according to the LED condition Table 6 Spine Status LED LED Condition LED Description Off No power to the spine Solid Green Spine is up and running Flashing Green Spine is powering up Solid Orange Non fatal error this spine needs troubleshooting but does not require chassis shutdown 24 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Table 6 Spine Status LED LED Condition LED Description Solid Red Fatal error 2 3 3 2 Fan LED The spine fan indicator LED has the following LED assignment Table 7 Spine Fan Status LED LED Color LED Description Solid Green Spine Fan is OK Solid Yellow One or more of the fans in this spine is not working Each spine has two fans in the fan module 2 3 3 3 Spine to Leaf IB Connection Status LEDs The leaf connection status on each spine displays the condition ofthe connection between the spine and each leaf There is a minimum of one LED per leaf per spine and a maximum of 3 LEDs per leaf These LEDs indicate a valid connection between a leaf and a spine Table 8 shows the leaf to spine status according to the LED condition Table 8 Spine to Leaf IB Link Status LED Condition LED Description Off Link is down Solid Green Logical connection Flashing Green Data activity Solid Orange Physical connection 2
37. U for the chassis and one U for the shelf Figure 16 Shelf Installation Kit Parts Angled x Bracket Nm cad N s Rail N Rail Slide M6 X 12 Caged nuts Z Vv M6 X 16 bolts ON Mellanox Technologies 35 Rev 1 8 Installation Figure 17 Chassis Installation Parts Rail Slide Pan head i ml 4 screws Ed 00000000 Washer A the handles SAA Screws for 3 5 2 3 Container Mishandling The container has shock and tilt stickers applied These will turn red if the container has been mishandled or roughly handled Upon receipt of the container look for and inspect the shock and tilt stickers to confirm that they have not tripped If one or more are red notify the shipper and Mellanox This on its own does not indicate damage to the contents But be sure to carefully inspect the contents if any of the shock and tilt stickers have tripped 36 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Figure 18 Shock and Tilt Stickers a Stickers unaffected by shipping 3 5 2 4 Opening the Container Stickers showing excessive shock or tilt 1 Before starting the procedure put the ESD strap on and connect it to a valid ground Mellanox Technologies 37 Rev 1 8 Installation 2 Open the crate by unscrewing the sides and top of the crate Container screws Y It is
38. Zealand i ZS CISPR 22 class A Safety EMC BSMI certificate for the power Aa S CIS class Emissions supply only Japan VCCI Class A Australia C Tick certificate EN 55022 2006 A1 07 ClassA EN 61000 3 2 2006 A1 09 A2 09 EN 61000 3 3 2008 EN 55024 1998 A1 01 A2 03 aaa Environmental Vibration i qi EU IEC 60068 2 64 Random EU IEC 60068 2 29 Shocks Vibration Type I II EU IEC 60068 2 32 Fall Test Mellanox Technologies 77 Rev 1 8 A 1 A 2 A 3 Acoustic Acoustic 76 DbA Shock and Shock and ISO 7779 Vibration ETSI EN 300 019 2 2 1999 09 Vibration ETS 300 753 Scalability and Performance Leafs Power Supplies Up to 12 leafs 18 ports each 4 Power Supplies standard Spines 6 spine boards Management All spines are needed for non Modules 2 available 1 required for opera blocking configuration tion Switching Addressing Leaf fans 4 SOO nen 48K Unicast Addresses Max per capacity Subnet 16K Multicast Addresses per Subnet Switching Switching Capacity 17 3Tb s Performance Simultaneous wire speed any port to any port EMI Certification EMI certification on the fully populated chassis was performed with the chassis installed in a closed two door rack using the chassis installation kit supplied by Mellanox Technologies Approved Cables For a list of all approved cables see http www mellanox com related docs user manuals Mellanox approved cables pdf
39. acle AN A 1 For a valid physical connection both ends of the cable must be connected to working N devices Take care to not impede the air flow through the ventilation holes next to the Infini Band ports Use cable lengths which allow routing horizontally around to the side of the chassis before bending upward or downward in the rack Mellanox Technologies 53 Rev 1 8 Installation 3 7 4 Supported Approved Cables For a list of approved cables for this switch see the Mellanox approved cable list http www mellanox com related docs user manuals Mellanox approved cables pdf 3 7 2 Cable Power Classes Chassis and switches need to be able to dissipate the heat generated by high power I O cables and modules The Mellanox IS5X00 se ries chassis are rated for cables up to class 2 as per the SFF committee classification SFF 8436 PDF See http www mellanox com content pages php pg cables for the cable class rating of Mella nox cables 54 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 4 Chassis Power Up Before starting any procedure on theIS5200 system put an ESD prevention wrist strap on your wrist and connect to the IS5200 chassis With N 1 PSU redundancy the chassis must be started with a full complement of possible PSUs thereafter it can run on one less than the total number of PSUs This final PSU is redundant and allows for hot swapping a PSU should
40. anagement module is not the master MM Green Green Management module is operating as a master 1 e the other management module is a slave 2 3 6 Port Connector Interfaces The connector side of the switch has 12 leaf boards and each leaf board has 18 QSFP ports The ports on each leaf board are placed in two rows 9 ports to a row The ports are labelled as shown in Figure 12 The bottom row ports are flipped from the top row See Figure 13 Figure 12 Port Numbering 1 3 5 7 9 11 18 19 17 RES A ee ee ear A SSS TA E SEO O posee 2 4 6 8 10 12 14 16 1 8 Mellanox Technologies 27 Rev 1 8 Hardware Figure 13 Top and Bottom Ports A E 2 4 Air Flow These switches come with the air fl ow pattern of air entering through the spine side and exiting through the connector side 2 5 QSFP Cable Power Budget Classification All IS5200 QSFP switches are designed for active cables with a max power per module of 2 0W This is power level 2 according to the QSFP Public Specification 2 6 Management Module Interfaces The switch system requires at least one management module The management module has five interfaces to connect to the IS5200 They are 112C port CONSOLE port this is an RS232 connector for connecting to a host machine 1 MGT this is an Ethernet connector e 1 USB port e RST reset button Figure 14 Management Module Interfaces CONSOLE MGT AR
41. ations 31 3 2 Ch ssis Package Contents Tas tep a ds Gada eae e INR ER ERES 32 33 Leaf Package Contents sereen inian net ip PRA Ete HE ERE ERIS REIR AS eda 33 34 Management Package Contents 33 3 5 Physical Installation cocoa ae ea ec ea 33 ISL ESD Connection a 2s eae vais sen sd cede toe sepia as alba 34 3 5 2 Installation Procedure 34 3 5 3 Installing the Cable Holder 40 3 5 4 Installing the Chassis ecc sae RU ERREUR dd d 44 315 5 Ground Connections cod eR eng ent else S esters Eos 50 3 6 Power Connections ooo Ree UE cie n hae eR UP bee Mete 51 3 61 Powering Up the Switch Platform 51 3 7 InfiniBand QSFP Cable Installation 53 3 7 1 Supported Approved Cables 54 3 12 Cable Power Classes e nenne 54 Chapter 4 Chassis Power Up 55 Mellanox Technologies 3 Rev 1 8 4 1 Power Supply and Spine Board Indicator Status at Power ON 55 Chapter 5 Insertion and or Hot swap Extraction 57 5 1 Power Supply Unitss 4o oth ste Lud e tps re eM ME ue 57 5 1 1 Extracting and Inserting the Power Supply Unit 57 5 2 Leaf Boards s reed eru a Vh
42. ay it is inserted incorrectly or it may be for the other side of the chassis 3 Push the fan module until the latch engages 4 Make sure that the green leaf LED on the module comes on indicating that fan is running 5 4 2 Spine Fan Modules Each spine module has a fan module with two individual fans built in When a fan module is not functioning the Fan Status LED on the spine will light up Figure 50 Spine Fan Module 19 21 2325 2729 31 33 35 ODODODODO BB N BN NL OOO 20 22 242628 30 32 34 36 Fan Module When a fan module is removed the indicator light will reset Mellanox Technologies 65 Rev 1 8 Insertion and or Hot swap Extraction Figure 51 Fan Status LED on the Spine Module 5 4 2 1 Extracting the Spine Fan Module 1 Push the two blue latch buttons together while pulling the fan module out 5 4 2 2 Inserting the Spine Fan Module 1 Make sure the fan module is oriented correctly top side up Confirm that the location of the connector in the chassis will line up with the connector in the fan module 2 Slowly slide in the new spine fan module 3 Push the fan module as far as it will go make sure the locking latches engage If the Fan LED continues to show red remove the fan module and check the pins on the connector inside if the spine to make sure that none of them are bent 5 5 Management Module When hot swapping any of the units it is necessary to wait 1 minute after re
43. cables See Figure 27 on page 46 Mellanox Technologies 45 Rev 1 8 Installation Figure 27 Screw the Handles Onto the Chassis Handles Figure 28 The Rails are Already Connected Onto the Chassis 46 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 When lifting e use a legs bend your knees and not your back lt 40 Ibs 40 70 lbs 70 121 lbs gt 121 lbs lt 18 kgs 18 32 kgs 32 55 kgs gt 55 kgs 3 Lift the chassis and slide it onto the shelf Use a mechanical lift or enough people to safely lift the chassis A full chassis weighs 120 kgs an empty chassis weighs 50 kgs Figure 29 Raising the Chassis Using a Mechanical Lift o 4 4 Remove the two handles from the sides of the chassis 5 Push the chassis into the rack until the faceplate is 20cm 8 from the vertical support 6 Put the rail slides onto the rails Mellanox Technologies 47 Rev 1 8 Installation Figure 30 Put on the Rail Slide 7 Slide the chassis further into the rack 8 Slide the chassis all the way into the rack until the faceplate is touching the vertical rack sup port The caged nuts placed in the last procedure should line up with the holes in the face plate 9 Screw the 8 screws through the faceplate and into the caged nuts 10
44. de Panel Display Status Indications 56 Figure 38 Management Module Status Indications for Normal Operation 56 Figure 39 Power Cord Numbering ST Figure 40 PSU Locations 12 254 uita usine sin dar taht tnt desde Whee NT A 58 Figur 41 Power Supply 2 2 2 ERA Eee eee Hed rn die when 58 Figure 42 Leaf Board Numbering 59 Figure 43 7 Ejeetor batch sert ie ee an ae In en 60 Fipure 44 Leaf Release nass me eisen Wa WE sten 60 Figure 45 Management Module Numbering 62 Figure 46 Ejector Lat die 62 Figure 47 Spine Board Extraction 63 Figure 48 Fan Locations on the Chassis 64 Figure 49 Leaf Fan Module Extraction 65 Figure 50 Spine Fan Module 2 nee a uns ant 65 Figure 51 Fan Status LED on the Spine Module 66 Figure 2 Ejector Latest een da 67 Figure 53 Management Module 67 Figure 54 QSFP Connector Male and Female Views 82 6 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 List of Tables Table 1 Revision History of this User
45. e management module Make sure the mating connectors of the unit are free of any dirt and or obstacles See Section 5 5 on page 66 7 If you are running the chassis with only one management module replace the management module 8 3 2 Yellow LED for the Leaf Fan on the Management Module is Lit 1 Check the MLNX OS management for confirmation and possible explanation of the alert 2 Make sure that there is nothing blocking the front or rear of the chassis and that the fan mod ules and ventilation holes are not blocked especially dust over the holes 3 If you find dust blocking the holes it is recommended to clean the fan unit and remove the dust from the front and rear panels of the switch using a vacuum cleaner 4 Determine which fan module is problematic by checking the status LED on each fan module 5 Remove and reinstall the problematic fan unit Make sure th e mating connector of the new unit is free of any dirt and or obstacles See Section 5 4 on page 64 6 Replace the Leaf fan module LA Replace defective leaf fan modules as soon as they are identified 74 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Should any of the boards shut down due to over temperature follow the procedure starting in Step 2 8 3 3 Yellow LED for the Spine Fan on the Management Module is Lit 1 Check the MLNX OS management for confirmation and possible explanation of the alert 2 Determine whic
46. e 15 Reset Button CONSOLE MGT Last PSU S FANS L FANS MASTER M ha eo UY UU Mellanox Technologies 29 Rev 1 8 Hardware This button resets the CPU of the management module A quick push of this button performs this reset When the reset button is pushed on the master management module this management mod ule is reset becoming the slave and the other management module becomes the master If there is only one management module in the chassis all of the leafs and ports are reset by bringing them down and powering them up when the reset button is pushed When the button is held down for 15 seconds the management module is reset and the password is deleted You will then be able to enter without a password and make a new password for the user admin 30 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 3 Installation This chassis can be installed in standard 19 racks that have depths between 65cm and 80cm between the vertical supports of the rack 5 Installation and initialization of the chassis is a simple process requiring attention to the normal mechanical power and thermal precautions for rack mounted equipment Your Chassis will be shipped in two packages one with the all of the leafs leaf package that are ordered and an extra management module should it be ordered and the second package chassis package with all of the rest of the parts The chassis requires initial con
47. e Errata For any possible errata due to hardware issues see the switch support product page This requires a customer support login Mellanox MLNX OS SwitchX WebUI Overview for MLNX OS software Software WebUI User s Manual Mellanox MLNX OS SwitchX This document contains information regarding configuring and manag Software User Manual ing Mellanox Technologies SwitchX Switch Platforms MLNX OS Software Command Ref Command Reference Guide for MLNX OS listing all of the commands erence Guide available through MLNX OS with explanations and examples MLNX OS Software Configuration Configuration Guide for MLNX OS displaying different configuration Guide scenarios Conventions Throughout this manual the name 155200 and the terms chassis and switch are used to describe the 216 port QSFP InfiniBand chassis unless explicitly indicated otherwise The following icons are used throughout this document to indicate information that is important to the user Y This symbol makes recommendations to the user Mellanox Technologies 9 Rev 1 8 10 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 1 Overview This User Manual provides an overview of the IS5200 QSFP Modular InfiniBand Switch Plat form known in this document as the chassis or switch and its operational environment Mellanox IS5200 switch systems provide the highest performing fabric sol
48. e PSUs 2 Plug the other end of the power cords into grounded outlets Make sure that the outlets and circuits will not be overloaded Spread out the load over at least two or three circuits or use a 3 phase circuit Figure 35 Multiple Power Inlets Electric Caution Notification CAUTION ACHTUNG ATTENTION Risk of electric shock and Gafahr des elektrischen Schocks Risque de choc et de danger energy hazard Entferrnen des Netzsteckers e lectriques Le de branchment The PSUS are all independent elnes Netzteils spannungsfrei d une seule alimentation stabi Um alle Einhieten spannungsfrei lise e ne de branch uniquement Disconnect all power supplies to zu machen sind die Netzstecker qu un module Alimentation ensure a powered down state aller Netzteile zu entfernen Stabilise e Pour isoler com inside of the switch platform pletement le module en cause Il faut de brancher toutes les ali mentations stabilise es 3 Check the Status LEDs on all of the management modules and confirm that all of the LEDs show status lights consistent with normal operation Any yellow or red status LEDs on any of the management modules is cause for con cern and must be dealt with immediately It can take up to 5 minutes to boot up the system Turn off the system if any LEDs remain red for more than 5 minutes 52 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 5 Check that the Leaf status LE
49. e transparent n est pas solidement fix ou s il est enlev 5 Systeme de fusible neutre double p le Avertissement Systeme de fusible neutre a double p le Veuillez debrancher tous les cordons d alimentation avant d ouvrir le boitier de ce produit ou de toucher un de ses composants internes 6 Orages dangers lectriques Pendant un orage il ne faut pas utiliser le mat riel et il ne faut pas brancher ou d brancher les c bles Mellanox Technologies 85 Rev 1 8 7 Branchement d branchement des c bles InfiniBand en cuivre Les c bles InfiniBand en cuivre sont lourds et ne sont pas flexibles il faut donc faire tr s attention en les branchant et en les d branchant des connecteurs Consultez le fab ricant des c bles pour conna tre les mises en garde et les instructions sp ciales gt 8 Risque de choc et de danger Risque de choc et de danger e lectriques Le de branchment d une seule alimentation stabilise e ne de branch unique ment qu un module Alimentation Stabilise e Pour isoler completement le module en cause Il faut de brancher toutes les alimentations stabilise es Risk of electric shock and energy hazard The PSUs are all independent Disconnect all power supplies to ensure a powered down state inside of the switch plat form gt gt 9 Montage et entretien sur baie Lorsque ce produit est mont ou entretenu sur baie il faut prendre des pr cautions
50. ement Module MAC 224er diode dat dede ve eel 12 1 1 4 Product Physical Specifications and Power 13 12 Beat res Bist Due edere Le Re nr ne et eee AN ce 15 Chapter 2 Hardware 16 2 1 Installation Safety Warnings 16 2 2 Power Supply Redundancy 20 2 3 EED Stat s Indicators 255 2564 aan aa RR 21 2 3 1 Power Supply Unit LEDS A ee ae sade RACER RT es 21 2 32 Leaf Board LED Indicators ee eA n abat ae SA id rr tan 23 2 3 3 Spine Board LED Indicators 24 2 3 4 Spine Side Panel Display LED Indicators 25 2 3 5 Management Module LED Indicators 26 2 3 6 Port Connector Interfaces sempe epo am pee a ERRARE DRE 27 24 CATE lia LERRA SUA HERES MENSIS Mn ceo ee la PR CARD UAR A 28 2 5 QSFP Cable Power Budget Classification 28 2 6 Management Module Interfaces 28 2 61 PA CN a an de E D OAE 28 2 62 CONSOLE bini D s Br UN S a Names 29 2 6 3 MGT Management ss e ot ee er nn 29 2 64 SUSB As v bad es wale ar dar pl sa OPI ie siegt Qe it cie dis ieh 29 26 55 Reset RS T sca da Abe Wh Sy sinc ana wy ee eet Pe 29 Chapter3 Installation 31 3 1 Environmental and Safety Recommend
51. erefore do not let go of it while sliding it out 4 Figure 53 Management Module AO M 00 5 an MN Ejectors 5 5 2 Inserting a Management Module 1 Start with the latch fully open that is 45 degrees to the front panel of the leaf 2 Carefully seat the management module into the slot and slowly slide the board into the chas sis until the ejectors begin to engage on the chassis edge Mellanox Technologies 67 Rev 1 8 Insertion and or Hot swap Extraction 3 Press the ejectors inward until the locks snap This indicates that the board is fully seated 4 Perform the initial configuration on the management module See the Installation Guide the master management module CPU and only then configure the slave By default the master CPU is the top management module For further information on the master and slave roles see the FabricIT EFM Software UM section High Availability On switch systems with dual management systems first connect the cable and configure All management modules in the chassis must go through an initial configuration proce dure See the Installation Guide for the initial configuration procedure 2 5 6 Switch Shut Down Procedures To shut down the chassis run the following command twice once for each MM Reload halt noconfirm The chassis cannot be restarted remotely To restart the chassis you must physically go to the switch and un
52. es y reglamentaciones nacionales 13 C digos el ctricos locales y nacionales LA Este equipo se debe instalar conforme a los c digos el ctricos locales y nacio nales 94 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 14 Cable de alimentaci n homologado por UL y con certificaci n CSA N En conexiones de America del Norte seleccionar un cable de alimentaci n homologado por UL y con certificacion CSA de tres conductores 16 AWG termi nado en un enchufe moldeado con capuch n de 125 voltios nominal 13 A con una longitud minima de 1 5 metros pero no mas de 4 5 metros En conexiones europeas seleccionar un cable de alimentacion armonizado internacionalmente y marcado lt HAR gt de tres conductores hilo de 1 0 mm2 como minimo 300 voltios nominal con cobertura protectora aislante de PVC El cable debe tener un enchufe moldeado con capuch n de 250 voltios nominal 10 A 15 Anadir conexi n a tierra N Antes de conectar el dispositivo ala linea de alimentaci n los tornillos del terminal de la puesta a tierra de protecci n del dispositivo se deben conectar a la puesta a tierra de protecci n de la instalaci n del edificio Informaci n de conexi n a tierra La instalaci n del edificio deber prover un medio para la conexi n con la puesta a tierra de protecci n y un t cnico de servicio deber conectar permanentemente el equipo a dicho medio de conex
53. ev 1 8 Hardware the other end of the connector plugged into a functioning port When a logical connection is made the LED will change to green When data is being transferred the light will blink green The switch does not provide a visual means to indicate the port speed configuration SDR DDR QDR and or the link width 1X or 4X The speed and link width config urations can be retrieved using management software 2 3 3 Spine Board LED Indicators Each leaf board is connected by links to each spine module Each spine has the following LEDs One status LED for the spine health One status LED for the spine fan modules 36 status LEDs showing the existence of the leaf to spine connection The 36 LEDs on each spine are divided by the number of leafs and the result N is the number of connections from each leaf that are connected to the spine 6 spines must be installed and work ing to ensure that full BW exists between nodes The maximum number of connections from each leaf is 3 Ifthe number of leafs x the maximum number of connections per leaf is less than 36 then some of the leaf to spine connection LEDs may be OFF The status LEDs for the spine and their descriptions are shown in Table 6 The LEDs indicate as follows Figure 9 Spine Status LEDs 19 21 2325 2729 31 33 35 YY YY ODODOODOOO 090000 0000000000 Jt ug 000000 00000000000 Spine Status Fan Status Spine Leaf IB Connection Status 2 3 3 1
54. f the FRUs and chassis according to the WEEE Directive 9 Dispose of these pieces in a legal and environmentally friendly way 6 1 1 Removing the Chassis Remove the screws connecting the upper brackets to the rack 2 Remove the screws holding the chassis to the rack These screws are located in the faceplate on the spine side of the chassis 3 Remove the upper brackets from the chassis B Dispose of the Chassis in a legal and environmental way 5 Go to the Mellanox website for detailed instructions for disassembly of the chassis according to the WEEE Directive 6 1 2 Removing the Bottom Shelf 1 Remove all of the bolts that are holding the shelf to the rack 2 Remove the shelf Mellanox Technologies 69 Rev 1 8 Disassembly and Disposal 3 Remove all of the caged nuts 6 2 Disposal According to the WEEE Directive 2002 96 EC all waste electrical and electronic equipment EEE should be collected separately and not disposed of with regular household waste Dispose of this product and all of its parts in a responsible and environmentally friendly way Go to the Mellanox website for detailed instructions for disassembly according to the WEEE Directive 70 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 7 Switch Management Tools This chapter describes the management module and tools available for Out of Band management of the switch system via FabricIT EFM
55. f the bolts for each rail slide install the rail slides onto the rack Check that both sides of the switch left and right are the same level in the rack Figure 20 Connect Rail Slide to Rack Vertical support Place the four bolts for the caged nuts within reach Put the shelf into place and screw the bolts into the nuts from step 2 Extend the rail slides to the rack vertical support and screw them to the rack Tighten all of the screws to 9 2 Nm or 81 5 pound inches ADM Insert 8 caged nuts for the facepl ate in the exact locations shown in Figure 21 Inserting the Caged Nuts for the Faceplate Mellanox Technologies 39 Rev 1 8 Installation Figure 21 Inserting the Caged Nuts for the Faceplate Place the caged nuts here 3 5 3 Installing the Cable Holder Y Now is the time to install the cable holder 40 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Figure 22 Cable Holders Right side cable holder pr di y 12 X M6 caged nut il 12 X M6 bolt for caged nut gr Left side cable holder 1 Place the Cable holder next to the rack on the connector side ofthe chassis and identify the holes where the caged nuts are to be placed Mellanox Technologies 41 Rev 1 8 Installation Figure 23 Inserting the Caged Nuts for the Cable Holder Top bracket connects here Place the caged nuts here Shelf
56. figuration to get the chassis and Fabric management up and run ning through remote management See the Installation Guide that is packed in the box for the instructions to make the initial configuration access area can be accessed only through the use of a special tool lock and key or P This unit is intended for installation in a Restricted Access Location A restricted n other means of security 3 1 Environmental and Safety Recommendations The following are Mellanox recommendations Y Recommended ambient temperature in the System room is 20 5 C Recommended humidity range is 40 15 without condensing Y It is highly recommended that the installation sites be as isolated as possible from all sources of radio transmissions and electrical interference It is highly recommended that the installation site building be equipped with a light ning rod Mellanox Technologies 31 Rev 1 8 Installation It is highly recommended that the installation site be equipped with smoke detectors and a fire alarm warning system The system requires a KVA rated UPS system It is recommended that a UPS system be installed to protect the equipment in the event of unexpected power failure Make sure that the outlets and circuits will not be overloaded Spread out the load over at least two or three circuits or use a 3 phase circuit 3 2 Chassis Package Contents The package includes e 1 chassis with the following module
57. generation When hot swapping any of the units it is necessary to wait 1 minute after removing the defective part before inserting the new part This is necessary so that the manage ment module will start a new cycle checking through the leafs and spines for the FW versions The leaf boards are numbered from top to bottom with corresponding numbers displayed to the outside of the leafs vertically along the side panel Figure 42 Leaf Board Numbering 5 2 1 Extracting a Leaf Board Each leaf board has an ejector that locks the board in place and serves as a lever for seating or extracting see Figure 53 1 Run the shut down command no power enable module For example to shut down leaf 16 run the command below switch 1 Orca648 master config no power enable L16 Disconnect all cables connected to the leaf Push the latch to unlock the ejector from the chassis Open the ejector until it is 45 degrees from the leaf Pull out on the leaf aA kw N Mellanox Technologies 59 Rev 1 8 Insertion and or Hot swap Extraction Figure 43 Ejector Latch Pull the leaf out of the chassis Leaf Leaf mM Push the latch oe release RS Ejector Y N The board is short therefore do not let go of it while sliding it out Figure 44 Leaf Release Push here to release the latch Pull the ejector to remove the leaf Inserting a Leaf Board
58. h barcode 2010 1 3 Oct 2010 Added air flow direction 1 2 Oct 2010 Removed Chassis Rails from installation Kit and Installation instruc tions Fixed Port Connector LED Assignment 1 1 Aug Added QSFP Cable Power Budget Classification 2010 Added Remove all protective plastic film from all sides and top of the chassis Added FCC Statement 15 19 Added QSFP Cable Power Budget Classification Changed Installation instructions Remove FRUs and 4 people can lift using handles pp37 38 40 Added weight warnings Added CoG and Max heat to the specs in Appendix A 1 0 July 2010 Initial release 8 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 About this Manual This manual provides an overview of the SwitchX based IS5200 modular InfiniBand chassis switch and guidelines for its operation Intended Audience This manual is intended for users and system administrators responsible for installing and setting up the chassis platform The manual assumes familiarity with the InfiniBand architecture specification Related Documentation The documentation set accompanying the QSFP Chassis InfiniBand Switch platform includes the following Table 2 Reference Documents and Web Sites Document Name Description InfiniBand Architecture Specifica The InfiniBand Architecture Specification that is provided by IBTA tion Vol 1 Release 1 2 1 Switch Hardwar
59. h spine has a defective fan by checking the Fan LEDs on all of the spines 3 Make sure that there is nothing blocking the front or rear of the chassis and that the fan mod ules and ventilation holes are not blocked especially dust over the holes 4 If you find dust blocking the holes it is recommended to clean the fan unit and remove the dust from the front and rear panels of the switch using a vacuum cleaner 5 Remove and reinstall the fan unit of the spine Make sure the mating connector of the new unit is free of any dirt and or obstacles See Section 5 4 on page 64 6 Replace the spine fan module LA Replace defective spine fan modules as soon as they are identified 8 4 Spine Board The yellow LED on the Spine board is lit 1 Check the MLNX OS management for confirmation and possible explanation of the alert 2 Make sure that there is nothing blocking the front or rear of the chassis and that the fan mod ules and ventilation holes are not blocked especially dust over the holes 3 If you find dust blocking the holes it is recommended to clean the fan unit and remove the dust from the front and rear panels of the switch using a vacuum cleaner 4 Remove and reinstall the spine board Make sure the mating connectors of the unit is free of any dirt and or obstacles See Section 5 3 on page 61 5 Make sure that the spine and the Leafs both have the same version of FW 6 Reburn the FW and remove and reinstall the spine
60. i n Un T CNICO DE SERVICIO comprobar si la toma el ctrica de la que se suminis trar corriente al equipo provee una conexi n con la puesta a tierra de protecci n del edificio De no ser as el TECNICO DE SERVICIO se encargar de instalar un CON DUCTOR DE CONEXI N A TIERRA DE PROTECCI N del terminal de puesta a tierra de protecci n separado al conductor de tierra de protecci n del edificio El equipo se instalar en un rea donde haya conexi n equipotencial como por ejemplo un centro de telecomunicaciones o una sala de computadoras dedicada 16 C digos de instalaci n Este dispositivo se debe instalar conforme a la versi n m s reciente de los c digos el ctricos nacionales del pa s en cuesti n En Am rica del Norte el equipo se debe instalar de acuerdo con las disposiciones vigentes del C digo El ctrico Nacional de los EE UU y del C digo El ctrico de Canad 17 Directiva sobre RAEE Conforme a la Directiva 2002 96 CE sobre RAEE todos los residuos de equipos el c tricos y electr nicos EEE se deben recolectar por separado y no se deben eliminar junto con residuos dom sticos Al deshacerse de este producto y de todas sus partes h galo de una manera respon EEN sable y respetuosa con el medio ambiente Mellanox Technologies 95
61. icator LEDs Figure 6 Power Supply Unit Status Indications pr DC AC OK FAIL OK AC When lit this LED indicates input voltage between 100 and 240 Volts DC FAIL When lit this LED indicates a fault in the power supply DC OK When lit this LED indicates that the output from the power supply is 48 VDC gt The PSUs are on the spine side of the chassis behind two cover panels The plugs for these PSUs are on the leaf side of the chassis 5 Mellanox Technologies 21 Rev 1 8 Hardware Figure 7 PSU Cover On and Off PSU Covers On Mx e ee ee o Mellanox Fan 1 Fan 2 Fan 4 El o a m IS5200 E E 1 ODODODODO FAN DDCDDOBDG O00000000 DODDDODOG PS1 s OOOOOCONO Fan DDCODOBDG aaa aa aa DDBDDODDO EEHEEHE OOOO FAN DOODBOGDO firn zr Cortes ets all CA E DODDDDDUDU y FAN CEE EEE LI Donna oonannnar DODDDODDO War 8 r1 el EENEN one aalala a a afa LIL 1 000000000 E E DUDDDIDOU y FAN DDODDDODDD DDDDDDBDD gooooooooo CAL PSU Covers Off N N ev oo Fan 3 eieae DUDDBHODU X Fan 4 155200 DOGGODRE alala a e eT aja 000000000 Ooooooooo DOGEDDEE eee ee 0000000 inl DODDDOODU poncanna eeaeee 000000000 Ooooooooo eleme eae ODODODODO emee poddana 22 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 2 3 2 Leaf Board LED Indicators Figure 8 Leaf Board Led Indicators
62. itch 7 1 InfiniBand Subnet Manager The InfiniBand Subnet Manager SM is a centralized entity running in the switch It discovers and configures all the InfiniBand fabric devices to enable traffic flow between those devices The SM applies network traffic related configurations such as QoS routing partitioning to the fabric devices You can view and configure the Subnet Parameters SM via the CLI WebUI Each InfiniBand subnet needs one subnet manager to discover activate and manage the subnet An InfiniBand network requires a Subnet Manager to be running in either the Infiniband switch itself switch based or on one of the nodes whic h is connected to the Infiniband fabric host based 7 2 Accessing the CPU via the Ethernet Connector Once the initial configuration is completed the management tools can be accessed through SSH Telnet WEB SNMP XML 7 3 Upgrading Software The new software and firmware update is available to the user from the Mellanox Support web site Copy the update to a known location on a Remote server within your LAN Use the CLI or the GUI in order to perform the Software upgrade For further information please refer to the FabricIT EFM user manual oy an earlier version than the minimum that the new version of the software can work with then the chassis management system may require up to 45 minutes to update all of the FW images in all of the leafs and spines If FabricIT
63. iter Sicherung im Netzteil Netzstecker ziehen um sicher zustellen da keine Spannung am Ger t anliegt Entfernen Sie alle Netzkabel vor dem ffnen der Abdeckung dieses Produkts oder dem Ber hren der Innenteile Mellanox Technologies 91 Rev 1 8 20 lt 40 lbs 40 70 lbs 70 121 lbs gt 121 lbs lt 18 kgs 18 32 kgs 32 55 kgs gt 55 kgs 92 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Appendix H Advertencias de seguridad para la insta laci n Spanish 1 Instrucciones de instalaci n Antes de conectar el equipo a la fuente de alimentaci n leer todas las instrucciones de Instalaci n 2 Lesi n corporal por peso e lt 40 lbs 40 70 lbs 70 121 lbs gt 121 lbs lt 18 kgs 18 32 kgs 32 55 kgs gt 55 kgs 3 Lesi n corporal por peso This equipment is very heavy and should be moved using a mechanical lift to avoid injuries 4 Sobrecalentamiento No se debe utilizar el equipo en un area con una temperatura ambiente superior a la m xima recomendada 45 C Adem s para garantizar una circulaci n de aire adec uada se debe dejar como m nimo un espacio de 8 cm 3 pulgadas alrededor de las aberturas de ventilaci n 5 Apilamiento del chasis Los chasis no se deben apilar sobre otros equipos La ca da del chasis podr a causar lesiones corporales as como da os al equipo 6 Dos fusibles uno en el polo y otro en el neutro
64. le that indicates power supplies are good 3 Spine Board indicators will display status ofin ternal links to the inst alled leaf boards All PHY links to existing leaf Boards should be ON 4 Check the Spine LEDs and make sure they coincide with Figure 37 Figure 37 Spine Side Panel Display Status Indications O O A E O STATUS PSU SPINE LEAF MNG 1 MNG 2 STATUS FANS FANS STATUS STATUS 5 Check the Management Module LEDs and make sure they coincide with Figure 38 Figure 38 Management Module Status Indications for Normal Operation TA Ina ci o nono gsal Il 56 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 5 Insertion and or Hot swap Extraction Before starting any procedure on the IS5200 switch system put an ESD prevention wrist strap on your wrist and connect to the IS5200 chassis Do NOT mix replacement parts based on different generations of chip gt Do not install InfiniScale IV based replacement parts within a SwitchX based chassis and vice versa All Replacement modules must be consistent with the Chassis family he and switch chip generation When hot swapping any of the units it is necessary to wait 1 minute after removing the defective part before inserting the new part This is necessary so that the manage ment module will start a new cycle checking through the leafs and spines for the FW versions This switch platform supports hot swap capabilities for the parts
65. listed below e Power supply units Leaf boards Spine boards Leaf fan module Spine fan module Management modules 5 1 Power Supply Units 5 1 1 Extracting and Inserting the Power Supply Unit With all of the 1000W power supplies installed the system is in N redundant configuration With all of the N N 1600W power supplies installed the system is in N N redundant configura tion half of the PSUs may be extracted without bringing down the system To extract a PSU 1 Determine which AC connector on the connector side of the chassis corresponds to the defec tive PSU 2 Remove the power cord from the power supply unit Note which power cord it is according to the AC numbering Figure 39 Power Cord Numbering Mellanox Technologies 57 Rev 1 8 Insertion and or Hot swap Extraction Figure 40 PSU Locations A eo eo SS CT S mm I i Ji lp Ip Fan 1 Fan 3 Fan 4 a a o o 155200 e Goooou000 Li aa aleja Ooo oooooo rte state fa a ODOOOOO0O pocongan DDODBDDDDG Oooonoooo 000000000 DODo0000d Oooooooo0 Oooonoooo DDODODODUS 000000000 DDODBDDDDD DO00000000 DODO00000 y 000000000 DOOOD0000 000000000 ooosoonoe E DODOODDDO COoooooon i Al DDDDDDDDD CELLIER 3 On the spine side of the
66. moving the defective part before inserting the new part This is necessary so that the manage ment module will start a new cycle checking through the leafs and spines for the FW versions Before starting any procedure on the 185200 series switch system put an ESD prevention wrist strap on your wrist and connect to the 185200 series chassis 5 5 1 Extracting a Management Module Management modules are located on the leaf side above the leafs There are two places to install the management modules P Only one management module is required to run the switch system Each management module has a pair of ejectors that lock the board in place and serve as a lever for seating or extracting see Figure 53 66 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 1 Run the shut down command no power enable lt module gt For example to shut down management module 2 run the command below switch 1 Orca648 master config no power enable MGMT2 Disconnect all cables connected to the management module Push outward on the latches to unlock the ejectors from the chassis Open the ejectors until they are 45 degrees from the module Ak YN Pull out on the module using both ejectors Figure 52 Ejector Latch Pull the module out of the chassis Management Module Spine Management Module Spine Push the latch E release _ 0 EPt IN A The board is short th
67. n cause bodily injury and equipment damage 16 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 6 Redundant Power Supply Connection Electrical Hazard power supply do not operate the product with the blank cover removed or not securely This product includes a redundant power or a blank in its place In case of a blank A fastened 7 Multiple Power Inlets Risk of electric shock and energy hazard The PSUs are all independent Disconnect all power supplies to ensure a powered down state inside of the switch platform 8 During Lightning Electrical Hazard During periods of lightning activity do not work on the equipment or connect or dis connect cables 9 Copper InfiniBand Cable Connecting Disconnecting attached to or detached from the connectors Refer to the cable manufacturer for spe Copper InfiniBand cables are heavy and not flexible as such they should be carefully A cial warnings instructions 10 Rack Mounting and Servicing to ensure that the system remains stable In general you should fill the rack with equip When this product is mounted or serviced in a rack special precautions must be taken A ment starting from the bottom to the top 11 Equipment Installation This equipment should be installed replaced and or serviced only by trained and qual ified personnel 12 Proper Enclosure A suitable electrical mechanical and fire enclosure shall be provided
68. n um einen einwandfreien Luftstrom zu gew hrleisten 3 Stapeln des Chassis Das Chassis sollte nicht auf andere Ger te gestapelt werden Wenn das Chassis herun terf llt kann es zu Verletzungen und Besch digungen an Ger ten f hren 4 Redundanter Stromversorgungsanschluss Elektrische Gefahr Stromversorgung Betreiben Sie das Produkt nicht wenn diese Abdeckung nicht sicher Dieses Produkt verf gt ber eine Abdeckung ber dem Bereich f r die redundante A festsitzt oder entfernt wurde 5 Bei Gewitter Elektrische Gefahr Arbeiten Sie w hrend eines Gewitters und Blitzschlag nicht am Ger t schlie en Sie keine Kabel an oder ab 6 Anschlie en Trennen von InfiniBand Kupferkabel an die Anschl sse angebracht bzw davon getrennt werden Lesen Sie die speziellen InfiniBand Kupferkabel sind schwer und nicht flexible Deshalb m ssen sie vorsichtig A Warnungen und Anleitungen des Kabelherstellers 7 Gafahr des elektrischen Schocks frei Um alle Einhieten spannungsfrei zu machen sind die Netzstecker aller Netzteile zu Gafahr des elektrischen Schocks Entferrnen des Netzsteckers elnes Netzteils spannungs A entfernen Mellanox Technologies 89 Rev 1 8 9 0 Risk of electric shock and energy hazard The PSUs are all independent Disconnect all power supplies to ensure a powered down state inside of the switch plat form 8 Rack Montage und Wartung sichtsma nahmen zu ergreifen um die Stabili
69. nox web site Basic installation is covered in Chapter 3 Installation on page 31 Hot swapping components and hardware maintenance is covered in Chapter 5 Insertion and or Hot swap Extraction on page 57 1 1 Product Information 1 1 1 Serial Number and Product Version Information The serial number GUID Identifier and product version information are found on the label that is attached to the pull out tab that is below the Mellanox logo on the spine side of the chassis Figure 1 Generic Product Label S N MTO806X0150 4 HEN P N MIS5200Q 4DNC AS O A ooozcoozoozzos c MUNINN Rev A1 GUID O INN 0002C90 MAC 20040 IMA SR Made in Israel N Mellanox Technologies 11 Rev 1 8 Overview The GUID is the System Image GUID according to the IB spec It is burned on the board which is in the chassis All the boards and the management SW look for this GUID in addition to their own Node GUID 1 1 2 License Key This chassis comes with a license for FabricIT Chassis Manager The license key to activate the license is found on the bottom side of the pull out tab Figure 2 License Key Label A MIN License Key LK2 EFM CONFIG 5G2L S5HG LOKA L64Q K6D2 Q60Q 3ADH G88A 1H3Q 7FT9 5M Top right hand corner of the spine side of the chas sis n Tab location with Product label and management MAC Fan 4 IS5200 1 1 3 Management Module MAC
70. of AWG 6 or Amm diameter is rec ommended for ground ing this device The chassis is concurrently grounded through each of the PSUs Only connect the PSU cords to properly grounded outlets Do not rely on the PSU grounds It is absolute ly necessary to connect the grounding post Make sure the connections are solid and permanent If you choose to not use the ground screw make 50 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 sure that the rack is properly grounded and that there is a valid ground connection between the chassis of the switch and the rack Test the ground using an Ohm meter Figure 34 Ground Connectio n Fan 3 Fan 4 IS5200 ja ry A A e ee oo oo so Mellanox I I Fan 1 Fan 2 o m E e Ground Post ESD Post 3 6 Power Connections The switch includes integrated hot swap power supplies which support up to 4 load sharing 1000W or 1600W supply units The slots for the power supply units PSUs are on the spine side The left side has odd numbered PSUS and the right side has even numbered PSUs Each PSU has a dedicated AC inlet This design enables the optional use of separate main and backup AC feeds The input voltage is Autorange 100 240 VAC 50Hz or 60Hz The output voltage for the PSUs is 48V The power cords should be standard 3 wire AC power cords including a safety ground 3
71. plug all of the power cords to the chassis and then replug in all of the power cords to the chas SIS The first time you run the command it shuts down the master management module and the sec ond time shuts down the slave management module To shut down a leaf run the following command no power enable lt module gt To shut down a spine run the following command no power enable lt module gt To shut down a management module run the following command no power enable lt module gt 68 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 6 Disassembly and Disposal 6 1 Disassembling the Chassis Power down the chassis Remove all power cables Remove all connector cables Disconnect the ground lug from the ground post Remove all leafs Remove all spines SLD Er D cl Remove all management modules Use enough people to safely lift this product LA dE Lil lt 40 lbs 40 70 lbs 70 121 lbs gt 121 lbs lt 18 kgs 18 32 kgs 32 55 kgs gt 55 kgs N CN h This product and all of its parts are NOT to be disposed of with household waste This product contains printed circuit boards cables and batteries According to the WEEE Directive 2002 96 EC all waste electrical and electronic equipment EEE should be collected separately and disposed of according to the directive 8 Go to the Mellanox website for detailed instructions for disassembly o
72. r standards IEC 60825 1 2007 03 and EN 60825 1 2007 21 Do Not Use the Switch as a Shelf or Work Space Caution Slide rail mounted equipment is not to be used as a shelf or a work space ra 22 WEEE Directive According to the WEEE Directive 2002 96 EC all waste electrical and electronic equipment EEE should be collected separately and not disposed of with regular household waste Dispose of this product and all of its parts in a responsible and environmentally friendly way gt lt 23 Country of Norway Power Restrictions N This unit is intended for connection to a TN power system and an IT power system of Norway only Mellanox Technologies 19 Rev 1 8 Hardware 2 2 Power Supply Redundancy This chassis has N 1 power supply redundancy With a full complement of power supplies one power supply can be removed and replaced without shutting down the chassis or stopping the flow of data 20 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 2 3 LED Status Indicators The LEDs are placed on the chassis for the convenience of the IT manager All chassis conditions and management options are available and controllable through the management SW either CLI or Web GUI Y It is recommended that all of the chassis sub systems be maintained and managed through the management software 2 3 1 Power Supply Unit LEDs Each Power Supply Unit has the following ind
73. rap on your wrist and make sure there is good contact between your body and the strap 2 Plug the other end of the wrist strap to a valid ground Make sure that this is a tight fit 3 5 2 Installation Procedure 3 5 2 1 Requirements You will need 2 phillips screwdriver 3 phillips screwdriver e a grounding lug ground wire to properly ground the chassis Y The installation will be much easier with a power screwdriver Y It is recommended to use AWG6 or 4mm diameter wire for grounding purposes It is recommended to have at least two people for the duration of the installation proce dure Use a mechanical lift to raise this chassis If not use enough manpower to ensure the safety and wellbeing of all of the people involved in the installation 34 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 3 5 2 2 Installation and Cable Management Kit Parts Parts for installing the shelf 1 shelf e 16 M6 bolts for the caged nuts 8 for the shelf and 8 for the faceplate 12 caged nuts 4 for the shelf and 8 forthe 2 Shelf rail slides faceplate Parts for installing the chassis 2 Chassis rail slides 4 Handles 8 pan head bolts 8 Allen head screws 2 Flat 4 hole metal spacers allen wrench 8 split lock washers Parts for installing the cable manager 1 RH cable holder LH cable holder 12 caged nuts M6 12 M6 bolts You will need 10 U of space in the rack Nine
74. rt 39 Figure 21 Inserting the Caged Nuts for the Faceplate 40 Figure22 CablesHolders 2 2 ur see en AA 41 Figure 23 Inserting the Caged Nuts for the Cable Holder 42 Figure 24 Cable Holder Placement 43 Figure 25 Chassis Rails and Rail Slides 44 Figure 26 Placement of Chassis in Rack 45 Figure 27 Screw the Handles Onto the Chassis 46 Figure 28 The Rails are Already Connected Onto the Chassis 46 Figure 29 Raising the Chassis Using a Mechanical Lift 47 Figure 30 Put on the Rail Slide 48 Figure 31 185200 Chassis on the Shelf 49 Figure 32 Face Plate Mounting Bolt Locations 49 Figure 33 Move the Rail Slide to the Vertical Support 50 Figure 34 Ground Connection 51 Figure 35 Multiple Power Inlets Electric Caution Notification 52 Fig re 36 Spine Mod le his radeln a Te p un e ptt 53 Mellanox Technologies 5 Rev 1 8 Figure 37 Spine Si
75. s installed e 6 spines 1 management module 4 fans 4PSUs If you are not using a mechanical lift to install the chassis redu ce the weight of the chas sis by removing all of the spines power supply units management modules and fan units and put aside for reinstallation after the chassis is installed in the rack e l installation kit 1 box containing 4 power cords 250v 15a 2 0M C14 to C13 USA UL Standard 1 cable management kit 1 Installation Guide Before you install your new 185200 series chassis unpack the system and check to make sure that all the parts have been sent check this against the parts list Check the parts for visible dam age that may have occurred during shipping If anything is damaged or missing contact your customer representative immediately 5 32 Mellanox Technologies J QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 The rack mounting holes conform to the EIA 310 standard for 19 inch racks Guaran tee proper ventilation by leaving 8cm 3 of space to the front and rear of the switch A This will ensure proper air flow through the chassis This is crucial for maintaining good airflow at ambient temperature In particular route cables such that they do not impede the air into or out of the chassis 3 3 Leaf Package Contents The leafs are ordered by the customer and are shipped 4 to a box The customer will receive as many boxes as needed to fill the order
76. sp ciales pour s assurer que le systeme reste stable En g n ral il faut remplir la baie avec du mat riel de bas en haut 10 Fuite gt 3 5mA Leakage gt 3 5mA ATTENTION La connexion la terre des forts courants de fuite est essentielle avant le branchement de l alimentation Avant de brancher l appareil la conduite d alimentation les vis de protection la terre du terminal de l appareil doivent tre appliqu es l installation de protection la Terre du b timent gt 11 Forts Courants de Fuite High Leakage Current Attention Forts courants de fuite Il est essentiel de relier a la terre avant de brancher l alimentation 12 Ajouter une information de connexion la masse Connect a Valid Ground to this Device terre du terminal de l appareil doivent tre appliqu es l installation de protection la Avant de brancher l appareil la conduite d alimentation les vis de protection la A Terre du b timent 86 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 13 Installation du mat riel Ce mat riel ne doit tre install remplac ou entretenu que par du personnel form et qualifi gt 14 Elimination du materiel L limination de ce mat riel doit s effectuer dans le respect de toutes les l gislations et reglementations nationales en vigueur gt 15 Codes lectriques locaux et nationaux Ce mat riel
77. t einer Mindestl nge von 1 5 m Six Feet aber nicht mehr als 4 5 m F r die europ ischen Zusammenhang w hlen Sie ein Netzkabel das international harmonisiert und der Aufschrift lt HAR gt 3 Leiter mindestens 0 75 mm2 Draht bewertet mit 300 V mit einem PVC Mantel isoliert Das Kabel muss eine angespritztem Stecker bewertet bei 250 V 10 A 15 Ableitstrom gt 3 5mA LEAKAGE gt 3 5mA WARNUNG Hohe Ableitstrom Earth Verbindung bevor Sie die Verbindung von wesentlicher Bedeutung werden 16 Add GND Verbindung Informationen Bevor Sie dieses Ger t an das Stromnetz die Schutzerde Terminal Schrauben dieses Ger t muss an den Schutzleiter in der Geb udeinstallation 17 Installation Codes Dieses Ger t muss installiert sein entsprechend auf die neueste Version des Landes National Electrical Code F r Nordamerika m ssen in bereinstimmung mit den gel YN tenden Vorschriften in der US amerikanischen National Electrical Code und dem Canadian Electrical Code 18 Zusammenschaltung von EINHEITEN Kabel f r den Anschluss an das Ger t RS232 und Ethernet Schnittstellen m ssen UL zertifiziert Typ DP 1 oder DP 2 Hinweis wenn nicht mit Wohnsitz in LPS Schal tung berstromschutz Eine leicht zug ngliche Auflistung Abzweigleitung berstrom Schutzeinrichtung 20 A bewertet werden m ssen in dem Geb ude Verkabelung 19 Zweipolige bzw Neutralleiter Sicherung im Netzteil Achtung Zweipolige bzw Neutralle
78. t t des Systems zu gew hrleisten Im Wenn dieses Produkt in einem Rack montiert oder gewartet wird sind besondere Vor A Allgemeinen sollten Sie das Gestell von unten nach oben mit Ger ten f llen 9 Ger teinstallation Diese Ger t sollte nur von geschultem und qualifiziertem Personal installiert ausge tauscht oder gewartet werden 10 Ger teentsorgung Die Entsorgung dieses Ger ts sollte unter Beachtung aller nationalen Gesetze Bestim mungen erfolgen 11 Regionale und nationale elektrische Bestimmungen Dieses Ger t sollte unter Beachtung der regionalen und nationalen elektrischen Bestimmungen installiert werden 12 Richtigen Schutz sicherstellen Geeigneter elekrischer mechanischer und Feuerschutz sind vom Hersteller des Endprodukts oder dem Endbenutzer bereitzustellen 13 Strahlenkontakt Klasse 1 Laserprodukt und Referenzen zu den aktuellsten Lasterstandards Achtung Nutzung von Steuerungen oder Einstellungen oder Ausf hrung von Proze duren die hier nicht spezifiziert sind kann zu gef hrlichem Strahlenkontakt f hren lt lt ICE 60 825 1 1993 A1 1997 A2 2001 und EN 60825 1 1994 A1 1996 A2 2001 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 14 UL und CSA Certified Netzkabel F r Nordamerika Stromanschluss w hlen Sie ein Netzkabel das UL und CSA Certi fied LA 3 Leiter 18 AWG mit einem angespritztem Stecker bewertet bei 125 V 15 mi
79. tch Platform Hardware User Manual Rev 1 8 Appendix E Replacement Parts Ordering Numbers Table 14 Replacement Parts Ordering Numbers Part Description OPN Power supply unit PSU 1000W for Modular Switch MTP005001 Family Power supply blank MTMO05001 Leaf board unit 18 port QDR MIS5001QC Modular Switch Family Leaf Blank MTM005004 Spine board unit for QDR Switch system MIS5002QC Spine board blank MTM005002 PPC460 Management Module MIS5600MDC Management module blank MTM005003 Fan unit for leafs MTF005004 Modular Switch Family Management Blank MTM005003 Rack Installation Kit for IS5600 SX6536 Series MTR005600 Cable Management Device Set for IS5600 SX6536 MTRO005601 Cables Set For IS5600 SX6536 Series MTR005602 155600 SX 6536 IS5300 SX6518 Series Modular MTF005001 Switch leaf cooling chassis fan unit located on Leaf Side 185600 5X6536 IS5300 SX6518 Modular Switch MTF005002 leaf cooling chassis fan unit located on Spine Side MIS5X00 Modular Switch Series Spine Fan Unit MTF005003 DB9 to RJ45 Harness HAR000028 Power cord 250V 15A 2 0M C14 to C13 ACC000334 Power cord 125V 15A 2 0M C14 TO C13 ACC000242 Power cord Type B for USA Canada Mexico Taiwan ACC000204 Power cord Type H for Israel ACC000205 Power cord Type E F for Sweden France Germany ACC000207 Netherlands Russia Power cord Type G for UK ACC000208 Power cord Type D for India ACC0002
80. ur final 20 Cordons lectriques CA homologu s UL f Pour les prises lectriques en Am rique du Nord choisissez un cordon lectrique 4 homologu UL et certifi CSA 3 conducteurs 18 AWG termin par une fiche moul e d une tension nominale de 125 V 15 A avec une longueur minimale de 1 5 m 6 pieds et d une longueur maxi male de 4 5 m 18 pieds Pour les prises lectriques en Europe choisissez un cordon lectrique harmonis inter nationalement et marqu lt HAR gt 3 conducteurs d un diam tre de fil minimum de 0 75 mm2 d une tension nominale de 300 V avec une gaine isol e en PVC Le cordon doit avoir une fiche moul e d une tension nominale de 250 V et d une intensit nominale de 10 A 21 40 lbs 40 70 lbs 70 121 lbs gt 121 los lt 18 kgs 18 32 kgs 32 55 kgs gt 55 kgs 88 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Appendix G Installation Sicherheitshinweise German 1 Installationsanleitungen Lesen Sie alle Installationsanleitungen bevor Sie das Gerat an die Stromversorgung anschlie en 2 bertemperatur Dieses Ger t sollte nicht in einem Bereich mit einer Umgebungstemperatur ber der maximal empfohlenen Temperatur von 45 C 113 F betrieben werden Es ist ein Luft A strom von 200 LFM bei maximaler Umgebungstemperatur erforderlich Au erdem sollten mindestens 8 cm 3 in Freiraum um die Bel ftungs ffnungen sei
81. ure wait 5 minutes and then follow the procedure starting with Step 2 5 The Physical link LED for the InfiniBand connector does not come on 1 Check that both ends of the cable are connected 2 Check that the locks on the ends are secured 3 Make sure that the latest FW version is installed on both the HCA card and the switch 4 If media adapters are used check that the all connections are good tight and secure Mellanox Technologies 73 Rev 1 8 TroubleShooting 5 Replace the cable The Activity indication does not come on Check that the Subnet Manager has been started 8 3 Management Module 8 3 1 Yellow Status LED for the Chassis on the Management Module is Lit 1 Check the MLNX OS management for confirmation and possible explanation of the alert 2 Reset the master management module by pushing the rest button If you have two manage ment modules installed this will convert the master management module to the slave and con vert the slave to the master If there is only one management module in the chassis all of the leafs and ports are reset by bringing them down and powering them up when the management module is removed Make sure the S Fans and L Fans LEDs are green Make sure that the spine and the leafs both have the same version of FW Reburn the FW and remove and reinstall the management module QN Mm pp If you are running the chassis with only one management module remove and reinstall th
82. ution by delivering high bandwidth and low latency to Enterprise Data Centers EDC High Performance Comput ing HPC and Embedded environments Networks built with the 1S5200 system can carry con verged traffic with the combination of assured bandwidth and granular quality of service Built with Mellanox s 4 generation InfiniScale IV InfiniBand switch device IS5200 systems pro vide up to 40Gb s full bidirectional bandwidth per port With up to 216 ports in a 9U high form factor these systems are among the densest switching systems available The switch platform comes pre installed with all necessary firmware for standard operation within an InfiniBand fabric and requires an InfiniBand compliant Subnet Manager running from one of the hosts or the management module of the switch system An initial configuration proce dure should be followed to initialize the switch before connecting it to the network after which normal operation can proceed See the installation guide for details regarding the initial configu ration Once connected to the network the Subnet Management software automatically discov ers and configures the fabric and begins utilizing the switch It is recommended that the Mellanox OpenFabrics software package be installed on all nodes connected to the switch The software package provides a subnet manager and network manage ment tools as well as connectivity software for servers and storage and is available on the Mella
83. y up 5 3 1 Extracting a Spine Board When a slave management board is not installed or not working hot swapping the spine board connected to the master management board will cause the chassis to crash or spine 2 as the management modules are connected to the chassis components Neither the CLI nor the GUI management tools will allow you to shut down spine 1 add through these spines 5 3 1 1 Extracting Spine Board 1 or 2 Spine board 1 is connected to management board 1 and spine board 2 is connected to man agement board 2 Mellanox Technologies 61 Rev 1 8 Insertion and or Hot swap Extraction Figure 45 Management Module Numbering MNGI management module 1 MNG2 management module 2 bottom Removing spine 1 causes management module 1 to reset Removing spine 2 causes management module 2 to reset Warning If the spine you want to hot swap is connected to the master management N module the management module will reboot when you take out the spine If you have only one management module the chassis will crash Figure 46 Ejector Latch Pull the module out of the chassis Management Module Spine e Management Module Spine Push the latch release 0 p 62 Mellanox Technologies QDR InfiniBand Switch Platform Hardware User Manual Rev 1 8 Figure 47 Spine Board Extraction Pull the latches to remove the spine 19 21 2325 27 2

216 Port InfiniBand QDR Switch Platform Hardware User Manual

Contents

Download Pdf Manuals

Related Search

Related Contents