Home

User Manual

1. indre rk cede 10 SENE MIA 10 1 2 Supported Mellanox Adapter Devices and 10 2 Burning the Expansion ROM IMAQe sssssssssssssssssssssrssssssssssrsnnrsnnnsnnrrn nnn sk RR nn RR RR RR RR RR RR RR RR nennt nsn nn nn 11 2 1 Burning the Image on ConnectX Family 0 annans 11 2 1 1 Prerequisites i eie eee MSS ERIKA ARA o HERE RE R R e 11 2 1 2 Image Burning 0448 11 2 2 Removing the Expansion ROM 11 3 Preparing the DHCP Server in Linux 12 3 1 Case l InfiniBand ARR KR RR 12 3 1 1 Installing dE t E ne ES 12 3 1 2 Configuring the DHCP 0 2 41 00 000 rr n anar ARA R nennen nenne 12 3 1 8 Running the DHCP esee entente snnt 14 3 1 4 Adding Device Driver Support in Linux Kernels Initrd 14 3 2 lls Ethernet Pofts su 14 3 2 1 Extracting the MAC Address Method 2 14 3 2 2 Extracting the MAC Address Method ll 15 3 2 3 Extracting the MAC Address 1 15 3 2 4 Placing MAC Addresses in
2. COT 2 2 Removing the Expansion ROM Image Remove the expansion ROM image Run flint d mst device name drom NOTE When removing the expansion ROM image you also remove Flexboot from the boot device list Depending on the OS the device name may be superseded with a prefix 11 Mellanox Technologies Rev 2 3 Preparing the DHCP Server in Linux 3 1 3 1 1 3 1 2 3 1 2 1 Preparing the DHCP Server in Linux When the boot session starts the PXE firmware attempts to bring up an adapter network link port If it succeeds to bring up a connected link the PXE firmware communicates with the DHCP server The DHCP server assigns an IP address to the PXE client and provides it with the location of the boot program Case I InfiniBand Ports Installing DHCP FlexBoot requires that the DHCP server runs on a machine which supports IP over IB 1 Prior to installing DHCP make sure that Mellanox OFED for Linux is already installed on your DHCP server see www mellanox com 2 ISC DHCP of versions above 3 1 3 have native support for IPoIB clients If using DHCP v3 1 3 the provided patch must be deployed Newer versions do not require patching 3 To download and install other supported DHCP server versions please download Mellanox OFED from www mellanox com gt Products gt InfiniBand VPI Drivers gt Linux SW Drivers and refer to docs dhcp README Standard DHCP fields holding MAC address
3. 15 4 Subnet Manager OpenSM nnne nn RR BR RR RR RR RR RR REKA RR RR RR RR RR RR RR RR RR RR Rn nn nn 16 5 m 17 Flexboot User Interface 18 6 1 Skipping FlexBoot FlexBoot User Interface and FlexBoot 18 6 2 Invoking the User 2 22 4 18 6 2 1 Browsing the User Interface Menu essen nennen nens 18 6 2 2 Accessing FlexBoot Sub 2 4 400 19 6 3 Configuring System Settings mssmosmsesssessseesseersserssersserssrrssrrssrrsnsr ss sr ss ener SR ASSR RK SR RKS RAS 19 6 3 1 Editing the Parameters using Free 19 6 3 2 Editing the Parameters using Set 20 6 3 3 Deleting Configurable Settings sse 20 6 3 4 Restoring Default Configurations sse 20 6 4 Exiting FlexBoot User 21 System Settings Configuration O PtiONS sss sss sss sss ssssssssss ssrsnsrrnnrrnnnrn nn rann rann nnn nennt nnns 22 712 System Set p uitio ro itai tente He Hte Ee Pega e etate Lond erbe dedito ceeds 22 7 1 1 22 ds Mellanox Technologie
4. EXE For a ConnectX device with ports configured as Ethernet comment out the following line hardware ethernet 00 02 c9 00 00 bb 39 Mellanox Technologies Rev 2 3 How to Bootl 8 3 Legacy Boot Priority Behavior The sections below describe various legacy boot priority behaviors based on the protocol used 8 3 1 Legacy Boot Protocol PXE The client first tries to perform PXE boot and if that fails it tries iSCSI boot The client goes to the DHCP to get an IP address The DHCP server might supply filename for PXE boot and root path for iSCSI boot if set by the user e If the iSCSI target parameters taken from the flash then the root path retrieved from the DHCP is deleted and if the Connect is set as then the values stored in the flash are used DHCP IP Enable DHCP Parameters Disable root path is supplied from either the DHCP or the flash the client is connected to the iSCSI target but no boot is performed filename is supplied a PXE boot is performed on the Operating System If PXE boot failed or no filename was supplied an iSCSI boot is performed to the connected target When the legacy boot protocol PXE is selected the following are not supported DHCP IP disable e iSCSI boot with static configurations after PXE boot Setting all iSCSI parameters from the flash including IP address 8 3 2 Legacy Boot Protocol iSCSI
5. In case of interoperability issues between iSCSI and Large Receive Offload LRO set the ib ipoib ko lro to 0 to disable LRO sbin insmod lib modules ib ib ipoib ko lro 0 44 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 11 Assign an IP address to the IB device Add either a call to ifconfig or to the DHCP client in the init file after loading the modules If you want to use the DHCP client then you need to add a call to the DHCP client in the init file after loading the IB modules For example sbin dhclient cf sbin dhclient conf ibl 12 Save the init file 13 Close initrd hostis eel iimpo l initro alo IMOSiclS rind 2 exo E gt scl 206 acing hostl gzip tmp new init ib img 14 Copy the modified initra including the IB driver which is located at tmp new init ib img gz and rename it properly Back up your current initrd file Make a new working directory and change to it Extract the initrd image Create a directory for the ConnectX EN modules and copy them A N Load the modules using the insmod executable If you do not have it in your initra add it using the following command Give your Ethernet device a static IP address 7 Add the commands for loading the copied modules into the file init at the point you wish the Ethernet driver to be loaded 8 Assign an IP address to the ConnectX EB network i
6. Mellanox FlexBoot User Manual Rev 2 3 6 2 2 Accessing FlexBoot Sub Menus To enter the required menu press Enter while the option is highlighted Figure 2 Main Menu Window System setup Firmware Image Properties netO Port 1 f4 52 14 05 1a d1 neti Port 2 4 52 14 05 1 42 Banner menu timeout Virtualization mode For further information on FlexBoot varies menus and the configuration options they provide please refer to section Configuring System Settings and its subsections 6 3 Configuring System Settings NOTE It is possible to save the changes made and exit the menu at any given time by pressing Ctrl S 6 3 1 Editing the Parameters using Free Text When a configurable setting is highlighted the instructions to edit it will be shown on screen If the setting is configurable via input the description will show the input restrictions gt To change the configuration 1 Press Enter or start typing while the setting is highlighted 2 Optional To discard the input while editing press Ctrl C 3 Press Enter and wait to verify the new value is accepted and shown In case the value entered is invalid an alert will be shown and the new value won t be accepted NOTE Although numeric values are shown in decimal base it is possible to insert values in different bases e For hexadecimal base type Ox before the number e For octal base type 0 before the number e For decimal bas
7. Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 Connect Location System setup Port main configurations iSCSI Configuration iSCSI First target Parameters Description Enable Disable connecting to the first iSCSI target This will be ignored if both DHCP IP and DHCP Parameters are enabled in iSCSI general parameters Configurable Yes built in options Persistency Persistent through AC cycle Available options Enabled 2 Disabled iSCSI Name Location System setup Port main configurations iSCSI Configuration iSCSI First target Parameters Description The iSCSI Qualified Name IQN of the first target Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI and Connect is enabled This will be the iSCSI Qualified name in the root path for the first target Value restrictions Maximum string length is 223 CHAP ID Location System setup Port main configurations iSCSI Configuration iSCSI First target Parameters Description The first iSCSI storage target Challenge Handshake Authentication Protocol CHAP ID Configurable Yes via input Persistency Persistent through AC cycle Effects If the Connect and the CHAP Authentication are enabled This will be the username used for the first target Challenge Handshake Authentication Protocol Value restrictions Maximum string length is 12
8. enable and CHAP Authentication enable the target CHAP parameters are used e If CHAP Mutual Authentication enable the Initiator CHAP parameters are used 9 UEFI PXE Boot The main differences UEFI PXE booting is determining the efi boot file editing the grub cfg and dhcpd conf files and setting the Boot Mode to UEFI Mode in the BIOS For further information please refer to the HowTo Configure PXE Server in UEFI Mode for IPoIB Network post in the Mellanox Community https community mellanox com gt Places gt Solutions gt PXE 7 Mellanox Technologies Rev 2 3 UEFI Bootl Appendix Diskless Machines A 1 Mellanox FlexBoot supports booting diskless machines NOTE The following configuration insturctions apply only to operation systems which do not include inbox driver and should be skipped in case an OS with an inbox driver is used To enable using an IB ETH driver the initrd image must include a device driver module and be configured to load that driver This can be achieved by adding the device driver module into the initrd image and loading it The initrd image of some Linux distributions such as SuSE Linux Enterprise Server and Red Hat Enterprise Linux cannot be edited prior or during the installation process If you need to install Linux distributions over Flexboot please replace your initrd images with the images found at www
9. 02 amp amp Ok a 7 18 8 JLIL P 13 Mellanox Technologies Rev 2 3 Preparing the DHCP Server Linux 3 2 3 2 1 Running the DHCP Server gt NOTE This section is applicable only when the Initiator s IP address is empty order for the DHCP server to provide configuration records for clients appropriate configuration file needs to be created By default the DHCP server looks for a configuration file called dhcpd conf under etc You can either edit this file or create a new one and provide its full path to the DHCP server using the c f flag See a file example at docs dhepd conf of this package The DHCP server must run on a machine which has loaded the IPoIB module To run the DHCP server from the command line enter dhcpd IB network interface name gt Example hostl dhcpd ibO d Adding Device Driver Support in Linux Kernels Initrd NOTE A DHCP client can be used if you need to prepare a diskless machine with an IB driver See Step 11 under Example Adding an IB Driver to initrd Linux Case Ethernet Ports When a FlexBoot client boots it sends the DHCP server various information including its DHCP hardware Ethernet address MAC The MAC address is 6 bytes long and it is used to distinguish between the various DHCP sessions Extracting the MAC Address Method I AII Mellanox Ethernet NICs have a label on the printed side of the adapter card
10. Persistent through AC cycle Effects If the Connect and the CHAP Mutual Authentication are enabled This will be the reverse username used for the iSCSI initiator Challenge Handshake Authentication Protocol Value restrictions Maximum string length is 128 31 Mellanox Technologies Rev 2 3 System Settings Configuration Options 7 6 2 CHAP Secret Location System setup Port main configurations iSCSI Configuration iSCSI Initiator Parameters Description The iSCST initiator Challenge Handshake Authentication Protocol CHAP secret password Configurable Yes via input Persistency Persistent through AC cycle Effects If the Connect and the CHAP Mutual Authentication are enabled This will be the reverse password used for the iSCSI initiator Challenge Handshake Authentication Protocol Value restrictions String length should be 0 or 12 to 16 7 6 3 IPv4 Address Location System setup Port main configurations iSCSI Configuration iSCSI Initiator Parameters Description iSCSI initiator s IPv4 address Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI This will be set as the IP address of the initiator Value restrictions Should be in IPv4 format 7 6 4 IPv4 Primary DNS Location System setup Port main configurations iSCSI Configuration iSC
11. 29 725 4SGSlI General Parameters reitera Rer eet re Ug tate Pan i ee vg Sener 29 7 51 DACP Pe RR 30 1 5 2 NDHOPSP Aall St lS a aae avenae va teat 30 7 5 9 IP terere tenete dete ee edle eoe de ne 30 75 4 CHAP Authentication eee te eet pete tice pe tice be dd da 30 7 5 5 CHAP Mutual Authentication 4 00 ss arr enne 30 7 6 iSCSI Initiator Parameters 0 1 000 rss or rr ner RKA BRA BKK RAR RK KR 31 7 61 Ane NN ARA ARA TARA EDR 31 7 62 CMAP SeCrEt 32 TOS 0 eias 32 7 6 4 4 Primary DNS esiisa 32 7 6 5 4 Default Gateway 2 2 0 1 00000 enne snnt ARR RK RR RK KR 32 7 66 150951 Names siste de eto umet tte t elec eta 33 7 627 sSu bnetMask S odds frate ior addet bo eere rta 33 7 7 iSCSI First Target Parameters sse eene nennen RK RR RK nennen 34 7 7 1 Boot LUN renaras 34 VOTES 35 LAB USCS Name uii eite ti ette praece ierit eevee is 35 iM Mellanox Technologies Table of Contents Rev 2 3 TAS OCHABJID a xe diete eet ie OE des 3
12. following menu you can set the following options e Flexboot Version e Family Firmware Version Figure 6 Firmware Image Properties Menu Firmware Image Properties Flexboot version 7 2 1 Flexboot Version Location System setup Firmware Image Properties Description Flexboot driver version information Configurable No 7 2 2 Family Firmware Version Location System setup Firmware Image Properties Description Device s firmware version information Configurable No 24 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 7 3 Main Configuration Initiator Parameters iSCSI First Target Parameters are applicable to ConnectX 3 ConnectX 3 Pro adapter cards only E 4 NOTE iSCSI Configuration menu and all its sub menus iSCSI General Parameters iSCSI In the following menu you can set the following options e Blink LEDs e MAC Address Figure 7 Main Configuration netO Port 1 4 52 14 05 1 41 NIC Configuration iSCSI Configuration Blink leds 7 3 1 Blink LEDs NOTE Blink LEDs will not appear in the boot menu if the feature is not supported by the firmware NOTE This configuration is applied when presssing Enter so it may take longer time to save than other configurations Location System setup Port main configurations Description Specifies the number of seconds the LEDs on physical network port should blink t
13. that has the card serial number and the card MAC address Figure 1 NIC Label 474 Serial S N MT0742X01849 abc nl x1 PIN MNKH 8 XTC EHE UAE UEM E AI MAC 000269000174 MAC EH E OH LUE ANN HI Made In IL 2 x 9 L 919 9222 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 3 2 2 Extracting the MAC Address Method II The six bytes of MAC address can be captured from the display upon the boot of the ConnectX device session as shown in the figure below lexBoot v3 4 637 PCI 00 03 0 starting execution ok FlexBoot initialising devices Initialising completed lexBoot v3 4 637 FlexBoot ht me llano Features HTTP 15 5 TP ULAN ELF MBOOT PXE bzImage COMBOOT Menu PXEXT netO 4 52 14 05 1 42 sing ConnectX 3 on PCIOO0 03 0 open Link down TX O TXE O RX O RXE 01 Link status Unknown http 7 ipxe org 1a086101 1 aiting for link up on netO 3 2 3 Extracting the Address Method III gt In case the previous methods fail to work perform the following as a last resort 1 Start mst hostl mst start hostl mst status The following MFT commands assume that the Mellanox Firmware Tools MFT package has been installed on the client machine 2 Obtain the MAC using the device name The device name will be of the form dev mst mt lt dev_id gt _pci _crOlconf0 flint d lt MST_DEVICE_NAME gt q Assuming that FlexBoot
14. 0 RHEL6 4 x86_64 DVD1 images pxeboot vmlinuz initrd http 14 7 6 30 RHEL6 4 x86 64 DVDl images pxeboot initrd img boot iSCSI Mellanox FlexBoot enables an iSCSI boot of an OS located on a remote iSCSI target It has a built in iSCSI initiator which can connect to the remote iSCSI target and load from it the kernel and initrd There are two instances of connection to the remote iSCSI target the first is for get ting the kernel and initrd via FlexBoot and the second is for loading other parts of the OS via initrd If you choose to continue loading the OS after boot through the HCA device driver please verify that the initrd image includes the HCA driver NOTE To boot via iSCSI Flexboot must be selected from the BIOS boot menu it will be shown as BEV entry Do not look for the entry in the BVC table aa Installing to iSCSI Target It is recommended to install to an iSCSI target using the same port performing the following 1 Connect to the iSCSI target to retrieve various information from it Due to the step above FlexBoot will register the remote target by hooking BIOS interrupt 13 2 Download the installation image using the same port by supplying the file name in the client s DHCP configuration file along with the root path 38 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 8 2 2 Configuring an iSCSI Target Linux Environment 8 2 2 1 Prerequisites e Make sure that an iS
15. 5 FAL GEAP S6SCIOD odd e dafs d v cee 35 10 AMPrAdGI688 mede eate epe side iis 36 LAL a a Pap LP e RUE end Uo dte 36 Si HOW to BOOT 37 84d PRE BOOD pp eroi iei iie 37 8 1 1 Prerequisites xe tu EU n 37 8 4 2 Starting Boot een esiti e aee opa ete e env ae 37 849 How to Boot IO ESXi cette HERD tede tid eed da 38 8 1 4 How to Boot Using HTTP sess 38 8 0 MSCS DEM 38 8 2 1 Installing to iSCSI 22 rer rn 38 8 2 2 Configuring an iSCSI Target in Linux 0 39 8 3 Legacy Boot Priority Behavior nennen ener nenne 40 8 3 1 Legacy Boot Protocol PAE 40 8 3 2 Legacy Boot Protocol 15 61 4 40000 00 40 8 3 3 CHAP Authentication and CHAP Mutual 41 EllZI S DEDI 41 Appendix A Diskless Machines esssssssssnssnssnsnnssnnnnssn nns sn nns eene nn mnnn nn nnn inne nnn inne nns 42 Gasel InfiniBand Pep tei Hp eoa ena uade 42 A 1 1 X Example Adding an IB Driver to initrd 43 LLL Mel
16. 8 CHAP Secret Location System setup Port main configurations iSCSI Configuration iSCSI First target Parameters Description The Challenge Handshake Authentication Protocol secret CHAP password of the first iSCSI storage target Configurable Yes via input Persistency Persistent through AC cycle Effects If the Connect and the CHAP Authentication are enabled This will be the N 35 Mellanox Technologies Rev 2 3 System Settings Configuration Options 7 7 6 7 7 7 36 J password used for the first target Challenge Handshake Authentication Protocol Value restrictions String length should be 0 or 12 to 16 IP Address Location System setup Port main configurations iSCSI Configuration iSCSI First target Parameters Description The IP address of the first iSCSI target Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI and Connect is enabled This will be the IP address in the root path for the first target Value restrictions Should be in IPv4 format TCP Port Location System setup Port main configurations iSCSI Configuration iSCSI First target Parameters Description TCP Port number of first iSCSI target Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI and Con
17. CSI Target is installed on your server side You can download and install an iSCSI Target from the following location http sourceforge net projects iscsitarget files iscsitarget e Dedicate a partition on your iSCSI Target on which you will later install the operating system e Configure your iSCSI Target to work with the partition you dedicated If for example you choose partition dev sda5 then edit the iSCSI Target configuration file etc ietd conf to include the following line under the iSCSI Target iqn line Lun 0 Path dev sda5 Type fileio Example of an iSCSI Target iqn line Target iqn 2007 08 7 3 4 10 iscsiboot e Start your iSCSI Target Example hostl etc init d iscsitarget start 8 2 2 2 Configuring the DHCP Server to Boot From an iSCSI Target 1 Configure DHCP as described in section Preparing the DHCP Server in Linux 2 Edit your DHCP configuration file etc dhcpd conf and add the following lines for the machine s you wish to boot from the iSCSI target Filename Gorton VASSSULSISSCSL ieget 1195555166 Cargat cpm p The following is an example for configuring an IB ETH device to boot from an iSCSI target host hostl filename EGGEN Laesi eget isos5cgei Targat acim 9 For ConnectX device with ports configured as InfiniBand comment out the follow ing line option dhcp client identifier Pe SOU OOS UUs OOP UU
18. H DAMAGE Mellanox TECHNOLOGIES Mellanox Technologies 350 Oakmead Parkway Suite 100 Sunnyvale CA 94085 U S A www mellanox com Tel 408 970 3400 Fax 408 970 3403 Copyright 2015 Mellanox Technologies All Rights Reserved Mellanox Mellanox logo BridgeX ConnectX Connect IB CoolBox CORE Direct GPUDirect InfiniBridge Infini Host InfiniScale amp Kotura Kotura logo Mellanox Connect Accelerate Outperform logo Mellanox Federal Systems Mellanox Open Ethernet Mellanox Virtual Modular Switch MetroDX MetroX MLNX OS Open Ethernet logo PhyX Scalable HPC SwitchX TestX The Generation of Open Ethernet logo UFM Virtual Protocol Interconnect Voltaire and Voltaire logo are registered trademarks of Mellanox Technologies Ltd CyPU ExtendX FabricIT FPGADirect HPC X Mellanox Care Mellanox CloudX Mellanox NEO Mellanox Open Ethernet Mellanox PeerDirect NVMeDirect StPU Spectrum Switch IB Unbreakable Link are trademarks of Mellanox Technologies Ltd All other trademarks are property of their respective owners 2 Document Number 3218 Mellanox Technologies Table of Contents Rev 2 3 Table of Contents Document Revision 8 1
19. LAN Configurable Yes built in options Persistency Persistent through AC cycle Available options 1 Enabled 2 Disabled 7 4 5 Option ROM NOTE Option ROM menu is applicable to ConnectX 3 ConnectX 9 3 Pro adapter cards only Location System setup Port main configurations NIC Configuration Description Controls the enablement of legacy Boot Protocols in the boot legacy pro If disabled no legacy boot protocol is chosen When set the boot protocol will be chosen by legacy boot protocol attribute Configurable Yes built in options Persistency Persistent through AC cycle Reboot required Yes reboot is needed in order for this change to apply Available options 1 Enabled 2 Disabled 7 4 6 Virtual LAN ID Location System setup Port main configurations NIC Configuration Description Uses the VLAN tag with the selected boot protocol Configurable Yes via input If virtual LAN mode is enabled 28 J Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 Value restrictions The minimum integer value is 1 and Maximum integer value is 4094 7 4 7 Virtual LAN Mode Location System setup Port main configurations NIC Configuration Description Enable Disable VLAN tagging on the selected boot protocol Configurable Yes built in options Persistency Persistent through AC cycle Avail
20. Mellanox TECHNOL Connect Accelerate Outperform Mellanox FlexBoot User Manual Rev 2 3 www mellanox com Mellanox Technologies NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT S AND OR THE SYSTEM USING IT THEREFORE MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT INDIRECT SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES OF ANY KIND INCLUDING BUT NOT LIMITED TO PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OFLIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY FROM THE USE OF THE PRODUCT S AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUC
21. OTE The following procedure modifies critical files used in the boot procedure It must be executed by users with expertise in the boot process Improper application of this pro cedure may prevent the diskless machine from booting 1 Back up your current initrd file 2 Make a new working directory and change to it hostis make um io hostis Cel enya aimaieiecl_alo 3 Extract the initrd MOSIELS Cui lt imulieimel cele ic l The initrd files are now located at tmp initrd_ib 4 Create a directory for the InfiniBand modules and copy them hostl1 mkdir tmp initrd ib lib modules ib hostl1 lib modules uname r updates kernel drivers host1 cp infiniband core ib addr ko tmp initrd ib lib modules ib 5 15 cp infiniband core ib core ko tmp initrd ib lib modules ib host1 cp infiniband core ib mad ko tmp initrd ib lib modules ib host1 cp infiniband core ib sa ko tmp initrd ib lib modules ib host1 cp infiniband core ib cm ko tmp initrd ib lib modules ib hostis cp infiniband core ib uverbs ko tmp initrd ib lib modules ib hostl cp infiniband core ib ucm ko tmp initrd ib lib modules ib host1 cp infiniband core ib umad ko tmp initrd ib lib modules ib host1 cp infiniband core iw cm ko tmp initrd ib lib modules ib hostl cp infiniband core rdma cm ko tmp initrd ib lib modules ib hostl cp infiniband core rdma ucm ko tmp initrd ib lib modules ib hostis cp net mlx4 mlx4 core
22. SI Initiator Parameters Description The iSCSI initiator Primary DNS IP Address Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI This will be set as the DNS for the IP address of the initiator Value restrictions Should be in IPv4 format 7 6 5 IPv4 Default Gateway Location System setup Port main configurations iSCSI Configuration iSCSI Initiator Parameters Description The iSCSI initiator default Gateway IP address Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI This will be set as the default gateway IP address of the initiator 32 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 Value restrictions Should be in IPv4 format 7 6 6 iSCSI Name Location System setup Port main configurations iSCSI Configuration iSCSI Initiator Parameters Description The initiator s iSCSI Qualified Name IQN Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI This will be set as the iSCSI Qualified Name of the initiator Value restrictions Maximum string length is 223 Subnet Mask Location System setup Port main configurations iSCSI Configuration iSCSI Initiator Parameters Description The iSCSI in
23. The client first tries to perform iSCSI boot and if that fails it tries PXE boot Upon iSCSI boot one of the following flows occurs e If all iSCSI configuration is taken from the flash the client waits 25 seconds for the switches to be configured and then tries to perform an iSCSI boot hook to target and then boot If the boot fails the client exits DHCP IP disable Or 1 If only the TCP parameters need to be retrieved from the DHCP DHCP IP enable DHCP Parameters disable then the IP address is taken from the DHCP the root path retrieved from the DHCP is deleted and if the Connect is set as Enable then the values stored in the flash are used Hook to the iSCSI target Try to boot to the iSCSI target If iSCSI boot failed and DHCP supplied a filename try to PXE boot If PXE boot failed exit AeA N When the legacy boot protocol iSCSI is selected the following is not supported e PXE boot after iSCSI boot with static configurations Setting all iSCSI parameters from the flash including IP address 40 Mellanox Technologies Mellanox FlexBoot User Manual 2 3 8 3 3 Authentication and Mutual Authentication iSCSI boot allows CHAP security option if the following conditions are met e If Connect disable or CHAP Authentication disable the server ignores both the CHAP Authentication and CHAP Mutual Authentication If Connect
24. User Interface 6 Flexboot User Interface NOTE Boot menu is displyed as READ ONLY if the HCA card does not support flash configuration 6 1 Skipping FlexBoot FlexBoot User Interface and FlexBoot Port You can skip FlexBoot in any of the given options below Following system POST when a FlexBoot advertisement banner and configuration prompt is displayed Press any key other than CTRL B When FlexBoot execution starts as depicted in the Initializing Devices screen output a UI prompt for escaping shortly follows After the BIOS called BEV_ENTRY Press ESC when prompted 2 3 seconds Skip a specific port when booting starts Press CTRL C 6 2 Invoking the User Interface Shortly after completion of the POST Power On Self Test sequence of the computer the user will be prompted to press CTRL B to invoke Mellanox FlexBoot User Interface The user has few seconds to press CTRL B before the message disappears Mellanox FlexBoot v3 4 435 iPXE http ipxe org 05 00 0 D300 PCI3 00 PnP PMM 002676C0 002816CO D300 Press Ctrl B to configure MLNX FlexBoot 3 4 435 PCI 05 00 0 6 2 1 Browsing the User Interface Menu To browse the menu use the following keys Table 3 PXE Browsing Keys Key Operation Arrows 11 Browse options Highlighted option is current Enter On forms to enter menu On configurable settings edit configuration Esc Exit current menu OE AN y Mellanox Technologies
25. able No Banner Menu Timeout Location System setup Description Defines the Banner s timeout Configurable Yes via input If virtualization mode is SR IOV Persistency Persistent through AC cycle Reboot required Available options Yes reboot is needed in order for this change to apply 1 14 sec Virtualization Mode Reboot required Location System setup Description Specifies the type of virtualization used by the controller on all ports Only valid for adapters that support SR IOV virtualization mode Configurable Yes built in options Persistency Persistent through AC cycle Yes reboot is needed in order for this change to apply Available options 1 None 2 SR IOV Number of Virtua Functions Location System setup Description The number of virtual functions advertised and usable by the driver Only valid for adapters that support SR IOV virtualization mode Configurable Yes via input If virtualization mode is SR IOV Mellanox Technologies Rev 2 3 System Settings Configuration Options Persistency Persistent through AC cycle Reboot required Yes reboot is needed in order for this change to apply Value restrictions Minimum integer value is 0 Maximum integer value is the value supported by the firmware installed 7 2 Firmware Image Properties In the
26. able options 1 Enabled 2 Disabled 7 5 iSCSI General Parameters NOTE iSCSI General Parameters menu is applicable to ConnectX 3 ConnectX 3 Pro adapter cards only In the following menu you can set the following options e DHCP IP e DHCP Parameters e IP Version e CHAP Authentication e CHAP Mutual Authentication Figure 10 iSCSI General Parameters iSCSI General Parameters DHCP IP Enabled DHCP Parameters Enabled CHAP Authentication Enabled CHAP Mutual Authentication Enabled DHCP IP 29 Mellanox Technologies Rev 2 3 System Settings Configuration Options 7 5 1 7 5 2 7 5 3 7 5 4 7 5 5 30 DHCP Location System setup Port main configurations iSCSI Configuration iSCSI General Parameters Description When the DHCP is enabled the initiator and target s IP netmask gateway and DNS will be taken from DHCP instead of from the menu Configurable Yes DHCP Parameters Location System setup Port main configurations iSCSI Configuration iSCSI General Parameters Description When the DHCP IP and DHCP Parameters are enabled all the initiator and target s parameters are taken from DHCP instead of from the menu Configurable Yes IP Version Location System setup Port main configurations iSCSI Configuration iSCSI General Parameters Description Specifies whether 4 or IPv6 network address
27. b bin hosti cp usr bin expr tmp initrd ib bin hostl cp sbin ifconfig tmp initrd ib bin hostl cp bin hostname tmp initrd ib bin 9 Create a configuration file for the DHCP client and place it under tmp initrd ib sbin Example of the dclient conf The value indicates a hexadecimal number For a ConnectX device interface ib0 send dhcp client identifier Le 00 2010 010 02 8 9 0 10 0 8 0 2 amp 8 0 0 e 0 0 9 102 6 9 e 0 LOS 10 Add the commands for loading the copied modules into the file init at the point you wish the IB driver to be loaded Please pay attention to the following commands order for loading modules as it is critical Loading ipoib_helper ko is not required for all OS kernels echo loading ipv6 sbin insmod lib modules ipv6 ko echo loading IB driver sbin insmod lib modules ib ib addr ko sbin insmod lib modules ib ib core ko sbin insmod lib modules ib ib_mad ko sbin insmod lib modules ib ib_sa ko sbin insmod lib modules ib ib_cm ko sbin insmod lib modules ib ib uverbs ko sbin insmod lib modules ib ib_ucm ko sbin insmod lib modules ib ib_umad ko sbin insmod lib modules ib iw_cm ko sbin insmod lib modules ib rdma_cm ko sbin insmod lib modules ib rdma_ucm ko sbin insmod lib modules ib mlx4 core ko sbin insmod lib modules ib mlx4 ib ko sbin insmod lib modules ib ib mthca ko
28. e type the number as is 19 Mellanox Technologies Rev 2 3 Flexboot User Interface 6 3 2 6 3 3 6 3 4 Figure 3 Free Text Editing Example NIC Configuration Legacy boot protocol Boot retry count No retries Option ROM Enabled Virtual LAN mode Enabled Virtual LAN ID Editing the Parameters using Set Values When a configurable setting is highlighted the instructions to edit it will be shown on screen If the setting is configurable with built in options the description will be Select to change value gt To change the configuration 1 Press Enter while setting is highlighted 2 Keep pressing Enter to choose the desired value Deleting Configurable Settings Configurable settings can be deleted Value will only be deleted from the current configuration and not from the flash If the configuration has a default value it will be set when deleting the current value gt To delete a configurable setting 1 Press Ctrl D while setting is highlighted Restoring Default Configurations All configurations can be restored to the default values set in the device All configurable values will be modified to default or deleted if there is no default value in the device gt To restore defaults per port e Press Ctrl R on the main menu of the port only when applicable To restore defaults per device e Press Ctrl R on System setup menu when applicable 20 Mellanox Techn
29. es are not large enough to contain an IPoIB hardware address To overcome this problem DHCP over InfiniBand messages convey a client identifier field in DHCP option 61 used to identify the DHCP session This client identifier field can be used to associate an IP address with a client identifier value such that the DHCP server will grant the same IP address to any client that conveys this client identifier NOTE In the DHCP discover packets flexboot sends hlen 6 rather than 0 chaddr will be the ethernet mac address rather than zeros In the DHCP server the GUID in the client identifier can still be used to identify clients Configuring the DHCP Server For ConnectX Family Devices When a FlexBoot client boots it sends the DHCP server various information including its DHCP client identifier This identifier is used to distinguish between the various DHCP sessions The value of the client identifier is composed of a prefix ff 00 00 00 00 00 02 00 00 02 c9 00 and an 8 byte port GUID all separated by colons and represented in hexadecimal digits Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 3 1 2 2 Extracting the Port GUID Method I gt To obtain the port GUID 1 Start mst hostl mst start hostl mst status The following MFT commands assume that the Mellanox Firmware Tools MFT package has been installed on the client machine 2 Obtain the Port GUID using the device na
30. ev 2 3 Mellanox FlexBoot 1 1 1 2 Mellanox FlexBoot PXE Overview Mellanox FlexBoot is a multiprotocol remote boot technology FlexBoot supports remote Boot over InfiniBand BoIB and over Ethernet Using Mellanox Virtual Protocol Interconnect VPI technologies available in ConnectX adapters FlexBoot gives IT Managers the choice to boot from a remote storage target iSCSI target or a LAN target Ethernet Remote Boot Server using a single ROM image on Mellanox ConnectX products FlexBoot is based on the open source project iPXE available at http www ipxe org FlexBoot first initializes the adapter device senses the port protocol Ethernet or InfiniBand and brings up the port Then it connects to a DHCP server to obtain its assigned IP address and network parameters and also to obtain the source location of the kernel OS to boot from The DHCP server instructs FlexBoot to access the kernel OS through a TFTP server an iSCSI target or some other service For an InfiniBand port Mellanox FlexBoot implements a network driver with IP over IB acting as the transport layer IP over IB is part of the Mellanox OFED for Linux software package see www mellanox com gt Products gt InfiniBand VPI Drivers gt FlexBoot Supported Mellanox Adapter Devices and Firmware The package supports the following HCA and firmware versions Table 2 Supported Mellanox Adapter Devices and Firmware HCAs Firm
31. evision History Table 1 Document Revision History Revision Date Description 2 3 September 10 2015 e Added the following sections e Banner Menu Timeout e PKey Value e Updated the following sections e Supported Mellanox Adapter Devices and Firmware e Main Configuration e Blink LEDs e NIC Configuration e Prerequisites e Removed the following section e FlexBoot Package 2 2 July 08 2015 e Added UEFI PXE Boot June 23 2015 e Updated the Value restrictions description in section Number of Virtual Functions May 15 2015 e Added the following sections e DHCP IP e DHCP Parameters e IP Version e CHAP Authentication e CHAP Mutual Authentication e Legacy Boot Protocol PXE e Legacy Boot Protocol iSCSI e CHAP Authentication and CHAP Mutual Authentication e Updated the following sections e Invoking the User Interface e CHAP ID CHAP Secret e Removed the following sections Max Number of VFs Supported e Virtual MAC Address 2 1 February 09 2015 Updated the note in section Running the DHCP Server e Removed sections e iSCSI General Parameters e DHCP Parameters e DHCP IP e DHCP Parameters e IP Version Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 Revision Date Description 2 0 January 15 2015 Initial release of the restructured new User Manual N 9 Mellanox Technologies R
32. ing will be used for iSCSI initiator and targets Configurable No CHAP Authentication Location System setup Port main configurations iSCSI Configuration iiSCSI General Parameters Description Enables CHAP Authentication for iSCSI initiator Configurable Yes CHAP Mutual Authentication Location System setup Port main configurations iSCSI Configuration iSCSI General Parameters Description Enables mutual CHAP authentication between the iSCSI initiator and target Configurable Yes Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 7 6 iSCSI Initiator Parameters E NOTE iSCSI Initiator Parameters menu is applicable to ConnectX 3 ConnectX 3 Pro adapter cards only In the following menu you can set the following options e CHAP ID CHAP Secret Pv4 Address Pv4 Primary DNS e IPv4 Default Gateway e iSCSI Name e Subnet Mask Figure 11 iSCSI Initiator Parameters Initiator Parameters CHAP gt Bpecif led gt CHAP Secret specified gt IPv4 Address specified IPv4 Primary DNS specified IPv4 Default Gateway specified iSCSI Name specified gt Subnet mask specified gt CHAP ID 7 6 1 CHAP ID Location System setup Port main configurations iSCSI Configuration iSCSI Initiator Parameters Description The iSCSI initiator Challenge Handshake Authentication Protocol CHAP ID Configurable Yes via input Persistency
33. is connected via Port 1 then the MAC address is 00 02 9 0 7 11 3 2 4 Placing Addresses etc dhcpd conf The following is an excerpt of a etc dhcpd conf example file showing the format of representing a client machine for the DHCP server running on a Linux machine INOS loxexsne ll next server 11 4 3 7 filename pxelinux 0 fixed address 11 4 3 130 hardware ethernet 00 02 9 0 78 12 15 Mellanox Technologies Rev 2 3 Subnet Manager OpenSMI 4 Subnet Manager OpenSM gt NOTE This section applies to ports configured as InfiniBand only FlexBoot requires a Subnet Manager to be running on one of the machines in the IB network OpenSM is part of the Mellanox OFED for Linux software package can be used to accomplish this Note that OpenSM may be run on the same host running the DHCP server but itis not mandatory For large InfiniBand clusters gt 100 nodes it is recommended to use OpenSM caching For further information please refer to the Mellanox OFED User Manual A AEN Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 5 BIOS Configuration The expansion ROM image presents itself to the BIOS as a boot device As a result the BIOS will add to the list of boot devices MLNX FlexBoot lt ver gt for a ConnectX family device The priority of this list can be modified through BIOS setup N 17 Mellanox Technologies Rev 2 3 Flexboot
34. itiator Subnet Mask for an IPv4 initiator configuration Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI This will be set as the subnet mask for the IP address of the initiator Value restrictions Should be in IPv4 format 33 Mellanox Technologies Rev 2 3 System Settings Configuration Options 7 7 iSCSI First Target Parameters NOTE iSCSI First Target Parameters menu is applicable to ConnectX 3 ConnectX 3 Pro adapter cards only In the following menu you can set the following options Boot LUN e Connect e iSCSI Name e CHAP ID e CHAP Secret e IP Address e TCP Port iSCSI First target Parameters Boot LUN 0 Connect Disabled iSCSI Name lt not specified gt CHAP ID not specified CHRP Secret lt not specified gt IP Address lt not specified gt TCP Port CHAP ID 7 7 1 Boot LUN Location System setup Port main configurations iSCSI Configuration iSCSI First target Parameters Description The boot Logical Unit Number LUN on the first iSCSI storage target Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI and Connect is enabled This will be the boot LUN in the root path for the first target Value restrictions The minimum integer value is 0 and the maximum is 1844674407370955 1615 UNE
35. ko tmp initrd ib lib modules ib hostl1 cp infiniband hw mlx4 mlx4 ib ko tmp initrd ib lib modules ib host1 cp infiniband hw mthca ib mthca ko tmp initrd ib lib modules ib host1 cp infiniband ulp ipoib ipoib helper ko tmp initrd ib lib modules ib host1 cp infiniband ulp ipoib ib ipoib ko tmp initrd ib lib modules ib 5 Load the IPv6 module If you do not have it in your initrd add it using the following command host1 cp lib modules unam r kernel net ipv6 ipv6 ko a J Mellanox Technologies Rev 2 3 UEFI Bootl tmp initrd_ib lib modules 6 Load the modules using the insmod executable If you do not have it in your initrd add it using the following command hostl cp sbin insmod tmp initrd ib sbin 7 Optional Give your IB device a static IP address hostis cio abin irceonitig ante lom 8 Optional Obtain an IP address for the IB device through DHCP Copy the DHCP client which was compiled specifically to support IB To continue with this step DHCP client needs to be already installed on the machine you are working with Copy the DHCP client file and all the relevant files as described below hostl cp path to DHCP client dhclient tmp initrd ib sbin hosti cp path to DHCP client dhclient script tmp initrd ib sbin hostl mkdir p tmp initrd ib var state dhcp 5 1 touch tmp initrd ib var state dhcp dhclient leases hostl cp bin uname tmp initrd i
36. lanox Technologies Rev 2 3 Table of Contents List of Figures Figure 1 NIG ec rte dite eiecit gears ae E e dead 14 Figure 2 Main Menu 19 Figure 3 Free Text Editing Example eed ette et etes 20 Figure 4 Exit Men obtu ne eee tare ERR ERE IURE OP ERR Pru e 21 Figure 5 System Setup KR RK KR RK KR RK Ken 22 Figure 6 Firmware Image Properties 24 Figure 7 Main RR RA RR RR RR RR RR RA RR RR KR RS SR RS SR RK 25 Figure 8 Ethernet NIC Configuration 1 1 1 ror anor snor eene RK RR RAR RR KR RK 26 Figure 9 InfiniBand NIC Configuration ARR RR DRAR 27 Figure 10 iSCSI General 04044 0 RK KR RK KR RK KR RK ens 29 Figure 11 iSCSI Initiator 2 4 441 10 on ANNAN FANAN ka 31 C MMC Mellanox Technologies Table of Contents Rev 2 3 List of Tables Table 1 Document Revision 8 Table 2 Supported Mellanox Adapter Devices and 10 Table g PXE Browsing Keys i ace o E et ede eed eie res 18 5 Mellanox Technologies Rev 2 3 Mellanox FlexBoot Document R
37. me The device name will be of the form dev mst mt lt dev_id gt _pci _crOlconf0 flint d lt MST_DEVICE_NAME gt q Assuming that FlexBoot is connected via Port 1 then the Port GUID is 00 02 c9 03 00 00 10 39 3 1 2 3 Extracting the Port GUID Method II An alternative method for obtaining the port GUID involves booting the client machine via Flex Boot This requires having a Subnet Manager running on one of the machines in the InfiniBand subnet The 8 bytes can be captured from the boot session as shown in the figure below LNX FlexBoot 3 4 454 PCI 07 00 0 starting execution ok 3 LNX FlexBoot 3 4 454 initialising devices Initialising completed Mellanox FlexBoot v3 4 454 iPXE 1 0 0 53438 Open Source Network Boot Firmware Features ULAN HTTP iSCSI DNS TFTP bzImage COMBOOT ELF MBOOT PXE PXEXT Menu netO GUID 4 52 14 03 00 6e ea f1 MAC f4 52 14 6e ea f1 Using ConnectxX3 Pro open Link down 0 RXE 01 Link status The socket not connected http ipxe org 38136001 Waiting for link up on net0 3 1 2 4 Placing Client Identifiers in etc dhcpd conf The following is an excerpt of a etc dhcpd conf example file showing the format of representing a client machine for the DHCP server Rost aoste i next server 11 4 3 7 filename pxelinux 0 fixed address 11 4 3 130 option dhcp client identifier 8 0 0 amp 0 0 0122 amp 000 amp 000 amp e 00 8 00 amp
38. mellanox com gt Products gt InfiniBand VPI Drivers gt FlexBoot Download Tab All OSes which doesn t include inbox driver should do the following OSes which include the inbox driver should skip these configurations Case I InfiniBand Ports The IB driver requires loading the following modules in the specified e ib_addr ko e ib core ko e ib mad ko e ib sa ko e ib cm ko e ib uverbs ko e ib ucm ko e ib umad ko e iw cm ko e rdma cm ko e rdma ucm ko e mlx4 core ko e mlx4 ib ko e ib mthca ko e ipoib helper ko this module is not required for all OS kernels Please check the release notes e jb ipoib ko 42 J Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 A 1 1 Example Adding an IB Driver to initrd Linux A 1 1 1 Prerequisites e The FlexBoot image is already programmed on the HCA card e The DHCP server is installed configured and connected to the client machine e An initrd file e To add an IB driver into initrd you need to copy the IB modules to the diskless image Your machine needs to be pre installed with a Mellanox OFED for Linux ISO image available for download from www mellanox com gt Products gt InfiniBand VPI Drivers gt Linux SW Drivers that is appropriate for the kernel version the diskless image will run The remainder of this section assumes that Mellanox OFED has been installed on your machine A 1 1 2 Adding the IB Driver to the initrd File N
39. nect is enabled This will be the TCP Port in the root path for the first target Value restrictions The minimum integer value is 1 and the maximum is 65535 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 8 How to Boot 8 1 PXE Boot 8 1 1 Prerequisites Make sure that your client is connected physically and logically to the server s The FlexBoot image is already programmed on the adapter card ForInfiniBand ports only Start the Subnet Manager Configure and start the DHCP server Configure and start at least one of the services such as iSCSI 8 1 2 Starting Boot Boot the client machine and enter BIOS setup to configure MLNX FlexBoot to be the first on the boot device priority list fails it switches to boot from Port 2 Note also that the driver waits up to 10 seconds for port p NOTE On dual port network adapters the client first attempts to boot from Port 1 If this a come up and up to 90 seconds to assign an IP from the DHCP server If MLNX FlexBoot was selected through BIOS setup the client will boot from FlexBoot The client will display FlexBoot attributes sense the port protocol Ethernet or InfiniBand The port sensing occurs only when the port is connected to a QSFP connector In case of an InfiniBand port the client will also wait for port configuration by the Subnet Manager If auto sensing proto col fails the port will be configured as an InfiniBand po
40. nterface 9 Save the init file 10 Close initrd 11 Copy the modified initrd including the Ethernet driver which is located at tmp new_init_en img gz and rename it properly amp J Mellanox Technologies
41. o assist with port identification Only valid for adapters with blink LEDs Configurable Yes via input Persistency Valid only for current cycle Effects Blink LEDs duration value of the port will be as set in the current cycle 25 Mellanox Technologies Rev 2 3 System Settings Configuration Options Value restrictions The minimum integer value is 0 and the maximum is 15 7 3 2 MAC Address Location System setup Port main configurations Description Permanent MAC address assigned during manufacturing Configurable No 7 4 NIC Configuration In the following menu you can set the following options Legacy Boot Protocol Boot Retry Count PKey Value Wake on LAN Option ROM Virtual LAN ID Virtual LAN Mode Figure 8 Ethernet NIC Configuration NIC Configuration Legacy boot protocol woven sees en nU Boot retry count No retries Option ROM Enabled Virtual LAN mode Disabled Legacy boot protocol 26 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 Figure 9 InfiniBand Configuration NIC Configuration PKey Value Legacy boot protocol Boot retry count No retries Option ROM Enabled PKey Value 7 4 1 Legacy Boot Protocol Location System setup Port main configurations NIC Configuration Description Use a non UEFI network boot protocol The chosen protocol is tried first if boot fails the other pro
42. ologies Mellanox FlexBoot User Manual Rev 2 3 6 4 Exiting FlexBoot User Interface To exit FlexBoot User Interface press ESC from the System Setup menu Figure 4 Exit Menu Save configurations 21 Mellanox Technologies Rev 2 3 System Settings Configuration Options 7 System Settings Configuration Options Below are listed the attributes shown in varies FlexBoot User Interface menus 7 1 System Setup In the following menu you can set the following options e Bus Device Function e Chip Type e Device Name e PCI Device ID e Banner Menu Timeout e Virtualization Mode e Number of Virtual Functions Figure 5 System Setup Menu System setup Firmware Image Properties netO Port 1 f4 52 14 05 1a d1 neti Port 2 f4 52 14 05 1a d2 Banner menu timeout Virtualization mode 7 1 1 Bus Device Function Location System setup Description Specifies the BIOS assigned PCI Bus Device Function identifier of the card Configurable No 22 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 N 23 Chip Location System setup Description Specifies the chip type Configurable No Device Name Location System setup Description The device s product name Configurable No PCI Device ID Location System setup Description The PCI Device ID of the controller Configur
43. rt After configuring the IB ETH port the client attempts to connect to the DHCP server to obtain an IP address and the source location of the kernel OS to boot from For ConnectX InfiniBand Mellanox FlexBoot v3 4 400 iPXE 1 0 04 9ae6 Open Source rk Boot Firmwar Features HTTP iSCSI DNS TFTP VLAN age COMBOOT ELF MBOOT PXEXT Menu netO GUID 00 02 c9 03 00 b7 10 a0 MA Using Connec n 00 05 0 open Link down TX 0 TXE 0 RX 0 RXE 0 Link status The socket is not connected http Waiting for ink r 4 ok Configur 2 02 7 0 neto 3 Next server 31 1 Filename pxelinux Root path var lib tftpboot ftp 31 134 38 7 pxel Next FlexBoot attempts to boot as directed by the DHCP server 37 Mellanox Technologies Rev 2 3 How to Bootl 8 2 8 2 1 How to Boot to ESXi For instructions booting to ESXi refer to the ESXi Installation Guide on VMWare site http pubs vmware com vsphere 55 index jsp com vmware vsphere upgrade doc GUID B9 DB94CA 4857 458B B6F1 6A688726AED0 html How to Boot Using HTTP Script After performing basic HTTP configuration add the following lines to the dhcp server configuration file Host il 4 hardware ethernet F4 52 14 7A 56 F1 iimecmackcicass 14 7 240 filename pxe script linux option vendor class identifier PXEClient Aore Sejr ipxe dhcp net0 kernel n vmlinuz http 14 7 6 3
44. s Rev 2 3 Table of Contents TAZ SCHIP V e oio deett eet etes 23 7 123 Device hoe edd ict eris eR etae Heber ER sta FRK da 23 7 14 POlDevice D i dte eaedem eue diede iie 23 7 1 5 Banner Menu Timeout ssssssssssrrssssrsrssosrsrsnsrsrsnnrrssnnrss rss nennen ener nennen nnn nenne 23 151 6 Virtualization Mode eee re ee tp ree rr ves Pee end 23 7 1 7 Number of Virtual Functions 424000 0 nennen nennen enn 23 7 2 Firmware Image Properties 044 0 NAS ERA enn nre nennen nennen nns 24 7 2 1 Flexboot Versione neve ete lee 24 7 2 2 ende et eee end dee et eid ees 24 7 3 Mairi Gonflg ratiori cec en qp deep e reddat eoe t d Sa EROR RETE Meo SPAN 25 TOA UBM INK DE DS NEPTUNUS 25 7 3 2 MAG Address i da eed e dp aa c dinde 26 FAQ NIC Configuration riecht ertet nit RU E be e un be E eene 26 7 4 1 Legacy Boot 2 00 ar ee 27 FAQ bue 27 74 8 Boot Retry Count nga ae eae ae ee 27 TAN Wake on re eed att eiii 28 7 4 5 Option 28 lt 4 6 EP ire Rabe pd 28 7 4 7 Mirtual eade
45. tocol will be used Notes 1 If set to NONE the boot will be skipped on this port 2 IF all the ports are set as NONE a BEV entry is not registered Configurable Yes built in options Persistency Persistent through AC cycle Available options 1 None no PXE boot no iSCSI boot 2 first and if failed try iSCSI 3 iSCSI iSCSI first and if failed try PXE 7 4 2 PKey Value Location System setup Port main configurations NIC Configuration Description PKey value to be used when booting IPoIB Configurable Yes Persistency Persistent through AC cycle Available options 0 65535 0 the default PKey value 7 4 3 Boot Retry Count Location System setup Port main configurations NIC Configuration Description The number of retries to attempt in case of a boot failure 27 Mellanox Technologies Rev 2 3 System Settings Configuration Options Configurable Yes built in options Persistency Persistent through AC cycle No retries Retry Available options Retries Retries Retries Retries Retries NY AVN BONG Indefinite Retries 7 4 4 Wake on LAN Location System setup Port main configurations NIC Configuration Description Status of Wake on LAN feature When set it enables the server to be powered using an in band magic packet Only valid for adapters that support Wake on
46. ware Version FlexBoot Version ConnectX 9 3 2 35 5000 3 4 648 ConnectX 9 3 Pro 2 35 5000 3 4 648 ConnectX 9 4 12 12 xxxx 3 4 650 ConnectX 4 Lx 14 12 xxxx 3 4 650 Connect IB 10 12 3 4 650 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 3 2 Burning the Expansion ROM Image 2 1 Burning the Image ConnectX Family Devices 2 1 1 Prerequisites Expansion ROM Image The expansion ROM images can be compiled using the Mellanox FlexBoot code source available in the FlexBoot webpage on www mellanox com For further information on how to compile the code source please refer to the pxebuild py file inside the code source e Firmware Burning Tools You need to install the Mellanox Firmware Tools MFT package version 4 1 0 or later in order to burn the PXE ROM image To download MFT see Firmware Tools under www mellanox com gt Products gt InfiniBand VPI Drivers gt Firmware Tools 2 1 2 Image Burning Procedure gt To burn the composite image perform the following steps 1 Obtain the MST device name Run mst start The device name will be of the form mt lt dev id pci cr0 conf0 2 Create and burn the composite image Run flint d lt mst device name gt brom expansion ROM image Example on Linux flint d dev mst mt4103 pci cr0 bron FlexBoot 3 4 644 4103 mrom Example on Windows Te ES x ISO Ote

User Manual

Contents

Download Pdf Manuals

Related Search

Related Contents