Mellanox FlexBoot User Manual
Contents
1. 38 Diskless Machines rernm 39 Gase IzInfiniBand Ports 2nd du ite ene e E gre coepi boe 39 A 1 1 X Example Adding IB Driver to initrd Linux 40 2 Gase ll Ethernet Ports scandinavia s dan pde Pep o i dd a e NAS NANA NNE akad rr 42 A 2 1 Example Adding an Ethernet Driver to initrd Linux 42 Mellanox Technologies Rev 2 2 Table of Contents List of Figures Figure 1 NIG Label ici cert redit e idee e ede gears ae E e dba 13 Figure 2 Main Menu 18 Figure Free Text Editing Example n remet e ed ette e ettet 19 Figure 4 Exit iibri nem ee eR Eae EC OP RR teur eR 20 Figure 5 System Setup 21 Figure 6 Firmware Image Properties MCNU sssssesrsrssosrrsrssssrrrsssrsrsrsnnrrr rss rs 23 Figure 7 Main Configuration 24 Figure 8 NIC Configu A 2 eren odd tiere bre eb t Ed tee ee ED Pee ibas 25 Figure 9 iSCSI General 27 Figure 9 iSCSI Initiator Parameters esses eene nne 29 r Mellanox Technologies Table of Contents Rev 2 2 List of Tables Table 1 Document Revision History sess nene eene ener KRK KR RR nennen nns 8 Table 2 Supported Mellanox Adapter Devices and2. 9 Table PXE Browsing Keys i ace o ete eie eda tee 17 RN MER EO Mellanox Technologies Rev 2 2 Mellanox FlexBoot PXE Document Revision History Table 1 Document Revision History Revision Date Description 2 2 July 08 2015 e Added UEFI PXE Boot June 23 2015 e Updated the Value restrictions description in section Number of Virtual Functions May 15 2015 e Added the following sections e DHCP IP e DHCP Parameters e IP Version e CHAP Authentication e CHAP Mutual Authentication e Legacy Boot Protocol PXE e Legacy Boot Protocol iSCSI e CHAP Authentication and CHAP Mutual Authentication e Updated the following sections e Invoking the User Interface e CHAP ID e CHAP Secret e Removed the following sections e Max Number of VFs Supported e Virtual MAC Address 2 1 February 09 2015 e Updated the note in section Running the DHCP Server e Removed sections e iSCSI General Parameters e DHCP Parameters e DHCP IP e DHCP Parameters e IP Version 2 0 January 15 2015 Initial release of the restructured new User Manual 1 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 1 Mellanox FlexBoot PXE 1 1 Overview Mellanox FlexBoot is a multiprotocol remote boot technology FlexBoot supports remote Boot over InfiniBand BoIB and over Ethernet Using Mellanox Virtual Protocol Interconnect VPI
3. 7 18 8 JUIL p qM Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 3 1 3 Running the DHCP Server E gt NOTE This section is applicable only when the Initiator s IP address is empty As In order for the DHCP server to provide configuration records for clients an appropriate configuration file needs to be created By default the DHCP server looks for a configuration file called dhcpd conf under etc You can either edit this file or create a new one and provide its full path to the DHCP server using the cf flag See file example at docs dhepd conf of this package The DHCP server must run on a machine which has loaded the IPoIB module To run the DHCP server from the command line enter dhcpd IB network interface name d Example hostl dhcpd ibO d 3 1 4 Adding Device Driver Support in Linux Kernels Initrd NOTE A DHCP client can be used if you need to prepare a diskless machine with an IB driver See Step 11 under Example Adding an IB Driver to initrd Linux 3 2 Case Il Ethernet Ports When a FlexBoot client boots it sends the DHCP server various information including its DHCP hardware Ethernet address MAC The MAC address is 6 bytes long and it is used to distinguish between the various DHCP sessions 3 2 1 Extracting the MAC Address Method I AII Mellanox Ethernet NICs have a label on the printed side of the adapter card that has the card serial nu
4. ance whee deed dance ciun 9 IEEE IIIA 9 1 2 Supported Mellanox Adapter Devices and 9 1 9 FlexBoot Package servern ven ads nr ee rec rete dece ete ve aee cete dee e dev o NAS BEN VARA 9 2 Burning the Expansion ROM 10 2 1 Burning the Image ConnectX Family Devices 10 2 1 1 Prerequisites uii ced eiu e boven sie osea cd ER E AR 10 2 1 2 Image Burning 10 2 2 Removing the Expansion ROM 10 3 Preparing the DHCP Server in Linux eeeeeeeeeeeeeenenee enne enne nennen nn RR anna RR RR nnn nnns nas 11 3 1 Case 1 InfiniBand 5 11 3 1 1 Installing DHC Piri ina tee eue he degraded eu gu Oe 11 3 1 2 Configuring the DHCP Server eene nnns 11 3 1 8 Running the DHCP 13 3 1 4 Adding Device Driver Support in Linux Kernels lnitrd 13 3 2 Gase II Ethernet Ports ieni deep x uie eite Mp IE da 13 3 2 1 Extracting the MAC Address Method I sse 13 3 2 2 Extracting the MAC Address Method ll sse 14 3 2 8 Extracting the MAC Address Method lll sene 14 3 2 4 Placing MAC Addresses etc dhcpd conf
5. sse 14 4 Subnet Manager OpenSM ariete er tie 15 BIOS Configuration aeo 16 Flexboot User Interface sssssssmsssssnsssssnssnssnsnnssnnnnnsn nns annan sn 17 6 1 Skipping FlexBoot FlexBoot User Interface and FlexBoot 17 6 2 Invoking the User Interface titer rettet eode iD te d 17 6 2 1 Browsing the User Interface Menu 17 6 2 2 Accessing FlexBoot Sub 18 6 3 Configuring System Settings 18 6 3 1 Editing the Parameters using Free Text sse 18 6 3 2 Editing the Parameters using Set Values sse 19 6 3 3 Deleting Configurable Settings sse 19 6 3 4 Restoring Default Configurations esses 19 6 4 Exiting FlexBoot User Iriterface ont tette xtti den teen 20 System Settings Configuration 21 T EE 21 NEUES Mellanox Technologies Rev 2 2 Table of Contents 7 1 1 Bus Device FUNCOM cetra a reete fal A 21 TAN GUI TY DG ec A T 22 713 Devico d ee rede ae deb ei AEA 22 PAA POl Device ID i ad Eti P Ead pe UD e BEC e d 22 715 Virtualization Mode ee ree eee e e ree rre ee Poe end 22 7 1 6 Number of Virtual
6. images with the images found at www mellanox com gt Products gt InfiniBand VPI Drivers gt FlexBoot Download Tab All OSes which doesn t include inbox driver should do the following OSes which include the inbox driver should skip these configurations A 1 Case I InfiniBand Ports The IB driver requires loading the following modules in the specified e ib addr ko e ib core ko e ib mad ko e ib sa ko e ib cm ko e ib uverbs ko e ib ucm ko e ib umad ko e iw cm ko e rdma cm ko e rdma ucm ko e mlx4 core ko e mlx4 ib ko e ib mthca ko e ipoib helper ko this module is not required for all OS kernels Please check the release notes 39 Mellanox Technologies Rev 2 2 UEFI Bootl e ib ipoib ko Example Adding IB Driver to initrd Linux A 1 1 1 Prerequisites e The FlexBoot image is already programmed on the HCA card e The DHCP server is installed configured and connected to the client machine e An initrd file e To add an IB driver into initrd you need to copy the IB modules to the diskless image Your machine needs to be pre installed with a Mellanox OFED for Linux ISO image available for download from www mellanox com gt Products gt InfiniBand VPI Drivers gt Linux SW Drivers that is appropriate for the kernel version the diskless image will run The remainder of this section assumes that Mellanox OFED has been installed on your machine A 1 1 2 Adding the IB Driver
7. comment out the follow ing line option dhcp client identifier ee SOOO UOT ERO DUCES 2 e OY e NDS For a ConnectX device with ports configured as Ethernet comment out the following line hardware ethernet 00 02 c9 00 00 bb 8 3 Legacy Boot Priority Behavior The sections below describe various legacy boot priority behaviors based on the protocol used 8 3 1 Legacy Boot Protocol PXE The client first tries to perform PXE boot and if that fails it tries iSCSI boot The client goes to the DHCP to get an IP address The DHCP server might supply filename for PXE boot and root path for iSCSI boot if set by the user e Ifthe iSCSI target parameters are taken from the flash then the root path retrieved from the DHCP is deleted and if the Connect is set as Enable then the values stored in the flash are used DHCP IP Enable DHCP Parameters Disable e Ifa root path is supplied from either the DHCP or the flash the client is connected to the iSCSI target but no boot is performed e Ifa filename is supplied a PXE boot is performed on the Operating System e If PXE boot failed or no filename was supplied an iSCSI boot is performed to the connected target When the legacy boot protocol PXE is selected the following are not supported DHCP IP disable N 37 Mellanox Technologies Rev 2 2 UEFI Bootl 8 3 2 8 3 3 e iSCSI boot with static c
8. 15 66 81 NIC Configuration iSCSI Configuration Blink leds 7 3 1 Blink LEDs NOTE This configuration is applied when presssing Enter so it may take longer time to save than other configurations Location System setup Port main configurations Description Specifies the number of seconds the LEDs on physical network port should blink to assist with port identification Only valid for adapters with blink LEDs Configurable Yes via input Persistency Valid only for current cycle Effects Blink LEDs duration value of the port will be as set in the current cycle Value restrictions The minimum integer value 15 0 and the maximum is 15 7 3 2 MAC Address Location System setup Port main configurations Description Permanent MAC address assigned during manufacturing Configurable No 24 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 7 4 Configuration In the following menu you can set following options e Legacy Boot Protocol e Boot Retry Count e Wake on LAN e Option ROM e Virtual LAN ID e Virtual LAN Mode Figure 8 NIC Configuration NIC Configuration Legacy boot Boot retry count No retries Option ROM Enabled Virtual LAN mode Disabled Legacy boot protocol 7 4 1 Legacy Boot Protocol Location System setup Port main configurations NIC Con
9. Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 7 6 iSCSI Initiator Parameters In the following menu you can set the following options e CHAP ID e CHAP Secret e IPv4 Address e Pv4 Primary DNS e IPv4 Default Gateway e iSCSI Name e Subnet Mask Figure 10 iSCSI Initiator Parameters Initiator Parameters CHAP Ian a sars Asa mam ene AE SPECIE ted gt CHAP Secret specif ied gt IPu4 Address specif ied gt IPv4 Primary DNS specif ied gt IPv4 Default Gateway specified iSCSI Name specified gt Subnet mask specified gt CHAP ID 7 6 1 CHAP ID Location System setup Port main configurations iSCSI Configuration iSCSI Initiator Parameters Description The iSCSI initiator Challenge Handshake Authentication Protocol CHAP ID Configurable Yes via input Persistency Persistent through AC cycle Effects If the Connect and the CHAP Mutual Authentication are enabled This will be the reverse username used for the iSCSI initiator Challenge Handshake Authentication Protocol Value restrictions Maximum string length is 128 29 Mellanox Technologies Rev 2 2 System Settings Configuration Options 7 6 2 CHAP Secret Location System setup Port main configurations iSCSI Configuration iSCSI Initiator Parameters Description The iSCST initiator Challenge Handshake Authentication Protocol CHAP secret password Configurable Yes via
10. SUCH DAMAGE Mellanox TECHNOLOGIES Mellanox Technologies Mellanox Technologies Ltd 350 Oakmead Parkway Suite 100 Hakidma 26 Sunnyvale CA 94085 Ofer Industrial Park U S A Y okneam 2069200 www mellanox com Israel Tel 408 970 3400 www mellanox com Fax 408 970 3403 Tel 972 0 74 723 7200 Fax 972 0 4 959 3245 Copyright 2015 Mellanox Technologies All Rights Reserved Mellanox Mellanox logo BridgeX ConnectX Connect IBG CoolBox CORE Direct GPUDirect InfiniBridge InfiniHost InfiniScale Kotura Kotura logo MetroX MLNX OS PhyX Scalable HPC SwitchX TestX Virtual Protocol Interconnect Voltaire and Voltaire logo are registered trademarks of Mellanox Technologies Ltd CyPU ExtendX FabricIT FPGADirect HPC X Mellanox Care Mellanox CloudX Mellanox NEO Mellanox Open Ethernet Mellanox PeerDirect Mellanox Virtual Modular Switch MetroDX NVMeDirect StPU Switch IBTM Unbreakable Link are trademarks of Mellanox Technologies Ltd All other trademarks are property of their respective owners 2 Document Number 3218 Mellanox Technologies Table of Contents Rev 2 2 Table of Contents Document Revision HiStOrVY sssssssssssssssssssssssssnssnsnnssnnnnssnnnnssn nns sn nn nana nnne nn sns AR Rn RR AR RR nasa tnn sana nne nnns 8 1 Mellanox FlexBoot PXE codes at eee uiuere
11. showing the format of representing a client machine for the DHCP server running on a Linux machine Tost Mosel i next server 11 4 3 7 filename pxelinux 0 fixed address 11 4 3 130 hardware ethernet 00 02 9 0 78 12 14 J Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 4 Subnet Manager OpenSM gt NOTE This section applies to ports configured as InfiniBand only aa FlexBoot requires a Subnet Manager to be running on one of the machines in the IB network OpenSM is part of the Mellanox OFED for Linux software package and can be used to accomplish this Note that OpenSM may be run on the same host running the DHCP server but itis not mandatory For large InfiniBand clusters gt 100 nodes it is recommended to use OpenSM caching For further information please refer to the Mellanox OFED User Manual 15 Mellanox Technologies Rev 2 2 BIOS Configuration 5 BIOS Configuration The expansion ROM image presents itself to the BIOS as a boot device As a result the BIOS will add to the list of boot devices MLNX FlexBoot lt ver gt for a ConnectX family device The priority of this list can be modified through BIOS setup _ MEME Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 6 Flexboot User Interface 6 1 Skipping FlexBoot FlexBoot User Interface and FlexBoot Port You can skip FlexBoot in any of the given options below e Following system POST w
12. var state dhcp dhclient leases hostl cp bin uname tmp initrd ib bin hostl cp usr bin expr tmp initrd ib bin hostl cp sbin ifconfig tmp initrd ib bin hostl cp bin hostname tmp initrd ib bin 9 Create a configuration file for the DHCP client and place it under tmp initrd ib sbin Example of the dclient conf The value indicates a hexadecimal number For ConnectX device interface ib0 send dhcp client identifier enu gp pO e D emn ego sg escogido 10 44 the commands for loading the copied modules into the file init at the point you wish the IB driver to be loaded Please pay attention to the following commands order for loading modules as it is critical Loading ipoib helper ko is not required for all OS kernels echo loading ipv6 sbin insmod lib modules ipv6 ko echo loading IB driver sbin insmod lib modules ib ib addr ko sbin insmod lib modules ib ib_core ko sbin insmod lib modules ib ib_mad ko sbin insmod lib modules ib ib_sa ko sbin insmod lib modules ib ib_cm ko sbin insmod lib modules ib ib uverbs ko sbin insmod lib modules ib ib ucm ko sbin insmod lib modules ib ib umad ko sbin insmod lib modules ib iw sbin insmod lib modules ib rdma cm ko sbin insmod lib modules ib rdma ucm ko sbin insmod lib modules ib mlx4 core ko sbin insmod lib modules ib mlx4 ib ko sbin insmod lib modules ib ib mthca k
13. 4 442 4103 mrom 2 2 Removing the Expansion ROM Image e Remove the expansion ROM image Run flint dev mst device name drom NOTE When removing the expansion ROM image you also remove Flexboot from the boot device list Depending on the OS the device name may be superseded with a prefix CAM NEN Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 3 Preparing the DHCP Server in Linux When the boot session starts the PXE firmware attempts to bring up an adapter network link port If it succeeds to bring up a connected link the PXE firmware communicates with the DHCP server The DHCP server assigns an IP address to the PXE client and provides it with the location of the boot program 3 1 Case I InfiniBand Ports 3 1 1 Installing DHCP FlexBoot requires that the DHCP server runs on a machine which supports IP over IB 1 Prior to installing DHCP make sure that Mellanox OFED for Linux is already installed on your DHCP server see www mellanox com 2 ISC DHCP of versions above 3 1 3 have native support for IPoIB clients If using DHCP v3 1 3 the provided patch must be deployed Newer versions do not require patching 3 To download and install other supported DHCP server versions please download Mellanox OFED from www mellanox com gt Products gt InfiniBand VPI Drivers gt Linux SW Drivers and refer to docs dhcp README Standard DHCP fields holding MAC addresses are not large enough to
14. DHCP IP 7 5 1 DHCP IP Location System setup Port main configurations iSCSI Configuration iSCSI General Parameters Description When the DHCP IP is enabled the initiator and target s IP netmask gateway and DNS will be taken from DHCP instead of from the menu Configurable Yes 7 5 2 DHCP Parameters 27 Mellanox Technologies Rev 2 2 System Settings Configuration Options Location System setup Port main configurations iSCSI Configuration iSCSI General Parameters Description When the DHCP IP and DHCP Parameters are enabled all the initiator and target s parameters are taken from DHCP instead of from the menu Configurable Yes 7 5 3 IP Version Location System setup Port main configurations iSCSI Configuration iSCSI General Parameters Description Specifies whether IPv4 or IPv6 network addressing will be used for iSCSI initiator and targets Configurable No 7 5 4 CHAP Authentication Location System setup Port main configurations iSCSI Configuration iSCSI General Parameters Description Enables CHAP Authentication for iSCSI initiator Configurable Yes 7 5 5 CHAP Mutual Authentication Location System setup Port main configurations iSCSI Configuration iSCSI General Parameters Description Enables mutual CHAP authentication between the iSCSI initiator and target Configurable Yes 28
15. Driver available for download from www mellanox com gt Products gt Ethernet Drivers that is appropriate for the kernel version the diskless image will run Adding the Ethernet Driver to the initrd File The following procedure modifies critical files used in the boot procedure It must be executed by users with expertise in the boot process Improper application of this procedure may prevent the diskless machine from booting 1 Back up your current initrd file 2 Make a new working directory and change to it host1 mkdir tmp initrd en host1 tmp initrd 3 Extract the initrd image host ior p cle amecge gt 42 J Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 The initrd files can be located at tmp initrd en 4 Create a directory for the ConnectX EN modules and copy them hostl1 mkdir p tmp initrd en lib modules mlnx hostl1 cd lib modules uname r updates kernel drivers hostis cp net mlx4 mlx4 core ko tmp initrd en lib modules mlnx host1 cp net mlx4 mlx4 enko tmp initrd en lib modules mlnx 5 Load the modules using the insmod executable If you do not have it in your init ra add it using the following command hostl cp sbin insmod tmp initrd en sbin 6 Optional Give your Ethernet device a static IP address host1 cp sbin ifconfig tmp initrd en sbin 7 Add the commands for loading the copied modules into the f
16. Functions 22 7 2 Firmware Image Properties 23 7 2 1 Flexboot Versione secet eene ee ere ee edes 23 73 2 2 Family Firmware Version ende et eee end dee et eid eene edes 23 7 3 Madin Gonflg ratiori cena eren qp deeper eed kat Meo SPAN 24 y NEN SIDES ITEM 24 7 3 2 MAG i d dede ed p AAA 24 FAQ NIC Configuration reci E ertet RU E be e un iUt Ee end 25 7 4 1 Legacy ProtoCOb en eesosast nennen nennen 25 7 4 2 Boot Retry 25 TAS WakeonLbAN eee ete sende e eg e eene e Re e 26 FAA Option ROM ite tae ete ete eite ended 26 74 5 Mirt al ANAD o dieci esee orate et Fate dior oed ped ean s 26 7 4 6 Virtual LAN Mode 3 nde dh eei a eed bein e ide eut 26 7 5 iSCSI General Parameters 27 7515 27 7 9 2 DHCP Parameters iti ederet eme ae eee es 27 f WJPSMOrSIODT i deno RUE 28 LOA CHAP AuthentiCation cuocere t viene cela e dee e indivi HA RN MER NN n 28 7 5 5 CHAP Mutual Authentication 28 7 6 iSCSI Initiator Parameters enne nre nennen streiten nnns 29 LEE LEE 29 76 2 5 eee eec EE p Eee tede ce EUR ence a 30 1 6 9 PYF AGIOS S itecto hr rur te Et e et amend ieee i S PR E
17. Installation Guide on VMWare site http pubs vmware com vsphere 55 index jspftcom vmware vsphere upgrade doc GUID B9 DB94CA 4857 458B B6F1 6A688726AED0 html How to Boot Using HTTP Script After performing basic HTTP configuration add the following lines to the dhcp server configuration file Host Wea il 4 hardware ethernet F4 52 14 7A 56 F1 iimecmackcicass 14 7 5 2419 filename pxe script linux option vendor class identifier PXEClient Aore Sejr 1 ipxe dhcp net0 kernel n vmlinuz http 14 7 6 30 RHEL6 4 x86_64 DVD1 images pxeboot vmlinuz initrd http 14 7 6 30 RHEL6 4 x86 64 DVDl images pxeboot initrd img boot iSCSI Mellanox FlexBoot enables an iSCSI boot of an OS located on a remote iSCSI target It has a built in iSCSI initiator which can connect to the remote iSCSI target and load from it the kernel and initrd There are two instances of connection to the remote iSCSI target the first is for get ting the kernel and initrd via FlexBoot and the second is for loading other parts of the OS via initrd If you choose to continue loading the OS after boot through the HCA device driver please verify that the initrd image includes the HCA driver as described in Section 1 2 NOTE To boot via iSCSI Flexboot must be selected from the BIOS boot menu it will be shown as a BEV entry Do not look for the entry in the BVC table Configuring an iSCSI Target in Linux Enviro
18. Mellanox TECHNOL Connect Accelerate Outperform Mellanox FlexBoot User Manual Rev 2 2 www mellanox com Mellanox Technologies NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT S AND OR THE SYSTEM USING IT THEREFORE MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT INDIRECT SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES OF ANY KIND INCLUDING BUT NOT LIMITED TO PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OFLIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY FROM THE USE OF THE PRODUCT S AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF
19. Port 2 4 52 14 15 66 82 Virtualization mode For further information on FlexBoot varies menus and the configuration options they provide please refer to section System Settings Configuration Options and its subsections Configuring System Settings NOTE It is possible to save the changes made and exit the menu at any given time by pressing Ctrl S Editing the Parameters using Free Text When a configurable setting is highlighted the instructions to edit it will be shown on screen If the setting is configurable via input the description will show the input restrictions gt To change the configuration 3 Press Enter or start typing while the setting is highlighted 4 Optional To discard the input while editing press Ctrl C 5 Press Enter and wait to verify the new value is accepted and shown In case the value entered is invalid an alert will be shown and the new value won t be accepted NOTE Although numeric values are shown in decimal base it is possible to insert values in different bases e For hexadecimal base type Ox before the number e For octal base type 0 before the number e For decimal base type the number as is E NN NN Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 Figure 3 Free Text Editing Example NIC Configuration Legacy boot protocol Boot retry count No retries Option ROM Enabled Virtual LAN mode Enabled Virtual LAN ID 6 3 2 Editin
20. boot required Yes reboot is needed in order for this change to apply Available options 1 None 2 SR IOV 7 1 6 Number of Virtual Functions Location System setup Description The number of virtual functions advertised and usable by the driver Configurable Yes via input If virtualization mode is SR IOV Persistency Persistent through AC cycle Reboot required Yes reboot is needed in order for this change to apply Value restrictions Minimum integer value is 0 Maximum integer value is the value supported by the firmware installed 22 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 7 2 Firmware Image Properties In the following menu you can set the following options e Flexboot Version e Family Firmware Version Figure 6 Firmware Image Properties Menu Firmware Image Properties Flexboot version 7 2 1 Flexboot Version Location System setup Firmware Image Properties Description Flexboot driver version information Configurable No 7 2 2 Family Firmware Version Location System setup Firmware Image Properties Description Device s firmware version information Configurable No 23 Mellanox Technologies Rev 2 2 System Settings Configuration Options 7 3 Main Configuration In the following menu you can set the following options e Blink LEDs e MAC Address Figure 7 Main Configuration netO Port 1 4 52 14
21. contain an IPoIB hardware address To overcome this problem DHCP over InfiniBand messages convey a client identifier field in DHCP option 61 used to identify the DHCP session This client identifier field can be used to associate an IP address with a client identifier value such that the DHCP server will grant the same IP address to any client that conveys this client identifier NOTE In the DHCP discover packets flexboot sends 6 rather than 0 aa chaddr will be the ethernet mac address rather than zeros In the DHCP server the GUID in the client identifier can still be used to identify clients 3 1 2 Configuring the DHCP Server 3 1 2 1 For ConnectX Family Devices When a FlexBoot client boots it sends the DHCP server various information including its DHCP client identifier This identifier is used to distinguish between the various DHCP sessions The value of the client identifier is composed of a prefix ff 00 00 00 00 00 02 00 00 02 c9 00 and an 8 byte port GUID all separated by colons and represented in hexadecimal digits 11 Mellanox Technologies Rev 2 2 Preparing the DHCP Server in Linux 3 1 2 2 3 1 2 3 3 1 2 4 Extracting the Port GUID Method I To obtain the port GUID 1 Start mst hostl mst start hostl mst status The following MFT commands assume that the Mellanox Firmware Tools MFT package has been installed on the client machine 2 Obtain the Port GUID using the
22. cy boot protocol is iSCSI This will be set as the default gateway IP address of the initiator 30 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 Value restrictions Should be in IPv4 format 7 6 6 iSCSI Location System setup Port main configurations iSCSI Configuration iSCSI Initiator Parameters Description The initiator s iSCSI Qualified Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI This will be set as the iSCSI Qualified Name of the initiator Value restrictions Maximum string length is 223 Subnet Mask Location System setup Port main configurations iSCSI Configuration iSCSI Initiator Parameters Description The iSCSI initiator Subnet Mask for an IPv4 initiator configuration Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI This will be set as the subnet mask for the IP address of the initiator Value restrictions Should be in IPv4 format s 1 7 Mellanox Technologies Rev 2 2 System Settings Configuration Options 7 7 iSCSI First Target Parameters In the following menu you can set the following options Boot LUN e Connect e iSCSI Name e CHAP ID e CHAP Secret e P Address e TCP Port iSCSI First target Parameters Boot LUN 0 Co
23. d tar zxf package file name gt The tarball contains PXE binary files with the mrom extension for the supported adapter devices See the release notes file FlexBoot lt flexboot_version gt _release_notes txt for details ConnectX 2 adapter cards support will be deprecated in the next release 9 Mellanox Technologies Rev 2 2 Burning the Expansion Image 2 Burning the Expansion ROM Image 2 1 Burning the Image on ConnectX Family Devices 2 1 1 Prerequisites e Expansion ROM Image The expansion ROM images are provided as part of the Mellanox FlexBoot package and are listed in the release notes file F1exBoot flexboot version release notes txt e Firmware Burning Tools You need to install the Mellanox Firmware Tools MFT package version 3 7 0 or later in order to burn the PXE ROM image To download MFT see Firmware Tools under www mellanox com gt Products gt InfiniBand VPI Drivers gt Firmware Tools 2 1 2 Image Burning Procedure Toburn the composite image perform the following steps 1 Obtain the MST device name Run mst start The device name will be of the form mt dev id pci cr0 conf0 2 Create and burn the composite image Run flint lt mst device name gt brom lt expansion ROM image gt Example on Linux flint d dev mst mt4103 pci cr0 bron FlexBoot 3 4 442 4103 mrom Example on Windows lint dev mt26428 pci 0 brom FlexBoot 3
24. device name The device name will be of the form dev mst mt lt dev_id gt _pci _crOlconf0 flint d lt MST DEVICE NAME q Assuming that FlexBoot is connected via Port 1 then the Port GUID is 00 02 c9 03 00 00 10 39 Extracting the Port GUID Method II An alternative method for obtaining the port GUID involves booting the client machine via Flex Boot This requires having a Subnet Manager running on one of the machines in the InfiniBand subnet The 8 bytes can be captured from the boot session as shown in the figure below MLNX FlexBoot 3 4 454 PCI 07 00 0 LNX FlexBoot 3 4 454 initialis Initialising completed Mellanox FlexBoot v3 4 454 iPXE 1 0 0 53438 Open Source Network Boot Firmware Features ULAN HTTP iSCSI DNS TFTP bzImage COMBOOT ELF MBOOT PXE PXEXT Menu netO GUID 4 52 14 03 00 6e ea f1 f4 52 14 6e ea f1 Using Connect Pro on PCIO 9 open Link down 7 X 0 RXE 0 Link status The socket is not connected http ipxe org 38136001 1 Maiting for link up on net Placing Client Identifiers in etc dhcpd conf The following is an excerpt of a etc dhcpd conf example file showing the format of representing a client machine for the DHCP server Rost oscil i next server 11 4 3 7 filename pxelinux 0 fixed address 11 4 3 130 option dhcp client identifier HIE 8 O0 0 amp 000 9 LOW amp 0122 amp 000 012 8 9 0 9 2 0 2 8 e 013 8
25. e aeta 30 7 6 4 Primary 30 7 6 5 IPv4 Default 30 7 66 iSCSI Be ae ieee s is 31 Kezo Subnet Mask umma testate et etas 31 7 7 iSCSI First Target Parameters 32 7 7 1 Boot EUN iced d eere debeo e E e asp doi ide 32 eZ Connects icc odd ER eid E De 33 fikade ISGSENAmO inte 33 FAA SCHAP UD iiic dette ee ed ett em 33 M Mellanox Technologies Table of Contents Rev 2 2 TRST CHAP SECTE sinister etos ortae ete eiectum delit et eius 33 7 5 doa odi d t iit TE 34 TET AT TOGPAPOR S uidi edere medietate etu iicet i ce 34 8 HOW t Boot ee 35 Bal PAE niter reet ott hee erre ed 35 8 1 1 Prerequisites x oneri Een etri He 35 94 2 Starting BOO Late ccrte 35 98 13 How to Bootto ESXi xor eei e aee et le e a Va d puede 36 8 1 4 How to Boot Using HTTP Script sess 36 NEN 36 8 2 1 Configuring an iSCSI Target in Linux 36 8 3 Legacy Boot Priority Behavior 37 8 3 1 Legacy Boot Protocol PAE Ut Eh iude E t Puta 37 8 3 2 Legacy Boot Protocol iSCSI 38 8 33 CHAP Authentication and CHAP Mutual 38 9 nu
26. egacy boot protocol is iSCSI and Connect is enabled This will be the iSCSI Qualified name in the root path for the first target Value restrictions Maximum string length is 223 CHAP ID Location System setup Port main configurations iSCSI Configuration iSCSI First target Parameters Description The first iSCSI storage target Challenge Handshake Authentication Protocol CHAP ID Configurable Yes via input Persistency Persistent through AC cycle Effects If the Connect and the CHAP Authentication are enabled This will be the username used for the first target Challenge Handshake Authentication Protocol Value restrictions Maximum string length is 128 CHAP Secret Location System setup Port main configurations iSCSI Configuration iSCSI First target Parameters Description The Challenge Handshake Authentication Protocol secret CHAP password of the first iSCSI storage target Configurable Yes via input Persistency Persistent through AC cycle Effects If the Connect and the CHAP Authentication are enabled This will be the 33 Mellanox Technologies Rev 2 2 System Settings Configuration Options 7 7 6 T T 7 UM password used for first target Challenge Handshake Authentication Protocol Value restrictions String length should be 0 or 12 to 16 IP Address Location System setup Port ma
27. figuration Description Use a non UEFI network boot protocol The chosen protocol is tried first if boot fails the other protocol will be used Configurable Yes built in options Persistency Persistent through AC cycle Available options 1 None no PXE boot no iSCSI boot 2 first and if failed try iSCSI 3 iSCSI iSCSI first and if failed try PXE 7 4 2 Boot Retry Count Location System setup Port main configurations NIC Configuration Description The number of retries to attempt in case of a boot failure 25 Mellanox Technologies Rev 2 2 System Settings Configuration Options 7 4 3 7 4 4 7 4 5 7 4 6 26 Configurable Yes built in options Persistency Persistent through AC cycle Available options 0 No retries 1 Retry 2 Retries 3 Retries 4 Retries 5 Retries 6 Retries 7 Indefinite Retries Wake on LAN Location System setup Port main configurations NIC Configuration Description Status of Wake on LAN feature When set it enables the server to be powered using an in band magic packet Only valid for adapters that support Wake on LAN Configurable Yes built in options Persistency Persistent through AC cycle Available options 1 Enabled 2 Disabled Option ROM Location System setup Port main configurations NIC Configuration Description Controls the enablement of legacy Boot Prot
28. g the Parameters using Set Values When a configurable setting is highlighted the instructions to edit it will be shown on screen If the setting is configurable with built in options the description will be Select to change value gt To change the configuration 1 Press Enter while setting is highlighted 2 Keep pressing Enter to choose the desired value 6 3 3 Deleting Configurable Settings Configurable settings can be deleted Value will only be deleted from the current configuration and not from the flash If the configuration has a default value it will be set when deleting the current value gt delete a configurable setting 1 Press Ctrl D while setting is highlighted 6 3 4 Restoring Default Configurations All configurations can be restored to the default values set in the device All configurable values will be modified to default or deleted if there is no default value in the device To restore defaults per port e Press Ctrl R on the main menu of the port only when applicable To restore defaults per device e Press Ctrl R on System setup menu when applicable 19 J Mellanox Technologies Rev 2 2 Flexboot User Interface 6 4 Exiting FlexBoot User Interface To exit FlexBoot User Interface press ESC from the System Setup menu Figure 4 Exit Menu Save configurations 20 J Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 7 System Setti
29. hen FlexBoot advertisement banner and configuration prompt is displayed Press any key other than CTRL B e When FlexBoot execution starts as depicted in the Initializing Devices screen output a UI prompt for escaping shortly follows e After the BIOS called ENTRY Press ESC when prompted 2 3 seconds e Skip a specific port when booting starts Press CTRL C 6 2 Invoking the User Interface Shortly after completion of the POST Power On Self Test sequence of the computer the user will be prompted to press CTRL B to invoke Mellanox FlexBoot User Interface The user has few seconds to press CTRL B before the message disappears Mellanox FlexBoot v3 4 435 iPXE C http ipxe org 05 00 0 D300 PCI3 00 PnP PMM 002676C0 002816CO D300 Press 1 to configure MLNX FlexBoot 3 4 435 PCI 05 00 0 6 2 1 Browsing the User Interface Menu To browse the menu use the following keys Table 3 PXE Browsing Keys Key Operation Arrows 11 Browse options Highlighted option is current Enter On forms to enter menu On configurable settings edit configuration Esc Exit current menu N 17 Mellanox Technologies Rev 2 2 Flexboot User Interface 6 2 2 6 3 6 3 1 Accessing FlexBoot Sub Menus To enter the required menu press Enter while the option is highlighted Figure 2 Main Menu Window System setup Firmware Image Properties netO Port 1 4 52 14 15 66 81 neti
30. ication e If Connect enable and CHAP Authentication enable the target CHAP parameters are used e If CHAP Mutual Authentication enable the Initiator CHAP parameters are used UEFI PXE Boot The main differences in UEFI PXE booting is determining the efi boot file editing the grub cfg and dhcpd conf files and setting the Boot Mode to UEFI Mode in the BIOS For further information please refer to the HowTo Configure PXE Server in UEFI Mode for IPoIB Network post in the Mellanox Community https community mellanox com gt Places gt Solutions gt PXE 38 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 Appendix Diskless Machines Mellanox FlexBoot supports booting diskless machines gt NOTE The following configuration insturctions apply only to operation systems which do not include an inbox driver and should be skipped in case an OS with an inbox driver is used To enable using an IB ETH driver the initrd image must include a device driver module and be configured to load that driver This can be achieved by adding the device driver module into the initrd image and loading it The initrd image of some Linux distributions such as SuSE Linux Enterprise Server and Red Hat Enterprise Linux cannot be edited prior or during the installation process If you need to install Linux distributions over Flexboot please replace your initrd
31. ile init at the point you wish the Ethernet driver to be loaded Please pay attention to the following commands order for loading modules as it is critical echo loading Mellanox ConnectX EN driver sbin insmod lib modules mlnx en mlx4 core ko sbin insmod lib modules mlnx en mlx4 en ko 8 Assign an IP address to the ConnectX EB network interface 9 Save the init file 10 Close initrd hostis xol ein Mostri rine 477 joie k mawe e gt aew tatc host1 gzip tmp new init en img 11 Copy the modified initra including the Ethernet driver which is located at tmp new init en img gz and rename it properly a J Mellanox Technologies
32. in configurations iSCSI Configuration iSCSI First target Parameters Description The IP address of the first iSCSI target Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI and Connect is enabled This will be the IP address in the root path for the first target Value restrictions Should be in IPv4 format TCP Port Location System setup Port main configurations iSCSI Configuration iSCSI First target Parameters Description TCP Port number of first iSCSI target Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI and Connect is enabled This will be the TCP Port in the root path for the first target Value restrictions The minimum integer value is 1 and the maximum is 65535 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 8 How to Boot 8 1 PXE Boot 8 1 1 Prerequisites e Make sure that your client is connected physically and logically to the server s e The FlexBoot image is already programmed on the adapter card e For InfiniBand ports only Start the Subnet Manager e Configure and start the DHCP server e Configure and start at least one of the services such as iSCSI 8 1 2 Starting Boot Boot the client machine and enter BIOS setup to configure MLNX FlexBoot to be the first on the boot device priority list fail
33. input Persistency Persistent through AC cycle Effects If the Connect and CHAP Mutual Authentication are enabled This will be the reverse password used for the iSCSI initiator Challenge Handshake Authentication Protocol Value restrictions String length should be 0 or 12 to 16 7 6 3 IPv4 Address Location System setup Port main configurations iSCSI Configuration iSCSI Initiator Parameters Description iSCSI initiator s IPv4 address Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI This will be set as the IP address of the initiator Value restrictions Should be in IP v4 format 7 6 4 IPv4 Primary DNS Location System setup Port main configurations iSCSI Configuration iSCSI Initiator Parameters Description The iSCST initiator Primary DNS IP Address Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI This will be set as the DNS for the IP address of the initiator Value restrictions Should be in IPv4 format 7 6 5 IPv4 Default Gateway Location System setup Port main configurations iSCSI Configuration iSCSI Initiator Parameters Description The iSCSI initiator default Gateway IP address Configurable Yes via input Persistency Persistent through AC cycle Effects If the lega
34. mber and the card MAC address Figure 1 NIC Label Serial S N MT0742X01849 number E I Wii HEUTE REV xi EU RU MAC 0002 9000174 MAC IN Made In Mellanox Technologies Rev 2 2 Preparing the DHCP Server in Linux 3 2 2 Extracting the MAC Address Method II The six bytes of MAC address can be captured from the display upon the boot of the ConnectX device session as shown in the figure below ellanox FlexBoot v3 4 454 iPXE 1 0 0 53438 Open Source Network Boot Firmware eatures ULAN HTTP iSCSI DNS TFTP bzImage COMBOOT ELF MBOOT PXE PXEXT Menu meti 4 52 14 6e ea f2 Using ConnectX3 Pro on PC107 0 Link doun TX 0 TXE O RX Link status Unknown http ipxe org 1a086101 1 Maiting for link up on net1 3 2 3 Extracting the MAC Address Method lll In case the previous methods fail to work perform the following as a last resort 1 Start mst hostl mst start hostl mst status The following MFT commands assume that the Mellanox Firmware Tools MFT package has been installed on the client machine 2 Obtain the MAC using the device name The device name will be of the form dev mst mt lt dev_id gt _pci _crOlconf0 flint d lt MST_DEVICE_NAME gt q Assuming that FlexBoot is connected via Port 1 then the MAC address is 00 02 9 0 7 11 3 2 4 Placing MAC Addresses etc dhcpd conf The following is an excerpt of a etc dhcpd conf example file
35. ngs Configuration Options Below are listed the attributes shown in varies FlexBoot User Interface menus 7 1 System Setup In the following menu you can set the following options e Bus Device Function e Chip Type e Device Name e PCI Device ID e Virtualization Mode e Number of Virtual Functions Figure 5 System Setup Menu System setup Firmware Image Properties netO Port 1 4 52 14 15 66 81 neti Port 2 f4 52 14 15 66 82 Virtualization mode 7 1 1 Bus Device Function Location System setup Description Specifies the BIOS assigned PCI Bus Device Function identifier of the card Configurable No 21 Mellanox Technologies Rev 2 2 System Settings Configuration Options 7 1 2 Chip Type Location System setup Description Specifies the chip type Configurable No 7 1 3 Device Name Location System setup Description The device s product name Configurable No 7 1 4 PCI Device ID Location System setup Description The PCI Device ID of the controller Configurable No 7 1 5 Virtualization Mode Location System setup Description Specifies the type of virtualization used by the controller on all ports Only valid for adapters that support SR IOV virtualization mode Configurable Yes built in options Persistency Persistent through AC cycle Re
36. nment Prerequisites e Make sure that an iSCSI Target is installed on your server side You can download and install an iSCSI Target from the following location http sourceforge net projects iscsitarget files iscsitarget e Dedicate a partition on your iSCSI Target on which you will later install the operating system e Configure your iSCSI Target to work with the partition you dedicated If for example you choose partition dev sda5 then edit the iSCSI Target configuration file etc ietd conf to include the following line under the iSCSI Target iqn line Lun 0 Path dev sda5 Type fileio 36 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 Example of an iSCSI Target iqn line target en 2007 08 763 4 e Start your iSCSI Target Example hostl etc init d iscsitarget start 8 2 1 2 Configuring the DHCP Server to Boot From an iSCSI Target 1 Configure DHCP as described in section Preparing the DHCP Server in Linux 2 Edit your DHCP configuration file etc dhcpd conf and add the following lines for the machine s you wish to boot from the iSCSI target Filename Ne Optio VASSSULSISSCSL ieget i9s5553cG893 lung acim 9 The following is an example for configuring an device to boot from an iSCSI target host hose filename Gorica roor oera 3ggsiciscsu ieget 190838 5 1966 Cargat acm p For a ConnectX device with ports configured as InfiniBand
37. nnect Disabled iSCSI Name lt not specified gt GINE U0 peru eee SOR a peGl teas CHAP Secret specif ied gt IP Address specif ied gt CHAP ID 7 7 1 Boot LUN Location System setup Port main configurations iSCSI Configuration iSCSI First target Parameters Description The boot Logical Unit Number LUN on the first iSCSI storage target Configurable Yes via input Persistency Persistent through AC cycle Effects If the legacy boot protocol is iSCSI and Connect is enabled This will be the boot LUN in the root path for the first target Value restrictions The minimum integer value is 0 and the maximum is 18446744073709551615 32 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 Connect Location System setup Port main configurations iSCSI Configuration iSCSI First target Parameters Description Enable Disable connecting to the first iSCSI target This will be ignored if both DHCP IP and DHCP Parameters are enabled in iSCSI general parameters Configurable Yes built in options Persistency Persistent through AC cycle Available options Enabled 2 Disabled iSCSI Name Location System setup Port main configurations iSCSI Configuration iSCSI First target Parameters Description The iSCSI Qualified Name IQN of the first target Configurable Yes via input Persistency Persistent through AC cycle Effects If the l
38. o In case of interoperability issues between iSCSI and Large Receive Offload LRO set the ib ipoib ko lro to 0 to disable LRO 7 Mellanox Technologies Rev 2 2 UEFI PXE Boot A 2 A 2 1 A 2 1 1 A 2 1 2 sbin insmod lib modules ib ib ipoib ko lro 0 11 Assign an IP address to the IB device Add either a call to ifconfig or to the DHCP client in the init file after loading the modules If you want to use the DHCP client then you need to add a call to the DHCP client in the init file after loading the IB modules For example sbin dhclient cf sbin dhclient conf ibl 12 Save the init file 13 Close initrd hostii kedi cmp miere hostis fine pio HH newe 0 gt host1 gzip tmp new init ib img 14 Copy the modified initrd including the IB driver which is located at tmp new init ib img gz and rename it properly Case Il Ethernet Ports The Ethernet driver requires loading the following modules in the specified order see the exam ple below e mlx4 core ko e mlx4 en ko Example Adding an Ethernet Driver to initrd Linux Prerequisites e The FlexBoot image is already programmed on the adapter card e The DHCP server is installed configured and connected to the client machine e An initra file e To add an Ethernet driver into initrd you need to copy the Ethernet modules to the diskless image Your machine needs to be pre installed with a MLNX EN Linux
39. ocols in the Option ROM If disabled no legacy boot protocol is chosen When set the boot protocol will be chosen by Boot legacy protocol attribute Configurable Yes built in options Persistency Persistent through AC cycle Reboot required Yes reboot is needed in order for this change to apply Available options 1 Enabled 2 Disabled Virtual LAN ID Location System setup Port main configurations NIC Configuration Description Uses the VLAN tag with the selected boot protocol Configurable Yes via input If virtual LAN mode is enabled Value restrictions The minimum integer value is I and Maximum integer value is 4094 Virtual LAN Mode Location System setup Port main configurations NIC Configuration Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 Description Enable Disable VLAN tagging on the selected boot protocol Configurable Yes built in options Persistency Persistent through AC cycle Available options 1 Enabled 2 Disabled 7 5 ISCSI General Parameters In the following menu you can set the following options DHCP IP e DHCP Parameters e IP Version e CHAP Authentication e CHAP Mutual Authentication Figure 9 iSCSI General Parameters iSCSI General Parameters DHCP IP Enabled DHCP Parameters Enabled CHAP Authentication Enabled CHAP Mutual Authentication Enabled
40. onfigurations after PXE boot Setting all iSCSI parameters from the flash including IP address Legacy Boot Protocol iSCSI The client first tries to perform iSCSI boot and if that fails it tries PXE boot Upon iSCSI boot one of the following flows occurs e If all iSCSI configuration is taken from the flash the client waits 25 seconds for the switches to be configured and then tries to perform an iSCSI boot hook to target and then boot If the boot fails the client exits DHCP IP disable Or 1 If only the TCP parameters need to be retrieved from the DHCP DHCP IP enable DHCP Parameters disable then the IP address is taken from the DHCP the root path retrieved from DHCP is deleted and if the Connect is set as Enable then the values stored in the flash are used Hook to the iSCSI target Try to boot to the iSCSI target If iSCSI boot failed and DHCP supplied a filename try to PXE boot If PXE boot failed exit d N When the legacy boot protocol iSCSI is selected the following is not supported e PXE boot after iSCSI boot with static configurations Setting all iSCSI parameters from the flash including IP address CHAP Authentication and CHAP Mutual Authentication iSCSI boot allows CHAP security option if the following conditions are met e If Connect disable or CHAP Authentication disable the server ignores both the CHAP Authentication and CHAP Mutual Authent
41. rdma ucm ko tmp initrd ib lib modules ib t1 cp net mlx4 mlx4 core ko tmp initrd ib lib modules ib t1 cp infiniband hw mlx4 mlx4 ib ko tmp initrd ib lib modules ib t1 cp infiniband hw mthca ib mthca ko tmp initrd ib lib modules ib t1 infiniband ulp ipoib ipoib helper ko tmp initrd_ib lib modules ib hos t1 cp infiniband ulp ipoib ib ipoib ko tmp initrd ib lib modules ib 40 Mellanox Technologies Mellanox FlexBoot User Manual Rev 2 2 5 Load the IPv6 module If you do not have it in your initrd add it using the following command host1 cp lib modules unam r kernel net ipv6 ipv6 ko tmp initrd ib lib modules 6 Load the modules using the insmod executable If you do not have it in your initrd add it using the following command hostl cp sbin insmod tmp initrd ib sbin 7 Optional Give your IB device a static IP address host1 cp sbin ifconfig tmp initrd ib sbin 8 Optional Obtain an IP address for the IB device through DHCP Copy the DHCP client which was compiled specifically to support IB To continue with this step DHCP client needs to be already installed on the machine you are working with Copy the DHCP client file and all the relevant files as described below hostl cp path to DHCP client dhclient tmp initrd ib sbin hostl cp path to DHCP client dhclient script tmp initrd ib sbin hostl mkdir p tmp initrd ib var state dhcp hostl touch tmp initrd ib
42. s it switches to boot from Port 2 Note also that the driver waits up to 10 seconds for port come up and up to 90 seconds to assign an IP from the DHCP server NOTE On dual port network adapters the client first attempts to boot from Port 1 If this If MLNX FlexBoot was selected through BIOS setup the client will boot from FlexBoot The client will display FlexBoot attributes sense the port protocol Ethernet or InfiniBand The port sensing occurs only when the port is connected to a QSFP connector In case of an InfiniBand port the client will also wait for port configuration by the Subnet Manager If auto sensing proto col fails the port will be configured as an InfiniBand port After configuring the IB ETH port the client attempts to connect to the DHCP server to obtain an IP address and the source location of the kernel OS to boot from For ConnectX InfiniBand Mellanox FlexBoot v3 4 400 iPXE 1 0 04 9ae6 Open Source Network Boot Firmw Features HTTP iSCSI DNS TFTP VLAN bzImage COMBOOT netO GUID 00 02 c9 03 00 b7 Using ConnectIB 0 0 Link down TX 0 TXE 0 RX 0 tus The socket is no ink up on 02 ilename pxelir Root path ftp 31 134 38 7 pxe Next FlexBoot attempts to boot as directed by the DHCP server 35 Mellanox Technologies Rev 2 2 How to Bootl 8 2 8 2 1 8 2 1 1 How to Boot to ESXi For instructions booting to ESXi refer to the ESXi
43. technologies available in ConnectX adapters FlexBoot gives IT Managers the choice to boot from a remote storage target iSCSI target or a LAN target Ethernet Remote Boot Server using a single ROM image on Mellanox ConnectX products FlexBoot is based on the open source project iPXE available at http www ipxe org FlexBoot first initializes the adapter device senses the port protocol Ethernet or InfiniBand and brings up the port Then it connects to a DHCP server to obtain its assigned IP address and network parameters and also to obtain the source location of the kernel OS to boot from The DHCP server instructs FlexBoot to access the kernel OS through a TFTP server an iSCSI target or some other service For an InfiniBand port Mellanox FlexBoot implements a network driver with IP over IB acting as the transport layer IP over IB is part of the Mellanox OFED for Linux software package see www mellanox com gt Products gt InfiniBand VPI Drivers gt FlexBoot 1 2 Supported Mellanox Adapter Devices and Firmware The package supports the following HCA and firmware versions Table 2 Supported Mellanox Adapter Devices and Firmware HCAs Firmware Version ConnectX 2 2 9 1000 ConnectX 3 2 34 5000 ConnectX 3 Pro 2 34 5000 ConnectX 9 4 12 0100 6000 Connect IB 10 0100 6000 1 3 FlexBoot Package The FlexBoot package is provided as a tarball tgz extension Uncompress it using the comman
44. to the initrd File gt NOTE The following procedure modifies critical files used in the boot procedure It must be Ae executed by users with expertise in the boot process Improper application of this pro cedure may prevent the diskless machine from booting 1 Back up your current initrd file 2 Make a new working directory and change to it Imoysie 9 Ino alt ia MOSES ul 3 Extract the initrd MOSIELS cle amecge gt cele d The initrd files are now located at tmp initrd_ib 4 Create a directory for the InfiniBand modules and copy them hos hos hos hos hos hos hos hos hos hos hos hos hos hos hos hos hos t1 mkdir p tmp initrd ib lib modules ib t1 cd lib modules uname r updates kernel drivers t1 cp infiniband core ib addr ko tmp initrd ib lib modules ib t1 cp infiniband core ib core ko tmp initrd ib lib modules ib t1 cp infiniband core ib mad ko tmp initrd ib lib modules ib t1 cp infiniband core ib sa ko tmp initrd ib lib modules ib t1 cp infiniband core ib cm ko tmp initrd ib lib modules ib t1 cp infiniband core ib uverbs ko tmp initrd ib lib modules ib t1 cp infiniband core ib ucm ko tmp initrd ib lib modules ib t1 cp infiniband core ib umad ko tmp initrd ib lib modules ib t1 cp infiniband core iw cm ko tmp initrd ib lib modules ib t1 cp infiniband core rdma cm ko tmp initrd ib lib modules ib t1 cp infiniband core
Download Pdf Manuals
Related Search