Mellanox OFED for FreeBSD for ConnectX
Contents
1. Usually low PCI locations are closest to NUMA number 0 and high PCI locations are closest to NUMA number 1 Here is how to verify the locations c Find the NIC s pcib by PCI location in this example try PCI 4 gt sysctl a grep pci 3 parent dev pci 3 parent pcib3 19 Mellanox Technologies Rev 3 0 0 d Find the NIC s pcib location gt sysctl a grep pcib 3 location dev pcib 3 10cation pci0 0 2 0 handle V SB PCIO PEX2 In handle PCIO is the value for locations near NUMAO and PCII is the value for locations near NUMAI e Find the cores list of the closest NUMA i sysctl a grep group level 2 A 1 group level 2 cache level 2 gt AGO CNN ess 1 2 3 4 By 7 By O 10 iios group level 2 cache level 2 gt e GewwE U129 meseUsOUDUSI2 19 n4 15 2 17 19 19 20 21 22 29 g Note Each list of cores refers to a different NUMA 2 Tune Mellanox NICs to work on desirable cores a Pin both interrupts and application processes to the relevant cores b Find the closest NUMA to the NIC c Find the device interrupts vmstat ia grep mlx5 cored awk print 1 sed s irg sed s 304 305 306 d Bind each interrupt to a core from the closest NUMA cores list Note It is best to avoid core number 0 cpuset x 304 1 1 cpuset x 305 1 2 cpuset x 306 1 3 e Bind the application to the closest NUMA cores list Note It is best to avoid core number 0 cpuset2. l 1 11 app name sever flag cpuset l 1 11 app name client flag IP For best performance change CPU s BIOS configuration to performance mode A Due to FreeBSD internal card memory allocation mechanism on boot it is preferred to insert the NIC to a NUMA O slot for max performance Mellanox Technologies 20 J
3. 00 00 00 00 00 00 00 00 00 00 00 00 30 00 00 2 Another option for reading cable information is by using the ifconfig gt ifconfig v mce X gt ifconfig vv mce lt X gt gt ifconfig vvv mce lt X gt Example gt ifconfig vvv mcel plugged QSFP 40GBASE CR4 No separate connector vendor Mellanox PN MCP1600 E00A SN MT1521VS06484 DATE 2015 05 26 compliance level SFF 8636 rev lt 1 5 nominal bitrate 25750 Mbps SFF8436 DUMP 0xA0 128 255 range 0D 00 23 88 00 00 00 00 00 00 00 00 FF 00 00 00 00 00 01 A0 4D 65 6C 6C 61 6E 6F 78 20 20 20 20 A 249 20 BO as WO 0 2 CO HD As tao SIL S5 30 340 21D 45 30 30 41 20 20 20 20 41 32 02 03 04 07 00 3F G2 OO OW OW Ap SA Sil 35 32 Sil Ws D3 BO sly se I 34 20 20 20 a 35 30 95 32 36 20 20 00 OO G7 Bia 31 32 38 38 35 35 32 33 38 44 33 33 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 30 00 00 SFF8436 DUMP 0xA0 0 81 range 0D 05 06 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 17 Mellanox Technologies Rev 3 0 0 4 Performance Tuning In order to improve performance please make sure the HW LRO is enabled PI 4 1 Receive Queue Interrupt Moderation An armed CQ will generate an event when either of the following conditions is met Thenumber of c
4. 2 EEPROM Cable Information Reader RP EEPROM is supported in ConnectX 4 only aw EEPROM cable reading feature allows reading important information about the plugged cable such as cable type cable speed vendor and more In order to read the cable EEPROM info 1 Read the cable information by enabling the following sysctl parameter Output will be printed in dmesg gt sysctl dev mce X conf eeprom info 1 Example gt sysctl dev mce 1 conf eeprom_info 1 dev mce 1 conf eeprom_info 0 gt 0 gt dmesg s Offset Values Mellanox Technologies 16 Rev 3 0 0 Features Overview and Configuration 0x0000 0d 05 06 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0060 00 00 00 00 00 00 00 00 00 00 00 00 01 00 04 00 0x0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0080 0d 00 23 88 00 00 00 00 00 00 00 00 ff 00 00 00 0x0090 00 00 01 a0 4d 65 6c 6c 61 6e 6f 78 20 20 20 20 0x00a0 20 20 20 20 1f 00 02 c9 4d 43 50 31 36 30 30 2d 0x00b0 AS 30 30 4 20 20 AQ 20 Al 32 02 03 3 07 OO Be 0x00c0 0b 00 00 00 4d 54 31 35 32 31 56 53 30 36 34 38 0x00d0 34 20 20 20 si 3 30 3 32 56 20 20 00 00 BT Ba 0x00e0 3 32 98 39 35 35 32 39 Sis 44 328 39 0 0 0 0 0x00 0 00
5. Re Eon a KC ORE ROCA Cae RE 4 Document Revision History isti bn e ERE IE Ru v RE E LT EE dax 5 About this Manual ic 3 5 escne eee oleae epu ies eee deh eee e che ee eens Chee he tk eke 6 Chapter 1 OVerview aov GUERRE CR OC EERE ELAS REN TAT ECC KO 8 1 1 Mellanox OFED EN for FreeBSD Package Contents 00005 8 11 1 Tarball Package cec vero Se AR ERU US E eds 8 112 XS ATV ER is oet rig ae tT e AOI damen a 8 Chapter 2 Installation 4 24a rh Rex VEREECHE E REELED PR 10 2 1 Software Dependencies 0 ccc cette n nes 10 2 2 Downloading Mellanox Driver for FreeBSD esse rer ere 10 2 3 Installing Mellanox Driver for FreeBSD 0 0 0 c eee ee eee 10 2 4 Firmware Programming sese e 11 2 4 1 Installing Firmware Tools ssleeeeeeeee II 11 2 4 2 Downloading Firmware 0 cece cee eect eee ER hen 12 2 4 3 Updating Firmware Using flint eens 12 2 4 4 Setting the Ports toETH 0 0 cece ec I 12 2 5 Driver Usage and Configuration 0 0 0 cece eee rr rese eee 12 Chapter 3 Features Overview and Configuration sesessesesseoesrt 16 3 1 Hardware Large Receive Offload HW LRO 0 0 00 c eee eee 16 3 2 EEPROM Cable Information Reader 20 0 eee eee 16 Chapter 4 Performance Tuning coser Rex AR vig x ERE Ex wi 18 4 1 Receive Queue Interrupt Moderation 0 0 0 ees 18 4 2 Tuning for NUMA Architecture 0
6. SHEAD share mk SYSDIR SHEAD sys clean cleandepend Mellanox Technologies 10 J Rev 3 0 0 Installation c Compile the mlx5_core module Run bmake m HEAD share mk SYSDIR HEAD sys d Install the mlx5_core module Run bmake m HEAD share mk SYSDIR HEAD sys install e Load the mlx5_core module Run kldload mlx5 mlx5en a Go to the mlx5en directory Run cd mlx5 modules mlx5en b Clean any previous dependencies Run bmake m SHEAD share mk SYSDIR S HEAD sys clean cleandepend c Compile the mlxSen module Run bmake m SHEAD share mk SYSDIR S HEAD sys d Install the mIx5en module Run bmake m SHEAD share mk SYSDIR S HEAD sys install e Load the mlxS5en module Run kldload mlx5en To load a module on reboot add mlx5_load YES mlxSen_load YES to the boot loader conf file create 1f does not exist Run kldstat in order to verify which modules are loaded on your server aa 2 4 Firmware Programming The adapter card was shipped with the most current firmware available This section is intended for future firmware upgrades and provides instructions for 1 installing Mellanox firmware update tools MFT 2 downloading FW and 3 updating adapter card firmware 2 4 4 Installing Firmware Tools Step 1 Download the current Mellanox Firmware Tools package MFT from www mellanox com gt Products gt Adapter IB VPI SW gt Firmware Tools The tools pa
7. THE POSSIBILITY OF SUCH DAMAGE Mellanox TECHNOLOGIES m Mellanox Technologies 350 Oakmead Parkway Suite 100 Sunnyvale CA 94085 U S A www mellanox com Tel 408 970 3400 Fax 408 970 3403 Copyright 2015 Mellanox Technologies All Rights Reserved Mellanox Mellanox logo BridgeX CloudX logo Connect IB ConnectX CoolBox amp CORE Direct GPUDirect InfiniHost InfiniScale Kotura Kotura logo Mellanox Federal Systems Mellanox Open Ethernet Mellanox ScalableHPC Mellanox Connect Accelerate Outperform logo Mellanox Virtual Modular Switch MetroDX MetroX MLNX OS Open Ethernet logo PhyX SwitchX TestX The Generation of Open Ethernet logo UFM Virtual Protocol Interconnect Voltaire and Voltaire logo are registered trademarks of Mellanox Technologies Ltd Accelio CyPU FPGADirect HPC X M InfiniBridge LinkX Mellanox Care Mellanox CloudX Mellanox Multi Host Mellanox NEO Mellanox PeerDirect Mellanox Socket Direct Mellanox Spectrum NVMeDirect StPU Spectrum logo Switch IB Unbreakable Link are trademarks of Mellanox Technologies Ltd All other trademarks are property of their respective owners 2 Mellanox Technologies Document Number 2950 Rev 3 0 0 Table of Contents Table of Contents 05 34 shes es ke un A cut RR ERAMRRARCERRESEREERASAKEETEEEXE 3 Lastof Fables exec Sate Ce Ka ae OS Hs ple x
8. dev mce l conf tx queue size max 8192 To modify rings size gt sysctl dev mce 0 conf rx queue size N gt sysctl dev mce 0 conf tx queue size N Note x is the OS assigned interface number To obtain device statistics gt sysctl a grep mce grep stat To obtain additional device statistics gt sysctl dev mce 0 conf debug stats 1 Sysctl a grep mce grep stats gt To show out of receive buffers counter gt sysctl a grep out of rx buffer dev mce l pstats out of rx buffer 0 dev mce 0 pstats out of rx buffer 0 Mellanox Technologies 14 Rev 3 0 0 Installation gt To verify support for Rx Tx pause frames e ifconfig gt ifconfig media Ethernet autoselect 100GBase CR4 lt full duplex rxpause txpause gt sysctl gt sysctl dev mce 0 conf rx pauseframe control dev mce 0 conf rx pauseframe control 1 gt sysctl dev mce 0 conf tx pauseframe control dev mce 0 conf tx pauseframe control 1 To enable disable Rx Tx pause frames sysctl dev mce 0 conf rx pauseframe control 1 sysctl dev mce 0 conf tx pauseframe control 1 Note 0 disable 1 enable To show all supported media gt ifconfig m mce lt x gt supported media media autoselect media 50GBase CR2 mediaopt full duplex media 25GBase SR mediaopt full duplex media 25GBase CR mediaopt full duplex media 100GBase LR4 mediaopt full duplex media 100GBase SR4 mediaopt full duplex media 100GB
9. group level 2 A 1 group level 2 cache level 2 gt cpu count 12 mask f gt 0 1 2 3 4 5 6 7 8 9 10 11 lt cpu gt 2 Tune Mellanox NICs to work on desirable cores Mellanox Technologies 18 J Rev 3 0 0 Performance Tuning a Find the device that matches the interface gt sysctl a grep mce grep mlx dev mce 1 conf device name mlx5 corel dev mce 0 conf device name mlx5 core0 b Find the device interrupts vmstat ia grep mlx5 cored awk print 1 sed s irq sed s 269 270 271 c Bind each interrupt to a desirable core cpuset x 269 1 1 cpuset x 270 1 2 joue ae 29 l 3 d Bind the application to the desirable core cpuset l 1 11 app name sever flag cpuset l 1 11 app name client flag IP Specifying a range of CPUs when using the cpuset command will allow the application to choose any of them This is important for applications that execute on multiple threads The range argument is not supported for interrupt binding 4 2 2 Dual NUMA Architecture 1 Find the CPU list closest to the NIC a Find the device that matches the interface gt sysctl a grep mce grep mlx dev mce 3 conf device name mlx5 core3 dev mce 2 conf device name mlx5 core2 dev mce 1 conf device name mlx5 corel dev mce 0 conf device name mlx5 core0 b Find the NIC s PCI location gt sysctl a grep mlx5 core 0 grep parent dev mlx5 core 0 parent pci3
10. gt Note lt x gt is the OS assigned interface number gt To set stateless offload status gt ifconfig mce lt x gt rxcsum rxcsum txcsum txcsum tso tso lro 1lro Note lt x gt is the OS assigned interface number 13 Mellanox Technologies Rev 3 0 0 To query and set interrupt coalescing modes gt sysctl a grep coalesce mode Example gt sysctl a grep coalesce mode dev mce 0 conf rx coalesce mode 1 dev mce 1 conf rx_ coalesce mode 1 e coalesce mode 0 indicates interrupt timer is resetting with each interrupt event e coalesce mode 1 indicates interrupt timer is resetting with each received packet gt To query and modify values for timer initialization between interrupts gt sysctl a grep tx coalesce usecs Sysctl a grep rx coalesce usecs gt To query and modify values for number of received packets between interrupts SySctl a grep tx coalesce pkts gt sysctl a grep rx coalesce pkts Example gt sysctl a grep rx coalesce usecs dev mce l conf rx coalesce usecs 3 dev mce 0 conf rx coalesce usecs 3 gt sysctl a grep rx coalesce pkts dev mce l conf rx coalesce pkts 32 dev mce 0 conf rx coalesce pkts 32 To query ring size values gt sysctl a grep mce grep size Example gt sysctl a grep mlx grep size dev mce l conf rx queue size 1024 dev mce 1 conf rx queue size max 8192 dev mce l conf tx queue size 1024
11. 0 eee eh 18 4 2 1 Single NUMA Architecture 5 18 4 2 2 Dua NUMA Architect te ox RR ROLE DORT BUS IRL E us 19 Mellanox Technologies 3 J Rev 3 0 0 List of Tables Table 1 Document Revision History Meu cpu ar m Maat dlc erac ep cto a Ar BEDA T Ses eras 5 Table2 Abbreviations and Acronyms 0 00 ccc en 7 Table3 Mellanox OFED EN for FreeBSD Software Components sssesseesrersrrr ere 9 Mellanox Technologies 4 Rev 3 0 0 Document Revision History Table 1 Document Revision History Revision Date Description 3 0 0 November 2015 Initial release 5 Mellanox Technologies Rev 3 0 0 About this Manual This Preface provides general information concerning the scope and organization of this User s Manual Intended Audience This manual is intended for system administrators responsible for the installation configuration management and maintenance of the software and hardware of VPI Infiniband Ethernet in ETH mode adapter cards Mellanox Technologies 6 J Rev 3 0 0 Common Abbreviations and Acronyms Table 2 Abbreviations and Acronyms us Whole Word Description B Capital B is used to indicate size in bytes or multiples of bytes e g IKB 1024 bytes and IMB 1048576 bytes b Small b is used to indicate size in bits or multiples of bits e g 1Kb 1024 bi
12. AMN Mellanox TECHNOLO GIES Connect Accelerate Outperform Mellanox OFED for FreeBSD for ConnectX 4 User Manual Rev 3 0 0 www mellanox com Rev 3 0 0 NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT S AND OR THE SYSTEM USING IT THEREFORE MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT INDIRECT SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES OF ANY KIND INCLUDING BUT NOT LIMITED TO PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY FROM THE USE OF THE PRODUCT S AND RELATED DOCUMENTATION EVEN IF ADVISED OF
13. ase CR4 mediaopt full duplex media 40Gbase LR4 mediaopt full duplex Note lt x gt is the OS assigned interface number The list of supported media is different in ConnectX 4 and ConnectX 4 Lx Toset new media gt ifconfig m mce x media y mediaopt full duplex Note x is the OS assigned interface number lt y gt is the relevant media When updating the media make sure to choose the right cable type Once the driver is loaded both ports will be activated meaning that an ifnet will be created for each port 15 Mellanox Technologies Rev 3 0 0 3 Features Overview and Configuration 3 1 Hardware Large Receive Offload HW LRO HW LRO is supported in ConnectX 4 only Large Receive Offload LRO increases inbound throughput of high bandwidth network connec tions by reducing CPU overhead It works by aggregating multiple incoming packets from a sin gle stream into a larger buffer before they are passed higher up the networking stack thus reducing the number of packets that have to be processed gt In order to turn on the LRO device run gt ifconfig mce lt x gt lro gt In order to turn off the LRO device run gt ifconfig mce lt x gt lro When the LRO device is on HW LRO can be turned on HW LRO is off by default gt In order to turn on HW LRO run gt sysctl dev mce 0 conf hw lro 1 gt In order to turn off HW LRO run gt sysctl dev mce 0 conf hw lro 0 3
14. ckage to download is MFT SW for FreeBSD tarball name is mft X X X tgz For ConnectX 4 you will need at least MFT 4 1 X X X Step 2 Extract the tarball and run the installation script 11 Mellanox Technologies Rev 3 0 0 2 4 Downloading Firmware 1 Retrieve device s PCI slot i e pci0 x 0 0 Run gt mst status 2 Verify your card s PSID gt flint d lt pci gt q 3 Download the desired firmware from the Mellanox website http www mellanox com page firmware_download 2 4 3 Updating Firmware Using flint 1 Before burning a new firmware make sure the modules are unloaded To unload the modules run gt kldunload mlx5en gt kldunload mlx5 2 Unzip the firmware binary file 3 Burn the firmware on your server flint d pci i img bin b 4 Reboot the server 2 4 4 Setting the Ports to ETH If you have a VPI HCA you will need to set the ports to ETH This is done by using the mlxcon fig tool part of the MFT 1 If you have a card with two ports run gt mlxconfig d pci set LINK TYPE P1 2 For the first port gt mlxconfig d pci set LINK TYPE P2 2 For the second port 2 Reboot the server 2 Driver Usage and Configuration Interface name has changed from mlx5en to mce Note that ifconfig and sysctl commands 7 were updated accordingly Ae To assign an IP address to the interface gt ifconfig mce x ip Note x is the OS assigned interface numbe
15. linuxapi 2 2 Downloading Mellanox Driver for FreeBSD 1 Verify that the system has a Mellanox network adapter HCA NIC installed The following example shows a system with an installed Mellanox HCA mlx5_core0 pci0 6 0 0 class 0x020000 card 0x000815b3 chip 0x101315b3 rev 0x00 hdr 0x00 vendor Mellanox Technologies device MT27620 Family class network subclass ethernet mlx5_corel pci0 6 0 1 class 0x020000 card 0x000815b3 chip 0x101315b3 rev 0x00 hdr 0x00 vendor Mellanox Technologies device MT27620 Family class network subclass ethernet 2 Download the tarball image to your host The image name has the format MLNX_OFED_FreeBSD lt ver gt tgz You can download it from http www mellanox com gt Products gt Software gt Ethernet Drivers gt FreeBSD 3 Use the md5sum utility to confirm the file integrity of your tarball image 2 3 Installing Mellanox Driver for FreeBSD FreeBSD v3 0 0 supports adapter cards based on the Mellanox ConnectX 4 family of adapter IC P devices only If you have ConnectX 3 and ConnectX 3 Pro on your server you will need to install E FreeBSD v2 1 6 driver For details on how to install FreeBSD v2 1 6 driver please refer to FreeBSD v2 1 6 User Manual at 1 Extract the tarball 2 Compile and load needed modules in the following order of dependencies mlx5 core a Go to the mlx5 directory Run cd mlx5 modules mlx5 b Clean any previous dependencies Run bmake m
16. nsertion e ifnet statistics 1 1 Mellanox OFED EN for FreeBSD Package Contents 1 1 4 Tarball Package Mellanox OFED EN for FreeBSD package includes the following directories mlx5 modules contains the relevant Makefiles for mlx5 core and EN drivers net mlx5 EN source code drnvex mlx5 generated freebsd core source code 1 1 2 mlx5 driver mlx5 is the low level driver implementation for the ConnectX 4 ConnectX 4 Lx adapters designed by Mellanox Technologies Mellanox Technologies 8 J Rev 3 0 0 Overview 1 1 2 1 Software Components Mellanox OFED EN for FreeBSD contains the following software components Table 3 Mellanox OFED EN for FreeBSD Software Components Components Description mlx5 Acts as a library of common functions required by the ConnectX 4 ConnectX 4 Lx adapter cards For example initializing the device after reset mlx5en Handles Ethernet specific functions and plugs into the ifnet mid layer Documentation Release Notes User Manual 9 Mellanox Technologies Rev 3 0 0 2 Installation This chapter describes how to install and test the Mellanox driver for FreeBSD package on a sin gle host machine with Mellanox adapter hardware installed 2 4 Software Dependencies To install the driver software kernel sources must be installed on the machine To load mlx5 linuxapi must be loaded as well e Compile and install linuxapi module under sys modules
17. ompletions generated since the one which trigged the last event genera tion reached a set in advance number The timer has expired and an event is pending The timer can be set to be restarted either upon event generation or upon completion generation Setting the timer to be restarted upon completion generation affects the interrupt receiving rate When receiving a burst of incoming packets the timer will not reach its limit therefore the inter rupt rate will be associated to the size of the packets Inorder to modify the timer restart mode run gt sysctl dev mce 1 conf rx coalesce mode 0 1 0 For timer restart upon event generation 1 For timer restart upon completion generation Inorder to modify the number of completions generated between interrupts run gt sysctl dev mce 1 conf rx coalesce pkts x gt In order to modify the time for the timer to finish run gt sysctl dev mce 1 conf rx coalesce usecs x Note The default values are e dev mce 1 conf rx_coalesce_mode 1 Timer restarts upon completion generation e dev mce l conf x coalesce pkts 32 32 completions generate interrupts e dev mce l conf x coalesce usecs 3 Timer count down 3 micro sec 4 2 Tuning for NUMA Architecture 4 2 1 Single NUMA Architecture When using a server with single NUMA no tuning is required Also make sure to avoid using core number 0 for interrupts and applications 1 Find a CPU list gt sysctl a grep
18. r To check driver and device information gt pciconf lv grep mlx d flint d pci w x y z C gt flint d pci0 6 0 0 dc grep Description Mellanox Technologies 12 Rev 3 0 0 Installation Example gt pciconf lv grep mlx C 3 mlx5_core0 pci0 33 0 0 class 0x020000 card 0x001415b3 chip 0x101315b3 rev 0x00 hdr 0x00 vendor Mellanox Technologies device MT27620 Family class network subclass ethernet mlx5_corel pci0 33 0 1 class 0x020000 card 0x001415b3 chip 0x101315b3 rev 0x00 hdr 0x00 vendor Mellanox Technologies device MT27620 Family class network gt flint d pci0 33 0 0 q Image type FS3 FW Version 12 12 0610 FW Release Date 3 9 2015 Description UID GuidsNumber Base GUID e41d2d03006094ec 20 Base MAC 0000e41d2d6094ec 20 Image VSD Device VSD PSID MT 2190110032 gt flint d pci0 6 0 0 dc grep Description Description ConnectX 4 VPI adapter card EDR IB 100Gb s and 100GbE dual port QSFP28 PCIe3 0 x16 ROHS R6 To check driver version gt sysctl a Example Sysctl a grep Mellanox dev mlx5 core 1 desc Mellanox Ethernet driver 3 0 0 RC2 dev mlx5 core 0 desc Mellanox Ethernet driver 3 0 0 RC2 To check firmware version dmesg dmesg Example lx5 core0 INFO firmware version 12 12 2008 sysctl Sysctl a Example dev mlx5 core 0 hw fw version 12 12 2008 To query stateless offload status gt ifconfig mce lt x
19. ts FW Firmware HCA Host Channel Adapter HW Hardware IB InfiniBand LSB Least significant byte Isb Least significant bit MSB Most significant byte msb Most significant bit NIC Network Interface Card SW Software VPI Virtual Protocol Interconnect PFC Priority Flow Control PR Path Record RDS Reliable Datagram Sockets RoCE RDMA over Converged Ethernet SL Service Level QoS Quality of Service ULP Upper Level Protocol VL Virtual Lane Support and Updates Webpage Please visit http www mellanox com gt Products gt Software gt Ethernet Drivers gt FreeBSD Driv ers for downloads FAQ troubleshooting future updates to this manual etc 7 Mellanox Technologies Rev 3 0 0 1 Overview This document provides information on the Mellanox EN driver for FreeBSD and instructions for installing the driver on Mellanox ConnectX adapter cards supporting the following uplinks to servers ConnectX 4 Ethernet 10GigE 25GigE 40GigE 50GigE and 100GigE ConnectX 4 Lx e Ethernet 10GigE 25GigE 40GigE and 50GigE The driver release introduces the following capabilities e Single Dual port Number of RX queues per port according to number of CPUs Number of TX queues per port according to number of CPUs e MSI X or INTx Hardware Tx Rx checksum calculation Large Send Offload 1 e TCP Segmentation Offload Large Receive Offload e VLAN Tx Rx acceleration Hardware VLAN stripping i
Download Pdf Manuals
Related Search