Building Highly Efficient Red Hat Enterprise Virtualization 3.0 Cloud
Contents
1. M You should now see the logical network name under the column Network Name for this interface 32 Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design Figure 36 Displaying the Network Interface of the Logical Network Expand All Collapse All 2 Hosts storage Virtual Machines 1emplates mw Es Events d ystem Edit Maintenance Assign tags 3 Default Name Host IP Cluster Status Load Memory CPU Network Spm Status E B storage 4 i xena001 lab mtl com 172 30 40 101 Default Up pem de 0 None E Templates Ela Clusters HL E ijj Default jj Hosts I 1 xena001 lab mtl com y iA VMs General Virtual Machines Network Interfaces Host Hooks Permissions Ei Events 2 Name Address MAC Speed mbps RX Mbps TX Mbps Drops Pts Bond Vian Network Name wv eths 192 168 39 10 00 00 00 00 03 01 10000 1 1 0 EoIB wv eth2 00 30 48 C4 40 1B 0 1 1 0 4A ethi 172 30 40 101 00 30 48 C4 40 1A 1000 1 lt i 0 rhevm 4 eth4 00 00 00 00 02 01 10000 y 1 0 4 7 Add an Interface to VM Step 1 Go to the VMs pane Step 2 Click on Network Interface tab Step 3 Click on New button a pop up will open Figure 37 Virtual Machine Network Interfaces View Expand All CollapseAll 23 Virtual Machines 4 3 system New Server New Desktop Edit Remove gt v Make Template Export Move E Guide2. Expand All Collapse Di 2 Virtual Machines B system New Server New Desktop E Guide Me 3 B Default Name Cluster Host IP Address Memory CPU Network Display Status Uptime Log Storage D vmi Default 0 0 0 Down Templates E y Y Clusters amp New Virtual Machine Guide Me 1j Default LJ Hosts Configuration completed xena001 lab r Optional actions e Add another Network Interface Add another Virtual Disk f Configure Later e Figure 30 Adding a New Virtual Machine VMs Screen r Walras Mies um gr y pex z aa rm Ja Zeg rd H i gt e e d t Us a ae ti Mame Cw Moat P Accrets Memory CPU Meteor Display forms LIA Loggec n iner y H wnt en Oefaut v rw ds Down Step 10 Right click on the line of the VM and choose Run Step 11 In order to start the VM console right click and select Console Figure 31 Adding a New Virtual Machine VNC Screen Expand All Collapse All e VIN PASS New Server New Desktop Edit TUN y Migrate amp Guide Me Assign tags System gt B Default Name Cluster Host IP Address Memory CPU Network Display Status Uptime 8 Storage gt own NC Up 3 min Templates a vmi Console co E 2 gt next seruer TFTP server L Clusters USEE ELABS sic lt not specified gt Default Ctrl X exit configuration utility 30H manufacturer Red Hat osts PXE autoboot LI xena00L lab r netO 00 1a 4a 1e 28 00 o
3. Ethernet Switch Optional 1GE Used for Management network Management network can be done over a separate IB partition as well Server refer to the UFM User Manual specific Used for UFM application server information Server refer to the VSA User Manual specific Used for VSA application server information Server refer to Red Hat Enterprise Virtualization Used for RHEV M application 3 0 Installation Guide Server refer to Red Hat Enterprise Linux 6 2 Used as virtual machine VM hosts in the Installation Guide clusters VL Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design 2 3 Required Software Packages e Mellanox OFED Driver Please contact cloudsupportOmellanox com to obtain this package e Unified Fabric Manager UFM e Mellanox Storage Accelerator VSA version 2 1 1 1 e Mellanox Network Manager MNM version 1 0 Please contact cloudsupport mellanox com to obtain this package e Red Hat Enterprise Linux RHEL 6 2 or higher e Red Hat Enterprise Virtualization 3 0 RHEV RHEV M or higher Mellanox Technologies packages are supported by Mellanox and not included in the Red Hat distributions d 9 Mellanox Technologies Rev 1 0 Software Installation and Basic Configuration 3 Software Installation and Basic Configuration Integrated cloud solution contains several
4. Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design Figure 4 Red Hat Network Register Copyright Am 2006 2016 lid Hat Inc ALL rights reserved 2121101010000 001001000000000000000101000010001000000101001000001000000000100001000040000101004100000000001000011000101000400200010000100000000001000110000000000000000 z pab Alt Iab between slamants Z lis naxt screen Step 5 Step 6 Step 7 d 11 Subscribe to the required channels Run reo oEC hose4 1 ran channel adc channel rhel xo 6 CI Se aie eyes Username meldcs Password root host2 1 rhn channel add Xe mdetiouore d 3 leew Omi S36 Ee Username meldcs Password eno ed los E ME Selber channel rhel s6 CA Server supplementary AS Username meldcs Password ooi dosi 21 s Confirm the list of channels to which the server 1s subscribed LrootChoseZ cine EE Ia ios S SIS ico rhel A SIS rhel x260 6d Server C rnevm gt oou hoc 2d If installed the classpathx jaf package must be removed It conflicts with some of the components installed to support JBoss root Chost Ir yum remove Classpathx jar Loaded plugins product 1d rh npluglin secas VS crio Eon manager Updating certificate based repositories Unable to read consumer identity Setting up Remove Process No Match for vargqumenk css spa Joar OM ane u
5. 45 Connect the Logical Sever to the NetWork nnnsnsennnnnnennnnnnsenrnnnsrnnrressrnrrrnsrnrrressnnrrresenne 39 Figure 46 UFM Network Connected to the UFM Logical Gener 39 ss Mellanox Technologies Rev 1 0 Contents List of Tables Fable t Required al OWAN Emo momen 8 Fable 2 sro melsted ME ett En dd oa de cL ah 42 A NNNM Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design Introduction This reference design describes how to integrate and use Red Hat Enterprise Virtualization Manager RHEV M to control a cloud based on e Servers with Red Hat OS and KVM e Mellanox products for network connectivity and storage Through significant customer engagements building data centers and working closely with IaaS architects and administrators Mellanox in collaboration with Red Hat formed a new architecture which enables an integrated computing network and storage technology cloud solution Through intelligent discovery awareness and automation the new joint solution provides the highest levels of virtualization and application performance The new collaboration is designed to deliver a high performance and efficient cloud infrastructure Performance application service levels security and usability no longer need to be compromised and importantly users will benefit from the most cost effective cloud infrastru
6. Hat Enterprise Linux 6 x x64 OK Cancel 26 Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design Step 3 Select VNC protocol in Console tab Figure 23 Adding a New Virtual Machine Console Expand All Collapse All 2 Virtual Machines 30 System New Server New Desktop 5 B Default Name New Server Virtual Machine Upt 8 Storage No results found m 7 Templates rotocol 3 Clusters USB Policy Disabled 1j Default Hon 3 j Hosts O xena001 lab r High Availability Resource Allocation Boot Options Custom Properties Cancel LZ Step 4 In the Host tab select the host you want the VM to run on Figure 24 Adding a New Virtual Machine Host Expand All Collapse All 2 Virtual Machines 3 Gi System New Server New Desktop E i bc 76 B Name New Server Virtual Machine LD PC O status Seier No results found Templates General Run On z Clusters Console 9 Any Host in Cluster Specific xena001 lab mtl com y Te j Hosts Run Migration Options O xena001 lab r High Availability Run VM on the selected host no migration allowed es Resource Allocation 7 Allow VM migration only upon Administrator specific request Boot Options system will not trigger automatic migration of this VM Custom Properties OK Cancel A 27 Mellanox Technologies R
7. b00506 3 14 il PUNATA 2 3600605b0032882501643ddec0204767e 890GB 0 LSI MR9265 8i 007e760402ecdd431650822803b b00506 3 14 1 PUNTI Ag 3 36000605p00328607601643c9ecd0d3de2c 890GB 0 DT MRIZ6S 31 W0Zededsdiece 34s leche Zs sb0 0S 0G 5 714 1 VSA add servers rhev servers ips 192 168 20 101 192 168 20 103 VSA add targets iqn iser 1 transport iser volumes dl VSA set targets iqn iser l server rhev servers VSA save For Mellanox VSA installation and advance configuration refer to the Mellanox VSA User Manual 3 5 Mellanox UFM Installation Host1 Mellanox s Unified Fabric Manager UFM is a powerful platform for managing scale out computing environments UFM enables data center operators to efficiently monitor and operate the entire fabric boost application performance and maximize fabric resource utilization Ll s Mellanox Technologies Rev 1 0 Software Installation and Basic Configuration For Mellanox UFM installation and basic configuration refer to the Mellanox UFM User Manual 3 6 Mellanox Network Manager Plugin 3 6 1 Installing Mellanox Network Manager Server Copy the file mellanox nm server tar gz to the server that is running UFM cd tmp E A IS Se it ene Ive cd mellanox nm server Noe alas ENS EM CN d 3 6 2 Installing Mellanox Network Client Copy the file mellanox nm client tar gz to each server in the fabric cd tmp deu cmedLlcsusiop rogue o dba Sol eens ope cd mellanox nm
8. client e Pulse d re illie s dal 20 Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design 4 RHEV Manager Configuration Before you begin configuring your Red Hat Enterprise Virtualization Manager RHEV M make sure it is installed and running 4 1 Add Data Center To add a new data center in your RHEV M portal click on the New Data Center button Figure 10 New Data Center New Data Center Ay e Name dcs1 Description Type ISCSI v Compatibility Version 3 0 OK Cancel 4 2 Add Cluster To add new cluster for your data center in your RHEV M click on the New Cluster button Figure 11 New Cluster New Cluster DNE o lt lt Memory Optimization Resilience Policy Name cluster Description CPU Name Intel Conroe Family v Compatibility Version 3 0 OK Cancel o as Mellanox Technologies Rev 1 0 RHEV Manager Configuration Figure 12 Data Center ENTERPRISE CH VIRTUALIZATION Logged in user admin internal Sign out Configure About Guide Search 1 x r GO Expand Ali Collapse Al Data Centers Clusters Hosts Storage Virtual Machines Templates ay System lp Guide Me Tree Name Storage Type Status Compatibility Version Description R LE e dest ISCSI Uninitialized 3 0 E f Templates E K Clusters 3 cluster Hosts A VMs 5 t
9. following documents Table 2 List of Related Documents Document Location Red Hat Enterprise Virtualization 3 0 Installationlhttp docs redhat com docs en US Red Ha Ente prise Virtualization 3 0 pdf Installation Guide Red Hat Enterprise Virtualization 3 0 Installati on Guide en US pdf Mellanox OFED User Manual www mellanox com Products Adapter IB VPI SW Linux SW Drivers http www mellanox com content pages php pg Mellanox UFM User Manual http licensel mellanox com Mellanox VSA User Manual http licensel mellanox com Mellanox Cloud Interface ccs Please contact Reeg com SE Hat Low Latency Performance Tuning https access redhat com knowledge articles 221 e com knowledge articles 221 153 Mellanox Low Latency Performance Tuning Performance Tuning Guidelines for Mellanox Network Adapters 42 Mellanox Technologies
10. following entry root ALL ALL ALL Enable SSH access for root user Add etc ssh sshd config file has the entry posmaiteb oobis Og es Restart the SSH server in case of a change in the etc ssh sshd config file roota host3 service sshd restart For advanced configuration of the RHEV M refer to Red Hat Enterprise Virtualization 3 0 Installation Guide GLEN Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design 3 3 Mellanox OFED Driver Installation All Hosts Any host in the fabric shall have Mellanox OFED installed Follow those steps for basic Mellanox OFED installation on all hosts Step 1 Make sure you download Mellanox OFED from www mellanox com and locate it in your file system Step 2 Install Mellanox OFED from the source yum install libstdc devel flex bison gcc c t 4 libstdc devel zlib devel libtool glibc devel gcc kernel devel rpm build iscsi initiator Utils rodha rpm c coniig tel deval Step 3 Download the OFED iso Run mkdir mnt tmp H moune O oo MA ORED INU a a ee mnt tmp cd mnt tmp mlnxofedinstall Step 4 Reboot the server in case the firmware is updated Step 5 Verify Mellanox OFED installation When running ibv devinfo you should see an output similar to this Hostia toy moles EE hea id md 30 1S le SUIS TO ONSE InfiniBand 0 fw ver UN node guid
11. is listening on TCP port netstat nap grep libvirtd Eep 0 UNO DEG EID EES HESE LISTEN SA Ao Ed Step 2 From the source run PULLIN Eet Eeer mos Syerem eepsod Li cd es Where target host is the host name of the target The command should return without errors Step 3 Check that the file etc sysconfig libvirtd has the following lines LSV Tee ll 1 Saee DPAENONE O Ehe IPTE EE Step 4 Check that the port libvirtd is not blocked by a firewall 7 3 Connection Verification of Virtual Machines Using elPolB Check that you can run the command virsh list without errors If you get a prompt for authentication edit the file etc 1ibvirt libvirt conf by changing this line su E me aos awe ees t To SO aos ewe MENE 7 4 Low Latency Performance Tuning The below links provides a tactical tuning overview of Red Hat Enterprise Linux 6 for latency sensitive workloads and describes important tuning parameters and settings that can improve performance for Mellanox adapters Each setting along with its potential effect is described to help in making an informed judgment concerning its relevance to the user s system the system workload and the performance goals e Performance Tuning Guidelines for Mellanox Network Adapters e Low Latency Performance Tuning Guide for Red Hat Enterprise Linux 6 41 Mellanox Technologies Rev 1 0 Related Documentation 8 Related Documentation For additional information see the
12. 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design Step 6 Click on Login locatedon the right of your chosen target Figure 18 Login to Target ExpandAll Collapse All 2 rei New Domain New Domain Import Do 3 9 System Name storel a Default Domain Name Data Center Default DA No results found Templates EA AA Data iSCSI v Format e Clusters Use Host xena001 lab mtl com v Discover Targets Address 192 168 20 120 User Authentication Port 3260 CHAP user name CHAP password Discover Login Target Name Address Port iqn ramdisk5 192 168 20 120 3260 Login LUNs gt Targets Targets gt LUNs ign rhev iser 1 192 168 20 120 3260 Step 7 Choose the LUN you wish to add and click OK Figure 19 Choosing an LUN Search Storage datacenter Default comico A e New Domain y Q System New Domain Import Dome Name storel B Default Domain Name Data Center Default Dido a No results found d Templates EAS EA Data iSCSI Y Format E Y Clusters Use Host xena00llab mtl com v Discover Targets i Address 192 168 20 120 User Authentication E Port 3260 CHAP user name CHAP password Discover Login All Target Name Address Port iqn ramdisk5 192 168 20 120 3260 Login VD z A Vi E m KE li ke 2 y E L ud ke A Cd z 2 iqn rhev iser 1 192 168 20 120 3260 LUN ID De
13. Add a Network to feCluster 31 47 deb ahinteraece toV Vois a e 33 5 Using UFM to Automate Network Mangement eeeeeeeeeeeeeeeeeer nennen 36 5 1 Basic UFM Configuration Flow eeessssesesesseeeeee nennen nnne nnne nennen nnn nnn 37 Mellanox Network Manager Plugin eeec eese eere erui eene nennen nne n nnnm n nnn nn nnn 40 Rei EE Le Le due tot 41 7 1 Hostis Not Added to Logical Server in UEM 41 7 2 Migration of VM Fails lssseeessessseeeeeeeenneenn nennen nennen nnns nnn nnne nnn nans 41 7 3 Connection Verification of Virtual Machines Using elo 41 TA bowiatency Performance UAM DE 41 9 Related Documentatlon croco eege E Pos SLE VEL CU Peau Ee CEN E 42 as Mellanox Technologies Rev 1 0 Contents List of Figures Figure 1 Basic Test Bed Scenario Exvample sse nennen nennen nnn nnns 8 Figure 2 Red Hat Enterprise Linux Installation cccccccccccecseeeeeeeeeeeeeeeeeeseeeceesssaeeeeeeessaaeeeeeessaeeeees 10 Figure 3 Red Hat Enterprise Linux Installation cccccccscccccsseeeeeeceeeeseeeeseeeeceeesseeeeceeessaaeeeeesssageeeees 10 Figure 4 Red Hat Network Register cccccoonnccncccccocconccccoonconncononncnnnnonnnncnnononnnncnnnnononnrnnnnonanrrnnnnnnnnnoss 11 ROUS Os ARIE VEN IP OM alain 14 Figure 6 Red Hat Enterprise Linux Installation 000nnnnn00annneneannnnnnennnnensennnnns
14. Default Storage Logical Networks Clusters Permissions Ka Events Attach Data Domain Name Domain Type Status Free Space Used Space Total Space No results found 4 3 Add Host Follow these steps in order to add a host Step 1 Go to Hosts tab and click on New Figure 13 Adding a Host ESA bacione e ERI Logged in user admin internal Sign out Configure About Guide Search Host datacenter Default x GO Expand All Collapse All 2 Data Centers Clusters Hosts Storage Virtual Machines Templates 2 System New T Default Name HostlP Cluster Status Load Memo CPU Network SpmStatus 8 Storage No results found New Host Templates Clusters General Data Center Default Power Management Host Cluster Default Bookmarks Name Tags Address Root Password Step 2 Fill in the details as desired After filling in the required details the installation starts NOTE If you intend to use UFM or Mellanox Network Plugin it is necessary that the hostname you give the host you add is the same as the hostname in UFM 4 A 22 Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design Figure 14 Installation in Progress Search Host datacenter Default x dr GO Expand All Collapse All 2 Data Centers Clusters Hosts Storage Virtual Machines Templates P Lal System New e
15. Me Assign tags Tree d 8 Default Name Cluster Host IP Address Memory CPU Network Display Status Uptime Logged in User t 8 Storage Templates Clusters tj Default j Hosts O xena001 lab r r vmi amp Default 0 0 0 Down BOOKMArKS 1ags General Network Interfaces Virtual Disks Snapshots Applications Permissions New Name Network Name Type MAC Speed mbps RX mbps TX Mbps Drops Pkts nici rhevm Red Hat Vi 00 1a 4a 1e 28 00 1000 1 lt 1 0 d 33 Mellanox Technologies Rev 1 0 RHEV Manager Configuration Step 4 Fill in the details for the HCA Figure 38 Adding a New Network Interface Expand All Collapse All 2 Virtual Machines 9 System New Server New Desktop Edit Remove P Make Template Export Move E Guide Me Assign tags B Default Name Cluster Host IP Address Network Display Status Uptime L T Samma fal 0 Templates 3 Clusters Default New Network Interface Y e j Hosts xm 0 xena001 lab r Network EoIB Type Red Hat VirtlO F Specify custom MAC address General Network Interfaces Virtual Disks S New Name Network Name Type MAC Speed mbps RX Mbps TX mbps Drops Prts nici rhevm Red Hat Vi 00 1a 4a 1e 28 00 1000 1 1 0 You should now see the new added network interface Figure 39 Displaying the New Network Interface Expand All Collapse All 2 Virtual Machines B 8 Q system New Server Ne
16. Mellanox TECHNOLOGIES Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design Rev 1 0 November 2012 www mellanox com Mellanox Technologies NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO S AND OR THE SYSTEM USING IT THEREFORE MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT INDIRECT SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES OF ANY KIND INCLUDING BUT NOT LIMITED TO PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT NCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY FROM THE USE OF THE PRODUCT S A
17. ND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE Mellanox TECHNOLOGIES Mellanox Technologies Mellanox Technologies Ltd 350 Oakmead Parkway Suite 100 Beit Mellanox Sunnyvale CA 94085 PO Box 586 Yokneam 20692 U S A Israel www mellanox com www mellanox com Tel 408 970 3400 Tel 972 0 74 723 7200 Fax 408 970 3403 Fax 972 0 4 959 3245 Copyright 2012 Mellanox Technologies All Rights Reserved Mellanox Mellanox logo BridgeX ConnectX CORE Direct InfiniBridge InfiniHost InfiniScale PhyX SwitchX Virtual Protocol Interconnect and Voltaire are registered trademarks of Mellanox Technologies Ltd Connect IB FabricIT MLNX OS ScalableHPC Unbreakable Link UFM and Unified Fabric Manager are trademarks of Mellanox Technologies Ltd All other trademarks are property of their respective owners Some of the text and illustrations in this document have been incorporated and or modified from https access redhat com knowledge docs en US Red_Hat_Enterprise_Virtualization 3 0 pdf Installation_Guide Red_Hat Enterprise_Virtualization 3 0 Installation_Guide en US pdf which is licensed by Red Hat under a Creative Commons Attribution Share Alike 3 0 Unported license CC BY SA An explanation of CC BY SA is available at http creativecommons org licenses by sa 3 0 l 2 Document Number 4008 Mellanox Technologies Contents Rev 1 01 Cont
18. QUOI e303 00 E Sore ee quid AOS als Wemclore ills Ox Aes vendo Ose ele 26428 hw ver 0x BO pomo EECHER poo EE 2 POCTE 1 state AOS ACI 2 max mtu 2048 4 active miU 2048 4 smeli d 24 poma 22 Portae RES Ee IB oou 2 Bisel Le PORTERA TIVER max mtu 2048 4 active miU 1024 3 MELLA 0 Ogee al 0 Pori ime 0x00 dote doe eu Ethernet Step 6 Set up your IP address for your 1b0 interface by editing the ifcfg ib0 file and running ifup as follows If your kernel version does not match with any of the offered pre built RPMs you can add your kernel version by using the mlnx add kernel support sh script located under the docs directory For further information please refer to MINX OFED User Manual Section Pre installation Notes mInx add kernel support sh tool VN A Mellanox Technologies Rev 1 0 Software Installation and Basic Configuration 3 4 vl Vetey sysconfig network scripts 1fcfg ribO DEVICE 1b0 BOOTPROTO none ONBOOT yes IPADDR 192 168 20 103 NE VANS GOS Oho A EI NM CONTROLLED yes TYPE Infiniband oO Step 7 Add elPoIB Interfaces Make sure the host is connected to an InfiniBand network and that you have the latest Mellanox OFED that supports eIPoIB Step 8 Locate the interface Make sure that you have the following line in the file etc infiniband openib conf E IPOIB LOAD yes Pehee ESTO IDE ESO se oo e ord eser vedo ud eerte heel CS doll pee S as follows e
19. V Manager a few moments to start up additional information SSL Certificate ina EE A BR Dor Co DD oo Cao Pec ao BoR E Do m ee EE aque e ae er ecr EA EE 18 95 tae a a eO lez The firewall has been updated the old iptables configuration file was seee uo use elacuseu icv comm e tee Secas 10495707 512 0012 5 20 9 lt The installation og file i gt available at e EE 10 47 1 5 Log Please use the user admin and password specified in order to login into RHEV Manager x TO Configure additional users Tirst Configure authentication domains using the rhevm manage domains utility A To access REEV Manager please go TO the following URL EECH Root mese 2 d res To ensure that the installation does not fail make sure that the locale settings are as follows host locale Ies erm US UTES LC CTYPE en US UTF 8 ILES EN Dae SEC eet Ores UMM ien E IM 10 5 MIRI ie Ie OLEATE s a Us UTET I MONETARY cen U LORI te EC EMNE O ACE en U WE LC PAPER en US UTF 8 EE ee EO TADDRE eme US US Ee LO ABRIL Ge tie SUREMENT M IIS UTET DEN ON SS AMO LC ALL NOTE You can access the administration portal by pointing your Internet Explorer to 4 http your_server 8080 assuming you followed the defaults A 13 Use the administrator username and password you supplied in the configuration step You Mellanox Technologies Rev 1 0 Software Installation and Basic Configuration will be instructed t
20. ackets errors B dropped B overruns B frame B TX packets B errors B dropped overruns B carrier B collisions B txqueuelen 18088 RX bytes B 8 8 b TX bytes B 8 8 b Link encap Local Loopback inet addr 127 8 80 1 Mask 255 6 6 86 inet6 addr 1 128 Scope Host UP LOOPBACK RUNNING MTU 16436 Metric 1 RX packets 1847 errors B dropped overruns B frame B TX packets 1847 errors B dropped B overruns B carrier B collisions txqueuelen RX bytes 72781 78 9 KiB TX bytes 2781 78 9 KiB A Me Assign tags twork Display 0 VNC d Status Up root localhost 14 ifconfig ethi 192 168 39 281 24 up B821q adding VLAN 8 to HW filter on device ethi ou have neu mail in var spool mail root Name root localhost 14 _ Drops Pkts nicl rhevm Red Hat Vi 00 1a 4a 1e 28 00 1000 1 1 0 nic2 EoIB Red Hat Vi 00 1a 4a 1e 28 01 1000 1 SOR 0 35 Mellanox Technologies Rev 1 0 Using UFM to Automate Network Mangement Using UFM to Automate Network Mangement Mellanox s Unified Fabric Manager UFM is a powerful platform for managing scale out computing environments UFM enables data center operators to efficiently monitor and operate the entire fabric boost application performance and maximize fabric resource utilization UFM s automated and application centric approach bridges the gap between servers applications and fabric elements thus enabling administrators to manage and optimize fr
21. cres 10 eed Tan asadas ere GEES VDS daemon server is running d 23 Mellanox Technologies Rev 1 0 RHEV Manager Configuration Figure 16 Host is UP Hosts New Edi Mantenance Assign tags Mame Hostir Cluster Status Load Memory CPU Network Spm Status 4 1 enago 172 30 40 106 cluster1 Up o ww _ und 0 None 4 4 Add Storage Perform the following operations in order to add storage domain using RHEV M Step 1 Connect to the RHEV M Portal Step 2 Click System gt Default gt Storage gt New Domain Step 3 Enter a name for the domain Step 4 Enter an IP of the VSA host Step 5 Click on Discover Figure 17 Discovering Targets Search Storage datacenter Default ExpandAll Collapse All 3 e New Domain New Domain Import Dome System Name storel a B Defautt Domain Name Data Center Default SCSI EE No results found Templates AAA A Data iSCSI Y Format 0 Clusters Use Host xena001 lab mtl com Bookmarks v Discover Targets Address 192 168 20 120 T User Authentication Tags Port 3260 CHAP user name CHAP password Discover Login All Target Name Address Port ign ramdisk5 192 168 20 120 3260 Login uv z 2 A Kal FI ES KE o gt 2 E o ic Ed z gt iqn rhev iser 1 192 168 20 120 3260 Login OK Cancel 24 j Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization
22. cture The purpose of this document is to describe virtualization networking management with RHEV M as cloud orchestrator and Mellanox InfiniBand HCAs and switches as a fabric interconnect This reference architecture demonstrates how to build a fully integrated InfiniBand FDR interconnect cloud infrastructure with RHEV M This reference architecture covers the installation and setup of the infrastructure including e Installation and configuration of the RHEV and Mellanox components Adapters switches storage accelerator and fabric manger e Datacenter configuration various configuration flows needed to operate the network e Monitoring and troubleshooting VN 121 Mellanox Technologies Rev 1 0 Getting Started 2 Getting Started 2 1 Basic Test Bed Configuration Figure 1 Basic Test Bed Scenario Example Host1 IP 172 30 40 104 Host IP 172 30 40 118 Host2 1 IP 172 30 40 147 mE Host3 IP 172 30 40 106 ER Host4 IP 172 30 40 120 1GE Management Network Optional InfiniBand nfiniBar d Management Ethernet 3 Management External Ethernet Network s InfiniBand SS Management InfiniBand Er VSA 2 2 Required Hardware Table 1 Required Hardware Mellanox SX6036 InfiniBand Gateway switch Used for data storage networks OR The gateway functionality is used for connecting Grid Director 4036E InfiniBand Gateway switch to external Ethernet networks
23. der this logical server cluster will be connected Figure 45 Connect the Logical Sever to the Network gt 3 Properties ad i Edit User Attributes a n p E e e Ki Monitor X Delete Logical Server Connect to mix6 Figure 46 UFM Network Connected to the UFM Logical Server rin LSI Refer to UFM User Manual for advanced configuration options 39 Mellanox Technologies Rev 1 0 Mellanox Network Manager Plugin Mellanox Network Manager Plugin Mellanox Network Manager Plugin performs seamless integration between Mellanox UFM and the RHEV Manager After installing the plugin see section 3 6 Mellanox Network Manager Plugin the interconnectivity between the hosts in the network over eIPoIB interface is performed seamlessly For advanced configuration please contact cloudsupport mellanox com 40 j Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design 7 Troubleshooting 7 1 Host is Not Added to Logical Server in UFM Check that you see the server in UFM If it does not appear there run eme o es Glas abe aloeuaol lost 0 ade dese The output should be something other than localhost HCA 1 You can change it by running for example Tecmo webl Heal gt sys class inminiband mix4 0 mode desc 7 2 Migration of VM Fails Step 1 Check that libvirtd on the target
24. e Default Name Host IP Cluster Status Load E B Storage dr 1 xena003 lab mtl com 172 30 40 103 Default Installing 0 VMs 0 Templates E E t 1 Clusters S 1ags Memory CPU Network SpmStatus 0 0 None After finishing the installation the installer restarts the host The virtual desktop and server manager daemon should be up and running Figure 15 Installation Complete WW Red Hat Enterprise Virtualization Man 9 ENTERPRISE VIRTUALIZATION Logged in user admin internal Sign out Configure About Guide Search Host datacenter Default Expand All Collapse All 2 Data Centers Clusters Hosts Storage Virtual Machines La System New E Default Name Host lP Cluster Status I B Storage X 1 xena003 lab mtl com 172 30 40 103 Default Reboot t Templates E E clusters 3 Vi a R fa x r GO Templates Load Memory CPU 0 VMs 0 0 fy xD sm y Page y Network SpmStatus 0 None Last Message 2011 Nov 03 13 37 53 Host xena003 lab mtl com installed Step 3 Add storage support by applying the iscsi py patch to enable SER support NOTE You should get the iscsi py patch from Mellanox support ready for RHEV 3 0 For RHEV 3 1 and up you should skip this section be Step 4 Copy iscsi py patch to tmp and run root host3 cd usr share vdsm storage ee Ee Ms esp Se root host3 4 4 service vdsmd restart Step 5 Verify that the VDSM daemon is up Run o
25. e Status Compatibility Version Description 8 Storage Default iSCSI Maintenance 3 0 The default Data Center Templates Clusters tj Default j Hosts O xena001 lab mtl Z VMs Storage Logical Networks Clusters Permissions New Name Description rhevm Management Network Step 3 Fill in the details for the new Logical Network Figure 33 Adding a New Logical Network Expand All Collapse All 2 Data Centers Clusters Hosts Storage Virtual Machines Templates Et Edit E Guide Me 3 Name Storage Type Status Compatibility Version Description Storage X Default SCSI Maintenance 30 The default Data Center Templates zi Q Clusters zu ee New Logical Network j Hosts O xena001 lab mtl Edit Network Parameters Assign Networks to Cluster s Optional aa VMs To allow this option detach all clusters from network Name EoIB 4 Default Description E STP support E Enable VLAN tagging Storage New Name Description rhevm Management Network You should now be able to see the new logical network o oa Mellanox Technologies Rev 1 0 RHEV Manager Configuration Figure 34 Displaying the New Logical Network Expand All Collapse All 2 Data Centers Clusters Hosts Storage Virtual Machines Templates P System Edit amp Guide Me J Name Storage Type Status Compatibility Version Description e 8 Storage X Default SCSI Maintenance 3 0 The default Data Center T
26. emplates zi Clusters ty Default J Hosts xena001 lab mtl 12 VMs Storage Logical Networks Clusters Permissions New Name Description rhevm Management Network EoIB gt Step 4 Go to each host you want to connect to the new logical network and click Edit on the interface Step 5 Find which interface is eIPoIB Run SOMA cus Gale V eS edis Mel eia seno Eeer EE etno sio Pori EO Step 6 Give it an IP address and save the configuration Figure 35 Adding a Network Interface to the Logical Network Expand All Collapse All 2 Hosts Storage Virtual Machines Templates stem Edit Maintenance Assign tags Default Name Host IP x ap CPU Network SpmStatus Stora e NIE ERROR RIA Edit Network Interface Ec RU E men EE EE SE GE 9 de i xena001 lab mti com 172 30 40 101 1 ee OB OR None d e i Templates Name eth8 Q Clusters 1j Default 3 j Hosts Network EoIB None J DHCP f xena001 lab mtl com Mi VMs O Static IP 192 255 168 255 39 Subnet Mask Changes done to the Networking configuration are temporary until explicitly saved Check the check box below to make the changes persistent General Virtual Machines Network Interface Add Edit Save network configuration Name Address MAC Vian A o A ethi 172 30 40 101 00 30 48 C4 40 1A 1000 S 1 lt L 0 4 eth4 192 168 39 10 00 00 00 00 02 01 10000 1 1 0
27. ents Mi iio o Ufo emet 7 2 SUING RE e nta dci 8 2 1 Basic Test Bed Configuration cccccccccnncnncccconnconcononancnnnnnnnnncnnnnonnncnnnnnnnnnrnnnnnnnnnrnnnnnnnnennnnnnnns 8 2 2 Required Hardware ccccccsssseccccceeeeeceeceaeeeeeeesseeeeceeesauaueceeessaaseceeessaaseceeessuaeeeessssaeeeesssaaaes 8 2 3 Required Software Packages nnnn00nnnnnoeennnnosennnnnosrnrrrrsrnrrressrnrrrrsrnrrrrssrnrrrrsnnrerensnnrrernnne 9 3 Software Installation and Basic Configuration ccceceeeesseessenneeeeeeeeeeeensseeeeeeseeeeeeenneeeees 10 3 1 RAEV M Installation BA et EE EE 10 3 2 RHEV Host Installation Hoer 14 3 3 Mellanox OFED Driver Installation All HOSTS oococcccononcconcccconccnncnonanccnnonnnancnncnnnnnncnnnnos 17 34 Melanox VSA Installation Hosti j uisi vri i DUO ved EC Hd a eL a o ve eens 18 45 Mellanox UFM Installation OSU iii Ger ue e Rum b tx ab vet eta a ah d pe e uos 19 3 6 M Mellanox Network Manager Plugin snannnnnnnnnnennensnnnnnennnnsrnnnnrnrrnsrnrrnsnrrnsrnernnrrersrrerennrrenne 20 3 6 1 Installing Mellanox Network Manager Server doccccononcccnccononcconcnononcconononanccnnononancnns 20 3 6 2 Installing Mellanox Network Cent 20 4 RHEV Manager CONTIGUA ON 21 4 1 Add Data Center nennen nenne nnnnn nnn nns nna antri sina sis r siena sanis snas asas nnns 21 A E MEME eR EUER RT t TTE 21 Ae AIFOS ccc MI RE 22 AA eeh e 24 4 5 Adding Virtual Machines to a Cluster 26 4 6
28. ev 1 0 RHEV Manager Configuration Step 5 Go to Boot Options tab and choose Hard Disk as the First Device and PXE as the Second Device then click the OK button at the bottom Figure 25 Adding a New Virtual Machine Boot Options Expand All Collapse All 2 Virtual Machines 3 system New Server New Desktop 5 Bi Default Name New Server Virtual Machine FS E O Status 8 e No results found ES A 3 ho Console First Device Hard Disk Default Host Second Device Network PXE E oa High Availability Attach CD VMs Resource Allocation Linux Boot Options z kernel path Boot Options P initrd path Custom Properties kernel parameters OK Cancel Step 6 A wizard will pop up Choose Configure Network Interface Figure 26 Adding a New Virtual Machine Configuration 350 System New Server New Desktop E Guide Me 3 B Default Name Cluster Host IP Address Memory CPU Network Display Status g Storage gm vmi en Default 0 0 0 Down Templates Q Custers amp New Virtual Machine Guide Me tj Default Hosts Virtual Machine created 0 xena001 lab r zm Required actions Configure Network Interfaces Configure Virtual Disks Configure Later 28 Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design Step 7 Complete the details for the new HCA Figure 27 Adding a N
29. ew Virtual Machine Adding a New Network Interface Expand All Collapse All Virtual Machines 7 3 D system New Server New Desktop E Guide Me z 8 Default Name Cluster Host IP Address Memory CPU Network Display Status B Storage F vmi amp amp Default 0 0 0 Down Templates 1 Clusters E 3 amp Default S New Network Interface fj Hosts Name nicl O xena00L lab r Network rhevm Type Red Hat VirtIO E Specify custom MAC address Step 8 Click on Configure Virtual Disks and fill in the details Figure 28 Adding a New Virtual Machine Adding a New Virtual Disk Expand All Collapse All Virtual Machines a Q system New Server New Desktop E Guide Me B Default Name Cluster Host IP Address Memory CPU Network Display Status Uptime Lo 8 Storage D nni Default 0 0 0 Down i Templates z s E New Virtual Machine Guide Me Li Default S d Hosts 0 xena001 lab r There are still unconfigured e Configure Virtual Disks Size GB Storage Domain Optional actions Advanced Add another Network Inte Disk type Interface Format New Virtual Disk e storel System VitIO Thin Provision d Wipe after delete V Is bootable Cancel d 29 Mellanox Technologies Rev 1 0 RHEV Manager Configuration Step 9 Click Configure Later to finish Figure 29 Adding a New Virtual Machine Finishing Configuratiin secon
30. ishing Configuratiin Secon ssseeeeessssss 30 Figure 30 Adding a New Virtual Machine VMS Screen oooooocccccccccnnnnoncccnncccnnnnnannccnncnnnnnnnnnncnnncnnnnnnnnas 30 Figure 31 Adding a New Virtual Machine VNC Screen eee cee cccccee cece cece cece ee eeee reese eeeaaaaaeeeeeeeeeeaaaaaees 30 Figure 32 Logical Networks ccccceccccceeeeeceeeeeeeeeeseeeeaeeeeeeeeeeeeseaueeessegeeessaueeeeseaueeessaueeessneesssagseeesaaes 31 Figure 33 Adding a New Logical Network nennen nnne nnn nnne nnn nnn nnn nnn nnns 31 Figure 34 Displaying the New Logical Network n nnnnnannnnsnnnnsnnnennnnnnsrnrnnrnrrnsrnnenrnrrnsrrrrnnnrrrsrrerennrenne 32 Figure 35 Adding a Network Interface to the Logical Network snaannnnenonnnnnnnennnnnsennnnnosnnnnnessnnnnennenne 32 Figure 36 Displaying the Network Interface of the Logical Network 33 Figure 37 Virtual Machine Network Interfaces View oooccccccccnncccconcncoconcnnnconnnonononnnnononennnnonnnnnnnos 33 Figure 38 Adding a New Network Interface ccccccccssseeeceeeseeseeeeeseeeeeceeeeeeeeeeeeessaaeecesesseaeeeeeeeseeeeees 34 Figure 39 Displaying the New Network Interface essere 34 Figure 40 Verifying the New HCA is Up 35 Figure 41 UEM Environment asdsa a a 37 Beete Mellanox Technologies Contents Rev 1 01 FIQUIS 42 New Logical Sel Vel zi ica do ita eae 37 Figure AS Ada O lo 38 Figure 44 Add Si Bina i een Mi ie 38 Figure
31. n PCIOO 03 0 open v Link up TX 0 TXE O RX 0 RXE 01 aiting for link up on netO ok DHCP netO 00 1a a 1e 28 00 ok weto 172 30 11 48 255 255 0 0 gw 172 30 0 1 Booting from filename pxelinux O ftp 172 30 0 108 pxelinux 0 ok PXELINUX 4 04 2011 04 18 Copyright C 1994 2011 H Peter Anvin et al PXE entry point found we hope at 9A 77 0379 via plan A NDI code segment at 9677 len 0 f6 NDI data segment at 9AF2 len 2CD8 setting cached packet 01 02 03 y IP address seems to be AC1EOB30 172 30 11 48 IN ip 172 30 11 48 172 30 0 108 172 30 0 1 255 255 0 0 GE General Nel IH I F 01 00 1a 4a 1e 28 00 SYSUUID 882c5272 c147 4129 ab82 83f5f 4ec 7631 Name Origin RHEV Description i pxelinux cfg default Run On Any Host in Cluster Template Custom Properties Not Configured Operating System Red Hat Enterprise Linux 6 x x64 Highly Available No Default Display Type VNC USB Policy Disabled Priority Low Resides on Storage Domain store 30 Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design 4 6 Add a Network to the Cluster Step 1 Goto System gt Default Step 2 Click on Logical Networks and then on New Figure 32 Logical Networks Expand All Collapse All 2 Data Centers Clusters Hosts Storage Virtual Machines Templates D soi Edit E Guide Me Default Name Storage Typ
32. nterprise Linux Installation 4 RED HAT ENTERPRISE LINUX 6 Ceci O TOCADO MA A LS JBA OIRAN AR AA A A Installing foomatic db ppds 4 0 7 20091126 e16 noarch 23 MB PFDs from porter manufacturers Step 2 Make sure your VM has installed RHEL 6 2 successfully Step 3 Make sure that the NTP service is activated root host3 etc init d ntpd status nep sone roore host3s 7 ere nid mapa stare Stare mig sno IV OE oct thoses Te solia Gakeconriag meed on eec OSEE eue ardens sss cis EG mpl mc Eq p NS II NEU roota Nosto Step 4 Register to Red Hat Network to be able to subscribe to the required channels Poor ano e a os en Figure 8 Red Hat Network Register Copyright Am 2006 2016 lad Hat Inc ALL rights reserved J a a a a a a a a a a a a middot lab lt labe between elements z5l E le next screen Step 5 Subscribe to the required channels Run Lee moses ie cla cla suma aclel elena d esie led x965 54 S 61 Sic Username meldcs Password root host3 rhn channel add Seran elle So odi genes os Mellanox Technologies Rev 1 0 Software Installation and Basic Configuration Step 6 Step 7 Step 8 Step 9 Step 10 Step 11 Username meldcs Password oo those le Confirm the list of channels to which the server 1s subscribed Run root host3 rhn channel list No ol e ME O ME lO cde cc CU EO root host3 Add a manual host en
33. o install NET Framework Step 11 You can access the administration portal by pointing your internet browser to http host2 1 8080 assuming you followed the defaults Figure 5 RHEV M Portal Red Hat Enterprise Virtualization Welcome to Red Hat Enterprise Virtualization Manager Version 3 0 5 0001 5 el6 3 Portals Web Admin Portal Tech Preview Documentation Administration Guide PDF Evaluation Guide PDF Hypervisor Deployment Guide PDF Installation Guide PDF Power User Portal Guide PDF Quick Start Guide PDF REST API Guide PDF Technical Reference Guide PDF User Portal Guide PDF V2V Guide PDF Support Getting Support For advance configuration of the RHEV M refer to Red Hat Enterprise Virtualization 3 0 Installation Guide 3 2 RHEV Host Installation Host3 Follow these steps for RHEV installation Step 1 Install RHEL 6 2 You may use the default installation of RHEL Basic Server Figure 6 Red Hat Enterprise Linux Installation The default installation of Red Hat Enterprise Linux is a basic server install You can optionally select a different set of software now Basic Server O Database Server O Web Server O Identity Management Server O Virtualization Host O Desktop 14 Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design Figure 7 Red Hat E
34. om the smallest to the largest and most performance demanding clusters UFM provides the ability to monitor troubleshoot configure and optimize all fabric aspects available via only one interface UFM s central dashboard provides a one view fabric wide status view UFM includes an advanced granular monitoring engine that provides real time access to switch and host data enabling cluster wide monitoring of fabric health and performance real time identification of fabric related errors and failures quick problem resolution via granular threshold based alerts and a fabric utilization dashboard Fabric congestion is difficult to detect when using traditional management tools resulting in unnoticed congestion and fabric under utilization UFM s unique traffic map quickly identifies traffic trends traffic bottlenecks and congestion events spreading over the fabric which enables the administrator to identify and resolve problems promptly and accurately Using UFM one can set specific service levels for different applications to ensure that critical applications get the right priority according to the fabric QoS management is performed using a unique intelligent algorithm that determines the optimal configuration for each device location in the topology and its QoS capabilities UFM uses a logical fabric model to manage the fabric as a set of business related entities such as time critical applications or services The logical fabric model enable
35. on WSS el Seo SIP wes las 08010 Jas lero gt 5 8 4 seve a OW MIE I OZ B 0 5 00 JOa rpe aie row m somo cese ens Sici ol 401 401 rhel x860 odos e ve 0 Nevis pie 6 ke 002 010 Mellanox Technologies Rev 1 0 Software Installation and Basic Configuration Step 8 Step 9 Step 10 ilael ex96 54 amp 8e1wen G xmehnn o portae Za la OG Se rhel x86 NOt SI E MENOS 1231 3121 rhel x80 CJ Server supplementary S SIS OO OO Ael 945 548 ver utei eme tren le primes ds EAR rhel x860 6l scrver supplementary o 249 249 Packages Classpeathx Jaf avamlable b t not installed No Packages marked for removal Root hoar Sl Use yum to ensure that the most up to date versions of all installed packages are in use root host2 1 yum upgrade Use yum to initiate installation of the RHEV M package and all dependencies woouGhosteZ 1 4yum install rhevm P NOTE You must run this command as the root user gt Once package installation is complete the RHEV MR must be configured Use the rhevm setup script command oorGhose2 as eumcsei Welcome to RHEV Manager setup utility HTTP Port 8080 HTTPS POCTE 8443 Host fully qualified domain mame Note that this mame should be rully resolvable lhaost2 1 lab dnte ds o com Password for Administrator admin internal Warning Weak Password Confirm password Database password required for secure authentication with the locally crea
36. ription Allocation Method Assigning Computers Network Interfaces Environment env2 0 OS Type Linux Icon URL Mellanox Technologies Rev 1 0 Using UFM to Automate Network Mangement Step 3 Add all hosts in the RHEV M cluster Figure 43 Add Hosts 44 xenaD04 lab mtl com New Logical Server Group L51 Allocate free computes and click Next Name and Description Pul n 4 Allocation Method P So Gees Assigning Computers Name ID IP hostname HOA 1 aoozesosooosfeso 0 0 0 0 localhost HC A 1 coo02cso3002eeseo 0 0 0 0 localhost HC A 1 ooo02eso3occ0s0c34 0 0 0 0 localhost HC A 1 0002c903004b51a8 0 0 0 0 xena006 HCA 1 0002cSO03003sdodo 0 0 0 0 xenaDO09 HC A 1 0002c903002e65d0 0 0 0 0 xena015 HCA 1 0002290300455940 0 0 0 0 xenaD20 HCA 1 co0o02csO3002ee400 0 0 0 0 Network Interfaces oOo oo aoa an OH STETCESTERTERTEE Step 4 Create a new network Add partition key PKey Figure 44 Add Hosts 44 xena004 lab mtl com New Network General Qos Filter IP Services CS Enviroment envz Mame m Description PKey 00006 Q Full Partial Event Script IP Configuration IP Subnet Network Mask Default Gateway 38 Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design Step 5 Connect the logical server cluster to the network By doing this all hosts located un
37. rnel version by using the A III mlnx add kernel support sh script located under the docs directory For further information please refer to MINX_OFED User Manual Section Pre installation Notes mInx add kernel support sh tool Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design the efforts of both Ethernet and InfiniBand communities and reduce the number of Storage protocols a user has to learn and maintain Figure 9 Using ConnectX 3 Adapter Results in Faster I O Traffic Delivery rather than using Multiple 10GbE Ports from Competitors iSER 4 VMs iSER 8 VMs iSER 16 VMs SCSI 8 VMs iSCSI 16 VMs 7000 e PCle 6000 Limit Y 5000 SF all 4000 10X Faster I O 3 3000 Z S 2000 t 1 2 4 8 16 32 64 128 256 UO Size 9KB Mellanox s Storage Accelerator VSA software is a highly scalable high performance low latency software solution for tier one storage and gateways that provides ultra fast remote block storage access and accelerates access to SAN DAS or Flash based storage Once the VSA is installed on your server run vscli and perform the following VSA commands to enter VSA configuration mode Mos vseli VA a OI e OMEN VSA show disks State Idx Name Size Cache Vendor Model Serial Rev paths CIO LAC 1 3600605b0032a49601601 f 69931f3bb42 667GB 0 LSI MR9265 8i 0042bbf33199f6011660492a03b
38. rnnnnnssrnrnrnsrnrrressnnrrrenenne 14 Figure 7 Red Hat Enterprise Linux Installation 4 15 Figure g Hed Hat Network e TEEN 15 Figure 9 Using ConnectX 3 Adapter Results in Faster UO Traffic Delivery rather than using Multiple TOGDE egen RE eu erte 19 Figure 10 New Data Generar cece aa ed acs ee inca hea ee a ce see ade hee 21 Figure EE team dde 21 wies up Data COn OS 22 Figure TS SAGGING s TOS EE 22 Figure 14 Installation in Progress muris da ANA dd 23 Figure 15 Ier ele RE eil e o oe 23 PIQUE 1O ROSS UP er RTT 24 Figure 17 Disc vering Tarea 24 FOUS TOS e rela to Par lisos EE 25 Figure 19 GAcOs Ma an LUN on T T 25 Figure 20 Aer Adding a storage Re UN sico ol ee eee 26 Figure 21 Successfully Adding a Storage Domain 26 Figure 22 Adding New Virtual Machine General 26 Figure 23 Adding a New Virtual Machine Console cccccsececeeeeeeeeeeeeeeeeeeeeeeesaeeeeeeeaeaeeeesessaeeeees 27 Figure 24 Adding a New Virtual Machine Hoer 27 Figure 25 Adding a New Virtual Machine Boot Options ooooooonccnnccccccnnnnccccnnnnnnnnnnnnnnnnnnnnnnnnnnnccnnnnnnos 28 Figure 26 Adding a New Virtual Machine Configuration ooocccnncccconcconccononncnnononannnonononannnnnnnnnnanoss 28 Figure 27 Adding a New Virtual Machine Adding a New Network Interface 29 Figure 28 Adding a New Virtual Machine Adding a New Virtual Disk 29 Figure 29 Adding a New Virtual Machine Fin
39. s fabric monitoring and performance optimization on the application level rather than just at the individual port or device level Managing the fabric using the logical fabric model provides improved visibility into fabric performance and potential bottlenecks improved performance due to application centric optimizations quicker troubleshooting and higher fabric utilization Refer to UFM User Manual for detailed installation and configuration options 36 Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design 9 1 Basic UFM Configuration Flow Follow the next steps for basic UFM configuration The following steps show how to create a logical server and UFM Network and finally connecting between them Step 1 Create an environment Figure 41 UFM Environment JES 172 30 40 104 xena004Jab mtl co AMIN Mellanox Unified Fabric Manager All Environments X a a F Network View M Vetal e lA AE m D w Environment E Glosa Metron Ki a 4 envi Di Events E da E Frer Event Name v Ge Source Date Time Category Description Event Name Severity gt Step 2 Add a logical server UFM logical server is equivalent to datacenter cluster in the RHEV M architecture model Figure 42 New Logical Server 44 xena004 lab mtl com New Logical Server Group LS1 Name and Desc
40. software applications and HW components The following chapter supplies basic software installations procedures for the cloud 3 1 RHEV M Installation Host2 1 To perform initial installation and configuration of the Red Hat Enterprise Virtualization Manager RHEV M follow the steps below on host2 1 Step 1 Install Red Hat Enterprise Linux RHEL 6 2 You may use the default installation of RHEL Basic Server Figure 2 Red Hat Enterprise Linux Installation The default installation of Red Hat Enterprise Linux is a basic server install You can optionally select a different set of software now Basic Server Database Server Web Server IO OD 9 Identity Management Server Virtualization Host O C Desktop Figure 3 Red Hat Enterprise Linux Installation RED HAT ENTERPRISE LINUX 6 CUNA O TOCADO MA AL JBA COREL AR AA Pee dh Eed 429 of 604 Installing foomatic db ppds 4 0 7 20091126 e16 noarch 23 MB PPOs from printer manufacturers Step 2 Make sure your VM has installed RHEL 6 2 successfully Step 3 Make sure that the NTP service is activated BOOBENOSE2 Nl tay Cle able ol d SEIEN EDS ntpd 1S stopped geet SEET Sie cm D erm de MISCO Lio tios e Z2 It seam dol ite ER EE Root host Sto E Ee epale tous maso comal Enc IE EES root El Eeer Z I Step 4 Register to Red Hat Network to be able to subscribe to the required channels acorde Mose 21 ela ces stes om Il
41. tc init d openibd restart Step 9 Run the following command after OFED installation to see all the eIPoIB interfaces Ss Ss Oli Mc ty Silao abi exo ble nghe ende eue s Sth EE lo 0 Step 10 To find the right interface run ibdev2netdev DoD DOE gt ee UM Um od cubos Io ENDO Here we can see that our interface eth5 1s associated with the first port on the first HCA Step 11 To further ensure that this interface is a PV IPoIB interface you may run ethtool i eth5 Giver erie Om yversions 1 0 0 firmware version 1 For additional options and advance installation refer to Mellanox OFED User Manual Mellanox VSA Installation Host4 iSER ConnectX s RDMA capabilities can be used to accelerate hypervisor traffic such as storage access VM migration data and VM replication The use of RDMA moves the data from node to node to the ConnectX hardware producing much faster performance lower latency access time and lower CPU overhead which provides zero copy message transfers for SCSI packets Thus the RAID for a cluster may now be connected via InfiniBand and result in saving additional storage connector such as Fibre Channel thereby greatly reducing the cost of the cluster when using RDMA based iSCSI GSER compared to traditional TCP IP based 1SCSI RDMA can provide 10X faster performance This will also consolidate gt Tf your kernel version does not match with any of the offered pre built RPMs you can add your ke
42. ted database Warning Weak Password Contirm password Organization Name for the Certificate Mellano The default storage CyPpPe voU wili sio e En EE Ee Should the installer configure NFS share on this server to be used as ans OD emana yes no Firewall ports need to be opened You can let the installer configure iptables automatically overriding the current c omnia mec old cmi aur eo no So a c e Alternately you can configure the firewall later using an example iptables file found under usr share rhevm conf iptables example Configure Iip ables yes ne yes RAEV Manager will be installed using Lhe following cone cm stato me Krep rore 8080 Herp o POR 8443 eg III 251 Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design Rost igdn Msi iL Wels ie O auth pass KKAKKKKKK db pass KKK ck ck kock org name Mellanox default dc type Is Sal override iptables yes Proceed with the contigurarion listed above yes mo yes asis bae Creating JBOSS Profile DONE Cren ange A DONE setting Database Security DONE Creating Database DONE Updating the Default Data Center Storage Type DONE Editing EE DONE Editing RAEV Manager Configuracion DONE Configuring Farewell ae tabiles DONE SE EES JBOSS Services e DONE pius e e IE cud die Cen Som suecas EE Please allow RHE
43. try to the etc hosts file on the Red Hat Enterprise Linux host for the RHEV M server to enable vdsm and other services to connect properly to the host 10 0 0 1 serverl example com rhev manager serverl example com For example queste OL kA hos cz ee aba de com They mama ger hos eet Open firewall ports The following commands will remove and existing firewall rules and add the ports required by RHEV M to the iptables rules that open the required ports for the agent to function properly Meco aos ee ooo so a pira les Sa pena mrstabe EE Bo AB cals AED Aa EE solitos Re ice elles eyereSiatel MONETE Sos htt der ACCME ROOE CHOSE sq Upbackes pena INPUT m iho eC T octo hoet a otto joio WN PUN oo ee ox OOO 22 qu NCCE Escorts ED bles o MOUTE Oe ero oa SIS dl ros EE pid les apena INPUIM p ben dor E IAS apar cds tp ral e SO eno OO EE OSM O y aC Bp ee divo ee de pta Dle oe NA A EE eis ES SS esq NOC Jane d Moor tao ssl abredbcdode s ecd ceder JEUNE EE xeomp bost peolirbated root host3 iptables append FORWARD m physdev physdewveurs bPuxdged J REJEC reece wita rompehostcprohmsosted root host3 etc init d iptables save asoma osas else dme idg dE Ee root host3 service iptables restart The RHEV M makes use of sudo to perform operations as root on the host The default configuration stored in etc sudoers contains values to allow this To configure sudo access Add etc sudoers the
44. v Size path VendorID ProductiD Serial 1d0ecc9431660672803 890GB 1 LSI MR9265 8i SLSIMRS2 OK Cancel Lb 25 Mellanox Technologies Rev 1 0 RHEV Manager Configuration Figure 20 After Adding a Storage Domain Si Eege Ad Collier Ad 2 gt E a a e ime Aa er Aa FA L LI f z a Uta Cer Rss Dora Tau fena Type P oes Cdi DI dl Bu Fh Beeb Heel Cate Her ita Wa Insite Ass A 5 T Lag F Lug a E E Lj Chiltern i d Reg Figure 21 Successfully Adding a Storage Domain Expand fl Colas Ad Sorge B h D I i a Sale Kr mgar Larra E B een Zeie HT Proin Pepa forage Tape Ferunt rar Dio ei dona Free dps Fe a goen Dum Maier Cat vi fw Han ou Template a ij Cluster 4 5 Adding Virtual Machines to a Cluster Step 1 Click on System gt Default gt Clusters gt Default gt VMs gt New Server Step 2 Fill the details in General tab Figure 22 Adding New Virtual Machine General Expand All Collapse Ai 2 Virtual Machines B 3 system New Server New Desktop 5 009 B Defaut Name New Server Virtual Machine LD E O Status SE de General O O Def Templates Data Center Default y 2 E 3 Clusters Console Host Cluster E 3 tf Default Host 3 j Hosts SE ilabili Name i O xena001 lab r O vanaoey uM d EI VMs Resource Allocation On BEE Based on Template Blank Custom Properties Memory Size 2 GB Total Cores 2 CPU Sockets 2 Operating System Red
45. w Desktop Edit Remove p Make Template Export Move E Guide Me Assign tags 9 ke i gt B Default Name Cluster Host IP Address Memory CPU Network Display Status Upti D storage vmi en Default 0 0 0 Down E Templates Clusters E E 1j Default j Hosts I O xena001 lab r y General Network Interfaces Virtual Disks Snapshots Applications Permissions New Name Network Name Type MAC Speed mbps RX mes TX mbps Drops Pkts nicl rhevm Red Hat Vi 00 1a 4a 1e 28 00 1000 lt 1 1 0 nic2 EoIB Red Hat Vi 00 1a 4a 1e 28 01 1000 lt 1 lt 1 0 Step 5 Start the VM Step 6 Verify that the host has a new network interface for the VM Run the command IICONnEPAG d La Mellanox Technologies Building Highly Efficient Red Hat Enterprise Virtualization 3 0 Cloud Infrastructure with Mellanox Interconnect Reference Design Figure 40 Verifying the New HCA is Up Search Vms cluster Default DUONIFPOVRT reg ray gt Expand All Collapse All e System 3 Default 8 Storage D store1 Templates z D Clusters Default 1 Hosts xena001 lab r Virtual New Sen Genen e VITIL COMSUIE eA A RX packets 11558 errors B dropped B overruns B frame B TX packets 1939 errors B dropped B overruns B carrier B collisions B txqueuelen 1888 RX bytes 4961519 4 7 MiB TX bytes 154762 151 1 KiB Link encap Ethernet HWaddr 88 16 40 1E 28 81 BROADCAST MULTICAST MTU 1588 Metric 1 RX p
Download Pdf Manuals
Related Search