Home

esxupdate

1. DESCRIPTION The SGL pages could not be unregistered from the firmware DATA None ACTION None required elx_mes2007 Only Limited Edition cmd Format supported lt iocb ulpCommand gt DESCRIPTION SLI 4 only supports the Limited Edition command format DATA None Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 55 gt E EMULEX ACTION None required elx_mes2008 Error lt rc gt posting all rpi headers DESCRIPTION The RPI headers could not be posted to the firmware DATA None ACTION None required elx_mes2009 Failed to allocate mbox for ADD_ FCF cmd DESCRIPTION Failed to allocate mailbox for ADD_FCF command DATA None ACTION None required elx_mes2010 Resume RPI Mailbox failed status lt status gt mbxStatus lt mbx status gt DESCRIPTION DATA None ACTION None required elx_mes2011 Unable to allocate memory for issuing SLI_CONFIG_SPECIAL mailbox command PTION Unable to allocate memory for issuing SLI_CONFIG_SPECIAL mailbox command DATA None ACTION None required elx_mes2012 Mailbox failed moxCmd lt mbx_cmd gt READ CONFIG mbxStatus lt mbx status gt DESCRIPTION The READ_CONFIG mailbox command failed DATA None ACTION None required elx_mes2013 Could not manually add FCF record 0 status lt rc gt DESCRIPTION Could not add FCF record to the FCF list DATA None ACTION None required elx_mes2014 Invalid command lt iocb ulpCommand gt DESCRIPTION The IOCB
2. DESCRIPTION The mailbox command sent to delete the SGL pages from the firmware has failed Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 58 gt E EMULEX DATA None ACTION None required elx_mes2513 POST_SGL_BLOCK mailbox command failed status lt shdr_status gt add_status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The mailbox command sent to post the SGL pages to the firmware has failed DATA None ACTION None required elx_mes2514 POST_RPI_HDR mailbox failed with status lt shdr_status gt add_ status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The mailbox command sent to post the RPUI header pages to the firmware has failed DATA None ACTION None required elx_mes2515 ADD_FCF_RECORD mailbox failed with status lt rc gt DESCRIPTION The mailbox command to add the FCF record has failed DATA None ACTION None required elx_mes2516 DEL FCF of default FCF Index failed mbx status lt rc gt status lt shdr_status gt d_status lt shdr_add_status gt DESCRIPTION The mailbox command to delete the FCF record has failed DATA None ACTION None required elx_mes2517 Unregister FCFI command failed status d mbxStatus x x rc bf_get lpfc_maqe_status amp mbox gt u mqge DESCRIPTION The driver was unable to unregister the FCFI from the firmware DATA None ACTION None required elx_mes2519 Unable to allocate memory for issuing NOP mailbox command DESCR
3. J gt G EMULEX ACTION None required elx_mes0387 Failed to allocate an iocbq DESCRIPTION Failed to get an IOCBQ from the list of available IOCBQs DATA None ACTION None required elx_mes0388 Not a valid WCQE code lt hex cqe_code gt DESCRIPTION The event code is invalid This event is dropped DATA None ACTION Ensure the adapter s firmware is current elx_mes0391 Error during rpi post operation DESCRIPTION The driver was trying to post pages to the firmware to be used to keep target login information and encountered a failure DATA None ACTION Unload and reload the driver elx_mes0393 Error lt rc gt during rpi post operation DESCRIPTION The driver was trying to post pages to the firmware to keep target login information and encountered a failure DATA None ACTION Unload and reload the driver elx_mes0394 Failed to allocate CQ_EVENT entry DESCRIPTION The asynchronous event handler was not able to allocate an event queue entry to which to transfer the asynchronous event DATA None ACTION This could be a V LINK clear from the switch or a fatal error from the firmware Perform a dump from the OneCommand Manager application elx_mes0395 The mboxq allocation failed DESCRIPTION The asynchronous link event handler could not allocate a mailbox command to issue the READ_LA read link attention mailbox command DATA None ACTION None required elx_mes0396 The lpfc_dmabuf allocation f
4. e Namespace Namespace is the namespace of the emulex provider For ESX ESXi 40 and 41 the namespace is root emulex For ESX 3 5 the namespace is elxhbacmpi cimv2 Note If the Emulex CIM Provider present in ESXi ESX is inbox provider then the namespace to be used is elxhbacmpi cimv2 If the out of box CIM Provider is installed then the namespace to be used is root emulex Table 2 Namespaces Used for Providers lists the namespaces to be used with the inbox providers and the out of the box providers for various versions of ESX ESXi Table 2 Namespaces Used for Providers Namespace Inbox Provider Out of Box Provider ESX ESXI3 5 elxhbacmpicimv2__ elxhbacmpicimv2 ESX ESXi4 0 elxhbacmpi cimv2 root emulex ESX ESXi4 1 elxhbacmpi cimv2 root emulex To check whether the CIM Provider is inbox or out of box enter the following command on the ESX ESxXi host esxupdate vib view query grep emulex cim provider Drivers for VMware ESX ESXi 4 1 CIM Section Page 18 gt S EMULEX If the provider name is prefixed with deb it is an inbox provider If the provider name is prefixed with cross it is an out of box provider as shown in the following response samples deb _emulex cim provider 410 2 0 32 1 207424 installed 2010 04 01T07 00 00 00 00 cross emulex cim provider 410 3 1 16 1235786 installed 2010 10 11T09 39 04 047082 00 00 6 Click OK to apply your changes Click Def
5. DATA None ACTION Software driver error If this problem persists report these errors to Technical Support elx_mes1813 Create VPORT failed Cannot get sparam DESCRIPTION The port could not be created because it could not be initialized possibly due to unavailable resources DATA None ACTION Software driver error If this problem persists report these errors to Technical Support Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 52 J gt G EMULEX elx_mes1814 Mbox lt u mb mbxCommanas gt failed no vport DESCRIPTION The VPort field of this mailbox command was not completed DATA None ACTION None required elx_mes1815 Could not issue unreg_did default rpis DESCRIPTION Attempt to unregister RPI failed DATA None ACTION Software driver error If this problem persists report these errors to Technical Support elx_mes1818 VPort failed init mobxCmd lt mailbox command gt READ _SPARM mbxStatus lt mailbox status gt rc lt status gt DESCRIPTION A pending mailbox command issued to initialize port failed DATA 1 mbxCommand 2 mbxStatus 3 rc ACTION Software driver error If this problem persists report these errors to Technical Support elx_mes1820 Unable to select SLI 3 Not supported by adapter DESCRIPTION The adapter is not capable of operating in a given mode DATA None ACTION This is an informational message SLI 3 mode is only available on some adapters Do not attem
6. snum lt serial number gt DESCRIPTION The abort handler timed out waiting for abort to complete DATA None ACTION None required Node Table Events 0900 0999 elx_mes0915 Register VPI failed lt mbxStatus gt DESCRIPTION Could not register the VPI DATA None ACTION None required Miscellaneous and FCoE Events 1200 1299 elx_mes1262 Failed to allocate dfc_host DESCRIPTION Could not allocate memory the dfc_host_struct DATA None ACTION None required Link Events 1300 1399 elx_mes1303 Link Up Event lt eventTag gt received Data x x x x x x X X X X X X d DESCRIPTION A link up event was received It is also possible for multiple link events to be received together DATA 1 fe_eventTag 2 granted_AL_PA 3 UlnkSpeed 4 alpa_map 0 Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 46 gt G EMULEX ACTION If numerous link events are occurring check the physical connections to the Fibre Channel network elx_mes1305 Link Down Event lt eventTag gt received Data x x x x X X DESCRIPTION A link down event was received DATA 1 fe_eventTag 2 hba_state 3 fc_flag ACTION If numerous link events are occurring check the physical connections to the Fibre Channel network elx_mes1306 Link Up Event in loop back mode lt eventTag gt received Data x x x x X X X X DESCRIPTION Link up notification configured for loopback DATA 1 fe_eventTag 2 granted_AL_PA 3 Uln
7. 0121 0133 0134 0135 should precede this message elx_mes0262 No NPIV Fabric support DESCRIPTION The switch to which the port is connected does not support NPIV DATA None ACTION Check the switch configuration elx_mes0263 Discovery Mailbox error state lt port_state gt lt sparam_mbox gt lt cfglink_mbox gt DESCRIPTION Either the driver could not allocate resources or it could not send sparam_mbox or cfglink_mbox DATA 1 address of sparam_mbox command 2 address of cfglink_mbox command ACTION Attempt to unload and reload the driver when it is convenient Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 27 J gt G EMULEX elx_mes0264 No NPIV Fabric support DESCRIPTION The switch to which the port is connected does not support NPIV DATA None ACTION Check the switch configuration elx_mes0266 Issue NameServer Reg lt cmdcode gt err lt rc gt Data lt fc_flag gt lt fc_rscn_id_cnt gt DESCRIPTION The driver was not able to send the nameserver CT command DATA 1 VPorts fc_flag 2 VPorts fe_rscn_id_cnt ACTION Check the switch and port configurations elx_mes0267 NameServer GFF Rsp lt did gt Error lt ulpStatus gt lt un ulpWord 4 gt Data lt fc_flag gt lt fc_rscn_id_cnt gt DESCRIPTION The nameServer GFF CT request failed DATA 1 VPorts fc_flag 2 VPorts fc_rscn_id_cnt ACTION Check the switch and port configurations elx_mes0268 NS cmd lt cmdcode gt Err
8. 32 gt E EMULEX ACTION If the mailbox command fails again set the Ipfc_log_verbose to LOG_MBOX and retry elx_mes0346 Ring lt ring number gt handler unexpected ASYNC_STATUS evt_code lt evicode gt WO lt hex w0 gt W1 lt hex w1 gt W2 lt hex W2 gt W3 lt hex W3 gt W4 lt hex W4 gt W5 lt hex Z5 gt W6 lt hex W6 gt W7 lt hex W7 gt W8 lt hex W8 gt W9 lt hex W9 gt W10 lt hex W10 gt W11 lt hex W11 gt DESCRIPTION The adapter received an asynchronous event that was not a temperature event DATA None ACTION None required elx_mes0347 Adapter is very hot please take corrective action DESCRIPTION Adapter temperature is above normal range DATA Temperature in Celsius ACTION Shutdown and remove the adapter Contact customer support elx_mes0348 NameServer login node freed DESCRIPTION The enable mode failed to free up the nameserver login DATA None ACTION None required elx_mes0349 rc should be MBX_SUCCESS DESCRIPTION The next mailbox command on the mailbox queue has failed DATA None ACTION None required elx_mes0350 rc should have been MBX_BUSY DESCRIPTION Attempting to unregister a default RPI from an interrupt context and the mailbox state is not busy DATA None ACTION None required elx_mes0352 Config MSI mailbox command failed moxCmd lt u mb mbxCommana gt mbxStatus lt u mb mbxStatus gt DESCRIPTION The mailbox command sent to the firmware to configure the adapter to use MS
9. CQ_CREATE mailbox failed with status lt shdr_status gt add_status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The mailbox command sent to create the completion queue has failed DATA None ACTION None required elx_mes2502 MQ_CREATE mailbox failed with status lt shdr_status gt add_ status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The mailbox command sent to create the mailbox queue has failed DATA None ACTION None required elx_mes2503 WQ_CREATE mailbox failed with status lt shdr_status gt add_status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The mailbox command sent to create the work queue has failed DATA None ACTION None required elx_mes2504 RQ_CREATE mailbox failed with status lt shdr_status gt add_status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The mailbox command sent to create the receive queue has failed DATA None Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 57 gt E EMULEX ACTION None required elx_mes2505 EQ_DESTROY mailbox failed with status lt shdr_status gt add_status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The mailbox command sent to delete the event queue has failed DATA None ACTION None required Preliminary Second Review elx_mes2506 CQ_DESTROY mailbox failed with status lt shdr_status gt add_status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The
10. G EMULEX New Events elx_mes2796 Mailbox memory allocation failed DESCRIPTION The driver failed to get memory resources to release an RPI DATA None ACTION None The driver s heap is exhausted A server reboot is required to fix the exhaustion Contact technical support if the problem persists elx_mes2798 Unreg_vpi failed vpi 0x x mb status 0x x DESCRIPTION The driver attempted to unregister a virtual port index and failed The failure status is printed DATA None ACTION This condition is not catastrophic but is unexpected If problems persist contact technical support elx_mes2813 Mgmt IO is Blocked x mbox cmd x still active DESCRIPTION The HBA s management interface is marked as blocked in preparation for an online or offline state transition All user space access to the HBA via libdfc interface will be blocked DATA None ACTION None Notification of a run state change only elx_mes2822 IOCB failed s iotag 0x x xri 0x x DESCRIPTION The driver is attempting to drain an internal queue and failed The failure reason and some state variables are written to the console DATA None ACTION None required This should be a transient condition If not please contact technical support elx_mes2823 txq empty and txq_cnt is d DESCRIPTION The driver has detected a discrepancy between the elements queued to the txq and the counter tracking the number or items DATA None ACTION None required Ther
11. GIM Providot s n cat vies ccttescgeas E 11 Working with VPorts Virtual Ports 0 000 12 Creating Deleting and Displaying VPortS essesssssererssssssrrrrrrrrrrsssssrrnssssss 12 NIC Network Driver Performance TUNING cccccceceeceeeeeeeeeeeeeeeeccneeeeeeeeeseneeeeees 12 Using vmxnet Emulation Enabling Jumbo Frames and TSO s a 12 Configuring a Virtual Switch to Use Jumbo Frames ccccccccceeeeessssttteeeeeees 13 Setting the MTU Size in the VSWItCH ccceeeceeeceeeeee eee eeteeeeeeeetaeeeeeeteee 13 Setting the MTU Size for a Linux Guest Operating System 0 13 Setting the MTU Size for a Windows Guest Operating System 13 Drivers for VMware ESX ESXi 4 1 User Manual Page iv Pinning VMs and Interrupts to CPUS cccccceeeeeeeeeeeeeeeeeeeeeeeeeeetaeeeeeeteeeeees 14 Managing Devices USING CIM cccceeseeeeeeeeeeeeeeeeeeeeeneeeseeeneeeeeseaeeees 15 Addnga Range 0f OSIS sissccctccstezccets betetassbetsteastesatataeanteanatatehehetenntstaietiisteiatsiaiaiess 15 Configuring Discovery CIM Credentials and TCP IP Settings eceeeeeeeees 17 How to Set the Default Credentials Using hbacmd ccccceeeseeeeeeeeeeeeees 19 Commands Using the Default CIM Credentials ccccccccceeeeeeeeeeeeeees 19 Commands Using the User Provided CIM Credentials sssssseeeee 19 CIM Provider Versions cccceeeeeeeeeeeeeeeeeee teeter e
12. HTTPS is not supported by the CIMOM Most often the CIMOM will be configured to use HTTPS So if you are trying to connect with HTTP you might get an error Try using HTTPS instead e The namespace specified is wrong Make sure the namespace conforms to Table 2 Namespaces Used for Providers on page 18 The username or the password specified is wrong Verify that the username given is correct and try retyping the password e The CIMOM is not running on the ESX ESXi host Try restarting the CIMOM You can check whether the CIMOM sfcb is running by typing the below commands etc init d sfcbd watchdog status or ps ef grep sfcb Also if the CIMOM is listening to a different port than 5988 or 5989 the connection might not take place You can configure the sfcb CIMOM settings by editing etc sfcb sfcb cfg If you still experience problems when adding the host execute the following commands on the ESXi host and send the output to the Emulex Technical Support team vm support dump esxcfg module 1 esxcfg scsidevs a esxupdate vib view query grep be2 esxupdate vib view query grep lpfc esxupdate vib view query grep emu esxcfg nics 1 lspci Send the var log messages file for all of the above operations Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 74 gt G EMULEX APPENDIX A DHCP Recommendations If you use a DHCP Dynamic Host Configuration Protocol server to obtain an IP add
13. None required elx_mes0285 Allocated DMA memory size lt alloclen gt is less than the requested DMA memo rysize lt reqlen gt DESCRIPTION Memory allocation was truncated DATA None ACTION None required elx_mes0286 Ipfc_nlp_state_cleanup failed to allocate statistical data buffer lt nlp_ DID gt DESCRIPTION Memory allocation failed for node s statistical data DATA None ACTION None required elx_mes0287 lpfc_alloc_bucket failed to allocate statistical data buffer lt nlp DID gt DESCRIPTION Memory allocation failed for node s statistical data DATA None ACTION None required elx_mes0288 Unknown FCoE event type lt event_type gt event tag lt event_tag gt DESCRIPTION The firmware has detected an unknown FCoE event DATA None ACTION Check the FCoE switch configuration and the adapter DCBX mode elx_mes0289 Issue Register VFI failed Err lt rc gt DESCRIPTION The driver could not register the Virtual Fabric Index for the FCFI DATA None ACTION Check the switch and port configurations elx_mes0290 The SLI4 DCBX asynchronous event is not handled yet DESCRIPTION The SLI 4 DCBX asynchronous event is not handled yet DATA None ACTION None required elx_mes0291 Allocated DMA memory size lt alloc_len gt is less than the requested DMA memo rysize lt req_len gt DESCRIPTION The asynchronous DCBX events are not handled in the driver DATA None ACTION Check the switch configuration elx
14. Remove afer 1600 seconds O immediately CIM Credentials Protocol htp C https Port Number fe Username fot Password Pe Namespace felchbampicew2 Defauits Refresh Rate Select when to start a new discovery cycle after a previous cycle completes Fibre Channel I Manual refresh requires pressing discovery refresh button Refresh every fis seconds TCP IP I Manual refresh requires pressing discovery refresh button Refresh every fs seconds TCP IP Management Port Warming The port number must be the same on all hosts being managed with out of band TCPAP connectivity V Enable TCPAP Management Port Number 23333 OK Carcel Figure 2 Discovery Settings dialog box Drivers for VMware ESX ESXi 4 1 CIM Section Page 17 gt G EMULEX To configure discovery settings 1 From the Discovery menu select Modify Settings The Discovery Settings dialog box appears 2 Define the discovery properties you want 3 If TCP IP Management is enabled the Enable TCP IP Management checkbox is selected and the current port number is displayed in the Port Number field If desired enter a different port number between 1024 and 65535 Click Defaults to select the Enable TCP IP Management checkbox if unchecked and set the port number to 23333 If the port number or the Enable TCP IP Management checkbox is changed a set of warning messages may appear before changes are made Click Yes on the
15. Reset d d reset status x x flush status x x DESCRIPTION A device reset has completed on tgt lun Status values are displayed DATA 1 tgt 2 lun 3 task mgmt status 4 flush status ACTION None required elx_mes0714 SCSI layer issued bus reset DESCRIPTION The SCSI layer is requesting the driver to abort all I Os to all targets on this adapter DATA 1 ret ACTION Check the state of the targets in question Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 45 gt G EMULEX elx_mes0718 Unable to dma_map single request_buffer x x DESCRIPTION The driver could not map a single virtual address to a dma address DATA 1 dma mapping error ACTION None The driver fails the IO back to ESX elx_mes0721 Device Reset rport failure rdata lt rdata gt DESCRIPTION The reset of the Rport failed DATA None ACTION None required elx_mes0724 I O flush failure for context lt cntx gt on lt tgt lun gt cnt x x DESCRIPTION The I O flush to the LUN TARGET or HOST has failed DATA 1 cnt of unrecovered IO ACTION None required The reset is retried elx_mes0727 TMF lt cmd gt to TGT lt TGT gt LUN lt LUN gt failed lt ulpStatus gt lt ulpWord 4 gt DESCRIPTION The task management command failed DATA None ACTION None required The TMF command gets retried elx_mes0748 Abort handler timed out waiting for abort to complete ret lt status gt ID lt target id gt LUN lt lun id gt
16. assumes no responsibility for its use nor for any infringements of patents or other rights of third parties that may result Periodic changes are made to information contained herein although these changes will be incorporated into new editions of this manual Emulex disclaims any undertaking to give notice of such changes US patent notice is given for one or more of the following 6226680 6247060 6334153 6389479 6393487 6427171 6427173 6434620 6591302 6658480 6697868 6751665 6757746 6941386 6965941 6687758 7042898 7133940 7124205 7089326 6938092 6996070 Emulex 3333 Susan Street Costa Mesa CA 92626 Drivers for VMware ESX ESXi 4 1 User Manual Page ii Supported Driver Versions Driver Version Comments FC FCoE Drivers 8 2 1 86 26 8 2 1 90 28 Ethernet Drivers 2 102 518 0 2 103 377 0 iSCSI Drivers 2 102 525 9 Drivers for VMware ESX ESXi 4 1 User Manual gt G EMULEX Page iii Installation ReePeene tee Dee nnn esr meron Ban ee ESSER Epes SRE Ate SoSnh r aen Wilts Bernese Smt Srneroae nee 1 Driver IMO MANON tscsctescteteccnatanatenatatatetanatatatetatatatatetatatatatadatatatadetanatatetanatanetatatatanann 1 Supported Features ccccecccccceceeeecceneeeeeeeeeeeecccaeeeeeeeeeeeeccaeeeeeeeeteeenceeeeeeeees 1 New Features in this RElCaSe cc cccccccccccceeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeteeeeeas 1 Prerequisites
17. bundle to be installed An offline bundle can be installed on a VMware ESX server locally using the esxupdate command The following example uses the offline bundle file name offline bundle zip when using this example as a reference make sure to use the file name of the offline bundle to be installed note the maintenance parameter is required for FCoE NIC and iSCSI driver installations root testmachine esxupdate nosig maintenance update bundle offline bundle zip Reboot the VMware ESX server to activate the new drivers Virtualization Features For the best performance you must install VMware Tools in each guest operating system For information on installing VMware Tools in a Linux or Windows guest operating system refer to the VMware ESX Server 4 1 documentation Enabling the NetQueue Feature To use the multiple interface feature in ESX Server you must enable the NetQueue feature in ESX Server The NetQueue feature is enabled by default e To check if NetQueue is enabled run the following command from the ESX Server console esxcfg advcfg j netNetqueueEnabled If it prints netNetqueueEnabled FALSE the feature is disabled e To enable NetQueue run the following command esxcfg advcfg k TRUE netNetqueueEnabled Reboot the ESX Server for the change to take effect How an ESX Server Creates and Names Interfaces The NIC driver supports a maximum of four UCNAs per system For dual channel UCNAs runnin
18. command is invalid DATA None ACTION None required elx_mes2015 Invalid CT x command lt iocb ulpCommand gt DESCRIPTION Invalid Command Type in the IOCB is not supported DATA None ACTION None required elx_mes2017 REG_FCFI mbxStatus error lt mbx status gt HBA state lt port_state gt DESCRIPTION The REG_FCFI mailbox command has failed DATA None ACTION None required Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 56 gt E EMULEX elx_mes2018 REG_VFI mbxStatus error lt mbx status gt HBA state lt port_state gt DESCRIPTION The REG_VFI mailbox command has failed DATA None ACTION None required elx_mes2020 Failed to allocate mbox for ADD_FCF cmd DESCRIPTION Failed to allocate mailbox for ADD_FCF command DATA None ACTION None required elx_mes2022 INIT VPI Mailbox failed status lt status gt mbxStatus lt mbxStatus gt DESCRIPTION The INIT VPI mailbox command has failed DATA None ACTION None required elx_mes2401 Failed to allocate memory for ELS XRI management array of size lt els_xri_cnt gt DESCRIPTION Initialization failed to allocate memory for the ELS XRI management array DATA None ACTION None required elx_mes2500 EQ_CREATE mailbox failed with status lt shdr_status gt add_status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The mailbox command sent to create the event queue has failed DATA None ACTION None required elx_mes2501
19. eee ein lil ll ln ln nnn nn eee eeeeeneeeeeeeeeees 2 Compatibility seca eee cece eee cece aae a e aa ee EAT 2 Things to Know Before You Download cccceceeeeeeeteeeeeteteetieaaaaaaaaaaaee 2 ISMOWIM ISSUCS i csicscicictenebshenehensbanenahehenehanetanshenshanehacshanshanesaceeetthaceaaawdevsvte chacenacas 2 Installing the FC FCOE Driver cceeccceeeeeeceeeeeeeeccneeeeeeeeeeeeeeeeecaaaeeeeeeeseseeneaeeeeeees 2 LILO e Elote 1EM E E E 2 Installing the FC FCoE Driver and Management Software n e 2 Uninstalling the FC FCOE DYriver cccccecceeeececeeeeeeeeeeeeeccneeeeeeeetesesscnaeeeeeees 2 Installing the FC FCoE Utilities ec eceeeeee eee eeeeeeeeeeeeeeeeeeeeeeccaaeeeeeeeeenee 3 Uninstalling the FC FCOE Utilities 2 0 00 eee cece eee eeeecneeeeeeeeeeeeenaaeeeeeeeeeetees 3 Installing the NIC Driver cccccscecceeeeeeeeeccneeeeeeeeeeececaaeeeeeeeeeeeecaaaeeeeeeeeesessnaeeeeeees 3 MOGU TION sate a cen a e ich xe ctu hieh bi natch at ata akin beth ah ei cha une ER 3 Installing the NIC Driver and Management Software eceseeeeeeeeeeeeeetaees 3 Uninstalling the NIC Driver cccccceeceeeeeeeeceeeeeeeeeeeteecaeeaaaeeeeeeeeeeeecnaeeeeettee 4 Installing All the Drivers 0 000 nl nnn nn nn nn nn nn nn nn nll i ilies 4 ed ODIVI Saiar ene eer rere eer rere eee ee eee ee eer 4 NIC Di iver cceeeeeeeceee cece eee eeeecaeee eee eee e eee aaeeeeeeeeeeeeeaaaeeeeeeeeegeecdaeeeeeeeennnee 4 Resi eer
20. er ere errr re 4 esxupdate Procedure eecceeeee cece cette teeta eeeeeeeeeeeneeeeeeeeeeeees 5 Vinyalizaton Feat o Sson E OOO 5 Enabling the NetQueue Feature sssssssssssiereeesssenirrrrnssssrnrnnnnntsssnnnnnnnnsnsnnnnn 5 How an ESX Server Creates and Names Interfaces cccccceeeeeeeeeeeeeeeees 5 Configuring VLANS cccceeeeeeneeeeeeeeeeeeeaaeeeeeeeeeeecaaaeeeeeeeeseeecaaeeeeeeeeesenenaeeeeeees 6 Configuring Network Heap Size in ESX Server ccccecccecceeceeeeeeeeeeestneeeeeees 6 Interrogating the NIC Driver c cccceeeeeeeeeeeeeeeeeeeeeeeeccaaeeeeeeeeeeeeenaeeeeeees 6 Co fig rati N sec escite cates scthcanecenctenceecee deteeceeserdtes dainai a aiai 7 Introduction to FC FCOE Configuration ccecesceeeeeeeeeeeeeeeeecneeeeeeeeeeseencaeeeeeeeeeetees 7 Temporary FC FCoE Configuration Methods Using Native ESX Tools 7 Permanent FC FCoE Configuration Methods Using Native ESX Tools 7 Dynamically Adding LUNs and Targets cceccceceeeeeeeeeeeeeeeeeeeeeeeeeeeeeeteeenenes 9 Emulex FC FCoE Driver Configuration Parameters cceccceeeeeeeeecteeeeeeeeeetees 9 Creating a Fibre Channel Remote Boot Disk 0 ccceeeeeceeeeeeeeeeeeeeeneeeeeeteeeeees 11 Managing ESX ESXi Server Through the CIM Interface ccsceeeeeeeeeeeneeeeees 11 Installing the Emulex CIM Providelr cccceseecceeeeeeeeeeeeeecnneeeeeeeeeeeeeenaeeeeeetees 11
21. for reg_login DESCRIPTION Memory allocation error DATA 1 nlp_DID 2 nlp_state 3 nlp_flag 4 nlp_rpi ACTION Memory allocation error Check system resources Unload unused modules elx_mes0134 PLOGI cannot issue reg_login DESCRIPTION The ELS PLOGI mailbox command has failed DATA 1 nlp_DID 2 nlp_state 3 nlp_flag 4 nlp_rpi ACTION Check the port and switch configuration elx_mes0135 cannot format reg_login DESCRIPTION Could not allocate an RPI or DMA buffer for the mailbox command DATA 1 nlp_DID 2 nlp_state 3 nlp_flag 4 nlp_rpi ACTION None required elx_mes0136 PLOGI completes to N_Port lt DID gt completion DESCRIPTION A PLOGI has completed for which there is no NDLP DATA 1 ulpStatus 2 ulpWord 4 ACTION None required elx_mes0137 No retry ELS command lt ELS_CMD gt to remote DESCRIPTION DATA 1 ulpStatus 2 ulpWord 4 ACTION None required elx_mes0138 ELS rsp Cannot issue reg_login for lt DID gt DESCRIPTION REG_LOGIN mailbox command failed DATA 1 nlp_DID 2 nlp_state 3 nlp_flag 4 nlp_rpi ACTION None required Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 23 gt E EMULEX elx_mes0139 Ignoring ELS cmd tag lt ioTag gt completion Data DESCRIPTION This ELS command was aborted DATA 1 ulpStatus 2 ulpWord 4 3 ulpTimeout ACTION None required elx_mes0140 PLOGI Reject invalid nname DESCRIPTION Invalid node WW
22. gt lt EMULEX Emulex Drivers for VMware ESX ESXI 4 1 User Manual One Network One Company P006614 01A Rev A Connect with Emulex gt G EMULEX Copyright 2011 Emulex All rights reserved worldwide No part of this document may be reproduced by any means or translated to any electronic medium without the prior written consent of Emulex Information furnished by Emulex is believed to be accurate and reliable However no responsibility is assumed by Emulex for its use or for any infringements of patents or other rights of third parties which may result from its use No license is granted by implication or otherwise under any patent copyright or related rights of Emulex Emulex the Emulex logo AutoPilot Installer AutoPilot Manager BlockGuard Connectivity Continuum Convergenomics Emulex Connect Emulex Secure EZPilot FibreSpy HBAnyware InSpeed LightPulse MultiPulse OneCommand OneConnect One Network One Company SBOD SLI and VEngine are trademarks of Emulex All other brand or product names referenced herein are trademarks or registered trademarks of their respective companies or organizations Emulex provides this manual as is without any warranty of any kind either expressed or implied including but not limited to the implied warranties of merchantability or fitness for a particular purpose Emulex may make improvements and changes to the product described in this manual at any time and without any notice Emulex
23. hba linkstate lt link_state gt DESCRIPTION Discovery has timed out and the adapter state is not ready DATA None ACTION None required elx_mes0231 RSCN timeout DESCRIPTION The driver has lost track of what N_Ports have RSCNs pending DATA 1 fe_ns_retry 2 lofc_max_ns_retry ACTION None required The driver should recover from this event elx_mes0233 Nodelist not empty DESCRIPTION Driver unloaded or hotplug detected a node still in use DATA None ACTION None required elx_mes0246 RegLogin failed DESCRIPTION The firmware returned a failure for the specified RegLogin DATA 1 Did 2 mbxStatus 3 hbaState ACTION This message indicates that the firmware could not do RegLogin for the specified Did There may be a limitation on how many nodes an adapter can see elx_mes0249 Cannot issue Register Fabric login Err lt err gt DESCRIPTION Could not issue the fabric reg login the err value is unique for each possible failure DATA None ACTION None required elx_mes0251 NameServer login no memory DESCRIPTION Could not allocate memory for the NDLP structure DATA None ACTION None required elx_mes0252 Cannot issue NameServer login DESCRIPTION Could not issue an ELS PLOGI to the nameserver DID DATA None ACTION Check the port connection and switch configuration elx_mes0253 Register VPI Can t send mbox DESCRIPTION Could not issue the REG_LOGIN command for this VPort DATA None ACT
24. mailbox command sent to delete the completion queue has failed DATA None ACTION None required elx_mes2507 MQ_DESTROY mailbox failed with status lt shdr_status gt add_status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The mailbox command sent to delete the mailbox queue has failed DATA None ACTION None required elx_mes2508 WQ_DESTROY mailbox failed with status lt shdr_status gt add_ status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The mailbox command sent to delete the work queue has failed DATA None ACTION None required elx_mes2509 RQ_DESTROY mailbox failed with status lt shdr_status gt add_status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The mailbox command sent to delete the receive queue has failed DATA None ACTION None required elx_mes2510 RQ_DESTROY mailbox failed with status lt shdr_status gt add_status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The mailbox command sent to delete the receive queue has failed DATA None ACTION None required elx_mes2511 POST_SGL mailbox failed with status lt shdr_status gt add_ status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The mailbox command sent to post the SGL pages to the firmware has failed DATA None ACTION None required elx_mes2512 REMOVE_ALL_SGL_PAGES mailbox failed with status lt shdr_status gt add_status lt shdr_add_status gt mbx status lt rc gt
25. run the following command ifconfig eth lt N gt mtu 9000 where lt N gt is the number of the Ethernet interface on which you are working Setting the MTU Size for a Windows Guest Operating System To set the MTU in each Windows guest operating system Go to the Start menu and select Control Panel gt System Select the Hardware tab and open Device Manager Expand the Network Adapters heading Right click on the NIC and select Properties Select the Advanced tab and set the MTU value a ey a Drivers for VMware ESX ESxXi 4 1 Configuration Section Page 13 gt G EMULEX Pinning VMs and Interrupts to CPUs The OneConnect driver supports MSI X interrupts The driver requests separate MSI X interrupt request IRQ vectors for each of the interfaces Pinning VMs and the IRQ vector to a CPU core helps provide the best performance from the OneConnect UCNA With a multi core CPU pinning a VM and the MSI X vector of the OneConnect driver interface configured in that VM to two cores sharing an L2 cache provides the best performance For example in a quad core Xeon CPUO and CPU1 share a 4MB L2 cache If you have configured vmnic16 in VM1 for the best performance pin VM1 to CPUO and the IRQ corresponding to vmnic16 to CPU1 To pin a VM to a CPU in the VI Client pe Ose 5 To pin an IRQ vector to a CPU run the following command in the console operating system Power off the VM Click Edit Settings in the Resources tab for
26. that VM Select Advanced CPU In the Scheduling Affinity frame select the CPU to pin it to Power on the VM echo move lt IRQ NO gt lt CPU ID gt gt proc vmware intr tracker You can find the IRQ NO corresponding to a vmnic by running cat proc vmware interrupts To see all IRQ Vectors and the CPU to which they are pinned run cat proc vmware intr tracker Drivers for VMware ESX ESXi 4 1 Configuration Section Page 14 gt G EMULEX Managing Devices using CIM VMware on the Visor based ESX platforms uses the Common Interface Model CIM as the only standard management mechanism for device management OneCommand Manager uses the standard CIM interfaces to manage the adapters in the ESX COS and Visor environments OneCommand Manager supports the CIM based device and HBA management in addition to the existing HBA management functionality around its proprietary management stack along with the standard HBAAPI interface For One Command Manager to manage the adapters of an ESXi host the ESX ESXi host should be installed with the Emulex CIM Provider ESX ESXi 3 5 4 0 and 4 1 come along with an inbox Emulex CIM Provider The inbox Emulex CIM Provider provides capabilities to manage the Emulex Light Pulse adapters but not the Emulex UCNA adapters For managing the Emulex UCNA adapters the out of the box Emulex CIM Provider needs to be installed explicitly The Emulex CIM Provider is available as a core kit rpm in ESX COS p
27. 2 if enabled depending on type of support needed Ipfc_link_speed 0 O auto select No Sets link speed 1 1 Gb s Note Not supported for 2 2 Gb s FCoE 4 4 Gb s 8 8 Gb s Ipfc_log_verbose 0x0 0x0 Ox7fffffff Yes Extra activity logging bit mask Ipfc_lun_queue_depth 30 1 128 Yes Default max commands sent to a single logical unit disk Drivers for VMware ESX ESXi 4 1 Configuration Section Page 9 Table 1 FC FCoE Driver Configuration Parameters Continued gt G EMULEX Variable Default Min Max Dynamic Comments lpfc_max_scsicmpl_time 0 0 60000 Yes Limits SCSI command completion time in mS to control I O queue depth The default 0 means the SCSI layer maintains control Ipfc_scan_down 0 Off 1 On No Select a method for scanning ALPA to assign a SCSI ID Ipfc_tgt_queue_depth 8192 10 8192 No Default maximum number of commands sent to a single target By default there is no effective limit at the target level Ipfc_topology 0x0 loop then P2P 0x1 internal loopback 0x2 P2P only 0x4 loop only Ox6 P2P then loop No FC link topology Defaults to loop If that fails the driver attempts to link in point to point mode Note Not supported for FCoE Ipfc_use_adisc 0 Off 1 On Yes Send ADISC instead of Port Login PLOGI for device discovery or Registered State Change Notification RSCN Ipfc_devloss_tmo 10 1 25
28. 4 3 ulpTimeout ACTION This error could indicate a software driver or firmware problem If problems persist report these errors to Technical Support elx_mes0113 An FLOGI ELS command lt elsCmd gt was received from DID lt did gt in Loop Mode DESCRIPTION While in Loop Mode an unknown or unsupported ELS command was received DATA None ACTION Check device DID elx_mes0115 Unknown ELS command lt elsCmd gt received from N_ Port lt did gt DESCRIPTION Received an unsupported ELS command from a remote N_Port DATA None ACTION Check remote N_Port for potential problem Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 22 gt E EMULEX elx_mes0125 FDISC Failed x x Fabric out of resources DESCRIPTION The fabric rejected an FDISC because the switch can not support any more Virtual ports DATA None ACTION Reconfigure the switch to support more NPIV logins If problem persists contact Technical Support elx_mes0126 FDISC failed ulpStatus ulpWord 4 n DESCRIPTION The ELS FDISC command has failed DATA None ACTION Check the port and switch configuration elx_mes0127 ELS timeout DESCRIPTION An ELS IOCB command was posted to a ring and did not complete within ULP timeout seconds DATA 1 elscmd 2 remote_id 3 ulpcommand 4 ulploTag ACTION If no ELS command is going through the adapter reboot the system If problem persists contact Technical Support elx_mes0133 PLOGI no memory
29. 5 Yes Number of seconds a remote port can drop from the SAN before that port is removed from the driver Ipfc_use_msi 0 use INTX min 1 use MSI 2 use MSI X max No Selects which interrupt mode to use By default the driver uses INTX VMware guidance is to use MSI X but MSI is available lpfc_fcp_waq_count No Configures the number of fast path work queues used by the host and port Ipfc_fcp_eq_count No Configures the number of fast path event queues used by the host and port lpfc_sg_seg_count 64 64 256 No Configures the maximum number of scatter gather elements the driver accepts in a single SCSI command Drivers for VMware ESX ESXi 4 1 Configuration Section Page 10 gt G EMULEX Table 1 FC FCoE Driver Configuration Parameters Continued Variable Default Min Max Dynamic Comments Ipfc_iocb_cnt 1 1 5 No OCBs allocated for ELS CT ABTS in 1024 increments Default is 1 Creating a Fibre Channel Remote Boot Disk For instructions on creating a Fibre Channel remote boot disk refer to the VMware SAN configuration documentation Chapter 6 Using Boot from SAN with ESX Server Systems Managing ESX ESXi Server Through the CIM Interface This manual also describes how OneCommand Manager can be configured to manage VMware ESX and ESXi servers through the CIM interface Please note that the manage
30. 820 Note Type the command all on one line without a carriage return Drivers for VMware ESX ESxXi 4 1 Configuration Section Page 8 2 Reboot the server Type reboot The new settings are used when the driver reloads To verify the settings type esxcfg module g lpfc820 Dynamically Adding LUNs and Targets gt G EMULEX For instructions on dynamically adding LUNs and targets refer to the Using Rescan section of the VMware SAN Config documentation Emulex FC FCoE Driver Configuration Parameters All adapter specific parameters have an lIpfcX_ prefix where X is the driver instance number For example setting lpfcO_lun_queue_depth 20 makes 20 the default maximum number of commands that can be sent to a single logical unit disk for Ipfc instance 0 Dynamic parameters do not require a system reboot for changes to take effect Table 1 FC FCoE Driver Configuration Parameters Variable Default Min Max Dynamic Commenis Ipfc_hba_queue_depth 8192 32 8192 No Maximum number of FCP commands that can queue to an Emulex adapter The value cannot exceed what the adapter supports Ipfc_ackO 0 0 Off 1 On No Use ACKO for class 2 Ipfc_discovery_threads 32 1 64 No Specifies the maximum number of PLOGI commands that can be outstanding for a discovery Ipfc_fcp_class 3 2 3 No FC class for FCP data transmission lpfc_fdmi_on 0 0 2 Yes False 0 if disabled 1 or
31. DID lt DID gt Status lt Status gt lt Extended Status gt DESCRIPTION A ADISC to lt DID gt was failed either by the driver firmware or target The lt status gt and lt extended status gt indicates why the ADISC failed DATA None ACTION If the problem persists report the error to Technical Support elx_mes2756 LOGO failure DID lt DID gt Status lt Status gt lt Extended Status gt DESCRIPTION A LOGO to lt DID gt was failed either by the driver firmware or target The lt status gt and Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 67 J gt G EMULEX lt extended status gt indicates why the LOGO failed DATA None ACTION If the problem persists report the error to Technical Support elx_mes2757 Protocol failure detected during processing of FCP 1 O op tgt lt tgt ID gt lun lt LUN gt cmnd lt CMD gt rsplnfo3 lt rsplnfo3 gt DESCRIPTION The FCP response from a target indicated that the response length is valid but rsplnfo3 indicates that there is no Failure This is a FCP spec violation by the target DATA None ACTION If the problem persists report the error to Technical Support elx_mes2758 Failed to allocate mempool for read FCF record mbox command DESCRIPTION The driver failed to allocate memory from the mempool for issuing FCF read mailbox command during the round robin FCF bmask update DATA None ACTION None required elx_mes2759 Failed to allocate memory for
32. Failed allocate memory for fast path per EQ handle array DESCRIPTION Failed to allocate memory for the fast path per EQ handle array DATA None Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 63 gt E EMULEX ACTION None required elx_mes2573 Failed allocate memory for msi x interrupt vector entries DESCRIPTION Failed to allocate memory for MSI X interrupt vector entries DATA None ACTION None required elx_mes2574 Not enough EQs lt sli4_hba max_cfg_param max_eq gt from the pci function for supporting FCP EQs lt cfg_fcp_eq_count gt DESCRIPTION Failed to create the minimum fast path event queues DATA None ACTION elx_mes2576 Failed allocate memory for fast path EQ record array DESCRIPTION Failed to allocate memory for the fast path EQ record array DATA None ACTION None required elx_mes2577 Failed allocate memory for fast path CQ record array DESCRIPTION Failed to allocate memory for the fast path CQ record array DATA None ACTION None required elx_mes2578 Failed allocate memory for fast path WQ record array DESCRIPTION Failed to allocate memory for the fast path WQ record array DATA None ACTION None required elx_mes2581 Not enough WGs lt sli4_hba max_cfg_param max_wq gt from the pci function for supporting FCP WGs lt cfg_fcp_wq_count gt DESCRIPTION The driver was not configured with the minimum number of fast path work queues DATA None ACTION None require
33. I X has failed DATA None ACTION Ensure the hardware platform supports MSI X elx_mes0353 Active Mailbox cleared mailbox timeout exiting DESCRIPTION The mailbox timeout handler has determined that the driver is in the process of completing this mailbox command DATA None ACTION None required elx_mes0359 Not a valid slow path completion event majorcode x x minorcode x x n bf_get lpfc_eqe_major_code eqe bf_get lpfc_eqe_minor_code eqe DESCRIPTION SLI 4 The EQE is not valid DATA None ACTION None required Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 33 J gt G EMULEX elx_mes0360 Unsupported EQ count lt entry_count gt DESCRIPTION Cannot create an event queue of this size DATA None ACTION None required elx_mes0361 Unsupported CQ count lt entry_count gt DESCRIPTION Cannot create a completion queue of this size DATA None ACTION None required elx_mes0362 Unsupported MQ count lt entry_count gt DESCRIPTION Cannot create MQ count of this size DATA None ACTION None required elx_mes0364 Invalid param DESCRIPTION SLI 4 The post SGL function was passed an invalid XRI DATA None ACTION None required elx_mes0365 Slow path CQ identifier lt cqid gt does not exist DESCRIPTION The Completion Queue ID passed in the Event Queue entry does not reference a valid completion queue DATA None ACTION None required elx_mes0366 Not a valid f
34. ION None required elx_mes0254 Register VPI no memory goto mbox_err_exit DESCRIPTION Could not allocate memory for the REG_LOGIN mailbox command Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 26 J gt G EMULEX DATA None ACTION None required elx_mes0255 Issue FDISC no IOCB DESCRIPTION All of the pre allocated IOCBs are in use DATA None ACTION None required elx_mes0256 Issue FDISC Cannot send IOCB DESCRIPTION Unable to send the fabric IOCB DATA None ACTION Check the switch configuration elx_mes0257 GID_FT Query error lt ulpStatus gt lt fc_ns_retry gt DESCRIPTION The GID_FT CT request for the nameserver has failed DATA None ACTION Check the switch configuration elx_mes0258 Register Fabric login error lt mbxStatus gt DESCRIPTION The REG_LOGIN for the fabric has failed DATA None ACTION Check the port and switch configuration elx_mes0259 No NPIVFabric support DESCRIPTION The switch to which the port is connected does not support NPIV DATA None ACTION Check the switch configuration elx_mes0260 Register NameServer error lt mbxStatus gt DESCRIPTION The REG_LOGIN mailbox command has failed for the nameserver DATA None ACTION Check the switch configuration elx_mes0261 Cannot Register NameServer login DESCRIPTION Either a memory allocation issue or an invalid parameter was sent to the REG_LOGIN DATA None ACTION At least one message 0142
35. ION None required elx_mes2747 Failed to issue read FCF record mailbox command DESCRIPTION The driver wanted to send a read FCF record mailbox command to start fast FCF failover FCF scan but failed to send the mailbox command due to state of the adapter DATA None ACTION None required elx_mes2751 Adapter failed to restart status reg lt status gt FW Data A8 lt 0xA8 gt AC lt 0xAC gt DESCRIPTION The adapter has failed to restart DATA None ACTION If the problem persists report the error to Technical Support elx_mes2752 KILL_BOARD command failed retval lt retval gt DESCRIPTION The KILL BOARD mailbox command failed to complete DATA None ACTION If the problem persists report the error to Technical Support elx_mes2753 PLOGI failure DID lt DID gt Status lt Status gt lt Extended Status gt DESCRIPTION A PLOGI to lt DID gt was failed either by the driver firmware or target The lt status gt and lt extended status gt indicates why the PLOGI failed DATA None ACTION If the problem persists report the error to Technical Support elx_mes2754 PRLI failure DID lt DID gt Status lt Status gt lt Extended Status gt DESCRIPTION A PRLI to lt DID gt was failed either by the driver firmware or target The lt status gt and lt extended status gt indicates why the PRLI failed DATA None ACTION If the problem persists report the error to Technical Support elx_mes2755 ADISC failure
36. IPTION Memory allocation for this mailbox command has failed DATA None ACTION None required elx_mes2521 READ_FCF_RECORD mailbox failed with status lt shdr_status gt add_status lt shdr_add_status gt mbox DESCRIPTION The READ FCF_RECORD mailbox command has failed DATA None ACTION None required elx_mes2523 Allocated DMA memory size lt alloc_len gt is less than the requested DMA memory size lt req_len gt DESCRIPTION The ADD_FCF_RECORD mailbox command failed to retrieve the length required from the firmware DATA None ACTION None required Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 59 gt E EMULEX elx_mes2524 Failed to get the non embedded SGE virtual address DESCRIPTION The READ_FCF_RECORD mailbox command could not retrieve the Scatter Gather Entry that was requested DATA None ACTION None required elx_mes2527 Failed to allocate non embedded SGE array DESCRIPTION Failed to allocate the non embedded SGE array DATA None ACTION None required elx_mes2528 Mailbox command lt vpi gt cannot issue DESCRIPTION The mailbox command could not be issued because the mailbox interrupt is disabled DATA 1 mbxCommand 2 sli_flag 3 flag ACTION None required elx_mes2529 Mailbox command lt vpi gt cannot issue DESCRIPTION DATA 1 mbxCommand 2 sli_flag 3 flag ACTION None required elx_mes2530 Mailbox command lt vpi gt cannot issue DESCRIPTION
37. If these problems persist report these errors to Technical Support elx_mes0340 Adapter temperature is OK now DESCRIPTION Adapter temperature has reverted to normal range DATA Temperature in Celsius ACTION No action needed informational elx_mes0341 Ring lt ringno gt Cannot find buffer for an unsolicited iocb tag lt un ulpWord 3 gt DESCRIPTION There are no more pre allocated buffers available to handle unsolicited buffers DATA None ACTION Ensure this port is not being managed by multiple ports elx_mes0342 Ring lt ringno gt Cannot find buffer for an unsolicited iocb tag lt unsli3 sli83Words gt DESCRIPTION This is a multiple IOCB unsolicited command and sufficient buffer space cannot be allocated for it DATA None ACTION None required elx_mes0343 Ring lt ringno gt Cannot find buffer for an unsolicited iocb tag lt un ulpWord 3 gt DESCRIPTION There are no more pre allocated buffers available to handle unsolicited buffers DATA None ACTION None required elx_mes0344 Ring lt ringno gt Cannot find buffer for an unsolicited iocb tag lt unsli3 sli3Words 7 gt DESCRIPTION There are no more pre allocated buffers available to handle unsolicited buffers DATA None ACTION None required elx_mes0345 Resetting board due to mailbox timeout DESCRIPTION A mailbox command failed to complete The driver is resetting the port DATA None Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page
38. LPFC_MIN_DEVLOSS_TMOs gt lt LPFC_MAX_DEVLOSS_TMOs DESCRIPTION Attempt to set the nodev timeout value is outside the range of the devloss timeout range DATA None ACTION Set the nodev timeout between the minimum and maximum devloss timeout range elx_mes0404 Config Param s set to x x DESCRIPTION Driver is setting a persistent VPort parameter to a new value DATA 1 New value ACTION None This message is notification only elx_mes0405 Config Param s set to x x DESCRIPTION Driver is setting a persistent VPort parameter to a new value DATA 1 New value ACTION None This message is notification only elx_mes0406 Adapter maximum temperature exceeded lt temperature gt taking this port offline DESCRIPTION The driver has received an error for the adapter indicating that the maximum allowable temperature has been exceeded DATA 1 work_hs 2 work_status O 3 work_status 1 ACTION Ensure the server fans are not blocked Shut down the server if the airflow is restricted elx_mes0410 Cannot find virtual addr for mapped buf on ring lt ringno gt DESCRIPTION The driver cannot find the specified buffer in its mapping table Thus it cannot find the Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 37 J gt G EMULEX virtual address needed to access the data DATA 1 phys 2 next 3 prev 4 postbufq_cnt ACTION This error could indicate a software driver or firmware problem If the
39. N Software driver error If this problem persists report these errors to Technical Support elx_mes1803 Bad hbq tag Data lt tag gt lt count gt DESCRIPTION An error occurred when processing queue related tags for an adapter in a particular slot DATA 1 tag 2 buffer_count ACTION Software driver error If this problem persists report these errors to Technical Support elx_mes1804 Invalid asynchronous event code lt evt code gt DESCRIPTION The asynchronous event code that the firmware passed to the driver is invalid DATA None ACTION None required elx_mes1805 Adapter failed to init Data lt command gt lt status gt lt queue num gt DESCRIPTION An error occurred when processing queue related tags for an adapter in a particular slot DATA 1 mbxCommand 2 mbxStatus 3 hbaqno ACTION Software driver error If this problem persists report these errors to Technical Support Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 51 J gt G EMULEX elx_mes1806 Mbox lt commanod gt failed No vport DESCRIPTION A mailbox command could not be communicated because there was no VPort associated with the mailbox command DATA 1 mbxCommand ACTION Software driver error If this problem persists report these errors to Technical Support elx_mes1807 IOCB lt value gt failed No vport DESCRIPTION An IOCB command could not be communicated because there was no VPort associated with the mailbox co
40. N provided DATA None ACTION None required elx_mes0141 PLOGI Reject invalid pname DESCRIPTION Invalid port WWN provided DATA None ACTION None required elx_mes0142 PLOGI RSP Invalid WWN DESCRIPTION The PLOGI sent to the port by a remote port had an invalid WWN DATA None ACTION None required elx_mes0144 Not a valid WCQE code lt Completion Code gt DESCRIPTION The completion queue handler detected an invalid type DATA None ACTION None required elx_mes0147 Failed to allocate memory for RSCN event DESCRIPTION Memory could not be allocated to send the RSCN event to the management application DATA None ACTION None required elx_mes0148 Failed to allocate memory for LOGO event DESCRIPTION Memory could not be allocated to send the LOGO event to the FC transport DATA None ACTION None required elx_mes0154 Authentication not complete DESCRIPTION Authentication was restarted because the previous authentication did not complete DATA None ACTION Check the switch configuration Link Discovery Events 0200 0299 elx_mes0200 CONFIG_LINK bad hba state lt hba_state gt DESCRIPTION A CONFIG_LINK mbox command completed and the driver was not in the right state DATA None ACTION Software driver error If this problem persists report these errors to Technical Support Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 24 J gt G EMULEX elx_mes0203 Devloss timeout on
41. NameServer login DESCRIPTION Our login request to the NameServer was not acknowledged within RATOV DATA None ACTION Check the fabric configuration The driver recovers from this and continues with device discovery elx_mes0224 NameServer Query timeout DESCRIPTION Node authentication timeout node Discovery timeout A NameServer Query to the Fabric or discovery of reported remote N_Ports is not acknowledged within R_A_TOV DATA 1 fc_ns_retry 2 f _max_ns_retry ACTION Check Fabric configuration The driver recovers from this and continues with device discovery elx_mes0226 Device discovery completion error DESCRIPTION This indicates that an uncorrectable error was encountered during device re discovery after a link up Fibre Channel devices are not accessible if this message is displayed DATA None ACTION Reboot the system If the problem persists report the error to Technical Support Run with verbose mode on for more details elx_mes0227 Node Authentication timeout DESCRIPTION The driver has lost track of what N_Ports are being authenticated DATA None ACTION None required The driver should recover from this event Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 25 J gt G EMULEX elx_mes0228 CLEAR LA timeout DESCRIPTION The driver issued a CLEAR_LA that never completed DATA None ACTION None required The driver should recover from this event elx_mes0230 Unexpected timeout
42. The SLI layer in the driver is inactive DATA 1 mb mbxCommand 2 sli_flag 3 flag ACTION None required elx_mes2531 Mailbox command lt cpi gt cannot issue DESCRIPTION DATA 1 mb mbxCommand 2 sli_flag 3 flag ACTION None required elx_mes2532 Mailbox command lt vpi gt lt mbxCommand gt cannot issue DESCRIPTION The mailbox bootstrap code detected that the SLI layer is active DATA 1 sli4_mbox_opcode 2 sli_flag 3 MBX_POLL ACTION None required elx_mes2533 Mailbox command lt vpi gt lt mbxCommand gt cannot issue DESCRIPTION DATA 1 sli4_mbox_opcode 2 sli_flag 3 MBX_NOWAIT ACTION None required elx_mes2535 Unsupported RQ count lt entry_count gt DESCRIPTION The receive queue ring can only be 512 1024 2048 or 4096 DATA None ACTION None required elx_mes2536 Unsupported RQ count lt entry_count gt DESCRIPTION The receive queue ring can only be 512 1024 2048 or 4096 Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 60 gt E EMULEX DATA None ACTION None required elx_mes2537 Receive Frame Truncated DESCRIPTION The receive unsolicited handler detected a truncated frame DATA None ACTION None required elx_mes2541 Mailbox command lt vpi gt lt mbxCommand gt cannot issue DESCRIPTION DATA 1 sli4_mbx_opcode 2 sli_flag 3 flag ACTION None required elx_mes2543 Mailbox command lt vpi gt lt mbxCommand gt cann
43. WWPN lt address gt N_Port lt nlp_DID gt DESCRIPTION A remote N_Port that was discovered by the driver disappeared for more than lofc_devloss_tmo seconds DATA 1 nlp_flag 2 nlp_state 3 nlp_rpi ACTION If the device generating this message is not a target to which the adapter is connected this error does not affect the data integrity of the I O between the adapter and the attached storage and can be ignored elx_mes0206 Device discovery completion error DESCRIPTION This indicates that an uncorrectable error was encountered during device re discovery after a link up Fibre Channel devices are not accessible if this message is displayed DATA None ACTION Reboot the system If the problem persists report the error to Technical Support Run with verbose mode on for more details elx_mes0207 Device lt DID gt lt WWN gt sent invalid service parameters Ignoring device DESCRIPTION Invalid service parameters were received from DID Ignoring this remote port DATA DID WWN ACTION Verify the remote port s configuration If the problem persists report the error to Technical Support Run with verbose mode on for more details elx_mes0222 Initial FLOG FDISKI timeout DESCRIPTION The driver sent the initial FLOGI or FDISK to the fabric and never got a response back DATA None ACTION Check Fabric configuration The driver recovers from this and continues with device discovery elx_mes0223 Timeout while waiting for
44. _mes0293 PM resume failed to start workerthread error lt error gt DESCRIPTION The PCI resume hotplug could not start the worker thread for the driver DATA None ACTION Unload and reload the driver elx_mes0294 PM resume Failed to enable interrupt DESCRIPTION The PCI resume hotplug could not get an interrupt vector DATA None ACTION Unload and reload the driver Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 29 gt G EMULEX invalid device group lt pci_dev_grp gt DESCRIPTION While unloading the driver the driver detect a PCI device that it should not have claimed DATA None ACTION None required elx_mes0299 Invalid SLI revision lt sli_rev gt DESCRIPTION While processing a host attention or unrecoverable error the driver detected an invalid SLI revision DATA None ACTION None required Mailbox Events 0300 0339 elx_mes0300 LATT Cannot issue READ _LA Data lt rc gt DESCRIPTION The link attention handler could not issue a READ_LA mailbox command DATA None ACTION None required elx_mes0303 Ring lt ringno gt handler portRspPut lt portRspPut gt is bigger then rsp ring lt portRspMax gt DESCRIPTION The port rsp ring put index is larger than the size of the rsp ring DATA None ACTION This error could indicate a software driver firmware or hardware problem Report these errors to Technical Support elx_mes0304 Stray mailbox interrupt moxCommand lt mbxcomm
45. _mes0472 Unknown PCI error state x x DESCRIPTION The PCI bus has detected an error DATA 1 state value ACTION Driver resets the adapter and attempts recovery If problem persists contact Emulex technical support elx_mes0474 Unable to allocate memory for issuing MBOX_CONFIG_MSI command DESCRIPTION Mailbox memory pool allocation error DATA None ACTION None required elx_mes0475 Not configured for supporting MSI X cfg_use_msi lt cfg_use_msi gt DESCRIPTION The lpfc_use_msi module parameter should have been set to 2 DATA None ACTION Set module parameter Ipfc_use_msi 2 elx_mes0476 HBA not supporting SLI 3 or later SLI Revision lt sli_rev gt DESCRIPTION The adapter does not support SLI 3 or SLI 4 DATA None ACTION This adapter does not support msi Set lpfc_use_msi 0 elx_mes0479 Deferred Adapter Hardware Error DESCRIPTION An adapter hardware error was sent to the driver DATA 1 work_hs 2 work_status O 3 work_status 1 ACTION Perform a dump using hbacmd elx_mes0483 Invalid link attention link speed x x bf_get lpfc_acqe_link_speed acqe_link DESCRIPTION The link speed reported in the link attention interrupt is invalid DATA None ACTION Check the switch configuration elx_mes0493 SLI CONFIG_SPECIAL mailbox failed with status lt rc gt DESCRIPTION Mailbox command failed DATA None ACTION Ensure the adapter s firmware is current Unload and reload the driver el
46. ailed DESCRIPTION The asynchronous link event handler could not allocate a DMA buffer for the mailbox command to issue the READ_LA read link attention mailbox command DATA None ACTION None required elx_mes0397 The mbuf allocation failed DESCRIPTION The asynchronous link event handler could not allocate DMA able memory for the READ_LA mailbox command DATA None ACTION None required Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 36 gt G EMULEX elx_mes0398 Invalid link fault code lt hex link_fault gt DESCRIPTION The attempt to read the link attention register has returned an unknown value DATA None ACTION None required elx_mes0399 Invalid link attention type lt hex link_type gt DESCRIPTION The READ_LA mailbox command has returned an invalid link type DATA None ACTION None required Initialization Events 0400 0599 elx_mes0400 Phys Attribute Count Exceeded Max d Actual d DESCRIPTION Too many driver configuration parameters have been set The limit is given as Max DATA 1 Maximum number 2 Actual number ACTION Reduce the number of actual parameters elx_mes0402 Cannot find virtual addr for buffer tag on ring lt ringno gt DESCRIPTION A DMA buffer is not available for this unsolicited command DATA 1 tag 2 next 3 prev 4 postbufq_cnt ACTION None required elx_mes0403 lpfc_nodev_tmo attribute cannot be set to lt val gt allowed range is lt
47. all the user can provide them along with the command like this gt hbacmd h 10 19 20 16 m cim u root p pass4321 n root emulex portattributes 10 00 00 00 c9 cc dd ee CIM Provider Versions The latest CIM Provider released is a bundle of the FC Provider and the UCNA Provider Table 3 CIM Provider Versions lists the versions of the providers used in each release Table 3 CIM Provider Versions UCNA Version CIM Provider Version FC Provider Version UCNA Provider Version UCNA 1 1 3 0 28 1 2 2 42 1 1 1 11 1 UCNA 1 2 3 1 17 1 2 2 57 1 1 2 17 1 UCNA 1 3 3 2 16 1 2 2 76 1 1 3 16 1 Drivers for VMware ESX ESXi 4 1 CIM Section Page 19 gt G EMULEX Troubleshooting Introduction There are several circumstances in which your system may operate in an unexpected manner The Troubleshooting section explains many of these circumstances and offers one or more workarounds for each situation FC FCoE Troubleshooting General Situations Table 4 General Driver Situations Situation Resolution Port link fails to come up If an FC link fails to come up verify that an 8 Gb s adapter is not attempting to connect to a 1 Gb s device Only 2 4 and 8 Gb s devices are supported on 8 Gb s adapters For LP21000 adapters ensure the adapter is not in maintenance mode and that it is not running the manufacturing firmware For the OCe10100 family of adapters ensure that the fabric port is enabled Th
48. alue Warning Failed to allocate memory for Memory allocation for pass through pass through command command failed The driver should continue to function The configuration utility that issued the pass through ioctl fails Warning Pass through command failed opcode The pass through firmware command with lt DDD gt status 0x lt XXX gt indicated opcode failed The driver should continue to function The configuration utility that issued the pass through ioctl fails Warning Command to modify EQ delay failed Firmware command to change the EQ delay failed The driver continues to function The adaptive interrupt coalescing does not function correctly Page 73 gt G EMULEX NIC Adapter Firmware Error The following POST message appears if you have loaded firmware on the OneConnect adapter that the controller does not support POST Error Firmware halted This firmware does not support this controller CIM Provider Troubleshooting The following error message may appear if the CIM hosts are not properly added to the OneCommand Manager Unknown or invalid host specified There could be instances when even with all the drivers CIM Provider and the CIM Client on a Windows machine are properly installed the CIM hosts might still not get added to OneCommand Manager Listed below are the most common reasons for this e The machine with the specified IP is not reachable Try pinging the machine e The specified protocol HTTP
49. and gt mbxStatus lt mbxstatus gt DESCRIPTION Received a mailbox completion interrupt and there are no outstanding mailbox commands DATA None ACTION This error could indicate a hardware or firmware problem If the problem persists report the error to Technical Support elx_mes0306 CONFIG_LINK mbxStatus error lt mbxStatus gt HBA state lt hba_state gt DESCRIPTION The driver issued a CONFIG_LINK mbox command to the adapter that failed DATA None ACTION This error could indicate a firmware or hardware problem Report these errors to Technical Support elx_mes0310 Mailbox command lt mbxcommand gt timeout DESCRIPTION A mailbox command was posted to the adapter and did not complete within 30 seconds DATA 1 hba_state 2 sli_flag 8 mbox_active ACTION This error could indicate a software driver or firmware problem If no I O is going through the adapter reboot the system If the problem persists report the error to Technical Support Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 30 gt G EMULEX elx_mes0312 Ring lt ringno gt handler portRspPut lt rspPutlnx gt is bigger then rsp ring lt numRiocb gt DESCRIPTION The IOCB command rings put pointer is ahead of the get pointer DATA None ACTION None required elx_mes0315 Ring lt ringno gt issue portCmdGet lt local_getidx gt is bigger then cmd ring lt max_cmd_idx gt DESCRIPTION The port cmd ring get index is greater than the
50. aring for recovery DESCRIPTION The driver writes this message to the console when it is preparing the port for a recovery operation DATA None ACTION None required This is a notification message for the recovery action elx_mes2831 FLOGI response with cleared Fabric bit fcf_index 0x x Switch Name 02x 02x 02x 02x 02x 02x 02x 02x Fabric Name 02x 02x 02x 02x 02x 02x 02x 02x DESCRIPTION When the driver completed a FLOGI the common service parameters did not indicate an FPort or NPort remote node The driver treats this as an error DATA None ACTION Validate the external cable connection and FPort Nport configuration Contact technical support if problems persist elx_mes2858 FLOGI failure Status x x x x TMO x x DESCRIPTION The driver issued a FLOGI but never received any completion with the timeout period The driver is marking the FLOGI as failed and stops discovery DATA None ACTION Please check your fabric to ensure it it operating correctly Contact technical support if problems persist elx_mes2862 FCF x x matches property of in use FCF x x DESCRIPTION The driver has found an FCF record that matches the properties of the current FCF record except for the VLAN id and Index The driver will attempt to use this FCF DATA None ACTION None required The driver is in its FCF discovery phase and is trying to recover a match to its in use FCF elx_mes2863 New FCF x x matches property of
51. ast path completion event majorcode lt major code hex gt minor code lt minor code hex gt DESCRIPTION The major or minor code in the Event Queue field is not valid DATA None ACTION None required elx_mes0367 Fast path completion queue does not exist DESCRIPTION The fast path completion queue referenced by the CQID does not exist DATA None ACTION None required elx_mes0368 Miss matched fast path completion queue identifier eqcqid lt cqid gt fcpcqid lt queue_id gt DESCRIPTION The CQID in the event queue entry does not match the fcp_cqid that was passed into the routine DATA None ACTION None required elx_mes0369 No entry from fast path completion queue fcpcqid lt queue_id DESCRIPTION There were no completions in the completion queue referenced by fcpcqid DATA None ACTION None required Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 34 J gt G EMULEX elx_mes0370 Invalid completion queue type lt type gt DESCRIPTION The event queue entry is not for a mailbox or a work queue entry DATA None ACTION None required elx_mes0371 No entry from the CQ identifier lt queue_id gt type lt type gt DESCRIPTION There was no completion queue event for this event queue entry DATA None ACTION None required elx_mes0372 iotag lt iotag gt is out of range max iotag lt sli last_iotag gt DESCRIPTION The IOCB lookup cannot be performed because the iocb_tag is out o
52. aults to return the discovery properties to their default settings How to Set the Default Credentials Using hbacmd The hbacmd setcimcred command can be used to set the default credentials that will be used in all the subsequent CIM management commands if the user does not provide any with those commands The syntax of this command is given below gt hbacmd setcimcred lt username gt lt password gt lt namespace gt lt port gt For example gt hbacmd setcimcred root pass123 root emulex 5989 The default credentials set by the OneCommand Manager GUI and hbacmd will be shared which means that if a set of default credentials is set by the OneCommand Manager GUI the same will be used by hbacmd and vice versa Commands Using the Default CIM Credentials Once the default CIM credentials are set using the above command these default credentials will be used for all the subsequent commands using the CIM interface To specify that the CIM interface is to be used the hbacmd commands should be provided with an m cin flag like this example gt hbacmd h 10 11 12 13 m cim portattributes 10 00 00 00 c9 11 22 33 In the preceding example this requests the port attributes of port 10 00 00 00 c9 11 22 33 which is present in the host with the IP address 10 11 12 13 The default credentials set will be used Commands Using the User Provided CIM Credentials If the credentials differ from the set default values or if the credentials are not set at
53. axSize 128 The new value takes effect after a reboot Interrogating the NIC Driver To get information on the installed NIC driver enter esxupdate query vib view grep be2net This is a sample output root blade10 esxupdate query vib view grep be2net cross _vmware esx drivers net be2net_ 400 2 102 225 12 1 0 4 164009 installed 2010 04 05T11 28 38 540372 07 00 cross _vmware esx drivers net be2net_400 2 102 200 10 1 0 4 164009 retired 2010 03 18T13 59 02 520994 07 00 Drivers for VMware ESX ESXi 4 1 Installation Section Page 6 gt G EMULEX Configuration Introduction to FC FCoE Configuration You can configure the driver parameters using native ESX tools or the Emulex OneCommand Manager application This document describes how to configure parameters using native ESX tools For a more comprehensive description of ESX tools refer to VMware s public website If you have further questions contact a VMware technical support representative Refer to the Emulex OneCommand Manager User Manual for more information about the utility Note For VMware ESX Server systems the firewall on the ESX Server must be opened to manage systems remotely using TCP IP To enable TCP port 23333 run the following commands esxcfg firewall openPort 23333 tcp in onecommand esxcfg firewall openPort 23333 tcp out onecommand To verify that the correct port is open run the following command esxcfg firewall q The TCP po
54. boot Note VMware does not officially support unloading the driver via vmkload_mod u If you must unload the driver contact VMware technical support Note NPIV port creation and deletion are performed by the VMware vSphere client or Virtual Center service Refer to the VMware documentation for more information Example of Permanent Global Configuration The following example sets lun_queue_depth the maximum number of commands that can be sent to a single LUN to 20 default is 30 for all Emulex adapters in your system Locate the parameter Ipfc_lun_queue_depth in Table 1 on page 9 2 Set the permanent value Type esxcfg module s lpfc_ lun queue depth 20 lpfc820 3 Reboot the server Type reboot The new setting is used when the driver reloads To verify the setting type esxcfg module g lpfc820 Example of Permanent Per Adapter Configuration The following example sets lun_queue_depth to 20 default is 30 for adapter 1 1 Set the adapter specific value Type esxcfg module s lpfcl lun queue depth 20 lpfc820 2 Reboot the server Type reboot The new setting is used when the driver reloads To verify the setting type esxcfg module g lpfc820 The following example sets lun_queue_depth to 20 default is 30 for adapter 1 and lun_queue_depth to 10 default is 30 for adapter 2 1 Set the adapter specific value Type esxcfg module s lpfcl_lun_queue_depth 20 lpfc2_ lun _queue_depth 10 lpfc
55. bose Mask x Saas Bit Definition FCoE EVENTS 1833 2800 The verbose bit remains with the previous rows LOG_ALL_MSG 0100 2600 Oxffffffff Log all messages Message Log Example The following is an example of a LOG message Jul 2 04 23 34 daffy kernel lpfc 0000 03 06 0 0 1305 Link Down Event x2f2 received Data x2f2 x20 x110 In the above LOG message e Ipfc 0000 03 06 0 identifies the identifies the PCI location of the particular Ipfc hw port e O identifies Emulex HBAO e 1305 identifies the LOG message number Note If the word Data is present in a LOG message any information to the right of Data is intended for Emulex technical support engineering use only Note Unless otherwise noted in the ACTION attribute report these errors to Technical Support Emulex requests that when reporting occurrences of these error messages you provide a tarball of all vmkernel files in var log ELS Events 0100 0199 elx_mes0100 FLOGI failure Status lt status gt lt extended_status gt TMO lt timeout gt DESCRIPTION An ELS FLOGI command that was sent to the fabric failed DATA 1 ulpStatus 2 ulpWord 4 3 ulpTimeout ACTION This error could indicate a fabric configuration error or internal driver issue If problems persist report these errors to Technical Support elx_mes0111 Dropping received ELS cmd DESCRIPTION The driver decided to drop an ELS Response ring entry DATA 1 ulpStatus 2 ulpWord
56. box failed rc lt rc gt HBA state lt port_state gt DESCRIPTION The unregister FCFI mailbox command has failed DATA None ACTION None required elx_mes2553 Ipfc_unregister_unused_fcf failed to read FCF record HBA state DESCRIPTION DATA None ACTION None required elx_mes2554 Could not allocate memory for fcf record DESCRIPTION DATA None ACTION None required elx_mes2555 UNREG_VFI mbxStatus error lt u mb mbxStatus gt HBA state lt port_state gt DESCRIPTION The unregister VFI mailbox command has failed DATA None ACTION None required elx_mes2556 UNREG_VFI mbox allocation failed HBA state lt port_state gt DESCRIPTION Could not allocate memory for UNREG_VFI mailbox command DATA None ACTION None required elx_mes2557 UNREG_VFI issue mbox failed rc lt rc gt HBA state lt port_state gt DESCRIPTION Could not issue the UNREG_VFI mailbox command DATA None ACTION None required elx_mes2558 ADD_FCF_RECORD mailbox failed with status lt shdr_status gt add_status lt shdr_add_status gt DESCRIPTION The ADD_ FCF_RECORD mailbox command has failed DATA None ACTION None required elx_mes2560 Failed to allocate mbox cmd memory DESCRIPTION Failed to allocate mailbox command memory DATA None ACTION None required Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 62 gt G EMULEX elx_mes2561 Allocated DMA memory size lt alloclen gt is less than the requested DMA me
57. ct TCP IP gt Add Range of Hosts The Add Range of TCP IP Hosts dialog box appears Enter the complete start and end address range and click Add The added address range appears in the dialog box Add any additional ranges you want to search Click Start Discovery If an address is remotely manageable it is added to the list of addresses that the discovery server will attempt to discover The utility creates a host file if necessary and checks each address in the range to determine if the host is available and remotely manageable The number of addresses of manageable hosts discovered is periodically updated on the dialog box To save the IP address ranges click Save Ranges to File to save the specified range s to a file so that these address ranges appear the next time you use the Add Range of TCP IP Hosts dialog box Drivers for VMware ESX ESXi 4 1 CIM Section Page 16 gt G EMULEX 5 Check the Include CIM hosts check box to include ESX hosts that can be managed by the CIM interface whose IP Address fall in the specified range of IP Addresses The default credentials configured will be used while trying to connect to the ESX hosts through the CIM interface Configuring the default CIM credentials is described in the section Configuring Discovery CIM credentials and TCP IP Settings Configuring Discovery CIM Credentials and TCP IP Settings Use the OneCommand Manager application Discovery Settings dialog box to co
58. d elx_mes2597 Mailbox command lt vpi gt lt mbxCommand gt cannot issue DESCRIPTION Synchronous mailbox command failed after blocking asynchronous mailbox commands DATA 1 sli4_mbx_opcode 2 sli_flag 3 flag ACTION None required elx_mes2598 Adapter Link is disabled DESCRIPTION The adapter link is disabled DATA None ACTION None required elx_mes2599 Adapter failed to issue DOWN_LINK mbox command rc lt rc gt DESCRIPTION The adapter failed to issue a DOWN_LINK mailbox command DATA None ACTION None required Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 64 gt S EMULEX elx_mes2600 Ipfc_sli_read_serdes_param failed to allocate mailbox memory DESCRIPTION Failed to allocate mailbox memory DATA None ACTION None required elx_mes2605 Ipfc_dump_static_vport memory allocation failed DESCRIPTION Memory allocation failed DATA None ACTION None required elx_mes2606 No NPIV Fabric support DESCRIPTION No NPIV Fabric support DATA None ACTION None required elx_mes2607 Failed to allocate init_vpi mailbox DESCRIPTION Failed to allocate init_vpi mailbox DATA None ACTION None required elx_mes2608 Failed to issue init_vpi mailbox DESCRIPTION Failed to issue init_vpi mailbox DATA None ACTION None required elx_mes2609 Init VPI mailbox failed lt u mb mbxStatus gt DESCRIPTION Initialization of VPI mailbox has failed DATA None ACTION None requ
59. d Contact Emulex Technical Support elx_mes0535 Failed setup of fast path FCP WQ lt fcp_waqidx gt rc lt rc gt DESCRIPTION The fast path FCP WQ setup failed DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0536 Slow path ELS WQ not allocated DESCRIPTION The slow path ELS WQ is not allocated DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0538 Slow path MQ not allocated DESCRIPTION The slow path MQ is not allocated DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0539 Failed setup of slow path MQ rc 0x x DESCRIPTION The slow path MQ is allocated but failed initial setup DATA 1 status ACTION The driver fails to load Contact Emulex Technical Support elx_mes0540 Receive Queue not allocated DESCRIPTION The Receive Queue is not allocated DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0541 Failed setup of Receive Queue rc 0x x DESCRIPTION The Receive Queue is allocated but failed setup DATA 1 status ACTION The driver fails to load Contact Emulex Technical Support elx_mes0542 lpfc_create_static_vport failed to allocate mailbox memory DESCRIPTION Failed to allocate mailbox memory for VPort creation DATA None ACTION Static VPorts does not load Contact Emulex Technical Support Drivers for VMware ESX ESXi 4 1 Troubleshooting S
60. e Emulex driver is not loaded and all Use Ispci to determine if the Emulex ports are being properly paths are down identified If not find out if the driver iso was correctly installed You must have the correct driver for the installed adapter because the device PCI IDs are installed with the driver package Examine the var log vmkernel file for Ipfc820 log messages indicating an error In this case contact Emulex support Ipfc driver fails to recognize an adapter Upgrade the adapter firmware to the minimum supported and logs unknown IOCB messages in revision or newer listed in the installation guide the system log during driver load The adapter is running outdated firmware System panics when booted with a failed Remove the failed adapter and reboot adapter installed Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 20 gt G EMULEX Ipfc Log Messages Introduction Log messages have traditionally been organized into logical groups based on code functionality in the FC driver With the introduction of OneConnect UCNAs by Emulex that grouping is modified to account for new behaviors The traditional grouping is maintained but new messages no longer group together nicely The messages provided in this section are unmaskable error conditions They are automatically added to the system console log You can examine the var log vmkernel file to see any of these messages If you have concerns t
61. e LPFC_DATA_READY was received on the worker thread DATA None ACTION Unload and reload the driver elx_mes0434 PM resume failed to start worker thread error lt error gt DESCRIPTION The driver s power management resume function could not start the worker thread DATA None ACTION Unload and reload the driver elx_mes0435 Adapter failed to get Option ROM version status lt rc gt DESCRIPTION The driver could not read the adapter s option ROM Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 38 J gt G EMULEX DATA None ACTION Reset the adapter Ensure the adapter s firmware is current elx_mes0436 Adapter failed to init timeout status reg lt status gt DESCRIPTION The adapter failed during powerup diagnostics after it was reset DATA None ACTION This error could indicate a hardware or firmware problem If the problem persists report the error to Technical Support elx_mes0437 Adapter failed to init chipset status reg lt status gt DESCRIPTION The adapter failed during powerup diagnostics after it was reset DATA None ACTION This error could indicate a hardware or firmware problem If the problem persists report the error to Technical Support elx_mes0438 Adapter failed to init chipset status reg lt status gt DESCRIPTION The adapter failed during powerup diagnostics after it was reset DATA None ACTION This error could indicate a hardware or firmware problem If
62. e eeecaaeeeeeeeeeeeeeiaeeeeeeneea 19 TROUBISS IO OUING ac cpeencescncasetcuncecacnncunssccunntesndecauntduenscndeteancecnensdnehecandecues 20 MUCH CEN OT ace cite wi ci ric Rc cc wi wc i cc ci thie i cid ci ie 20 FC FCOE Troubleshooting cccceeeeeeeececeeeeeeeeeeeeeceaaeeeeeeeeeeeceaaaeeeeeeeeseeeneaaeeeeeees 20 General SUT ALONG cg st eg ws es ns E eae 20 lpfc Mole mY Sckct 0 ee 21 HATHOGUCTIOMN sana ia a E O A 21 Message Log Example cccccecceccceceeeeeeeeeeeeeeeeeeeeaaaaaaaaaaaaaaaaaaaaaeeeeeees 22 ELS Events 0100 0199 painiin anaa aaan iaa aiaa 22 Link Discovery Events 0200 0299 ssssrrrsrrrsrrrrrrrrrrrrunruunnunnnunnnunnnnunnruneeenenan 24 Mailbox Events 0300 0339 cssecricscsrseecseicieenieseieecieieieeeieieieeeiedeieanendtiiiiiiia 30 Initialization Events 0400 0599 0 ee ceceeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeenes 37 Node Table Events 0900 0999 li i iii 46 Miscellaneous and FCoE Events 1200 1299 cccccecceeeeeeeeeeeeeeneeeeeeeeeeeees 46 Link Events 1300 1399 02 cece cece eee e eee e eee eeeeeeeeeeeeeeeeeeteeeeeeeeeeeeetes 46 Reserved 1400 1499 ll nn nn I I I nn I Hn nn i ill iii 47 IOG TE Events 1600 1699 swisieitemtetstes caste tiatatetadeaaaetedediitadsecis tia tddaeveleaised 51 VPort Events 1800 1832 ooo ccce cee ee eee ee cece eect ee eeeeeeeeeeeeeeeeeeeeeeeeeeeeees 51 ELS Events 1833 2800 ecccccceeeeeeeeecceeeeeeeeeeeeeeeaaeaaeeeeee
63. e is nothing for the driver to do except correct the counter the txq is empty elx_mes2824 Cannot re enable interrupt after slot reset DESCRIPTION The driver failed to re enable interrupts following a PCI slot reset command DATA None ACTION A system reboot may be required to fully recover Contact technical support if problems persist elx_mes2825 Unknown PCI error state x x n state DESCRIPTION The driver writes this message to the console when the PCI subsystem has detected an error on an Emulex port and called the driver The driver reacts by resetting the port DATA None ACTION None required The driver resets the device in an attempt to recover Contact technical support if problems persist Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 69 gt G EMULEX elx_mes2826 PCI channel disable preparing for reset DESCRIPTION The driver writes this message to the console when it is preparing the port for a reset operation DATA None ACTION None required This message is notification of a corrective measure Contact technical support if problems persist elx_mes2827 PCI channel permanent disable for failure DESCRIPTION The driver writes this message to the console when a recovery mechanism has failed and the driver wants to mark the port with a permanent failure DATA None ACTION A system reboot may correct the failure If not contact technical support elx_mes2828 PCI channel I O abort prep
64. echnical Support elx_mes0503 Failed allocate fast path FCP DESCRIPTION Failed to allocate fast path FCP DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0504 Failed allocate slow path ELS WQ DESCRIPTION Failed to allocate slow path ELS WQ DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0505 Failed allocate slow path MQ DESCRIPTION Failed to allocate slow path MQ DATA None ACTION The driver fails to load Contact Emulex Technical Support Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 42 elx_mes0506 Failed allocate receive HRQ DESCRIPTION Failed to allocate receive HRQ DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0507 Failed allocate receive DRQ DESCRIPTION Failed to allocate receive DRQ DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0520 Slow path EQ not allocated DESCRIPTION The slow path EQ is not allocated DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0521 Failed setup of slow path EQ rc 0x x DESCRIPTION The slow path EQ setup failed with status rc DATA 1 status code ACTION The driver fails to load Contact Emulex Technical Support elx_mes0522 Fast path EQ lt fcp_eqidx gt not allocated DESCRIPTION The fast path EQ is not allocated DATA None ACTION The driver fail
65. ection Page 44 gt E EMULEX elx_mes0543 Ipfc_create_static_vport failed to allocate vport_info DESCRIPTION Failed to allocate VPort_info DATA None ACTION Static VPorts does not load Contact Emulex Technical Support elx_mes0545 Ipfc_create_static_vport bad information header 0x x Ox x le32_to_cpu vport_info gt signature le32_to_cpu vport_info gt rev amp VPORT_INFO_REV_MASk DESCRIPTION Invalid information header the signature or revision is invalid DATA None ACTION Static VPorts does not load Contact Emulex Technical Support elx_mes0582 Error lt rc gt during sgl post operation DESCRIPTION The SGL post operation failed DATA None ACTION None required elx_mes0602 Failed to allocate CQ_EVENT entry DESCRIPTION Failed to allocate a CQ_EVENT entry DATA None ACTION None required elx_mes0603 Invalid work queue CQE subtype lt subtype gt DESCRIPTION Invalid work queue CQE DATA None ACTION None required FCP Traffic History 0700 0799 elx_mes0700 Bus Reset on target lt i gt failed DESCRIPTION The bus reset for the specified target failed DATA None ACTION None required elx_mes0706 0706 IOCB Abort failed outstanding d failed d DESCRIPTION The driver did not recover all IO following a reset task management command DATA 1 outstanding IO count 2 number of unrecovered IO ACTION Reset call fails to ESX tries to recover elx_mes0713 SCSI layer issued Device
66. equired elx_mes1429 Invalid PCI device group lt pci_dev_grp gt DESCRIPTION Invalid adapter PCl device group detected DATA None ACTION None required elx_mes1430 Failed to initialize sgl list DESCRIPTION Failed to initialize SGL list DATA None ACTION None required elx_mes1431 Invalid HBA PCl device group lt dev_grp gt DESCRIPTION Invalid adapter PCl device group detected Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 50 gt G EMULEX DATA None ACTION None required elx_mes1432 Failed to initialize rpi headers DESCRIPTION RPI headers required by the firmware failed to initialize DATA None ACTION None required elx_mes1476 Failed to allocate sysfs attr DESCRIPTION Failed to allocate sysfs attribute DATA None ACTION None required elx_mes1477 Failed to set up hba DESCRIPTION Failed to set up adapter DATA None ACTION None required IOCTL Events 1600 1699 None VPort Events 1800 1832 elx_mes1800 Could not issue unreg_vpi DESCRIPTION Driver attempt to unregister VPI failed DATA None ACTION Software driver error If this problem persists report these errors to Technical Support elx_mes1802 HBQ lt index gt local_hbqGetldx lt index gt is gt than hbqp gt entry_count lt count gt DESCRIPTION An error occurred when processing queue related to an adapter in a particular slot DATA 1 hbqno 2 local_hbqGetldx 3 entry_count ACTIO
67. f range DATA None ACTION None required elx_mes0376 READ_REV Error SLI Level lt sli_rev gt FCoE enabled lt hba_flag amp HBA_FCOE_SUPPORT gt DESCRIPTION This SLI 4 only adapter setup function was called for a non SLlI 4 device DATA None ACTION None required elx_mes0377 Error lt rc gt parsing vpd Using defaults DESCRIPTION Could not parse the VPD data so the driver is using the default values DATA None ACTION None required elx_mes0381 Error lt rc gt during queue setup DESCRIPTION Could not set up all the queues that driver requires to exchange IOs with the adapter DATA None ACTION Reload the driver elx_mes0382 READ_SPARAM command failed status lt issue status gt mbxStatus lt mailbox status gt DESCRIPTION The READ_SPARAM mailbox command has failed during initialization The adapter has been set to error state DATA None ACTION Take a dump with hbacmd and then try reloading the driver elx_mes0384 There is pending active mailbox cmd DESCRIPTION The mailbox commands have overlapped This command should have been added to the mailbox queue DATA None ACTION None required elx_mes0385 rc should have been MBX_BUSY DESCRIPTION The completion handler for REG_LOGIN detected the IMMED_UNREG flag and tried to issue the unreg_login command from an interrupt level The mailbox status should still be busy DATA None Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 35
68. g Section Page 48 gt G EMULEX elx_mes1412 Failed to set up driver resource DESCRIPTION Initialization failed to set up driver resource DATA None ACTION None required elx_mes1413 Failed to initialize iocb list DESCRIPTION Initialization failed to initialize the IOCB list DATA None ACTION None required elx_mes1414 Failed to set up driver resource DESCRIPTION Initialization failed to set up driver resource DATA None ACTION None required elx_mes1415 Failed to create scsi host DESCRIPTION Initialization failed to create SCSI host DATA None ACTION None required elx_mes1416 Failed to allocate sysfs attr DESCRIPTION Initialization failed to sysfs attribute DATA None ACTION None required elx_mes1418 Invalid HBA PCl device group lt dev_grp gt DESCRIPTION Invalid adapter PCl device group detected DATA None ACTION None required elx_mes1419 Invalid HBA PCl device group lt dev_grp gt DESCRIPTION Invalid adapter PCl device group detected DATA None ACTION None required elx_mes1420 Invalid HBA PCl device group lt dev_grp gt DESCRIPTION Invalid adapter PCl device group detected DATA None ACTION None required elx_mes1421 Failed to set up hba DESCRIPTION Initialization failed to set up the adapter DATA None ACTION None required elx_mes1422 HBA Unrecoverable error uerr_lo_reg lt ue lo gt uerr_hi_reg lt ue hi gt onlineO_reg lt OnlineO gt onli
69. g in standard operating mode the driver creates two interfaces one for each physical port The first and second interfaces are respectively named vmnicO and vmnici1 assuming there are no other network interfaces in your configuration The same applies to virtual NIC VNIC capable UCNAs when vNIC mode is disabled in the adapter BIOS for those boards that support vNIC When vNIC mode is enabled in the adapter BIOS and the UCNA is in NIC only mode the driver creates eight interfaces four for each physical port When vNIC mode is enabled in the adapter BIOS and the UCNA is in FCoE or iSCSI mode the driver creates only 6 NIC interfaces The other two interfaces are Drivers for VMware ESX ESXi 4 1 Installation Section Page 5 gt G EMULEX reserved for the FCoE and iSCSI The interfaces are labeled vmnicO through vmnic7 assuming there are no other network interfaces in your configuration All eight vNICs are fully functional and support the same feature set as a standard NIC The vNICs can also be linked to a virtual switch in the same way esxcfg nics 1 list recognized nics esxcfg vswitch 1 list available vswitches esxcfg vswitch a vSwitch0 create vSwitcho esxcfg vswitch A VMNetO vSwitch0 create virtual machine network VMNetO and add it to vSwitcho esxcfg vswitch L vmnicO vSwitcho link vmnicO to vSwitcho The only difference being that in vNIC mode each of the four vNICs tied to a physical port share the po
70. he best policy is to execute a vm support dump and push it to the VMware Emulex support staff Log messages are organized into logical groups based on code functionality within the Fibre Channel driver Each group consists of a block of 100 log message numbers Most groups require a single block of 100 message numbers however some groups INIT FCP require two blocks Table 5 the Message Log table shows the groups and defines the associated number ranges Table 5 Message Log Table rip cares From To Verbose Verbose Description Definition al LOG_ELS 0100 0199 0x1 ELS events LOG_DISCOVERY 0200 0299 0x2 Link discovery events LOG_SLI 0300 0399 0x800 SLI events LOG_MBOX 0300 0339 0x4 Mailbox events LOG_TEMP 0340 0347 0x100 Temperature sensor events LOG_INIT 0400 0599 0x8 Initialization events LOG_FCP 0700 0799 0x40 FCP traffic history Reserved 0800 0899 LOG_NODE 0900 0999 0x80 Node table events Reserved 1000 1099 Reserved 1100 1199 LOG_MISC 1200 1299 0x400 Miscellaneous and FCoE events LOG_FCoE LOG_LINK_EVENT 1300 1399 0x10 Link events Reserved 1400 1499 Reserved 1500 1599 LOG_LIBDFC 1600 1699 0x2000 IOCTL events LOG_VPORT 1800 1832 0x4000 NPIV events Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 21 gt G EMULEX Table 5 Message Log Table Continued LOG Message From To Verbose Verbose Description Ver
71. his file and add the following line for the OneConnect driver interface ethernetO features 15 4 To enable vmxnet emulation add the following line for the OneConnect driver interface ethernet0O virtualDev vmxnet 5 Restart the VMs Steps 1 through 5 assume that eth0 is the interface added to the VM from the OneConnect Network Note The use of vmx NIC emulation requires VMware Tools to be installed in the guest operating systems For information on installing VMware Tools in a Linux or Windows guest operating system refer to the VMware ESX Server 4 1 documentation Configuring a Virtual Switch to Use Jumbo Frames To use jumbo frames you must increase the MTU Maximum Transmission Unit size in the vSwitch and also in the guest operating system For the best performance set the MTU to the maximum supported by the OneConnect driver which is 9000 bytes This requires the MTU to be changed in the virtual switch as well as the guest operating systems Note You can configure TSO TCP Segmentation Offload and jumbo frames using the vSphere client Setting the MTU Size in the vSwitch To change the MTU to 9000 run the following command for each switch in the console operating system esxcfg vswitch vSwitch lt N gt m 9000 where lt N gt is the number of the switch Setting the MTU Size for a Linux Guest Operating System To set the MTU of the OneConnect driver interface in each Linux Guest operating system to 9000
72. hooting Section Page 39 gt S EMULEX ACTION This error could indicate a hardware or firmware problem If the problem persists report the error to Technical Support elx_mes0446 Adapter failed to init moxCmd lt mbxCommand gt CFG_RING mbxStatus lt mbxStatus gt ring lt num gt DESCRIPTION Adapter initialization failed when issuing a CFG_RING mailbox command DATA None ACTION This error could indicate a hardware or firmware problem If the problem persists report the error to Technical Support elx_mes0448 Adapter failed to init moxCmd lt mbxCommand gt READ_SPARM mbxStatus lt mbxStatus gt DESCRIPTION Adapter initialization failed when issuing a READ_SPARM mailbox command DATA None ACTION This error could indicate a hardware or firmware problem If the problem persists report the error to Technical Support elx_mes0449 Phys attribute Instance Error Defaulting to lpfc_ attr to d Allowed range is min max DESCRIPTION A physical device attribute has an out of range value The driver is correcting it DATA 1 value written 2 minimum value 3 maximum value ACTION Write the default value elx_mes0450 Ipfc_ attr attribute cannot be set to d allowed range is Yomin max DESCRIPTION Sysfs attribute value written exceeds attribute range DATA 1 attribute name 2 value written 3 minimum value 3 maximum value ACTION Write a value within the supported range elx_mes0453 Adapter failed
73. in use FCF x x DESCRIPTION The driver has found a new FCF record that matches the properties of the current FCF Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 70 gt G EMULEX record but the record instance numbers don t match DATA None ACTION None required The driver is in its FCF discovery phase and is trying to recover a match to its in use FCF NIC Troubleshooting The following section includes NIC troubleshooting information Be sure to check the readme txt file located on CD1 for other troubleshooting issues The following table provides ESX Server NIC troubleshooting information for the OneConnect UCNA Table 6 ESX Server NIC Problem Answer Solution 1 When there is a great deal of network traffic in some VMs a few VMs appear to have lost network connectivity 2 A lot of alloc_skb failed messages appear in the log file proc vmware log This could be due to low configured value for netPktHeapMaxSize Try increasing it to a higher value To read the current value run esxcfg advcfg j netPktHeapMaxSize A value of 0 indicates default 64MB To increase the size to for example 128 MB run esxcfg advcfg k 128 netPktHeapMaxSize netPktHeapMaxSize can also be configured through VI Client using Configuration gt Advanced Settings gt VMKernel After configuring the size reboot the system Unable to ping from one VM to another VM When inserting o
74. ired elx_mes2610 HBA FCF index goes beyond driver s resource dimension DESCRIPTION During updating the round robin FCF bmask the FCF index goes beyond the driver s internal resource dimension DATA None ACTION None required elx_mes2611 UNREG_FCEFI issue mbox failed DESCRIPTION Could not issue the UNREG_FCFI mailbox command DATA None ACTION None required elx_mes2619 Config region 23 has bad signature DESCRIPTION Configuration region 23 has an invalid signature DATA None ACTION None required elx_mes2620 Config region 23 has bad version DESCRIPTION Configuration region 23 has an invalid version Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 65 gt E EMULEX DATA None ACTION None required elx_mes2621 Failed to allocate mbox for query firmware config cmd DESCRIPTION Failed to allocate mailbox memory DATA None ACTION None required elx_mes2622 Query Firmware Config failed mbx status lt rc gt status lt shdr_status gt add_status lt shdr_add_status gt DESCRIPTION Could not read the firmware configuration DATA None ACTION None required elx_mes2623 FCoE Function not supported by firmware Function mode lt function_mode gt DESCRIPTION FCoE is not supported by this firmware DATA None ACTION Use the OneCommand Manager application to update to the latest firmware elx_mes2707 Ring lt Ring gt handler Failed to allocate iocb Retl lt fh_rctl gt Type l
75. kSpeed 4 alpa_map 0 ACTION None required elx_mes1308 Menlo Maint Mode Link up Event x x rcvd Data x x x x x x DESCRIPTION Link up notification in Menlo maintenance mode DATA 1 fe_eventTag 2 port_state 3 VPort fc_flag ACTION None required elx_mes1309 Link Down Event x x received Data x x x x x x DESCRIPTION The port generated a link down event to the host DATA 1 fe_eventTag 2 port_state 3 VPort fc_flag ACTION None required elx_mes1310 Link Up Event npiv not supported in loop topology DESCRIPTION Loop topologies are not supported when NPIV is enabled DATA None ACTION Put link into Fabric mode Reserved 1400 1499 elx_mes1400 Failed to initialize sgl list DESCRIPTION Failed to initialize SGL list during initialization DATA None ACTION Reboot the server If the issue persist contact technical support elx_mes1401 Failed to enable pci device DESCRIPTION Failed to enable PCI device during initialization DATA None ACTION Reboot the server If the issue persist contact technical support elx_mes1402 Failed to set up pci memory space DESCRIPTION PCI initialization failed DATA None ACTION Reboot the server If the issue persist contact technical support Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 47 J gt G EMULEX elx_mes1403 Failed to set up driver resource DESCRIPTION Driver resource initialization failed DATA None ACTION None re
76. latform and as an offline bundle in ESXi platforms The offline bundle is the VMWare s recommended way for software upgrades for the VMWare platforms For more information about the ESX Patch Management activities please refer to the below documents http Awww vmware com pdf vsphere4 r41 vsp_41_esxupdate pdf Adding a Range of Hosts You can find the TCP IP accessed manageable hosts by searching a range of IP addresses The Add Range of TCP IP Hosts dialog box enables you to build the initial list of TCP IP accessed manageable hosts Not available in read only mode Note The ranges of IP addresses are only scanned each time you open the Add Remote TCP IP Hosts dialog box and click Start Discovery The ranges are NOT automatically scanned by the discovery server during its discovery cycles Drivers for VMware ESX ESXi 4 1 CIM Section Page 15 gt G EMULEX Add Range of TCP IP Hosts xi r New IP Address Range Range sat i0 fase zs fias Range End fio ps o faze rasp Current IP Address Ranges Eei Saye Ranges to File e Total addresses to search 0 Unscanned addresses 0 New hosts found 0 Discovery Status Idle Figure 1 Add Range of TCP IP Hosts dialog box The only prerequisite for this procedure is that the OneCommand Manager application must be installed on all remote hosts To add a range of remote hosts 1 From the Discovery menu sele
77. lx_mes1832 No pending MBOX command to handle DESCRIPTION DATA None ACTION Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 54 gt E EMULEX ELS Events 1833 2800 elx_mes1835 Vport discovery quiesce failed state lt port_state gt fc_flags lt fc_flag gt wait msecs lt jiffies_to_msecs jiffies start_time gt DESCRIPTION Could not pause discovery on this VPort DATA None ACTION None required elx_mes1836 Could not issue unreg_login all_rpis status lt rc gt DESCRIPTION The unreg_login cannot be issued DATA None ACTION None required elx_mes1837 vport_delete failed Cannot delete static vport DESCRIPTION Static VPorts cannot be deleted DATA None ACTION None required elx_mes1838 Failed to INIT_VPI on vpi lt vpi gt status lt rc gt DESCRIPTION Failed to INIT_VPI DATA None ACTION None required elx_mes2000 Failed to allocate mbox for READ FCF cmd DESCRIPTION Failed to allocate mbox for READ_FCF command DATA None ACTION None required elx_mes2001 Unable to allocate memory for issuing SLI CONFIG_SPECIAL mailbox command DESCRIPTION Unable to allocate memory for issuing the SLI_CONFIG_SPECIAL mailbox command DATA None ACTION None required elx_mes2002 Error Could not grow rpi count DESCRIPTION An error occurred because the RPI count could not be increased DATA None ACTION None required elx_mes2005 Unable to deregister pages from HBA lt rc gt
78. ment of adapters in the VMware ESX ESxXi hosts through the CIM interface is enabled only for the OneCommand Manager applications running on the Windows environment Installing the Emulex CIM Provider To install the Emulex CIM provider use the esxupdate utility provided by VMware in ESX and ESXi platforms For esxupdate procedure refer to esxupdate Procedure on page 5 for more details CIM Provider The Emulex CIM Provider is available as a core kit rom in ESX platform and as an offline bundle in ESXi platforms To install the CIM Provider core kit installation which is only for ESX execute the following command in the command prompt of the ESX host rpm ivh lt CIM Provider rpm gt where lt CIM Provider rpm gt is the released CIM Provider rpm For example elxocmcore esx40 5 1 32 2 1 x86 64 rpm To install the CIM Provider offline bundle installation which is only for ESXi execute the following command in the command prompt of the ESXi host tesxupdate maintenancemode nosigcheck update bundle lt offline bundle zip gt where lt offline bundle zip gt is the offline bundle for the Emulex CIM Provider For example ELX ESX 4 1 0 emulex cim provider 3 2 16 1 offline bundle 311859 zip For more information about the ESX Patch Management activities please refer to the below documents http Awww vmware com pdf vsphere4 r40 vsp_40_esxupdate pdf http Awww vmware com pdf vsphere4 r41 vsp_41_esxu
79. mmand DATA 1 ulpCommand ACTION Software driver error If this problem persists report these errors to Technical Support elx_mes1808 Create VPORT failed NPIV is not enabled SLImode lt mode gt DESCRIPTION The driver failed to create a port because the adapter was in wrong mode or was not capable of NPIV DATA 1 sli_rev ACTION Load the driver with npiv enabled on an adapter that supports SLI 3 elx_mes1809 Create VPORT failed Max VPORTSs lt vpi gt exceeded DESCRIPTION The driver failed to create a port because the maximum number of port supported by the driver is exceeded DATA 1 max_vpi ACTION No Action The driver can not create any more VPorts elx_mes1810 Create VPORT failed Cannot get instance number DESCRIPTION The driver failed to allocate resources for an adapter and could not assign an instance number DATA None ACTION Software driver error If this problem persists report these errors to Technical Support elx_mes1811 Create VPORT failed vpi x lt vpi gt DESCRIPTION The driver failed to create a port and had to eliminate all its resources DATA 1 vpi ACTION Software driver error If this problem persists report these errors to Technical Support elx_mes1812 vport_delete failed Cannot delete physical host DESCRIPTION An attempt to delete a port failed because it was to delete a physical port and not a virtual port Only VPorts on physical ports can be deleted on an NPIV system
80. mory size lt reqlen gt DESCRIPTION Could not get the memory required for the number of XRIs that are attempting to be posted DATA None ACTION None required elx_mes2562 No room left for SCSI XRI allocation max_xri lt sli4_hba max_cfg_param max_xri gt els_xri lt els_xri_cnt gt DESCRIPTION The number of allocated XRIs has reached the max_xri value DATA None ACTION None required elx_mes2563 Failed to allocate memory for SCSI XRI management array of size lt sli4_hba scsi_xri_max gt DESCRIPTION Initialization could not allocate memory to hold the XRIs DATA None ACTION None required elx_mes2564 POST_SGL_BLOCK mailbox command failed status lt shdr_status gt add_status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION The list of XRI SGEs failed to be registered with the firmware DATA None ACTION None required elx_mes2566 Failed to allocate connection table entry DESCRIPTION Failed to allocate connection table entry DATA None ACTION None required elx_mes2567 Config region 23 has bad signature DESCRIPTION Configuration region 23 has an invalid signature DATA None ACTION None required elx_mes2568 Config region 23 has bad version DESCRIPTION Configuration region 23 has an invalid version DATA None ACTION None required elx_mes2570 Failed to read FCoE parameters DESCRIPTION Failed to read the FCoE parameters DATA None ACTION None required elx_mes2572
81. ne1_reg lt Online1 gt DESCRIPTION The adapter has notified the driver that it has encountered an unrecoverable error Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 49 gt G EMULEX DATA None ACTION A dump from the OneCommand Manager application should be taken Then the driver should be unloaded and reloaded elx_mes1423 HBA Unrecoverable error uerr_lo_reg lt ue lo gt uerr_hi_reg lt ue hi gt onlineO_reg lt OnlineO gt online1_reg lt Online1 gt DESCRIPTION The adapter has notified the driver that it has encountered an unrecoverable error DATA None ACTION A dump from the OneCommand Manager application should be taken Then unload and reload the driver elx_mes1424 Invalid PCI device group lt pci_dev_grp gt DESCRIPTION Invalid adapter PCl device group detected DATA None ACTION None required elx_mes1425 Invalid PCI device group lt pci_dev_grp gt DESCRIPTION Invalid adapter PCl device group detected DATA None ACTION None required elx_mes1426 Invalid PCI device group lt pci_dev_grp gt DESCRIPTION Invalid adapter PCl device group detected DATA None ACTION None required elx_mes1427 Invalid PCI device group lt pci_dev_grp gt DESCRIPTION Invalid adapter PCl device group detected DATA None ACTION None required elx_mes1428 Invalid PCI device group lt pci_dev_grp gt DESCRIPTION Invalid adapter PCl device group detected DATA None ACTION None r
82. nfigure several discovery server parameters You can define when to start the discovery server when to refresh FC and TCP IP accessed discoveries and when to remove previously discovered adapters that are no longer being discovered You can also define default CIM credentials such as the protocol user name port number password and name space For TCP IP management you can specify an IP port number change an IP port number and enable a port for TCP IP management Note The number of addresses does not correspond directly to the number of hosts added to the discovery tree For example some of the addresses discovered may be for hosts that have already been discovered over FC However new adapters can be discovered on those hosts that were not discovered over FC Also a host can have more than one IP address assigned to it If multiple IP addresses for a host are discovered during the search the host will be added to the discovery tree only once If the same host name appears for more than one host the adapters of all these hosts will be displayed in the OneCommand Manager as a single host entry OneCommand Manager Discovery Settings xi Discovery Server Stertup Select when to start the discovery server prennennemnennennenng C At system boot yhen this utility starts slower Undiscovered Adapter Expiration Select when to remove a previously discovered adapter that is no longer being discovered I Never remove
83. nnect failed UCNA or the allocation of some resource for initializing the driver failed In most cases this message is accompanied by another more specific error message Try rebooting the system after a power cycling If the problem persists this could indicate a hardware problem or corrupted firmware Warning Using INTx interrupts Net The driver could not allocate MSIx vector for Queues feature are disabled interrupt The driver may continue to work but the performance may be impacted Warning WARNING Found a BE2 card in OneConnect is a x8 Gen2 PCl e device For Gen lt D gt x lt D gt PCI e slot Should best performance OneConnect should be be in Gen 2 x8 slot for best installed in a Gen2 PCl e slot 8 or 16 performance channels wide The driver prints this warning if it finds the device in a slower or narrower PCl e slot The device continues to work with lesser performance Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 72 Table 7 ESX Server NIC Event Log Entries Continued gt G EMULEX Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Severity Message Displayed Description Warning Command to get pause frame set The firmware command to get PAUSE tings failed settings failed Warning Command to set pause frame set The firmware command to change PAUSE tings failed settings failed Warning Command to apply MAC add
84. o install the OneCommand Manager agent Log in as root 2 Copy the elxocmcore esx41 lt kit version gt x86 64 rpmfile to a directory on the install machine 3 CD to the directory to which you copied the RPM file Install the RPM file Type rpm U elxocmcore esx41 lt kit version gt x86 64 rpm For example rpm U elxocmcore esx41 5 0 43 6 1 x86 64 rpm The RPM file contents are installed in usr sbin ocmanager The CLI utility is also located in this directory Uninstalling the FC FCoE Utilities Follow these instructions to uninstall the Emulex configuration utilities To uninstall the OneCommand Manager agent Log in as root 2 Type rpm qa grep elx to verify that this kit is installed This command should list elxocmcore esx41 lt kit version gt for the current release 3 Type rpm e elxocmcore esx41 lt kit version gt Installing the NIC Driver Introduction This part of the guide provides installation and troubleshooting information for the Emulex OneConnect UCNA Before using this product you need a working knowledge of the NIC network interface card TOE TCP offload engine technology and the fundamentals of network storage devices Installing the NIC Driver and Management Software The Emulex NIC driver for VMware ESX 4 1 is available through the VMware support site The VMware support website also includes a documentation link to the correct installation procedures for this driver Follo
85. or lt ulpStatus gt lt un ulpWord 4 gt DESCRIPTION The nameServer CT request failed DATA None ACTION Check the switch and port configurations elx_mes0271 Illegal State Transition node lt nlp_DID gt event lt evt gt state lt nlp_state gt Data lt nlp_rpi gt lt nlp_flag gt DESCRIPTION The current node state does not have a handler for this event DATA 1 nlp_rpi 2 nlp_flag ACTION Verify that all targets are still visible to the SCSI mid layer elx_mes0272 Illegal State Transition node lt nlp_DID gt event lt evt gt state lt nlp_state gt Data lt nlp_rpi gt lt nlp_flag gt DESCRIPTION The driver is completing a PLOGI but do not have the rcv_plogi flag set DATA 1 nlp_rpi 2 nlp_flag ACTION Verify that all targets are still visible to the SCSI mid layer elx_mes0273 Unexpected discovery timeout vport State lt port_state gt DESCRIPTION The discovery process has timed out DATA None ACTION Ensure all targets are visible elx_mes0282 did x x ndlp x pusgmap x x refcnt d ndlp gt nlp_DID void ndlp lpfc_init c ndlp gt nlp_usg_map DESCRIPTION Driver clean up has found a node that is still on the node list during driver unload or PCI hotplug removal DATA None ACTION None required elx_mes0283 Failed to allocate mbox cmd memory DESCRIPTION Mailbox allocation error DATA None Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 28 gt G EMULEX ACTION
86. or VMware ESX ESXi 4 1 Troubleshooting Section Page 53 J gt G EMULEX elx_mes1825 Vport Created DESCRIPTION This message is displayed to indicate that a port was created in the system It is displayed at this level to ensure it is always appears at all log levels DATA None ACTION No action informational elx_mes1826 Vport Disabled DESCRIPTION The port had to be disabled in the system DATA None ACTION No action informational elx_mes1827 Vport Enabled DESCRIPTION The port had to be enabled after possible recovery from some errors DATA None ACTION No action informational elx_mes1828 Vport Deleted DESCRIPTION A VPort was deleted DATA None ACTION No action informational elx_mes1829 CT command failed to delete objects on fabric DESCRIPTION A command issued to the fabric to delete an associated resource for an object such as for a port failed DATA None ACTION Software driver error If this problem persists report these errors to Technical Support elx_mes1830 Signal aborted moxCmd lt command gt DESCRIPTION A pending mailbox command was aborted because the thread received a signal DATA None ACTION The command is retried elx_mes1831 Create VPORT Interrupted DESCRIPTION The port creation process was unexpectedly interrupted at a critical time and the operation was unsuccessful DATA None ACTION The process was interrupted while creating a VPort Retry the command e
87. ot issue DESCRIPTION The mailbox command does not have all of the fields set correctly DATA 1 sli4_mbx_opcode 2 sli_flag 3 flag ACTION None required elx_mes2544 Mailbox command lt vpi gt lt mbxCommand gt cannot issue DESCRIPTION The adapter cannot be accessed on the PCI bus DATA 1 sli4_mbx_opcode 2 sli_flag 3 flag ACTION None required elx_mes2546 New FCF found index lt index gt tag lt event_tag gt DESCRIPTION A new FCF has been found DATA None ACTION None required elx_mes2547 Read FCF record failed DESCRIPTION Could not read the FCF record from the firmware DATA None ACTION None required elx_mes2548 FCF Table full count lt count gt tag lt event_tag gt DESCRIPTION The FCF table is full DATA None ACTION None required elx_mes2549 FCF disconnected from network index lt index gt tag lt event_tag gt DESCRIPTION The FCF has disconnected from the network DATA None ACTION None required elx_mes2550 UNREG_FCFI mbxStatus error lt u mb mbxStatus gt HBA state lt port_state gt DESCRIPTION The unregistered FCFI has failed DATA None ACTION None required Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 61 gt E EMULEX elx_mes2551 UNREG_FCFI mbox allocation failed HBA state lt port_state gt DESCRIPTION The allocation for the UNREG_FCFI mailbox command has failed DATA None ACTION None required elx_mes2552 UNREG_FCFI issue m
88. pdate pdf Once the above steps are complete the system has to be rebooted for the changes to be effective Drivers for VMware ESX ESxXi 4 1 Configuration Section Page 11 gt G EMULEX After the reboot is completed verify that the drivers and the CIM Provider are correctly installed by typing the following commands esxupdate vib view query grep be2 esxupdate vib view query grep lpfc esxupdate vib view query grep emu For additional configuration information refer to Managing Devices using CIM on page 15 for more details Working with VPorts Virtual Ports Creating Deleting and Displaying VPorts The Emulex driver for VMware supports NPIV by default The only management API for creating and deleting a VPort and creating an NPIV enabled virtual machine comes from ESX VPorts in the driver discover the fabric just like physical ports do and are subject to the same SAN delays As the number of VPorts increases the amount of time it takes to complete remote port discovery increases This is because the VPorts are created sequentially and each VPort executes discovery synchronously If your NPIV enabled virtual machines power on automatically powering on could take longer than usual This is normal for NPIV virtual machines Note Ensure you are using the latest recommended firmware for VPort functionality Check the Emulex website for the latest firmware Note Loop devices and NPIV are not supported on
89. pl DESCRIPTION A unknown mailbox command completed DATA None ACTION This error could indicate a software driver firmware or hardware problem Report these errors to Technical Support elx_mes0324 Config port initialization error moxCmd lt mbxCommand gt READ_NVPARM mbxStatus lt mbxStatus gt DESCRIPTION A read nvparams mailbox command failed during port configuration DATA None ACTION This error could indicate a software driver firmware or hardware problem Report these errors to Technical Support Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 31 J gt G EMULEX elx_mes0330 IOCB wake NOT set DESCRIPTION The completion handler associated with the IOCB was never called DATA 1 timeout 2 timeleft jiffies ACTION This error could indicate a software driver firmware or hardware problem If the problem persists report the error to Technical Support elx_mes0334 Unknown IOCB command DESCRIPTION Received an unknown IOCB command completion DATA 1 type 2 ulpCommand 3 ulpStatus 4 ulploTag 5 ulpContext ACTION This error could indicate a software driver or firmware problem If these problems persist report these errors to Technical Support elx_mes0335 Unknown IOCB command DESCRIPTION Received an unknown IOCB command completion DATA 1 ulpCommand 2 ulpStatus 3 ulploTag 4 ulpContext ACTION This error could indicate a software driver or firmware problem
90. ported features Supports the following protocols e NIC e Fibre Channel FC Fibre Channel over Ethernet FCoE e Fibre Channel initiator mode SCSI FCP e FCP 2 e SNIA CTP compliant SMI S 1 1 Provider e Supports the following topologies e FC AL Fibre Channel Arbitrated Loop e Point to point e Fabric with auto topology negotiation e Supports FC in band management e Supports 1 2 4 8 and 10 Gb s capable adapters with auto rate negotiation 1 Gb s is not supported on 8 Gb s adapters e Tested with up to sixteen adapter ports e Support for Common HBA API e Batch firmware download capability e PCI hot plug support vendor specific e VPD Vital Product Data support e NPIV N_Port ID Virtualization support e Adapter personality change support New Features in this Release e Supports the Emulex OneConnect OCe 11102 UNCAs e Supports iSCSI e Supports iSCSI hardware offload Drivers for VMware ESX ESXi 4 1 Installation Section Page 1 gt G EMULEX Prerequisites e There are no prerequisites at this time Compatibility e Fora list of adapters that are compatible with this driver see the driver Downloads page on the Emulex website For compatible firmware versions see the Downloads page for the specific adapter Things to Know Before You Download e The driver is matched to a particular version of the Emulex Core Kit the adapter firmware and the NIC driver Refer to the Downloads page on the Emulex websi
91. problem persists report these errors to Technical Support elx_mes0423 Vport Attribute Instance Error Defaulting lpfc_ attr to d error value d allowed range is min max DESCRIPTION A Vport attribute was set out of range The driver reset the parameter to its default DATA None ACTION Set the module parameter between the minimum and maximum values elx_mes0424 Vport Attribute Count Exceeded Max d Actual d DESCRIPTION The total number of Vport attributes set exceeded the max allowed DATA None ACTION Reduce the number set attributes below max elx_mes0425 Ipfc_ attr attribute cannot be set to d allowed range is min max DESCRIPTION Driver attribute lpfc_ attr was defined with an out of range value DATA None ACTION Set the parameter between the minimum and maximum value elx_mes0427 Cannot re enable interrupt after slot reset DESCRIPTION The driver was not able to enable the interrupt after an adapter reset DATA None ACTION Unload and reload the driver elx_mes0430 PM resume Failed to enable interrupt DESCRIPTION The driver s power management resume function could not enable the interrupt DATA None ACTION Perform another PM suspend and resume or adapter reset elx_mes0431 Failed to enable interrupt DESCRIPTION The driver failed to start the interrupt DATA None ACTION Unload and reload the driver elx_mes0433 Wakeup on signal rc lt rc gt DESCRIPTION A signal other than th
92. pt to force an adapter to run in SLI mode 3 if that adapter does not support SLI 3 mode Adapters that do not support SLI 3 are configured to run in SLI 2 mode Nevertheless it is recommended to use the auto setting 0 elx_mes1821 Create VPORT failed Invalid WWN format DESCRIPTION The port could not be created due to an invalid WWNN or WWPN format DATA None ACTION Provide a valid WWN when creating VPorts elx_mes1822 Invalid lt name gt lt XX XX XX XX XX XX XX XX gt DESCRIPTION An invalid WWN was used when creating a VPort DATA 1 type_name 2 wwn 1 3 wwn 3 3 wwn 5 4 wwn 7 ACTION When creating a VPort you must furnish a valid WWN elx_mes1823 Create VPORT failed Duplicate WWN on HBA DESCRIPTION The port could not be created because it would duplicate an existing WWNN adapter address The resources for the port had to be discarded DATA None ACTION Provide a WWN that is unique elx_mes1824 NPIV enabled Override Ipfc_sli_mode parameter lt mode gt to auto 0 DESCRIPTION The Ipfc_enable_npiv and Ipfc_sli_mode driver parameter settings conflict The adapter must be configured for SLI 3 mode to support NPIV DATA 1 lpfc_sli_mode ACTION This is an informational message that indicates that the lpfc_enable_npiv and Ipfc_sli_mod parameter settings are not compatible Resolve the parameter conflict by setting the SLI mode to 0 or 3 or if SLI 2 mode is required then disable NPIV Drivers f
93. quired elx_mes1404 Failed to set up driver resource DESCRIPTION Driver resource initialization failed DATA None ACTION None required elx_mes1405 Failed to initialize iocb list DESCRIPTION IOCB initialization failed DATA None ACTION None required elx_mes1406 Failed to set up driver resource DESCRIPTION Initialization failed to set up driver resource DATA None ACTION None required elx_mes1407 Failed to create scsi host DESCRIPTION Initialization failed to create SCSI host DATA None ACTION None required elx_mes1408 Failure HBA POST Status sta_reg lt status reg gt perr lt port error gt sfi lt sfi reg gt nip lt nip reg gt ipc lt ipc reg gt xrom lt xrom gt dl lt dl reg gt pstatus lt port status gt DESCRIPTION The adapter s power on self test has failed DATA None ACTION Make sure the adapter firmware is up to date Contact the technical support if the issue persists after system reboot elx_mes1409 Failed to enable pci device DESCRIPTION Failed to enable PCI device during initialization DATA None ACTION None required elx_mes1410 Failed to set up pci memory space DESCRIPTION Initialization failed to set up PCI memory space DATA None ACTION None required elx_mes1411 Failed to set up driver resource DESCRIPTION Initialization failed to set up driver resource DATA None ACTION None required Drivers for VMware ESX ESXi 4 1 Troubleshootin
94. r removing a 1Gb SFP RJ45 module on a OneConnect adapter without RJ45 copper cables attached the operating system indicates link up or down status When inserting or removing copper cables attached to a switch to the module link up or down events are not reported to the operating system OneConnect driver creates two vmnic interfaces one for each port If these interfaces are configured as uplinks in two separate vSwitches the VMs in each of these switches are in separate networks with no network path between them Thus pinging between the VMs in the two groups fails If you want all these VMs in the same network configure them as teaming uplinks to one vSwitch option Each of the vmnics vmnic1 to vmnic16 must be configured in a separate vSwitch In this configuration there is no network path between the vSwitches and pinging between these VMs does not work There is no solution to this issue as the link status is not reported due to a PHY limitation on the card Flow control setting is not stored per port after rebooting the system With flow control there is no persistence across reboot It always starts with both RX and TX on For persistence run a config command from an RC file at reboot Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 71 gt G EMULEX NIC Event Error Logging Retrieving ESX Server NIC Error Log Codes For ESX Server systems the NIC OneConnect be2net driver genera
95. ress Driver could not set the MAC address filter on filter failed the hardware The device continues to work There may be an impact on the performance Warning Command to delete MAC address The firmware command to delete a MAC filter failed address filter failed The device should continue to work Warning Unable to get Firmware Version The command to get firmware revision number failed The version number is not shown The device must continue to work Warning Did not receive completions for While unloading the driver some outstanding all TX requests transmit requests are found This is an indication that the hardware is not functioning properly Warning Failed to register char device Could not create the char device used for certain management functions The driver must still work You may not be able to use HBACMD to interact with the device Warning alloc_skb failed Try increas Could not allocate skb structure to send a ing netPktHeapMaxSize frame received from the network to the OS Transient failure can be ignored Persistent message points to insufficient memory allocated for network heap For example to increase the heap size to 128MB run esxcfg advcfg k 128 netPk tHeapMaxSize Warning Invalid MTU requested Must be Invalid MTU size in MTU configuration ioctl between 64 and 8174 bytes The MTU is not changed Warning Invalid vlan priority labeled Request to set VLAN priority tag was made Must be 0 7 with invalid v
96. ress for the OneConnect UCNA Emulex recommends that you set up a reservation A reservation assigns a specific IP address based on the MAC address of the OneConnect UCNA If you do not reserve an IP Address through DHCP then you must set the lease length for the OneConnect UCNA IP address to unlimited to prevent the IP address lease from expiring Drivers for VMware ESX ESXi 4 1 Appendix Page 75
97. round robin FCF failover bmask DESCRIPTION The driver failed to allocate memory for the round robin FCF failover bmask DATA None ACTION Make sure system has enough kernel memory might need to reload the driver after memory problem resolved elx_mes2762 HBA reported FCF index go beyond driver bmask dimension DESCRIPTION Adapter reports an FCF record index goes beyond the driver s internal resource dimension for the bmask DATA None ACTION Inform Emulex about this elx_mes2763 Failed to allocate mempool for read FCF record mbox command DESCRIPTION The driver failed to allocate memory from the mempool for issuing an FCF read mailbox command during the round robin FCF failover DATA None ACTION None required elx_mes2772 Failed to issue FCF rediscovery mailbox command due to dead FCF DESCRIPTION The driver wanted to send an FCF rediscovery mailbox command to start fast FCF failover due to a dead FCF asynchronous event but failed to send the mailbox command due to state of the adapter DATA None ACTION None required elx_mes2774 Failed to issue FCF rediscovery mailbox command due to CVL DESCRIPTION The driver wanted to send an FCF rediscovery mailbox command to start fast FCF failover due to a Clear Virtual Link asynchronous event but failed to send the mailbox command due to state of the adapter DATA None ACTION None required Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 68 gt
98. rt s 10GbE bandwidth Configuring VLANs The OneCommand Manager application supports VLAN filtering in the hardware To configure VLANs on a OneConnect interface create the vSwitch with the required VLAN ID and use this interface as an adapter to this vSwitch A native VLAN can also be configured in the guest operating system in VGT Virtual Guest Tagging mode For example using vconfig in the Linux guest operating system Configuring Network Heap Size in ESX Server The amount of memory allocated by default for a network heap depends on the amount of memory configured in the system The ESX Server network stack allocates a minimum of 64MB to the network heap to handle network data More memory is allocated to the network heap if the system is configured with more memory If the network load requires more than 64MB of memory the OneConnect driver cannot allocate it When this happens the driver logs messages in the file proc vmware log indicating that the alloc_skb call failed This impacts network performance considerably e To read the current size of the network heap run esxcfg advcfg j netPktHeapMaxSize netPktHeapMaxSize 0 e Ifthe default value of 64MB is in effect this command shows the size as 0 If any other value is in effect the command prints that value e For example to set the heap size to 128MB run the command esxcfg advcfg k 128 netPktHeapMaxSize esxcfg advcfg j netPktHeapMaxSize netPktHeapM
99. rt number can be changed The default is 23333 Refer to the VMware Server Configuration Guide for more details on how to configure the ESX firewall Temporary FC FCoE Configuration Methods Using Native ESX Tools There are four ways to configure the driver parameters e Permanent global e Permanent per adapter global per adapter e Temporary e Temporary Note The OneCommand Manager application version 5 1 also supports all four ways to configure driver parameters This is the preferred method of setting configuration parameters Refer to the OneCommand Manager 5 1 User Manual for more information Permanent FC FCoE Configuration Methods Using Native ESX Tools Permanent configuration requires that the new values be saved in the ESX environment These changes are considered permanent because they stay in effect across system reboots To make changes that impact all adapters in the system global changes follow these steps See FC FCoE Driver Configuration Parameters on page 9 for parameter names and values Parameter values are in both hexadecimal and decimal Drivers for VMware ESX ESxXi 4 1 Configuration Section Page 7 gt G EMULEX 1 From the Console Operating System COS terminal window type esxcfg module s param value param2 value lt driver_name gt The lt driver_name gt is obtained from the vmkload_mod 1 call Look for the Ipfc prefix 2 Reboot the server Type re
100. s to load Contact Emulex Technical Support elx_mes0523 Failed setup of fast path EQ lt fcp_eqidx gt rc lt rc gt DESCRIPTION The fast path EQ setup failed DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0526 Fast path FCP CQ lt fcp_cqidx gt not allocated DESCRIPTION The fast path FCP is not allocated DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0527 Failed setup of fast path FCP CQ lt fcp_cqidx gt rc lt rc gt DESCRIPTION The fast path FCP CQ setup failed DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0528 Mailbox CQ not allocated DESCRIPTION The mailbox CQ is not allocated DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0530 ELS CQ not allocated DESCRIPTION The ELS CQ is not allocated DATA None Drivers for VMware ESX ESXi 4 1 Troubleshooting Section J gt G EMULEX Page 43 J gt G EMULEX ACTION The driver fails to load Contact Emulex Technical Support elx_mes0531 0531 Failed setup of slow path ELS CQ rc 0x x DESCRIPTION The ELS CQ is allocated but failed initial setup DATA 1 status ACTION The driver fails to load Contact Emulex Technical Support elx_mes0534 Fast path FCP WQ lt fcp_eqidx gt not allocated DESCRIPTION The fast path FCP WQ is not allocated DATA None ACTION The driver fails to loa
101. size of cmd ring DATA None ACTION This error could indicate a software driver firmware or hardware problem Report these errors to Technical Support elx_mes0317 iotag lt ulp_loTag gt is out of range max iotag lt max_iotag gt wd0 lt wd0 gt DESCRIPTION The loTag in the completed IOCB is out of range DATA None ACTION This error could indicate a software driver firmware or hardware problem Report these errors to Technical Support elx_mes0318 Failed to allocate IOTAG last IOTAG is lt last_allocated_iotag gt DESCRIPTION The driver cannot allocate an loTag Display the last value used DATA None ACTION This message indicates the adapter I O queue is full Typically this happens when heavy I O is running on a low end 3 digit adapter We suggest you upgrade to a higher end adapter elx_mes0319 READ_SPARAM mbxStatus error lt mbxStatus gt hba state lt hba_state gt DESCRIPTION The driver issued a READ_SPARAM mbox command to the adapter that failed DATA None ACTION This error could indicate a firmware or hardware problem Report these errors to Technical Support elx_mes0320 CLEAR_LA mbxStatus error lt mbxStatus gt hba state lt hba_state gt DESCRIPTION The driver issued a CLEAR_LA mbox command to the adapter that failed DATA None ACTION This error could indicate a firmware or hardware problem Report these errors to Technical Support elx_mes0323 Unknown Mailbox command lt mbxCommand gt Cm
102. t fh_type gt received DESCRIPTION Could not allocate an IOCB with which to associate this received frame DATA None ACTION None required elx_mes2718 Clear Virtual Link Received for VPI lt index gt tag lt event_tag gt DESCRIPTION A Clear virtual link was received from the Fabric for this VPI DATA None ACTION None required elx_mes2719 Invalid response length tgt lt TGT_ID gt lun lt LUN gt cmnd lt CMD gt rsplen lt RSPLEN gt DESCRIPTION The response length for this FCP command is not supported DATA None ACTION None required elx_mes2726 READ_FCF_RECORD Indicates empty FCF table DESCRIPTION The driver requested the firmware provide a list of FCF entries to connect to and the firmware responded that the FCF table is empty DATA None ACTION None required elx_mes2731 Cannot find fabric controller node DESCRIPTION Driver not able to find Fabric controller node in its data base DATA None ACTION None required Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 66 J gt G EMULEX elx_mes2732 Failed to issue INIT_VPI mailbox command DESCRIPTION The driver wanted to send a INIT_VPI mailbox command to initialize a VPort but failed to send the mailbox command due to state of the adapter DATA None ACTION None required elx_mes2746 Failed FCF rediscover mailbox command failure DESCRIPTION The adapter returned failure on FCF rediscover mailbox command DATA None ACT
103. te for details Known Issues e See the product release notes for the latest information Installing the FC FCoE Driver Introduction This part of the guide provides installation and troubleshooting information for the Emulex OneCommand Manager application using the FC FCoE interface Before using this product you need a working knowledge of FC FCoE TOE TCP Offload Engine technology and the fundamentals of network storage devices Installing the FC FCoE Driver and Management Software The Emulex driver is available through the VMware support site Refer to the VMware support website for further details See the OneCommand Manager Command Line User Interface Manual for instructions on how to install the application on the ESX COS Note Before installing the Emulex OneCommand Manager Kit you must install the Ipfc driver from the VMware website Note Before installing OneConnect network drivers and management software verify that the firmware version is correct If it is proceed with the installation If it is not update the firmware using the OneCommand Manager application and reboot your system before proceeding with the installation Uninstalling the FC FCoE Driver See the VMware Patch Download page for instructions Drivers for VMware ESX ESXi 4 1 Installation Section Page 2 gt G EMULEX Installing the FC FCoE Utilities Follow these instructions to install the OneCommand Manager application T
104. teeenaaaaaeeeeeeeeeeas 55 New Events eeii iiki ii k i e k e e 69 NIC Troubleshooting isecisceen asnan nna aana Aaa EASA AAAA EAEE AAAA EAE T ATAA N AE 71 NIC Event Error Logging sssssesssssrresssesssrrrnrsssesrstnnnntsssnnrnnnnnsssrnnnnnnnsseennnnn 72 Retrieving ESX Server NIC Error Log Codes cseceeeeeeeeeeeecneeeeeeees 72 ESX Server NIC Event Log Entries c ccceeeeeeeeeeceneeeeeeeeeeeseneneeeees 72 NIC Adapter Firmware Error ccccccecceeeeeeeeeeecnneeeeeeeeeeeeecaeeeeeeeeeteeenaaaes 74 CIM Provider Troubleshooting ccceeeeeeeeeeneeeeeeeeecneeeeeeeeeeeeeeeeccaeeeeeeeeeseneaaaaes 74 Appendix A DHCP Recommendations 22 cccceseeeeeeeeeeeeeeeeeeeeeeeeeees 75 Drivers for VMware ESX ESXi 4 1 User Manual Page v gt G EMULEX Installation Driver Information Supported Features e Supports the Emulex OneConnect family of UNCAs e Supports dynamic parameter setting using the OneCommand Manager application version 5 1 from Emulex It is a GUI and CLl based configuration utility used as part of a master kit enabling driver configuration including Supports the CIM interface through OneCommand Manager to manage ESX ESXi Out of band TCP IP remote SAN management capability Diagnostics loopback and diagnostic dump except for OneConnect adapters LUN logical unit number masking Virtual port support See the OneCommand Manager User Manual for a complete list of sup
105. tes error codes to the var log vmkernel log file The vmkernel log file is an ASCII text file and can be viewed and searched with a text editor such as vi The vmkernel log file is automatically rotated as it gets larger and the rotated log files are named vmkernel x where x is an integer To search the log file for error messages at the command prompt type cd var log less vmkernel For example you might see the following message Sep 9 19 48 04 esx server vmkernel WARNING Found a BE2 card in Gen 1 x8 PCI e slot Should be in Gen 2 x8 slot for best performance ESX Server NIC Event Log Entries The following is a list of ESX Server network event log error messages It includes the severity of the error the message displayed and the message description When reporting a problem with the OneConnect UCNA to Emulex check the message log proc vmware log and report any of these entries that may be present Note In the following table lt D gt lt DD gt or lt DDD gt in the Message Displayed column refers to decimal values that appear in the actual error messages Table 7 ESX Server NIC Event Log Entries Severity Message Displayed Description Error BladeEngine POST failed Power ON Self Test of the OneConnect UCNA failed This indicates either a hard ware or a firmware problem Try rebooting the system after a reset Error BladeEngine 2 initialization Either the initialization of the OneCo
106. the problem persists report the error to Technical Support elx_mes0439 Adapter failed to init moxCmd lt mbxCommand gt READ_REV mbxStatus lt mbxStatus gt DESCRIPTION Adapter initialization failed when issuing a READ_REV mailbox command DATA None ACTION This error could indicate a hardware or firmware problem If the problem persists report the error to Technical Support elx_mes0440 Adapter failed to init READ REV has missing revision information DESCRIPTION A firmware revision initialization error was detected DATA None ACTION This error could indicate a hardware or firmware problem Update the firmware If the problem persists report the error to Technical Support elx_mes0442 Adapter failed to init moxCmd lt mbxCommand gt CONFIG_PORT mbxStatus lt mbxStatus gt DESCRIPTION Adapter initialization failed when issuing a CONFIG_PORT mailbox command DATA 1 hbainit ACTION This error could indicate a hardware or firmware problem If the problem persists report the error to Technical Support elx_mes0443 Adapter failed to set maximum DMA length mbxStatus lt u mb mbxStatus gt DESCRIPTION Cannot set the maximum DMA length to reflect cfg_pci_max_read DATA None ACTION Set module parameter Ipfc_pci_max_read to 512 1024 2048 or 4096 elx_mes0445 Firmware initialization failed DESCRIPTION The driver was unable to initialize the hardware DATA None Drivers for VMware ESX ESXi 4 1 Troubles
107. the same port at the same time If you are running a loop topology and you create a VPort the VPort s link state is offline VMware ESX supports only fabric mode Note You can create VPorts only on 4 Gb s 8 Gb s and OneConnect adapters You cannot create VPorts on 1 Gb s or 2 Gb s adapters Note The OneCommand Manager application sees all VPorts created by the driver but the application has read only access to them NIC Network Driver Performance Tuning Network driver performance tuning improves performance of the network and the TCP Offload driver The OneConnect Universal Converged Network Adapter UCNA is an x8 Generation 2 Gen 2 or Gen2 PCI Express PCle device and requires substantial system memory bandwidth to support 10 Gb s data streams Using vmxnet Emulation Enabling Jumbo Frames and TSO The OneConnect UCNA supports jumbo frames and TSO both of which are necessary to achieve optimal performance with the OneConnect UCNA Also the use of the vmxnet NIC emulator can provide a significant performance boost These features are not enabled by default in ESX Server To enable these features 1 Log into the console operating system 2 For each guest operating system there is a vmx file in the path vmts volumes lt VM NAME gt VM NAME gt vmx where lt VM NAMES gt is the name of the VM Drivers for VMware ESX ESxXi 4 1 Configuration Section Page 12 gt G EMULEX 3 For each VM edit t
108. to init moxCmd lt mbxCommand gt READ_CONFIG mbxStatus lt mbxStatus gt DESCRIPTION Adapter initialization failed when issuing a READ_CONFIG mailbox command DATA None ACTION This error could indicate a hardware or firmware problem If the problem persists report the error to Technical Support elx_mes0454 Adapter failed to init moxCmd lt mbxCommand gt INIT_LINK mbxStatus lt mbxStatus gt DESCRIPTION Adapter initialization failed when issuing an INIT_LINK mailbox command DATA None ACTION This error could indicate a hardware or firmware problem If the problem persists report the error to Technical Support elx_mes0456 Adapter failed to issue ASYNCEVT_ENABLE mbox status lt rc gt DESCRIPTION The mailbox command to enable an asynchronous event notification failed DATA None ACTION Ensure the adapter firmware is current Reload the driver elx_mes0457 Adapter Hardware Error DESCRIPTION The driver received an interrupt indicating a possible hardware problem Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 40 gt G EMULEX Data 1 status 2 status1 3 status2 ACTION This error could indicate a hardware or firmware problem If the problem persists report the error to Technical Support elx_mes0459 Adapter heartbeat failure taking this port offline DESCRIPTION The Heartbeat mailbox command failed DATA None ACTION Ensure the adapter firmware is current Reload the driver elx
109. ver for the NIC ports of the UCNA adapters To install the NIC driver execute the following command in the command prompt of the ESX ESXi host tesxupdate maintenancemode nosigcheck update b lt BE NIC driver gt where lt BE NIC driver gt is the be NIC driver released as an iso For example vmware esx drivers net be2net 400 2 102 499 0 lvmw 2 17 249663 x86 64 iso iSCSI Driver The iSCSI driver is the driver for the iSCSI ports of the UCNA adapters To install the iSCSI driver execute the following command in the command prompt of the ESX ESXi host esxupdate maintenancemode nosigcheck update b lt BE iSCSI driver gt where lt BE iSCSI driver gt is the iSCSI driver released as an iso For example vmware esx drivers scsi be2iscsi 400 2 103 308 0 lvmw 0 0 235786 x86 64 1iso Drivers for VMware ESX ESXi 4 1 Installation Section Page 4 gt G EMULEX esxupdate Procedure For esxupdate installs perform the following procedure Directly mount the ISO image in this manner root testmachine mkdir tmp mountpoint root testmachine mount r o loop path to driver iso tmp mountpoint Once the ISO image is mounted the offline bundle can be accessed in the tmp mountpoint offline bundle directory Since the offline bundle file name will vary depending on the package that is to be installed the example below uses the name offline bundle zip please replace this file name with the name of the offline
110. w the installation instructions provided on the VMware support website See the OneCommand Manager Command Line User Manual for instructions on installing the Emulex core kit Note Before installing the Emulex OneCommand Manager Kit you must install the Ipfc driver from the VMware website Drivers for VMware ESX ESXi 4 1 Installation Section Page 3 gt G EMULEX Note Before installing OneConnect network drivers and management software verify that the firmware version is correct If it is proceed with the installation If it is not update the firmware using the OneCommand Manager application and reboot your system before proceeding with the installation Uninstalling the NIC Driver See the VMware Patch Download page for instructions Installing All the Drivers To install all drivers use the esxupdate utility provided by VMware in ESX and ESXi platforms The drivers and components that can be installed include LPFC gt NIC e iSCSI LPFC Driver The LPFC driver is the driver used for LP adapters and the UCNA adapter FCoE ports To install the Ipfc driver execute the following command in the command prompt of the ESX ESXi host esxupdate maintenancemode nosigcheck update b lt lpfc driver iso gt where lt lpfc driver iso gt is the Ipfc driver released as an iso For example vmware esx drivers scsi lpfc820 400 8 2 0 86 26 lvmw 2 17 00000 x86 64 1iso NIC Driver The NIC driver is the dri
111. warning message to continue with the change 4 Ifthe IP port number is changed the utility restarts the OneCommand Manager Application discovery server and management agent to use the new settings If the servers cannot be stopped and restarted you are prompted to reboot the host for the new TCP IP management settings to take effect CAUTI ON The IP port number must be the same for all hosts that are to be managed Setting an IP port number for one host to a different value than the other hosts will make the host unable to manage other hosts over TCP IP as well as make the host unmanageable over TCP IP from other hosts 5 The CIM credentials group can be used to set the default CIM credentials which will be used by default to connect to all the ESX hosts that are managed through the CIM interface e Protocol The http or https protocol can be used to connect to the ESX hosts The default port numbers used for http and https are 5988 and 5989 respectively The port number will change automatically according to the protocol selected The user can also manually change the port number Since by default the HTTP is disabled on sfcb in ESXi host user should use HTTPS to communicate to the ESXi host e User name The user name field contains the username with which to connect to the ESX hosts By default this will be root e Password This password field will contain the password of the user name which will be used to connect to the ESX host
112. x_mes0494 Unable to allocate memory for issuing SLI_FUNCTION RESET mailbox commana DESCRIPTION Mailbox memory pool allocation error Drivers for VMware ESX ESXi 4 1 Troubleshooting Section Page 41 J gt G EMULEX DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0495 SLI FUNCTION_RESET mailbox failed with status lt shdr_status gt add_status lt shdr_add_status gt mbx status lt rc gt DESCRIPTION Mailbox command failed DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0496 Failed allocate slow path EQ DESCRIPTION The event queue for the slow path was not allocated DATA None ACTION Unload and reload the driver elx_mes0497 Failed allocate fast path EQ DESCRIPTION The event queue for the fast path was not allocated DATA None ACTION Unload and reload the driver elx_mes0499 Failed allocate fast path FCP CQ lt fcp_cqidx gt DESCRIPTION The completion queue event for the fast path could not be allocated DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0500 Failed allocate slow path mailbox CQ DESCRIPTION Failed to allocate slow path mailbox CQ DATA None ACTION The driver fails to load Contact Emulex Technical Support elx_mes0501 Failed allocate slow path ELS CQ DESCRIPTION Failed to allocate slow path ELS CQ DATA None ACTION The driver fails to load Contact Emulex T

esxupdate

Contents

Download Pdf Manuals

Related Search

Related Contents