Home

Cisco Systems DSC9148D8G48PK9 User's Manual

1. A a p gt 120489 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Cu OL 29284 01 Release 6 x Chapter 6 Configuring PortChannels Information About PortChannels The channel group numbers are selected dynamically and as such the administrative configuration of the ports forming the channel group at either end are applicable to the newly created channel group The channel group number being chosen dynamically may be different across reboots for the same set of PortChannels based on the order of ports that are initialized in the switch Table 6 2 identifies the differences between user configured and auto configured channel groups Table 6 2 User Configured Channel Group Channel Group Configuration Differences Autocreated Channel Group Manually configured by the user Created automatically when compatible links come up between two compatible switches if channel group autocreation is enabled in all ports at both ends Member ports cannot participate in autocreation of channel groups The autocreation feature cannot be configured None of these ports are members of a user configured channel group You can form the PortChannel with a subset of the ports in the channel group Incompatible ports remain in a suspended or isolated state depending on the ON or ACTIVE mode configuration All ports included in the channel group participate in the PortCha
2. Chapter Title Description Chapter 7 Configuring N Port Virtualization Provides an overview of N Port Virtualization and includes quidelines and requirements for configuring and verifying NPV Chapter 8 Configuring FlexAttach Virtual FlexAttach virtual pWWN feature facilitates pWWN server and configuration management In a SAN environment the server installation or replacement requires interaction and coordination among the SAN and server administrators Document Conventions Command descriptions use these conventions boldface font Commands and keywords are in boldface italic font Arguments for which you supply values are in italics Elements in square brackets are optional xlylz Optional alternative keywords are grouped in brackets and separated by vertical bars Screen examples use these conventions screen font Terminal sessions and information the switch displays are in screen font boldface screen font italic screen font Information you must enter is in boldface screen font Arguments for which you supply values are in italic screen font Nonprinting characters such as passwords are in angle brackets Default responses to system prompts are in square brackets An exclamation point or a pound sign at the beginning of a line of code indicates a comment line This document uses the following conventions Means reader
3. Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Verifying Interfaces Configuration W LR RX Delta 60 5 4 1 4 Not enabled LR TX Delta 60 5 4 ul 4 Not enabled Timeout Discards Delta 60 200 4 10 4 Not enabled Credit Loss Reco Delta 60 T 4 0 4 Not enabled TX Credit Not Available Delta 60 10 4 0 4 Not enabled The following default port monitor policy will be active when the switch comes up Policy Name slowdrain Admin status Not Active Oper status Not Active Port type All Ports Counter Threshold Interval Rising Threshold event Falling Threshold event Portguard Credit Loss Reco Delta 5 4 4 1 4 Not enabled TX Credit Not Available Delta al 20 4 10 4 Not enabled Configuring Average Credit Nonavailable Duration Threshold and Action When the average credit nonavailable duration exceeds the set threshold the port can be error disabled a trap can be sent with interface details and a syslog can be generated with interface details One or more of these actions can also be combined together These actions can be turned on or off depending on the situation The port monitor feature provides the command line interface to configure the thresholds and action The threshold configuration can be a percentage of credit nonavailable duration in an interval The thresholds are that the credit nonavailable duration can be 0 percent to 100 percen
4. fig i shut switchport mode F switchport trunk mode off switchport speed 4000 switchport rate mode shared channel group 1 no shut exit Step5 Create the PortChannel on the NPV switch swi swi swi swi tch tch tch tch con con con con fig interface port channel 1 fig if fig if fig if switchport mode NP switchport rate mode shared exit Step6 Configure the PortChannel member interfaces on the NPV switch Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter6 Configuring PortChannels Step 7 Configuration Examples for F and TF PortChannels W switch config interface fc2 1 3 shut switchport mode NP switchport speed 4000 switchport rate mode shared switchport trunk mode off channel group 1 no shut exit switch config i switch config i switch config i switch config i switch config i switch config i switch config i switch config i Fh Fh FH OF oPh Ph Fh orh Set the administrative state of all the PortChannel member interfaces in both NPIV core switch and the NPV switch to ON switch config interface fc1 1 3 switch config if shut switch config if no shut switch config interface fc2 1 3 switch config if shut switch config if no shut Note Step 1 Step 2 Step 3 Step 4 Step 5 The speed configuration must be the same for all member interfaces in a PortCha
5. Command Purpose switch config t switch config Enters configuration mode switch config interface fc8 13 switch config if Enters the configuration mode for the selected interface s switch config if channel group auto Automatically creates the channel group for the selected interface s switch config if no channel group auto Disables the autocreation of channel groups for this interface even if the system default configuration may have autocreation enabled Manually Configured Channel Groups You can convert autocreated channel group to a user configured channel group using the port channel channel group number persistent EXEC command If the PortChannel does not exist this command is not executed To display PortChannel configuration information perform one of the following tasks Command Purpose show port channel summary Displays a summary of PortChannels within the switch A one line summary of each PortChannel provides the administrative state the operational state the number of attached and active interfaces up and the first operational port FOP which is the primary operational interface selected in the PortChannel to carry control plane traffic no load balancing The FOP is the first port that comes up in a PortChannel and can change if the port goes down The FOP is also identified by an asterisk show port channel database Di
6. Interfaces in the Port Group B2B Credit Bandwidth Rate Mode Buffers Gbps fc7 1 32 4 0 shared fc7 2 32 4 0 shared fc7 3 32 4 0 shared fc7 4 32 4 0 shared fc7 5 32 4 0 shared fc7 6 32 4 0 shared fc7 7 32 4 0 shared fc7 8 32 4 0 shared fc7 9 32 4 0 shared fc7 10 32 4 0 shared fc7 11 32 4 0 shared fc7 12 32 4 0 shared Port Group 2 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Bandwidth Rate Mode Buffers Gbps Cisco MDS 9000 Family NX OS Interfaces Configuration Guide 33a OL 29284 01 Release 6 x Chapter3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces W Fay 7 13 32 4 0 shared Fo7 14 32 4 0 shared f c7 15 32 4 0 shared fc7 16 32 4 0 shared Fo7 17 32 4 0 shared fc7 18 32 4 0 shared fc7 19 32 4 0 shared fc7 20 32 4 0 shared f c7 21 32 4 0 shared c7 22 32 4 0 shared fc7 23 32 4 0 shared fc7 24 32 4 0 shared Port Group 3 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Bandwidth Rate Mode Buffers Gbps 67 25 32 4 0 shared fc7 26 32 4 0 shared c7 27 32 4 0 shared c7 28 32 4 0 shared fc7 29 32 4 0 shared fc7 30 32 4 0 shared fc7 31 32 4 0 shared fc7 32 32 4 0 shared fc7 33 32 4 0 shared fc7 34 32 4 0 shared fc7 35 32 4 0 shared fc7 36 32 4 0 sha
7. OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Information About Interfaces W Slow Drain Device Detection and Congestion Avoidance amp All data traffic between end devices in a SAN fabric is carried by Fibre Channel Class 3 In some cases the traffic is carried by Class 2 services that use link level per hop based and buffer to buffer flow control These classes of service do not support end to end flow control When there are slow devices attached to the fabric the end devices do not accept the frames at the configured or negotiated rate The slow devices lead to ISL credit shortage in the traffic destined for these devices and they congest the links The credit shortage affects the unrelated flows in the fabric that use the same ISL link even though destination devices do not experience slow drain This feature provides various enhancements to detect slow drain devices that are causing congestion in the network and also provides a congestion avoidance function This feature is focused mainly on the edge ports that are connected to slow drain devices The goal is to avoid or minimize the frames being stuck in the edge ports due to slow drain devices that are causing ISL blockage To avoid or minimize the stuck condition configure lesser frame timeout for the ports No credit timeout drops all packets once the slow drain is detected using the configured thresholds The lesser frame timeout value helps to alleviate t
8. If you configure this feature the linked port is up only when the VSAN is up on the tracked port Tip The specified VSAN does not have to be the same as the port VSAN of the linked port Monitoring Ports in a VSAN Detailed Steps To monitor a tracked port in a specific VSAN follow these steps Command Purpose Step1 switch config t Enters configuration mode Step2 switch config interface fc8 6 Configures the specified interface and enters the interface configuration submode You can now configure tracked ports Step3 switch config if port track Enables tracking of the PortChannel in VSAN 2 interface port channel 1 vsan 2 switch config if no port track Removes the VSAN association for the linked port The interface port channel 1 vsan 2 PortChannel link remains in effect Cisco MDS 9000 Family NX OS Quality of Service Configuration Guide I oL 29284 01 g os Chapter9 Configuring Port Tracking W Displaying Port Tracking Information Information AboutForceful Shutdown If a tracked port flaps frequently then tracking ports using the operational binding feature may cause frequent topology change In this case you may choose to keep the port in the down state until you are able to resolve the reason for these frequent flaps Keeping the flapping port in the down state forces the traffic to flow through the redundant path until the primary tracked port problems are resolved When the proble
9. This section describes how to enable the F port trunking and channeling protocol Prerequisites e To avoid inconsistent configurations shut all ports before enabling or disabling the trunking protocols Detailed Steps To enable or disable the F port trunking and channeling protocol follow these steps Command Purpose Step1 switch config tasf Enters configuration mode Step2 switch config feature fport channel trunk Enables the F port trunking and channeling switch config protocol default switch config no feature fport channel trunk Disables the F port trunking and channeling switch config protocol Configuring Trunk Mode Detailed Steps To configure trunk mode follow these steps Command Purpose Step1 switch config t Enters configuration mode Step2 switch config interface fc1 1 Configures the specified interface switch config if Step3 switch config if switchport trunk mode on Enables default the trunk mode for the specified interface switch config if switchport trunk mode off Disables the trunk mode for the specified interface switch config if switchport trunk mode auto Configures the trunk mode to auto mode which provides automatic sensing for the interface Configuring an Allowed Active List of VSANs Detailed Steps To configure an allowed active list of VSANs for an interface follow these steps Command Purpose St
10. green light that helps you identify the physical location of the specified interface Configuring the beacon mode has no effect on the operation of the interface To enable beacon mode for a specified interface or range of interfaces follow these steps Command Purpose switch config t Enters configuration mode switch config switch config interface fc1 1 Selects a Fibre Channel interface and enters switch config if interface configuration submode I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide ii Chapter2 Configuring Interfaces HZ Configuring Interfaces Command Purpose Step3 9 switch config if switchport beacon Enables the beacon mode for the interface switch config if no switchport beacon Disables the beacon mode for the interface Troubleshooting Tips e The flashing green light turns on automatically when an external loopback is detected that causes the interfaces to be isolated The flashing green light overrides the beacon mode configuration The state of the LED is restored to reflect the beacon mode configuration after the external loopback is removed Disabling Bit Error Threshold Detailed Steps To disable the bit error threshold for an interface follow these steps Command Purpose Step1 switch config t Enters configuration mode Step2 switch config interface fc1 1 Selects a Fibre Channel interfac
11. W Configuring Fibre Channel Interfaces Step5 Configure the traffic speed 1 Gbps 2 Gbps 4 Gbps or autosensing with a maximum of 2 Gbps or 4 Gbps to use See the Configuring Port Speed section on page 3 29 Task Flow for Configuring 12 Port 4 Gbps Module Interfaces The 12 port 4 Gbps switching modules support the following features e 1 Gbps 2 Gbps and 4 Gbps speed traffic e Only dedicated rate mode e ISL E or TE and Fx F or FL port modes e Extended BB credits e Performance buffers To configure 4 port 10 Gbps switching modules when starting with the default configuration follow these steps Step 1 Configure the traffic speed 1 Gbps 2 Gbps 4 Gbps or autosensing with a maximum of 2 Gbps or 4 Gbps to use See the Configuring Port Speed section on page 3 29 Step2 Configure the port mode See the Configuring Interface Modes section on page 2 22 Step 3 Configure the BB_credits performance buffers and extended BB_credits as necessary See the Configuring Buffer to Buffer Credits section on page 4 21 and the Configuring Extended BB _credits section on page 4 23 Task Flow for Configuring 4 Port 10 Gbps Module Interfaces The 4 port 10 Gbps switching modules support the following features e Only 10 Gbps speed traffic e Only dedicated rate mode e ISL E or TE and F port modes e Extended BB credits e Performance buffers To configure 4 port 10 Gbps switching modules when starting with
12. config if config if the interface config if config if switchport speed 1000 switchport rate mode shared switchport mode f s and return to configuration mode no shutdown exit Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter 3 Configuring Fibre Channel Interfaces W Configuration Examples for Fibre Channel Interfaces Cisco MDS 9000 Family NX OS Interfaces Configuration Guide M350 BS OL 29284 01 Release 6 x ee Sate Configuring Interface Buffers Information About Interface Buffers page 4 1 Configuring Interface Buffers page 4 21 Verifying BB_Credit Configuration page 4 25 Information About Interface Buffers Fibre Channel interfaces use buffer credits to ensure all packets are delivered to their destination This section includes the following topics Buffer to Buffer Credits page 4 1 Performance Buffers page 4 2 Buffer Pools page 4 2 BB_Credit Buffers for Switching Modules page 4 5 BB_Credit Buffers for Fabric Switches page 4 16 Extended BB_Credits page 4 17 Buffer to Buffer Credit Recovery page 4 20 Buffer to Buffer State Change Number page 4 20 Receive Data Field Size page 4 21 Buffer to Buffer Credits Buffer to buffer credits BB_credits are a flow control mechanism to ensure that Fibre Channel switches do not run out of buffers so that switches do not drop frames BB_credits are negotiated on a per hop basis
13. fcip200 up The show port channel consistency command has two options without details and with details Example 6 4 Displays the Consistency Status without Details switch show port channel consistency Database is consistent Example 6 5 Displays the Consistency Status with Details switch show port channel consistency detail Authoritative port channel database totally 3 port channels port channel 77 2 ports first operational port is none fcipl down fcip2 down port channel 78 2 ports first operational port is none fe271 down fc2 5 down port channel 79 2 ports first operational port is fcip200 fcip101 up fcip200 up database 1 from module 5 totally 3 port channels port channel 77 2 ports first operational port is none fcip1 down fcip2 down port channel 78 2 ports first operational port is none fc2 1 down fc2 5 down port channel 79 2 ports first operational port is fcip200 fcip101 up fcip200 up database 2 from module 4 totally 3 port channels Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter6 Configuring PortChannels Verifying PortChannel Configuration W port channel 77 2 ports first operational port is none fcipl down fcip2 down port channel 78 2 ports first operational port is none fc2 1 down e2 5 down port channel 79 2 ports first operational port is fcip200 fcip101 up fcip200 up
14. switch config interface fc 4 5 6 Configure the port speed rate mode and port mode on the interfaces switch config if switchport speed auto max 4000 switch config if switchport rate mode shared switch config if switchport mode f Enable the interfaces and return to configuration mode switch config if no shutdown switch config if exit switch Configuration Example for 24 Port 8 Gbps Module Interfaces These steps describe how to configure the 24 port 8 Gbps module interfaces Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter3 Configuring Fibre Channel Interfaces Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Configuration Examples for Fibre Channel Interfaces Select interfaces fc 3 1 swi swi tch config t tch config interface fc 3 1 Configure the port speed rate mode and port mode on the interfaces swi swi swi tch config if switchport speed 8000 tch config if switchport rate mode dedicated tch config if switchport mode f Enable the interfaces and return to configuration mode swi swi swi tch config if no shutdown tch config if exit tch Select the interfaces fc 3 2 through fc 3 3 swi swi tch config t tch config interface fc 3 2 3 Configure the port speed rate mode and port mode on the interfaces swi swi swi tch config if switchport speed 8000 tch con
15. 0 discards 0 errors 0 CRC 0 unknown class 0 too long 0 too short 1 frames output 128 bytes 0 discards 0 errors input OLS output OLS COO amp 0 LRR 0 LRR receive B2B credit remaining transmit B2B credit remaining 0 NOS 0 loop inits 0 NOS 0 loop inits Example 9 3 Displays a Tracked Port Configuration for a PortChannel Interface switch show interface port channel 1 port channel 1 is down No operational members Hardware is Fibre Channel Port WWN is 24 01 00 05 30 00 0d de Admin port mode is auto trunk mode is on Port vsan is 2 Linked to 1 port s Port linked to interface fcl1 1 5 minutes input rate 0 bits sec 0 bytes sec 0 frames sec 5 minutes output rate 0 bits sec 0 bytes sec 0 frames sec 0 frames input 0 discards 0 bytes 0 errors 0 CRC 0 unknown class 0 too long 0 too short 0 frames output 0 bytes 0 discards 0 errors 0 input OLS 0 output OLS No members 0 LRR 0 LRR 0 NOS 0 loop inits 0 NOS 0 loop inits OL 29284 01 Cisco MDS 9000 Family NX OS Quality of Service Configuration Guide E Chapter9 Configuring Port Tracking E Example 9 4 Displays a Forced Shutdown Configuration switch show interface fc 1 5 fc1 5 is up Hardware is Fibre Channel FCOT is short wave laser Port WWN is 20 05 00 05 30 00 47 9e Admin port mode is F Port mode is F FCID is 0x710005 Port vsan is 1 Speed is 1 Gbps Transmit B2B Credit is 64 Receive B2
16. 63 bytes sec 0 frames sec 5 minutes output rate 520 bits sec 65 bytes sec 0 frames sec 47050 frames input 10311824 bytes 0 discards 0 errors 0 CRC 0 unknown class 0 too long 0 too short 62659 frames output 10676988 bytes 0 discards 0 errors 0 input OLS 0 LRR 0 NOS 0 loop inits 1 output OLS 1 LRR 0 NOS 1 loop inits 16 receive B2B credit remaining 3 transmit B2B credit remaining Example 2 5 Displays a Specific Interface switch show interface fc2 2 fc2 2 is trunking Port description is Trunk to Core 4 Hardware is Fibre Channel SFP is short wave laser Port WWN is 20 42 00 05 30 00 97 9e Peer port WWN is 20 cc 00 05 30 00 50 9e Admin port mode is E trunk mode is on Port mode is TE Port vsan is 1 Speed is 2 Gbps Transmit B2B Credit is 255 Receive B2B Credit is 255 Receive data field Size is 2112 Beacon is turned off Belongs to port channel 2 Trunk vsans admin allowed and active 1 100 3000 Trunk vsans up 1 Trunk vsans isolated 100 3000 Trunk vsans initializing 5 minutes input rate 0 bits sec 0 bytes sec 0 frames sec 5 minutes output rate 32 bits sec 4 bytes sec 0 frames sec Cisco MDS 9000 Family NX OS Interfaces Configuration Guide M244 OL 29284 01 Release 6 x Chapter 2 Configuring Interfaces 2214834 frames input 0 discards 0 errors 98673588 bytes 0 CRC 0 unknown class 0 too long 0 too short 2262415 frames output 343158368 bytes 0 d
17. NPIV is used by edge switches in the NPV mode to log in to multiple end devices that share a link to the core switch This feature is available only for Cisco MDS Blade Switch Series the Cisco MDS 9124 Multilayer Fabric Switch and the Cisco MDS 9134 Multilayer Fabric Switch FlexAttach Cisco NX OS supports the FlexAttach feature One of the main problems in a SAN environment is the time and effort required to install and replace servers The process involves both SAN and server administrators and the interaction and coordination between them can make the process time consuming To alleviate the need for interaction between SAN and server administrators the SAN configuration should not be changed when a new server is installed or an existing server is replaced FlexAttach addresses these problems reducing configuration changes and the time and coordination required by SAN and server administrators when installing and replacing servers This feature is available only for Cisco MDS 9000 Blade Switch Series the Cisco MDS 9124 and the Cisco MDS 9134 when NPV mode is enabled Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter1 _ Interfaces Overview FlexAttach W Cisco MDS 9000 Family NX OS Interfaces Configuration Guide I OL 29284 01 Release 6 x EEN Chapter1 Interfaces Overview E FlexAttach Cisco MDS 9000 Family NX OS Interfaces Configuration Guide a OL 29284 01 Relea
18. The buffer to buffer credit recovery feature is not compatible with the distance extension DE feature also known as buffer to buffer credit spoofing If you use intermediate optical equipment such as DWDM transceivers or Fibre Channel bridges on ISLs between switches that use DE then buffer to buffer credit recovery on both sides of the ISL needs to be disabled Buffer to Buffer State Change Number The BB_SC_N field word 1 bits 15 12 specifies the buffer to buffer state change BB_SC number The BB_SC_N field indicates that the sender of the port login PLOGI fabric login FLOGI or ISLs E or TE ports frame is requesting 2 SC_BB_N number of frames to be sent between two consecutive BB_SC send primitives and twice the number of R_RDY primitives to be sent between two consecutive BB_SC receive primitives For Generation 2 and Generation 3 modules the BB_SCN on ISLs E or TE ports is enabled by default This can fail the ISLs if used with optical equipment using distance extension DE also known as buffer to buffer credit spoofing On a Generation 2 module one port will not come up for the following configuration for all ports e Port Mode auto or E for all the ports e Rate Mode dedicated e Buffer Credits default value On a Generation 3 module one or two ports will not come up for the following configuration for the first half of the ports the second half of the ports or all ports e Port Mode auto or E for the firs
19. The receive BB_credit fcrxbbcredit value may be configured for each Fibre Channel interface In most cases you do not need to modify the default configuration The receive BB_credit values depend on the module type and the port mode as follows For 16 port switching modules and full rate ports the default value is 16 for Fx mode and 255 for E or TE modes The maximum value is 255 in all modes This value can be changed as required For 32 port switching modules and host optimized ports the default value is 12 for Fx E and TE modes These values cannot be changed I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter4 Configuring Interface Buffers HZ Information About Interface Buffers e For Generation 2 Generation 3 and Generation 4 switching modules see the Buffer Pools section on page 4 2 In the Cisco MDS 9100 Series switches the groups of ports on the left outlined in white are in dedicated rate mode The other ports are host optimized Each group of 4 host optimized ports have the same features as for the 32 port switching module Note Because Generation modules do not support as many buffer to buffer credits as Generation 4 modules supports you cannot configure an ISL on E or TE ports between a Generation 1 module such as the 16 port 1 2 Gbps Fibre Channel Switching Module DS X9016 and a Generation 4 module such as the 48 port 8 Gbps Advanced F
20. The show port channel usage command displays details of the used and unused PortChannel numbers Example 6 6 Displays the PortChannel Usage switch show port channel usage Totally 3 port channel numbers used Used 77 79 Unused 1 76 80 256 Example 6 7 Displays the PortChannel Compatibility switch show port channel compatibility parameters physical port layer fibre channel or ethernet port mode E AUTO only trunk mode speed port VSAN port allowed VSAN list Use the existing show commands to obtain further details on autocreated channel group attributes Autocreated PortChannels are indicated explicitly to help differentiate them from the manually created PortChannels See Examples 6 8 to 6 10 Example 6 8 Displays Autocreated PortChannels switch show interface fc1 1 fc1 1 is trunking Hardware is Fibre Channel FCOT is short wave laser Port WWN is 20 0a 00 0b 5f 3b fe 80 Receive data field Size is 2112 Beacon is turned off Port channel auto creation is enabled Belongs to port channel 123 Example 6 9 Displays the Specified PortChannel Interface switch show port channel database interface port channel 128 port channel 128 Administrative channel mode is active Operational channel mode is active Last membership update succeeded Channel is auto created First operational port is fcl 1 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter6 Configuring Por
21. Tip When enabling autocreation in any switch in the Cisco MDS 9000 Family we recommend that you retain at least one interconnected port between the switches without any autocreation configuration If all ports between two switches are configured with the autocreation feature at the same time you may face a possible traffic disruption between these two switches as the ports are automatically disabled and reenabled when ports are added to an autocreated PortChannel Manually Configured Channel Groups Tip A user configured channel group cannot be converted to an autocreated channel group However you can convert an autocreated channel group to a manual channel group Once performed this task is irreversible The channel group number does not change but the member ports operate according to the properties of the manually configured channel group and the autocreation of channel group is implicitly disabled for all member ports If you enable persistence be sure to enable it at both ends of the PortChannel Prerequisites for PortChannels Before configuring a PortChannel consider the following guidelines e Configure the PortChannel across switching modules to implement redundancy on switching module reboots or upgrades e Ensure that one PortChannel is not connected to different sets of switches PortChannels require point to point connections between the same set of switches Cisco MDS 9000 Family NX OS Interfaces Confi
22. Total shared bandwidth is 32 4 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Buffers fc5 1 32 c5 2 32 c5 3 32 c5 4 32 Port Group 2 Total bandwidth is 32 4 Gbps Total shared bandwidth is 32 4 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Buffers f c5 5 32 fc5 6 32 fc5 7 32 fc5 8 32 Port Group 3 Total bandwidth is 32 4 Gbps Total shared bandwidth is 32 4 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Buffers fc5 9 32 fc5 10 32 5 11 32 port group 1 are 3970 1 port group 2 are 3970 2 port group 3 are 3970 3 port group 4 are 3970 4 port group 5 are 3058 5 port group 6 are 3058 6 port group 7 are 3970 7 port group 8 are 3970 Bandwidth Rate Mode Gbps 8 0 shared 8 0 shared 8 0 shared 8 0 shared Bandwidth Rate Mode Gbps 8 0 shared 8 0 shared 8 0 shared 8 0 shared Bandwidth Rate Mode Gbps 8 0 shared 8 0 shared 8 0 shared OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Fc5 12 32 Port Group 4 Total bandwidth is 32 4 Gbps Total shared bandwidth is 32 4 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Buffers fc5 13 32 fc5 14 32 fc5 15 32
23. configured for autosensing auto and dedicated rate mode then 4 Gbps of bandwidth is reserved even though the maximum operating speed is 2 Gbps For the same interface if autosensing with a maximum speed of 2 Gbps auto max 2000 is configured then only 2 Gbps of bandwidth is reserved and the unused 2 Gbps is shared with the other interface in the port group Note e The Generation 2 4 port 10 Gbps switching module supports 10 Gbps traffic only e On Generation 2 4 Gbps modules setting the port speed to auto enables autosensing which negotiates to a maximum speed of 4 Gbps e On Generation 3 8 Gbps modules setting the port speed to auto enables autosensing which negotiates to a maximum speed of 8 Gbps e On Generation 4 8 Gbps modules setting the port speed to auto enables autosensing which negotiates to a maximum speed of 8 Gbps Dynamic Bandwidth Management On port switching modules where bandwidth is shared the bandwidth available to each port within a port group can be configured based on the port rate mode and speed configurations Within a port group some ports can be configured in dedicated rate mode while others operate in shared mode E Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces W Ports configured in dedicated rate mode are allocated the required bandwidth to sust
24. e Management Interfaces page 2 17 e VSAN Interfaces page 2 18 Interface Description For the Fibre Channel interfaces you can configure the description parameter to provide a recognizable name for the interface Using a unique name for each interface allows you to quickly identify the interface when you are looking at a listing of multiple interfaces You can also use the description to identify the traffic or the use for that interface Interface Modes Each physical Fibre Channel interface in a switch may operate in one of several port modes E port F port FL port TL port TE port SD port ST port and B port see Figure 2 1 Besides these modes each interface may be configured in auto or Fx port modes These two modes determine the port type during interface initialization Figure 2 1 Cisco MDS 9000 Family Switch Port Modes NL port NL port Private loop a E port K TL port ISL link 79528 Note Interfaces are created in VSAN 1 by default See the Cisco MDS 9000 Family NX OS Fabric Configuration Guide Each interface has an associated administrative configuration and an operational status e The administrative configuration does not change unless you modify it This configuration has various attributes that you can configure in administrative mode Cisco MDS 9000 Family NX OS Interfaces Configuration Guide P22 E OL 29284 01 Release 6 x Chapter2 Configuring Interfaces D Informat
25. e The ports cannot be configured in any other speed e Ports that are capable of 10 Gbps that are disabled or out of service cannot be put back in service using the no out of service command To put these ports back in service all ports in the module first have to be moved to the out of service state Then they can be brought back to the in service state e Local switching must be disabled otherwise ports cannot be configured in dedicated mode Only certain ports on the 48 port and 32 port 8 Gbps Advanced Fibre Channel modules are 10 Gbps capable When running in 10 Gbps mode the non 10 Gbps ports cannot be operational They have to be either in shut state or out of service state Cisco MDS 9000 Family NX OS Interfaces Configuration Guide 224 E OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Detailed Steps Configuring Interfaces To configure the interface mode follow these steps Command Purpose Step1 switch config t Enters configuration mode Step 2 switch config interface fc1 1 12 switch config if Selects a Fibre Channel interface and enters interface configuration submode Note Ensure that one full ASIC range of ports are selected before executing this command For example 1 12 for a 48 port module or fcY 1 8 for a 32 port module The first ASIC on the 48 port module has ports fc1 1 12 Step 3 switch config if 10G speed mode Configures all the ports fcl to
26. fc4 8 1 F sfpAbsent c4 9 I F sfpAbsent Example 2 16 Displays Interface Information in a Brief Format After Two Interfaces Are Individually Configured for Mode FL switch show interface brief Interface Vsan Admin Admin Status SFP Oper Oper Port Mode Trunk Mode Speed Channel Mode Gbps fc4 1 i FL notConnected swl ote fc4 2 1 F notConnected swl a fc4 3 1 FL notConnected swl as fc4 4 1 F notConnected swl fc4 5 1 F sfpAbsent c4 6 1 F sfpAbsent f c4 7 1 F sfpAbsent c4 8 1 F sfpAbsent c4 9 1 F sfpAbsent c4 10 1 F sfpAbsent Displaying TL Port Information The show tlport command displays the TL port interface configurations This command provides a list of all TL ports configured in a switch and shows the associated VSAN the FC ID for the port only domain and area are valid and the current operational state of the TL port up or initializing See Example 2 17 through Example 2 20 Example 2 17 Displays the TL Ports in All VSANs switch show tlport list Interface Vsan FC ID State fc1 16 a 0x420000 Init c2 26 1 0x150000 Up TL ports allow a private device devices that physically reside on the loop to see a fabric device and vice versa by proxying fabric devices on the loop Fabric devices are proxied by allocating each fabric device an ALPA on this loop Cisco MDS 9000 Family NX OS Inter
27. nested NPIV Only F NP and SD ports are supported in NPV mode In the case of servers that are booted over the SAN with NPV if an NPV link failover occurs servers will lose access to their boot LUN temporarily NPV switches do not recognize the BB_SCN configuration on the xNP ports because of interoperability issues with the third party core switches NPV Traffic Management Guidelines When deploying NPV traffic management follow these guidelines Use NPV traffic management only when the automatic traffic engineering by the NPV device is not sufficient for the network requirements Do not configure traffic maps for all the servers For non configured servers NPV will use automatic traffic engineering Configure the Persistent FC ID on the core switch Traffic engineering directs the associated server interface to external interfaces that lead to the same core switch The server will be assigned the same FC ID for every log in This guideline is not applicable if a 91x4 switch is used as the core switch Server interfaces configured to a set of external interfaces cannot use any other available external interfaces even if the configured interfaces are not available Do not configure disruptive load balancing because this involves moving a device from one external interface to another interface Moving the device between external interfaces requires NPV relogin to the core switch through F port leading to traffic disruption Lin
28. page 4 5 e 48 Port 8 Gbps Advanced Fibre Channel Module BB_Credit Buffers page 4 6 e 48 Port 8 Gbps Fibre Channel Module BB_Credit Buffers page 4 7 e 24 Port 8 Gbps Fibre Channel Module BB_Credit Buffers page 4 8 e 4 44 Port 8 Gbps Host Optimized Fibre Channel Module BB_Credit Buffers page 4 9 e 48 Port 4 Gbps Fibre Channel Module BB_Credit Buffers page 4 10 e 24 Port 4 Gbps Fibre Channel Module BB_Credit Buffers page 4 12 e 18 Port Fibre Channel 4 Port Gigabit Ethernet Multiservice Module BB_Credit Buffers page 4 13 e 4 Port 10 Gbps Switching Module BB_Credit Buffers page 4 15 Configuring Buffer Credits on a Generation 2 Generation 3 or Generation 4 Module When you configure port mode to auto or E on a Generation 2 module one of the ports will not come up for the following configuration e Port Mode auto or E for all of the ports e Rate Mode dedicated e Buffer Credits default value Cisco MDS 9000 Family NX OS Interfaces Configuration Guide I OL 29284 01 Release 6 x Chapter4 Configuring Interface Buffers Information About Interface Buffers When you configure port mode to auto or E on a Generation 3 module one or two of the ports will not come up for the following configuration e Port Mode auto or E for the first half of the ports the second half of the ports or for all of the ports e Rate Mode dedicated e Buffer Credits default value When you configure port mode to auto or E for all ports in t
29. switch config system default switchport shutdown switch config Configures the default setting for administrative state of an interface as Down This is the factory default setting Tip This command is applicable only to interfaces for which no user configuration exists for the administrative state switch config system default switchport trunk mode auto switch config Configures the default setting for administrative trunk mode state of an interface as Auto Note The default setting is trunk mode on Configuring TL Ports Private loops require setting the interface mode to TL Use the switchport mode command to configure a TL port See the Configuring Interface Modes section on page 2 22 Manually Inserting Entries into the ALPA Cache Detailed Steps To manually insert entries into the ALPA cache follow these steps Command Purpose Step1 switch config t Enters configuration mode Step2 switch config tlport alpa cache interface Configures manual entries into the ALPA cache fc1 2 pwwn 22 00 00 20 37 46 09 bd alpa 0x02 Step3 switch config tlport alpa cache interface Removes this entry from the ALPA cache fc1 3 pwwn 22 00 00 20 37 46 09 bd Clearing the ALPA Cache The clear tlport alpa cache command clears the entire content of the ALPA cache Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x EZE Chapter2 Configurin
30. 2 Gbps 0 5 0 135 2 switching module 1 Gbps 0 25 0 067 1 All ports in the 48 port and 24 port 4 Gbps modules can be configured to operate at 4 Gbps in shared mode even if other ports in the port group are configured in dedicated mode regardless of available bandwidth However when oversubscription ratio restrictions are enabled you may not have all shared 4 Gbps module ports operating at 4 Gbps All ports in the 48 port 32 Port and 24 port 8 Gbps modules can be configured to operate at 8 Gbps in shared mode even if other ports in the port group are configured in dedicated mode regardless of available bandwidth However when oversubscription ratio restrictions are enabled you may not have all shared 8 Gbps module ports operating at 8 Gbps On the 48 port 32 Port and 24 port 8 Gbps modules if you have configured one 8 Gbps dedicated port in one port group no other ports in the same port group can be configured to operate at 8 Gbps dedicated mode You can have any number of 8 Gbps shared and 4 Gbps dedicated or shared ports On the 4 44 port 8 Gbps module only one port per port group can be configured in 8 Gbps dedicated or shared mode In the following example a 24 port 4 Gbps module has oversubscription ratios enabled and three dedicated ports in one port group operating at 4 Gbps No other ports in the same port group can be configured to operate at 4 Gbps switch show port resources module 8 Module 8 Availa
31. 2147483648 Not enabled TX Performance Delta 60 2147483648 Not enabled Policy Name default Admin status Not Active Oper status Not Active Port type All Ports Counter Threshold Interval Rising Threshold portguard Link Loss Delta 60 5 Not enabled Sync Loss Delta 60 5 Not enabled Protocol Error Delta 60 R Not enabled Signal Loss Delta 60 5 Not enabled Invalid Words Delta 60 I Not enabled Invalid CRC s Delta 60 5 Not enabled RX Performance Delta 60 2147483648 Not enabled TX Performance Delta 60 2147483648 Not enabled Falling Threshold event 1 4 1 4 0 4 1 4 0 4 1 4 524288000 4 524288000 4 Falling Threshold event 1 i 1 4 0 4 1 4 0 4 1 4 524288000 4 524288000 4 switch show port monitor active Policy Name sample Admin status Active Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Oper status Port type Active All Access Ports Verifying Interfaces Configuration W Counter portguard Link Loss enabled Sync Loss enabled Protocol Error enabled Signal Loss enabled Invalid Words enabled Invalid CRC s enabled RX Performance enabled TX Performance enabled Threshold Delta Delta Delta Delta Delta Delta Interval Rising Threshold event Falling Threshold event 60 60 60 60 60 60 5 4 1 4 5 4 1 4 1 4 0 4 5 4 1 4 1 4 0 4 5 4 d 4 21474836
32. 28 description 2 11 2 28 B port mode description 2 5 interface modes 2 5 bridge port mode See B port mode buffer pools Generation 2 switching modules 4 2 buffer to buffer credits See BB_credits buffer to buffer start change See BB_SC Cc Cisco MDS 92161 switches configuring extended BB_credits 4 18 configuring NPV 7 10 D dedicated rate mode description 3 7 migrating from shared rate mode 3 26 migrating to shared rate mode 3 27 destination IDs exchange based 6 5 flow based 6 4 domain IDs assignment failures 2 9 domain manager isolation 2 9 downgrading disabling ACL adjacency sharing 3 43 dynamic bandwidth management description 3 10 EISLs E Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x PortChannel links 6 2 enhanced ISLs See EISLs E port mode classes of service 2 3 description 2 3 E ports 32 port guidelines 2 18 32 port switching module configuration guidelines 6 14 configuring 2 22 isolation 2 9 exchange link parameter See ELP expansion port mode See E port mode extended BB credits configuring 4 23 displaying information 4 25 Generation 2 switching modules 4 17 licensing 4 17 F fabric loop port mode See FL port mode fabric port mode See F port mode fcdomains overlap isolation 2 9 Fiber Channel interfaces configuring system default port mode f 2 23 Fibre Channel interfaces administrative states 2 6 BB credits 4 1 confi
33. 3 6 Bandwidth Reserved for the Port Speeds on Generation 2 Switching Modules Configured Speed Reserved Bandwidth Auto 4 Gbps 4 Gbps Auto with 2 Gbps maximum 2 Gbps 2 Gbps 1 Gbps 1 Gbps S Note The 4 Port 10 Gbps Fibre Channel module ports in auto mode only support auto speed mode at 10 Gbps Shared Rate Mode Cisco MDS 9000 Family NX OS Interfaces Configuration Guide When port rate mode is configured as shared multiple ports within a port group share data paths to the switch fabric so that fabric bandwidth and related resources are shared Often the available bandwidth to the switch fabric may be less than the negotiated operating speed of a port Ports in this mode use local buffering for the BB_credit buffers OL 29284 01 Release 6 x _ Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces W All ports in 8 Gbps Advanced Fibre Channel switching modules where bandwidth is shared support 1 Gbps 2 Gbps 4 Gbps or 8 Gbps traffic However it is possible to configure one or more ports in a port group to operate in dedicated rate mode with 1 Gbps 2 Gbps 4 Gbps or 8 Gbps operating speed All ports in 4 Gbps Fibre Channel switching modules where bandwidth is shared support 1 Gbps 2 Gbps or 4 Gbps traffic However it is possible to configure one or more ports in a port group to operate in dedicated rate mode with 1 Gbps 2 Gbps or 4 Gbps operating speed All ports in the 32 Port
34. 4 0 shared 4 0 shared 4 0 shared 4 0 shared When you want to reconfigure the interfaces in a port group on a Generation 2 or Generation 3 module you can return the port group to the default configuration to avoid problems with allocating shared resources e The interface cannot be a member of a PortChannel e Releasing shared resources disrupts traffic on the port Traffic on other ports in the port group is not affected Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter3 Configuring Fibre Channel Interfaces Detailed Steps Step 1 Step 2 Step 3 Step 4 Step5 Step 6 Verifying Fibre Channel Interfaces Configuration W To release the shared resources for a port group follow these steps Command Purpose switch config t Enters configuration mode switch config switch config interface fc 1 1 Selects the interface and enters interface switch config if configuration submode Tip You can use an interface range to release the resources for all interfaces in a port group switch config if no channel group Removes the interface from a PortChannel switch config if shutdown Disables the interface switch config if out of service Takes the interface out of service Putting an interface into out of service will cause its shared resource configuration to revert to default Do you wish to continue y n n y switch config
35. 6 NPV CFS Distribution over IP 7 7 NPV Traffic Management 7 7 Auto 7 7 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide 6 17 I OL 29284 01 Release 6 x Contents W E Contents Traffic Map 7 7 Disruptive 7 8 Multiple VSAN Support 7 8 Guidelines and Limitations 7 8 NPV Guidelines and Requirements 7 8 NPV Traffic Management Guidelines 7 9 DPVM Configuration Guidelines 7 9 NPV and Port Security Configuration Guidelines 7 10 Configuring N Port Virtualization 7 10 Enabling N Port Identifier Virtualization 7 10 Configuring NPV 7 10 Configuring NPV Traffic Management 7 12 Configuring List of External Interfaces per Server Interface 7 12 Enabling the Global Policy for Disruptive Load Balancing 7 13 Verifying NPV Configuration 7 13 Verifying NPV 7 14 Verifying NPV Traffic Management 7 15 CHAPTER 8 Configuring FlexAttach Virtual pWwWN 8 1 Information About FlexAttach Virtual pWWN 8 1 FlexAttach Virtual pWWN 8 1 Difference Between San Device Virtualization and FlexAttach Port Virtualization 8 2 FlexAttach Virtual pWWN CFS Distribution 8 2 Security Settings for FlexAttach Virtual pWWN 8 3 Guidelines and Limitations 8 3 Configuring FlexAttach Virtual pWWN 8 3 Automatically Assigning FlexAttach Virtual pWWN 8 3 Manually Assigning FlexAttach Virtual pWWN _ 8 4 Mapping pWWN to Virtual pWWN 8 4 Verifying FlexAttach Virtual pWWN Configuration 8 5 Verifying the End Device 8 6 Monitoring FlexAttach Virtual pWWN 8 6
36. BB Credit Buffer Allocation ISL Fx Port ISL Fx Port Default BB_credit buffers 250 16 16 16 Maximum BB credit buffers 250 250 16 16 Total number of BB_credit buffers per module 4509 1 ISL E port or TE port The following considerations apply to BB_credit buffers on18 port 4 Gbps Fibre Channel switching modules e BB credit buffers for ISL connections can be configured from a minimum of 2 buffers to a maximum of 250 buffers for dedicated rate mode or 16 buffers for shared rate mode e BB credit buffers for Fx port mode connections can be configured The minimum is 2 buffers and the maximum of 250 buffers for dedicated rate mode or 16 buffers for shared rate mode e Performance buffers are not supported on this module 12 Port 4 Gbps Switching Module BB_Credit Buffers Table 4 8 lists the BB_credit buffer allocation for 12 port 4 Gbps switching modules Table 4 8 12 Port 4 Gbps Switching Module BB_Credit Buffer Allocation BB_Credit Buffers Per Port Dedicated Rate Mode 4 Gbps Speed BB_Credit Buffer Allocation Type ISL Fx Port Default BB_credit buffers 250 16 Maximum BB credit buffers 250 16 Default Performance buffers 145 12 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x EEN Chapter4 Configuring Interface Buffers Information About Interface Buffers Note Table 4 8 12 Port 4 Gbps Switching Module BB_Credit Buffer Allocation continued BB_Credit B
37. Command Purpose switch config t Enters configuration mode switch config interface fc1 1 Selects a Fibre Channel interface and enters interface switch config if configuration submode Note When a Fibre Channel interface is configured it is automatically assigned a unique world wide name WWN If the interface s operational state is up it is also assigned a Fibre Channel ID FC ID To configure a range of interfaces follow these steps Command Purpose switch config t Enters configuration mode switch config interface fc1 1 4 fc2 1 3 Selects the range of Fibre Channel interfaces switch config if and enters interface configuration submode3 Note In this command provide a space before and after the comma For the Cisco Fabric Switch for HP c Class BladeSystem and the Cisco Fabric Switch for IBM BladeCenter you can configure a range of interfaces among internal ports or external ports but you cannot mix both interface types within the same range For example bay 1 10 bay 12 or ext 0 ext 15 18 are valid ranges but bay 1 5 ext 15 17 is not I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces HZ Configuring Interfaces Setting the Interface Administrative State Detailed Steps To gracefully shut down an interface follow these steps Command Purpose Ste
38. Credit is 16 Receive data field Size is 2112 Beacon is turned off 5 minutes input rate 0 bits sec 0 bytes sec 5 minutes output rate 0 bits sec 0 bytes sec 134 frames input 8468 bytes 0 discards 0 errors 0 CRC 0 unknown class 0 too long 0 too short 154 frames output 46072 bytes 0 discards 0 errors 1 input OLS 1 LRR 0 NOS 0 loop inits 1 output OLS 0 LRR 1 NOS 0 loop inits 16 receive B2B credit remaining 3 transmit B2B credit remaining fcl 9 is trunking Port WWN is 20 09 00 05 30 00 97 9e Peer port WWN is 20 0b 00 0b 5f a3 cc 00 Admin port mode is E trunk mode is on Port mode is TE Port vsan is 100 Speed is 2 Gbps Transmit B2B Credit is 255 Receive B2B Credit is 255 Receive data field Size is 2112 Beacon is turned off 5 minutes input rate 280 bits sec 35 bytes sec 5 minutes output rate 176 bits sec 22 bytes sec 4609939 frames input 8149405708 bytes 0 discards 0 errors Hardware is Fibre Channel SFP is short wave laser 0 frames sec 0 frames sec Hardware is Fibre Channel SFP is long wave laser cost reduced Trunk vsans admin allowed and active 1 100 3000 Trunk vsans up 1 100 3000 Trunk vsans isolated Trunk vsans initializing 0 frames sec 0 frames sec E Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Verifying Interfaces Configuration 0 CRC 0 unknown class 0 too
39. Dynamic negotiation of operational trunk mode Selection of a common set of trunk allowed VSANs Detection of a VSAN mismatch across an ISL Table 5 1 specifies the protocols used for trunking and channeling I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter5 Configuring Trunking W Information About Trunking Table 5 1 Supported Trunking Protocols Trunk Link Default TE TE port link Cisco EPP PTP TF TN port link FC LS Rev 1 62 EVFP TF TNP port link Cisco EPP PTP E or F PortChannel Cisco EPP PCP TF Port Channel Cisco EPP PTP and PCP Third party TF TNP port link FC LS Rev 1 62 EVFP 1 These features are not currently supported By default the trunking protocol is enabled on E ports and disabled on F ports If the trunking protocol is disabled on a switch no port on that switch can apply new trunk configurations Existing trunk configurations are not affected The TE port continues to function in trunk mode but only supports traffic in VSANs that it negotiated with previously when the trunking protocol was enabled Also other switches that are directly connected to this switch are similarly affected on the connected interfaces In some cases you may need to merge traffic from different port VSANs across a non trunking ISL If so disable the trunking protocol Note We recommend that both ends of a trunking link belong to the same port
40. E ports into one logical interface to provide higher aggregated bandwidth load balancing and link redundancy PortChannels can connect to interfaces across switching modules so a failure of a switching module cannot bring down the PortChannel link A PortChannel has the following features and restrictions Provides a point to point connection over ISL E ports or EISL TE ports Multiple links can be combined into a PortChannel Increases the aggregate bandwidth on an ISL by distributing traffic among all functional links in the channel Load balances across multiple links and maintains optimum bandwidth utilization Load balancing is based on the source ID destination ID and exchange ID OX ID Cisco MDS 9000 Family NX OS Interfaces Configuration Guide E OL 29284 01 Release 6 x Chapter6 Configuring PortChannels Information About PortChannels W e Provides high availability on an ISL If one link fails traffic previously carried on this link is switched to the remaining links If a link goes down in a PortChannel the upper protocol is not aware of it To the upper protocol the link is still there although the bandwidth is diminished The routing tables are not affected by link failure PortChannels may contain up to 16 physical links and may span multiple modules for added high availability amp Note See the Cisco MDS 9000 Family NX OS Fabric Configuration Guide for information about failover scenarios for PortC
41. Exits configuration mode switch copy running config startup config Saves the new oversubscription ratio configuration to the startup configuration and then the new configuration is enforced upon subsequent reboots of the module Use the show running config command to view oversubscription ratios for a module If oversubscription ratios are enabled then no restriction appears in the output Example 3 1 Module with Restrictions on Oversubscription Ratios Disabled switch show running config version 3 1 1 no rate mode oversubscription limit module 2 interface fc2 1 switchport speed 2000 interface fc2 1 To disable restrictions on oversubscription ratios for ports on a 48 port Generation 2 switch that is configured with both shared and dedicated ports follow these steps To disable restrictions on oversubscription ratios you must shut down any shared ports Use the show port resources command to view the configuration on a module and to identify shared ports switch show port resources module 2 Module 2 Available dedicated buffers are 4656 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces HI Configuring Fibre Channel Interfaces Port Group 1 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Bandwidth Rate Mode Buf
42. Family NX OS Interfaces Configuration Guide Chapter6 Configuring PortChannels HI Prerequisites for PortChannels e Newly created PortChannels are allocated from the maximum possible PortChannel 128 for Generation 1 or a combination of Generation 1 and Generation 2 switches or 256 for Generation 2 switches in a decreasing order based on availability If all 128 or 256 numbers are used up aggregation is not allowed e You cannot change the membership or delete an autocreated PortChannel e When you disable autocreation all member ports are removed from the autocreated PortChannel e Once the last member is removed from an autocreated PortChannel the channel is automatically deleted and the number is released for reuse e An autocreated PortChannel is not persistent through a reboot An autocreated PortChannel can be manually configured to appear the same as a persistent PortChannel Once the PortChannel is made persistent the autocreation feature is disabled in all member ports e You can enable or disable the autocreation feature on a per port basis or for all ports in the switch When this configuration is enabled the channel group mode is assumed to be active The default for this task is disabled e If autocreation of channel groups is enabled for an interface you must first disable autocreation before downgrading to earlier software versions or before configuring the interface in a manually configured channel group
43. Gbps When migrating a host that supports up to 4 Gbps traffic that is not 8 Gbps with autosensing capabilities to the 8 Gbps switching modules use autosensing with a maximum bandwidth of 4 Gbps Frame Encapsulation The switchport encap eisl command only applies to SD port interfaces This command determines the frame format for all frames transmitted by the interface in SD port mode If the encapsulation is set to EISL all outgoing frames are transmitted in the EISL frame format regardless of the SPAN sources The switchport encap eisl command is disabled by default If you enable encapsulation all outgoing frames are encapsulated and you will see a new line Encapsulation is eisl in the show interface SD_port_interface command output See the Cisco MDS 9000 Family NX OS System Management Configuration Guide Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Eu OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Information About Interfaces W You can set the frame format to EISL for all frames transmitted by the interface in SD port mode If you sent the frame encapsulation to EISL all outgoing frames are transmitted in the EISL frame format regardless of the SPAN sources See the Cisco MDS 9000 Family NX OS System Management Configuration Guide Beacon LEDs Figure 2 2 displays the status link and speed LEDs in a 16 port switching module Figure 2 2 Cisco MDS 9000 Family Switch Interface Mod
44. Gbps Fabric Switches e BB_credit buffers can be configured from a minimum of 1 buffer to a maximum of 32 buffers per port when the ports are in F or FL mode e BB_credit buffers can be configured from a minimum of 2 buffers to a maximum of 32 buffers per port when the ports are in E or TE mode Cisco MDS 9134 Fabric Switch BB_Credit Buffers Table 4 11 lists the BB_credit buffer allocation for MDS 9134 Fabric Switches Table 4 11 MDS 9134 Fabric Switch BB_Credit Buffer Allocation BB_Credit Buffers Per BB_Credit Buffer Allocation Type Port Group BB_Credit Buffers Per Port ISL Fx Port Maximum user configurable BB_credit buffers 64 61 61 Minimum user configurable BB_credit buffers 2 1 Default BB_credit buffers on 10 Gbps mode 64 64 64 Default BB_credit buffers on 4 Gbps mode 64 16 16 1 ISL E port or TE port Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x _ Chapter 4 Configuring Interface Buffers Information About Interface Buffers W Cisco MDS 9124 Fabric Switch BB_ Credit Buffers Table 4 12 lists the BB_credit buffer allocation for MDS 9124 Fabric Switches Table 4 12 MDS 9124 Fabric Switch BB_Credit Buffer Allocation Defaults BB_Credit Buffers Per BB_Credit Buffers Per Port BB_Credit Buffer Allocation Type Port Group Defaults ISL Fx Port Maximum user configurable BB_credit buffers 64 61 61 Minimum user configurable BB_cr
45. Guide Pax OL 29284 01 Audience Preface This preface describes the audience organization and conventions of the Cisco MDS 9000 Family NX OS Interfaces Configuration Guide It also provides information on how to obtain related documentation This guide is for experienced network administrators who are responsible for configuring and maintaining the Cisco MDS 9000 Family of multilayer directors and fabric switches Organization The Cisco MDS 9000 Family NX OS Interfaces Configuration Guide is organized as follows Chapter Title Description Chapter 1 Interfaces Overview Provides an overview of all the features in this guide Chapter 2 Configuring Interfaces Explains Generation 1 and Generation 2 module port and operational state concepts in Cisco MDS 9000 Family switches and provides details on configuring ports and interfaces Chapter 3 Configuring Fibre Channel Interfaces Explains configuration concepts for Fibre Channel module ports and interfaces Chapter 4 Configuring Interface Buffers Explains configuration concepts for Interface Buffers Chapter 5 Configuring Trunking Explains TE ports and trunking concepts Chapter 6 Configuring PortChannels Explains PortChannels and load balancing concepts and provides details on configuring PortChannels adding ports to PortChannels and deleting ports from PortChannels I OL 29284 01 Cisco MDS 9000 Family 1 0 Accelerator Configuration Guide
46. Size is 2112 Beacon is turned off Port tracked with interface port channel 1 vsan 2 trunking lt Tracked port Port tracked with interface fcip 5 lt nn n nnnnncnnnnnn rene nnn nesses nn ns en enn n Tracked port 5 minutes input rate 0 bits sec 0 bytes sec 0 frames sec 5 minutes output rate 0 bits sec 0 bytes sec 0 frames sec Cisco MDS 9000 Family NX OS Quality of Service Configuration Guide lt OL 29284 01 Chapter 9 Configuring Port Tracking Displaying Port Tracking Information 269946 frames input 22335204 bytes 0 discards 0 errors 0 CRC 0 unknown class 0 too long 0 too short 205007 frames output 10250904 bytes 0 discards 0 errors input OLS 0 LRR 0 NOS 0 loop inits output OLS 2 LRR 0 NOS 1 loop inits receive B2B credit remaining transmit B2B credit remaining oO NO Example 9 2 Displays a Tracked Port Configuration for a Fibre Channel Interface switch show interface fcl1 1 fcl 1 is down Administratively down Hardware is Fibre Channel FCOT is short wave laser w o OFC SN Port WWN is 20 01 00 05 30 00 0d de Admin port mode is FX Port vsan is 1 Receive data field Size is 2112 Beacon is turned off Port tracked with interface fcl 2 down Port tracked with interface port channel 1 vsan 2 down Port tracked with interface fcip1 down 5 minutes input rate 0 bits sec 0 bytes sec 0 frames sec 5 minutes output rate 0 bits sec 0 bytes sec 0 frames sec 1 frames input 128 bytes
47. Step 9 Select the interfaces fc 4 7 through fc 4 10 Configuration Examples for Fibre Channel Interfaces switch config t switch config interface fc 4 7 10 Configure the port speed rate mode and port mode on the interfaces switch config if switchport speed 1000 switch config if switchport rate mode shared switch config if switchport mode f Enable the interfaces and return to configuration mode switch config if no shutdown switch config if exit switch Configuration Example for 24 Port 4 Gbps Module Interfaces Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 These steps describe how to configure the example shown in Figure 4 8 on page 4 13 Select interfaces fc 3 1 through fc 3 3 swi swi tch tch config t config interface fc 3 1 3 Configure the port speed rate mode and port mode on the interfaces swi swi swi tch tch tch Enable swi swi swi tch tch tch config if config if config if the interface config if config if switchport speed 4000 switchport rate mode dedicated switchport mode e s and return to configuration mode no shutdown exit Select the interfaces fc 3 4 through fc 3 6 swi swi tch tch config t config interface fc 3 4 6 Configure the port speed rate mode and port mode on the interfaces swi swi swi tch tch tch Enable swi swi swi tch tch tch config if
48. and local buffer pools The receive buffers allocated from the global buffer pool to be shared by a port group are called a global receive buffer pool Global receive buffer pools include the following buffer groups e Reserved internal buffers e Allocated BB_credit buffers for each Fibre Channel interface user configured or assigned by default e Common unallocated buffer pool for BB_credits if any to be used for additional BB_credits as needed e Performance buffers only used on 12 port 4 Gbps and 4 port 10 Gbps switching modules Cisco MDS 9000 Family NX OS Interfaces Configuration Guide P42 E OL 29284 01 Release 6 x Chapter4 Configuring Interface Buffers Information About Interface Buffers W Note The 48 port and 24 port 8 Gbps modules have dual global buffer pools Each buffer pool in the 48 port modules support 24 ports and in the 24 port modules each buffer pool supports 12 ports Figure 4 1 shows the allocation of BB_credit buffers on line cards 24 port and 48 port 4 Gbps line cards Figure 4 1 Receive Buffers for Fibre Channel Ports in a Global Buffer Pool A Reserved Internal Buffers not user configurable o 2 9 Performance Buffers Shared Pool Ess Common unallocated buffer pool for BB_credits D gja a S 3 Allocated BB_credit buffers T5 1 2 3 for each front panel FC ports N T a y D 2 Figure 4 2 shows the default BB_credit buffer allocation model for 48
49. and the Device Manager create the PortChannel in ON mode in the NPIV core switches and ACTIVE mode on the NPV switches DCNM SAN creates all PortChannels in ACTIVE mode We recommend that you create PortChannels in ACTIVE mode e An F PortChannel is supported only on ACTIVE mode To configure ACTIVE mode follow these steps Command Purpose switch config t switch config Enters configuration mode switch config interface port channel 1 switch config if Configures the specified PortChannel 1 using the default ON mode I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter6 Configuring PortChannels HZ Configuring PortChannels Command Purpose Step 3 switch config if channel mode active Configures the ACTIVE mode switch config if no channel mode active Reverts to the default ON mode Deleting PortChannels Detailed Steps To delete a PortChannel follow these steps Command Purpose Step 1 switch config t Enters configuration mode switch config Step2 switch config no interface port channel 1 Deletes the specified PortChannel 1 its port channel 1 deleted and all its members associated interface mappings and the disabled aoe hardware associations for this PortChannel please do the same operation on the switch at the other end of the port channel switch config Adding an Interfac
50. brought down indicating that no frames are lost when the interface is going down see the Graceful Shutdown section on page 2 9 Interface Deletion from a PortChannel When a physical interface is deleted from the PortChannel the channel membership is automatically updated If the deleted interface is the last operational interface then the PortChannel status is changed to a down state Deleting an interface from a PortChannel decreases the channel size and bandwidth of the PortChannel e Ifyou use the default ON mode to avoid inconsistent states across switches and to maintain consistency across switches then the ports shut down You must explicitly enable those ports again e Ifyou use the ACTIVE mode then the PortChannel ports automatically recover from the deletion After the members are deleted regardless of the mode ACTIVE and ON used the ports at either end are gracefully brought down indicating that no frames are lost when the interface is going down see the Generation 1 PortChannel Limitations section on page 6 14 and Graceful Shutdown section on page 2 9 PortChannel Protocols In earlier Cisco SAN OS releases PortChannels required additional administrative tasks to support synchronization The Cisco NX OS software provides robust error detection and synchronization capabilities You can manually configure channel groups or they can be automatically created In both cases the channel groups have the sam
51. config Step2 switch config interface fc1 1 Selects a Fibre Channel interface and enters switch config if interface configuration submode Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x ZN Chapter4 Configuring Interface Buffers W Configuring Interface Buffers Step 3 Step 4 Command Purpose switch config if switchport fcrxbbcredit default Applies the default operational value to the selected interface The operational value depends on the port mode The default values are assigned based on the port capabilities switch config if switchport fcerxbbcredit 5 Assigns a BB_credit of 5 to the selected interface The range to assign BB_credits is between 1 and 255 switch config if switchport fcerxbbcredit 5 mode E Assigns this value if the port is operating in E or TE mode The range to assign BB_credits is between 1 and 255 switch config if switchport fcrxbbcredit 5 mode Fx Assigns this value if the port is operating in F or FL mode The range to assign BB_credits is between 1 and 255 switch config if do show int fc1 1 Displays the receive and transmit BB_credit along with other pertinent interface information for this interface Note The BB credit values are correct at the time the registers are read They are useful to verify situations when the data traffic is slow This example shows the output of the show
52. connected to a third party switch the trunking protocol ensures seamless operation as an E port e Trunking F ports and trunking F PortChannels are not supported on the following hardware 91x4 switches if NPIV is enabled and used as the NPIV core switch Generation 1 2 Gbps Fibre Channel switching modules e On core switches the FC SP authentication will be supported only for the physical FLOGI from the physical pWWN Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x a 57 Chapter5 Configuring Trunking HZ Guidelines and Limitations amp No FC SP authentication is supported by the NPV switch on the server F ports MDS does not enforce the uniqueness of logical pWWNs across VSANs DPVM is not supported on trunked F port logins The DPVM feature is limited to the control of the port VSAN since the EVFP protocol does not allow changing the VSAN on which a logical pWWN has done FLOGI The port security configuration will be applied to both the first physical FLOGI and the per VSAN FLOGIs Trunking is not supported on F ports that have FlexAttach enabled On MDS 91x4 core switches hard zoning can be done only on F ports that are doing either NPIV or trunking However in NPV mode this restriction does not apply since zoning is enforced on the core F port Note Fibre Channel Security Protocol FC SP is not supported for 6 2 1 release on MDS 9710 but targeted for a future r
53. default behavior PortChannel Limitations PortChannels have the following restrictions I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide ii Chapter 3 Configuring Fibre Channel Interfaces HE Guidelines and Limitations Note e The maximum number of PortChannels allowed is 256 if all switching modules are Generation 2 or Generation 3 or both e The maximum number of PortChannels allowed is 128 whenever there is a Generation 1 switching module in use with a Generation 2 or Generation 3 switching module e Ports need to be configured in dedicated rate mode on the Generation 2 and Generation 3 switching module interfaces to be used in the PortChannel The number of PortChannels allowed does not depend on the type of supervisor module However Generation 3 modules require the Supervisor 2 module on the MDS 9506 and 9509 switches The Generation1l Generation 2 and Generation 3 modules have the following restrictions for PortChannel configuration e Generation 1 switching module interfaces do not support auto speed with a maximum of 2 Gbps e Generation 1 and Generation 2 module interfaces do not support auto speed with maximum of 4 Gbps e Generation 2 and Generation 3 switching module interfaces cannot be forcefully added to a PortChannel if sufficient resources are not available Note Before adding a Generation 2 or Generation 3 interface to a PortChannel use th
54. device module Cisco MDS 9124 Cisco MDS 9134 or Cisco MDS 9148 Fabric Switch The module or switch is rebooted and when it comes back up is in NPV mode Note A write erase is performed during the reboot switch config interface fe 1 1 On the NPV device selects the interfaces that au leeo conti g tera CeRpere mode ny will be connected to the aggregator switch and configure them as NP ports switch config if no shutdown Changes Admin status to bring up the interfaces switch config vsan database Configures the port VSANs for the NP port on switch config vsan db vsan 9 interface fc the NPV device 1 1 switch config vsan db vsan 11 interface fc 1 1 switch config if exit Exits interface mode for the port switch config interface fc 1 2 6 Selects the remaining interfaces 2 through 6 eC Er DkCOu tA gett switchport mode F on the NPV enabled device and configures them as F ports switch config if no shutdown Changes Admin status to bring up the interfaces I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter7 Configuring N Port Virtualization WE Configuring N Port Virtualization Step 10 Step 11 Command Purpose switch config vsan database switch config vsan db vsan 12 interface fc 1 1 6 switch config vsan db vsan 13 interface fc 1 1 6 Configures the port VSANs for the F ports on the NPV device switch
55. discusses the following aspects of load balancing e Auto page 7 7 e Traffic Map page 7 7 e Disruptive page 7 8 Auto Before Cisco MDS SAN OS Release 3 3 1a NPV supported automatic selection of external links When a server interface is brought up an external interface with the minimum load is selected from the available links There is no manual selection on the server interfaces using the external links Also when a new external interface was brought up the existing load was not distributed automatically to the newly available external interface This newly brought up interface is used only by the server interfaces that come up after this interface Traffic Map As in Cisco MDS SAN OS Release 3 3 1a and NX OS Release 4 1 1a NPV supports traffic management by allowing you to select and configure the external interfaces that the server uses to connect to the core switches Note When the NPV traffic management is configured the server uses only the configured external interfaces Any other available external interface will not be used The NPV traffic management feature provides the following benefits e Facilitates traffic engineering by providing dedicated external interfaces for the servers connected to NPV e Uses the shortest path by selecting external interfaces per server interface e Uses the persistent FC ID feature by providing the same traffic path after a link break or reboot of the NPV or core switch e
56. down 10 1 1 2 8 GigabitEthernet4 7 down Toz T 27 24 GigabitEthernet4 8 down Interface Status Oper Mode iscsi4 1 down Interface Status sup fc0 up Interface Status IP Address mgmt 0 up 172 19 48 96 25 Interface Vsan Admin Status Trunk Mode port channel 1 1 on trunking port channel 2 1 on trunking Interface Vsan Admin Admin Status Oper Mode Trunk Mode Mode fcipl0 eli auto on notConnected Example 2 8 Displays Interface Counters switch show interface counters c3 1 5 minutes input rate 24 bits sec 5 minutes output rate 16 bits sec 3502 frames input 268400 bytes 0 discards 0 CRC 0 unknown class 0 too long 0 too short 3505 frames output 198888 bytes 0 discards 1 input OLS 1 LRR 2 output OLS 1 LRR 1 link failures 3 bytes sec 1 NOS O loop inits 1 NOS 0 loop inits 1 sync losses c9 8 5 minutes input rate 0 bits sec 5 minutes output rate 0 bits sec 0 frames input 0 bytes class 2 frames 0 bytes class 3 frames 0 bytes class f frames 0 bytes discards 0 CRC 0 unknown class too long 0 too short 0 bytes sec 0 bytes sec oo oo o amp 2 bytes sec 1 signal losses 0 frames sec 0 frames sec 0 frames sec 0 frames sec E Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter 2 Configuring Interfaces Verifying Interfaces Configuration 0 frames output 0 bytes 0 class 2 frames 0 bytes 0 class 3 frames 0 bytes 0 cl
57. for 24 port 4 Gbps Fibre Channel switching modules Table 4 6 24 Port 4 Gbps Switching Module BB_Credit Buffer Allocation BB_Credit Buffers Per Port Dedicated Rate Mode Shared Rate Mode 4 Gbps Speed 4 Gbps Speed BB_Credit Buffer Allocation ISL Fx Port Fx Port Default BB_credit buffers 250 16 16 Maximum BB credit buffers 250 250 16 Total number of BB_credits buffers per module 6000 1 ISL E port or TE port The following considerations apply to BB_credit buffers on 24 port 4 Gbps Fibre Channel switching modules e BB credit buffers for ISL connections can be configured from a minimum of 2 buffers to a maximum of 250 buffers for dedicated rate mode or 16 buffers for shared rate mode e BB_credit buffers for Fx port mode connections can be configured The minimum is 2 buffers and the maximum of 250 buffers for dedicated rate mode or 16 buffers for shared rate mode e Performance buffers are not supported on this module Each port group on the 24 port 4 Gbps Fibre Channel switching module consists of six ports The ports in shared rate mode have a bandwidth oversubscription of 2 1 by default However some configurations of the shared ports in a port group can have a maximum bandwidth oversubscription of 4 1 considering that each port group has 12 8 Gbps bandwidth The following example configurations are supported by the 24 port 4 Gbps Fibre Channel switching modules e Six ports with shared rate mode and 4
58. have just entered the switch or would have stayed in the switch for the configured timeout value These are preemptive drops and will clear the congestion completely compared to the stuck frame timeout value No credit timeout feature is disabled by default We recommend that you retain the default configuration for ISLs and configure a value not exceeding 500 ms 200 to 300 ms for fabric F ports I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 2 Configuring Interfaces HZ Configuring Interfaces wy Note Step 1 Step 2 Configuring Credit Loss Recovery Threshold and Action The no credit timeout value and stuck frame timeout value are interlinked The no credit timeout value must always be greater than the stuck frame timeout value To configure the no credit timeout value follow these steps Command Purpose switch config t Enters configuration mode switch config system timeout no credit drop seconds mode E F Specifies the no credit timeout value and port mode for the switch switch config system timeout no credit drop default mode E F Specifies the default no credit timeout value port mode for the switch When the port detects the credit loss condition and recovers then the port can be error disabled a trap can be sent with interface details and a syslog can be generated with interface details When the configured th
59. int fc1 1 command intfcl 1 is up 16 receive B2B credit remaining 3 transmit B2B credit remaining Configuring Performance Buffers Detailed Steps Step 1 Step 2 Step 3 To configure performance buffers for a Fibre Channel interface follow these steps Command Purpose switch config t Enters configuration mode switch config interface fc1 1 switch config if Selects a Fibre Channel interface and enters interface configuration submode switch config if switchport fcrxbbcredit performance buffers 45 Assigns a performance buffer of 45 to the selected interface The value ranges from 1 to 145 switch config if switchport fcrxbbcredit performance buffers default Reverts to the factory default of using the built in algorithm Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x _ Chapter 4 Configuring Interface Buffers wy Configuring Interface Buffers W Note Use the show interface bbcredit command to display performance buffer values and other BB_credit information Configuring Extended BB_credits Detailed Steps Step 1 Step 2 Step 3 Step 4 Step5 To configure extended BB_credits for a MDS 14 2 interface for a Generation 2 switching module interface not including the Cisco MDS 9124 Fabric Switch or for an interface in a Cisco MDS 92161 switch follow these steps Command Purpose
60. is automatically calculated by the switch For internal ports on the Cisco Fabric Switch for HP c_Class BladeSystem and Cisco Fabric Switch for IBM BladeCenter a port speed of 1 Gbps is not supported Auto negotiation is supported between 2 Gbps and 4 Gbps only Also if the BladeCenter is a T chassis then port speeds are fixed at 2 Gbps and auto negotiation is not enabled Autosensing Autosensing speed is enabled on all 4 Gbps and 8 Gbps switching module interfaces by default This configuration enables the interfaces to operate at speeds of 1 Gbps 2 Gbps or 4 Gbps on the 4 Gbps switching modules and 8 Gbps on the 8 Gbps switching modules When autosensing is enabled for an interface operating in dedicated rate mode 4 Gbps of bandwidth is reserved even if the port negotiates at an operating speed of 1 Gbps or 2 Gbps To avoid wasting unused bandwidth on 48 port and 24 port 4 Gbps and 8 Gbps Fibre Channel switching modules you can specify that only 2 Gbps of required bandwidth be reserved not the default of 4 Gbps or 8 Gbps This feature shares the unused bandwidth within the port group provided that it does not exceed the rate limit configuration for the port You can also use this feature for shared rate ports that are configured for autosensing Tip When migrating a host that supports up to 2 Gbps traffic that is not 4 Gbps with autosensing capabilities to the 4 Gbps switching modules use autosensing with a maximum bandwidth of 2
61. long 0 too short 4638491 frames output 7264731728 bytes 0 discards 0 errors 3 input OLS 9 LRR 1 NOS O loop inits 9 output OLS 7 LRR 1 NOS 0 loop inits 16 receive B2B credit remaining 3 transmit B2B credit remaining fc1 13 is up Hardware is Fibre Channel SFP is short wave laser Port WWN is 20 0d 00 05 30 00 97 9e Admin port mode is auto trunk mode is on Port mode is F FCID is 0x650100 Port vsan is 100 Speed is 2 Gbps Transmit B2B Credit is 3 Receive B2B Credit is 16 Receive data field Size is 2112 Beacon is turned off 5 minutes input rate 0 bits sec 0 bytes sec 0 frames sec 5 minutes output rate 0 bits sec 0 bytes sec 0 frames sec 8696 frames input 3227212 bytes 0 discards 0 errors 0 CRC 0 unknown class 0 too long 0 too short 16799 frames output 6782444 bytes 0 discards 0 errors 0 input OLS 0 LRR 0 NOS 0 loop inits 1 output OLS 1 LRR 0 NOS 1 loop inits 16 receive B2B credit remaining 3 transmit B2B credit remaining sup fcO is up Hardware is Fibre Channel Speed is 1 Gbps 139597 packets input 13852970 bytes 0 multicast frames 0 compressed 0 input errors 0 frame 0 overrun 0 fifo 139516 packets output 16759004 bytes 0 underruns 0 output errors 0 collisions 0 fifo 0 carrier errors You can also specify arguments a range of interfaces or multiple specified interfaces to display interface information You can specify a range of interfaces by issuing a command with the following example
62. only on NPV switches e Cisco Fabric Services CFS IP version 4 IPv4 distribution should be enabled e Virtual WWNs should be unique across the fabric Configuring FlexAttach Virtual pWWN This section includes the following topics e Automatically Assigning FlexAttach Virtual pWWN page 8 3 e Manually Assigning FlexAttach Virtual pWWN page 8 4 e Mapping pWWN to Virtual pWWN page 8 4 Automatically Assigning FlexAttach Virtual pWWN Automatic assignment of virtual pWWN can be configured on an NPV switch globally per VSAN or per port When assigned automatically a virtual WWN is generated from the device local switch WWN Prerequisites The port must be in a shut state when the virtual pWWN is enabled Detailed Steps To assign a virtual pWWN automatically perform this task Command Purpose Step1 switch config t Enters configuration mode Step 2 switch config flex attach virtual pwwn auto Assigns FlexAttach virtual pWWN linterface interface list automatically for the interfaces switch config flex attach virtual pwwn auto Assigns FlexAttach virtual pWWN vsan vsan range automatically for the VSANSs Step 3 switch config flex attach commit Commits the configuration Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x EN Chapter8 Configuring FlexAttach Virtual pWWN WE Configuring FlexAttach Virtual pWWN Troubleshooting Tips e When th
63. output errors 0 collisions 0 fifo 0 carrier errors Displaying VSAN Interface Information To display VSAN interface information use the show interface vsan command switch show interface vsan 2 vsan2 is up line protocol is up WWPN is 10 00 00 05 30 00 59 1f FCID is 0xb90100 Internet address is 10 1 1 1 24 MTU 1500 bytes BW 1000000 Kbit 0 packets input 0 bytes 0 errors 0 multicast 0 packets output 0 bytes 0 errors 0 dropped Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x mw 255 Chapter2 Configuring Interfaces W Verifying Interfaces Configuration Cisco MDS 9000 Family NX OS Interfaces Configuration Guide P2568 i OL 29284 01 Release 6 x CHAPTER Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces page 3 1 Guidelines and Limitations page 3 18 Default Settings page 3 25 Configuring Fibre Channel Interfaces page 3 26 Verifying Fibre Channel Interfaces Configuration page 3 43 Configuration Examples for Fibre Channel Interfaces page 3 45 Information About Fibre Channel Interfaces This section includes the following topics Generations of Modules and Switches page 3 1 Port Groups page 3 3 Port Rate Modes page 3 5 Port Speed page 3 10 Dynamic Bandwidth Management page 3 10 Out of Service Interfaces page 3 11 Oversubscription Ratio Restrictions page 3 11 Bandwidth Fairness page 3 17 Generations of Modules an
64. percentagei event event id falling threshold percentage event event id Specifies the delta protocol error poll interval in seconds the thresholds in percentage and the event IDs of events to be triggered switch config port monitor counter rx performance poll interval seconds delta rising threshold percentagei event event id falling threshold percentage event event id Specifies the delta Rx counter poll interval in seconds and thresholds in percentage switch config port monitor counter signal loss poll interval seconds delta rising threshold percentagei event event id falling threshold percentage event event id Specifies the delta signal loss poll interval in seconds the thresholds in percentage and the event IDs of events to be triggered switch config port monitor counter state change poll interval seconds delta rising threshold percentagel event event id falling threshold percentage event event id Specifies the delta state change poll interval in seconds the thresholds in percentage and the event IDs of events to be triggered switch config port monitor counter sync loss poll interval seconds delta rising threshold percentagei event event id falling threshold percentage event event id Specifies the delta sync loss poll interval in seconds the thresholds in percentage and the event IDs of events to be triggered switch config port monitor counter tx performance po
65. port modes of operation Note TL ports and SD ports cannot be configured automatically They must be administratively configured Note You cannot configure Fibre Channel interfaces on SSMs in auto mode Cisco MDS 9000 Family NX OS Interfaces Configuration Guide a OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Configuring Interfaces W Configuring System Default Port Mode F Restrictions Detailed Steps Step 1 Step 2 The system default switchport mode F command sets the administrative mode of all Fibre Channel ports to mode F while avoiding traffic disruption caused by the formation of unwanted Inter Switch Links ISLs This command is part of the setup utility that runs during bootup after a write erase or reload It can also be executed from the command line in configuration mode This command changes the configuration of the following ports to administrative mode F e All ports that are down and that are not out of service e All F ports that are up whose operational mode is F and whose administrative mode is not F This command does not affect the configuration of the following ports e All user configured ports even if they are down e All non F ports that are up however if non F ports are down this command changes the administrative mode of those ports e To ensure that ports that are part of ISLs do not get changed to port mode F configure the ports in port mode E rather than in A
66. switch config t switch config Enters configuration mode switch config ferxbbcredit extended enable Enables the extended BB credits feature switch config no ferxbbcredit extended enable Disables default the extended BB_credits feature switch config interface fc1 1 switch config if Selects a Fibre Channel interface and enters interface configuration submode switch config if switchport fcrxbbcredit extended 1500 Applies the extended BB_credit value of 1 500 credits to the selected interface The valid range is from 256 to 3 500 credits switch config if no switchport fcrxbbcredit extended 1500 Clears the configured extended BB_credit configuration for this port switch do show interface fc3 2 fc3 2 is trunking Hardware is Fibre Channel laser w o OFC SN Port WWN is 20 82 00 05 30 00 2a le Peer port WWN is 20 42 00 0b 46 79 f 1 80 Admin port mode is auto trunk mode is SFP is short wave on Port mode is TE Port vsan is 1 Speed is 2 Gbps Transmit B2B Credit is 255 Receive B2B Credit is 1500 Receive data field Size is 2112 Displays the receive and transmit BB_credit values along with other pertinent interface information for this interface if the interface is in the up state Note The receive BB credit value reflects the extended BB_credit configuration if applicable I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interf
67. take note Notes contain helpful suggestions or references to material not covered in the manual Caution Means reader be careful damage or loss of data In this situation you might do something that could result in equipment Cisco MDS 9000 Family 1 0 Accelerator Configuration Guide OL 29284 01 Related Documentation The documentation set for the Cisco MDS 9000 Family includes the following documents To find a document online use the Cisco MDS NX OS Documentation Locator at http www cisco com en US docs storage san_switches mds9000 roadmaps doclocater htm Release Notes e Cisco MDS 9000 Family Release Notes for Cisco MDS NX OS Releases e Cisco MDS 9000 Family Release Notes for MDS SAN OS Releases e Cisco MDS 9000 Family Release Notes for Cisco MDS 9000 EPLD Images e Cisco DCNM Release Notes Regulatory Compliance and Safety Information e Regulatory Compliance and Safety Information for the Cisco MDS 9000 Family Compatibility Information e Cisco Data Center Interoperability Support Matrix e Cisco MDS 9000 NX OS Hardware and Software Compatibility Information and Feature Lists e Cisco MDS 9000 Family Switch to Switch Interoperability Configuration Guide Hardware Installation e Cisco MDS 9500 Series Hardware Installation Guide e Cisco MDS 9200 Series Hardware Installation Guide e Cisco MDS 9100 Series Hardware Installation Guide e Cisco MDS 9124 and Cisco MDS 9134 Multilayer Fabric Switch Qu
68. the 48 port 8 Gbps Fibre Channel switching module to use the full 96 Gbps backplane bandwidth with any to any connectivity Generation 3 Fabric Switch DS C9148 K9 Cisco MDS 9148 Fabric switch 48 port 8 Gbps Fabric switch Generation 2 Modules DS X9148 48 port 4 Gbps Fibre Channel switching module DS X9124 24 port 4 Gbps Fibre Channel switching module DS X9304 18K9 18 port 4 Gbps Fibre Channel switching module with 4 Gigabit Ethernet ports Cisco MDS 9000 Family NX OS Interfaces Configuration Guide E OL 29284 01 Release 6 x _ Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Table 3 1 Fibre Channel Modules and Fabric Switches continued Part Number Product Name and Description DS X9112 12 port 4 Gbps Fibre Channel switching module DS X9704 4 port 10 Gbps Fibre Channel switching module DS X9530 SF2 K9 Supervisor 2 module for Cisco MDS 9500 Series switches Generation 2 Fabric Switches DS C9134 K9 Cisco MDS 9134 Fabric switch 32 port 4 Gbps Fabric switch with 2 additional 10 Gbps ports DS C9124 K9 Cisco MDS 9124 Fabric switch 24 port 4 Gbps Fabric switch Cisco MDS 92221 Multiservice Modular switch DS C92221 K9 18 port 4 Gbps switch with 4 Gigabit Ethernet IP storage services ports and a modular expansion slot to host Cisco MDS 9000 Family switching and services modules Note Port Groups Gen
69. the Cisco MDS 9000 Family NX OS Licensing Guide e Configure this feature in any port of the full rate 4 port group in either the Cisco MDS 9216i Switch or in the MPS 14 2 module see Figure 4 9 Figure 4 9 Port Group Support for the Extended BB_Credits Feature Group 1 Group 2 Group 3 T Extended credits not supported 120479 The port groups that support extended credit configurations are as follows Any one port in ports 1 to 4 identified as Group 1 Any one port in ports 5 to 8 identified as Group 2 Any one port in ports 9 to 12 identified as Group 3 amp Note The last two Fibre Channel ports port 13 and port 14 and the two Gigabit Ethernet ports do not support the extended BB_credits feature e Explicitly enable this feature in the required Cisco MDS switch e Disable the remaining three ports in the 4 port group if you need to assign more than 2 400 BB_credits to the first port in the port group Ifyou assign less than 2 400 extended BB_credits to any one port in a port group the remaining three ports in that port group can retain up to 255 BB_credits based on the port mode Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Pais OL 29284 01 Release 6 x Chapter4 Configuring Interface Buffers Information About Interface Buffers W Note The receive BB_credit value for the remaining three ports depends on the port mode The default value is 16 for
70. the Fx mode and 255 for E or TE modes The maximum value is 255 in all modes This value can be changed as required without exceeding the maximum value of 255 BB credits If you assign more than 2 400 up to a maximum of 3 500 extended BB_credits to the port in a port group you must disable the other three ports e If you change the BB_credit value the port is disabled and then reenabled Disable explicitly this feature if you need to nondisruptively downgrade to Cisco SAN OS Release 1 3 or earlier When you disable this feature the existing extended BB_credit configuration is completely erased amp Note The extended BB_credit configuration takes precedence over the receive BB_credit and performance buffer configurations Extended BB_credits on Generation 2 and Generation 3 Switching Modules To use this feature on Generation 2 or Generation 3 switching modules you must meet the following requirements e Display the interface configuration in the Information pane e Obtain the Enterprise package ENTERPRISE_PKG license see the NX OS Family Licensing Guide e Configure this feature in any port on a Generation 2 switch module See the Extended BB_Credits section on page 4 17 for more information on extended BB_credits on Generation 2 switching modules amp Note Extended BB_credits are not supported on the Cisco MDS 9124 Fabric Switch Cisco MDS 9134 Fabric Switch the Cisco Fabric Switch for HP c
71. the default configuration follow these steps Step 1 Configure the port mode See the Configuring Interface Modes section on page 2 22 Step 2 Configure the BB_credits performance buffers and extended BB_credits as necessary Cisco MDS 9000 Family NX OS Interfaces Configuration Guide e328 OL 29284 01 Release 6 x Chapter3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces W See the Configuring Buffer to Buffer Credits section on page 4 21 and the Configuring Extended BB_credits section on page 4 23 Configuring Port Speed Restrictions e Changing port speed and rate mode disrupts traffic on the port Traffic on other ports in the port group is not affected Detailed Steps To configure the port speed on an interface on a 4 Gbps or 8 Gbps switching module follow these steps Command Purpose Step1 switch config t Enters configuration mode switch config Step2 switch config interface fc 1 1 Selects the interface and enters interface switch config if configuration submode Step3 switch config if switchport speed 4000 Configures the port speed in megabits per second Valid values are 1000 2000 4000 and auto The auto parameter enables autosensing on the interface switch config if switchport speed 8000 8 Gbps modules only Configures the port speed in megabits per second to 8 Gbps Valid values are 1000 2000 4000 8000 and auto Th
72. the port is flappped is up or down when the event occurs switch config port monitor counter link loss poll interval seconds delta rising threshold percentagei event event id falling threshold percentage event event id portguard errordisable Specifies the delta link loss poll interval in seconds the thresholds in percentage and the event IDs of events to be triggered It also specifies that the interface is down error disabled when the event occurs Note By default the port monitor port guard is disabled To enable this feature you must explicitly configure the port monitor port guard feature on a particular counter by performing Step 3 or Step 4 Configuring Port Group Monitor This section includes the following topics e Enabling Port Group Monitor page 2 35 e Configuring a Port Group Monitor Policy page 2 35 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Configuring Interfaces e Reverting to the Default Policy for a Specific Counter page 2 36 e Turning Off the Monitoring of Specific Counter page 2 36 e Activating a Port Group Monitor Policy page 2 37 Enabling Port Group Monitor Detailed Steps Step 1 Step 2 To enable port group monitor follow these steps Command Purpose switch config t Enters configuration mode switch config port group monitor enable Enables de
73. these guidelines when creating or deleting VSAN interfaces e Create a VSAN before creating the interface for that VSAN If a VSAN does not exist the interface cannot be created e Create the interface VSAN it is not created automatically e If you delete the VSAN the attached interface is automatically deleted e Configure each interface only in one VSAN je Tip After configuring the VSAN interface you can configure an IP address or Virtual Router Redundancy Protocol VRRP feature See the Cisco MDS 9000 Family NX OS IP Services Configuration Guide Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x SEN Chapter 2 Configuring Interfaces HZ Default Settings Default Settings Table 2 7 lists the default settings for interface parameters Table 2 7 Default Interface Parameters Parameters Default Interface mode Auto Interface speed Auto Administrative state Shutdown unless changed during initial setup Trunk mode On unless changed during initial setup on non NPV and NPIV core switches Off on NPV switches Trunk allowed VSANSs or VF IDs 1 to 4093 Interface VSAN Default VSAN 1 Beacon mode Off disabled EISL encapsulation Disabled Data field size 2112 bytes Configuring Interfaces This section includes the following topics Configuring Fibre Channel Interfaces page 2 21 Setting the Interface Administrative State pag
74. this feature up to 16 expansion ports E ports or trunking E ports TE ports can be bundled into a PortChannel ISL ports can reside on any switching module and they do not need a designated master port If a port or a switching module fails the PortChannel continues to function properly without requiring fabric reconfiguration Cisco NX OS software uses a protocol to exchange PortChannel configuration information between adjacent switches to simplify PortChannel management including misconfiguration detection and autocreation of PortChannels among compatible ISLs In the autoconfigure mode ISLs with compatible parameters automatically form channel groups no manual intervention is required PortChannels load balance Fibre Channel traffic using a hash of source FC ID and destination FC ID and optionally the exchange ID Load balancing using PortChannels is performed over both Fibre Channel and FCIP links Cisco NX OS software also can be configured to load balance across multiple same cost FSPF routes Fibre Channel Port Rate Limiting The Fibre Channel port rate limiting feature for the Cisco MDS 9100 Series controls the amount of bandwidth available to individual Fibre Channel ports within groups of four host optimized ports Limiting bandwidth on one or more Fibre Channel ports allows the other ports in the group to receive a greater share of the available bandwidth under high utilization conditions Port rate limiting is also beneficial fo
75. unless specified explicitly Forcing an Interface Addition Detailed Steps Step 1 Step 2 Step 3 To force the addition of a port to a PortChannel follow these steps Command Purpose switch config t switch config Enters configuration mode switch config interface fc1 1 switch config if Specifies the interface fc1 1 switch config if channel group 1 force Forces the addition of the physical port for interface fc1 1 to channel group 1 The port is shut down Deleting an Interface from a PortChannel Detailed Steps Step 1 Step 2 To delete a physical interface or a range of physical interfaces from a PortChannel follow these steps Command Purpose switch config interface fc1 1 switch config if Enters the selected physical interface level switch config interface fc1 1 5 switch config if Enters the selected range of physical interfaces switch config if no channel group 2 switch config if Deletes the physical Fibre Channel interfaces in channel group 2 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter6 Configuring PortChannels W Verifying PortChannel Configuration Enabling and Configuring Autocreation Detailed Steps Step 1 Step 2 Step 3 Converting to Verifying PortChannel Configuration To configure automatic channel groups follow these steps
76. when the values cross the threshold settings The default port monitor policy has the following threshold values Threshold Interval Rising Falling Counter Type Seconds Threshold Event Threshold Event Link Loss Delta 60 5 4 1 4 Sync Loss Delta 60 5 4 1 4 Protocol Error Delta 60 1 4 0 4 Signal Loss Delta 60 5 4 1 4 Invalid Words Delta 60 1 4 0 4 Invalid CRCs Delta 60 5 4 1 4 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 2 Configuring Interfaces HZ information About Interfaces Threshold Interval Rising Falling Counter Type Seconds Threshold Event Threshold Event RX Performance Delta 60 2147483648 4 524288000 4 TX Performance Delta 60 2147483648 4 524288000 4 Port Monitor Port Guard Port monitor port guard is a feature that disables or shuts down a port when an event occurs Depending on the configuration when an event occurs the port is either error disabled or flapped Port monitor port guard is a different or separate feature that functions based on the configuration of the errordisable command Port Group Monitor Each line card or module has a predefined set of ports which share the same backplane bandwidth called port groups While oversubscription is a feature the port group monitor feature helps to monitor the spine bandwidth utilization An alarm syslog is generated so th
77. 0 shared 0 shared Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x TEEN Chapter 3 Configuring Fibre Channel Interfaces HI Configuring Fibre Channel Interfaces Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Bandwidth Rate Mode Buffers Gbps c9 13 16 4 0 shared c9 14 16 4 0 shared fc9 15 16 4 0 shared fc9 16 16 4 0 shared fc9 17 16 4 0 shared fc9 18 16 4 0 shared Port Group 4 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Bandwidth Rate Mode Buffers Gbps fc9 19 16 4 0 shared fc9 20 16 4 0 shared fc9 21 16 4 0 shared fc9 22 16 4 0 shared fc9 23 16 4 0 shared fc9 24 16 4 0 shared This example shows the port rate mode configuration for interfaces on a 48 port 8 Gbps module switch show port resource module 4 Module 4 Available dedicated buffers for global buffer 0 port groups 1 4 are 5016 Available dedicated buffers for global buffer 1 port groups 5 8 are 5016 Port Group 1 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Bandwidth Rate Mode Buffers Gbps fc4 1 32 8 0 shared fc4 2 32 8 0 shared fc4 3 32 8 0 shared fc4 4 32 8 0 shared fc4 5 32 8 0 shared fc4 6 32 8 0 shared Port Gr
78. 00 Family NX OS Interfaces Configuration Guide 2e W OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Information About Interfaces W Table 2 3 Reason Codes for Interface States Administrative Operational Configuration Status Reason Code Up Up None Down Down Administratively down If you administratively configure an interface as down you disable the interface No traffic is received or transmitted Up Down See Table 2 4 Note Only some of the reason codes are listed in Table 2 4 If the administrative state is up and the operational state is down the reason code differs based on the nonoperational reason code as described in Table 2 4 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x EN Chapter2 Configuring Interfaces Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Information About Interfaces Table 2 4 Reason Codes for Nonoperational States Applicable Reason Code long version Description Modes Link failure or not connected The physical layer link is not operational All SFP not present The small form factor pluggable SFP hardware is not plugged in Initializing The physical layer link is operational and the protocol initialization is in progress Reconfigure fabric in progress The fabric is currently being reconfigured Offline The Cisco NX OS software waits for the specified R_A
79. 1 shows an example application using NPIV Figure 7 1 NPIV Example Application MDS switch LUN 1 a N_Port_ID 1 _ gt LUN 2 be gt i N_Port_ID 2 a f LUN 3 N_Port_ID 3 N port F port controller 154019 You must globally enable NPIV for all VSANs on the MDS switch to allow the NPIV enabled applications to use multiple N port identifiers amp Note All of the N port identifiers are allocated in the same VSAN N Port Virtualization Typically Fibre Channel networks are deployed using a core edge model with a large number of fabric switches connected to edge devices Such a model is cost effective because the per port cost for director class switches is much higher than that of fabric switches However as the number of ports in the fabric increases the number of switches deployed also increases and you can end up with a significant increase in the number of domain IDs the maximum number supported is 239 This challenge becomes even more difficult when additional blade chassis are deployed in Fibre Channel networks NPV addresses the increase in the number of domain IDs needed to deploy a large number of the ports by making a fabric or blade switch appear as a host to the core Fibre Channel switch and as a Fibre Channel switch to the servers in the fabric or blade switch NPV aggregates multiple locally connected N ports into one or more external NP links which shares the do
80. 10 1 oversubscription Two ports with dedicated rate mode and 4 Gbps speed plus four ports with shared rate mode and 4 Gbps speed 4 1 oversubscription I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter4 Configuring Interface Buffers HZ Information About Interface Buffers e One port with dedicated rate mode and 4 Gbps speed plus three ports with dedicated rate mode and 2 Gbps speed plus two ports with shared rate mode and 4 Gbps speed 4 1 oversubscription e Six ports with dedicated rate mode and 2 Gbps speed 24 Port 8 Gbps Fibre Channel Module BB_Credit Buffers Table 4 3 lists the BB_credit buffer allocation for the 24 port 8 Gbps Fibre Channel switching module Table 4 3 24 Port 8 Gbps Switching Module BB_Credit Buffer Allocation BB_Credit Buffers Per Port Dedicated Rate Mode Shared Rate Mode 8 Gbps Speed 8 Gbps Speed BB_Credit Buffer Allocation ISL Fx Port Fx Port Default BB_credit buffers 500 32 32 Maximum BB_credit buffers 500 500 32 Total Number of BB_Credit Buffers per Module Ports 1 through 12 6000 Ports 13 through 24 6000 1 When connected to Generation 1 modules reduce the maximum BB credit allocation to 250 The following guidelines apply to BB_credit buffers on 24 port 8 Gbps Fibre Channel switching modules e BB credit buffers allocated for ports 1 through 12 and 13 through 24 can be a maximum of 6000 each so that the load is di
81. 12 to out of service state Moves the ports that are capable of a10 Gbps configuration fc1 4 8 and fc1 10 to in service state Sets the speed on ports fc1 4 8 and fc 1 10 to 10 Gbps Sets port modes on these ports to dedicated Performs a no shut on these ports switch config if no 10G speed mode Reverts the settings and puts all the ports fcl to 12 in out of service state and moves them to in service State Configuring Port Administrative Speeds Restrictions e Changing the port administrative speed is a disruptive operation Detailed Steps To configure the port speed of the interface follow these steps Command Purpose Step1 switch config t Enters configuration mode Step 2 switch config interface fc 1 1 Selects the fc interface and enters interface configuration mode Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x g 225 Chapter2 Configuring Interfaces HZ Configuring Interfaces Command Purpose Step3 switch config if switchport speed 1000 Configures the port speed of the interface to 1000 Mbps The number indicates the speed in megabits per second Mbps switch config if switchport speed Configures the port speed of the interface to 10000 10000 Mbps for 10 Gbps Note All the 10 Gbps capable interfaces except the interface that is being configured must be in the out of service state At lea
82. 140 and 9120 switch interfaces amp Note Due to the hardware design of the MDS 9134 switch we do not support interface out of service action on either of its two 10 Gigabit ports This is because no internal port hardware resource is released when an out of service action is performed on these 10 Gigabit ports When configuring these host optimized ports the following port mode guidelines apply e You can configure only the first port in each 4 port group for example the first port in ports 1 4 the fifth port in ports 5 8 and so on as an E port If the first port in the group is configured as an E port the other three ports in each group ports 2 4 6 8 and so on are not usable and remain shutdown e If you execute the write erase command on a 32 port switching module and then copy a saved configuration to the switch from a text file that contains the no system default switchport shutdown command you need to copy the text file to the switch again for the E ports to come up without manual configuration e If any of the other three ports are enabled you cannot configure the first port as an E port The other three ports continue to remain enabled Cisco MDS 9000 Family NX OS Interfaces Configuration Guide P2418 E OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Guidelines and Limitations W e The auto mode is not allowed in a 32 port switching module or the host optimized ports in the Cisco 9100 Seri
83. 2000 Auto Generation 2 or Normal Fail Generation 3 Horse Pass Auto Auto max 2000 Generation2 or Normal Fail Generation 3 Fort Pass Generation 3 Auto Auto Generation 1 Normal or force Fail interfaces Auto max 2000 Auto Generation Normal or force __ Pass Auto max 2000 Auto Generation 2 Normal Fail Force Pass Auto Auto max 2000 Generation 2 Normal Fail Force Pass Auto max 2000 Auto Generation 3 Normal Fail Force Pass Auto Auto max 2000 Generation 3 Normal Fail Force Pass Generation 4 Auto Auto Generation Normal or force Fail interfaces Auto max 2000 Auto Generation Normal or force Pass Auto max 2000 Auto Generation 2 Normal Fail Force Pass Auto Auto max 2000 Generation 2 Normal Fail Force Pass Auto max 2000 Auto Generation3 or Normal Fail Generation 4 Foree Pass Auto Auto max 2000 Generation3or Normal Fail Generation 4 Hore Pass 1 If resources are not available Use the show port channel compatibility parameters command to obtain information about PortChannel addition errors Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x _ Chapter 3 Configuring Fibre Channel Interfaces Default Settings Table 3 12 lists the default settings for Generation 2 interface parameters Default Settings W Table 3 12 Default Generation 2 Interface Parameters Default 48 Port 4 Gbps 24 Port 4 Gbps 12 Port 4 Gbps 4 Port 10 Gbps Paramete
84. 3 1 is not trunking fc3 7 is trunking Vsan 1000 is down fc3 10 is trunking Vsan 1 is up FCID is 0x760001 Vsan 2 is up FCID is 0x6f0001 fc3 11 is trunking Belongs to port channel 6 Vsan 1 is up FCID is 0xef0000 Vsan 2 is up FCID is 0xef0000 port channel 6 is trunking Vsan 1 is up FCID is 0xef0000 Vsan 2 is up FCID is 0xef0000 Configuration Example for F Port Trunking This example shows how to configure trunking and bring up the TF TNP link between an F port in the NPIV core switch and an NP port in the NPV switch Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Enable the F port trunking and channeling protocol on the MDS core switch swi tch Enable swi tch con con fig feature fport channel trunk NPIV on the MDS core switch fig feature npiv Configure the port mode to auto F or Fx on the MDS core switch swi swi tch tch con con fig interface fc1 2 fig if switchport mode F Configure the trunk mode to ON on the MDS core switch swi tch con fig if switchport trunk mode on Configure the port mode to NP on the NPV switch swi swi tch tch con con fig interface fcl1 2 fig if switchport mode NP Configure the trunk mode to ON on the NPV switch swi tch con fig if switchport trunk mode on Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Isolation due to vsan not configured on peer
85. 48 4 524288000 4 2147483648 4 524288000 4 switch show port monitor sample Policy Name Admin status Oper status Port type sample Active Active All Access Ports Counter portgurard Link Loss enabled Sync Loss enabled Protocol Error enabled Signal Loss enabled Invalid Words enabled Invalid CRC s enabled RX Performance enabled TX Performance enabled Threshold Delta Delta Delta Delta Delta Delta Interval Rising Threshold event Falling Threshold event 60 60 60 60 60 60 5 4 J 4 5 4 1 4 al 4 0 4 5 4 in 4 ai 4 0 4 5 4 al 4 2147483648 4 524288000 4 2147483648 4 524288000 4 Displaying Port Group Monitor Status and Policies The following commands display information about port group monitor switch show port group monitor status Port Group Monitor Active Policies Last 10 logs switch pgm2 Enabled I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces W Verifying Interfaces Configuration switch show port group monitor Policy Name pgml Admin status Not Active Oper status Not Active Port type All Port Groups Counter Threshold Interval ge Rising Threshold RX Performance Delta 60 50 TX Performance Delta 60 50 Policy Name pgm2 Admin status Active Oper status Active Port type All Port Groups Counter Threshold Interval ge Rising Threshold R
86. 5 BB credits Figure 4 3 shows the default BB_credit buffer allocation model for 24 port 8 Gbps switching modules The minimum BB credits required to bring up a port is two buffers 189047 Figure 4 4 shows the default BB_credit buffer allocation model for 4 44 port 8 Gbps host optimized switching modules The minimum BB_credits required to bring up a port is two buffers 189049 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Configuring Interface Buffers Figure 4 5 shows the default BB_credit buffer allocation model for 24 port 4 Gbps switching modules The minimum BB_credits required to bring up a port is two buffers Information About Interface Buffers Figure 4 5 BB_Credit Buffer Allocation in 24 Port 4 Gbps Switching Modules 24 port module All ports All ports shared 4 Gpbs Mixed dedicated 2 Gbps 16 BB credits 250 BB credits Dedicated Shared 16 BB credits 1 Gbps 2 Gbps 4 Gbps e 250 BB credits 250 BB credits 250 BB credits wy Note The default BB_credit buffer allocation is the same for all port speeds BB_Credit Buffers for Switching Modules This section describes how buffer credits are allocated to Cisco MDS 9000 switching modules and includes the following topics e Configuring Buffer Credits on a Generation 2 Generation 3 or Generation 4 Module
87. 7 Configuring N Port Virtualization NP Ports NP Links Information About N Port Virtualization W An NP port proxy N port is a port on a device that is in NPV mode and connected to the NPV core switch using an F port NP ports behave like N ports except that in addition to providing N port behavior they also function as proxies for multiple physical N ports An NP link is basically an NPIV uplink to a specific end device NP links are established when the uplink to the NPV core switch comes up the links are terminated when the uplink goes down Once the uplink is established the NPV switch performs an internal FLOGI to the NPV core switch and then if the FLOGI is successful registers itself with the NPV core switch s name server Subsequent FLOGIs from end devices in this NP link are converted to FDISCs For more details refer to the Internal FLOGI Parameters section on page 7 5 Server links are uniformly distributed across the NP links All the end devices behind a server link will be mapped to only one NP link Internal FLOGI Parameters When an NP port comes up the NPV device first logs itself in to the NPV core switch and sends a FLOGI request that includes the following parameters e The fWWN fabric port WWN of the NP port used as the pWWN in the internal login e The VSAN based sWWN switch WWN of the NPV device used as nWWN node WWN in the internal FLOGI After completing its FLOGI request the NPV devi
88. 9509 Directors IPS 4 and IPS 8 modules must be upgraded to the MSM 18 4 Multiservice modules Fabric 1 modules must be upgraded to Fabric 2 modules on the MDS 9513 Director to use the 48 port or the 24 port 8 Gbps module Fabric 2 modules must be upgraded to Fabric 3 modules on the MDS 9513 Director to get the maximum backplane bandwidth of 256 Gbps Cisco Fabric Manager Release 4 x supports MDS SAN OS Release 3 x and NX OS 4 x in mixed mode through Interswitch Link ISL connectivity Cisco MDS 9000 Family NX OS Interfaces Configuration Guide P38 OL 29284 01 Release 6 x Chapter3 Configuring Fibre Channel Interfaces wy Guidelines and Limitations W Note Local Switchi When a Cisco or another vendor switch port is connected to a Generation 1 module port ISL connection the receive buffer to buffer credits of the port connected to the Generation 1 module port must not exceed 255 ng Limitations e All ports in the module must be in shared mode Use the switchport ratemode shared command to ensure that all the ports in the module are in shared mode e NoE ports are allowed in the module because E ports must be in dedicated mode Port Index Limitations Note Cisco MDS 9000 switches allocate index identifiers for the ports on the modules These port indexes cannot be configured You can combine Generation 1 Generation 2 Generation 3 and Generation 4 switching modules with either Supervisor 1 modules o
89. A is assigned to a device Each device is identified by its port world wide name pWWN When a device is allocated an ALPA an entry for that device is automatically created in the ALPA cache A cache contains entries for recently allocated ALPA values These caches are maintained on various TL ports If a device already has an ALPA the Cisco NX OS software attempts to allocate the same ALPA to the device each time The ALPA cache is maintained in persistent storage and saves information across switch reboots The maximum cache size is 1000 entries If the cache is full and a new ALPA is allocated the Cisco NX OS software discards an inactive cache entry if available to make space for the new entry See the TL Port section on page 2 4 for more information on TL ports Port Guard The port guard feature is intended for use in environments where the system and application environment does not adapt quickly and efficiently to a port going down and back up or to a port rapidly cycling up and down which can happen in some failure modes For example if a system takes five seconds to stabilize after a port goes down but the port is going up and down once a second a more severe failure in the fabric might occur Cisco MDS 9000 Family NX OS Interfaces Configuration Guide P24 E OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Port Monitor Information About Interfaces W The port guard feature gives the SAN administrato
90. Activates the specified port group monitor policy switch config port group monitor activate Activates the default port group monitor policy switch config no port group monitor activate policyname Deactivates the specified port group monitor policy Configuring Management Interfaces Detailed Steps Step 1 Step 2 Step 3 Step 4 Step5 Step 6 Step7 Step 8 To configure the mgmt0 Ethernet interface to connect over IPv4 follow these steps Command Purpose switch config terminal switch config Enters configuration mode switch config interface mgmt0 switch config if Selects the management Ethernet interface on the switch and enters interface configuration submode switch config if ip address 10 16 1 2 255 255 255 0 Configures the IPv4 address and IPv4 subnet mask switch config if no shutdown Enables the interface switch config if exit switch config Returns to configuration mode switch config ip default gateway 1 1 1 4 Configures the default gateway IPv4 address switch config exit switch Returns to EXEC mode switch copy running config startup config Optional Saves your configuration changes to the file system Note If you want to save your configuration you can enter this command at any time I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guid
91. Apafi CISCO Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Cisco MDS NX OS Release 6 2 1 April 2013 Americas Headquarters Cisco Systems Inc 170 West Tasman Drive San Jose CA 95134 1706 USA http www cisco com Tel 408 526 4000 800 553 NETS 6387 Fax 408 527 0883 Text Part Number OL 29284 01 THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE ALL STATEMENTS INFORMATION AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND EXPRESS OR IMPLIED USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California Berkeley UCB as part of UCB s public domain version of the UNIX operating system All rights reserved Copyright 1981 Regents of the University of California NOTWITHSTANDING ANY OTHER WARRANTY HEREIN ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED AS IS WITH ALL FAULTS CISCO AND THE ABOVE NAMED SUPPLIERS DISCLAIM ALL WARRANTIES EXPR
92. B Credit is 16 Receive data field Size is 2112 Beacon is turned off Port track mode is force_shut lt this port remains shut even if the tracked port is back up Cisco MDS 9000 Family NX OS Quality of Service Configuration Guide OL 29284 01 Symbols asterisk first operational port 6 20 Numerics 12 port 4 Gbps switching modules BB credit buffers 4 13 configuration guidelines 3 28 default settings 3 25 See also switching modules 16 port switching modules configuring BB_credits 4 1 LEDs 2 11 See also switching modules 24 port 4 Gbps switching modules bandwidth fairness 3 17 default settings 3 25 example configurations 3 46 3 49 4 12 oversubscription 3 11 shared resources 3 11 See also switching modules 24 port 8 Gbps switching modules default settings 3 25 example configurations 4 8 32 port switching modules configuring BB_credits 4 1 See also switching modules 4 44 port 8 Gbps switching modules default settings 3 25 example configurations 4 9 48 port 4 Gbps switching modules INDEX bandwidth fairness 3 17 default settings 3 25 example configurations 3 47 3 48 4 10 oversubscription 3 11 shared resources 3 11 See also switching modules 48 port 8 Gbps switching modules default settings 3 25 example configurations 3 46 4 7 See also switching modules 4 port 10 Gbps switching modules BB credit buffers 4 15 configuration guidelines 3 28 default settings 3 25 S
93. Balances the load by allowing the user to evenly distribute the load across external interfaces Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x 77 Chapter7 Configuring N Port Virtualization HZ Guidelines and Limitations Disruptive Disruptive load balance works independent of automatic selection of interfaces and a configured traffic map of external interfaces This feature forces reinitialization of the server interfaces to achieve load balance when this feature is enabled and whenever a new external interface comes up To avoid flapping the server interfaces too often enable this feature once and then disable it whenever the needed load balance is achieved If disruptive load balance is not enabled you need to manually flap the server interface to move some of the load to a new external interface Multiple VSAN Support By grouping devices into different NPV sessions based on VSANs it is possible to support multiple VSANs on the NPV enabled switch The correct uplink must be selected based on the VSAN that the uplink is carrying Guidelines and Limitations This section includes the guidelines and limitations for this feature NPV Guidelines and Requirements page 7 8 NPV Traffic Management Guidelines page 7 9 DPVM Configuration Guidelines page 7 9 NPV and Port Security Configuration Guidelines page 7 10 NPV Guidelines and Requirements Following are recommended guidelines
94. CHAPTER 9 Configuring Port Tracking 9 1 Information About Port Tracking 9 1 Guidelines and Limitations 9 2 Default Settings 9 2 Configuring Port Tracking 9 3 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Enabling Port Tracking 9 3 Information About Configuring Linked Ports 9 3 Binding a Tracked Port Operationally 9 4 Information About Tracking Multiple Ports 9 4 Tracking Multiple Ports 9 5 Information About Monitoring Ports ina VSAN 9 5 Monitoring Ports ina VSAN 9 5 Information AboutForceful Shutdown 9 6 Forcefully Shutting Down a Tracked Port 9 6 Displaying Port Tracking Information 9 6 9 8 I OL 29284 01 Release 6 x Contents W Cisco MDS 9000 Family NX OS Interfaces Configuration Guide E Contents Cisco MDS 9000 Family NX OS Interfaces Configuration Guide exit OL 29284 01 Release 6 x New and Changed Information As of Cisco MDS NX OS Release 4 2 1 software configuration information is available in new feature specific configuration guides for the following information e System management e Interfaces e Fabric e Quality of service e Security e IP services e High availability and redundancy The information in these new guides previously existed in the Cisco MDS 9000 Family CLI Configuration Guide and in the Cisco MDS 9000 Family Fabric Manager Configuration Guide Those configuration guides remain available on Cisco com and should be used for al
95. Chapter 3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces W f c2 7 1 TE up swl mH fc2 8 1 FX down swl c2 48 1 FX down swl Step4 Disable restrictions on oversubscription ratios for the ports switch config t Enter configuration commands one per line End with CNTL Z switch config no rate mode oversubscription limit module 2 Step 5 Bring up the ports that you shut down in step 2 and display their status to confirm that they are no longer shut down switch config interface fc2 1 2 c2 4 5 c2 8 38 c2 43 48 switch config if no shutdown switch config if end switch show interface brief Interface Vsan Admin Admin Status SFP Oper Oper Port Mode Trunk Mode Speed Channel Mode Gbps fc2 1 1 FX up swl c2 2 1 FX up swl ooo o2 3 1 P up swl c2 4 1 FX up swl 35 c2 5 T FX up swl c2 6 1 TE up swl f c2 7 ak TE up swl c2 8 1 FX up swl fc2 48 1 FX S up sw1 e Step6 Confirm that the ports are now operating with no restrictions on oversubscription ratios switch show running config include oversubscription limit no rate mode oversubscription limit module 2 lt indicates no restrictions on oversubscription ratios Step7 Save the new oversubscription ratio configuration to the startup configuration switch copy running config startup config Enabling Restrictions on Oversubscription Rati
96. Class BladeSystem and the Cisco Fabric Switch for IBM BladeCenter Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x EN Chapter4 Configuring Interface Buffers HZ Information About Interface Buffers Buffer to Buffer Credit Recovery amp Although the Fibre Channel standards require low bit error rates bit errors do occur Over time the corruption of receiver ready messages known as R_RDY primitives can lead to a loss of credits which can eventually cause a link to stop transmitting in one direction The Fibre Channel standards provide a feature for two attached ports to detect and correct this situation This feature is called buffer to buffer credit recovery Buffer to buffer credit recovery functions as follows the sender and the receiver agree to send checkpoint primitives to each other starting from the time that the link comes up The sender sends a checkpoint every time it has sent the specified number of frames and the receiver sends a checkpoint every time it has sent the specified number of R_RDY primitives If the receiver detects lost credits it can retransmit them and restore the credit count on the sender The buffer to buffer credit recovery feature can be used on any non arbitrated loop link This feature is most useful on unreliable links such as MANs or WANs but can also help on shorter high loss links such as a link with a faulty fiber connection Note
97. Configure the port speed rate mode and port mode on the interfaces switchport speed auto max 4000 switchport rate mode shared switchport mode f switch config i switch config i switch config i Step9 Enable the interfaces and return to configuration mode no shutdown exit switch config i switch config i switch Configuration Example for 48 Port 4 Gbps Module Interfaces These steps describe how to configure the example shown in Figure 4 6 on page 4 11 Step 1 Select interfaces fc 4 11 through fc 4 12 switch config t switch config interface fc 4 11 12 Step2 Disable the interfaces and take them out of service switch config if shutdown switch config if out of service Step3 Return to configuration mode switch config if exit switch Step4 Select the interfaces fc 4 1 through fc 4 6 switch config t switch config interface fc 4 1 6 Step5 Configure the port speed rate mode and port mode on the interfaces switch config if switchport speed auto max 2000 switch config if switchport rate mode dedicated switch config if switchport mode e Step6 Enable the interfaces and return to configuration mode switch config if no shutdown switch config if exit switch Cisco MDS 9000 Family NX OS Interfaces Configuration Guide M348 OL 29284 01 Release 6 x _ Chapter 3 Configuring Fibre Channel Interfaces Step 7 Step 8
98. DS 9000 Family NX OS Interfaces Configuration Guide Chapter6 Configuring PortChannels HI Information About PortChannels Figure 6 5 SID1 DID1 and Exchange Based Load Balancing Frame 1 ____ Link Frame 2 Link 2 Frame 3 beh SID1 DID1 Exchange 1 Frame n Frame 1 Link 1 Frame 2 Link 2 Frame 3 gt gt gt SID1 DID1 Exchange 2 Frame n 8 For more information on configuring load balancing and in order delivery features see the Cisco MDS 9000 Family NX OS Fabric Configuration Guide PortChannel Modes You can configure each PortChannel with a channel group mode parameter to determine the PortChannel protocol behavior for all member ports in this channel group The possible values for a channel group mode are as follows e ON default The member ports only operate as part of a PortChannel or remain inactive In this mode the PortChannel protocol is not initiated However if a PortChannel protocol frame is received from a peer port the software indicates its nonnegotiable status This mode is backward compatible with the existing implementation of PortChannels in releases prior to Release 2 0 1b where the channel group mode is implicitly assumed to be ON In Cisco MDS SAN OS Releases 1 3 and earlier the only available PortChannel mode was the ON mode PortChannels configured in the ON mode require you to explicitly enable and disable the PortChan
99. E mode is supported By default the mode is ACTIVE on the NPV switches Devices logged in through F PortChannel on an MDS switch are not supported in IVR non NAT configuration The devices are supported only in IVR NAT configuration Port security rules are enforced only on physical pWWNs at the single link level FC SP authenticates only the first physical FLOGI of every PortChannel member Since the FLOGI payload carries only the VF bits to trigger the use of a protocol after the FLOGI exchange those bits will be overridden In the case of the NPV switches the core has a Cisco WWN and will try to initiate the PCP protocol The name server registration of the N ports logging in through an F PortChannel will use the fWWN of the PortChannel interface DPVM configuration is not supported The PortChannel port VSAN cannot be configured using DPVM Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter6 Configuring PortChannels Guidelines and Limitations e The Dynamic Port VSAN Management DPVM database will be queried only for the first physical FLOGI of each member so that the port VSAN can be configured automatically e DPVM does not bind FC_IDs to VSANs but pWWNs to VSANSs It will be queried only for the physical FLOGI Valid and Invalid PortChannel Examples PortChannels are created with default values You can change the default configuration just like any other physical in
100. ESSED OR IMPLIED INCLUDING WITHOUT LIMITATION THOSE OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING USAGE OR TRADE PRACTICE IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT SPECIAL CONSEQUENTIAL OR INCIDENTAL DAMAGES INCLUDING WITHOUT LIMITATION LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and or its affiliates in the U S and other countries To view a list of Cisco trademarks go to this URL www cisco com go trademarks Third party trademarks mentioned are the property of their respective owners The use of the word partner does not imply a partnership relationship between Cisco and any other company 1110R Any Internet Protocol IP addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers Any examples command display output network topology diagrams and other figures included in the document are shown for illustrative purposes only Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental Cisco MDS 9000 Family NX OS Interfaces Configuration Guide 2012 2013 Cisco Systems Inc All rights reserved CHAPTER 1 CHAPTER 2 CONTENTS New and Cha
101. Gbps speed 2 1 oversubscription default e Two ports with dedicated rate mode and 4 Gbps speed plus four ports with shared rate mode and 4 Gbps speed with 4 1 oversubscription e One port with dedicated rate mode and 4 Gbps speed plus three ports with dedicated rate mode and 2 Gbps speed plus two ports with shared rate mode and 4 Gbps speed 4 1 oversubscription e Six ports with dedicated rate mode and 2 Gbps speed e Three ports with dedicated rate mode and 4 Gbps speed plus three ports with shared rate mode and 1 Gbps speed see Figure 4 8 Note For detailed configuration steps of this example see the Configuration Example for 24 Port 4 Gbps Module Interfaces section on page 3 49 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Ca OL 29284 01 Release 6 x Chapter4 Configuring Interface Buffers Information About Interface Buffers W Figure 4 8 Example Speed and Rate Configuration on a 24 Port 4 Gbps Switching Module 1 2 3 4 5 6 4 Gbps 4 Gbps 4 Gbps 1 Gbps 1 Gbps 1 Gbps Dedicated Dedicated Dedicated Shared Shared Shared 144857 18 Port Fibre Channel 4 Port Gigabit Ethernet Multiservice Module BB_Credit Buffers Table 4 7 lists the BB_credit buffer allocation for 18 port 4 Gbps multiservice modules Table 4 7 18 Port 4 Gbps Multiservice Module BB_Credit Buffer Allocation BB_Credit Buffers Per Port Dedicated Rate Mode Shared Rate Mode 4 Gbps Speed 4 Gbps Speed
102. IDs in SANs Switches operating in the NPV mode do not join a fabric They pass traffic between NPV core switch links and end devices which eliminates the domain IDs for these edge switches NPV Overview page 7 1 N Port Identifier Virtualization page 7 2 N Port Virtualization page 7 2 NPV Mode page 7 4 NP Ports page 7 5 NP Links page 7 5 Default Port Numbers page 7 6 NPV CFS Distribution over IP page 7 7 NPV Traffic Management page 7 7 Multiple VSAN Support page 7 8 NPV is supported by the following Cisco MDS 9000 switches and Cisco Nexus 5000 Series switches only Cisco MDS 9124 Multilayer Fabric Switch Cisco MDS 9134 Fabric Switch Cisco MDS 9148 Multilayer Fabric Switch I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter7 Configuring N Port Virtualization HZ information About N Port Virtualization e Cisco Fabric Switch for HP c Class BladeSystem e Cisco Fabric Switch for IBM BladeCenter e Cisco Nexus 5000 Series switches D Note NPV is available on these switches only while in NPV mode if in switch mode NPV is not available N Port Identifier Virtualization N port identifier virtualization NPIV provides a means to assign multiple FC IDs to a single N port This feature allows multiple applications on the N port to use different identifiers and allows access control zoning and port security to be implemented at the application level Figure 7
103. LOGI is mapped to the virtual pWWN Subsequent FDSIDs will have different mappings Several checks are done by the NPV core to ensure the uniqueness of virtual pWWNs in the switch across the NPV switches When duplicate virtual pWWNs are configured the subsequent logins are rejected by the NPV core switch Prerequisites e The interface must be in a shut state and the specified virtual pWWN should not be logged in Cisco MDS 9000 Family NX OS Interfaces Configuration Guide lt OL 29284 01 Release 6 x Chapter8 Configuring FlexAttach Virtual pWWN Verifying FlexAttach Virtual pWWN Configuration Hi Restrictions e The specified virtual pWWN and the real pWWN must not be logged in Detailed Steps To map pWWN to virtual pWWN perform this task Command Purpose Step1 switch config t Enters configuration mode Step2 switch config flex attach virtual pwwn Maps the pWWN to the virtual pWWN vpwwn pwwn pwwn Step3 switch config flex attach commit Commits the configuration Verifying FlexAttach Virtual pWWN Configuration To display FlexAttach configuration information perform one of the following tasks Command Purpose show flex attach virtual wwn Displays the type and value of virtual pWWNs show fcns database Displays if the end device is logged with the correct virtual WWNs For detailed information about the fields in the output from these commands refer to the Cisco MDS NX OS Comm
104. OL 29284 01 Release 6 x Chapter5 Configuring Trunking Configuration Example for F Port Trunking W Step7 Set the port administrative state on NPIV and NPV switches to ON switch config interface fc1 2 switch config if shut switch config if no shut Step8 Save the configuration switch config copy running config startup config Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x nE Chapter5 Configuring Trunking W Configuration Example for F Port Trunking Cisco MDS 9000 Family NX OS Interfaces Configuration Guide u OL 29284 01 Release 6 x CHAPTER Configuring PortChannels Information About PortChannels page 6 1 Prerequisites for PortChannels page 6 12 Guidelines and Limitations page 6 13 Default Settings page 6 16 Configuring PortChannels page 6 17 Verifying PortChannel Configuration page 6 20 Configuration Examples for F and TF PortChannels page 6 24 Information About PortChannels This section includes the following topics PortChannels Overview page 6 2 E PortChannels page 6 2 F and TF PortChannels page 6 3 PortChanneling and Trunking page 6 3 Load Balancing page 6 4 PortChannel Modes page 6 6 PortChannel Deletion page 6 7 Interfaces in a PortChannel page 6 7 PortChannel Protocols page 6 9 Channel Group Creation page 6 10 Autocreation page 6 11 Manually Configured Channel Groups page 6 12 Cisco MDS 9000 Family NX OS I
105. PV you must adhere to the following requirements The internal FLOGI must be in the port security database so that the port on the NPV core switch will allow communications and links e All of the end device pWWNs must also be in the port security database Once these requirements are met you can enable port security as you would in any other context For details about enabling port security see the Cisco MDS 9000 Family NX OS Security Configuration Guide Configuring N Port Virtualization This section includes the following topics e Enabling N Port Identifier Virtualization page 7 10 e Configuring NPV page 7 10 e Configuring NPV Traffic Management page 7 12 Enabling N Port Identifier Virtualization You must globally enable NPIV for all VSANs on the MDS switch to allow the NPIV enabled applications to use multiple N port identifiers S Note All of the N port identifiers are allocated in the same VSAN Detailed Steps To enable or disable NPIV on the switch follow these steps Command Purpose Step1 switch config t Enters configuration mode Step2 switch config feature npiv Enables NPIV for all VSANs on the switch Step3 switch config no feature npiv Disables default NPIV on the switch Configuring NPV When you enable NPV the system configuration is erased and the system reboots with the NPV mode enabled S Note We recommend that you save the current configuration either on boot
106. RISE_PKG license is required to use extended BB_credits on Generation 2 Generation 3 and Generation 4 switching modules Also extended BB_credits are not supported by ports in shared rate mode OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter4 Configuring Interface Buffers Information About Interface Buffers All ports on the Generation 2 and Generation 3 switching modules support extended BB_credits There are no limitations for how many extended BB_credits you can assign to a port except for the maximum and minimum limits If necessary you can take interfaces out of service to make more extended BB_credits available to other ports You can use the extended BB_credits flow control mechanism in addition to BB_credits for long haul links This section includes the following topics e Extended BB_credits on Generation 1 Switching Modules page 4 18 e Extended BB_credits on Generation 2 and Generation 3 Switching Modules page 4 19 Extended BB_credits on Generation 1 Switching Modules The BB_credits feature allows you to configure up to 255 receive buffers on Generation 1 switching modules To facilitate BB_credits for long haul links you can configure up to 3 500 receive BB_credits on a Fibre Channel port on a Generation 1 switching module To use this feature on Generation switching modules you must meet the following requirements e Obtain the ENTERPRISE_PKG license See
107. Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces HE Guidelines and Limitations When you are downgrading to a release earlier than Cisco SAN OS Release 3 1 2 all modules keep operating in the same bandwidth fairness configuration prior to the downgrade After the downgrade any new module that is inserted has bandwidth fairness disabled S Note After the downgrade any insertion of a module or module reload will have bandwidth fairness disabled Guidelines and Limitations This section includes the following topics Combining Generation 1 Generation 2 Generation 3 and Generation 4 Modules page 3 18 Local Switching Limitations page 3 19 Port Index Limitations page 3 19 PortChannel Limitations page 3 21 Combining Generation 1 Generation 2 Generation 3 and Generation 4 Modules Cisco MDS NX OS Release 5 2 x and later supports combining Generation 1 Generation 2 Generation 3 and Generation 4 modules and switches with the following considerations MDS NX OS Release 4 1 1 and later features are not supported on the following Generation 1 switches and modules Supervisor 1 module 4 Port IP Storage Services module 8 Port IP Storage Services module MDS 9216 switch MDS 9216A switch MDS 9020 switch MDS 9120 switch MDS 9140 switch Supervisor 1 modules must be upgraded to Supervisor 2 modules on the MDS 9506 and MDS
108. V Traffic Map Information Server If External If s fel 3 fFol 10 fc1 11 felys fFol 1 f c1 2 To display the NPV internal traffic details enter the show npv internal info traffic map command switch show npv internal info traffic map NPV Traffic Map Information Server If External If s rel 3 Fol 10 fc1 11 ECL 5S fFol 1 f c1 2 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter7 Configuring N Port Virtualization W Verifying NPV Configuration Cisco MDS 9000 Family NX OS Interfaces Configuration Guide za OL 29284 01 Release 6 x CHAPTER Configuring FlexAttach Virtual pWWN Information About FlexAttach Virtual pWWN page 8 1 Guidelines and Limitations page 8 3 Configuring FlexAttach Virtual pWWN page 8 3 Verifying FlexAttach Virtual pWWN Configuration page 8 5 Monitoring FlexAttach Virtual pWWN page 8 6 Information About FlexAttach Virtual pWWN This section includes the following topics FlexAttach Virtual pWWN page 8 1 Difference Between San Device Virtualization and FlexAttach Port Virtualization page 8 2 FlexAttach Virtual pWWN CFS Distribution page 8 2 Security Settings for FlexAttach Virtual pWWN page 8 3 FlexAttach Virtual pWWN FlexAttach virtual pWWN feature facilitates server and configuration management In a SAN environment the server installation or replacement requires interaction and coordination among the SAN and server ad
109. VSAN On certain switches or fabric switches where the port VSANs are different one end returns an error and the other end is not connected Trunk Modes By default trunk mode is enabled on all Fibre Channel interfaces Mode E F FL Fx ST and SD on non NPV switches On NPV switches by default trunk mode is disabled You can configure trunk mode as on enabled off disabled or auto automatic The trunk mode configuration at the two ends of an ISL between two switches determine the trunking state of the link and the port modes at both ends see Table 5 2 Table 5 2 Trunk Mode Status Between Switches Your Trunk Mode Configuration Resulting State and Port Mode Port Type Switch 1 Switch 2 Trunking State Port Mode E ports On Auto or on Trunking EISL TE port Off Auto on or off No trunking ISL E port Auto Auto No trunking ISL E port Port Type Core Switch NPV Switch Trunking State Link Mode Fand NP On Auto or on Trunking TF TNP link ports Auto On Trunking TF TNP link Off Auto on or off No trunking F NP link Cisco MDS 9000 Family NX OS Interfaces Configuration Guide a OL 29284 01 Release 6 x Chapter5 Configuring Trunking je Information About Trunking W Tip The preferred configuration on the Cisco MDS 9000 Family switches is one side of the trunk set to auto and the other side set to on Note When connected to a third party switch the
110. X Performance Delta 60 80 TX Performance Delta 60 80 Policy Name default Admin status Not Active Oper status Not Active Port type All Port Groups Counter Threshold Interval ge Rising Threshold RX Performance Delta 60 80 TX Performance Delta 60 80 switch show port group monitor active Policy Name pgm2 Admin status Active Oper status Active Port type All Port Groups Counter Threshold Interval ge Rising Threshold RX Performance Delta 60 80 TX Performance Delta 60 80 switch show port group monitor PGMON_policy PPolicy Name PGMON_policy Admin status Not Active Oper status Not Active Port type All Port Groups Counter Threshold Interval ge Rising Threshold RX Performance Delta 26 450 TX Performance Delta 60 100 ge Falling Threshold ge Falling Threshold Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Verifying Interfaces Configuration W Displaying Management Interface Configuration To display the management interface configuration use the show interface mgmt 0 command switch show interface mgmt 0 mgmt0 is up Hardware is FastEthernet Address is 000c 30d9 fdbc Internet address is 10 16 1 2 24 MTU 1500 bytes BW 100 Mbps full Duplex 26388 packets input 6101647 bytes 0 multicast frames 0 compressed 0 input errors 0 frame 0 overrun 0 fifo 10247 packets output 2389196 bytes 0 underruns 0
111. Z information About Interfaces The bit errors can occur for the following reasons e Faulty or bad cable e Faulty or bad GBIC or SFP e GBIC or SFP is specified to operate at 1 Gbps but is used at 2 Gbps e GBIC or SFP is specified to operate at 2 Gbps but is used at 4 Gbps e Short haul cable is used for long haul or long haul cable is used for short haul e Momentary sync loss e Loose cable connection at one or both ends e Improper GBIC or SFP connection at one or both ends A bit error rate threshold is detected when 15 error bursts occur in a 5 minute period By default the switch disables the interface when the threshold is reached You can enter a shutdown and no shutdown command sequence to re enable the interface You can configure the switch to not disable an interface when the threshold is crossed By default the threshold disables the interface SFP Transmitter Types The small form factor pluggable SFP hardware transmitters are identified by their acronyms when displayed Table 2 5 defines the acronyms used for SFPs The small form factor pluggable SFP hardware transmitters are identified by their acronyms when displayed in the show interface brief command If the related SFP has a Cisco assigned extended ID then the show interface and show interface brief commands display the ID instead of the transmitter type The show interface transceiver command and the show interface fc s ot port transceiver command display both val
112. _TOV time before retrying initialization Inactive The interface VSAN is deleted or is in a suspended state To make the interface operational assign that port to a configured and active VSAN Hardware failure A hardware failure is detected Error disabled Error conditions require administrative attention Interfaces may be error disabled for various reasons For example e Configuration failure e Incompatible buffer to buffer credit configuration To make the interface operational you must first fix the error conditions causing this state and next administratively shut down or enable the interface FC redirect failure A port is isolated because a Fibre Channel redirect is unable to program routes No port activation license available A port is not active because it does not have a port license SDM failure A port is isolated because SDM is unable to program routes OL 29284 01 Release 6 x Chapter 2 Configuring Interfaces Information About Interfaces Ti Table 2 4 Reason Codes for Nonoperational States continued Applicable Reason Code long version Description Modes Isolation due to ELP failure The port negotiation failed Only E ports Isolation due to ESC failure The port negotiation failed and TE ports Isolation due to domain The Fibre Channel domains fcdomain overlap overlap Isolation due to domain ID The assigne
113. aces page 3 48 Configuration Example for 24 Port 4 Gbps Module Interfaces page 3 49 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces W Configuration Examples for Fibre Channel Interfaces Configuration Example for 48 Port 8 Gbps Module Interfaces Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Step 7 Step 8 Step 9 These steps describe how to configure the 48 port 8 Gbps module interfaces Select the interfaces fc 4 1 through fe 4 2 switch config t switch config interface fc 4 1 2 Configure the port speed rate mode and port mode on the interfaces switch config if switchport speed 8000 switch config if switchport rate mode shared switch config if switchport mode f Enable the interfaces and return to configuration mode switch config if no shutdown switch config if exit switch Select the interfaces fc 4 3 through fe 4 4 switch config t switch config interface fc 4 3 4 Configure the port speed rate mode and port mode on the interfaces switch config if switchport speed auto max 4000 switch config if switchport rate mode dedicated switch config if switchport mode f Enable the interfaces and return to configuration mode switch config if no shutdown switch config if exit switch Select the interfaces fc 4 5 through fc 4 6 switch config t
114. aces Configuration Guide Chapter4 Configuring Interface Buffers W Configuring Interface Buffers Enabling Buffer to Buffer Credit Recovery Buffer to buffer credit recovery on ISLs E or TE ports is enabled by default Detailed Steps To use buffer to buffer credit recovery on a port follow these steps Command Purpose Step1 switch config t Enters configuration mode switch config Step2 switch config interface fc 1 1 Selects the interface and enters interface switehlconfig if configuration submode Step3 switch config if switchport fcbbscn Enables buffer to buffer credit recovery on the interface switch config if no switchport fcbbscn Disables default buffer to buffer credit recovery on the interface Enabling the Buffer to Buffer State Change Number Detailed Steps To use the BB_SC_N field during PLOGI or FLOGI follow these steps Command Purpose Step1 switch config t Enters configuration mode switch config Step2 switch config interface fc 1 1 Selects the interface and enters interface switch config if configuration submode Step3 switch config if switchport fcbbscn Enables the use of buffer to buffer state change number for PLOGIs and FLOGIs on the interface switch config if no switchport fcbbscn Disables default the use of buffer to buffer state change number for PLOGIs and FLOGIs on the interface Configuring Re
115. acking feature is unique to the Cisco MDS 9000 Family of switches This feature uses information about the operational state of the link to initiate a failure in the link that connects the edge device This process of converting the indirect failure to a direct failure triggers a faster recovery process towards redundant links When enabled the port tracking feature brings down the configured links based on the failed link and forces the traffic to be redirected to another redundant link This chapter includes the following sections e Information About Port Tracking page 9 1 e Guidelines and Limitations page 9 2 e Default Settings page 9 2 e Configuring Port Tracking page 9 3Configuring Port Tracking page 9 3 e Displaying Port Tracking Information page 9 6 Information About Port Tracking Generally hosts can instantly recover from a link failure on a link that is immediately direct link connected to a switch However recovering from an indirect link failure between switches in a WAN or MAN fabric with a keep alive mechanism is dependent on several factors such as the time out values TOVs and on registered state change notification RSCN information In Figure 9 1 when the direct link 1 to the host fails recovery can be immediate However when the ISL 2 fails between the two switches recovery depends on TOVs RSCNs and other factors I OL 29284 01 Cisco MDS 9000 Family NX OS Quality of Service Configuration Guide Chapte
116. ain a line rate of traffic at the maximum configured operating speed and ports configured in shared mode share the available remaining bandwidth within the port group Bandwidth allocation among the shared mode ports is based on the operational speed of the ports For example if four ports operating at speeds 1 Gbps 1 Gbps 2 Gbps and 4 Gbps share bandwidth of 8 Gbps the ratio of allocation would be 1 1 2 4 Unutilized bandwidth from the dedicated ports is shared among only the shared ports in a port group as per the ratio of the configured operating speed A port cannot be brought up unless the reserved bandwidth is quaranteed for the shared ports see Table 3 9 For dedicated ports configured bandwidth is taken into consideration while calculating available bandwidth for the port group This behavior can be changed using bandwidth fairness by using the rate mode bandwidth fairness module number command For example consider a 48 port 8 Gbps module This module has 6 ports per port group with 12 8 Gbps bandwidth Ports 3 to 6 are configured at 4 Gbps If the first port is configured at 8 Gbps dedicated rate mode and the second port is configured at 4 Gbps dedicated rate mode then no other ports can be configured at 4 Gbps or 8 Gbps because the left over bandwidth of 0 8 Gbps 12 8 8 4 cannot meet the required 0 96 Gbps for the remaining four ports A minimum of 0 24 Gbps reserved bandwidth is required for the for the rest of the four ports H
117. ame Dedicated Rate Shared Rate Mode and Rate Part Number Description Mode Mode Mode on All Ports DS C9134 K9 32 port 4 Gbps Fabric Yes Yes Auto Shared Cisco MDS 9134 Switch Fabric switch 2 port 10 Gbps Fabric Yes No Auto Dedicated switch DS C9124 K9 24 port 4 Gbps Fabric Yes No Auto Dedicated taht Cisco MDS 9124 SWiteh Fabric switch DS C9222i K9 18 port 4 Gbps Fibre Yes Yes Auto Shared Cisco MDS 9222i Channel switch with 4 Gigabit Ethernet IP Multiservice Modular switch storage services ports and a modular expansion slot to host Cisco MDS 9000 Family Switching and Services Modules Supports shared rate mode 2 Shared rate mode is supported on Fx ports only and no ISLs 3 All ports in a 48 port 4 Gbps switching module can operate in dedicated rate mode with a 1 Gbps operating speed However if you configure one or more ports to operate in 2 Gbps or 4 Gbps dedicated rate mode some of the other ports in the port group would have to operate in shared mode 4 All ports in a 24 port 4 Gbps switching module can operate in dedicated rate mode with a 2 Gbps operating speed However if you configure one or more ports to operate in 4 Gbps dedicated rate mode some of the other ports in the port group would have to operate in shared mode Dedicated Rate Mode When port rate mode is configured as dedicated a port is allocated required fabric bandwidth and related resources to sustain line rate traffic at the ma
118. and 4 44 Port 8 Gbps Fibre Channel switching modules support the following features e 1 Gbps 2 Gbps 4 Gbps and 8 Gbps speed traffic e Shared and dedicated rate mode e ISL and Fx port modes e Extended BB credits The 48 port and 24 port 4 Gbps Fibre Channel switching modules support the following features e 1 Gbps 2 Gbps and 4 Gbps speed traffic e Shared and dedicated rate mode e ISL E or TE and Fx F or FL port modes e Extended BB credits Cisco MDS 9000 Family NX OS Interfaces Configuration Guide M326 OL 29284 01 Release 6 x Chapter3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces W Note Step 1 Step 2 Step 3 Step 4 Step 5 If you change the port bandwidth reservation parameters on a 48 port or 24 port 4 Gbps module the change affects only the changed port No other ports in the port group are affected To configure the 4 Gbps and 8 Gbps Fibre Channel switching modules when starting with the default configuration or when migrating from shared rate mode to dedicated rate mode follow these steps Take unused interfaces out of service to release resources for other interfaces if necessary See the Taking Interfaces Out of Service section on page 3 41 Configure the traffic speed to use 1 Gbps 2 Gbps 4 Gbps 8 Gbps or autosensing with a maximum of 2 Gbps or 4 Gbps See the Configuring Port Speed section on page 3 29 Configure the rate mode dedicated
119. and Reference To view and confirm that the type and value of virtual pWWNs are correct enter the show flex attach virtual wwn command See Example 8 1 Example 8 1 Displaying the Type and Value of Virtual pWWNs switch show flex attach virtual wwn VIRTUAL PORT WWNS ASSIGNED TO INTERFACES VSAN INTERFACE VIRTUAL PWWN AUTO LAST CHANGE Ec1 1 00 00 00 00 00 00 00 00 Fel 2 22 73 00 05 30 6e 1e TRUE Thu Jan 31 01 58 52 2008 fe1 3 22 5e 00 05 30 01 6e 1le TRUE Thu Jan 31 01 58 52 2008 fc1 4 22 5f 00 05 30 01 6e 1e TRUE Thu Jan 31 01 58 52 2008 fet 5 22 74 00 05 30 01 6e 1e TRUE Thu Jan 31 01 26 24 2008 fc1 6 22 60 00 05 30 01 6e 1e TRUE Thu Jan 31 01 58 52 2008 c1 7 22 61 00 05 30 01 6e 1e TRUE Thu Jan 31 01 58 52 2008 fe1 8 22 62 00 05 30 01 6e 1le TRUE Thu Jan 31 01 58 52 2008 fc1 9 22 63 00 05 30 01 6e 1e TRUE Thu Jan 31 01 58 52 2008 fc1 10 22 64 00 05 30 01 6e le TRUE Thu Jan 31 01 58 52 2008 fol fil 22 65 00 05 30 01 6e 1le TRUE Thu Jan 31 01 58 52 2008 el i2 22 66 00 05 30 01 6e 1le TRUE Thu Jan 31 01 58 52 2008 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x nE Chapter8 Configuring FlexAttach Virtual pWWN W Monitoring FlexAttach Virtual pWWN Verifying the End Device To verify that the end device is logged with the correct virtual WWNs use the show fens database command on the NPV core See Example 8 2 Example 8 2 Verifying th
120. and requirements when deploying NPV NPV core switches must support NPIV You can have up to 100 NPV devices Nondisruptive upgrades are supported See the Cisco MDS 9000 Family NX OS Fundamentals Configuration Guide Port tracking is supported See the Cisco MDS 9000 Family NX OS Security Configuration Guide You can configure zoning for end devices that are connected to NPV devices using all available member types on the NPV core switch If fWWN sWWN domain or port based zoning is used then fWWN sWWN or the domain port of the NPV core switch should be used Port security is supported on the NPV core switch for devices logged in via NPV NPV uses a load balancing algorithm to automatically assign end devices in a VSAN to one of the NPV core switch links in the same VSAN upon initial login If there are multiple NPV core switch links in the same VSAN then you cannot assign a specific one to an end device Both servers and targets can be connected to an NPV device Remote SPAN is not supported Local switching is not supported all traffic is switched using the NPV core switch Cisco MDS 9000 Family NX OS Interfaces Configuration Guide e E OL 29284 01 Release 6 x Chapter7 Configuring N Port Virtualization Guidelines and Limitations Hi NPV devices can connect to multiple NPV core switches In other words different NP ports can be connected to different NPV core switches NPV supports NPIV capable module servers
121. andwidth of the PortChannel A port can be configured as a member of a static PortChannel only if the following configurations are the same in the port and the PortChannel e Speed e Mode e Rate mode e Port VSAN e Trunking mode e Allowed VSAN list or VF ID list After the members are added regardless of the mode ACTIVE and ON used the ports at either end are gracefully brought down indicating that no frames are lost when the interface is going down see the Generation PortChannel Limitations section on page 6 14 and Graceful Shutdown section on page 2 9 Compatibility Check A compatibility check ensures that the same parameter settings are used in all physical ports in the channel Otherwise they cannot become part of a PortChannel The compatibility check is performed before a port is added to the PortChannel The check ensures that the following parameters and settings match at both ends of a PortChannel e Capability parameters type of interface Gigabit Ethernet at both ends or Fibre Channel at both ends e Administrative compatibility parameters speed mode rate mode port VSAN allowed VSAN list and port security amp Note Ports in shared rate mode cannot form a PortChannel or a trunking PortChannel e Operational parameters remote switch WWN and trunking mode A port addition procedure fails if the capability and administrative parameters in the remote switch are incompatible wi
122. annel member ports at either end Port initialization is not synchronized There is synchronized startup of all ports in a channel across peer switches All misconfigurations are not detected as no Consistently detect misconfigurations using a protocol is exchanged PortChannel protocol Transitions misconfigured ports to the suspended Transitions misconfigured ports to the isolated state You must explicitly disable shut and state to correct the misconfiguration Once you enable no shut the member ports at either end correct the misconfiguration the protocol ensures automatic recovery This is the default mode You must explicitly configure this mode PortChannel Deletion When you delete the PortChannel the corresponding channel membership is also deleted All interfaces in the deleted PortChannel convert to individual physical links After the PortChannel is removed regardless of the mode used ACTIVE and ON the ports at either end are gracefully brought down indicating that no frames are lost when the interface is going down see the Graceful Shutdown section on page 2 9 If you delete the PortChannel for one port then the individual ports within the deleted PortChannel retain the compatibility parameter settings speed mode port VSAN allowed VSAN and port security You can explicitly change those settings as required e If you use the default ON mode to avoid inconsistent states across switche
123. ared pool e There are 2488 extra buffers available as extended BB_credits after allocating all the default BB_credit buffers for all the ports in ISL mode 5488 750 4 amp Note Extended BB_credits are allocated across all ports on the switch That is they are not allocated by port group Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x EEN Chapter 4 Configuring Interface Buffers HZ Information About Interface Buffers BB_ Credit Buffers for Fabric Switches This section describes how buffer credits are allocated to Cisco MDS 9000 Fabric switches and includes the following topics e Cisco MDS 9148 Fabric Switch BB_Credit Buffers page 4 16 e Cisco MDS 9148 Fabric Switch BB_Credit Buffers page 4 16 e Cisco MDS 9124 Fabric Switch BB_Credit Buffers page 4 17 e Cisco MDS 9222 Multiservice Modular Switch BB_Credit Buffers page 4 17 Cisco MDS 9148 Fabric Switch BB_ Credit Buffers Table 4 10 lists the BB_credit buffer allocation for 48 port 8 Gbps Fibre Channel switches Table 4 10 48 Port 8 Gbps Fabric Switch BB_Credit Buffer Allocation BB_Credit Buffers Per BB_Credit Buffer Allocation Type Port Group BB_Credit Buffers Per Port ISL Fx Port Default BB_credit buffers 128 32 32 Maximum configurable BB_credit buffers on 8 Gbps 128 125 125 mode 1 ISL E port or TE port The following considerations apply to BB_credit buffers on 48 port 8
124. ass f frames 0 bytes 0 discards 0 input OLS 0 LRR 0 NOS 0 loop inits 0 output OLS 0 LRR 0 NOS 0 loop inits 0 link failures 0 sync losses 0 signal losses 16 receive B2B credit remaining 3 transmit B2B credit remaining sup fc0 114000 packets input 0 multicast frames 0 input errors 113997 packets output 0 output errors 0 carrier errors mgmt0 31557 packets input 0 multicast frames 0 input errors 26618 packets output 0 frame 0 collisions 0 frame 11585632 bytes 0 compressed 0 overrun 0 fifo 10969672 bytes 0 underruns 0 fifo 2230860 bytes 0 compressed 0 overrun 0 fifo 16824342 bytes 0 underruns 0 output errors 0 collisions 7 fifo 0 carrier errors vsanl 0 packets input 0 bytes 0 errors 0 multicast 0 packets output 0 bytes 0 errors 0 dropped port channel 1 5 minutes input rate 0 bits sec 5 minutes output rate 0 frames input class 2 frames class 3 frames class f frames discards 0 CRC 0 too long 0 frames output O O 2 4 0 bytes sec 0 bytes sec 0 frames sec 0 bits sec 0 frames sec 0 bytes 0 bytes 0 bytes 0 bytes 0 unknown class 0 too short 0 bytes 0 class 2 frames 0 bytes 0 class 3 frames 0 bytes 0 class f frames 0 bytes 0 discards 0 input OLS 0 LRR 0 NOS 0 loop inits 0 output OLS 0 LRR 0 NOS 0 loop inits 0 link failures amp 0 sync losses 0 signal losses Note Interfaces 9 8 and 9 9 are not trunkin
125. at you can provision the ports across port groups evenly to manage the oversubscription better When the port group monitor feature is enabled and a policy consisting of polling interval in seconds and the raising and falling thresholds in percentage are specified port group monitor generates a syslog if a port group traffic goes above the specified percentage of the maximum supported bandwidth for that port group for rx and for tx and another syslog if the value falls below the specified threshold The default port group policy has the following threshold values Counter Threshold Type Rising Threshold Falling Threshold RX Performance Delta 60 80 120 Delta 60 80 20 Interval Seconds TX Performance Local Switching Local switching can be enabled in Generation 4 modules which allows traffic to be switched directly with a local crossbar when the traffic is directed from one port to another on the same line card By using local switching an extra switching step is avoided which decreases the latency When using local switching note the following guidelines e All ports need to be in shared mode which usually is the default state To place a port in shared mode enter the switchport ratemode shared command e E ports are not allowed in the module because they must be in dedicated mode Note Local Switching is not supported on MDS 9710 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide
126. atabase command switch show fens database FCID TYPE PWWN VENDOR FC4 TYPE FEATURE 0x010000 N 20 01 00 0d ec 2f c1 40 Cisco npv 0x010001 N 20 02 00 0d ec 2f c1 40 Cisco npv 0x010200 N 21 00 00 e0 8b 83 01 al Qlogic scsi fcp init 0x010300 N 21 01 00 e0 8b 32 1a 8b Qlogic scsi fep init Total number of entries 4 For additional details such as IP addresses switch names interface names about the NPV devices you see in the show fens database output enter the show fens database detail command switch show fens database detail port wwn vendor 20 01 00 0d ec 2f c1 40 Cisco node wwn 20 00 00 0d ec 2f c1 40 class 2 3 node ip addr 172 20 150 38 ipa ff ff ff ff ff ff ff ff fc4 types fc4_features npv symbolic port name para 3 fcl1 1 symbolic node name para 3 port type N port ip addr 0 0 0 0 fabric port wwn 20 01 00 0d ec 04 99 40 hard addr 0x000000 permanent port wwn 20 01 00 0d ec 2f c1 40 vendor Cisco port wwn vendor 20 02 00 0d ec 2f c1 40 node wwn 20 00 00 0d ec 2f c1 40 class 22 3 node ip addr 172 20 150 38 ipa f ff ff fc4 types fc4_features npv symbolic port name para 3 fcl1 2 symbolic node name para 3 port type N port ip addr 0 0 0 0 fabric port wwn 20 02 00 0d ec 04 99 40 hard addr 0x000000 permanent port wwn Cisco vendor 20 02 00 0d ec 2f c1 40 Cisco If you need to contact support enter the show tech suppor
127. ation 3 or Generation 4 Auto Auto Generation l or Normal or force Pass Generation 2 or Generation 3 or Generation 4 Auto Auto max 2000 Generation2 or Normal Fail Generation 3 or Farce Pass t Generation 4 fail Auto Auto max 4000 Generation 3 or Generation 4 Auto max 2000 Auto Generation2 or Normal Fail Generation 3 or Forc Pass Generation 4 Auto max 2000 Auto max 4000 Generation 3 or or Generation 4 Auto max 4000 Auto Generation 2 or Generation 3 or or Generation 4 Auto max 4000 Auto max 2000 Generation 2 or Generation 3 or or Generation 4 Generation 1 Auto Auto Generation2 or Normal Fail interfaces Generation 3 Bored Pass Auto max 2000 Auto Generation 1 Normal or force Pass Auto max 2000 Auto Generation 2or Normal Fail Generation 3 Force Pass or fail Auto max 4000 Auto Generation 1 or Generation 2 Auto max 4000 Auto Generation 3 OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide E Chapter 3 Configuring Fibre Channel Interfaces W Guidelines and Limitations Table 3 11 PortChannel Configuration and Addition Results continued PortChanne Configured Speed New Member Members PortChannel New Member Type Addition Type Result Generation 2 Auto Auto Generation 1 Normal or force Fail interfaces Auto max 2000 Auto Generation Normal or force Pass Auto max
128. auto Rate mode shared shared I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces W Configuring Fibre Channel Interfaces Table 3 14 Default Generation 4 Interface Parameters continued Default 48 Port 8 Gbps Advanced Fibre 32 Port 8 Gbps Advanced Fibre Parameter Channel Switching Module Channel Switching Module Port mode Fx Fx BB credit buffers 32 32 Configuring Fibre Channel Interfaces This section includes the following topics e Task Flow for Migrating Interfaces from Shared Mode to Dedicated Mode page 3 26 e Task Flow for Migrating Interfaces from Dedicated Mode to Shared Mode page 3 27 e Task Flow for Configuring 12 Port 4 Gbps Module Interfaces page 3 28 e Task Flow for Configuring 4 Port 10 Gbps Module Interfaces page 3 28 e Configuring Port Speed page 3 29 e Configuring Rate Mode page 3 30 e Configuring Local Switching page 3 36 e Disabling Restrictions on Oversubscription Ratios page 3 37 e Enabling Restrictions on Oversubscription Ratios page 3 39 e Enabling Bandwidth Fairness page 3 40 e Disabling Bandwidth Fairness page 3 41 e Taking Interfaces Out of Service page 3 41 e Releasing Shared Resources in a Port Group page 3 42 e Disabling ACL Adjacency Sharing for System Image Downgrade page 3 43 Task Flow for Migrating Interfaces from Shared Mode to Dedicated Mode The 48 Port 24 Port
129. bar counter poll interval in seconds the thresholds in the count of error frame events and the event IDs of events switch config port monitor counter invalid cre poll interval seconds delta rising threshold percentagei event event id falling threshold percentage event event id Specifies the delta invalid CRC poll interval in seconds the thresholds in percentage and the event IDs of events to be triggered switch config port monitor counter invalid words poll interval seconds delta rising threshold percentagei event event id falling threshold percentage event event id Specifies the delta invalid words poll interval in seconds the thresholds in percentage and the event IDs of events to be triggered Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter 2 Configuring Interfaces Step 10 Step 11 Step 12 Step 13 Step 14 Step 15 Step 16 Step 17 Step 18 Step 19 Configuring Interfaces Command Purpose switch config port monitor counter link loss poll interval seconds delta rising threshold percentagei event event id falling threshold percentage event event id Specifies the delta link failure counter poll interval in seconds the thresholds in percentage and the event IDs of events to be triggered switch config port monitor counter protocol error poll interval seconds delta rising threshold
130. ble dedicated buffers are 5478 Port Group 1 Total bandwidth is 12 8 Gbps Total shared bandwidth is 0 8 Gbps Allocated dedicated bandwidth is 12 0 Gbps B2B Credit Bandwidth Rate Mode Buffers Gbps OL 29284 01 Release 6 x Chapter 3 Configuring Fibre Channel Interfaces fc8 1 16 fc8 2 16 fc8 3 16 fc8 4 out of service fc8 5 out of service fc8 6 out of service Information About Fibre Channel Interfaces 4 0 dedicated 4 0 dedicated 4 0 dedicated For dedicated ports oversubscription ratio restrictions do not apply to the shared pool in port groups So if oversubscription ratio restrictions are disabled and you have configured three 4 Gbps dedicated ports in one port group then you can configure all other ports in the same port group to operate at a shared rate of 4 Gbps In the following example a 48 port module has a group of six ports four dedicated ports are operating at 8 Gbps and the two shared ports are also operating at 8 Gbps switch show port resources module 5 Module 5 Available dedicated buffers for global buffer 0 Available dedicated buffers for global buffer Available dedicated buffers for global buffer Available dedicated buffers for global buffer Available dedicated buffers for global buffer Available dedicated buffers for global buffer Available dedicated buffers for global buffer Available dedicated buffers for global buffer Port Group 1 Total bandwidth is 32 4 Gbps
131. bps 8 0 shared 8 0 shared 8 0 shared 8 0 shared 8 0 shared 8 0 shared are are are are are are are are 3880 3880 3880 3056 3880 3880 3880 3880 OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces HZ information About Fibre Channel Interfaces Interfaces in the Port Group B2B Credit Buffers f c13 19 250 f c13 20 250 fc13 21 250 c13 22 250 c13 23 32 c13 24 32 Port Group 5 Total bandwidth is 32 4 Gbps Total shared bandwidth is 32 4 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Buffers fc13 25 32 fc13 26 32 fc13 27 32 fc13 28 32 fc13 29 32 fc13 30 32 Port Group 6 Total bandwidth is 32 4 Gbps Total shared bandwidth is 32 4 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Buffers fc13 31 32 c13 32 32 c13 33 32 c13 34 32 c13 35 32 c13 36 32 Port Group 7 Total bandwidth is 32 4 Gbps Total shared bandwidth is 32 4 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Buffers c13 37 32 c13 38 32 c13 39 32 f c13 40 32 c13 41 32 c13 42 32 Port Group 8 Total bandwidth is 32 4 Gbps Total shared bandwidth is 32 4 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Buffers Bandwidth Gbps Band
132. bre Channel switching module 32 port 8 Gbps Advanced Yes Yes Auto Shared Fibre Channel switching DS X9232 256K9 module Generation 3 Modules DS X9248 96K9 48 Port 8 Gbps Fibre Yes Yes Auto Shared Channel switching module DS X9224 96K9 24 Port 8 Gbps Fibre Yes Yes Auto Shared Channel switching module DS X9248 48K9 4 44 Port 8 Gbps Yes Yes Auto Max 4 Gbps Host Optimized Fibre Shared Channel switching module Generation 3 Fabric Switches DS C9148 K9 48 port 8 Gbps Fabric Yes No Auto Dedicated Cisco MDS 9148 Switch Fabric switch Generation 2 Modules DS X9148 48 port 4 Gbps Fibre Yes Yes Auto Shared Channel switching module 3 DS X9124 24 port 4 Gbps Fibre Yes Yes Auto Shared Channel switching module DS X9304 18K9 18 port 4 Gbps Fibre Yes Yes Auto Shared MSM 18 4 Channel switching module Multi with 4 Gigabit Ethernet ultiservice module ports DS X9112 12 port 4 Gbps Fibre Yes No Auto Dedicated Channel switching module DS X9704 4 port 10 Gbps Fibre Yes No Auto Dedicated Channel switching module Generation 2 Fabric Switches E Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x _ Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Table 3 3 Port Rate Mode Support on Generation 2 Generation 3 and Generation 4 Modules and Switches continued Supports Supports Default Speed Product N
133. buffers Available dedicated buffers Available dedicated buffers Port Group 1 Total bandwidth is 32 4 Gbps module 13 for for for for for for for for Q 9 9 9 9 9 9 Q lobal Lobal Lobal Lobal Lobal Lobal Lobal Lobal Total shared bandwidth is 32 4 Gbps 32 32 buffer buffer buffer buffer buffer buffer buffer buffer Allocated dedicated bandwidth is 0 0 Gbps fc13 1 c13 2 fc13 3 fc13 4 fc13 5 fc13 6 Port Group 2 Total bandwidth is 32 4 Gbps B2B Credit Total shared bandwidth is 32 4 Gbps Buffers Allocated dedicated bandwidth is 0 0 Gbps fc13 7 fc13 8 fc13 9 fc13 10 13711 Fo13 12 Port Group 3 Total bandwidth is 32 4 Gbps B2B Credit Total shared bandwidth is 32 4 Gbps Buffers Allocated dedicated bandwidth is 0 0 Gbps c13 13 c13 14 fc13 15 fc13 16 c13 17 c13 18 Port Group 4 Total bandwidth is 32 4 Gbps Total shared bandwidth is 0 4 Gbps B2B Credit Buffers Allocated dedicated bandwidth is 32 0 Gbps Information About Fibre Channel Interfaces 8 0 shared 8 0 shared 0 port group 1 1 port group 2 2 port group 3 3 port group 4 4 port group 5 5 port group 6 6 port group 7 7 port group 8 Bandwidth Rate M Gbps 8 0 shared 8 0 shared 8 0 shared 8 0 shared 8 0 shared 8 0 shared Bandwidth Rate M Gbps 8 0 shared 8 0 shared 8 0 shared 8 0 shared 8 0 shared 8 0 shared Bandwidth Rate M G
134. c 2 modules but requires Fabric 2 module for support of the 48 port and the 24 port 8 Gbps Fibre Channel switching modules The MDS 92221 switch supports the 4 44 port Host Optimized Fibre Channel switching module The Cisco 9500 Series switches support the following Generation 4 modules the 48 port 8 Gbps Advanced Fibre Channel switching module DS X9248 256K9 and the 32 port 8 Gbps Advanced Fibre Channel module DS X9232 256K9 Cisco MDS NX OS Release 5 2 1 or higher is required to support the Generation 4 modules Table 3 1 identifies the Generation 2 Generation 3 and Generation 4 modules as well as the Fabric switches Table 3 1 Fibre Channel Modules and Fabric Switches Part Number Product Name and Description Generation 4 Modules DS X9248 256K9 48 port 8 Gbps Advanced Fibre Channel switching module DS X9232 256K9 32 port 8 Gbps Advanced Fibre Channel switching module DS X9530 SF2A K9 Supervisor 2A module for Cisco MDS 9500 Series switches DS 13SLT FAB3 Fabric 3 module that enables the 32 port and the 48 port 8 Gbps Advanced Fibre Channel switching module to use the full 96 Gbps or 256 Gbps backplane crossbar bandwidth Generation 3 Modules DS X9248 96K9 48 port 8 Gbps Fibre Channel switching module DS X9224 96K9 24 port 8 Gbps Fibre Channel switching module DS X9248 48K9 4 44 port 8 Gbps Host Optimized Fibre Channel switching module DS 13SLT FAB2 Fabric 2 module that enables the 24 port and
135. ce registers itself with the fabric name server using the following additional parameters e Switch name and interface name for example fc1 4 of the NP port is embedded in the symbolic port name in the name server registration of the NPV device itself e The IP address of the NPV device is registered as the IP address in the name server registration of the NPV device Note The BB_SCN of internal FLOGIs on NP ports is always set to zero The BB_SCN is supported at the F port of the NPV device I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter7 Configuring N Port Virtualization HZ information About N Port Virtualization Figure 7 4 shows the internal FLOGI flows between an NPV core switch and an NPV device Figure 7 4 Internal FLOGI Flows NPV Core Switch aan fe 5 10 fwwn fe 1 5 wwn az NPV Device 184572 Table 7 1 identifies the internal FLOGI parameters that appear in Figure 7 4 Table 7 1 Internal FLOGI Parameters Parameter Derived From pWWN The fWWN of the NP port nWWN The VSAN based sWWN of the NPV device fWWN The fWWN of the F port on the NPV core switch symbolic port name The switch name and NP port interface string Note If there is no switch name available then the output will display switch For example switch fcl 5 IP address The IP address of the NPV device symbolic node name The NPV switc
136. ceive Data Field Size Detailed Steps To configure the receive data field size follow these steps Command Purpose Step1 switch config t Enters configuration mode Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Ez OL 29284 01 Release 6 x Chapter4 Configuring Interface Buffers Step 2 Step 3 Verifying B Verifying BB_Credit Configuration Command Purpose switch config interface fc1 1 Selects a Fibre Channel interface and enters switch config if interface configuration submode switch config if switchport fcrxbufsize 2000 Reduces the data field size for the selected interface to 2000 bytes The default is 2112 bytes and the range is from 256 to 2112 bytes B Credit Configuration To display BB_credit configuration information perform one of the following tasks Command Purpose show interface Displays the interface configuration show interface bbcredit Displays the BB_credit configuration for all the interfaces show interface fc numbers bbcredit Displays the BB_credit configuration for the specified interfaces For detailed information about the fields in the output from these commands refer to the Cisco NX OS Command Reference To display the BB_credit information use the show interface bbcredit command see Example 4 1 and Example 4 2 Example 4 1 Displays BB_credit Information switch show interface bbcredit fc2 1 is down SFP not pr
137. ches and is referred to as a nontrunking interface See Figure 6 2 and Figure 6 3 See Chapter 5 Configuring Trunking for information on trunked interfaces Figure 6 2 Trunking Only Switch 1 Any other Switch 1 I 2 switch ISL P EISL pafe port E port a pajre port a 79938 lea Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x a 63 Chapter6 Configuring PortChannels HZ information About PortChannels Figure 6 3 PortChanneling and Trunking Switch 1 Switch 2 Switch 1 Sieh 2 aA RA Port e Port channel 79939 and trunking PortChanneling and trunking are used separately across an ISL e PortChanneling Interfaces can be channeled between the following sets of ports E ports and TE ports F ports and NP ports TF ports and TNP ports e Trunking Trunking permits carrying traffic on multiple VSANs between switches See the Cisco MDS 9000 Family NX OS Fabric Configuration Guide e Both PortChanneling and trunking can be used between TE ports over EISLs Load Balancing Two methods support the load balancing functionality e Flow based All frames between source and destination follow the same links for a given flow That is whichever link is selected for the first exchange of the flow is used for all subsequent exchanges e Exchange based The first frame in an exchange picks a link and subsequent frames in the exchange follow
138. config npv no npv enable switch config Terminates session and disables NPV mode which results in a reload of the NPV device Configuring NPV Traffic Management Configuring List of External Interfaces per Server Interface Detailed Steps Step 1 Step 2 The NPV traffic management feature is enabled after configuring NPV Configuring NPV traffic management involves configuring a list of external interfaces to the servers and enabling or disabling disruptive load balancing This section includes the following topics e Configuring List of External Interfaces per Server Interface page 7 12 e Enabling the Global Policy for Disruptive Load Balancing page 7 13 A list of external interfaces are linked to the server interfaces when the server interface is down or if the specified external interface list includes the external interface already in use To configure the list of external interfaces per server interface perform the following tasks Command Purpose switch config t switch config Enters configuration mode on the NPV switch config npv traffic map server interface svr if range external interface fc ext fc 1f range switch config Allows you to configure a list of external FC interfaces per server interface by specifying the external interfaces in the svr if range The server to be linked is specified in the ext fc if range switch config npv traffic map server
139. configuration with information for all interfaces The interfaces have multiple entries in the configuration files to ensure that the interface configuration commands execute in the correct order when the switch reloads Example 2 11 Displays the Running Configuration for All Interfaces switch show running config interface fc9 1 switchport speed 2000 interface fc9 1 switchport mode E interface fc9 1 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide e248 E OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Verifying Interfaces Configuration W channel group 11 force no shutdown Example 2 12 displays the running configuration information for a specified interface The interface configuration commands are grouped together Example 2 12 Displays the Running Configuration for a Specified Interface switch show running config interface fcl1 1 interface fc9 1 switchport speed 2000 switchport mode E channel group 11 force no shutdown Example 2 13 displays the running configuration after the system default switchport mode F command is executed Example 2 13 Displays the Running Configuration After the System Default Switchport Mode F Command is Executed switch show running config version 3 1 3 system default switchport mode F interface fc4 1 interface fc4 2 interface fc4 3 interface fc4 4 interface fc4 5 interface fc4 6 interface fc4 7 interface fc4 8 interface fc4 9 in
140. counter tx performance 3Turns off Tx performance monitoring 1 See Reverting to the Default Policy for a Specific Counter page 2 36 See Port Group Monitor page 2 16 See Turning Off the Monitoring of Specific Counter page 2 36 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces HZ Configuring Interfaces Reverting to the Default Policy for a Specific Counter When the no counter command is used in the config port group monitor mode the specified counter polling values will revert to the default values as seen in the following example switch config port group monitor name PGMON_policy switch config port group monitor counter tx performance poll interval 100 delta rising threshold 65 falling threshold 25 switch config show port group monitor PGMON_policy Policy Name PGMON_policy Admin status Not Active Oper status Not Active Port type All Port Groups Counter Threshold Interval ge Rising Threshold ge Falling Threshold RX Performance Delta 60 80 10 TX Performance Delta 100 65 25 switch config port group monitor name PGMON_policy switch config port group monitor no counter tx performance switch config show port group monitor PGMON_policy Policy Name PGMON_policy Admin status Not Active Oper status Not Active Port type All Port Groups Counter Threshold Interval ge Rising Threshold ge Falling Thre
141. ct as a peer with NPV This prevents the FlexAttach configuration from getting distributed to other peer NPVs in the network NP port uses physical pWWN instead of virtual pWWN configured through FlexAttach This occurs when NP port uses physical pWWN instead of virtual pWWN that is configured through FlexAttach FlexAttach is supported on server interfaces such as F ports and not on external interfaces such as NP ports Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter8 Configuring FlexAttach Virtual pWWN Error Description Monitoring FlexAttach Virtual pWwWN W Workaround real port WWN and virtual WWN cannot be same This occurs when you try to configure FlexAttach with a similar value for pWWN and virtual pWWN Use different values for pWWN and virtual pWWYN as similar values for pWWN and virtual pWWan are not allowed Virtual port WWN already exists This occurs when you try to configure an already defined pWWN to a different interface Use an undefined virtual pWWN for a new interface I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter8 Configuring FlexAttach Virtual pWWN W Monitoring FlexAttach Virtual pWWN Cisco MDS 9000 Family NX OS Interfaces Configuration Guide 8 8 OL 29284 01 Release 6 x CHAPTER Configuring Port Tracking The port tr
142. ctive 5 7 configuring allowed active lists 5 12 configuring trunk allowed lists 5 5 to loop devices 2 50 mismatches 2 9 port tracking 9 5 TE port mode 2 4 TF port mode 2 4 trunk allowed 5 7 VSAN trunking See trunking Index W Ww WWNs suspended connections 2 9 Z zones merge failures 2 9 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide W index Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Pie i OL 29284 01 Release 6 x
143. d Switches Cisco MDS 9000 Family hardware modules and switches are categorized into generations based on the time of introduction capabilities features and compatibilities Generation 1 Modules and switches with a maximum port speed of 2 Gbps Generation 2 Modules and switches with a maximum port speed of 4 Gbps Generation 3 Modules and switches with a maximum port speed of 8 Gbps Generation 4 Modules with a maximum port speed of 8 Gbps or 10 Gbps I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces HZ Information About Fibre Channel Interfaces The Cisco MDS 9500 Series switches Cisco MDS 92221 Cisco MDS 9216A and Cisco MDS 92161 switches support the Generation 2 modules Each module or switch can have one or more ports in port groups that share common resources such as bandwidth and buffer credits In addition to supporting Generation 2 modules the Cisco MDS 9500 Series switches and the Cisco MDS 922721 switch support the Generation 3 modules Similar to Generation 2 each Generation 3 or Generation 4 module can have one or more ports in port groups that share common resources such as bandwidth and buffer credits Generation 3 modules are supported on the Cisco MDS 9506 and 9509 switches with Supervisor 2 modules The MDS 9513 Director supports 4 44 port Host Optimized Fibre Channel switching module with either Fabric 1 or Fabri
144. d Trunking 6 3 Load Balancing 6 4 PortChannel Modes 6 6 PortChannel Deletion 6 7 Interfaces ina PortChannel 6 7 5 11 5 12 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x CHAPTER 7 Interface Addition to a PortChannel 6 8 Forcing an Interface Addition 6 9 Interface Deletion from a PortChannel 6 9 PortChannel Protocols 6 9 Channel Group Creation 6 10 Autocreation 6 11 Manually Configured Channel Groups 6 12 Prerequisites for PortChannels 6 12 Guidelines and Limitations 6 13 General Guidelines and Limitations 6 13 Generation 1 PortChannel Limitations 6 14 F and TF PortChannel Limitations 6 14 Valid and Invalid PortChannel Examples 6 15 Default Settings 6 16 Configuring PortChannels 6 17 Configuring PortChannels Using the WizardCreating a PortChannel Configuring the PortChannel Mode 6 17 Deleting PortChannels 6 18 Adding an Interface to a PortChannel 6 18 Forcing an Interface Addition 6 19 Deleting an Interface from a PortChannel 6 19 Enabling and Configuring Autocreation 6 20 Converting to Manually Configured Channel Groups 6 20 Verifying PortChannel Configuration 6 20 Configuration Examples for F and TF PortChannels 6 24 Configuring N Port Virtualization 7 1 Information About N Port Virtualization 7 1 NPV Overview 7 1 N Port Identifier Virtualization 7 2 N Port Virtualization 7 2 NPV Mode 7 4 NP Ports 7 5 NP Links 7 5 Internal FLOGI Parameters 7 5 Default Port Numbers 7
145. d domain ID is not valid assignment failure Isolation due to the other side The E port at the other end of the link is isolated of the link E port isolated Isolation due to invalid fabric The port is isolated due to fabric reconfiguration reconfiguration Isolation due to domain The fcdomain feature is disabled manager disabled Isolation due to zone merge The zone merge operation failed failure Isolation due to VSAN The VSANs at both ends of an ISL are different mismatch Nonparticipating FL ports cannot participate in loop operations It may Only FL happen if more than one FL port exists in the same ports and TL loop in which case all but one FL port in that loop ports automatically enters nonparticipating mode PortChannel administratively The interfaces belonging to the PortChannel are down Only down PortChannel interfaces Suspended due to incompatible speed The interfaces belonging to the PortChannel have incompatible speeds Suspended due to incompatible mode The interfaces belonging to the PortChannel have incompatible modes Suspended due to incompatible remote switch WWN An improper connection is detected All interfaces in a PortChannel must be connected to the same pair of switches Graceful Shutdown Interfaces on a port are shut down by default unless you modified the initial configuration The Cisco NX OS software implicitly performs a graceful shutdown in response to either of the fo
146. dth oversubscription of 1 5 1 considering that each port group has 32 Gbps bandwidth In case of 32 Port version each port group of 4 ports has sufficient bandwidth 32 Gbps to handle the line rate traffic without any oversubscription The following example configurations are supported by the 48 port Advanced 8 Gbps Fibre Channel switching modules e Six ports with shared rate mode and 8 Gbps speed 1 5 1 oversubscription default e Two port with dedicated rate mode and 8 Gbps speed plus four ports with shared rate mode and 8 Gbps speed 2 1 oversubscription Cisco MDS 9000 Family NX OS Interfaces Configuration Guide mae W OL 29284 01 Release 6 x Chapter4 Configuring Interface Buffers Information About Interface Buffers W Two ports with dedicated rate mode and 8 Gbps speed plus four ports with shared rate mode and 8 Gbps speed 2 1 oversubscription One port with dedicated rate mode and 8 Gbps speed plus three ports with dedicated rate mode and 4 Gbps speed plus two ports with shared rate mode and 8 Gbps speed 1 33 1 oversubscription Six ports with dedicated rate mode and 8 Gbps speed 48 Port 8 Gbps Fibre Channel Module BB_Credit Buffers Table 4 2 lists the BB_credit buffer allocation for the 48 port 8 Gbps Fibre Channel switching module Table 4 2 48 Port 8 Gbps Switching Module BB_Credit Buffer Allocation po BB Credit Buffers Per Pot Dedicated Rate Mode Shared Rate Mode 8 Gb
147. dth for the interface Note If you cannot reserve dedicated bandwidth on an interface you might have exceeded the port group maximum bandwidth Use the show port resources command to determine what resources are already allocated switch config if switchport rate mode shared Reserves shared default bandwidth for the interface switch config if no switchport rate mode Reverts to the default state shared Displaying the Rate Mode Configuration for Interfaces Use show port resources module command to verify the rate mode configuration for interfaces on a 48 port or 24 port 4 Gbps or any 8 Gbps Fibre Channel switching module This example shows the port rate mode configuration for interfaces on a 4 Gbps module switch show port resources module 9 Module 9 Available dedicated buffers are 5400 Port Group 1 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Buffers fc9 1 16 fc9 2 16 fc9 3 16 fc9 4 16 fc9 5 16 fc9 6 16 Port Group 2 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Buffers fc9 7 16 fc9 8 16 fc9 9 16 fc9 10 16 fc9 11 16 Ee9 T2 16 Port Group 3 Total bandwidth is 12 8 Gbps Gbps shared shared shared shared shared Gbps 0 shared 0 shared 0 shared 0 shared
148. e auto parameter enables autosensing on the interface switch config if switchport speed auto On 4 Gbps modules configures autosensing for the interface with 4 Gbps of bandwidth reserved On 8 Gbps modules configures autosensing for the interface with 8 Gbps of bandwidth reserved 7 switch config if switchport speed auto max Configures autosensing with a maximum of 2000 2 Gbps of bandwidth reserved switch config if switchport speed auto max 8 Gbps modules only Configures autosensing 4000 with a maximum of 4Gbps of bandwidth reserved switch config if no switchport speed Reverts to the default speed for the interface auto 1 The 8000 and auto max 4000 speed configurations are available only for the 8 Gbps modules 2 The default speed on 48 port and 24 port 4 Gbps modules is 4000 The default speed on 48 port and 24 port 8 Gbps modules is 8000 The default speed on the 4 44 port 8 Gbps module is auto max 4000 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x EN Chapter 3 Configuring Fibre Channel Interfaces W Configuring Fibre Channel Interfaces Use the show interface command to verify the port speed configuration for an interface on a 4 Gbps or 8 Gbps Fibre Channel switching module switch show interface fc 9 1 fc9 1 is up Hardware is Fibre Channel SFP is short wave laser w o OFC SN Port WWN is 22 01 00 05 30 01 9f 02 Admin port mod
149. e 2 22 Configuring Interface Modes page 2 22 Configuring System Default Port Mode F page 2 23 Configuring Port Administrative Speeds page 2 25 Configuring Port Speed Group page 2 26 Configuring the Interface Description page 2 26 Specifying a Port Owner page 2 27 Configuring Beacon Mode page 2 27 Disabling Bit Error Threshold page 2 28 Configuring Switch Port Attribute Default Values page 2 28 Configuring TL Ports page 2 29 Manually Inserting Entries into the ALPA Cache page 2 29 Clearing the ALPA Cache page 2 29 Configuring Port Guard page 2 30 Configuring Port Monitor page 2 31 Configuring Port Group Monitor page 2 34 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Configuring Interfaces e Configuring Management Interfaces page 2 37 e Creating VSAN Interfaces page 2 38 e Configuring Slow Drain Device Detection and Congestion Avoidance page 2 38 For more information on configuring mgmt0 interfaces refer to the Cisco MDS 9000 Family NX OS Fundamentals Configuration Guide and Cisco MDS 9000 Family NX OS IP Services Configuration Guide For more information on configuring Gigabit Ethernet interfaces see the Cisco MDS 9000 Family NX OS IP Services Configuration Guide Configuring Fibre Channel Interfaces Detailed Steps Step 1 Step 2 Step 1 Step 2 To configure a Fibre Channel interface follow these steps
150. e End Device switch show fens database VSAN 1 PWWN VENDOR FC4 TYPE FEATURE 0x010000 N 20 01 00 0d ec 0x010001 N 20 02 00 0d ec 0x010200 N 21 00 00 e0 8b 83 01 0x010300 N 21 01 00 e0 8b 32 2f c1 40 Cisco 2 c1 40 Cisco al Qlogic la 8b Qlogic Total number of entries 4 Monitoring FlexAttach Virtual pWWN npv npv scsi fep init scsi fcp init Table 8 2 lists the errors that might be displayed and provides the workarounds Table 8 2 FlexAttach Errors and Workarounds Error Description Workaround fcl 1 interface is not down FlexAttach configuration fails because the configuration is enabled for an active interface with the operation state as up To move the port to the shut state enable the FlexAttach configuration and then move the port to no shut state FlexAttach configuration is not distributed to the peers The FlexAttach configuration on one peer NPV is not available to any other peer NPV FlexAttach configuration will not be distributed if cfs ipv4 distribute or cfs ipv6 distribute is disabled Enable cfs ipv4 distribute or cfs ipv6 distribute Even with CFS distribution enabled Inagua does not become a peer with other NPVs CFS over IP is enabled and the Inagua in one blade center is not the peer NPV for other NPVs CFS over IP uses IP multicast to discover the NPV peers in the network IBM MM does not support multicast and cannot a
151. e and enters switch config if interface configuration submode Step3 9 switch config if switchport ignore Prevents the detection of bit error threshold DIES SEEOES events from disabling the interface switch config if no switchport ignore Prevents the detection of bit error threshold bit errors events from enabling the interface Troubleshooting Tips e Regardless of the setting of the switchport ignore bit errors command the switch generates a syslog message when bit error threshold events are detected Configuring Switch Port Attribute Default Values You can configure attribute default values for various switch port attributes These attributes will be applied globally to all future switch port configurations even if you do not individually specify them at that time Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Detailed Steps Step 1 Step 2 Configuring Interfaces W To configure switch port attributes follow these steps Command Purpose switch config t Enters configuration mode switch config no system default switchport shutdown switch config Configures the default setting for administrative state of an interface as Up The factory default setting is Down Tip This command is applicable only to interfaces for which no user configuration exists for the administrative state
152. e capability and configurational parameters Any change in configuration applied to the associated PortChannel interface is propagated to all members of the channel group A protocol to exchange PortChannel configurations is available in all Cisco MDS switches This addition simplifies PortChannel management with incompatible ISLs An additional autocreation mode enables ISLs with compatible parameters to automatically form channel groups without manual intervention OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter6 Configuring PortChannels HZ information About PortChannels The PortChannel protocol is enabled by default The PortChannel protocol expands the PortChannel functional model in Cisco MDS switches It uses the exchange peer parameters EPP services to communicate across peer ports in an ISL Each switch uses the information received from the peer ports along with its local configuration and operational values to decide if it should be part of a PortChannel The protocol ensures that a set of ports are eligible to be part of the same PortChannel They are only eligible to be part of the same PortChannel if all the ports have a compatible partner The PortChannel protocol uses two subprotocols e Bringup protocol Automatically detects misconfigurations so you can correct them This protocol synchronizes the PortChannel at both ends so that all frames for a given flow as ide
153. e fc2 15 fc2 15 is trunking Hardware is Fibre Channel SFP is short wave laser w o OFC SN Cisco MDS 9000 Family NX OS Interfaces Configuration Guide a OL 29284 01 Release 6 x Chapter5 Configuring Trunking Guidelines and Limitations W Port WWN is 20 4f 00 0d ec 6d 2b 40 Peer port WWN is 20 0a 00 0d ec 3f ab 80 Admin port mode is auto trunk mode is on snmp link state traps are enabled Port mode is TE Port vsan is 1 Speed is 2 Gbps Rate mode is dedicated Transmit B2B Credit is 16 Receive B2B Credit is 250 B2B State Change Number is 14 Receive data field Size is 2112 Beacon is turned off Trunk vsans admin allowed and active Trunk vsans up Trunk vsans isolated Trunk vsans 1 100 101 1101 1163 1166 1216 2172 2182 2183 1 1101 1163 1166 1216 2172 2182 2183 100 101 initializing In case of TF ports after the handshake one of the allowed VSANs will be moved to the up state All other VSANs will be in initializing state even though the handshake with the peer is completed and successful Each VSAN will be moved from initializing state to up state when a server or target logs in through the trunked F or NP ports in the corresponding VSAN Note In case of TF or TNP ports the Device Manager will show the port status as amber even after port is up and there is no failure It will be changed to green once all the VSAN has successful logins This example shows a TF port informa
154. e ii Chapter2 Configuring Interfaces W Configuring Interfaces Step 1 Step 2 Step 3 Step 4 Step5 Step 6 Step7 To configure the mgmt0 Ethernet interface to connect over IPv6 follow these steps Command Purpose switch config terminal switch config Enters configuration mode switch config interface mgmt0 switch config if Selects the management Ethernet interface on the switch and enters interface configuration submode switch config if ipv6 enable Enables IPv6 and assigns a link local address on the interface switch config if address 2001 0db8 ipv6 address ipv6 800 200c 417a 64 switch config if no shutdown Specifies an IPv6 unicast address and prefix length on the interface Enables the interface switch con end switch fig if Returns to EXEC mode switch copy running config startup config Optional Saves your configuration changes to the file system Note If you want to save your configuration you can enter this command at any time Creating VSAN Interfaces Detailed Steps Step 1 Step 2 Step 3 To create a VSAN interface follow these steps Command Purpose switch config t Enters configuration mode switch config interface vsan 2 switch config if Configures a VSAN with the ID 2 switch config if no shutdown Enables the VSAN interface Confi
155. e interface list value is not included in the command virtual pWWN is assigned globally e All the interfaces mentioned in the interface list value must be in a shut state Manually Assigning FlexAttach Virtual pWWN You can manually assign a WWN to the interface without generating it through the switch Several checks are done by the NPV core to ensure the uniqueness of virtual pWWNs in the switch When duplicate virtual pWWNs are configured the subsequent logins are rejected by the NPV core switch Prerequisites e Some ports may be in automode some in manual mode and the virtual pWWNs need not be assigned e The port must be in a shut state when a virtual pWWN is assigned Restrictions e The interface mentioned in the interface value must be in a shut state Detailed Steps To assign virtual pWWN manually perform this task Command Purpose Step1 switch config t Enters configuration mode Step2 switch config flex attach virtual pwwn Configures the FlexAttach virtual pWWN for vpwwn interface interface the interface switch config lex attach virtual pwwn Configures the FlexAttach virtual pWWN for vpwwn interface isnterface Nsan vsan the interface in the VSAN Step3 9 switch config flex attach commit Commits the configuration Mapping pWWN to Virtual pWWN You can configure virtual pWWNs through real pWWNs This process is required for NPIV hosts containing multiple pWWNs of which only F
156. e is F snmp traps are enabled Port mode is F FCID is 0xeb0002 Port vsan is 1 Speed is 2 Gbps Rate mode is shared Transmit B2B Credit is 64 Receive B2B Credit is 16 Receive data field Size is 2112 Beacon is turned off 5 minutes input rate 0 bits sec 0 bytes sec 0 frames sec 5 minutes output rate 0 bits sec 0 bytes sec 0 frames sec 226 frames input 18276 bytes 0 discards 0 errors 0 CRC 0 unknown class 0 too long 0 too short 326 frames output 21364 bytes 0 discards 0 errors 0 input OLS 0 LRR 1 NOS 0 loop inits 3 output OLS 2 LRR 0 NOS 0 loop inits 16 receive B2B credit remaining 64 transmit B2B credit remaining Configuring Rate Mode Restrictions e Changing port speed and rate mode disrupts traffic on the port Detailed Steps To configure the rate mode dedicated or shared on an interface on a 48 port or 24 port 4 Gbps or any 8 Gbps Fibre Channel switching module follow these steps Command Purpose Step1 switch config t Enters configuration mode switch config Step 2 switch config interface fc 1 1 Selects the interface and enters interface switch config if configuration submode Cisco MDS 9000 Family NX OS Interfaces Configuration Guide M330 By OL 29284 01 Release 6 x Chapter3 Configuring Fibre Channel Interfaces Command Configuring Fibre Channel Interfaces Purpose Step 3 switch config if switchport rate mode dedicated Reserves dedicated bandwi
157. e mode e BB_credit buffers for Fx port mode connections can be configured The minimum is 2 buffers and the maximum of 250 buffers for dedicated rate mode or 16 buffers for shared rate mode e Performance buffers are not supported on this module Each port group on the 48 port 4 Gbps Fibre Channel switching module consists of 12 ports The ports in shared rate mode have bandwidth oversubscription of 2 1 by default However some configurations of the shared ports in a port group can have maximum bandwidth oversubscription of 4 1 considering that each port group has 12 8 Gbps bandwidth The following example configurations are supported by the 48 port 4 Gbps Fibre Channel switching modules e Twelve ports with shared rate mode and 4 Gbps speed 4 1 oversubscription default e One port with dedicated rate mode and 4 Gbps speed plus 11 ports with shared rate mode and 4 Gbps speed 5 1 oversubscription e One port with dedicated rate mode and 4 Gbps speed plus 11 ports with shared rate mode and 2 Gbps speed 2 5 1 oversubscription e Two ports with dedicated rate mode and 2 Gbps speed plus 10 ports with shared rate mode and 4 Gbps speed 5 1 oversubscription e Two ports with dedicated rate mode and 2 Gbps speed plus 10 ports with shared rate mode and 2 Gbps speed 2 5 1 oversubscription e Twelve ports with dedicated rate mode and 1 Gbps speed e Three ports with dedicated rate mode and 4 Gbps speed plus four ports with shared rate
158. e show port resources module command to check for resource availability When configuring PortChannels on switches with Generation 1 Generation 2 and Generation 3 switching modules follow one of these procedures e Configure the PortChannel and then configure the Generation 2 and Generation 3 interfaces to auto with a maximum of 2 Gbps e Configure the Generation 1 switching modules followed by the Generation 2 switching modules and then the Generation 3 switching modules and then configure the PortChannel When configuring PortChannels on switches with only Generation 2 and Generation 3 switching modules follow one of these procedures e Configure the PortChannel and then configure the Generation 3 interfaces to auto with a maximum of 4 Gbps e Configure the Generation 2 switching modules followed by the Generation 3 switching modules and then configure the PortChannel Table 3 11 describes the results of adding a member to a PortChannel for various configurations Cisco MDS 9000 Family NX OS Interfaces Configuration Guide 322 OL 29284 01 Release 6 x Chapter 3 Configuring Fibre Channel Interfaces Guidelines and Limitations Table 3 11 PortChannel Configuration and Addition Results PortChannel Configured Speed New Member Members PortChannel New Member Type Addition Type Result No members Any Any Generation or Force Pass Generation 2 or Gener
159. e to a PortChannel Detailed Steps To add an interface to a PortChannel follow these steps Command Purpose Step1 switch config t Enters configuration mode switch config Step2 switch config interface fc1 15 Configures the specified port interface fc1 15 switch config if Step3 switch config if channel group 15 Adds physical Fibre Channel port 1 15 to channel group 15 If channel group 15 does not exist it is created The port is shut down To add a range of ports to a PortChannel follow these steps Command Purpose Step 1 switch config t Enters configuration mode switch config Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x _ Chapter 6 Configuring PortChannels Step 2 Step 3 wy Command Configuring PortChannels W Purpose switch config interface fc1 1 5 switch config if Configures the specified range of interfaces In this example interfaces from 1 1 to 1 5 are configured switch config if channel group 2 Adds physical interfaces 1 1 1 2 1 3 1 4 and 1 5 to channel group 2 If channel group 2 does not exist it is created If the compatibility check is successful the interfaces are operational and the corresponding states apply to these interfaces Note By default the CLI adds a interface normally to a PortChannel while DCNM SAN adds the interface by force
160. edit buffers 2 1 Default BB_credit buffers 64 16 16 1 ISL E port or TE port Cisco MDS 9222 Multiservice Modular Switch BB_ Credit Buffers Table 4 13 lists the BB_credit buffer allocation for 18 port 4 Gbps Multiservice Modular switches Table 4 13 18 Port 4 Gbps Fabric Switch BB_Credit Buffer Allocation Defaults BB_Credit Buffers Per BB_Credit Buffers Per Port BB_Credit Buffer Allocation Type Port Group Defaults ISL Fx Port User configurable BB_credit buffers 4509 250 16 1 ISL E port or TE port Extended BB_ Credits amp To facilitate BB_credits for long haul links the extended BB_credits feature allows you to configure the receive buffers above the maximum value on all Generation 2 Generation 3 and Generation 4 switching modules When necessary you can reduce the buffers on one port and assign them to another port exceeding the default maximum The minimum extended BB_credits per port is 256 and the maximum is 4095 Note amp Extended BB_credits are not supported on the Cisco MDS 9148 Fabric Switch Cisco MDS 9134 Fabric Switch Cisco MDS 9124 Fabric Switch the Cisco Fabric Switch for HP c Class BladeSystem and the Cisco Fabric Switch for IBM BladeCenter In general you can configure any port in a port group to dedicated rate mode To do this you must first release the buffers from the other ports before configuring larger extended BB_credits for a port Note The ENTERP
161. ee also switching modules A ACL adjacency sharing disabling for downgrading 3 43 administrative speeds configuring 2 10 2 25 administrative states description 2 6 setting 2 22 ALPA caches allocation 2 50 clearing 2 29 description 2 14 displaying contents 2 51 inserting entries manually 2 29 auto mode configuring 2 22 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide W index auto port mode description 2 5 interface configuration 2 2 autosensing speed Generation 2 switching modules 2 10 bandwidth fairness disabling 3 41 enabling 3 40 BB credit buffers 12 port 4 Gbps switching module allocations 4 13 12 port 4 Gbps switching module considerations 4 14 24 port 4 Gbps switching module allocations 4 12 24 port 4 Gbps switching module considerations 4 12 4 13 24 port 8 Gbps switching module considerations 4 8 4 44 port 8 Gbps switching module considerations 4 9 48 port 4 Gbps switching module considerations 4 10 48 port 8 Gbps switching module considerations 4 7 4 port 10 Gbps switching module allocations 4 15 4 port 10 Gbps switching module considerations 4 15 4 16 allocation defaults table 4 6 4 7 4 8 4 9 4 10 BB credits configuring 4 21 description 4 1 reason codes 2 8 BB_SC enabling 4 20 4 24 beacon modes configuring 2 27 identifying LEDs 2 11 bit errors reasons 2 12 bit error thresholds configuring 2 11 2
162. elease Upgrade and Downgrade Limitations The trunking and channeling feature includes the following upgrade and downgrade limitations When F port trunking or channeling is configured on a link the switch cannot be downgraded to Cisco MDS SAN OS Release 3 x and NX OS Release 4 1 1b or earlier If you are upgrading from a SAN OS Release 3 x to NX OS Release 5 0 1 and you have not created VSAN 4079 the NX OS software will automatically create VSAN 4079 and reserve it for EVFP use If VSAN 4079 is reserved for EVFP use the switchport trunk allowed vsan command will filter out VSAN 4079 from the allowed list as shown in the following example switch config if switchport trunk allowed vsan 1 4080 1 4078 4080 switch config if If you have created VSAN 4079 the upgrade to NX OS Release 5 0 1 will have no affect on VSAN 4079 If you downgrade after NX OS Release 5 0 1 the VSAN will no longer be reserved for EVFP use Difference Between TE Ports and TF TNP Ports In case of TE ports the VSAN will in be initializing state when VSAN is coming up on that interface and when peers are in negotiating phase Once the handshake is done VSAN will be moved to up state in the successful case and isolated state in the case of failure Device Manager will show the port status as amber during initializing state and it will be green once VSANSs are up This example shows the trunk VSAN states of a TE port Switch show interfac
163. elease 6 x _ Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces W Table 3 2 Bandwidth and Port Groups for the Fibre Channel Modules and Fabric Switches continued Bandwidth Per Maximum Product Name Number of Ports Port Group Bandwidth Per Port Part Number Description Per Port Group Gbps Gbps DS C9134 K9 32 port 4 Gbps 4 16 4 Gbps Cisco MDS 9134 Fannie een Fabric switch 2 port 10 Gbps 1 10 10 Gbps Fabric switch DS C9124K9 Cisco 24 port 4 Gbps 4 16 4 Gbps MDS 9124 Fabric Fabric switch switch DS C9222i K9 18 port 4 Gbps 4 6 12 8 4 Gbps Cisco MDS 92221 Multiservice Modular switch Gigabit Ethernet ports and a modular expansion slot 1 This bandwidth is available with the Fabric 3 module DS 13SLT FAB3 in the MDS 9513 switch 2 This bandwidth is available with the Fabric 2 module DS 13SLT FAB2 in the MDS 9513 switch and with the Supervisor 2 DS X9530 SF2 K9 or Supervisor 2A module DS X9530 SF2AK9 in the MDS 9509 switch or MDS 9506 switch 3 A maximum of four ports one per port group in a 4 44 port 8 Gbps switching module can operate at 8 Gbps bandwidth in dedicated or shared mode All the other ports can operate at a maximum of 4 Gbps in shared mode or dedicated mode Port Rate Modes In Generation 2 Generation 3 and Generation 4 modules you can configure the port rate modes The port rate mode configuration is used to determine
164. eneration 3 Fibre Channel switching modules 3 9 port indexes description 3 19 port modes auto 2 5 description 2 2 to 2 6 port rate modes configuring 3 30 dedicated 3 7 description 3 5 oversubscribed 3 8 shared 3 8 See also rate modes I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide W index port speed group configuring 2 26 port speeds configuring 2 10 2 25 configuring on Generation 2 switching module interfaces 3 29 displaying configuration 3 30 port tracking default settings 9 2 description 9 1 displaying information 9 6 enabling 9 3 guidelines 9 2 monitoring ports in a VSAN 9 5 multiple ports 9 4 shutting down ports forcefully 9 6 private devices TL ports 2 50 rate modes configuring on Generation 2 switching module interfaces 3 30 verifying configuration 3 31 See also port rate modes reason codes description 2 6 receive buffer groups See buffer groups receive data field sizes configuring 4 21 recovery from powered down state 3 21 S SD port mode description 2 5 interface modes 2 5 SD ports configuring 2 22 SFPs displaying transmitter types 2 48 transmitter types 2 12 shared rate mode description 3 8 migrating from dedicated rate mode 3 27 migrating to dedicated rate mode 3 26 oversubscription 3 11 source IDs exchange based 6 5 flow based 6 4 SPAN destination port mode See SD port mode SPAN tunnel port mode See ST port
165. ep 1 switch config t Enters configuration mode Step2 switch config interface fc1 1 Configures the specified interface switch config if Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter5 Configuring Trunking Step 3 Verifying Trunking Configuration W Command Purpose switch config if switchport trunk allowed vsan 2 4 Changes the allowed list for the specified VSANs switch config if switchport trunk allowed vsan add 5 Expands the specified VSAN 5 to updated trunking membership the new allowed list switch config if no switchport trunk allowed vsan 2 4 Deletes VSANSs 2 3 and 4 switch config if no switchport trunk allowed vsan add 5 Deletes the expanded allowed list Verifying Trunking Configuration To display trunking configuration information perform one of the following tasks Command Purpose show interface fc slot port Displays the interface configuration information that includes trunking trunk mode allowed VSANSs and status show trunk protocol Displays whether the trunk protocol is enabled show interface trunk vsan numbers Displays whether the interface is trunking and the allowed VSAN list for each trunking interface For detailed information about the fields in the output from these commands refer to the Cisco MDS NX OS Command Reference The show interface command i
166. eration 2 Fibre Channel switching modules are not supported on the Cisco MDS 9216 switch however they are supported by both the Supervisor 1 module and the Supervisor 2 module For detailed information about the installation and specifications for these modules and switches refer to the hardware installation guide for your switch Each module or switch can have one or more ports in port groups that share common resources such as bandwidth and buffer credits Port groups are defined by the hardware consisting of sequential ports For example ports 1 through 6 ports 7 through 12 ports 13 through 18 ports 19 through 24 ports 25 through 30 31 through 36 and ports 37 through 42 43 through 48 are the port groups on the 48 port 8 Gbps Advanced Fibre Channel switching modules Table 3 2 shows the bandwidth and number of ports per port group for the Generation 2 Generation 3 and Generation 4 Fibre Channel modules and Generation 2 and Generation 3 Fabric switches Table 3 2 Bandwidth and Port Groups for the Fibre Channel Modules and Fabric Switches Bandwidth Per Maximum Product Name Number of Ports Port Group Bandwidth Per Port Part Number Description Per Port Group Gbps Gbps Generation 4 Modules DS X9248 256K9 48 port 8 Gbps 6 32 4 or 12 8 8 or 10 Advanced Fibre Gbps depending Channel switching on the module configuration I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuratio
167. erface functions as a fabric port This port may be connected to a peripheral device host or disk operating as an N port An F port can be attached to only one N port F ports support class 2 and class 3 service In fabric loop port FL port mode an interface functions as a fabric loop port This port may be connected to one or more NL ports including FL ports in other switches to form a public arbitrated loop If more than one FL port is detected on the arbitrated loop during initialization only one FL port becomes operational and the other FL ports enter nonparticipating mode FL ports support class 2 and class 3 service Note NP Ports FL port mode is not supported on 4 port 10 Gbps switching module interfaces An NP port is a port on a device that is in NPV mode and connected to the core switch via an F port NP ports function like N ports except that in addition to providing N port operations they also function as proxies for multiple physical N ports For more details about NP ports and NPV see Chapter 7 Configuring N Port Virtualization OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces HZ information About Interfaces TL Port TE Port TF Port In translative loop port TL port mode an interface functions as a translative loop port It may be connected to one or more private loop devices NL ports TL ports are specific
168. es so Oe TT ee aaar aar nar 77686 i i l ha 0000 0000 1 Status LED 3 Link LEDs and speed LEDs 2 1 2 Gbps Fibre Channel port group 4 Asset tag See the Cisco MDS 9000 Family NX OS Fundamentals Configuration Guide See the Speed LEDs section on page 2 11 See the Generation 1 Interface Configuration Guidelines section on page 2 18 we PS Refer to the Cisco MDS 9000 Family hardware installation guide for your platform Speed LEDs Each port has one link LED on the left and one speed LED on the right The speed LED displays the speed of the port interface e Off tThe interface attached to that port is functioning at 1000 Mbps On solid green The interface attached to that port is functioning at 2000 Mbps for 2 Gbps interfaces The speed LED also displays if the beacon mode is enabled or disabled e Off or solid green Beacon mode is disabled e Flashing green The beacon mode is enabled The LED flashes at one second intervals Note Generation 2 Generation 3 and Generation 4 modules and fabric switches do not have speed LEDs Bit Error Thresholds The bit error rate threshold is used by the switch to detect an increased error rate before performance degradation seriously affects traffic Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter2 Configuring Interfaces H
169. es 16 host optimized ports in the Cisco MDS 9120 switch and 32 host optimized ports in the Cisco MDS 9140 switch e The default port mode is Fx Fx negotiates to F or FL for 32 port switching modules e The 32 port switching module does not support FICON Note We recommend that you configure your E ports on a 16 port switching module If you must configure an E port on a 32 port host optimized switching module the other three ports in that 4 port group cannot be used amp Note In the Cisco MDS 9100 Series the groups of ports that are located on the left and outlined in white are full line rate The other ports are host optimized Each group of 4 host optimized ports have the same features as for the 32 port switching module Private Loop Configuration Guidelines Follow these guidelines when configuring private loops e A maximum of 64 fabric devices can be proxy to a private loop e Fabric devices must be in the same zone as private loop devices to be proxy to the private loop e Each private device on a TL port may be included in a different zone e All devices on the loop are treated as private loops You cannot mix private and public devices on the loop if the configured port mode is TL e The only FC4 type supported by TL ports is SCSI FCP e Communication between a private initiator to a private target on the same private loop does not invoke TL port services VSAN Interface Configuration Guidelines Follow
170. es 4 19 Buffer to Buffer Credit Recovery 4 20 Buffer to Buffer State Change Number 4 20 Receive Data Field Size 4 21 Configuring Interface Buffers 4 21 Configuring Buffer to Buffer Credits 4 21 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x vi E Contents CHAPTER 5 CHAPTER 6 Configuring Performance Buffers 4 22 Configuring Extended BB_credits 4 23 Enabling Buffer to Buffer Credit Recovery 4 24 Enabling the Buffer to Buffer State Change Number Configuring Receive Data Field Size 4 24 Verifying BB_Credit Configuration 4 25 Configuring Trunking 5 1 Information About Trunking 5 1 Trunking E Ports 5 1 Trunking F Ports 5 2 Key Concepts 5 3 Trunking Protocols 5 3 Trunk Modes 5 4 Trunk Allowed VSAN Lists and VF_IDs 5 5 Guidelines and Limitations 5 7 General Guidelines and Limitations 5 7 Upgrade and Downgrade Limitations 5 8 Difference Between TE Ports and TF TNP Ports 5 8 Trunking Misconfiguration Examples 5 10 Default Settings 5 11 Configuring Trunking 5 11 Enabling the Cisco Trunking and Channeling Protocols Enabling the F Port Trunking and Channeling Protocol Configuring Trunk Mode 5 12 Configuring an Allowed Active List of VSANs 5 12 Verifying Trunking Configuration 5 13 Configuration Example for F Port Trunking 5 14 Configuring PortChannels_ 6 1 Information About PortChannels 6 1 PortChannels Overview 6 2 EPortChannels 6 2 Fand TF PortChannels 6 3 PortChanneling an
171. esent fc2 17 is trunking Transmit B2B Credit is 255 Receive B2B Credit is 12 Receive B2B Credit performance buffers is 375 12 receive B2B credit remaining 255 transmit B2B credit remaining fc2 18 is down SFP not present fc2 19 is down SFP not present f c2 20 is down SFP not present fc2 21 is down Link failure or not connected fc2 31 is up Transmit B2B Credit is 0 Receive B2B Credit is 12 Receive B2B Credit performance buffers is 48 12 receive B2B credit remaining 0 transmit B2B credit remaining fc2 32 is down Link failure or not connected Example 4 2 Displays BB_credit Information for a Specified Fibre Channel Interface switch show interface fc2 31 bbcredit c2 31 is up Cisco MDS 9000 Family NX OS Interfaces Configuration Guide I OL 29284 01 Release 6 x Chapter4 Configuring Interface Buffers Verifying BB_Credit Configuration Transmit B2B Credit is 0 Receive B2B Credit is 12 Receive B2B Credit performance buffers is 48 12 receive B2B credit remaining 0 transmit B2B credit remaining Cisco MDS 9000 Family NX OS Interfaces Configuration Guide M426 E OL 29284 01 Release 6 x CHAPTER 5 Configuring Trunking e Information About Trunking page 5 1 e Guidelines and Limitations page 5 7 e Default Settings page 5 11 e Configuring Trunking page 5 11 e Verifying Trunking Configuration page 5 13 e Configuration Example for F Port Trunking page 5 14 Information About Trunking Trunk
172. faces Configuration Guide P250 fy OL 29284 01 Release 6 x Chapter 2 Configuring Interfaces Verifying Interfaces Configuration W In addition to these proxied devices other virtual devices local or remote domain controller addresses are also allocated ALPAs on the loop A switch reserves the ALPA for its own communication with private devices and the switch acts as a SCSI initiator The first column in the output of the show tlport interface command is the ALPA identity of the device on the loop The columns that follow include the port WWNs the node WWNs for each device the device as a SCSI initiator or target and the real FC ID of the device Example 2 18 Displays the Detailed Information for a Specific TL Port switch show tlport interface fc1 16 all fcl 16 is up vsan 1 FCID 0x420000 alpa pWWN nWWN SCSI Type Device FC ID 0x01 20 10 00 05 30 00 4a de 20 00 00 05 30 00 4a de Initiator Proxied Oxfffc42 0x73 22 00 00 20 37 39 ae 54 20 00 00 20 37 39 ae 54 Target Private 0x420073 Oxef 20 10 00 05 30 00 4a de 20 00 00 05 30 00 4a de Initiator Switch 0x0000ef Example 2 19 Displays TL Port Information for Private Devices switch show tlport interface fc 1 16 private fcl 16 is up vsan 1 FCID 0x420000 0x73 22 00 00 20 37 39 ae 54 20 00 00 20 37 39 ae 54 Target 0x420073 0x74 22 00 00 20 37 38 d3 de 20 00 00 20 37 38 d3 de Target 0x420074 Example 2 20 Displays TL Port Information for Proxied Devices switch show t
173. fault PortChannels FSPF is enabled by default Create PortChannel Administratively up ON mode on non NPV and NPIV core switches ACTIVE mode on NPV switches Disabled Default PortChannel mode Autocreation E Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter6 Configuring PortChannels Configuring PortChannels This section includes the following topics Configuring PortChannels e Configuring PortChannels Using the WizardCreating a PortChannel page 6 17Configuring PortChannels Using the WizardCreating a PortChannel page 6 17 e Configuring the PortChannel Mode page 6 17 e Deleting PortChannels page 6 18 e Adding an Interface to a PortChannel page 6 18 e Forcing an Interface Addition page 6 19 e Deleting an Interface from a PortChannel page 6 19 e Enabling and Configuring Autocreation page 6 20 e Converting to Manually Configured Channel Groups page 6 20 Configuring PortChannels Using the WizardCreating a PortChannel Detailed Steps Step 1 Step 2 To create a PortChannel follow these steps Command Purpose switch config t switch config Enters configuration mode switch config interface port channel 1 switch config if Configures the specified PortChannel 1 using the default ON mode Configuring the PortChannel Mode Restrictions Detailed Steps Step 1 Step 2 By default the CLI
174. fault port group monitoring switch config no port group monitor enable Configuring a Port Group Monitor Policy Detailed Steps Step 1 Step 2 Step 3 Step 4 Disables port group monitoring To configure port group monitor policy follow these steps Command switch config t Purpose Enters configuration mode switch config port group monitor name policyname Specifies the policy name and enters the port group monitoring policy configuration mode switch config no port group monitor name policyname Removes the policy switch config port group monitor counter rx performance poll interval seconds delta rising threshold percentagel falling threshold percentage Specifies the delta Rx counter poll interval in seconds and thresholds in percentage switch config port group monitor counter tx performance poll interval seconds delta rising threshold percentagel falling threshold percentage Specifies the delta Tx counter poll interval in seconds and thresholds in percentage switch config port group monitor no counter tx performance Reverts to the default policy switch config port group monitor monitor counter rx performance Turns on Rx performance monitoring switch config port group monitor monitor counter tx performance Turns on Tx performance monitoring switch config port group monitor no monitor
175. fc5 16 32 Port Group 5 Total bandwidth is 32 4 Gbps Total shared bandwidth is 16 4 Gbps Allocated dedicated bandwidth is 16 0 Gbps Interfaces in the Port Group B2B Credit Buffers fc5 17 32 fc5 18 32 fc5 19 500 fc5 20 500 Port Group 6 Total bandwidth is 32 4 Gbps Total shared bandwidth is 16 4 Gbps Allocated dedicated bandwidth is 16 0 Gbps Interfaces in the Port Group B2B Credit Buffers fc5 21 500 fc5 22 500 fc5 23 32 fc5 24 32 Port Group 7 Total bandwidth is 32 4 Gbps Total shared bandwidth is 32 4 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Buffers c5 25 32 c5 26 32 c5 27 32 c5 28 32 Port Group 8 Total bandwidth is 32 4 Gbps Total shared bandwidth is 32 4 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Buffers fc5 29 32 fc5 30 32 Gbps Gbps Gbps Gbps Gbps shared shared shared shared shared shared shared dedicated dedicated dedicated dedicated shared shared shared shared shared shared E Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter3 Configuring Fibre Channel Interfaces c5 31 c5 32 Tsola 13 show port resources Module 13 Available dedicated buffers Available dedicated buffers Available dedicated buffers Available dedicated buffers Available dedicated buffers Available dedicated
176. fers Gbps fc2 1 16 4 0 shared fc2 2 16 4 0 shared fc2 3 16 4 0 dedicated fc2 4 16 4 0 shared fc2 5 16 4 0 shared fc2 6 16 4 0 dedicated fc2 7 16 4 0 dedicated fc2 8 16 4 0 shared fc2 9 16 4 0 shared fc2 10 16 4 0 shared fc2 11 16 4 0 shared fc2 12 16 4 0 shared Port Group 4 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Bandwidth Rate Mode Buffers Gbps fc2 37 16 4 0 shared fc2 38 16 4 0 shared fc2 39 16 4 0 dedicated fc2 40 16 4 0 dedicated fc2 41 16 4 0 dedicated fc2 42 16 4 0 shared fc2 43 16 4 0 shared fc2 44 16 4 0 shared fc2 45 16 4 0 shared fc2 46 16 4 0 shared fc2 47 16 4 0 shared fc2 48 16 4 0 shared Step2 Shut down all shared ports for which you want to remove restrictions on oversubscription ratios switch config interface fc2 1 2 c2 4 5 fc2 8 38 c2 43 48 switch config if shutdown Step3 Display the interface status to confirm the shutdown of all shared ports switch config if end switch show interface brief Interface Vsan Admin Admin Status SFP Oper Oper Port Mode Trunk Mode Speed Channel Mode Gbps fc2 1 1 FX down swl c2 2 1 FX down swl f c2 3 1 T up swl c2 4 1 FX down swl c2 5 1 FX down swl fc2 6 1 TE up swl Cisco MDS 9000 Family NX OS Interfaces Configuration Guide P3338 OL 29284 01 Release 6 x
177. fig if switchport rate mode shared tch config if switchport mode f Enable the interfaces and return to configuration mode swi swi swi tch config if no shutdown tch config if exit tch Configuration Example for 4 44 Port 8 Gbps Module Interfaces Step 1 Step 2 Step 3 Step 4 Step 5 These steps describe how to configure the 4 44 port 8 Gbps module interfaces Select interfaces fc 4 11 through fc 4 12 swi swi tch config t tch config interface fc 4 11 12 Disable the interfaces and take them out of service swi swi tch config if shutdown tch config if out of service Return to configuration mode swi swi tch config if exit tch Select the interfaces fc 4 1 swi swi tch config t tch config interface fc 4 1 Configure the port speed rate mode and port mode on the interfaces swi tch config if switchport speed 8000 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces WE Configuration Examples for Fibre Channel Interfaces switch config if switchport rate mode shared switch config if switchport mode f Step6 Enable the interfaces and return to configuration mode switch config if no shutdown switch config if exit switch Step7 Select the interfaces fc 4 2 through fc 4 10 switch config t switch config interface fc 4 2 10 Step8
178. figuring extended BB_credits 4 18 E Cisco MDS 9000 Family NX OS Interfac OL 29284 01 Release 6 x NL ports interface modes 2 6 nonparticipating codes description 2 9 NPIV description 7 2 enabling 7 10 NP links 7 5 N port identifier virtualization See NPIV NL ports See also Nx ports NP ports 7 5 NPV configuring 7 10 NPV mode 7 4 O operational states configuring on Fibre Channel interfaces 2 22 description 2 6 out of service interfaces description 3 11 oversubscription disabling restrictions 3 37 enabling restrictions 3 39 Generation 2 switching modules 3 11 ratios 3 12 P performance buffers configuring 4 22 description 4 2 PortChannel modes description 6 6 PortChannel Protocol autocreation 6 11 Index W configuring autocreation 6 20 converting autocreated groups to manually configured 6 12 creating channel group 6 10 description 6 9 enabling autocreation 6 20 PortChannels adding interfaces 6 8 6 18 administratively down 2 9 comparison with trunking 6 3 compatibility checks 6 8 creating 6 17 default settings 6 16 deleting 6 7 deleting interfaces 6 9 examples 6 2 forcing interface additions 6 9 6 19 Generation 2 switching module interfaces 3 21 interface states 6 9 load balancing 6 4 misconfiguration error detection 6 13 verifying configurations 6 21 to 6 24 port groups assigning extended BB_credits 4 18 description 3 3 Generation 2 Fibre Channel switching modules 3 3 G
179. flash or a TFTP server before NPV if the configuration is required for later use Use the following commands to save either your non NPV or NPV configuration Cisco MDS 9000 Family NX OS Interfaces Configuration Guide zu OL 29284 01 Release 6 x _ Chapter 7 Configuring N Port Virtualization Detailed Steps Step 1 Step 2 Step 3 Step 4 Step5 Step 6 Step7 Step 8 Step 9 switch copy running bootflash filename Configuring N Port Virtualization The configuration can be reapplied later using the following command switch copy bootflash filename running config To configure NPV using the CLI perform the following tasks Command Purpose swi CC h config t On the NPV core switch enters configuration switch config mode switch config feature npiv Enables NPIV mode on the NPV core switch switch config switch config no feature npiv Disables NPIV mode on the NPV core switch switch config interface fe 2 1 Configures the NPIV core switch port as an F switch config if switchport mode F ort port switch config if no shutdown Changes Admin status to bring up the interfaces switch config vsan database Configures the port VSANSs for the F port on the switch config vsan db vsan 8 interface fc NPIV core switch 2 1 switch config vsan db vsan 10 interface fc 2 1 switch config npv enable Enables NPV mode on a NPV
180. format interface fc1 1 5 fc2 5 7 Note The spaces are required before and after the dash and before and after the comma Example 2 4 Displays Multiple Specified Interfaces switch show interface fc3 13 c3 16 fc3 13 is up Hardware is Fibre Channel SFP is short wave laser Port WWN is 20 8d 00 05 30 00 97 9e Admin port mode is FX Port mode is F FCID is 0x7b0300 Port vsan is 1 Speed is 2 Gbps Transmit B2B Credit is 3 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces W Verifying Interfaces Configuration Receive B2B Credit is 12 Receive data field Size is 2112 Beacon is turned off 5 minutes input rate 0 bits sec 0 bytes sec 0 frames sec 5 minutes output rate 0 bits sec 0 bytes sec 0 frames sec 1856 frames input 116632 bytes 0 discards 0 errors 0 CRC 0 unknown class 0 too long 0 too short 1886 frames output 887712 bytes 0 discards 0 errors 0 input OLS 0 LRR 0 NOS 1 loop inits 1 output OLS 1 LRR 0 NOS 1 loop inits 16 receive B2B credit remaining 3 transmit B2B credit remaining fc3 16 is up Hardware is Fibre Channel SFP is short wave laser Port WWN is 20 90 00 05 30 00 97 9e Admin port mode is FX Port mode is F FCID is 0x7d0100 Port vsan is 3000 Speed is 2 Gbps Transmit B2B Credit is 3 Receive B2B Credit is 12 Receive data field Size is 2112 Beacon is turned off 5 minutes input rate 504 bits sec
181. g 2 16 Slow Drain Device Detection and Congestion Avoidance Management Interfaces 2 17 VSAN Interfaces 2 18 Prerequisites for Interfaces 2 18 Guidelines and Limitations 2 18 Generation 1 Interface Configuration Guidelines 2 18 Private Loop Configuration Guidelines 2 19 VSAN Interface Configuration Guidelines 2 19 Default Settings 2 20 Configuring Interfaces 2 20 Configuring Fibre Channel Interfaces 2 21 Setting the Interface Administrative State 2 22 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Configuring Interface Modes 2 22 Configuring System Default Port Mode F 2 23 Configuring ISL between Two Switches 2 24 Configuring 10 Gbps FC Mode 2 24 Configuring Port Administrative Speeds 2 25 Configuring Port Speed Group 2 26 Configuring the Interface Description 2 26 Specifying a Port Owner 2 27 Configuring Beacon Mode 2 27 Disabling Bit Error Threshold 2 28 Configuring Switch Port Attribute Default Values 2 28 Configuring TL Ports 2 29 Manually Inserting Entries into the ALPA Cache 2 29 Clearing the ALPA Cache 2 29 Configuring Port Guard 2 30 Configuring Port Monitor 2 31 Enabling Port Monitor 2 31 Configuring a Port Monitor Policy 2 32 Activating a Port Monitor Policy 2 34 Configuring a Port Monitor Port Guard 2 34 Configuring Port Group Monitor 2 34 Enabling Port Group Monitor 2 35 Configuring a Port Group Monitor Policy 2 35 Reverting to the Default Policy for a Specif
182. g Interfaces HZ Configuring Interfaces Configuring Port Guard Detailed Steps Examples Step 1 Step 2 Step 3 Step 4 To enable or disable the port guard for a port follow these steps Command Purpose switch config t Enters configuration mode switch config interface fc1 1 Selects the port interface switch config errdisable detect cause Brings the port to down state if the link flaps link down once switch config errdisable detect cause Enables the port guard configuration for the link down num times zumber duration interface Brings the port to down state if the link seconds flaps for the number of instances within the specified seconds switch config no errdisable detect cause Removes default the port guard configuration tions for the interface The link resumes flapping and sending error reports normally switch config errdisable detect cause Brings the port to down state if the specified error trustsec violation bit errors occurs even once credit loss link reset signal loss sync loss switch config errdisable detect cause Brings the port to down state if the specified error trustsec violation bit errors occurs for the number of instances within the credit loss link reset signal loss specified seconde sync loss num times zumber duration P seconds switch config no errdisable detect cause Removes default the port guard configuration t
183. g if no shutdown Returns all shared ports to service switch config exit Exits configuration mode switch copy running config startup config Saves the new oversubscription ratio configuration to the startup configuration and then the new configuration is enforced upon subsequent reboots of the module Enabling Bandwidth Fairness Detailed Steps Step 1 Step 2 Step 3 To enable bandwidth fairness on a switching module follow these steps Command Purpose switch config t switch config Enters configuration mode switch config rate mode bandwidth fairness module 1 Enables bandwidth fairness for a module Note You must enter this command separately for each module for which you want to enable bandwidth fairness You must reload the module for the command to take effect switch config exit Exits configuration mode Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces W Disabling Bandwidth Fairness Restrictions e If you disable bandwidth fairness up to a 20 percent increase in internal bandwidth allocation is possible for each port group however bandwidth fairness is not guaranteed when there is a mix of shared and full rate ports in the same port group Detailed Steps To disable bandwidth fairness on a switching module fo
184. g ports and display class 2 3 and F information as well Example 2 9 Displays Interface Counters in Brief Format switch show interface counters brief Interface Input rate is 5 min avg Output rate is 5 min avg OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces W Verifying Interfaces Configuration Mbits s Frames Mbits s Frames fc3 1 0 3871 0 3874 o3 2 0 3902 0 4232 c3 3 0 3901 0 4138 c3 4 0 3895 0 3894 f c3 5 0 3890 0 3897 c9 8 0 0 0 0 c9 9 0 5 0 4 c9 10 0 4186 0 4182 c9 11 0 4331 0 4315 Interface Input rate is 5 min avg Output rate is 5 min avg Rate Total Rate Total Mbits s Frames Mbits s Frames port channel 1 0 0 0 0 port channel 2 0 3946 0 3946 Note The show interface transceiver command can only be issued on a switch in the Cisco MDS 9100 Series if the SFP is present see Example 2 10 Example 2 10 Displays Transceiver Information switch show interface transceiver fc1 1 SFP is present name is CISCO AGILENT part number is QFBR 5796L revision is serial number is A00162193 fc transmitter type is short wave laser cisco extended id is unknown 0x0 fcl 9 SFP is present name is FINISAR CORP part number is FTRJ 1319 7D CSC revision is serial number is H11A6ER fc transmitter type is long wave laser cost reduced cisco extended id is unknown 0x0 Example 2 11 displays the entire running
185. guration Guide Pei OL 29284 01 Release 6 x Chapter6 Configuring PortChannels Guidelines and Limitations W Note On switches with Generation 1 switching modules or a combination of Generation and Generation 2 switching modules you can configure a maximum of 128 PortChannels On switches with only Generation 2 switching modules or Generation 2 and Generation 3 switching modules you can configure a maximum of 256 PortChannels If you misconfigure PortChannels you may receive a misconfiguration message If you receive this message the PortChannel s physical links are disabled because an error has been detected A PortChannel error is detected if the following requirements are not met e Each switch on either side of a PortChannel must be connected to the same number of interfaces e Each interface must be connected to a corresponding interface on the other side see Figure 6 8 for an example of an invalid configuration e Links in a PortChannel cannot be changed after the PortChannel is configured If you change the links after the PortChannel is configured be sure to reconnect the links to interfaces within the PortChannel and reenable the links If all three conditions are not met the faulty link is disabled Enter the show interface command for that interface to verify that the PortChannel is functioning as required Guidelines and Limitations This section includes the guidelines and limitations for this fea
186. guration mode Step2 switch config interface fc8 6 Configures the specified interface and enters the interface switch config if configuration submode You can now configure tracked ports Note This link symbolizes the direct link 1 in Figure 9 1 Step 3 switch config if port track Tracks interface fc8 6 with interface port channel 1 When interface port channel 1 port channel 1 goes down interface fc8 6 is also brought down Note This link symbolizes the ISL 2 in Figure 9 1 switch config if no port track Removes the port tracking configuration that is currently tntertace port channel 1 applied to interface fc8 6 Information About Tracking Multiple Ports You can control the operational state of the linked port based on the operational states of multiple tracked ports When more than one tracked port is associated with a linked port the operational state of the linked port will be set to down only if all the associated tracked ports are down Even if one tracked port is up the linked port will stay up In Figure 9 2 only if both ISLs 2 and 3 fail will the direct link 1 be brought down Direct link 1 will not be brought down if either 2 or 3 are still functioning as desired Figure 9 2 Traffic Recovery Using Port Tracking Port Channel 2 p FCIP Cisco MDS 9000 Family NX OS Quality of Service Configuration Guide OL 29284 01 Chapter 9 Configuring Port Tracking T
187. guring 2 21 configuring auto port mode 2 22 configuring beacon modes 2 27 configuring bit error thresholds 2 11 2 28 configuring descriptions 2 26 configuring frame encapsulation 2 10 configuring port modes 2 22 configuring port speed group 2 26 configuring receive data field sizes 4 21 Index W configuring speeds 2 10 2 25 default settings 2 20 3 25 5 11 6 16 deleting from PortChannels 6 9 disabling 2 22 displaying capabilities on Generation 2 switching modules 3 44 displaying information 2 42 to 2 49 enabling 2 22 graceful shutdown 2 9 modes 2 2 to 2 6 operational states 2 6 performance buffers 4 2 reason codes 2 6 states 2 6 taking out of service on Generation 2 switching modules 3 41 troubleshooting operational states 2 8 See also interfaces 2 6 FL port mode classes of service 2 3 description 2 3 FL ports configuring 2 22 description 2 3 nonparticipating code 2 9 See also Fx ports F port mode classes of service 2 3 description 2 3 F ports configuring 2 22 description 2 3 See also Fx ports frame encapsulation configuring 2 10 Fx ports 32 port default 2 19 configuring 2 22 description 2 5 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide W index interface modes 2 5 See also F ports FL ports 2 5 G Generation 1 switching modules 4 18 3 19 extended BB_credits port index allocations Generation 2 switching modules buffer gro
188. guring Slow Drain Device Detection and Congestion Avoidance This section includes the following topics e Configuring Congestion Frame Timeout Value page 2 39 e Configuring Stuck Frame Timeout Value page 2 39 e Configuring No Credit Timeout Value page 2 39 e Configuring Credit Loss Recovery Threshold and Action page 2 40 e Configuring Average Credit Nonavailable Duration Threshold and Action page 2 41 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Configuring Interfaces W Configuring Congestion Frame Timeout Value Step 1 Step 2 Step 3 Step 4 The default congestion frame timeout value is 500 milliseconds We recommend that you retain the default configuration for the ISLs and configure a value that does not exceed the default value for the edge ports If the frame is in the switch for a longer time than the configured congestion frame timeout it gets dropped which empties the buffer space in the ISL and alleviates the congestion To configure the congestion frame timeout value follow these steps Command Purpose switch config t Enters configuration mode switch config system default interface Configures a new congestion frame timeout congestion timeout milliseconds mode core value in milliseconds and the port mode for the edge device switch config system default interface Configures the default congestio
189. h is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps shared Gbps shared shared shared shared shared Interfaces in the Port Group B2B Credit Buffers c4 31 32 c4 32 32 c4 33 32 c4 34 32 c4 35 32 c4 36 32 Port Group 7 Total bandwidth is 12 8 Gbps shared I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces HI Configuring Fibre Channel Interfaces Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Bandwidth Rate Mode Buffers Gbps c4 37 32 8 0 shared c4 38 32 8 0 shared c4 39 32 8 0 shared c4 40 32 8 0 shared c4 41 32 8 0 shared Fc4 42 32 8 0 shared Port Group 8 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Bandwidth Rate Mode Buffers Gbps c4 43 32 8 0 shared c4 44 32 8 0 shared c4 45 32 8 0 shared c4 46 32 8 0 shared c4 47 32 8 0 shared c4 48 32 8 0 shared This example shows the port rate mode configuration for interfaces on a 4 44 port 8 Gbps module switch show port resources module 7 Module 7 Available dedicated buffers are 3888 Port Group 1 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps
190. h name Although f WWN based zoning is supported for NPV devices it is not recommended because e Zoning is not enforced at the NPV device rather it is enforced on the NPV core switch e Multiple devices behind an NPV device log in via the same F port on the core they use same fWWN and cannot be separated into different zones e The same device might log in using different fWWNs on the core switch depending on the NPV link it uses and may need to be zoned using different fWWNs Default Port Numbers Port numbers on NPV enabled switches will vary depending on the switch model For details about port numbers for NPV eligible switches see the Cisco NX OS Family Licensing Guide Cisco MDS 9000 Family NX OS Interfaces Configuration Guide re E OL 29284 01 Release 6 x Chapter7 Configuring N Port Virtualization Information About N Port Virtualization W NPV CFS Distribution over IP NPV devices use only IP as the transport medium CFS uses multicast forwarding for CFS distribution NPV devices do not have ISL connectivity and FC domain To use CFS over IP multicast forwarding has to be enabled on the Ethernet IP switches all along the network that physically connects the NPV switch You can also manually configure the static IP peers for CFS distribution over IP on NPV enabled switches For more information see the Cisco MDS 9000 Family NX OS System Management Configuration Guide NPV Traffic Management This sections
191. hannel modules I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces HZ Information About Fibre Channel Interfaces Port Speed Table 3 8 Dedicated Rate Mode Bandwidth Reservation for Generation 3 Fibre Channel Modules Maximum Dedicated Allowed Ports Product Name Bandwidth That Can Come Part Number Description per Port Up Ports in Shared Mode DS X9224 96K9 24 port 8 Gbps 8 Gbps 8 Ports All the remaining ports Fibre Channel 4 Gbps 24 Ports are 8 Gbps shared switching module DS X9248 96K9 48 port 8 Gbps 8 Gbps 8 Ports All the remaining ports Fibre Channel 4 Gbps 24 Ports are 8 Gbps shared switching module 2 Gbps 48 Ports DS X9248 48K9 4 44 port 8 Gbps 8 Gbps 4 Ports All the remaining ports Host Optimized 4 Gbps 12 Ports are 4 Gbps shared 8 Gbps Fibre Channel of bandwidth can be switching module 2 Gbps 24 Ports provided only to one port 1 Gbps 48 Ports per port group in dedicated or shared rate mode The port speed on an interface combined with the rate mode determines the amount of shared resources available to the ports in the port group on a 48 port 24 port 4 Gbps or any 8 Gbps Fibre Channel switching module Especially in the case of dedicated rate mode the port group resources are reserved even though the bandwidth is not used For example on Generation 2 modules if an interface is
192. hannels and FSPF links F and TF PortChannels An F PortChannel is also a logical interface that combines a set of F ports connected to the same Fibre Channel node and operates as one link between the F ports and the NP ports The F PortChannels support bandwidth utilization and availability like the E PortChannels F PortChannels are mainly used to connect MDS core and NPV switches to provide optimal bandwidth utilization and transparent failover between the uplinks of a VSAN An F PortChannel trunk combines the functionality and advantages of a TF port and an F PortChannel This logical link uses the Cisco PTP and PCP protocols over Cisco EPP ELS gt Note Ifa Cisco MDS 9124 or 9134 switch is used as a core switch only a nontrunking F PortChannel is supported Trunking is not supported on this platform when NPIV enabled PortChanneling and Trunking Trunking is a commonly used storage industry term However the Cisco NX OS software and switches in the Cisco MDS 9000 Family implement trunking and PortChanneling as follows e PortChanneling enables several physical links to be combined into one aggregated logical link e Trunking enables a link transmitting frames in the EISL format to carry trunk multiple VSAN traffic For example when trunking is operational on an E port that E port becomes a TE port A TE port is specific to switches in the Cisco MDS 9000 Family An industry standard E port can link to other vendor swit
193. hapter 5 Configuring Trunking TL ports and SD ports are not determined during initialization and are administratively configured amp Note Fibre Channel interfaces on Storage Services Modules SSMs cannot be configured in auto mode Interface States The interface state depends on the administrative configuration of the interface and the dynamic state of the physical link Administrative States The administrative state refers to the administrative configuration of the interface as described in Table 2 1 Table 2 1 Administrative States Administrative State Description Up Interface is enabled Down Interface is disabled If you administratively disable an interface by shutting down that interface the physical link layer state change is ignored Operational States The operational state indicates the current operational state of the interface as described in Table 2 2 Table 2 2 Operational States Operational State Description Up Interface is transmitting or receiving traffic as desired To be in this state an interface must be administratively up the interface link layer state must be up and the interface initialization must be completed Down Interface cannot transmit or receive data traffic Trunking Interface is operational in TE or TF mode Reason Codes Reason codes are dependent on the operational state of the interface as described in Table 2 3 Cisco MDS 90
194. he global buffer pool you need to reconfigure buffer credits on one or more of the ports The total number of buffer credits configured for all the ports in the global buffer pool should be reduced by 64 48 Port 8 Gbps Advanced Fibre Channel Module BB_Credit Buffers Table 4 1 lists the BB_credit buffer allocation for the 48 port 8 Gbps Advanced Fibre Channel switching module Table 4 1 48 Port 8 Gbps Advanced Switching Module BB_Credit Buffer Allocation BB_Credit Buffers Per Port Dedicated Rate Mode Shared Rate Mode 8 Gbps Speed 8 Gbps Speed BB_Credit Buffer Allocation ISL Fx Port Fx Port Default BB_credit buffers 250 for 48 32 32 port 500 for 32 port Maximum BB credit buffers 500 500 32 The following guidelines apply to BB_credit buffers on 32 48 port Advanced 8 Gbps Fibre Channel switching modules e BB credit buffers for ISL connections can be configured from a minimum of 2 buffers to a maximum of 500 buffers for dedicated rate mode e BB credit buffers for Fx port mode connections can be configured The minimum is 2 buffers and the maximum of 500 buffers for dedicated rate mode or 32 buffers for shared rate mode e Performance buffers are not supported on this module e The buffers should not be allocated automatically Each port group on the 32 48 port Advanced 8 Gbps Fibre Channel switching module consists of four six ports The ports in shared rate mode in a port group can have a maximum bandwi
195. he slow drain condition that affects the fabric by dropping the packets on the edge ports sooner than the time they actually get timed out 500 ms This function frees the buffer space in ISL which can be used by other unrelated flows that do not experience slow drain condition Note Management Note This feature is used mainly for edge ports that are connected to slow edge devices Even though this feature can be applied to ISLs as well we recommend that you apply this feature only for edge F ports and retain the default configuration for ISLs as E and TE ports This feature is not supported on Generation 1 modules Interfaces You can remotely configure the switch through the management interface mgmt0 To configure a connection on the mgmt0 interface you must configure either the IP version 4 IPv4 parameters IP address subnet mask and default gateway or the IP version 6 IPv6 parameters so that the switch is reachable Before you begin to configure the management interface manually obtain the switch s IPv4 address and subnet mask or the IPv6 address The management port mgmt0 is autosensing and operates in full duplex mode at a speed of 10 100 1000 Mbps Autosensing supports both the speed and the duplex mode On a Supervisor 1 module the default speed is 100 Mbps and the default duplex mode is auto On a Supervisor 2 module the default speed is auto and the default duplex mode is auto You need to explic
196. ibre Channel module DS X9248 256K9 or the 32 port 8 Gbps Advanced Fibre Channel module DS X9232 256K9 Performance Buffers Regardless of the configured receive BB_credit value additional buffers called performance buffers improve switch port performance Instead of relying on the built in switch algorithm you can manually configure the performance buffer value for specific applications for example forwarding frames over FCIP interfaces Note Buffer Pools Performance buffers are not supported on the Cisco MDS 9148 Fabric Switch Cisco MDS 9124 Fabric Switch the Cisco Fabric Switch for HP c Class BladeSystem and the Cisco Fabric Switch for IBM BladeCenter For each physical Fibre Channel interface in any switch in the Cisco MDS 9000 Family you can specify the amount of performance buffers allocated in addition to the configured receive BB_credit value The default performance buffer value is 0 If you set the performance buffer value to 0 the built in algorithm is used If you do not specify the performance buffer value 0 is automatically used The default performance buffer value is 0 If you use the default option the built in algorithm is used If you do not specify this command the default option is automatically used In the architecture of Generation 2 Generation 3 and Generation 4 modules receive buffers shared by a set of ports are called buffer groups The receive buffer groups are organized into global
197. ic Counter 2 36 Turning Off the Monitoring of Specific Counter 2 36 Activating a Port Group Monitor Policy 2 37 Configuring Management Interfaces 2 37 Creating VSAN Interfaces 2 38 Configuring Slow Drain Device Detection and Congestion Avoidance 2 38 Configuring Congestion Frame Timeout Value 2 39 Configuring Stuck Frame Timeout Value 2 39 Configuring No Credit Timeout Value 2 39 Configuring Credit Loss Recovery Threshold and Action 2 40 Configuring Average Credit Nonavailable Duration Threshold and Action 2 41 Verifying Interfaces Configuration 2 4 Displaying Interface Information 2 42 Displaying TL Port Information 2 50 Displaying the ALPA Cache Contents 2 51 Displaying Port Monitor Status and Policies 2 51 Displaying Port Group Monitor Status and Policies 2 53 I OL 29284 01 Release 6 x Contents W Cisco MDS 9000 Family NX OS Interfaces Configuration Guide E Contents Displaying Management Interface Configuration 2 55 Displaying VSAN Interface Information 2 55 CHAPTER 3 Configuring Fibre Channel Interfaces 3 1 Information About Fibre Channel Interfaces 3 1 Generations of Modules and Switches 3 1 Port Groups 3 3 Port Rate Modes 3 5 Dedicated Rate Mode 3 7 Shared Rate Mode 3 8 Dedicated Rate Mode Configurations for the 8 Gbps Modules 3 9 Port Speed 3 10 Dynamic Bandwidth Management 3 10 Out of Service Interfaces 3 11 Oversubscription Ratio Restrictions 3 11 Bandwidth Fairness 3 17 Upgrade or Downgrade Scena
198. ick Start Guide Software Installation and Upgrade e Cisco MDS 9000 NX OS Software Upgrade and Downgrade Guide Cisco NX 0S e Cisco MDS 9000 Family NX OS Licensing Guide e Cisco MDS 9000 Family NX OS Fundamentals Configuration Guide e Cisco MDS 9000 Family NX OS Interfaces Configuration Guide e Cisco MDS 9000 Family NX OS Fabric Configuration Guide Cisco MDS 9000 Family 1 0 Accelerator Configuration Guide oL 29284 01 ETN e Cisco MDS 9000 Family NX OS Quality of Service Configuration Guide e Cisco MDS 9000 Family NX OS Security Configuration Guide e Cisco MDS 9000 Family NX OS IP Services Configuration Guide e Cisco MDS 9000 Family NX OS Intelligent Storage Services Configuration Guide e Cisco MDS 9000 Family NX OS High Availability and Redundancy Configuration Guide e Cisco MDS 9000 Family NX OS Inter VSAN Routing Configuration Guide e Cisco MDS 9000 Family Cookbook for Cisco MDS SAN OS Command Line Interface e Cisco MDS 9000 Family Command Reference Intelligent Storage Networking Services Configuration Guides e Cisco MDS 9000 Family I O Acceleration Configuration Guide e Cisco MDS 9000 Family SANTap Deployment Guide e Cisco MDS 9000 Family Data Mobility Manager Configuration Guide e Cisco MDS 9000 Family Storage Media Encryption Configuration Guide Troubleshooting and Reference e Cisco MDS 9000 Family and Nexus 7000 Series System Messages Reference e Cisco MDS 9000 Family SAN OS Troubleshooting Guide e Cisc
199. if no out of service Makes the interface available for service Repeat Step 2 through Step 6 for all the interfaces in the port group Disabling ACL Adjacency Sharing for System Image Downgrade Fibre Channel ACL adjacency sharing is enabled by default on the switches with an active Generation 2 switching module as of Cisco MDS SAN OS Release 3 0 3 and with an active Generation 3 module as of MDS NX OS Release 4 1 1 Fibre Channel ACL adjacency sharing improves the performance for zoning and inter VSAN routing IVR network address translation NAT To prevent disruptions when downgrading the system image on your switch to a release prior to Cisco SAN OS Release 3 0 3 enter the following command in EXEC mode switch system no acl adjacency sharing To reenable Fibre Channel ACL adjacency sharing on your switch enter the following command in EXEC mode switch system acl adjacency sharing Verifying Fibre Channel Interfaces Configuration To display Fibre Channel interface configuration information perform one of the following tasks Command Pupe show module Displays the module show module s ot recovery steps Displays the slot for the module show port resources module slot Displays the port resources for the slot I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces HZ Verifying Fibre Channel Interfaces Configurat
200. ing also known as VSAN trunking is a feature specific to switches in the Cisco MDS 9000 Family Trunking enables interconnect ports to transmit and receive frames in more than one VSAN over the same physical link Trunking is supported on E ports and F ports See Figure 5 1 and Figure 5 2 This section includes the following topics e Trunking E Ports page 5 1 e Trunking F Ports page 5 2 e Key Concepts page 5 3 e Trunking Protocols page 5 3 e Trunk Modes page 5 4 e Trunk Allowed VSAN Lists and VF_IDs page 5 5 Trunking E Ports Trunking the E ports enables interconnect ports to transmit and receive frames in more than one VSAN over the same physical link using enhanced ISL EISL frame format Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x EN Chapter5 Configuring Trunking W Information About Trunking Figure 5 1 Trunking E Ports Switch 1 Any other Switch 1 Switch 2 E switch n f E j TE poria EISL Trunking Eao o E e E port E port an pajre port 79938 Note Trunking is not supported by internal ports on both the Cisco Fabric Switch for HP c_Class BladeSystem and the Cisco Fabric Switch for IBM BladeCenter Trunking F Ports Trunking F ports allows interconnected ports to transmit and receive tagged frames in more than one VSAN over the same physical link Figure 5 2 represents the possible trunking scenarios in a SAN with MDS core switches NPV sw
201. ing PortChannels Hs Configuration Examples for F and TF PortChannels swi swi swi tch tch tch config if switchport mode NP config if no shut config if exit Step6 Configure the PortChannel member interfaces on the NPV switch in dedicated mode swi swi swi swi swi swi swi swi swi tch tch tch tch tch tch tch tch tch config interface f c3 1 3 config i config i config i config i con con config i fig i config i Fh FH FH FH EF Fh Fh Fh shut switchport mode NP switchport speed 4000 switchport rate mode dedicated switchport trunk mode on channel group 2 no shut exit Step 7 Set the administrative state of all the PortChannel member interfaces in both NPIV core switch and the NPV switch to ON switch config interface fc1 4 6 switch config if shut switch config if no shut switch config interface c3 1 3 switch config if shut switch config if no shut Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x CHAPTER 7 Configuring N Port Virtualization Information About N Port Virtualization page 7 1 Guidelines and Limitations page 7 8 Configuring N Port Virtualization page 7 10 Verifying NPV Configuration page 7 13 information About N Port Virtualization This section includes the following topics NPV Overview N port virtualization NPV reduces the number of Fibre Channel domain
202. interface svr if range external interface port channel ext pc if range switch config Allows you to configure a list of external PortChannel interfaces per server interface by specifying the external interfaces in the svr if range The server to be linked is specified in the ext pc if range switch config no npv traffic map server interface svr if range external interface ext i1f range switch config Disables the NPV traffic management feature on the NPV 1 While mapping non PortChannel interfaces and PortChannel interfaces to the server interfaces include them separately in two steps Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter7 Configuring N Port Virtualization Enabling the Global Policy for Disruptive Load Balancing Detailed Steps Step 1 Step 2 Step 3 Verifying NPV Configuration W Disruptive load balancing allows you to review the load on all the external interfaces and balance the load disruptively Disruptive load balancing is done by moving the servers using heavily loaded external interfaces to the external interfaces running with fewer loads To enable or disable the global policy for disruptive load balancing perform the following tasks Command Purpose switch config t switch config Enters configuration mode on the NPV switch config npv auto load balance disrup tive switch config Enables d
203. ion Command Purpose show interface fc slot port Displays the slot or port information show interface brief Displays the interface show port index allocation Displays the port in the index allocation show port index allocation startup Displays the startup port in the index allocation show port channel compatibility parameters Displays the PortChannel compatibility parameters show module slot bandwidth fairness Displays the module slot bandwidth fairness information For detailed information about the fields in the output from these commands refer to the Cisco MDS NX OS Command Reference Displaying Interface Capabilities Before configuring a Generation 2 or Generation 3 interface you can use the show interface capabilities command to display detailed information about the capabilities of the interface This example shows the capabilities of a Generation 2 Fibre Channel interface switch show interface fc 9 1 capabilities Min Speed is 1 Gbps Max Speed is 4 Gbps FC PH Version high low 0 6 Receive data field size max min 2112 256 bytes Transmit data field size max min 2112 128 bytes Classes of Service supported are Class 2 Class 3 Class F Class 2 sequential delivery supported Class 3 sequential delivery supported Hold time max min 100 1 micro sec BB state change notification supported Maximum BB state change notifications 14 Rate Mode change supp
204. ion About Interfaces W e The operational status represents the current status of a specified attribute like the interface speed This status cannot be changed and is read only Some values may not be valid when the interface is down for example the operational speed Note E Port D When a module is removed and replaced with the same type of module the configuration is retained If a different type of module is inserted then the original configuration is no longer retained Each interface is briefly described in the sections that follow In expansion port E port mode an interface functions as a fabric expansion port This port may be connected to another E port to create an Inter Switch Link ISL between two switches E ports carry frames between switches for configuration and fabric management They serve as a conduit between switches for frames destined to remote N ports and NL ports E ports support class 2 class 3 and class F service An E port connected to another switch may also be configured to form a PortChannel see Chapter 6 Configuring PortChannels Note F Port FL Port We recommend that you configure E ports on 16 port modules If you must configure an E port on a 32 port oversubscribed module then you can only use the first port in a group of four ports for example ports 1 through 4 5 through 8 and so forth The other three ports cannot be used In fabric port F port mode an int
205. is determined at run time either when booting up the switch or when installing the modules In some cases the sequence in which switching modules are inserted into the chassis determines if one or more modules is powered up When a module does not power up because of a resource limitation you can display the reason by using the show module command switch show module Mod Ports Module Type Model Status 5 32 1 2 4 8 10 Gbps Advanced FC Module DS X9232 256K9 ok 7 0 Supervisor Fabric 2 DS X9530 SF2 K9 active 13 48 1 2 4 8 10 Gbps Advanced FC Module DS X9248 256K9 ok Mod Sw Hw World Wide Name s WWN 5 5 2 2 0 207 21 01 00 0d ec b7 28 c0 to 21 20 00 0d ec b7 28 c0 T 5 2 2 1 9 Suas 13 5 2 2 0 212 23 01 00 0d ec b7 28 c0 to 23 30 00 0d ec b7 28 c0 Mod MAC Address es Serial Num 5 68 ef bd a8 45 cc to 68 ef bd a8 45 d0 JAF1450CHQT 7 00 24 c4 60 00 f 8 to 00 24 c4 60 00 fc JAE141502L2 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide M320 fy OL 29284 01 Release 6 x Chapter 3 Configuring Fibre Channel Interfaces Guidelines and Limitations W 13 68 ef bd a8 40 00 to 68 ef bd a8 40 04 JAF1450BMBP Xbar Ports Module Type Model Status 1 0 Fabric Module 3 DS 13SLT FAB3 ok 2 0 Fabric Module 3 DS 13SLT FAB3 ok Xbar Sw Hw World Wide Name s WWN T NA 0 4 2 NA 0 4 Xbar MAC Address es Serial Num 1 NA JAF1451AMHG 2 NA JAF1451AMHN this terminal session The running configuration is upda
206. iscards 0 errors 1 input OLS 1 LRR 1 NOS 0 loop inits 2 output OLS 1 LRR 0 NOS 0 loop inits 16 receive B2B credit remaining 3 transmit B2B credit remaining Example 2 6 Displays Port Description switch Verifying Interfaces Configuration c3 10 fc3 11 fc3 12 fc3 16 show interface description ace Description test intest TE port Next hop switch 5 ace Description port channel 1 port channel 5 aS port channel 6 Example 2 7 Displays Interface Information in a Brief Format switch show interface brief Interf c3 16 fc3 17 Gigabi ace Vsan Admin Admin Status Mode Trunk Mode ak E on trunking 1 E on trunking 1 auto on SFPAbsent 1 auto on SFPAbsent 3000 auto on up 1 E on trunking 1 auto on down 1 auto on down 3000 auto on notConnected 1 auto on SFPAbsent 3000 FX up 1 FX SFPAbsent ace Status IP Address tEthernet4 1 SFPAbsent SFP Oper Oper Port Mode Speed Channel Gbps swl TE 2 1 swl TE 2 al swl F 2 swl TE 2 2 c1610 C1590 lwer oo swl F 2 Speed MTU auto 1500 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 2 Configuring Interfaces W Verifying Interfaces Configuration GigabitEthernet4 6 auto 3000 auto 1500 auto 1500 Oper Speed Gbps Speed Gbps 1 Speed MTU 100 Mbps 1500 Oper Oper Mode Speed Gbps TE 4 TE 4 Profile Port channel 10 ial
207. isruptive load balancing on the NPV core switch switch config no npv auto load balance disruptive Disables disruptive load balancing on the NPV core switch Verifying NPV Configuration This section includes the following topics e Verifying NPV page 7 14 e Verifying NPV Traffic Management page 7 15 To display NPV configuration information perform one of the following tasks Command Purpose show fens database Displays all the NPV devices in all the VSANs that the aggregator switch belongs to show fens database detail Displays additional details such as IP addresses switch names interface names about the NPV devices show npv flogi table Displays a list of the NPV devices that are logged in along with VSANs source information pWWNys and FCIDs show npy status Displays the status of the different servers and external interfaces show npv traffic map Displays the NPV traffic map show npv internal info traffic map Displays the NPV internal traffic details For detailed information about the fields in the output from these commands refer to the Cisco MDS NX OS Command Reference I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 7 Configuring N Port Virtualization W Verifying NPV Configuration Verifying NPV To view all the NPV devices in all the VSANs that the aggregator switch belongs to enter the show fens d
208. ist of trunk allowed VF IDs when trunking is enabled on the link If F port trunking and channeling is enabled or if switchport trunk mode on is configured in NPV mode for any interface or if NP PortChannel is configured the VSAN and VF ID ranges available for the configuration are as described in Table 5 3 Table 5 3 VSAN and VF ID Reservations VSAN or VF ID Description 000h Cannot be used as virtual fabric identifier 001h 1 to EFFh 3839 This VSAN range is available for user configuration FO0h 3840 to FEEh 4078 Reserved VSANSs and they are not available for user configuration FEFh 4079 EVFP isolated VSAN FFOh 4080 to FFEh 4094 Used for vendor specific VSANs FFFh Cannot be used as virtual fabric identifier Note If the VF_ID of the F port and the N port do not match then no tagged frames can be exchanged I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter5 Configuring Trunking Information About Trunking Figure 5 3 Default Allowed Active VSAN Configuration Switch 2 VSAN1 VSAN2 ao gy Sans ex gt Switch 1 VSAN1 VSAN2 VSAN3 VSAN4 VSAN5 VSANs 1 and 2 are operational 79945 You can configure a select set of VSANs from the allowed active list to control access to the VSANs specified in a trunking ISL Using Figure 5 3 as an example you can configure the list of allowed VSANs on a per interface basis
209. itches third party core switches and HBAs Figure 5 2 Trunking F Ports 3rd party Core MDS Core EVEP Switch Switch rT TF F TF TF TN ipa EPP EPP EPP 5 NP TNP TNP TNP NPV Switch EVFP HBA HBA 192090 Link Number Link Description la and 1b F port trunk with N port 2 F port trunk with NP port Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter5 Configuring Trunking Key Concepts Information About Trunking W Link Number Link Description 3 F PortChannnel with NP port 4 Trunked F PortChannel with NP port 5 Trunking NP port with third party core switch F port 1 These features are not supported currently The trunking feature includes the following key concepts TE port If trunk mode is enabled in an E port and that port becomes operational as a trunking E port it is referred to as a TE port TF port TIf trunk mode is enabled in an F port see the link 2 in Figure 5 2 and that port becomes operational as a trunking F port it is referred to as a TF port TN port lIf trunk mode is enabled not currently supported in an N port see the link 1b in Figure 5 2 and that port becomes operational as a trunking N port it is referred to as a TN port TNP port If trunk mode is enabled in an NP port see the link 2 in Figure 5 2 and that po
210. itly configure a default gateway to connect to the switch and send IP packets or add a route for each subnet I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces W Prerequisites for Interfaces VSAN Interfaces VSANs apply to Fibre Channel fabrics and enable you to configure multiple isolated SAN topologies within the same physical infrastructure You can create an IP interface on top of a VSAN and then use this interface to send frames to this VSAN To use this feature you must configure the IP address for this VSAN VSAN interfaces cannot be created for nonexisting VSANs Prerequisites for Interfaces Before you begin configuring the interfaces ensure that the modules in the chassis are functioning as designed To verify the status of a module at any time enter the show module command in EXEC mode For information about verifying the module status refer to the Cisco NX OS Fundamentals Configuration Guide Guidelines and Limitations This section includes the following topics e Generation 1 Interface Configuration Guidelines page 2 18 e Private Loop Configuration Guidelines page 2 19 e VSAN Interface Configuration Guidelines page 2 19 Generation 1 Interface Configuration Guidelines The Generation 1 interfaces configuration guidelines apply to the following hardware e The 32 port 2 Gbps or 1 Gbps switching module interfaces e The Cisco MDS 9
211. k a set of servers to a core switch by configuring the server to a set of external interfaces that are linked to the core switch DPVM Configuration Guidelines When NPV is enabled the following requirements must be met before you configure DPVM on the NPV core switch You must explicitly configure the WWN of the internal FLOGI in DPVM If DPVM is configured on the NPV core switch for an end device that is connected to the NPV device then that end device must be configured to be in the same VSAN Logins from a device connected to an NPV device will fail if the device is configured to be in a different VSAN To avoid VSAN mismatches ensure that the internal FLOGI VSAN matches the port VSAN of the NP port The first login from an NP port determines the VSAN of that port If DPVM is configured for this first login which is the internal login of the NPV device then the NPV core switch s VSAN F port is located in that VSAN Otherwise the port VSAN remains unchanged For details about DPVM configuration see the Cisco MDS 9000 Family NX OS Fabric Configuration Guide I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter7 Configuring N Port Virtualization WE Configuring N Port Virtualization NPV and Port Security Configuration Guidelines Port security is enabled on the NPV core switch on a per interface basis To enable port security on the NPV core switch for devices logging in via N
212. l software releases prior to Fabric Manager Release 5 0 1a Each guide addresses the features introduced in or available in a particular release Select and view the configuration guide that pertains to the software installed in your switch For a complete list of document titles see the list of Related Documentation in the Preface http www cisco com en US products ps5989 prod_release_notes_list htm About This Guide Table 1 lists the New and Changed features for this guide starting with MDS NX OS Release 5 2 1 Table 1 New and Changed Features Changed in Feature GUI Change Description Release Where Documented Port Monitor Configuring Port Added information about the 5 2 2a Chapter 2 Configuring Enhancements Monitor Policy feature Port Monitor Port Guard and Interfaces three new counters for the port monitor command Cisco MDS 9000 Family NX OS Interfaces Configuration Guide oL 29284 01 m Table 1 New and Changed Features continued Changed in Feature GUI Change Description Release Where Documented FlexAttach Disable FlexAttach Added information about disabling 5 0 1a Chapter 8 Configuring FlexAttach FlexAttach Virtual pWWN Port Group Check Oversubscription Added information about 5 0 1a Chapter 2 Configuring Monitoring gt Monitor monitoring a selected port group Interfaces Enhancements Cisco MDS 9000 Family NX OS Interfaces Configuration
213. le interfaces e Cisco MDS 9124 Fabric Switch e Cisco Fabric Switch for HP c Class BladeSystem e Cisco Fabric Switch for IBM BladeCenter Table 2 6 lists the TL port translations supported in Cisco MDS 9000 Family switches Table 2 6 Supported TL Port Translations Translation from Translation to Example Private initiator Private target From I1 to T1 or vice versa Private initiator Public target N port From I1 to T2 or vice versa Private initiator Public target NL port From 14 to T3 or vice versa Public initiator N port Private target From I2 to T1 or vice versa Public initiator NL port Private target From I3 to T1 or vice versa OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces HZ information About Interfaces Figure 2 3 shows examples of TL port translation support Figure 2 3 TL Port Translation Support Examples Private initiator 11 y Public a target T2 SS NL port Private Sas initiator 14 tees yO eae Private initiator 2 g tae ae Public Wag NL port Public target 3 EE loop eee NL port y Public initiator 13 91699 TL Port ALPA Caches Although TL ports cannot be automatically configured you can manually configure entries in arbitrated loop physical address ALPA caches Generally ALPA cache entries are automatically populated when an ALP
214. ll interval seconds delta rising threshold percentagel event event id falling threshold percentage event event id Specifies the delta Tx counter poll interval in seconds and thresholds in percentage switch config port monitor no counter sync loss Reverts to the default policy for sync loss performance counter values switch config port monitor no counter tx performance Reverts to the default policy for Tx performance counter values switch config port monitor monitor counter rx datarate Turns on Rx performance counter switch config port monitor monitor counter tx datarate Turns on Tx performance counter switch config port monitor no monitor counter tx datarate Turns off Tx performance counter switch config port monitor no monitor counter sync loss Turns off monitoring sync loss switch config port monitor no monitor counter state change Turns off monitoring state change 1 The error pkt port counter the err pkt from xbar counter and the err pkt to xbar counter are all ASIC counters All ASIC counters are turned off by default The Asic counters are queried every 10 seconds If the ASIC corresponding to a specific ASIC counter sends or receives any error packets during a 10 second interval an error frame event occurs during the interval for that counter 2 Falling threshold value should be less than the rising threshold
215. llow these steps Command Purpose Step1 switch config t Enters configuration mode switch config Step2 switch config no rate mode Disables bandwidth fairness for a module bandwidth fairness module 1 Note You must enter this command separately for each module for which you want to disable bandwidth fairness You must reload the module for the command to take effect Step3 switch config exit Exits configuration mode Taking Interfaces Out of Service You can take interfaces out of service on Generation 2 and Generation 3 switching modules When an interface is out of service all the shared resources for the interface are released as well as the configuration associated with those resources Prerequisites The interface must be disabled using a shutdown command before it can be taken out of service Restrictions e The interface cannot be a member of a PortChannel Taking interfaces out of service releases all the shared resources to ensure that they are available to other interfaces This causes the configuration in the shared resources to revert to default when the interface is brought back into service Also an interface cannot come back into service unless the default shared resources for the port are available The operation to free up shared resources from another port is disruptive Detailed Steps To take an interface out of service follow these steps Command Purpose Ste
216. llowing actions for interfaces operating in the E port mode e If you shut down an interface e Ifa Cisco NX OS software application executes a port shutdown as part of its function A graceful shutdown ensures that no frames are lost when the interface is shutting down When a shutdown is triggered either by you or the Cisco NX OS software the switches connected to the shutdown link coordinate with each other to ensure that all frames in the ports are safely sent through the link before shutting down This enhancement reduces the chance of frame loss I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces HZ information About Interfaces A graceful shutdown is not possible in the following situations e If you physically remove the port from the switch e Ifin order delivery IOD is enabled for information about IOD refer to the Cisco MDS 9000 Family NX OS Fabric Configuration Guide e Ifthe Min_LS_interval interval is higher than 10 seconds For information about FSPF global configuration refer to the Cisco MDS 9000 Family NX OS Fabric Configuration Guide amp Note This feature is only triggered if both switches at either end of this E port interface are MDS switches and are running Cisco SAN OS Release 2 0 1b or later or MDS NX OS Release 4 1 1a or later Port Administrative Speeds By default the port administrative speed for an interface
217. lport interface fc 1 16 proxied fc1 16 is up vsan 1 FCID 0x420000 0x01 20 10 00 05 30 00 4a de 20 00 00 05 30 00 4a de Initiator Oxfffc42 0x02 21 00 00 e0 8b 01 95 e7 20 00 00 e0 8b 01 95 e7 Initiator 0x420100 Displaying the ALPA Cache Contents The show tlport alpa cache command displays the contents of the ALPA cache switch show tlport alpa cache alpa PWWN Interface 0x02 22 00 00 20 37 46 09 bd fc1 2 0x04 23 00 00 20 37 46 09 bd fc1 2 The first entry indicates that if a device with a pWWN of 22 00 00 20 37 46 09 bd is exported on TL port fc 1 2 then the pWWN is allocated an alpa 0x02 if available Displaying Port Monitor Status and Policies The following commands display information regarding port monitor OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces W Verifying Interfaces Configuration switch show port monitor status Port Monitor Active Policies Last 10 logs switch show po Enabled sample rt monitor Policy Name Admin status Oper status sample Not Active Not Active Port type All Ports Counter Threshold Interval Rising Threshold portguard Link Loss Delta 60 5 Not enabled Sync Loss Delta 60 5 Not enabled Protocol Error Delta 60 T Not enabled Signal Loss Delta 60 5 Not enabled Invalid Words Delta 60 ak Not enabled Invalid CRC s Delta 60 5 Not enabled RX Performance Delta 60
218. lve ports with shared rate mode and 4 Gbps speed 5 1 oversubscription default e One port with dedicated rate mode and 8 Gbps speed plus eleven ports with shared rate mode and 4 Gbps speed 10 1 oversubscription e One port with dedicated rate mode and 4 Gbps speed plus three ports with dedicated rate mode and 3 Gbps speed plus eight ports with shared rate mode and 4 Gbps speed 2 1 oversubscription e Twelve ports with dedicated rate mode and 1 Gbps speed Cisco MDS 9000 Family NX OS Interfaces Configuration Guide I OL 29284 01 Release 6 x Chapter4 Configuring Interface Buffers Information About Interface Buffers 48 Port 4 Gbps Fibre Channel Module BB_Credit Buffers Table 4 5 lists the BB_credit buffer allocation for 48 port 4 Gbps Fibre Channel switching modules Table 4 5 48 Port 4 Gbps Switching Module BB_Credit Buffer Allocation BB_Credit Buffers Per Port Dedicated Rate Mode Shared Rate Mode 4 Gbps Speed 4 Gbps Speed BB_Credit Buffer Allocation ISL Fx Port Fx Port Default BB_credit buffers 125 16 16 Maximum BB credit buffers 250 250 16 Total number of BB_credit buffers per module 6000 1 ISL E port or TE port The following considerations apply to BB_credit buffers on 48 port 4 Gbps Fibre Channel switching modules e BB credit buffers for ISL connections can be configured from a minimum of 2 buffers to a maximum of 250 buffers for dedicated rate mode or 16 buffers for shared rat
219. main ID of the NPV core switch among multiple NPV switches NPV also allows multiple devices to attach to same port on the NPV core switch which reduces the need for more ports on the core Cisco MDS 9000 Family NX OS Interfaces Configuration Guide P72 E OL 29284 01 Release 6 x _ Chapter 7 Configuring N Port Virtualization Information About N Port Virtualization Figure 7 2 Cisco NPV Fabric Configuration NPV Core Switch MDS or 3rd party switch with NPIV support fs zZ lt S NP port qD Can have multiple uplinks NPV Device uses on different Up to 100 the same domains VSANs NPV switches as the NPV core lt lt switches gt Ea Cisco Fabric Switch for HP c Class BladeSystem Cisco Fabric Switch for IBM BladeCenter in a Blade Chassis Blade Server 1 E port server port gt dy 10 5 2 10 5 7 as ha 5 1 Initiator no FL ports y Target Blade Server 2 Blade Server n 184639 While NPV is similar to N port identifier virtualization NPIV it does not offer exactly the same functionality NPIV provides a means to assign multiple FC IDs to a single N port and allows multiple applications on the N port to use different identifiers NPIV also allows access control zoning and port security to be implemented at the application level NPV makes use of NPIV to get multiple FCIDs allocated from the core switch on the NP port Figure 7 3 shows a mo
220. ministrators For coordination it is important that the SAN configuration does not change when a new server is installed or when an existing server is replaced FlexAttach virtual pWWN minimizes the interaction between the server administrator and the SAN administrator by abstracting the real pWWN using virtual pWWNs When FlexAttach virtual pWWN is enabled on an interface a virtual pWWN is assigned to the server interface The real pWWN is replaced by a virtual pWWN which is used for a SAN configuration such as zoning Server administrators can benefit from FlexAttach in the following scenarios Pre configure Pre configure SAN for new servers that are not available physically yet For example they may be on order FlexAttach can be enabled on the ports designated for the new servers and use the virtual WWNs assigned for configuring SAN The new servers are then plugged into the fabric without any change needed in the SAN I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter8 Configuring FlexAttach Virtual pWWN W Information About FlexAttach Virtual pWWN e Replacement to the same port A failed server can be replaced onto the same port without changing the SAN The new server gets a same pWWN as the failed server because the virtual pWWN is assigned to the port e Replacement to spare A spare server which is on the same NPV device or a different NPV device can be brought o
221. mode ST port mode description 2 5 interface modes 2 5 limitations 2 5 ST ports interface modes 2 5 subnet masks configuring mgmt0 interfaces 2 17 switch ports configuring attribute default values 2 28 T TE port mode classes of service 2 4 description 2 4 TE ports trunking restrictions 5 3 TF port mode classes of service 2 4 description 2 4 TL port mode classes of service 2 4 description 2 4 E Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x TL ports ALPA caches 2 14 configuring 2 22 2 29 description 2 13 displaying information 2 50 private devices 2 50 virtual devices 2 51 tracked ports binding operationally 9 4 translative loop port mode See TL port mode trunk allowed VSAN lists description 5 5 to 5 7 trunking comparison with PortChannels 6 3 configuration guidelines 5 10 configuring modes 5 4 link state 5 4 merging traffic 5 10 restrictions 5 3 5 7 trunking E port mode See TE port mode trunking F port mode See TF port mode trunking protocol default settings 5 11 default state 5 4 detecting port isolation 5 10 disabling 5 11 5 12 enabling 5 11 5 12 trunk mode administrative default 2 29 configuring 5 4 5 12 status 5 4 trunk ports displaying information 5 14 Vv virtual devices TL ports 2 51 VSAN IDs allowed list 5 11 multiplexing traffic 2 4 VSAN interfaces creating 2 38 description 2 18 displaying information 2 55 VSANs allowed a
222. mode and 1 Gbps speed plus five ports put out of service see Figure 4 6 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Cu OL 29284 01 Release 6 x Chapter4 Configuring Interface Buffers Information About Interface Buffers W Figure 4 6 Example Speed and Rate Configuration on a 48 Port 4 Gbps Switching Module 1 3 5 7 9 11 4 Gbps 4 Gbps 1 Gbps 1 Gbps Dedicated Dedicated Shared Shared Out of Out of Service Service 2 4 6 8 4 Gbps 1 Gbps 1 Gbps Dedicated Shared Shared Out of Service 144858 e Six ports with dedicated rate mode and 2 Gbps speed plus four ports with shared rate mode and 1 Gbps speed plus two ports put out of service see Figure 4 7 S Note For detailed configuration steps of this example see Configuration Example for 48 Port 4 Gbps Module Interfaces section on page 3 48 Figure 4 7 Example Speed and Rate Configuration on a 48 Port 4 Gbps Switching Module 1 7 9 3 5 2 Gbps 2 Gbps 2 Gbps 1 Gbps 1 Gbps Dedicated Dedicated Dedicated Shared Shared 2 4 6 8 10 2 Gbps 2 Gbps 2 Gbps 1 Gbps 1 Gbps C Dedicated Dedicated Dedicated Shared Shared Out of Service 144859 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x EZIN Chapter4 Configuring Interface Buffers HZ Information About Interface Buffers 24 Port 4 Gbps Fibre Channel Module BB_Credit Buffers Table 4 6 lists the BB_credit buffer allocation
223. ms are resolved and the tracked port is back up you can explicitly enable the interface Tip If you configure this feature the linked port continues to remain in the shutdown state even after the tracked port comes back up You must explicitly remove the forced shut state by administratively bringing up this interface of the linked port once the tracked port is up and stable Forcefully Shutting Down a Tracked Port Detailed Steps To forcefully shut down a tracked port follow these steps Command Purpose Step1 switch config t Enters configuration mode Step2 switch config interface fc1 5 Configures the specified interface and enters the interface configuration submode You can now configure tracked ports Step3 switch config if port track force shut Forcefully shuts down the tracked port switch config if no port track force shut Removes the port shutdown configuration for the tracked port Displaying Port Tracking Information Examples The show commands display the current port tracking settings for the Cisco MDS switch see Examples 9 1 to 9 4 Example 9 1 Displays the Linked and Tracked Port Configuration switch show interface fc8 6 is down All tracked ports down Kn nnnnn enna nnn n ne enna nnn e eee eee Linked port Hardware is Fibre Channel FCOT is short wave laser Port WWN is 21 c6 00 05 30 00 37 le Admin port mode is auto trunk mode is on Port vsan is 1 Receive data field
224. n Guide Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Table 3 2 Bandwidth and Port Groups for the Fibre Channel Modules and Fabric Switches continued Bandwidth Per Maximum Product Name Number of Ports Port Group Bandwidth Per Port Part Number Description Per Port Group Gbps Gbps 32 port 8 Gbps 4 32 4 or 12 8 8 or 10 Advanced Fibre Gbps depending Channel switching on the DS X9232 256K9 module configuration Generation 3 Modules DS X9248 96K9 48 port 8 Gbps 6 12 8 8 Gbps Fibre Channel switching module DS X9224 96K9 24 port 8 Gbps 3 12 8 8 Gbps Fibre Channel switching module DS X9248 48K9 4 44 port 8 Gbps 12 12 8 8 4 Gbps Host Optimized Fibre Channel switching module Generation 3 Fabric Switches DS C9148 K9 48 port 8 Gbps 4 32 8 Gbps Cisco MDS 9148 Eppa Fabric switch Generation 2 Modules DS X9148 48 port 4 Gbps 12 12 8 4 Gbps Fibre Channel switching module DS X9124 24 port 4 Gbps 6 12 8 4 Gbps Fibre Channel switching module DS X9304 18K9 18 port 4 Gbps 6 12 8 4 Gbps MSM 18 4 Fibre Channel Multiservice module switching module with 4 Gigabit Ethernet ports DS X9112 12 port 4 Gbps 3 12 8 4 Gbps Fibre Channel switching module DS X9704 4 port 10 Gbps 1 10 10 Gbps Fibre Channel switching module Generation 2 Fabric Switches Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 R
225. n frame timeout congestion mode core edge value in milliseconds and the port mode for the device The congestion timeout range is 100 1000 milliseconds switch show logging onboard flow control Optional Displays the request timeout for a request timeout source destination pair per module with the timestamp information Configuring Stuck Frame Timeout Value Step 1 Step 2 The default stuck frame timeout value is 500 ms We recommend that you retain the default configuration for ISLs and configure a value not exceeding 500 ms 100 to 200 ms for fabric F ports To configure the stuck frame timeout value follow these steps Command Purpose switch config t Enters configuration mode switch config system timeout Specifies the stuck frame timeout value in ms congestion drop seconds mode E F and the port mode for the switch switch config system timeout Specifies the default stuck frame timeout port congestion drop default mode E F mode for the switch Configuring No Credit Timeout Value When the port does not have the credits for the configured period no credit timeout can be enabled on that port This will result in all frames coming to that port getting dropped in the egress This will free the buffer space in the ISL link which carries traffic for this port This will help reduce fabric slow down and congestion on other unrelated flows using the same link The frames that will be dropped would
226. nd 24 port 4 Gbps and all 8 Gbps Fibre Channel switching modules support oversubscription on switches with shared rate mode configurations By default all 48 port and 24 port 4 Gbps and 8 Gbps Fibre Channel switching modules have restrictions on oversubscription ratios enabled As of Cisco SAN OS Release 3 1 1 and NX OS Release 4 1 1 you can disable restrictions on oversubscription ratios Table 3 9 describes the bandwidth allocation for oversubscribed interfaces configured in shared mode on the 4 Gbps and 8 Gbps modules I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Information About Fibre Channel Interfaces Table 3 9 Bandwidth Allocation for Oversubscribed Interfaces Reserved Bandwidth Gbps Maximum Switching Bandwidth Module Configured Speed Ratios enabled Ratios disabled Gbps 24 Port 8 Gbps_ Auto 8 Gbps 0 8 0 8 8 Fibre Channel Auto Max 4 Gbps 0 4 0 4 4 Module Auto Max 2 Gbps 0 2 0 2 2 4 44 Port 8 Gbps 0 87 0 16 8 8 Gbps Auto Max 4 Gbps 0 436 0 08 4 Host Optimized Fibre Channel Auto Max 2 Gbps 0 218 0 04 2 Module 1 Gbps 0 109 0 02 1 48 port 4 Gbps_ Auto 4 Gbps 0 8 0 09 4 Fibre Channel Auto Max 2 Gbps 0 4 0 045 2 switching module 1 Gbps 0 2 0 0225 1 24 port 4 Gbps_ Auto 4 Gbps 1 0 27 4 Fibre Channel Auto Max
227. nel member ports at either end if you add or remove ports from the PortChannel configuration You must physically verify that the local and remote ports are connected to each other e ACTIVE The member ports initiate PortChannel protocol negotiation with the peer port s regardless of the channel group mode of the peer port If the peer port while configured in a channel group does not support the PortChannel protocol or responds with a nonnegotiable status it will default to the ON mode behavior The ACTIVE PortChannel mode allows automatic recovery without explicitly enabling and disabling the PortChannel member ports at either end Cisco MDS 9000 Family NX OS Interfaces Configuration Guide lt OL 29284 01 Release 6 x Chapter6 Configuring PortChannels Information About PortChannels W Table 6 1 compares ON and ACTIVE modes Table 6 1 Channel Group Configuration Differences ON Mode ACTIVE Mode No protocol is exchanged A PortChannel protocol negotiation is performed with the peer ports Moves interfaces to the suspended state if its Moves interfaces to the isolated state if its operational values are incompatible with the operational values are incompatible with the PortChannel PortChannel When you add or modify a PortChannel member When you add or modify a PortChannel interface port configuration you must explicitly disable the PortChannel automatically recovers shut and enable no shut the PortCh
228. nfiguration Example for 4 44 Port 8 Gbps Module Interfaces 3 47 Configuration Example for 48 Port 4 Gbps Module Interfaces 3 48 Configuration Example for 24 Port 4 Gbps Module Interfaces 3 49 CHAPTER 4 Configuring Interface Buffers 4 1 Information About Interface Buffers 4 1 Buffer to Buffer Credits 4 1 Performance Buffers 4 2 Buffer Pools 4 2 BB_Credit Buffers for Switching Modules 4 5 Configuring Buffer Credits on a Generation 2 Generation 3 or Generation 4 Module 4 5 48 Port 8 Gbps Advanced Fibre Channel Module BB_Credit Buffers 4 6 48 Port 8 Gbps Fibre Channel Module BB_Credit Buffers 4 7 24 Port 8 Gbps Fibre Channel Module BB_Credit Buffers 4 8 4 44 Port 8 Gbps Host Optimized Fibre Channel Module BB_Credit Buffers 4 9 48 Port 4 Gbps Fibre Channel Module BB_Credit Buffers 4 10 24 Port 4 Gbps Fibre Channel Module BB_Credit Buffers 4 12 18 Port Fibre Channel 4 Port Gigabit Ethernet Multiservice Module BB_Credit Buffers 4 13 12 Port 4 Gbps Switching Module BB_Credit Buffers 4 13 4 Port 10 Gbps Switching Module BB_Credit Buffers 4 15 BB_Credit Buffers for Fabric Switches 4 16 Cisco MDS 9148 Fabric Switch BB_Credit Buffers 4 16 Cisco MDS 9134 Fabric Switch BB_Credit Buffers 4 16 Cisco MDS 9124 Fabric Switch BB_Credit Buffers 4 17 Cisco MDS 9222 Multiservice Modular Switch BB_Credit Buffers 4 17 Extended BB_Credits 4 17 Extended BB_credits on Generation 1 Switching Modules 4 18 Extended BB_credits on Generation 2 and Generation 3 Switching Modul
229. nged Information xiii Preface xv Audience xv Organization xv Document Conventions xvi Related Documentation xvii Release Notes xvii Regulatory Compliance and Safety Information xvii Compatibility Information xvii Hardware Installation xvii Software Installation and Upgrade xvii Cisco NX OS xvii Command Line Interface xviii Intelligent Storage Networking Services Configuration Guides xviii Troubleshooting and Reference xviii Obtaining Documentation and Submitting a Service Request xviii Interfaces Overview 1 1 Trunks and PortChannels 1 1 Fibre Channel Port Rate Limiting 1 1 Extended Credits 1 2 N Port Virtualization 1 2 FlexAttach 1 2 Configuring Interfaces 2 1 Information About Interfaces 2 1 Interface Description 2 2 Interface Modes 2 2 EPort 2 3 FPort 2 3 FLPort 2 3 NP Ports 2 3 TL Port 2 4 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide I OL 29284 01 Release 6 x E Contents TEPort 2 4 TF Port 2 4 TNP Port 2 5 SD Port 2 5 ST Port 2 5 Fx Port 2 5 BPort 2 5 Auto Mode 2 5 Interface States 2 6 Administrative States 2 6 Operational States 2 6 Reason Codes 2 6 Graceful Shutdown 2 9 Port Administrative Speeds 2 10 Autosensing 2 10 Frame Encapsulation 2 10 Beacon LEDs 2 11 Speed LEDs 2 11 Bit Error Thresholds 2 11 SFP Transmitter Types 2 12 TL Ports 2 13 TL Port ALPA Caches 2 14 Port Guard 2 14 Port Monitor 2 15 Port Monitor Port Guard 2 16 Port Group Monitor 2 16 Local Switchin
230. nline without changes to the SAN This action is achieved by moving the virtual port WWN from the current server port to the spare port e Server Mobility A server can be moved to another port on the same NPV device or another NPV device without changing the SAN This is accomplished by moving the virtual pWWN to the new port No change is needed if FlexAttach was configured using the physical port WWN of the server to the virtual port WWN mapping Difference Between San Device Virtualization and FlexAttach Port Virtualization Table 8 1 describes the difference between SAN device virtualization SDV and FlexAttach port virtualization Table 8 1 Difference Between SDV and FlexAttach Virtualization SAN Device Virtualization SDV FlexAttach Virtualization Facilitates target and disk management and only facilitates disk and data migration Facilitates server management and has no restriction on the end devices used WWN NAT and Fibre Channel ID FC ID are allocated on the virtual device both primary and secondary WWN and Network Address Transport NAT is allocated to host bus adapter HBA FC ID rewrite on the switch indicates a rewrite capable switch on the path No rewrite requirements Configuration is distributed This allows programming rewrites and connectivity anywhere Configuration distribution is not required for any of the interface based configurations Configuration is secured
231. nnel no member port becomes isolated or suspended instead the member port is removed from the channel group when the link is found to be incompatible Any administrative configuration made to the PortChannel is applied to all ports in the channel group and you can save the configuration for the PortChannel interface Any administrative configuration made to the PortChannel is applied to all ports in the channel group but the configurations are saved for the member ports no configuration is saved for the PortChannel interface You can explicitly convert this channel group if required You can remove any channel group and add members to a channel group You cannot remove a channel group or add remove any of its members The channel group is removed when no member ports exist Note Autocreation Autocreation is not supported as of MDS NX OS Release 4 1 1b and later The autocreation protocol has the following functionality e A portis not allowed to be configured as part of a PortChannel when the autocreation feature is enabled These two configurations are mutually exclusive e Autocreation must be enabled in both the local and peer ports to negotiate a PortChannel e Aggregation occurs in one of two ways A portis aggregated into a compatible autocreated PortChannel A port is aggregated with another compatible port to form a new PortChannel I OL 29284 01 Release 6 x Cisco MDS 9000
232. nnel While configuring the channel in dedicated mode ensure that required bandwidth is available to the ports This example shows how to configure channeling in dedicated mode and bring up the TF TNP PortChannel link between TF ports in the NPIV core switch and TNP ports in the NPV switch Enable the F port trunking and channeling protocol on the MDS core switch switch config feature fport channel trunk Enable NPIV on the MDS core switch switch config feature npiv Create the PortChannel on the MDS core switch switch config interface port channel 2 switch config if switchport mode F switch config if switchport rate mode dedicated switch config if channel mode active switch config if exit Configure the PortChannel member interfaces on the MDS core switch in dedicated mode switch config interface fc1 4 6 shut switchport mode F switchport speed 4000 switchport rate mode dedicated switchport trunk mode on channel group 2 no shut exit switch config i switch config i switch config i switch config i switch config i switch config i switch config i switch config i Fh Ph FH Ph FH Fh Ph Pi Create the PortChannel in dedicated mode on the NPV switch switch config interface port channel 2 switch config if switchport rate mode dedicated I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 6 Configur
233. nnel Interfaces HE Guidelines and Limitations Generation switching modules have specific numbering requirements If these requirements are not met the module does not power up The port index numbering requirements include the following e If port indexes in the range of 256 to 1020 are assigned to operational ports Generation 1 switching modules do not power up e A block of contiguous port indexes is available If this block of port indexes is not available Generation modules do not power up Table 3 10 shows the port index requirements for the Generation modules Note If the switch has Supervisor 1 modules the block of 32 contiguous port indexes must begin on the slot boundary The slot boundary for slot 1 is 0 for slot 2 is 32 and so on For Supervisor 2 modules the contiguous block can start anywhere Table 3 10 Port Index Requirements for Generation 1 Modules Number of Port Indexes Required Generation 1 Module Supervisor 1 Module Supervisor 2 Module 16 port 2 Gbps Fibre Channel module 16 16 32 port 2 Gbps Fibre Channel module 32 32 8 port Gigabit Ethernet IP Storage Services module 32 32 4 port Gigabit Ethernet IP Storage Services module 32 16 32 port 2 Gbps Fibre Channel Storage Services 32 32 Module SSM 14 port Fibre Channel 2 port Gigabit Ethernet 32 22 Multiprotocol Services MPS 14 2 module The allowed mix of Generation and Generation 2 switching modules in a chassis
234. nterfaces Configuration Guide I OL 29284 01 Release 6 x Chapter6 Configuring PortChannels HZ information About PortChannels PortChannels Overview PortChannels refer to the aggregation of multiple physical interfaces into one logical interface to provide higher aggregated bandwidth load balancing and link redundancy See Figure 6 1 PortChannels can connect to interfaces across switching modules so a failure of a switching module cannot bring down the PortChannel link Figure 6 1 PortChannel Flexibility Switch 1 Switch 2 PortChannel A J J PortChannel B 00000000 60000000 00000000 SOO0000000 00000000 UY 0000000 00000000 dies aaa eoo00000 8 PortChannels on Cisco MDS 9000 Family switches allow flexibility in configuration This illustrates three possible PortChannel configurations E PortChannels PortChannel A aggregates two links on two interfaces on the same switching module at each end of a connection PortChannel B also aggregates two links but each link is connected to a different switching module If the switching module goes down traffic is not affected PortChannel C aggregates three links Two links are on the same switching module at each end while one is connected to a different switching module on switch 2 An E PortChannel refers to the aggregation of multiple
235. ntified by the source FC ID destination FC ID and OX_ID are carried over the same physical link in both directions This helps make applications such as write acceleration work for PortChannels over FCIP links e Autocreation protocol Automatically aggregates compatible ports into a PortChannel This section describes how to configure the PortChannel protocol and includes the following sections e Channel Group Creation page 6 10 e Autocreation page 6 11 e Manually Configured Channel Groups page 6 12 Channel Group Creation amp Note Channel groups are not supported on internal ports in the Cisco Fabric Switch for HP c Class BladeSystem and the Cisco Fabric Switch for IBM BladeSystem Assuming link A1 B1 comes up first see Figure 6 6 that link is operational as an individual link When the next link comes up for example A2 B2 the PortChannel protocol identifies if this link is compatible with link A1 B1 and automatically creates channel groups 10 and 20 in the respective switches If link A3 B3 can join the channel groups the PortChannels the respective ports have compatible configurations If link A4 B4 operates as an individual link it is because of the incompatible configuration of the two end ports with the other member ports in this channel group Figure 6 6 Autocreating Channel Groups Channel Group 10 Channel Group 20 po10 po20 1 A ll Uh Cisco MDS Cisco MDS Switch A 3 3 Switch B
236. o MDS 9000 Family NX OS MIB Quick Reference e Cisco DCNM for SAN Database Schema Reference Obtaining Documentation and Submitting a Service Request For information on obtaining documentation submitting a service request and gathering additional information see the monthly What s New in Cisco Product Documentation which also lists all new and revised Cisco technical documentation at http www cisco com en US docs general whatsnew whatsnew html e Subscribe to the What s New in Cisco Product Documentation as a Really Simple Syndication RSS feed and set content to be delivered directly to your desktop using a reader application The RSS feeds are a free service and Cisco currently supports RSS version 2 0 Cisco MDS 9000 Family 1 0 Accelerator Configuration Guide avii OL 29284 01 CHAPTER 1 Interfaces Overview e Trunks and PortChannels page 1 1 e Fibre Channel Port Rate Limiting page 1 1 e Extended Credits page 1 2 e N Port Virtualization page 1 2 e FlexAttach page 1 2 Trunks and PortChannels Trunking also known as VSAN trunking is a feature specific to switches in the Cisco MDS 9000 Family Trunking enables interconnect ports to transmit and receive frames in more than one VSAN over the same physical link Trunking is supported on E ports and F ports PortChannels aggregate multiple physical ISLs into one logical link with higher bandwidth and port resiliency for both Fibre Channel and FICON traffic With
237. ocal switching module 5 Disables local switching for a module If you want to proceed with disabling local switching when prompted press Y to continue or N to exit switch config exit Exits configuration mode Use the show system internal xbar local switching command to verify the local switching configuration status on a module switch show system internal xbar local switching Slot Local switching enabled 1 no 2 no 3 no 4 no 5 yes 6 no 9 no 10 no 11 no 12 no 13 no Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter3 Configuring Fibre Channel Interfaces Configuring Fibre Channel Interfaces W Disabling Restrictions on Oversubscription Ratios Prerequisites Detailed Steps Step 1 Step 2 Step 3 Step 4 Examples Step 1 e Before disabling restrictions on oversubscription ratios ensure that you have explicitly shut down shared ports To disable restrictions on oversubscription ratios on a 48 port or 24 port 4 Gbps or any 8 Gbps Fibre Channel switching module follow these steps Command Purpose switch config t Enters configuration mode switch config switch config no rate mode Disables restrictions on oversubscription ratios for a oversubscription limit module 1 module Note You must enter this command separately for each module for which you want to remove the restrictions switch config exit
238. ode and 4 Gbps speed Information About Interface Buffers W 4 44 Port 8 Gbps Host Optimized Fibre Channel Module BB_Credit Buffers Table 4 4 lists the BB_credit buffer allocation for the 4 44 port 8 Gbps Fibre Channel switching module Table 4 4 4 44 Port 8 Gbps Switching Module BB_Credit Buffer Allocation BB_Credit Buffers Per Port Dedicated Rate Mode 8 Gbps Speed Shared Rate Mode 8 Gbps Speed BB_Credit Buffer Allocation ISL Fx Port Fx Port Default BB_credit buffers 125 32 32 Maximum BB credit buffers 250 250 32 Total number of BB_credit buffers per module 6000 The following guidelines apply to BB_credit buffers on 4 44 port 8 Gbps Fibre Channel switching modules e BB credit buffers for ISL connections can be configured from a minimum of 2 buffers to a maximum of 500 buffers for dedicated rate mode e BB credit buffers for Fx port mode connections can be configured The minimum is 2 buffers and the maximum of 250 buffers for dedicated rate mode or 32 buffers for shared rate mode e Performance buffers are not supported on this module Each port group on the 24 port 8 Gbps Fibre Channel switching module consists of 12 ports The ports in shared rate mode in a port group can have a maximum bandwidth oversubscription of 10 1 considering that each port group has 12 8 Gbps bandwidth The following example configurations are supported by the 4 44 port 8 Gbps Fibre Channel switching modules e Twe
239. or 48 Port 8 Gbps Advanced Fibre Channel modules where bandwidth is shared support 1 Gbps 2 Gbps 4 Gbps or 8 Gbps traffic in a maximum or 32 or 48 ports All ports in the 48 Port and 24 Port 8 Gbps Fibre Channel switching modules where bandwidth is shared support 1 Gbps 2 Gbps 4 Gbps or 8 Gbps traffic In the 4 44 Port 8 Gbps Host Optimized Fibre Channel switching module all the ports where bandwidth is shared support 1 Gbps 2 Gbps 4 Gbps in a maximum of 44 ports or 8 Gbps in a maximum of 4 ports Dedicated Rate Mode Configurations for the 8 Gbps Modules Table 3 7 shows the maximum possible dedicated rate mode configuration scenarios for the Generation 4 Fibre Channel modules Table 3 7 Dedicated Rate Mode Bandwidth Reservation for Generation 4 Fibre Channel Modules Maximum Dedicated Allowed Ports Product Name Bandwidth That Can Come Part Number Description per Port Up Ports in Shared Mode DS X9248 256K9 48 port 8 Gbps 10 Gbps 24 Ports All the remaining ports Advanced Fibre 8 Gbps 32 Ports are 8 Gbps shared Channel switching module 4 Gbps 48 Ports 2 Gbps 48 Ports 1 Gbps 48 Ports DS X9232 256K9 32 port 8 Gbps 10 Gbps 24 Ports All the remaining ports Advanced Fibre 8 Gbps 32 Ports are 8 Gbps shared Channel switching module 4 Gbps 32 Ports 2 Gbps 32 Ports 1 Gbps 32 Ports Table 3 8 shows the maximum possible dedicated rate mode configuration scenarios for the Generation 3 Fibre C
240. or shared See the Configuring Rate Mode section on page 3 30 Configure the port mode See the Configuring Interface Modes section on page 2 22 amp Note ISL ports cannot operate in shared rate mode Configure the BB_credits and extended BB_credits as necessary See the Configuring Buffer to Buffer Credits section on page 4 21 and the Configuring Extended BB_credits section on page 4 23 Task Flow for Migrating Interfaces from Dedicated Mode to Shared Mode Step 1 Step 2 Step 3 Step 4 To configure the 4 Gbps and 8 Gbps Fibre Channel switching modules migrating from dedicated rate mode to shared rate mode follow these steps Take unused interfaces out of service to release resources for other interfaces if necessary See the Taking Interfaces Out of Service section on page 3 41 Configure the BB_credits and extended BB_credits as necessary See the Configuring Buffer to Buffer Credits section on page 4 21 and the Extended BB_Credits section on page 4 17 Configure the port mode See the Configuring Interface Modes section on page 2 22 Note ISL ports cannot operate in shared rate mode Configure the rate mode dedicated or shared to use See the Configuring Rate Mode section on page 3 30 OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces
241. orted Rate Mode Capabilities Shared Dedicated Receive BB Credit modification supported yes yes FX mode Receive BB Credit min max default 1 16 16 1 250 16 ISL mode Receive BB Credit min max default am 2 250 250 Performance buffer modification supported no no Out of Service capable yes Beacon mode configurable yes This example shows the capabilities of an interface module switch show interface fc 4 1 capabilities Min Speed is 1 Gbps Max Speed is 8 Gbps on the 48 port 8 Gbps Fibre Channel switching FC PH Version high low 0 6 Receive data field size max min 2112 256 bytes Transmit data field size max min 2112 128 bytes Classes of Service supported are Class 2 Class 3 Class F Class 2 sequential delivery supported Class 3 sequential delivery supported Hold time max min 100000 1 micro sec Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x _ Chapter 3 Configuring Fibre Channel Interfaces BB state change notification Maximum BB state change notifications Rate Mode change Rate Mode Capabilities Receive BB Credit modification supported FX mode Receive BB Credit ISL mode Receive BB Credit Performance buffer modification supporte Out of Service capable Beacon mode configurable Extended B2B credit capable Displaying SFP Diagnostic Information min max default min max default s Configuration Examples for Fibre Channel Interfaces
242. os Prerequisites e You must enable restrictions on oversubscription ratios before you can downgrade modules to a previous release e Before enabling restrictions on oversubscription ratios ensure that you have explicitly configured shared ports to out of service mode I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces W Configuring Fibre Channel Interfaces Detailed Steps Step 1 Step 2 Step 3 Step 4 Step5 Step 6 Step7 To enable restrictions on oversubscription ratios on a 48 port or 24 port 4 Gbps or any 8 Gbps Fibre Channel switching module follow these steps Command Purpose switch config t switch config Enters configuration mode switch config interface fc2 1 2 c2 4 5 fc2 8 38 c2 43 48 Specifies the port interfaces for which you want to enable restrictions on oversubscription ratios switch config if shutdown Shuts down shared ports switch config if out of service Takes shared ports out of service switch rate mode oversubscription limit module 1 Enables restrictions on oversubscription ratios for the module Note You must enter this command separately for each module for which you want to add the restriction switch config t switch config interface fc2 1 2 c2 4 5 c2 8 38 fc2 43 48 switch config if no out of service switch confi
243. oup 2 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Bandwidth Rate Mode Buffers Gbps fc4 7 32 8 0 shared c4 8 32 8 0 shared c4 9 32 8 0 shared fc4 10 32 8 0 shared fc4 11 32 8 0 shared Cisco MDS 9000 Family NX OS Interfaces Configuration Guide 3 32 By OL 29284 01 Release 6 x Chapter3 Configuring Fibre Channel Interfaces Fc4 12 32 Port Group 3 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Configuring Fibre Channel Interfaces shared Gbps shared shared shared shared shared Interfaces in the Port Group B2B Credit Buffers c4 13 32 c4 14 32 c4 15 32 fc4 16 32 fc4 17 32 fc4 18 32 Port Group 4 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps shared Gbps shared shared shared shared shared Interfaces in the Port Group B2B Credit Buffers fc4 19 32 fc4 20 32 fc4 21 32 fc4 22 32 c4 23 32 Fc4 24 32 Port Group 5 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps shared Gbps shared shared shared shared shared Interfaces in the Port Group B2B Credit Buffers c4 25 32 c4 26 32 c4 27 32 c4 28 32 c4 29 32 c4 30 32 Port Group 6 Total bandwidt
244. owever if the two ports for example 5 and 6 are taken out of service which is not same as shutdown required reserved bandwidth for the two ports 3 and 4 is 0 48 and port 2 can be configured at 4 Gbps in dedicated rate mode This behavior can be overridden by the bandwidth fairness command in which case reserved bandwidth is not enforced Once the port is up ports 3 and 4 can share the unutilized bandwidth from ports 1 and 2 Out of Service Interfaces A On supported modules and fabric switches you might need to allocate all the shared resources for one or more interfaces to another interface in the port group or module You can take interfaces out of service to release shared resources that are needed for dedicated bandwidth When an interface is taken out of service all shared resources are released and made available to the other interface in the port group or module These shared resources include bandwidth for the shared mode port rate mode BB_credits and extended BB_credits All shared resource configurations are returned to their default values when the interface is brought back into service Corresponding resources must be made available in order for the port to be successfully returned to service Caution If you need to bring an interface back into service you might disrupt traffic if you need to release shared resources from other interfaces in the same port group Oversubscription Ratio Restrictions The 48 port a
245. p1 switch config t Enters configuration mode switch config Step 2 switch config interface fe 1 1 Selects the interface and enters interface switch config if configuration submode Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x o Chapter 3 Configuring Fibre Channel Interfaces W Configuring Fibre Channel Interfaces Step 3 Step 4 Step5 Command Purpose switch config if no channel group Removes the interface from a PortChannel switch config if shutdown Disables the interface switch config if out of service Putting an interface into out of service will cause its shared resource configuration to revert to default Do you wish to continue y n n Y Takes the interface out of service Use the show port resources module command to verify the out of service configuration for interfaces on a Generation 2 and Generation 3 switching module This example shows a 24 port 4 Gbps module switch show port resources module 9 Module 9 Available dedicated buffers are 5429 Port Group 1 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Buffers c9 1 16 c9 2 out of service fc9 3 16 c9 4 16 c9 5 16 c9 6 16 Releasing Shared Resources in a Port Group Restrictions Bandwidth Rate Mode Gbps 4 0 shared
246. p1 switch config t Enters configuration mode Step2 switch config interface fc1 1 Selects a Fibre Channel interface and enters interface configuration submode Step3 9 switch config if shutdown Gracefully shuts down the interface and administratively disables traffic flow default To enable traffic flow follow these steps Command Purpose Step1 switch config t Enters configuration mode Step2 switch config interface fc1 1 Selects a Fibre Channel interface and enters interface configuration submode Step3 switch config if no shutdown Enables traffic flow to administratively allow traffic when the no prefix is used provided the operational state is up Configuring Interface Modes Detailed Steps To configure the interface mode follow these steps Command Purpose Step1 switch config t Enters configuration mode Step2 switch config interface fcl 1 Selects a Fibre Channel interface and enters interface switch config if configuration submode Step3 switch config if switchport mode F Configures the administrative mode of the port You switch config if can set the operational state to auto E F FL Fx TL NP or SD port mode Note Fx ports refers to an F port or an FL port host connection only but not E ports switch config if switchport mode auto Configures the interface mode to auto negotiate an E switch config if F FL or TE port mode not TL or SD
247. policy to all ports switch config port monitor counter credit loss reco poll interval seconds percentagel event event id delta rising threshold percentagei event event id Specifies the delta credit loss recovery counter poll interval in seconds the thresholds in percentage and the event IDs of events switch config port monitor monitor counter err pkt from port switch config port monitor counter err pkt from port poll interval seconds delta rising threshold eventcounti event event id falling threshold eventcount2 event event 1d Activates the err pkt from port counter Specifies the delta err pkt from port counter poll interval in seconds the thresholds in percentage and the event IDs of events switch config port monitor monitor counter err pkt from xbar switch config port monitor counter err pkt from xbar poll interval seconds delta rising threshold eventcounti event event id switch config port monitor monitor counter err pkt to xbar switch config port monitor counter err pkt to xbar poll interval seconds delta rising threshold eventcounti event event 1d falling threshold eventcount event event id Activates the err pkt from xbar counter Specifies the delta err pkt from xbar counter poll interval in seconds the thresholds in the count of error frame events and the event IDs of events Activates the err pkt to xbar counter Specifies the delta err pkt to x
248. port 8 Gbps switching modules The minimum BB_credits required to bring up a port is two buffers Figure 4 2 BB_Credit Buffer Allocation in 48 Port 8 Gbps Switching Modules 48 port module s y All ports All ports shared 8 Gpbs Mixed dedicated 2 Gbps 32 BB credits 250 BB credits Dedicated Shared 32 BB credits 1 Gbps 2 Gbps 4 Gbps 8 Gbps 2 250 BB credits 250 BB credits 250 BB credits 250 BB credits 3 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x EN Chapter 4 Configuring Interface Buffers Figure 4 3 Information About Interface Buffers BB_Credit Buffer Allocation in 24 Port 8 Gbps Switching Modules 24 port module er a m All ports All ports shared 8 Gpbs Mixed dedicated 2 Gbps 32 BB credits 500 BB credits Dedicated Shared 32 BB credits 1 Gbps 2 Gbps 4 Gbps 8 Gbps 500 BB credits 500 BB credits 500 BB credits 500 BB credits Figure 4 4 BB_Credit Buffer Allocation in 4 44 Port 8 Gbps Switching Modules 4 44 port module All ports All ports shared 8 Gpbs Mixed dedicated 2 Gbps 32 BB credits 125 BB credits Dedicated Shared 32 BB credits 1 Gbps 2 Gbps 4 Gbps 8 Gbps 250 BB credits 250 BB credits 250 BB credits 12
249. port trunk mode ON on non NPV and MDS core switches OFF on NPV switches Allowed VSAN list 1 to 4093 user defined VSAN IDs Allowed VF ID list 1 to 4093 user defined VF IDs Trunking protocol on E ports Enabled Trunking protocol on F ports Disabled Configuring Trunking This section includes the following topics e Enabling the Cisco Trunking and Channeling Protocols page 5 11 e Enabling the F Port Trunking and Channeling Protocol page 5 12 e Configuring Trunk Mode page 5 12 e Configuring an Allowed Active List of VSANs page 5 12 Enabling the Cisco Trunking and Channeling Protocols This section describes how to enable the required trunking and channeling protocols Prerequisites e To avoid inconsistent configurations disable all ports with a shutdown command before enabling or disabling the trunking protocols Detailed Steps To enable or disable the Cisco trunking and channeling protocol follow these steps Command Purpose Step1 switch config t Enters configuration mode Step2 switch config trunk protocol enable Enables the Cisco PTP trunking protocol switch config default switch config no trunk protocol enable Disables the Cisco PTP trunking protocol switch config Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x m Chapter5 Configuring Trunking WE Configuring Trunking Enabling the F Port Trunking and Channeling Protocol
250. ps Speed 8 Gbps Speed BB_Credit Buffer Allocation ISL Fx Port Fx Port Default BB_credit buffers 250 32 32 Maximum BB_credit buffers 500 500 32 Total Number of BB_Credit Buffers per Module Ports 1 through 24 6000 Ports 25 through 48 6000 The following guidelines apply to BB_credit buffers on 48 port 8 Gbps Fibre Channel switching modules BB_credit buffers allocated for ports through 24 and 25 through 48 can be a maximum of 6000 each so that the load is distributed BB_credit buffers for ISL connections can be configured from a minimum of 2 buffers to a maximum of 500 buffers for dedicated rate mode BB_credit buffers for Fx port mode connections can be configured The minimum is 2 buffers and the maximum of 500 buffers for dedicated rate mode or 32 buffers for shared rate mode Performance buffers are not supported on this module The buffers should not be allocated automatically Each port group on the 48 port 8 Gbps Fibre Channel switching module consists of six ports The ports in shared rate mode in a port group can have a maximum bandwidth oversubscription of 10 1 considering that each port group has 12 8 Gbps bandwidth The following example configurations are supported by the 48 port 8 Gbps Fibre Channel switching modules Six ports with shared rate mode and 8 Gbps speed 4 1 oversubscription default One port with dedicated rate mode and 8 Gbps speed plus five ports with shared rate mode and 8 Gbps speed
251. r Supervisor 2 modules However combining switching modules and supervisor modules has the following port index limitations e Supervisor modules only support a maximum of 252 port indexes regardless of the type of switching modules e Supervisor 2 modules support a maximum of 1020 port indexes when all switching modules in the chassis are Generation 2 or Generation 3 e Supervisor 2 modules only support a maximum of 252 port indexes when only Generation 1 switching modules or a combination of Generation 1 Generation 2 Generation 3 or Generation 4 switching modules are installed in the chassis On a switch with the maximum limit of 252 as port index any new module that exceeds the limit does not power up when installed You can use the show port index allocation command to display the allocation of port indexes on the switch switch show port index allocation Module index distribution Slot Allowed Allotted indices info range Total Index values ee eee ana a a eos ere ees 1 None 2 None 3 None 4 None 5 0 1023 32 0 31 6 None 9 None 10 None 11 None 12 None 13 0 1023 48 32 79 SUP 253 255 3 253 255 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Cha
252. r Switching Module Switching Module Switching Module Switching Module Speed mode _ auto auto auto auto Rate mode shared shared dedicated dedicated Port mode Fx Fx auto auto BB_credit 16 16 250 250 buffers Performance 1454 1455 buffers The 4 port 10 Gbps switching module only supports 10 Gbps traffic 2 Auto port mode on the 12 port 4 Gbps switching module interfaces can operate in E port mode TE port mode and Fx port mode 3 Auto port mode on the 4 port 10 Gbps switching module interfaces can operate in E port mode TE port mode and F port mode 4 Performance buffers are shared among all ports on the module Table 3 13 lists the default settings for Generation 3 interface parameters Table 3 13 Default Generation 3 Interface Parameters Default 4 44 Port 8 Gbps 48 Port 8 Gbps 24 Port 8 Gbps Host Optimized Switching Parameter Switching Module Switching Module Module Speed mode auto auto auto_max_4G Rate mode shared shared shared Port mode Fx Fx Fx BB credit buffers 32 32 32 1 Auto_max_4G speed mode on the 4 44 port 8 Gbps switching module negotiates to a maximum speed of 4 Gbps Table 3 14 lists the default settings for Generation 4 interface parameters Table 3 14 Default Generation 4 Interface Parameters Default 48 Port 8 Gbps Advanced Fibre 32 Port 8 Gbps Advanced Fibre Parameter Channel Switching Module Channel Switching Module Speed mode auto
253. r the ability to prevent this issue from occurring in environments that are vulnerable to these problems The port can be configured to stay down after the first failure or after a specified number of failures in a specified time period This allows the SAN administrator to intervene and control the recovery avoiding any problems caused by the cycling Using the port guard feature you can restrict the number of error reports and bring a malfunctioning port to down state dynamically A port can be configured to go into error disabled state for specific types of failures A general link failure caused by link down is the superset of all other causes The sum of the number of all other causes equals to the number of link down link failures This means a port is brought to down state when it reaches the maximum number of allowed link failures or the number of specific causes The causes of link failure can be any of the following e ESP trustsec violation e Bit errors e Signal loss e Sync loss e Link reset e Credit loss e Additional causes might be the following Not operational NOS Too many interrupts Cable is disconnected Hardware recoverable errors The connected device rebooted F ports only The connected linecard rebooted ISL only Port monitor helps to monitor the performance and the status of ports and generate alerts when problems occur You can configure the thresholds for various counters and trigger an event
254. r throttling WAN traffic at the source to help eliminate excessive buffering in Fibre Channel and IP data network devices I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter1 Interfaces Overview HI Extended Credits Extended Credits Full line rate Fibre Channel ports provide at least 255 buffer credits standard Adding credits lengthens distances for Fibre Channel SAN extension Using extended credits up to 4095 buffer credits from a pool of more than 6000 buffer credits for a module can be allocated to ports as needed to greatly extend the distance for Fibre Channel SANs N Port Virtualization Cisco NX OS software supports industry standard N port identifier virtualization NPIV which allows multiple N port fabric logins concurrently on a single physical Fibre Channel link HBAs that support NPIV can help improve SAN security by enabling zoning and port security to be configured independently for each virtual machine OS partition on a host In addition to being useful for server connections NPIV is beneficial for connectivity between core and edge SAN switches N port virtualizer NPV is a complementary feature that reduces the number of Fibre Channel domain IDs in core edge SANs Cisco MDS 9000 family fabric switches operating in the NPV mode do not join a fabric they only pass traffic between core switch links and end devices which eliminates the domain IDs for these switches
255. r9 Configuring Port Tracking HZ Guidelines and Limitations Figure 9 1 Traffic Recovery Using Port Tracking ISL2 rT Direct link 1 ggl gy i i Dx 120490 The port tracking feature monitors and detects failures that cause topology changes and brings down the links connecting the attached devices When you enable this feature and explicitly configure the linked and tracked ports the Cisco NX OS software monitors the tracked ports and alters the operational state of the linked ports on detecting a link state change The following terms are used in this chapter e Tracked ports A port whose operational state is continuously monitored The operational state of the tracked port is used to alter the operational state of one or more ports Fibre Channel VSAN PortChannel FCIP or a Gigabit Ethernet port can be tracked Generally ports in E and TE port modes can also be Fx ports e Linked ports A port whose operational state is altered based on the operational state of the tracked ports Only a Fibre Channel port can be linked Guidelines and Limitations Before configuring port tracking consider the following guidelines e Verify that the tracked ports and the linked ports are on the same Cisco MDS switch e Do not track a linked port back to itself for example Port fc1 2 to Port fc2 5 and back to Port fc1 2 to avoid recursive dependency e Be aware that the linked port is automatically brought down when the
256. racking Multiple Ports Detailed Steps Step 1 Step 2 Step 3 Step 4 Configuring Port Tracking W To track multiple ports follow these steps Command Purpose switch config t Enters configuration mode switch config interface fc8 6 Configures the specified interface and enters the interface configuration submode You can now configure tracked ports Note This link symbolizes the direct link 1 in Figure 9 2 switch config if port track interface port channel 1 Tracks interface fc8 6 with interface port channel 1 When port channel 1 goes down interface fc8 6 is also brought down Note This link symbolizes the ISL 2 in Figure 9 2 switch config if port track interface fcip 5 Tracks interface fc8 6 with interface fcip 5 When FCIP 5 goes down interface fc8 6 is also brought down Note This link symbolizes the ISL 3 in Figure 9 2 Information About Monitoring Ports ina VSAN je You can optionally configure one VSAN from the set of all operational VSANs on the tracked port with the linked port by specifying the required VSAN This level of flexibility provides higher granularity in tracked ports In some cases when a tracked port is a TE port the set of operational VSANs on the port can change dynamically without bringing down the operational state of the port In such cases the port VSAN of the linked port can be monitored on the set of operational VSANs on the tracked port
257. re granular view of an NPV configuration at the interface level Figure 7 3 Cisco NPV Configuration Interface View Host N Port NPV Core Switch F Port Fal NPIV enabled NPV Device F Port Pe Port CT foe Port Host 184631 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 7 Configuring N Port Virtualization Hi information About N Port Virtualization NPV Mode A switch is in NPV mode after a user has enabled NPV and the switch has successfully rebooted NPV mode applies to an entire switch All end devices connected to a switch that is in NPV mode must log in as an N port to use this feature loop attached devices are not supported All links from the edge switches in NPV mode to the NPV core switches are established as NP ports not E ports which are used for typical interswitch links NPIV is used by the switches in NPV mode to log in to multiple end devices that share a link to the NPV core switch Note In order data delivery is not required in NPV mode because the exchange between two end devices always takes the same uplink to the core from the NPV device For traffic beyond the NPV device core switches will enforce in order delivery if needed and or configured After entering NPV mode only the following commands are available aaa arp banner boot callhome cli clock do end exit fcanalyzer fcorxbbcredit fip
258. red Port Group 4 Total bandwidth is 12 8 Gbps Total shared bandwidth is 12 8 Gbps Allocated dedicated bandwidth is 0 0 Gbps Interfaces in the Port Group B2B Credit Bandwidth Rate Mode Buffers Gbps fc7 37 32 4 0 shared fc7 38 32 4 0 shared fc7 39 32 4 0 shared fc7 40 32 4 0 shared fc7 41 32 4 0 shared fc7 42 32 4 0 shared fc7 43 32 4 0 shared fc7 44 32 4 0 shared fc7 45 32 4 0 shared fc7 46 32 4 0 shared fc7 47 32 4 0 shared fc7 48 32 4 0 shared Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x PEEN Chapter 3 Configuring Fibre Channel Interfaces W Configuring Fibre Channel Interfaces Configuring Local Switching amp Note Detailed Steps Step 1 Step 2 Step 3 Step 4 We recommend that you shut down all of the ports on the module before you execute the local switching command If local switching is enabled then ports cannot be configured in dedicated mode If there are dedicated ports and you enter the local switching command a warning is displayed and the operation is prevented To enable or disable local switching module follow these steps Command Purpose switch config t switch config Enters configuration mode switch config local switching module 5 Enables local switching for a module If you want to proceed with enabling local switching when prompted press Y to continue or N to exit switch config no l
259. reshold is exceeded one or more of these actions can be combined together These actions can be turned on or off depending on situation The port monitor feature provides the command line interface to configure the thresholds and action The thresholds are that the credit loss recovery can be between 1 and 10 and the interval can be 1 second to 1 hour The default value is 3 in 10 minutes and generates a syslog When the port sees the credit loss condition and fails to recover the port flaps This function is already part of port guard and you can configure the supported actions using the Port Guard feature To configure credit loss recovery threshold and action refer to the following example The following example shows the credit loss recovery threshold and action configuration switch show port monitor Policy Name Admin status Oper status Port type Cisco Active Active All Ports Counter event Portguard Link Loss Not enabled Sync Loss Not enabled Protocol Error Not enabled Signal Loss Not enabled Invalid Words Not enabled Invalid CRC s Not enabled RX Performance Not enabled TX Performance Not enabled TX Discards Not enabled Threshold Interval Rising Threshold event Falling Threshold Delta Delta Delta Delta Delta Delta Delta 60 60 60 60 60 60 60 2147483648 2147483648 200 1 4 1 4 0 4 1 4 0 4 1 4 524288000 4 524288000 4 10 4
260. rio 3 17 Guidelines and Limitations 3 18 Combining Generation 1 Generation 2 Generation 3 and Generation 4 Modules 3 18 Local Switching Limitations 3 19 Port Index Limitations 3 19 PortChannel Limitations 3 21 Default Settings 3 25 Configuring Fibre Channel Interfaces 3 26 Task Flow for Migrating Interfaces from Shared Mode to Dedicated Mode 3 26 Task Flow for Migrating Interfaces from Dedicated Mode to Shared Mode 3 27 Task Flow for Configuring 12 Port 4 Gbps Module Interfaces 3 28 Task Flow for Configuring 4 Port 10 Gbps Module Interfaces 3 28 Configuring Port Speed 3 29 Configuring Rate Mode 3 30 Displaying the Rate Mode Configuration for Interfaces 3 31 Configuring Local Switching 3 36 Disabling Restrictions on Oversubscription Ratios 3 37 Enabling Restrictions on Oversubscription Ratios 3 39 Enabling Bandwidth Fairness 3 40 Disabling Bandwidth Fairness 3 41 Taking Interfaces Out of Service 3 41 Releasing Shared Resources ina Port Group 3 42 Disabling ACL Adjacency Sharing for System Image Downgrade 3 43 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide i E OL 29284 01 Release 6 x Contents W Verifying Fibre Channel Interfaces Configuration 3 43 Displaying Interface Capabilities 3 44 Displaying SFP Diagnostic Information 3 45 Configuration Examples for Fibre Channel Interfaces 3 45 Configuration Example for 48 Port 8 Gbps Module Interfaces 3 46 Configuration Example for 24 Port 8 Gbps Module Interfaces 3 46 Co
261. rmation About Interface Buffers W 4 Port 10 Gbps Switching Module BB_Credit Buffers Table 4 9 lists the BB_credit buffer allocation for 4 port 10 Gbps switching modules Table 4 9 4 Port 10 Gbps Switching Module BB_Credit Buffer Allocation BB_Credit Buffers Per Port Dedicated Rate Mode 10 Gbps Speed BB_Credit Buffer Allocation Type ISL F port Default BB_credit buffers 250 16 Maximum BB_credit buffers 750 16 Maximum BB_credit buffers on one of the ports with 4095 Enterprise license Total number of BB_credit buffers per module 5488 Default Performance buffers 145 12 Total number of performance buffers per module 512 shared 1 ISL E port or TE port 2 Ports on the 4 port 10 Gbps cannot operate in FL port mode Note The ports in the 4 port 10 Gbps switching module only support 10 Gbps dedicated rate mode FL port mode and shared rate mode are not supported The following considerations apply to BB_credit buffers on 4 port 10 Gbps switching modules e BB credit buffers for ISL connections can be configured from a minimum of 2 buffers to a maximum of 750 buffers e BB_credit buffers for Fx port mode connections can be configured from a minimum of 2 buffers to a maximum of 750 buffers e By default 512 performance buffers are preallocated and are shared by all the ports These buffers are configurable and the buffers are assigned to the port based on the availability of the buffers in the sh
262. rt becomes operational as a trunking NP port it is referred to as a TNP port TF PortChannel If trunk mode is enabled in an F PortChannel see the link 4 in Figure 5 2 and that PortChannel becomes operational as a trunking F PortChannel it is referred to as TF PortChannel Cisco Port Trunking Protocol PTP is used to carry tagged frames TF TN port link A single link can be established to connect an F port to an HBA to carry tagged frames see the link la and 1b in Figure 5 2 using Exchange Virtual Fabrics Protocol EVFP A server can reach multiple VSANs through a TF port without inter VSAN routing IVR TF TNP port link A single link can be established to connect an TF port to an TNP port using the PTP protocol to carry tagged frames see the link 2 in Figure 5 2 PTP is used because PTP also supports trunking PortChannels amp Note The TF TNP port link between a third party NPV core and a Cisco NPV switch is established using the EVFP protocol A Fibre Channel VSAN is called Virtual Fabric and uses a VF_ID in place of the VSAN ID By default the VF_ID is for all ports When an N port supports trunking a pWWN is defined for each VSAN and called a logical pWWN In the case of MDS core switches the pWWNs for which the N port requests additional FC_IDs are called virtual pWWNs Trunking Protocols The trunking protocol is important for trunking operations on the ports The protocols enable the following activities
263. rt speed so the bandwidth allocation for the same three ports mentioned in the example would be 1 1 1 Bandwidth Fairness A This feature improves fairness of bandwidth allocation among all ports and provides better throughput average to individual data streams Bandwidth fairness can be configured per module As of Cisco SAN OS Release 3 1 2 all 48 port and 24 port 4 Gbps Fibre Channel switching modules as well as 18 port Fibre Channel 4 port Gigabit Ethernet Multiservice modules have bandwidth fairness enabled by default As of Cisco NX OS Release 4 1 1 all the 8 Gbps Fibre Channel switching modules have bandwidth fairness enabled by default Caution When you disable or enable bandwidth fairness the change does not take effect until you reload the module Use the show module bandwidth fairness command to check whether ports in a module are operating with bandwidth fairness enabled or disabled switch show module 2 bandwidth fairness Module 2 bandwidth fairness is enabled Note This feature is supported only on the 48 port and 24 port 4 Gbps modules the 8 Gbps modules and the 18 4 port Multiservice Module MSM Upgrade or Downgrade Scenario When you are upgrading from a release earlier than Cisco SAN OS Release 3 1 2 all modules operate with bandwidth fairness disabled until the next module reload After the upgrade any new module that is inserted has bandwidth fairness enabled I OL 29284 01
264. rustsec violation bit errors for the interface The link resumes flapping and credit loss link reset signal loss sending error reports normally sync loss 8 P y This example shows how to configure port guard to bring a port to down state if the link flaps 5 times within 120 seconds based on multiple causes Switch config t Switch config interface fc1 1 Switch config if errdisable detect cause link down num times 5 duration 120 Switch config if errdisable detect cause bit errors num times 5 duration 120 Switch config if errdisable detect cause credit loss num times 5 duration 120 The above example sets the configuration to the following status e The port will be error disabled due to bit errors if the port suffers link failure due to bit errors 5 times in 120 seconds e The port will be error disabled due to credit loss if the port suffers link failure due to credit loss 5 times in 120 seconds e The port will be error disabled due to link down if the port suffers link failure due to bit errors 2 times and link failure due to credit loss 3 times in 120 seconds This example shows the internal information about a port in down state because of TrustSec violation Cisco MDS 9000 Family NX OS Interfaces Configuration Guide e230 1 OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Switch show port internal info interface fc8 3 fc8 3 is down Error disabled port down due to trust
265. s hw module interface ip ipv6 line logging no npv ntp port track power poweroff radius radius server rate mode rmon role snmp server ssh switchname system tacacs telnet username wwn Configure aaa functions no remove an entry from the ARP cache Configure banner message Configure boot variables Enter the callhome configuration mode CLI configuration commands Configure time of day clock EXEC command Exit from configure mode Exit from configure mode Configure cisco fabric analyzer Enable extended rx b2b credit configuration Enable Disable FIPS mode Enable Disable OBFL information Select an interface to configure Configure IP features Configure IPv6 features Configure a terminal line Modify message logging facilities Negate a command or set its defaults Config commands for FC N_port Virtualizer NTP Configuration Configure Switch port track config Configure power supply Poweroff a module in the switch Configure RADIUS configuration Configure RADIUS related parameters Configure rate mode oversubscription limit Remote Monitoring Configure roles Configure snmp server Configure SSH parameters Configure system s network name System config command Enable tacacs Enable telnet Configure user information Set secondary base MAC addr and range for additional WWNs Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter
266. s and to maintain consistency across switches then the ports shut down You must explicitly enable those ports again e If you use the ACTIVE mode then the PortChannel ports automatically recover from the deletion Interfaces in a PortChannel You can add or remove a physical interface or a range of interfaces to an existing PortChannel The compatible parameters on the configuration are mapped to the PortChannel Adding an interface to a PortChannel increases the channel size and bandwidth of the PortChannel Removing an interface from a PortChannel decreases the channel size and bandwidth of the PortChannel This section describes interface configuration for a PortChannel and includes the following topics e Interface Addition to a PortChannel page 6 8 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter6 Configuring PortChannels HZ information About PortChannels e Forcing an Interface Addition page 6 9 e Interface Deletion from a PortChannel page 6 9 amp Note For information about PortChannel support on Generation 2 switching modules see the PortChannel Limitations section on page 3 21 Interface Addition to a PortChannel You can add a physical interface or a range of interfaces to an existing PortChannel The compatible parameters on the configuration are mapped to the PortChannel Adding an interface to a PortChannel increases the channel size and b
267. s interfaces to operate in any other mode for example preventing an interface to connect to another switch While E ports typically interconnect Fibre Channel switches some SAN extender devices such as the Cisco PA FC 1G Fibre Channel port adapter implement a bridge port B port model to connect geographically dispersed fabrics This model uses B ports as described in the T11 Standard FC BB 2 If an FCIP peer is a SAN extender device that only supports Fibre Channel B ports you need to enable the B port mode for the FCIP link When a B port is enabled the E port functionality is also enabled and they coexist If the B port is disabled the E port functionality remains enabled see the Cisco MDS 9000 Family NX OS IP Services Configuration Guide Interfaces configured in auto mode can operate in one of the following modes F port FL port E port TE port or TF port The port mode is determined during interface initialization For example if the interface is connected to a node host or disk it operates in F port or FL port mode depending on the N I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces HZ information About Interfaces port or NL port mode If the interface is attached to a third party switch it operates in E port mode If the interface is attached to another switch in the Cisco MDS 9000 Family it may become operational in TE port mode see C
268. s invoked from the EXEC mode and displays trunking configurations for a TE port Without any arguments this command displays the information for all of the configured interfaces in the switch See Examples 5 1 to 5 3 Example 5 1 Displays a Trunked Fibre Channel Interface switch show interface fc1 13 fc1 13 is trunking Admin port mode is auto Port mode is TE Port vsan is 1 Speed is 2 Gbps Beacon is turned off Trunk vsans up Trunk vsans isolated Hardware is Fibre Channel Port WWN is 20 0d 00 05 30 00 58 1le Peer port WWN is 20 0d 00 05 30 00 59 1le Receive B2B Credit is 255 Trunk vsans admin allowed and active trunk mode is on 1 1 Trunk vsans initializing 5 minutes input rate 0 bits sec 0 bytes sec 0 frames sec 5 minutes output rate 0 bits sec 0 bytes sec 0 frames sec 233996 frames input 14154208 bytes 0 discards 0 CRC 0 unknown class 0 too long 0 too short 236 frames output 13818044 bytes 0 discards 11 input OLS 12 LRR 10 NOS 28 loop inits I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter5 Configuring Trunking W Configuration Example for F Port Trunking 34 output OLS 19 LRR 17 NOS 12 loop inits Example 5 2 Displays the Trunking Protocol switch show trunk protocol Trunk protocol is enabled Example 5 3 Displays Per VSAN Information on Trunk Ports switch show interface trunk vsan 1 1000 fc
269. se 6 x CHAPTER Configuring Interfaces e Information About Interfaces page 2 1 e Prerequisites for Interfaces page 2 18 e Guidelines and Limitations page 2 18 e Default Settings page 2 20 e Configuring Interfaces page 2 20 e Verifying Interfaces Configuration page 2 41 Information About Interfaces The main function of a switch is to relay frames from one data link to another To relay the frames the characteristics of the interfaces through which the frames are received and sent must be defined The configured interfaces can be Fibre Channel interfaces Gigabit Ethernet interfaces the management interface mgmt0 or VSAN interfaces This section includes the following topics e Interface Description page 2 2 e Interface Modes page 2 2 e Interface States page 2 6 e Graceful Shutdown page 2 9 e Port Administrative Speeds page 2 10 e Frame Encapsulation page 2 10 e Beacon LEDs page 2 11 e Speed LEDs page 2 11 e Bit Error Thresholds page 2 11 e SFP Transmitter Types page 2 12 e TL Ports page 2 13 e TL Port ALPA Caches page 2 14 e Port Guard page 2 14 e Port Monitor page 2 15 e Port Monitor Port Guard page 2 16 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces HZ information About Interfaces e Port Group Monitor page 2 16 e Local Switching page 2 16 e Slow Drain Device Detection and Congestion Avoidance page 2 17
270. sec violation Hardware is Fibre Channel SFP is short wave laser w o OFC SN Port WWN is 21 c3 00 0d ec 10 57 80 Admin port mode is E trunk mode is on snmp link state traps are enabled Port vsan is 1 Receive data field Size is 2112 Beacon is turned off 5 minutes input rate 0 bits sec 0 bytes sec 0 frames sec 5 minutes output rate 0 bits sec 0 bytes sec 0 frames sec 11274 frames input 1050732 bytes 0 discards 0 errors 0 CRC 0 unknown class 0 too long 0 too short 11242 frames output 971900 bytes 0 discards 0 errors 11 input OLS 34 LRR 10 NOS 0 loop inits 72 output OLS 37 LRR 2 NOS 0 loop inits Interface last changed at Sun Nov 27 07 34 05 1988 admin port down trustsec violation 3 num_times 0 duration 0 state reason Error disabled port down due to trustsec violation Port guard trustsec violation is Enabled errdisabled on trustsec violation TRUE oper cnt 1 port guard first trustsec violation Sun Nov 27 07 34 05 1988 Troubleshooting Tips Configuring Interfaces e Link down is the superset of all other causes A port is brought to down state if the total number of other causes equals to the number of allowed link down failures e Even if the link does not flap due to failure of the link and port guard is not enabled the port goes into a down state if too many invalid FLOGI requests are received from the same host Use the shut and the no shut commands consecutively to bring up the link Config
271. see Figure 5 4 For example if VSANs 2 and 4 are removed from the allowed VSAN list of ISLs connecting to switch 1 the operational allowed list of VSANs for each ISL would be as follows e The ISL between switch 1 and switch 2 includes VSAN 1 and VSAN 3 e The ISL between switch 2 and switch 3 includes VSAN 1 and VSAN 2 e The ISL between switch 3 and switch 1 includes VSAN 1 2 and 5 Consequently VSAN 2 can only be routed from switch through switch 3 to switch 2 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter5 Configuring Trunking Guidelines and Limitations W Figure 5 4 Operational and Allowed VSAN Configuration Switch 2 Switch 1 VSAN1 VSAN2 VSANS3 VSAN4 VSAN5 lt n gt Z A 79946 Guidelines and Limitations Trunking has the following configuration guidelines and limitations e General Guidelines and Limitations page 5 7 e Upgrade and Downgrade Limitations page 5 8 e Difference Between TE Ports and TF TNP Ports page 5 8 e Trunking Misconfiguration Examples page 5 10 General Guidelines and Limitations The trunking feature has the following general configuration guidelines and limitations e F ports support trunking in Fx mode e The trunk allowed VSANs configured for TE TF and TNP links are used by the trunking protocol to determine the allowed active VSANs in which frames can be received or transmitted e Ifa trunking enabled E port is
272. shold RX Performance Delta 60 80 10 TX Performance Delta 60 80 10 Turning Off the Monitoring of Specific Counter When the no monitor counter command is used in the config port group monitor mode it turns off the monitoring of the specified counter in the given policy as seen in the following example switch config show port group monitor PGMON_policy Policy Name PGMON_policy Admin status Not Active Oper status Not Active Port type All Port Groups Counter Threshold Interval ge Rising Threshold ge Falling Threshold RX Performance Delta 26 450 250 TX Performance Delta 60 100 80 switch config port group monitor name PGMON_policy switch config port group monitor no monitor counter rx performance switch config show port group monitor PGMON_policy Policy Name PGMON_policy Admin status Not Active Oper status Not Active Cisco MDS 9000 Family NX OS Interfaces Configuration Guide e236 i OL 29284 01 Release 6 x Chapter 2 Configuring Interfaces Configuring Interfaces W Port type All Port Groups Counter Threshold Interval ge Rising Threshold ge Falling Threshold RX Performance Delta 26 450 250 TX Performance Delta 60 100 80 Activating a Port Group Monitor Policy Step 1 Step 2 To activate a port group monitor policy follow these steps Command Purpose switch config t Enters configuration mode switch config port group monitor activate policyname
273. show port channel database port channel 77 Administrative channel mode is on Operational channel mode is on Last membership update succeeded 2 ports in total 0 ports up Ports fcipl down fcip2 down port channel 78 Administrative channel mode is on Operational channel mode is on Last membership update succeeded 2 ports in total 0 ports up Ports fc2 1 down f c2 5 down port channel 79 Administrative channel mode is on Operational channel mode is on Last membership update succeeded First operational port is fcip200 2 ports in total 2 ports up Ports fcip101 up fcip200 up Example 6 3 Displays the PortChannel Configured in the ACTIVE Mode switch show port channel database port channel 77 Administrative channel mode is active Operational channel mode is active Last membership update succeeded 2 ports in total 0 ports up Ports fcipl down Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x EN Chapter6 Configuring PortChannels HZ Verifying PortChannel Configuration fcip2 down port channel 78 Administrative channel mode is active Operational channel mode is active Last membership update succeeded 2 ports in total 0 ports up Ports fc2 1 down c2 5 down port channel 79 Administrative channel mode is active Operational channel mode is active Last membership update succeeded First operational port is fcip200 2 ports in total 2 ports up Ports fcip101 up
274. splays the PortChannel configured in the default ON mode and ACTIVE mode show port channel consistency Displays the consistency status without details show port channel consistency detail Displays the consistency status with details show port channel usage Displays the PortChannel usage show port channel compatibility parameters Displays the PortChannel compatibility Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter6 Configuring PortChannels Verifying PortChannel Configuration Hi Command Purpose show interface fc slot port Displays autocreated PortChannels show port channel database interface Displays the specified PortChannel interface port channel number For detailed information about the fields in the output from these commands refer to the Cisco MDS NX OS Command Reference You can view specific information about existing PortChannels at any time from EXEC mode The following show commands provide further details on existing PortChannels You can force all screen output to go to a printer or save it to a file See Examples 6 1 to 6 6 Example 6 1 Displays the PortChannel Summary switch show port channel summary Interface Total Ports Oper Ports First Oper Port port channel 77 2 0 port channel 78 2 0 oe port channel 79 2 2 fcip200 Example 6 2 Displays the PortChannel Configured in the Default ON Mode switch
275. ss the same physical link This feature is referred to as trunking in the Cisco MDS 9000 Family switches see Chapter 5 Configuring Trunking TE ports support class 2 class 3 and class F service In trunking F port TF port mode an interface functions as a trunking expansion port It may be connected to another trunked N port TN port or trunked NP port TNP port to create a link between a core switch and an NPV switch or an HBA to carry tagged frames TF ports are specific to Cisco MDS 9000 Family switches They expand the functionality of F ports to support VSAN trunking In TF port mode all frames are transmitted in EISL frame format which contains VSAN information Interconnected switches use the VSAN ID to multiplex traffic from one or more VSANSs across the same physical link This feature is referred to as trunking in the Cisco MDS 9000 Family see Chapter 5 Configuring Trunking TF ports support class 2 class 3 and class F service Cisco MDS 9000 Family NX OS Interfaces Configuration Guide P24 E OL 29284 01 Release 6 x Chapter2 Configuring Interfaces TNP Port SD Port ST Port Information About Interfaces W In trunking NP port TNP port mode an interface functions as a trunking expansion port It may be connected to a trunked F port TF port to create a link to a core NPIV switch from an NPV switch to carry tagged frames In SPAN destination port SD port mode an interface functions as a
276. st one other 10 Gbps capable interface must be in the in service state switch config if no switchport speed Reverts the factory default auto administrative speed of the interface Configuring Port Speed Group Detailed Steps To configure the port speed group of the interface follow these steps Command Purpose Step1 switch config t Enters configuration mode Step2 switch config interface fc 1 1 Selects the fc interface and enters interface configuration mode Step3 9 switch config if speed group 10g Configures the port speed group to 10 Gbps Note The preferred way of changing the speed group is the 10g speed mode command switch config if no speed group 10g Unsets the port speed group and reverts to the factory default auto administrative speed group of the interface Configuring the Interface Description The interface description can be any alphanumeric string Detailed Steps To configure a description for an interface follow these steps Command Purpose Step1 switch config t Enters configuration mode Step2 switch config interface fc1 1 Selects a Fibre Channel interface and switch config if enters interface configuration submode Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter2 Configuring Interfaces Step 3 Configuring Interfaces Command Purpose switch config if switchpor
277. stributed e BB credit buffers for ISL connections can be configured from a minimum of 2 buffers to a maximum of 500 buffers for dedicated rate mode e BB credit buffers for Fx port mode connections can be configured The minimum is 2 buffers and the maximum of 500 buffers for dedicated rate mode or 32 buffers for shared rate mode e Performance buffers are not supported on this module Each port group on the 24 port 8 Gbps Fibre Channel switching module consists of three ports The ports in shared rate mode in a port group can have a maximum bandwidth oversubscription of 10 1 considering that each port group has 12 8 Gbps bandwidth The following example configurations are supported by the 24 port 8 Gbps Fibre Channel switching modules e Three ports with shared rate mode and 8 Gbps speed 2 1 oversubscription default e One port with dedicated rate mode and 8 Gbps speed plus two ports with shared rate mode and 8 Gbps speed 4 1 oversubscription e One port with dedicated rate mode and 8 Gbps speed plus one port with dedicated rate mode and 4 Gbps speed plus one port with shared rate mode and 8 Gbps speed 10 1 oversubscription e Two ports with dedicated rate mode and 4 Gbps speed plus one port with shared rate mode and 8 Gbps speed 2 1 oversubscription Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Pas W OL 29284 01 Release 6 x Chapter4 Configuring Interface Buffers e Three ports with dedicated rate m
278. supported 14 supported Shared Dedicated yes yes 1 32 32 1 500 32 2 500 250 d no no yes yes yes You can use the show interface transceiver command to display small form factor pluggable SFP diagnostic information for Generation 2 switching modules switch show interface transceiver fc12 12 sfp is present name is CISCO FINISAR part number is FTRJ 8519 7D2CS1 revision is A serial number is H11TVQB fc transmitter type is short wave lase fc transmitter supports intermediate d media type is multi mode 62 5m M6 Supported speed is 200 MBytes sec Nominal bit rate is 2100 MBits sec Link length supported for 50 125mm fib Link length supported for 62 5 125mm f cisco extended id is unknown 0x0 no tx fault rx loss no sync exists SFP Diagnostics Information Temperature 24 33 Celsius Voltage 3 33 Volt Current 0 04 mA Optical Tx Power N A dBm Optical Rx Power N A dBm Note high alarm high warning r w o OFC SN istance link length er is 300 m s iber is 150 m s Diag mon type 104 low alarm low warning Configuration Examples for Fibre Channel Interfaces This section describes example configurations and includes the following sections Configuration Example for 48 Port 8 Gbps Module Interfaces page 3 46 Configuration Example for 24 Port 8 Gbps Module Interfaces page 3 46 Configuration Example for 4 44 Port 8 Gbps Module Interfaces page 3 47 Configuration Example for 48 Port 4 Gbps Module Interf
279. switched port analyzer SPAN The SPAN feature is specific to switches in the Cisco MDS 9000 Family It monitors network traffic that passes though a Fibre Channel interface This monitoring is done using a standard Fibre Channel analyzer or a similar switch probe that is attached to an SD port SD ports do not receive frames they only transmit a copy of the source traffic The SPAN feature is nonintrusive and does not affect switching of network traffic for any SPAN source ports see the Cisco MDS 9000 Family NX OS System Management Configuration Guide In the SPAN tunnel port ST port mode an interface functions as an entry point port in the source switch for the RSPAN Fibre Channel tunnel The ST port mode and the remote SPAN RSPAN feature are specific to switches in the Cisco MDS 9000 Family When configured in ST port mode the interface cannot be attached to any device and thus cannot be used for normal Fibre Channel traffic see the Cisco MDS 9000 Family NX OS System Management Configuration Guide Note Fx Port B Port Auto Mode ST port mode is not supported on the Cisco MDS 9124 Fabric Switch the Cisco Fabric Switch for HP c Class BladeSystem and the Cisco Fabric Switch for IBM BladeCenter Interfaces configured as Fx ports can operate in either F port or FL port mode The Fx port mode is determined during interface initialization depending on the attached N port or NL port This administrative configuration disallow
280. t NPV command and save the output so that support can use it to troubleshoot if necessary To display a list of the NPV devices that are logged in along with VSANs source information pWWNs and FCIDs enter the show npv flogi table command Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter7 Configuring N Port Virtualization Verifying NPV Configuration W switch show npv flogi table SERVER EXTERNAL INTERFACE VSAN FCID PORT NAME NODE NAME INTERFACE fc1 19 1 Oxee0008 10 00 00 00 c9 60 e4 9a 20 00 00 00 c9 60 e4 9a fc1 9 fc1 19 1 Oxee0009 20 00 00 00 0a 00 00 01 20 00 00 00 c9 60 e4 9a fc1 1 fc1 19 1 Oxee000a 20 00 00 00 0a 00 00 02 20 00 00 00 c9 60 e4 9a fc1 9 fc1 19 1 Oxee000b 33 33 33 33 33 33 33 33 20 00 00 00 c9 60 e4 9a fc1 1 Total number of flogi 4 To display the status of the different servers and external interfaces enter the show npv status command switch show npv status npiv is enabled External Interfaces Interface fc1 1 VSAN 2 FCID 0x1c0000 State Up Interface fc1 2 VSAN 3 FCID 0x040000 State Up Number of External Interfaces 2 Server Interfaces Interface fc1 7 VSAN 2 NPIV No State Up Interface fc1 8 VSAN 3 NPIV No State Up Number of Server Interfaces 2 Verifying NPV Traffic Management To display the NPV traffic map enter the show npv traffic map command switch show npv traffic map NP
281. t Tracking Detailed Steps Step 1 Step 2 The port tracking feature is disabled by default in all switches in the Cisco 9000 Family When you enable this feature port tracking is globally enabled for the entire switch To configure port tracking enable the port tracking feature and configure the linked port s for the tracked port To enable port tracking follow these steps Command Purpose switch config t Enters configuration mode switch config feature port track Enables port tracking switch config no feature Removes the currently applied port tracking configuration port track and disables port tracking Information About Configuring Linked Ports You can link ports using one of two methods Operationally binding the linked port s to the tracked port default Continuing to keep the linked port down forcefully even if the tracked port has recovered from the link failure I OL 29284 01 Cisco MDS 9000 Family NX OS Quality of Service Configuration Guide Chapter9 Configuring Port Tracking HI Configuring Port Tracking Binding a Tracked Port Operationally When you configure the first tracked port operational binding is automatically in effect When you use this method you have the option to monitor multiple ports or monitor ports in one VSAN Detailed Steps To operationally bind a tracked port follow these steps Command Purpose Step1 switch config t Enters confi
282. t description cisco HBA2 Configures the description of the interface The string can be up to 80 characters long switch config if no switchport description Clears the description of the interface Specifying a Port Owner wy Using the port owner feature you can specify the owner of a port and the purpose for which a port is used so that the other administrators are informed Note Detailed Steps Step 1 Step 2 Step 3 The port guard and port owner features are available for all ports regardless of the operational mode To specify or remove the port owner follow these steps Command Purpose switch config t Enters configuration mode switch config interface fc1 1 Selects the port interface switch config switchport owner description Specifies the owner of the switch port The description can include name of the owner and the purpose for which the port is used The description can be up to 80 characters long switch config no switchport owner Removes default the port owner description To display the owner description specified for a port use the following commands switch show running interface fc module number Interface number switch show port internal info interface fc module number tnterface number Configuring Beacon Mode Detailed Steps Step 1 Step 2 By default the beacon mode is disabled on all switches The beacon mode is indicated by a flashing
283. t half of the ports the second half of the ports or for all of the ports e Rate Mode dedicated e Buffer Credits default value When you configure port mode to auto or E and rate mode to dedicated for all ports in the global buffer pool you need to reconfigure buffer credits on one or more ports other than default Cisco MDS 9000 Family NX OS Interfaces Configuration Guide M420 i OL 29284 01 Release 6 x Chapter4 Configuring Interface Buffers Configuring Interface Buffers W wy Note If you use distance extension buffer to buffer credit spoofing on ISLs between switches the BB_SCN parameter on both sides of the ISL needs to be disabled Receive Data Field Size You can also configure the receive data field size for Fibre Channel interfaces If the default data field size is 2112 bytes the frame length will be 2148 bytes Configuring Interface Buffers This section includes the following topics e Configuring Buffer to Buffer Credits page 4 21 e Configuring Performance Buffers page 4 22 e Configuring Extended BB_credits page 4 23 e Enabling Buffer to Buffer Credit Recovery page 4 24 e Enabling the Buffer to Buffer State Change Number page 4 24 e Configuring Receive Data Field Size page 4 24 Configuring Buffer to Buffer Credits Detailed Steps To configure BB_credits for a Fibre Channel interface follow these steps Command Purpose Step1 switch config t Enters configuration mode switch
284. t in multiples of 10 and the interval can be 1 second to 1 hour The default is 10 percent in 1 second and generates a syslog To configure average credit nonavailable duration threshold and action refer to the Port Monitor section on page 2 15 amp Note This feature is not supported on 1 RU fabric switches Verifying Interfaces Configuration This section includes the following topics e Displaying Interface Information page 2 42 e Displaying TL Port Information page 2 50 e Displaying the ALPA Cache Contents page 2 51 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x EN Chapter2 Configuring Interfaces W Verifying Interfaces Configuration Displaying Port Monitor Status and Policies page 2 51 Displaying Port Group Monitor Status and Policies page 2 53 Displaying Management Interface Configuration page 2 55 Displaying VSAN Interface Information page 2 55 Displaying Interface Information The show interface command is invoked from the EXEC mode and displays the interface configurations Without any arguments this command displays the information for all the configured interfaces in the switch See Examples 2 3 to 2 10 Example 2 3 Displays All Interfaces switch show interface fc1 1 is up Port WWN is 20 0b 00 05 30 00 8d de Admin port mode is F Port mode is F FCID is 0x610000 Port vsan is 2 Speed is 2 Gbps Transmit B2B Credit is 3 Receive B2B
285. tChannels Hs Configuration Examples for F and TF PortChannels 1 ports in total 1 ports up Ports fcl 1 up Example 6 10 Displays the PortChannel Summary switch show port channel summary port channel port channel port channel port channel 1 2 3 port channel 4 5 6 port channel Configuration Examples for F and TF PortChannels This example shows how to configure F PortChannel in shared mode and bring up the link not supported on the MDS 91x4 switches between F ports on the NPIV core switches and NP ports on the NPV switches Step 1 Enable the F port trunking and channeling protocol on the MDS core switch swi tch Step2 Enable swi Step3 Create the PortC swi swi swi swi swi swi tch tch tch tch tch tch tch con NPIV on con con contr conf con conf conf Step4 Configure t swi swi swi swi swi swi swi swi swi tch tch tch tch tch tch tch tch tch con con con con con con con con con fig feature fport channel trunk the MDS core switch fig feature npiv hannel on the MDS core switch fig interface port channel 1 Fig TELS Figs if switchport mode F channel mode active switchport trunk mode off switchport rate mode shared exit he PortChannel member interfaces on the core switch fig interface fc2 1 3 fig if fig i fFig i fFig i fig i fFig i fFig i Fh FH FH FH FEF Fh Fh
286. ted when modules are installed If you save the running configuration to the startup configuration using the copy running config startup config command during reboot the switch powers up the same set of modules as before the reboot regardless of the sequence in which the modules initialize You can use the show port index allocation startup command to display the index allocation the switch uses at startup switch show port index allocation startup Startup module index distribution Slot Allowed Alloted indices info range Total Index values Re E Ea EE enaj ee ee an eee 34 0 31 80 81 2 Wl cease 32 32 63 a ee 16 64 79 Slot 1 shares 80 81 fi Sse 48 96 127 224 239 SUP 253 255 3 253 255 Note The output of the show port index allocation startup command does not display anything in the Allowed range column because the command extracts the indices from the persistent storage service PSS and displaying an allowed range for startup indices is meaningless If a module fails to power up you can use the show module slot recovery steps command to display the reason For information on recovering a module powered down because port indexes are not available refer to the Cisco MDS 9000 Family Troubleshooting Guide Tip Whenever using mixed Generation 1 and Generation 2 modules power up the Generation 1 modules first During a reboot of the entire switch the Generation 1 modules power up first
287. terface Figure 6 7 provides examples of valid PortChannel configurations Figure 6 7 Channel Group 10 Cisco MDS SwitchA 1 Channel Group 20 Valid PortChannel Configurations Channel Group 10 MM 2 1 1 1 1 i A 1 Cisco MDS 3 Switch B Cisco MDS Switch A A ld i2 ir Cisco MDS AN 3Switch B Channel Group 1 i i 4 i K Channel Group 20 120480 Channel Group 2 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter6 Configuring PortChannels HZ Default Settings Figure 6 8 provides examples of invalid configurations Assuming that the links are brought up in the 1 2 3 4 sequence links 3 and 4 will be operationally down as the fabric is misconfigured Figure 6 8 Misconfigured Configurations Channel Group 10 Channel Group 20 Channel Group 10 Channel Group 20 1 1 A 1 2 2 2 Cisco MDS Cisco MDS Cisco MDS Cisco MDS Switch A Switch B Switch A Switch B 3 3 3 X X Channel Group 10 Channel Group 20 1 1 Cisco MDS 2 b Switch B Cisco MDS Switch A 3 X 3 Cisco MDS 4 4 Switch C 7 Default Settings Table 6 3 lists the default settings for PortChannels Table 6 3 Default PortChannel Parameters Parameters De
288. terface fc4 10 Example 2 14 displays the running configuration after two interfaces are individually configured for mode FL Example 2 14 Displays the Running Configuration After Two Interfaces Are Individually Configured for Mode FL switch show running config version 3 1 3 system default switchport mode F interface fc4 1 switchport mode FL interface fc4 2 interface fc4 3 switchport mode FL interface fc4 4 interface fc4 5 interface fc4 6 interface fc4 7 interface fc4 8 interface fc4 9 interface fc4 1 Example 2 15 displays interface information in a brief format after the system default switchport mode F command is executed Example 2 16 displays interface information in a brief format after two interfaces are individually configured for mode FL I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces W Verifying Interfaces Configuration Example 2 15 Displays Interface Information in a Brief Format After the System Default Switchport Mode F Command is Executed switch show interface brief Interface Vsan Admin Admin Status SFP Oper Oper Port Mode Trunk Mode Speed Channel Mode Gbps fc4 1 T F notConnected swl fc4 2 1 F notConnected swl fc4 3 1 F notConnected swl a c4 4 1 F notConnected swl f c4 5 1 F sfpAbsent fc4 6 1 F sfpAbsent fc4 7 1 F sfpAbsent
289. th the capability and administrative parameters in the local switch If the compatibility check is successful the interfaces are operational and the corresponding compatibility parameter settings apply to these interfaces Cisco MDS 9000 Family NX OS Interfaces Configuration Guide lt OL 29284 01 Release 6 x Chapter6 Configuring PortChannels Information About PortChannels W Suspended and Isolated States If the operational parameters are incompatible the compatibility check fails and the interface is placed in a suspended or isolated state based on the configured mode e An interface enters the suspended state if the interface is configured in the ON mode e An interface enters the isolated state if the interface is configured in the ACTIVE mode Forcing an Interface Addition You can force the port configuration to be overwritten by the PortChannel In this case the interface is added to a PortChannel e Ifyou use the default ON mode to avoid inconsistent states across switches and to maintain consistency across switches then the ports shut down You must explicitly enable those ports again e Ifyou use the ACTIVE mode then the PortChannel ports automatically recover from the addition Note When PortChannels are created from within an interface the force option cannot be used After the members are forcefully added regardless of the mode ACTIVE and ON used the ports at either end are gracefully
290. the bandwidth allocation for ports in a port group Two port rate modes are supported e Dedicated Rate Mode A port is allocated required fabric bandwidth to sustain line traffic at the maximum operating speed configured on the port For more information see the Dedicated Rate Mode section on page 3 7 e Shared Rate Mode Multiple ports in a port group share data paths to the switch fabric and share bandwidth For more information see the Shared Rate Mode section on page 3 8 In Generation 1 modules you cannot configure the port rate modes The mode is determined implicitly based on the port mode and line card type Note Port rate modes are not supported on the Cisco Fabric Switch for HP c Class Blade System and the Cisco Fabric Switch for IBM Blade Center I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces Information About Fibre Channel Interfaces Table 3 3 shows the modules that support dedicated shared and the default rate modes Table 3 3 Port Rate Mode Support on Generation 2 Generation 3 and Generation 4 Modules and Switches Supports Supports Default Speed Product Name Dedicated Rate Shared Rate Mode and Rate Part Number Description Mode Mode Mode on All Ports Generation 4 Modules DS X9248 256K9 48 port 8 Gbps Advanced _ Yes Yes Auto Shared Fi
291. the same link However subsequent exchanges can use a different link This provides more granular load balancing while preserving the order of frames for each exchange Figure 6 4 illustrates how source ID 1 SID1 and destination ID1 DID1 based load balancing works When the first frame in a flow is received on an interface for forwarding link 1 is selected Each subsequent frame in that flow is sent over the same link No frame in SID1 and DID1 utilizes link 2 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide lt OL 29284 01 Release 6 x Chapter6 Configuring PortChannels Figure 6 4 SID1 and DID1 Based Load Balancing Frame gt Frame Frame SID1 DID1 Exchange 1 Frame Frame o Frame Frame KE SID1 DID1 Exchange 2 Frame Frame ry Frame ry Frame eoe SID2 DID2 Exchange 1 Frame Information About PortChannels W Link 1 Link 2 Link 1 Link 2 Link 1 Link 2 79530 Figure 6 5 illustrates how exchange based load balancing works When the first frame in an exchange is received for forwarding on an interface link 1 is chosen by a hash algorithm All remaining frames in that particular exchange are sent on the same link For exchange 1 no frame uses link 2 For the next exchange link 2 is chosen by the hash algorithm Now all frames in exchange 2 use link 2 I OL 29284 01 Release 6 x Cisco M
292. tion after the port is in the up state sw7 show interface fc1 13 fc1 13 is trunking Not all VSANs UP on the trunk Hardware is Fibre Channel SFP is short wave laser w o OFC SN Port WWN is 20 0d 00 0d ec 6d 2b 40 Admin port mode is FX trunk mode is on snmp link state traps are enabled Port mode is TF Port vsan is 1 Speed is 4 Gbps Rate mode is shared Transmit B2B Credit is 16 Receive B2B Credit is 32 Receive data field Size is 2112 Beacon is turned off Trunk vsans admin allowed and active Trunk vsans up Trunk vsans isolated Trunk vsans initializing 1 100 101 1101 1163 1166 1216 2172 2182 2183 1 1101 1163 1166 1216 2172 2182 This example shows the TF port information when a server logs in on noninternal FLOGI VSAN VSAN 2183 is moved to the up state when the server logs in to VSAN 2183 w7 show interface fc1 13 fc1 13 is trunking Not all VSANs UP on the trunk Hardware is Fibre Channel SFP is short wave laser w o OFC SN Port WWN is 20 0d 00 0d ec 6d 2b 40 Admin port mode is FX trunk mode is on snmp link state traps are enabled Port mode is TF Port vsan is 1 Speed is 4 Gbps Rate mode is shared Transmit B2B Credit is 16 I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter5 Configuring Trunking HZ Guidelines and Limitations Receive B2B Credit is 32 Receive data field Size is 2112 Beacon is turned off Trunk
293. to Cisco MDS 9000 Family switches and have similar properties as FL ports TL ports enable communication between a private loop device and one of the following devices e A device attached to any switch on the fabric e A device on a public loop anywhere in the fabric e A device on a different private loop anywhere in the fabric e A device on the same private loop TL ports support class 2 and class 3 services Private loop devices refer to legacy devices that reside on arbitrated loops These devices are not aware of a switch fabric because they only communicate with devices on the same physical loop see the TL Port ALPA Caches section on page 2 14 Tip S We recommend configuring devices attached to TL ports in zones that have up to 64 zone members Note TL port mode is not supported on Generation 2 switching module interfaces In trunking E port TE port mode an interface functions as a trunking expansion port It may be connected to another TE port to create an extended ISL EISL between two switches TE ports are specific to Cisco MDS 9000 Family switches They expand the functionality of E ports to support the following e VSAN trunking e Transport quality of service QoS parameters e Fibre Channel trace fctrace feature In TE port mode all frames are transmitted in EISL frame format which contains VSAN information Interconnected switches use the VSAN ID to multiplex traffic from one or more VSANs acro
294. to device alias Does not require device alias for virtual pWWN Does not allow automapping to the secondary device Allows automapping to the new HBA Mapping process is manual for NPIV FlexAttach Virtual pWWN CFS Distribution The FlexAttach virtual pWWN configuration is distributed for CFS through IPv4 and is enabled by default The FlexAttach virtual pWWN distribution by default is on CFS region 201 The CFS region 201 links only to the NPV enabled switches Other CFS features such as syslog is on region 0 Region 0 will be linked through IPv4 for all NPV switches on the same physical fabric If CFS has an option to link through IPv4 or ISL then CFS will select the ISL path Note NPV switches do not have ISL E or TE ports and are linked through IPv4 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter8 Configuring FlexAttach Virtual pWWN Guidelines and Limitations W Security Settings for FlexAttach Virtual pWWN Security settings for the FlexAttach virtual pWWN feature are done by port security at the NPV core Node WWN of the end device is used to provide physical security For more details on enabling port security refer to the Cisco MDS 9000 Family NX OS Security Configuration Guide Guidelines and Limitations Following are recommended guidelines and requirements when deploying FlexAttach virtual pWWN e FlexAttach configuration is supported
295. tracked port goes down Be aware that the linked port is automatically brought down when the tracked port goes down Default Settings Table 9 1 lists the default settings for port tracking parameters Table 9 1 Default Port Tracking Parameters Parameters Default Port tracking Disabled Operational binding Enabled along with port tracking Cisco MDS 9000 Family NX OS Quality of Service Configuration Guide OL 29284 01 Chapter9 Configuring Port Tracking Configuring Port Tracking W Configuring Port Tracking Port tracking has the following features The application brings the linked port down when the tracked port goes down When the tracked port recovers from the failure and comes back up again the tracked port is also brought up automatically unless otherwise configured You can forcefully continue to keep the linked port down even though the tracked port comes back up In this case you must explicitly bring the port up when required This section includes the following topics Enabling Port Tracking page 9 3 Information About Configuring Linked Ports page 9 3 Binding a Tracked Port Operationally page 9 4 Information About Tracking Multiple Ports page 9 4 Tracking Multiple Ports page 9 5 Information About Monitoring Ports in a VSAN page 9 5 Monitoring Ports in a VSAN page 9 5 Information AboutForceful Shutdown page 9 6 Forcefully Shutting Down a Tracked Port page 9 6 Enabling Por
296. trunk mode configuration on E ports has no effect The ISL is always in a trunking disabled state In the case of F ports if the third party core switch ACC s physical FLOGI with the EVFP bit is configured then EVFP protocol enables trunking on the link Trunk Allowed VSAN Lists and VF_IDs Each Fibre Channel interface has an associated trunk allowed VSAN list In TE port mode frames are transmitted and received in one or more VSANs specified in this list By default the VSAN range 1 through 4093 is included in the trunk allowed list The common set of VSANs that are configured and active in the switch are included in the trunk allowed VSAN list for an interface and they are called allowed active VSANs The trunking protocol uses the list of allowed active VSANs at the two ends of an ISL to determine the list of operational VSANs in which traffic is allowed Switch 1 see Figure 5 3 has VSANs 1 through 5 switch 2 has VSANs 1 through 3 and switch 3 has VSANSs 1 2 4 and 5 with a default configuration of trunk allowed VSANs All VSANs configured in all three switches are allowed active However only the common set of allowed active VSANs at the ends of the ISL become operational see Figure 5 3 For all F N and NP ports the default VF_ID is 1 when there is no VF_ID configured The trunk allowed VF_ID list on a port is same as the list of trunk allowed VSANs VF_ID 4094 is called the control VF_ID and it is used to define the l
297. ture e General Guidelines and Limitations page 6 13 e Generation 1 PortChannel Limitations page 6 14 e F and TF PortChannel Limitations page 6 14 e Valid and Invalid PortChannel Examples page 6 15 General Guidelines and Limitations Cisco MDS 9000 Family switches support the following number of PortChannels per switch e Switches with only Generation 1 switching modules do not support F and TF PortChannels e Switches with Generation 1 switching modules or a combination of Generation 1 and Generation 2 switching modules support a maximum of 128 PortChannels Only Generation 2 ports can be included in the PortChannels e Switches with only Generation 2 switching modules or Generation 2 and Generation 3 modules support a maximum of 256 PortChannels with 16 interfaces per PortChannel e A PortChannel number refers to the unique identifier for each channel group This number ranges from of 1 to 256 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x a cis Chapter6 Configuring PortChannels HI Guidelines and Limitations Generation 1 PortChannel Limitations This section includes the restrictions on creation and addition of PortChannel members to a PortChannel on Generation hardware The 32 port 2 Gbps or 1 Gbps switching module The MDS 9140 and 9120 switches When configuring the host optimized ports on Generation 1 hardware the following PortChannel guidelines apply If you exec
298. ues for Cisco supported SFPs Table 2 5 defines the acronyms used in the command output see the Displaying Interface Information section on page 2 42 Table 2 5 SFP Transmitter Acronym Definitions Definition Acronym Standard transmitters defined in the GBIC specifications short wave laser swl long wave laser lwl long wave laser cost reduced lwcr electrical elec Extended transmitters assigned to Cisco supported SFPs CWDM 1470 c1470 CWDM 1490 c1490 CWDM 1510 c1510 CWDM 1530 c1530 CWDM 1550 c1550 CWDM 1570 c1570 Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Ea OL 29284 01 Release 6 x Chapter2 Configuring Interfaces TL Ports Information About Interfaces W Table 2 5 SFP Transmitter Acronym Definitions continued Definition Acronym Standard transmitters defined in the GBIC specifications CWDM 1590 c1590 CWDM 1610 c1610 Private loop devices refer to legacy devices that reside on arbitrated loops These devices are not aware of a switch fabric because they only communicate with devices on the same physical loop The legacy devices are used in Fibre Channel networks and devices outside the loop may need to communicate with them The communication functionality is provided through TL ports See the Interface Modes section on page 2 2 TL port mode is not supported on the following hardware e Generation 2 switching modu
299. uffers Per Port Dedicated Rate Mode 4 Gbps Speed BB_Credit Buffer Allocation Type ISL Fx Port Total number of BB_credit buffers per module 5488 Total number of performance buffers per module 512 shared 1 ISL E port or TE port The following considerations apply to BB_credit buffers on 12 port 4 Gbps switching modules e BB credit buffers for ISL connections can be configured from a minimum of 2 buffers to a maximum of 250 buffers e BB credit buffers for Fx port mode connections can be configured from a minimum of 2 buffers to a maximum of 250 buffers e By default 512 performance buffers are preallocated and are shared by all the ports These buffers are configurable and the buffers are assigned to the port based on the availability of the buffers in the shared pool e There are 2488 extra buffers available as extended BB_credit buffers after allocating all the default BB_credit buffers for all the ports in ISL mode 5488 250 12 amp Note Extended BB_credits are allocated across all ports on the switch That is they are not allocated by port group By default the ports in the 12 port 4 Gbps switching modules come up in 4 Gbps dedicated rate mode but can be configured as 1 Gbps and 2 Gbps dedicated rate mode Shared mode is not supported Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Pais OL 29284 01 Release 6 x Chapter4 Configuring Interface Buffers Info
300. unking or non trunking depending on the trunking port mode switch conf t gt Enter configuration commands one per line End with CNTL Z switch config interface fc lt Slot No Port No gt switch config if switchport mode E switch config if end Ensure you follow this on both the switches between which you are attempting to bring up the ISL link Configuring 10 Gbps FC Mode The 48 port 8 Gbps Advanced Fibre Channel module DS X9248 256K9 and the 32 port 8 Gbps Advanced Fibre Channel module DS X9232 256K9 can switch between two speed modes the 1 2 4 8 Gbps or 10 Gbps By default the modules are online in the 1 2 4 and 8 Gbps modes when they are loaded for the first time There are two ways to change the ports to the 10 Gbps speed mode e Using the 10G speed mode command which is the recommended method e Using the generic speed configuration switchport speed command which has certain constraints The following conditions apply when the ports in the module can be configured to 10 Gbps speed mode e The ports in the module can be configured to 10 Gbps speed only when the DS 13SLT FAB3 module bandwidth is 256 G Any other combination of fabric modules will not let the ports come up in 10 Gbps e When in 10 Gbps mode the ports in the module that are not 10 Gbps capable are disabled and will be in out of service state e The ports function only in full rate mode They cannot be moved to shared rate mode
301. ups 4 2 to 4 15 configuring port speeds 3 29 configuring rate modes 3 30 default settings 3 25 description 3 1 to 3 3 disabling ACL adjacency sharing 3 43 displaying port resources 3 42 dynamic bandwidth management 3 10 4 17 4 19 3 44 3 11 extended BB_credits interface capabilities out of service interfaces port groups 3 3 port index allocations 3 19 port rate modes 3 5 recovering from powered down state releasing shared resources 3 42 taking interfaces out of service 3 41 Generation 3 switching modules default settings 3 25 3 21 indirect link failures recovering 9 1 interfaces 6 8 6 18 configuring descriptions 2 26 2 20 3 25 5 11 6 16 adding to PortChannels default settings deleting from PortChannels 6 9 es Configuration Guide displaying information 2 42 to 2 49 displaying SFP information 2 48 forced addition to PortChannels 6 9 6 19 isolated states 6 9 suspended states 6 9 IPv4 default gateways configuring mgmt0 interfaces 2 17 ISLs PortChannel links 6 2 L LEDs beacon mode states 2 11 speed 2 11 licenses extended BB_credits 4 17 4 18 link failures recovering 9 1 load balancing description 6 4 PortChannels 6 2 management interfaces configuring 2 17 2 37 default settings 2 20 3 25 5 11 6 16 displaying information 2 55 features 2 17 mgmt0 interfaces configuring 2 17 2 37 default settings 2 20 3 25 5 11 6 16 features 2 17 MPS 14 2 modules con
302. uring Port Monitor This section includes the following topics e Enabling Port Monitor page 2 31 e Configuring a Port Monitor Policy page 2 32 e Activating a Port Monitor Policy page 2 34 Enabling Port Monitor Detailed Steps To enable port monitor follow these steps Command Purpose Step1 switch config t Enters configuration mode Step2 switch config port monitor enable Enables default port monitoring switch config no port monitor enable Disables port monitoring Cisco MDS 9000 Family NX OS Interfaces Configuration Guide I OL 29284 01 Release 6 x Chapter2 Configuring Interfaces HZ Configuring Interfaces Configuring a Port Monitor Policy Detailed Steps Step 1 Step 2 Step 3 Step 4 Step5 Step 6 Step7 Step 8 Step 9 To configure a port monitor policy follow these steps Command Purpose switch config t Enters configuration mode switch config port monitor name policyname Specifies the policy name and enters the port monitoring policy configuration mode switch config no port monitor name policyname Removes the policy switch config t Enters configuration mode switch config port monitor port type access port Applies the policy to the access ports switch config port monitor port type trunks Applies the policy to the trunk ports switch config port monitor port type all Applies the
303. ute the write erase command on a 32 port switching module and then copy a saved configuration to the switch from a text file that contains the no system default switchport shutdown command you need to copy the text file to the switch again for the E ports to come up without manual configuration Any or all full line rate port s in the Cisco MDS 9100 Series can be included in a PortChannel The host optimized ports in the Cisco MDS 9100 Series are subject to the same PortChannel rules as 32 port switching modules only the first port of each group of 4 ports is included in a PortChannel You can configure only the first port in each 4 port group as an E port for example the first port in ports 1 4 the fifth port in ports 5 8 and so on If the first port in the group is configured as a PortChannel the other three ports in each group ports 2 4 6 8 and so on are not usable and remain in the shutdown state If any of the other three ports are configured in a no shutdown state you cannot configure the first port to be a PortChannel The other three ports continue to remain in a no shutdown state F and TF PortChannel Limitations The following guidelines and restrictions are applicable for F and TF PortChannels The ports must be in F mode Automatic creation is not supported The PortChannel interface must be in ACTIVE mode when multiple FCIP interfaces are grouped with WA ON mode is not supported Only ACTIVE ACTIV
304. uto mode e When the command is executed from the command line switch operation remains graceful No ports are flapped To set the administrative mode of Fibre Channel ports to mode F in the CLI follow these steps Command Purpose switch config t Enters configuration mode switch config system default Sets the administrative mode of Fibre Channel ports to switchport mode F mode F if applicable switch config no system default Sets the administrative mode of Fibre Channel ports to switchport mode F the default unless user configured Note For detailed information about the switch setup utility see the Cisco MDS 9000 Family NX OS Fundamentals Configuration Guide Example 2 1 shows the command in the setup utility and Example 2 2 shows the command from the command line Example 2 1 Setup Utility Configure default switchport mode F yes no n y Example 2 2 Command Line switch config system default switchport mode F I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces HZ Configuring Interfaces Configuring ISL between Two Switches amp Note Ensure the FC cable is connected between the ports and do a no shut on each port E port mode is used when the port functions as one end of an Inter Switch Link ISL setting When you set the port mode to E you restrict the port coming up as an E port tr
305. value I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter2 Configuring Interfaces W Configuring Interfaces Activating a Port Monitor Policy Detailed Steps Step 1 Step 2 Configuring a Detailed Steps Step 1 Step 2 Step 3 Step 4 To activate a port monitor policy follow these steps Command Purpose switch config t Enters configuration mode switch config port monitor activate policyname Activates the specified port monitor policy switch config port monitor activate Activates the default port monitor policy switch config no port monitor activate policyname Deactivates the specified port monitoring policy Port Monitor Port Guard To configure a port monitor policy follow these steps Command switch config t Purpose Enters configuration mode switch config port monitor name ypolicyname Specifies the policy name and enters the port monitoring policy configuration mode switch config no port monitor name policyname Removes the policy switch config port monitor counter link loss poll interval seconds delta rising threshold percentagei event event id falling threshold percentage event event id portguard flap Specifies the delta link loss poll interval in seconds the thresholds in percentage and the event IDs of events to be triggered It also specifies that
306. vsans admin allowed and active 1 100 101 1101 1163 1166 1216 2172 2 182 2183 Trunk vsans up 1 2183 Trunk vsans isolated Trunk vsans initializing 1101 1163 1166 1216 2172 2182 Trunking Misconfiguration Examples If you do not configure the VSANs correctly issues with the connection may occur For example if you merge the traffic in two VSANs both VSANs will be mismatched The trunking protocol validates the VSAN interfaces at both ends of a link to avoid merging VSANs see Figure 5 5 Figure 5 5 VSAN Mismatch Switch 1 Switch 2 Isolated VSAN mismatch The trunking protocol detects potential VSAN merging and isolates the ports involved see Figure 5 5 The trunking protocol cannot detect merging of VSANs when a third party switch is placed in between two Cisco MDS 9000 Family switches see Figure 5 6 Figure 5 6 Third Party Switch VSAN Mismatch Third party switches Switch 1 Switch 2 Switch 3 N N st to VSAN 2 and VSAN 3 are effectively merged with overlapping entries in the name server and the zone applications Cisco DCNM SAN helps detect such topologies Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Eu OL 29284 01 Release 6 x Chapter5 Configuring Trunking Default Settings W Default Settings Table 5 4 lists the default settings for trunking parameters Table 5 4 Default Trunk Configuration Parameters Parameters Default Switch
307. width Gbps Bandwidth Gbps Bandwidth Gbps Bandwidth Gbps dedicated dedicated dedicated dedicated shared shared shared shared shared shared shared shared shared shared shared shared shared shared shared shared shared shared shared E Cisco MDS 9000 Family NX OS Interfaces Configuration Guide OL 29284 01 Release 6 x Chapter3 Configuring Fibre Channel Interfaces wy Information About Fibre Channel Interfaces W c13 43 32 8 0 shared c13 44 32 8 0 shared c13 45 32 8 0 shared fc13 46 32 8 0 shared fc13 47 32 8 0 shared fc13 48 32 8 0 shared When disabling restrictions on oversubscription ratios all ports in shared mode on 48 port and 24 port 4 Gbps or any 8 Gbps Fibre Channel switching modules must be shut down When applying restrictions on oversubscription ratios you must take shared ports out of service Note When restrictions on oversubscription ratios are disabled the bandwidth allocation among the shared ports is proportionate to the configured speed If the configured speed is auto on Generation 2 modules then bandwidth is allocated assuming a speed of 4 Gbps For example if you have three shared ports configured at 1 2 and 4 Gbps then the allocated bandwidth ratio is 1 2 4 As of Cisco SAN OS Release 3 0 and NX OS Release 4 1 1 or when restrictions on oversubscription ratios are enabled the port bandwidths are allocated in equal proportions regardless of po
308. ximum operating speed configured for the port In this mode ports do not use local buffering and all receive buffers are allocated from a global buffer pool see the Buffer Pools section on page 4 2 Table 3 4 shows the bandwidth provided by the various port speed configurations on the 8 Gbps Advanced Fibre Channel switching modules Table 3 4 Bandwidth Reserved for the Port Speeds on Generation 4 Switching Modules Configured Speed Reserved Bandwidth Auto 8 Gbps 8 Gbps Auto with 4 Gbps maximum 4 Gbps 4 Gbps I OL 29284 01 Release 6 x Cisco MDS 9000 Family NX OS Interfaces Configuration Guide Chapter 3 Configuring Fibre Channel Interfaces HI information About Fibre Channel Interfaces Table 3 4 Bandwidth Reserved for the Port Speeds on Generation 4 Switching Modules Configured Speed Reserved Bandwidth Auto with 2 Gbps maximum 2 Gbps 2 Gbps 1 Gbps 1 Gbps Table 3 5 shows the bandwidth provided by the various port speed configurations on the 8 Gbps Fibre Channel switching modules Table 3 5 Bandwidth Reserved for the Port Speeds on Generation 3 Switching Modules Configured Speed Reserved Bandwidth Auto 8 Gbps 8 Gbps Auto with 4 Gbps maximum 4 Gbps 4 Gbps Auto with 2 Gbps maximum 2 Gbps 2 Gbps 1 Gbps 1 Gbps Table 3 6 shows the amount of bandwidth reserved for a configured port speed on 4 Gbps switching modules Table

Cisco Systems DSC9148D8G48PK9 User's Manual

Contents

Download Pdf Manuals

Related Search

Related Contents