Home

Planning and Implementing an IBM SAN - ps

image

Contents

1. Input Signal Driver Skew Cable Skew Receiver Skew Data1 __ Driver pp CReceiver 1 __ Data2 _ _ _ _ Driver Receiver Data3 __ Driver a sCReceiver _ Data4 __ _ Driver lt sCReceiver Clock __ Driver pp CReceiver _ __ gt gt gt lt gt lt A A A A Data valid window reduces Figure 4 SCSI propagation delay results in skew Arrival occurs during a small window of time depending on the transmission speed and the physical length of the SCSI bus The need to minimize the skew limits the distance that devices can be positioned away from the initiating server to between 2 meters to 25 meters depending on the cycle time Faster speed means shorter distance The distances refer to the maximum length of the SCSI bus including all attached devices The SCSI distance limitations are shown in Figure 5 These distance limitations may severely restrict the total GB capacity of the disk storage which can be attached to an individual server Planning and Implementing an IBM SAN SCSI Distance limitations 1 2 Host Systems 15 Devices per connection Termination veo lt r Device _ Device 4 i 20 MB sec q
2. Parallel Transfer Serial Transfer Skewed bit arrival Streamed bit arrival m o gt oO ba b 3 o Fo 3 0 0 0 0 0 0 0 0 D a Data packet or frame a Impervious to cycle time and distance L o Data Valid Window affected by cycle time and distance Figure 10 Parallel data transfers versus serial data transfers Chapter 1 Introduction to Storage Area Networks 15 16 Serial transfer enables simpler cabling and connectors and also routing of information through switched networks Today Fibre Channel can operate over distances of up to 10 km link distances up to 90 km by implementing cascading and longer with the introduction of repeaters Just as LANs can be interlinked in WANs by using high speed gateways so can campus SANs be interlinked to build enterprise wide SANs Whatever the topology information is sent between two nodes which are the source transmitter or initiator and destination receiver or target A node is a device such as a server personal computer workstation or mainframe or peripheral device such as disk or tape drive or video camera Frames of information are passed between nodes and the structure of the frame is defined by a protocol Logically a source and target node must utilize the same protocol but each node may support several different protocols or data types Therefore Fibre Channel archi
3. Up to 25 meters e Fast Wide SCSI 1 2 Host Systems 15 Devices per connection Termination 1a TJ i Device gt Device gt iw 40 MB sec lt Upto 12 meters Ultra Wide SCSI 1 2 Host Systems 15 Devices per connection Termination mam i Device 2 Devicsli i 80 MB sec lt Up to 12 meters SS SS Ultra2 SCSI Figure 5 SCSI bus distance limitations Many applications require the system to access several devices or for several systems to share a single device SCSI can enable this by attaching multiple servers or devices to the same bus This is known as a multi drop configuration A multi drop configuration is shown in Figure 6 Multi Drop Data Control Bus SCSI Initiator Data Bus Receiver Terminator Control Signals Driver Receiver Driver Receiver Driver Receiver Driver Receiver Disk Drive Disk Drive Disk Drive Disk Drive Figure 6 Multi drop bus structure Chapter 1 Introduction to Storage Area Networks 7 To avoid signal interference and therefore possible data corruption all unused ports on a parallel SCSI bus must be properly terminated Incorrect termination can result in transaction errors or failures Normally only a single server can access data on a specific disk by means of a SCSI bus In a shared bus environmen
4. Scalability and fault tolerance Mission Critical Enterprise Directors Clustered Servers Business Critical FC AL Loops SAN FC Switches Costs Figure 15 IBM s hierarchy of Fibre Channel SAN offerings It is worth bearing in mind that new products features and functions are regularly announced For the latest information on the products described in this book and for details of new solutions refer to IBM s SAN Web site http www storage ibm com ibmsan The following section summarizes the major reasons for selecting a device type 2 3 1 1 When will you use a Router or a Data Gateway Routers and Gateways act as bridges between different protocols You would select them to allow you to provide investment protection for IBM or non IBM storage devices which use SCSI or SSA protocols and provide attachment to the Fibre Channel SAN The IBM SAN Data Gateway Router IBM 2108 R03 is a low cost solution supporting attachment between a Fibre Channel attached host and a SCSI tape library such as a 3575 Magstar MP Chapter 2 SAN planning and design considerations 35 36 The Vicom Fibre Channel SLIC Router Model FC SL 7139 model 111 enables all IBM 7133 7131 and 3527 SSA Disk Systems to attach to host systems using fibre channel host adapters and drivers The IBM SAN Data Gateway 2108 G07 provides protocol conversion for connection of SCSI and Ultra SCSI storage devices to Fibre Channe
5. 220 7 4 5 Adding an AIX zone to the existing zone set 231 7 5 Cascading with McDATA building a multiswitch fabric 237 7 5 1 Multiswitch fabric considerations 000 05 238 7 5 2 Setting up our McDATA multiswitch fabric 242 Chapter 8 Implementing the SAN Data Gateway 251 8 1 SAN Data Gateway 0 es 251 8 2 Installation Sonra aa Gu ahd to te oe ed Seley ie AG 252 8 2 1 Setting the Ethernet address 00 00000 254 8 2 2 Using Telnet on the SAN Data Gateway 255 8 2 3 Startup Sequence eue os reaa cee ee 255 8 3 StorWatch SAN Data Gateway Specialist 256 8 3 1 Installing StorWatch Specialist 00 257 8 3 2 Using the StorWatch SAN Data Gateway Specialist 259 8 4 SCSIIDeVICES v5 a Hh VAs eee de hye ciel S 269 6 4 41 LON SUPPO a2 ky Stuart tela eed itn ae ee Pec alee eas 269 8 4 2 Device discovery 0 0 cc ee ees 270 8 4 3 Assigning LUN IDS yia a eama naa a a a aaa eee 270 8 4 4 Adding SCSI devices 0 2c eee ee 270 8 5 IBM Storage Area Network Data Gateway access options 271 B D ZONIN Guay srereetectod Guay ayafety avedieny airertives dat sisi a a eta 272 vi Planning and Implementing an IBM SAN 8 5 2 Virtual Private SAN 0 000 00000000000 ee ee 272 8 5 3 Combining Zoning and Virtual Private SAN 283 8 6 Adding Fibre Ch
6. LUN N A OK Cancel Figure 287 Mapping a general spare The Original Mapping box will indicate that no LUN was assigned previously Click on the Get New Mapping button and the next sequential available LUN number will appear in the New Mapping box Click OK The drive will now appear in the Physical Drive window of the Control Center This can also be used to modify the LUN number on an existing mapped drive as well as remove the LUN number to unmap a drive and create a general spare It is not necessary to map a general spare A general spare can be used to creating a composite mirror or copy drive Mapping a general spare will create a drive that has a LUN number that can then be used by the host 9 8 3 Removing a mirror containing composite drive 336 The mirror in this case was made from logical drives on their own Once the mirror is removed the composite drives that made up the mirror will return to the Composite Drive window as viewed from the Control Center However since each composite drive had its attributes changed as it became a member of the mirror it will no longer be mapped The composite drives will show up aS UnMapped in the Control Center window This is shown in Figure 288 The mirror created in 9 7 2 Creating the mirror on page 331 was removed Planning and Implementing an IBM SAN T33282 ITSOm T33025 ITSOl 733027 ITSO3 TOOOOO 35E68229 Figure
7. 0 ce es 324 9 6 1 Creating an Instant Copy drive 00 000 0 324 9 6 2 Instant copy drive properties 00 002 326 9 6 3 Adding an Instant Copy Drive to a mirror 327 9 6 4 Detach Instant Copy Drive from a mirror 329 9 7 Combining composite and Mirroring sssaaa aaa aaaea 329 9 7 1 Creating a second composite drive 00 330 9 7 2 Creating the mirror 2 0 ce es 331 9 7 3 Viewing mirror drive using composite drives 332 9 8 Reusing logical drives 0 0 es 334 9 8 1 Remove a logical drive 2 20 ee ee 334 9 8 2 Mapping a general Spare 0 0 cee eee eee 335 9 8 3 Removing a mirror containing composite drive 336 9 9 Expanding the SLIC system 2 0 0 0 eee 337 9 91 1 Adding diskid renens aa Opie De PS wad eadaa ed Fee ae 338 9 9 2 Adding Routers 0 eee eet ees 338 vii g JI GAGGING NOSTS iia seca cod alta eine ck een Ce Peace eile dill Cece tie tala 341 Appendix A Special notices 0 2 cee 343 Appendix B Related publications 0 0 005 347 B 1 IBM Redbooks 0 ccc eee eee eens 347 B 2 IBM Redbooks collections 0 000000 cee ee eee 347 B 3 Other fESOUlCES e465 cc a nae Als aug oda suet gels NE ere Aaa 348 B 4 Referenced Web sites 0 0 0 000 eee 349 How to get IBM Redbooks 00 cee eee ees 351
8. Fibre Channel Host 10000000 c921da0a Host Name unknown Host Type unknown Host Connection unknown Done downloading SAN Data Gateway 9 113 24 110 data Figure 224 Fibre Channel host data As you select and highlight the different ports or devices there are different options available from the top toolbar If an SCSI channel is highlighted select Controls from the toolbar You will notice that all options are grayed out except for SCSI Channel Once selected a dialog box will appear as shown in Figure 225 and display the settings for the SCSI channel Max LUNs per ID 32 Host ID k F Re scan SCSI Bus Channel Mode Initiator Target Identify SCSI Channel MaxIDs perbus 16 _oK Cancel Max Speed Ultra 20Mhz Advanced Options Figure 225 SCSI channel parameters Chapter 8 Implementing the SAN Data Gateway 267 268 Selecting the Advanced Options button displays a dialog box as shown in Figure 226 These settings are not typically changed and may disrupt normal operations Refer to a service representative before changing any Advanced Options PA Advanced SCSI Channel Options x v Bus Reset On Power Up 1 v Alternate Initiator ID Termination Enabled Reset OK Cancel Figure 226 Advanced SCSI parameters Fibre Channel parameters are displayed in a similar fashion Highlight a Fibre Channel port and select Controls from the toolbar you will notice that now all option
9. Welcome to IBM StorWatch Introduction a Enterprise Storage ja Server Specialist Status Soares Machine Type 2105 Machine Model F20 F Serial Number IBM FCA13 Communications WNN 5005076300C00C0D Storage Allocation Copyright IBM Corp 1998 2000 Licensed Materials Property of IBM All Rights Reserved IBM is a registered trademark of the IBM Corp U S Government Users Restricted Rights Use duplication or disclosure restricted Users 4 by GSA ADP Schedule Contract with IBM Corp View README Licensed Internal Code Copy Services har EDE Document Done SESE Figure 25 Introduction panel of the native FC ESS s Z 64 Planning and Implementing an IBM SAN Here we can see the machine type model and serial number displayed Also we see the identification of the Fibre Channel ESS in the Fibre Channel fabric in the form of its WWNN To the left we see the main button panel from where we start to configure the options for the ESS The Introduction button will load the Introduction panel 4 1 1 2 Logon to the ESS Using one of the buttons to the left we are forced to login to the site with the login window shown in Figure 26 The default user is storwatch and the password is specialist Both are case sensitive and must be entered in lower case Username and Password Required x Enter username for Enterprid Storage authorization at tpkal3c1 User Name ee Password Cancel
10. Perform Configuration Update Cancel Configuration Update Figure 47 Modify Host Systems panel In the Host Attributes entry fields we identify the host Fibre Channel ports that will access the ESS The first entry is a nickname to identify the port in the ESS Also we have to choose the host type which is a PC server the type of the host attachment which is Fibre Channel and the WWPN of the host adapter port to identify the host adapter If you plan to use the StorWatch Enterprise Storage Server Expert later on we recommend that you also enter the host IP address field This is used to show which Fibre Channel host adapters are in a specific IP host By pressing the Add button we add the specified host Fibre Channel port of a Fibre Channel host to the Host Systems List as shown in Figure 48 Chapter 4 Configuring the ESS with native Fibre Channel 81 Modify Host Systems Host Atiributes Nickname Pey roa Host Type rc Server Win NT 4 0 or higher gt Host Attachment Fibre Charmel attached Hosiname IP Address pmen o World Wide Port Name 200000EISB00525D iesea tom list of novm WWPHs me Perform Configuration Update Cancel Configuration Update Figure 48 Adding a FC host adapter port If we wanted to we can define a second host FC port belonging to the same system or we can define ports for other systems Adding ports to the list puts together a script in the background which is executed by pressi
11. 4 1 7 1 Selecting the interface port In this panel we select the port of the ESS interface adapters to be configured depending on the type of the adapter There are different configuration options for SCSI and Fibre Channel For example if we select a Chapter 4 Configuring the ESS with native Fibre Channel 97 SCSI port we can configure the port for SCSI as shown in Figure 74 However we will focus on the FC part By selecting an FC port we are presented with the Configure Host Adapter Ports panel as shown in Figure 75 Configure Host Adapter Ports a Host Adapter Port By Adapter 2 Pott A FC Sto Server Atiribuies Access Restricted Fibre Channel Access Mode FC Port Attributes Fixe Channel Point to Point Fibre Channel Topology Perform Configuration Update Cancel Configuration Update Figure 75 Configuring ESS interface ports FC The port we used is in bay 1 adapter 2 and is port A It is called A even it is the only port on the adapter 4 1 7 2 Storage Server Attributes field As shown in Figure 76 the Storage Server Attributes field also known as the logical unit number LUN access mode which specifies the channel access mode can only be changed by an IBM service support representative SSR 98 Planning and Implementing an IBM SAN Storage Server Atiribuies Fibre Channel Access Mode Access_Any Fire Channel Point to Point Fibre Channel Topology FC Port Atir
12. Figure 26 Login window 4 1 1 3 Status Panel Using the Status button we get to the Status Graphical View panel as shown in Figure 27 Chapter 4 Configuring the ESS with native Fibre Channel 65 Refresh Data Problem Log Figure 27 Status panel graphical view This panel allows us to quickly check for any errors in the ESS Here we see a problem detected in the storage arrays with access to data not affected To get a detailed view on the problems encountered we use the Problem Log button and get the Status Problem Log panel as shown in Figure 28 Problem Log 66 Planning and Implementing an IBM SAN Refresh Data Graphical View a Cancel the selected problem Figure 28 Status panel problem log 4 1 1 4 The Problem Notification panel Using the Problem Notification button we get the Problem Notification panel shown in Figure 29 Problem Notification Chapter 4 Configuring the ESS with native Fibre Channel 67 Problem Notification Email and Pager Notification p1 201 4408777 SEN fo Abbemmese est SNMP Notification MIB Read Access Disabled SNMP Trap Destination Modi Ena Noa SNMP Figure 29 Problem notification panel From within this panel we can configure the notification options provided with the ESS Also we can configure the Simple Network Management Protocol SNMP 4 1 1 5 The Communications panel Using the Communications button we get the Communications panel as sh
13. Ronda Hruby IBM Storage Subsystems Division Scott Jensen Brocade Chris Morton IBM Storage Subsystems Division Dietmar Kurpanek Vicom Systems Inc Paul Radu Pathlight Technology Inc Tammy Sokol IBM Storage Subsystems Division Karen Ward IBM Storage Subsystems Division John Young IBM Storage Subsystems Division Norm Weinberg Vicom Systems Inc xxi Sven Eichelbaum IBM Storage Subsystems Division Comments welcome Your comments are important to us We want our Redbooks to be as helpful as possible Please send us your comments about this or other Redbooks in one of the following ways e Fax the evaluation form found in IBM Redbooks review on page 371 to the fax number shown on the form e Use the online evaluation form found at ibm com redbooks e Send your comments in an Internet note to redbook us ibm com xx Planning and Implementing an IBM SAN Chapter 1 Introduction to Storage Area Networks Everyone working in the Information Technology industry is familiar with the continuous developments in technology which constantly deliver improvements in performance capacity size functionality and so on A few of these developments have far reaching implications because they enable applications or functions which allow us fundamentally to rethink the way we do things and go about our everyday business The advent of Storage Area Networks SANs is one such development SANs can lead to a
14. Status Ready Figure 251 McDATA Director connection to a Gateway In the right hand view pane under Port Type you see that the port is in N_Port mode to denote a fabric connection However in the left hand view pane there are no devices listed under the Fibre Channel port 4 description As hosts begin to attach to the McDATA Director and are restarted they will begin to login their own information to the Gateway s VPS database The McDATA is seen as completely transparent to the SAN Data Gateway 8 7 High availability considerations Connections from the host to the SAN Data Gateway and from the Gateway to the ESS or other SCSI devices have been discussed until now with single host adapters or utilizing single SCSI connections Special considerations must be taken into account in deciding to add host adapters or providing redundant SCSI connections 8 7 1 Single host with multiple Fibre Channel adapters A host can have several adapters installed If each adapter is assigned to different LUNs using the SAN Data Gateway there are no contention issues As each adapter is assigned specific LUNs it cannot see or access any other LUNs that may be present Chapter 8 Implementing the SAN Data Gateway 295 However there may be situations where the adapters are used to perform load balancing and failover much like the SCSI connections from a host to the ESS that are in place today As the host in the SCSI situation re
15. This IBM Redbook is a follow on from the redbook Designing an IBM Storage Area Network SG24 5758 In that book we introduced Fibre Channel basics described the technical topology of a Storage Area Network SAN and detailed Fibre Channel products and IBM SAN initiatives We also designed configurations that were able to maximize the benefits of Fibre Channel products that are currently supported by IBM and that are available in the marketplace today Where this IBM Redbook picks up the story is how to implement those products that are in the IBM product armory today It is not possible to duplicate each and every SAN installation that is feasible or practical What we want to achieve is a consolidated reference guide that details how the basic products can be swiftly and in some cases easily implemented We will show the various features that each of these products contributes and how the most common and important benefits of each are taken advantage of We will show how they can be employed in some of the more commonly encountered environments and platforms With this in mind we have two objectives within this redbook The first is to show practical decisions to be considered when planning a SAN the second objective is to show how the following products can be installed configured and tailored e IBM Enterprise Storage Server with native Fibre Channel e IBM Fibre Channel Storage Hub e IBM SAN Fibre Channel Managed Hub e IBM SAN Fibr
16. To create a new zone set select Configure gt Zone Sets This displays the zone set library and provides us with options for changing our zone definitions for example creating new zone sets deleting zone sets or modifying existing zone sets as shown in Figure 183 Chapter 7 Implementing the McDATA ED 5000 221 gt McDATA Fabric Manager d View j Configure Zoning View Default Zone Enabled Zone Set Libra ctive Zone Set Replace World Wide Name Find Zone Find World Wide Name Save active zone set as Deactivate zone set Figure 183 Actions for Zone Sets 222 Because there are no zone sets in the zone library we will create one Starting from the Actions button in the Zone Sets window we select New We are presented with a window to define a zone set From this window we will assign a zone set name The Actions button provides us with the different tasks to maintain the zones in this zone set as shown in Figure 184 Because there is no zone defined in the zone set we can only assign a new zone to the zone set Planning and Implementing an IBM SAN Figure 184 Defining a new zone set Now a window opens where we can define the members of the zone as shown in Figure 185 We want to define a zone that includes our Windows NT machines and one ESS The ESS is already connected to the switch and the hosts are currently detach
17. 131 5 7 4 QuickLoop feature 0 0 ee ee 131 5 7 5 IBM 3534 managed hub zoning 0 00 sees 137 Chapter 6 Implementing an IBM SAN Fibre Channel Switch 139 6 1 IBM SAN Fibre Channel switch 000 000 eee 140 6 1 1 IBM 2109 S08 hardware components 0 140 6 1 2 IBM 2109 software features 0 0 00 00 eee 142 6 2 The ITSO environment e r EKERI EEKE es 143 6 2 1 Installing the 2109 S08 switch 0 0005 143 6 2 2 Setting the IP address using the serial port 2109 S08 only 144 6 2 3 Setting the IP address on an IBM 2109 S16 146 6 2 4 Attaching the switch to a network server and a storage device 150 6 2 5 SAN installation verification 00 ee eee 151 6 3 Zoning in an IBM SAN Fibre Channel Switch environment 154 6 3 1 The role of zoning ina SAN 0 0 00 155 6 3 2 Zoning components 0 0 0 es 156 6 3 3 Methods of implementing zones 0 00005 157 6 4 Implementing zoning aaaea 161 6 4 1 Adding a second host to the ESS 161 6 4 2 Setting up zones in your SAN fabric 4 162 6 5 Cascading IBM 2109 switches 20000020 ee eeee 171 6 5 1 Cascading overview 0 es 172 6 5 2 Cascading examples 0 0 00 cee eee ees 172 6 5 3 Cascading using IBM switches 0 00005 174 6 5 4 Implementing cascading with
18. 3 Install and configure the Ethernet hub We will not be installing the optional Ethernet hub This is described in the ED 5000 Enterprise Fibre Channel Director Installation Manual should you want to implement this 4 Unpack and inspect the ED 5000 Note Verify that your ED 5000 is not damaged and that no items are missing For a complete list of items that are shipped refer to the ED 5000 Enterprise Fibre Channel Director Installation Manual Planning and Implementing an IBM SAN 5 Position the ED 5000 and route power cables This consists of placing the ED 5000 in the planned area with enough ventilation and then connecting the power and the network Note You may also have to adjust the leveling pads 6 Switch power on to the ED 5000 and verify operation After the hardware installation we power on the ED 5000 After power on the ED 5000 runs a couple of Power On Self Tests POST At the successful completion of the tests the power LED is illuminated and the operator panel shows that the ED 5000 is online 7 Set IP address Subnetmask and Gateway address To configure the ED 5000 from within the EFC Manager the switch itself needs to be configured for the network For operations like this we will use the operator panel at the front of the switch We can display and change the status of the switch and the current network information there Since the switch is initially set up to work on the dedicated Ethernet segme
19. Channel that reserves the full Fibre Channel bandwidth for a dedicated Class 1 connection but also allows connection less Class 2 traffic to share the link if the bandwidth is available I O Input output IP Internet Protocol IPI Intelligent Peripheral Interface Isochronous Transmission Data transmission which supports network wide timing requirements A typical application for isochronous transmission is a broadcast environment which needs information to be delivered at a predictable time JBOD Just a bunch of disks Jukebox A device that holds multiple optical disks and one or more disk drives and can swap disks in and out of the drive as needed L_Port Loop Port A node or fabric port capable of performing Arbitrated Loop functions and protocols NL Ports and FL_Ports are loop capable ports LAN See Local Area Network A network covering a relatively small geographic area usually not larger than a floor or small building Transmissions within a Local Area Network are mostly digital carrying data among stations at rates usually above one megabit s Latency A measurement of the time it takes to send a frame between two locations Link A connection between two Fibre Channel ports consisting of a transmit fibre and a receive fibre Link_Conitrol_Facility A termination card that handles the logical and physical control of the Fibre Channel link for each mode of use LIP A Loop Initialization Primitive sequence is a
20. IBM Redbooks fax order form 0 0 0 0 ccc ee 352 GIOSS ALY ei eet ah Re OR Ae a a PR ORS 353 ING OX sra Seta ee a eine eee eae ated etek ia eee A 363 IBM Redbooks review 0 00000 cee ee 371 viii Planning and Implementing an IBM SAN Figures 1 Typical distributed systems or client server infrastructure 2 2 Inefficient use of available disk capacity attached to individual servers 3 3 Distributed computing model tends to create islands of information 4 4 SCSI propagation delay results in skew 20000 0 ee eee 6 5 SCSI bus distance limitations 0 00 ee eee 7 6 Multi drop bus structure 2 eee 7 7 Network attached storage utilizing the network in front of the servers 9 8 Storage Area Network the network behind the servers 11 9 FICON enhances ESCON 0 000 cee eae 12 10 Parallel data transfers versus serial data transfers 15 11 Consolidated storage efficiently shared capacity 18 12 Logical consolidation of dispersed disk subsystems 19 13 LAN backup restore today loading the IP network 21 14 SAN solutions match e business strategic needs 25 15 IBM s hierarchy of Fibre Channel SAN offerings 35 16 Valid and invalid inter switch links 00 000 c eee ee eee 39 17 A fault tolerant fabric design 0
21. R Modify Host Systems Configure Host Adapter Poris Configure Disk Groups Modify Volume Assignment Figure 51 Newly configured Fibre Channel host adapter port no volume assigned At this point we can work with other configurations for instance configuring the options for the ESS Fibre Channel ports we want to use or assigning volumes to the newly defined host Fibre Channel port To configure disk groups and volumes from the beginning even in a partially configured ESS we first want to configure disk groups 4 1 5 Configuring disk groups Pressing the Configure Disk Groups button takes us to the Fixed Block Storage panel as shown in Figure 52 Modify Host Systems Configure Host Adapter Poris Configure Disk Groups Modify Volune Assignment 84 Planning and Implementing an IBM SAN Fixed Block Storage ia aan sete ial clas a ae a i Block FB Fonmatted 2 62 GB ima i Block FB Fomnatted 52 62 GB mow RAD g Perform Configuration Update Cancel Configuration Update Figure 52 Fixed Block Storage panel The Available Storage table lists all of the available physical storage in the ESS with their location storage type track format and capacity Also modifications made prior to performing an update are indicated The storage type can be RAID Array or non RAID Non RAID means the disks are going to be configured as just a bunch of disks JBOD We are going to configure the disk groups connected to
22. an optical fiber that is designed for the transmission of a single ray or mode of light as a carrier Itis a single light path used for long distance signal transmission See also Multi Mode Fiber SMART Self Monitoring and Reporting Technology SM Single Mode See Single Mode Fiber SMF Single Mode Fiber In optical fiber technology an optical fiber that is designed for the transmission of a single ray or mode of light as a carrier It is a single light path used for long distance signal transmission See also MMF SNIA Storage Networking Industry Association A non profit organization comprised of more than 77 companies and individuals in the storage industry SN Storage Network See also SAN SNMP Simple Network Management Protocol The Internet network management protocol which provides a means to monitor and set network configuration and run time parameters SNMWG Storage Network Management Working Group is chartered to identify define and support open standards needed to address the increased management requirements imposed by storage area network environments SSA Serial Storage Architecture A high speed serial loop based interface developed as a high speed point to point connection for peripherals particularly high speed storage arrays RAID and CD ROM storage by IBM Star The physical configuration used with hubs in which each user is connected by communications links radiating out of a central hub that h
23. and then detach from the mirror The drive can then be used to back up the data or used elsewhere if desired 9 6 1 Creating an Instant Copy drive To create an Instant Copy drive select Tools gt Instant Copy Drive Setup Wizard You will see a dialog box Instant Copy Drive List that will be blank If there were copy drives created it would display the names of the drives Click on the Next gt gt button and the Instant Copy Drive Members Selection window is displayed The window on the left named Available Drive Pool contains a list off all drives that are candidates to become a copy drive Select a drive by highlighting it and click on the Add gt gt button The drive name will be added to the Member window An asterisk will appear on the left hand side 324 Planning and Implementing an IBM SAN of the drive just selected in the Available Drive window to denote that the drive has been selected An example is shown in Figure 276 Instant Copy Drive Setup Wizard Instant Copy Drive Member Selection TOOOOG 35E68462 T0000 LOO 733025 L001 Figure 276 Instant Copy Drive Member Selection To remove a drive from the Member window select the desired drive and click on the Remove lt lt button Below each window there is a Drive Capacity box As a drive is selected its capacity in megabytes is shown Click on the Next gt button to continue to the Assigning Instant Copy Drive Properties window as shown in Fig
24. from a Windows NT platform To review the commands that are available for UNIX platforms and installation instructions on other operating systems please refer to the SLIC Manager Installation and User Guide 310 605807 Installing the SLIC Manager software The SLIC Manager software can run on many operating systems The following discussion will describe an installation on a Netfinity 5500 with Windows NT 4 0 To install the SLIC Manager server software for local access the Vicom Utilities CD ROM is placed in the CD drive Select Start gt Run and Browse the CD drive Go to slicmgr i386 server setup exe and click OK Follow the prompts displayed on the window to install the Manager software This will install the daemon service also For remote or client access the daemon service is not required To load the Manager software only go to slicmgr i386 client setup exe instead Planning and Implementing an IBM SAN 9 2 2 Communicating to the Router For the SLIC Manager server software to communicate to the Router it requires space on a disk or several disks that are within the SSA loop This is referred to as a SLIC Zone To create space on a disk a file or partition depending on the operating system used is created for the Manager software to use To create this SLIC Zone a configuration file must be created or edited 9 2 2 1 Editing the configuration file When the software is loaded a sample configuration file called
25. from the hosts We only describe what we have to configure for use with FC AL and FC SW environments For further information on the configuration of the Fibre Channel ESS refer to the ESS documentation as described in 4 2 Related information on page 109 In the following sections we cover these topics e 4 1 1 Introduction to the ESS Specialist on page 64 e 4 1 2 Viewing the Storage Allocation of the ESS on page 75 e 4 1 3 Accessing the Open System Storage panel on page 79 e 4 1 4 Defining a new host FC port with its WWPN on page 80 e 4 1 5 Configuring disk groups on page 84 e 4 1 6 Defining volumes for Fibre Channel host adapter ports on page 88 e 4 1 7 Configuring an ESS Fibre Channel port on page 96 e 4 1 8 Modifying volume assignments and sharing volumes on page 104 Copyright IBM Corp 2000 63 4 1 1 Introduction to the ESS Specialist The ESS Specialist is a Web and Java based interface to configure the ESS To connect to it we use the Netscape Navigator and use the hostname or IP address of the ESS in the Uniform Resource Locator URL field 4 1 1 1 Introduction panel This takes us to the introduction panel of the ESS Specialist as shown in Figure 25 Seascape Web User Interface Netscape File Edit View Go Communicator Help EEEE ut Bookmarks 4 Location D StorWatch Enterprise Storage Server Specialist Solutions gt z
26. gt SCSI Channel 1 gt SCSI Channel 2 lt 3 SCSI Channel 3 lt SCSI Channel 4 T Fibre Channel 1 T Fibre Channel 4 Fibre Channel Host 10000000 c9221802 Host Name plymouth Host Type AIX Host Connection idevfscsi0 Oxet 0 Emulex Current SAN Data Gateway ITSO IP Address 9 113 24 110 Channel Description Fibre Channel 4 Up Time 00 41 51 Figure 232 Specialist after VPS enabled and host registration If no registration software is loaded on the host or is not available for a specific operating system for example a Fibre Channel switch only the WWN of the attached system will register to the VPS database all other fields will have unknown This is shown in Figure 233 276 Planning and Implementing an IBM SAN ES otorWatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc File View Admin Tools Controls Help WH 9 113 24 110 ITSO SAN Data Gateway lt 3 SCSI Channel 1 lt 3 SCSI Channel 2 lt 3 SCSI Channel 3 lt SCSI Channel 4 T Fibre Channel 1 gt Fibre Channel 4 Current SAN Data Gateway ITSO IP Address 9 113 24 110 Channel Description Fibre Channel 1 Fibre Channel Host 10000000 c921da0a Host Name unknown Host Type unknown Hast Connection unknown Figure 233 Host system with no host registration software A diagram to depict the setup that was described is shown in Figure 234 Not
27. multiplexed into a stream of light transmitted on a single optical fiber WEBM Web Based Enterprise Management A consortium working on the development of a series of standards to enable active management and monitoring of network based elements Zoning In Fibre Channel environments the grouping together of multiple ports to form a virtual private storage network Ports that are members of a group or zone can communicate with each other but are isolated from ports in other zones Index Numerics 1 way mirror 319 2 Gbit speeds 50 2103 HO7 111 2108 36 2109 S08 143 2 way mirror 319 3527 299 3534 36 35341RU 111 3 way mirror 319 7131 299 7133 299 A access levels 72 access profile 99 Access_Any 99 Access_Restricted 99 active zone sets 218 Add Volumes 88 Adding an Instant Copy Drive to a Mirror 327 Adding Disk 338 Adding Hosts 341 Adding Routers 338 Adding SCSI devices 270 administrator 259 Advanced Options 268 Affordability 30 Agent 256 Aliases 166 American National Standards Institute 14 anonymous host 99 ANSI 14 arbitrated loop 14 153 parameters 177 Assign Fibre Channel Target 301 Assigned Volumes 80 Assigning LUN IDs 270 Attaching hosts 290 auditlogs 186 availability 28 B backbone 47 Copyright IBM Corp 2000 fabric 43 BI 3 bottlenecks 11 238 bridge solutions 34 bus 4 Business Intelligence 27 Cc callhome 69 campus 33 campus SAN 16 cascaded 37 237 fabric 38
28. name the switch description and the location in the main window This is useful to distinguish among a number of installed directors if installed To configure this information we select Configure gt Identification and we are presented with a dialog window with data entry fields as shown in Figure 170 Chapter 7 Implementing the McDATA ED 5000 209 i Configure Identification x Fibre Channel Director IBM SAN JOSE BLD 51 3rd FLOOR End User Contact please configure e es Figure 170 Setting the switch identification After activation the display of the main window changes and places the name of the switch in the title bar and the name description and location displayed in the window as shown in Figure 171 This information is used in various locations of the Product Manager to identify the selected director The name of the ED 5000 can be the same as the host name assigned to the DNS This prevents too many names for the same director W ED 5000 ED 1 X E Figure 171 Switch information By selecting the Hardware View we are presented with the front and the back of the selected director These are interactive views where you can query the status of the FRUs by clicking on them Once clicked more information of the FRU is displayed with indicators relating to its operational status For example clicking on one of the G_Port modules opens the Port Card View Now we can select a port by clicking on it an
29. therefore limiting capacity scalability Access to data is lost in the event of a failure of any of the SCSI connections to the disks This also applies in the event of reconfiguration or servicing of a disk device attached to the SCSI bus because all the devices in the string must be taken offline In today s environment when many applications need to be available continuously this downtime is unacceptable The data rate of the SCSI bus is determined by the number of bits transferred and the bus cycle time measured in megahertz MHz Decreasing the cycle time increases the transfer rate but due to limitations inherent in the bus architecture it may also reduce the distance over which the data can be successfully transferred The physical transport was originally a parallel cable comprising eight data lines to transmit eight bits in parallel plus control lines Later implementations widened the parallel data transfers to 16 bit paths SCSI Wide to achieve higher bandwidths Propagation delays in sending data in parallel along multiple lines lead to a well known phenomenon known as skew meaning that all bits may not arrive at the target device at the same time This is shown in Figure 4 Chapter 1 Introduction to Storage Area Networks 5 6 Signal Skew on Parallel Data Bus
30. will have to be refreshed by selecting View gt Refresh SAN Data Gateway This will cause the updated data to be shown on the Specialist window 274 Planning and Implementing an IBM SAN The registration service will re register the host to the SAN Data Gateway ata default of 15 minute intervals This interval can be changed if so desired Previous to enabling the VPS feature you will have seen the Specialist display a window similar to Figure 231 There is no key beside the Fibre Channel ports to indicate that no host is recognized PA storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc File vew Admin Tools Controls Help WH 9 113 24 246 fchridge1 SAN Data Gateway lt SCSI Channel lt SCSI Channel lt SCSI Channel lt SCSI Channel 4 Fibre Channel gt Fibre Channel M k UU ied Figure 231 Specialist display without VPS enabled After the VPS is enabled and a host has registered with the Gateway all its information will load automatically into the database of the Gateway Figure 232 shows in the right hand side view pane the host name host type HBA and connection information and the unique WWN that was obtained automatically by host registration Chapter 8 Implementing the SAN Data Gateway 275 Pa storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc File View Admin Tools Controls Help Q E 9 113 24 110 ITSO SAN Data Gateway
31. 1 of 2 panel with Not Allocated space selected 89 Add Volumes 2 of 2 panel 1 2 0 0 eee eee eee 90 Volume definition sesi iinoa sia neo iea n aea eee ee 91 Volume placement 0 0 00 eee eee ee 91 Performing the volume definitions 000020 eee eaee 92 Warning window 000 eee eet ee 92 Progress window for volume definition 220000005 93 Successful volume update 0 0 eee 93 Storage Allocation button 0 2 0 0 ce eee 93 Storage allocation with host FC port selected 44 94 Storage allocation with host FC port and ESS FC port selected 95 Host system port with associated volumes 2200005 96 Entry point for configuring ESS FC ports 000 eae 97 Configuring ESS interface ports SCSI 0 222000055 97 Configuring ESS interface ports FC 00000 eee eee 98 Storage Server Attributes field 00 000 0c eee eee 99 Anonymous host in access any mode 000 e eee eee eee 100 FC Port Attributes configured for Point to Point 101 FC Port Attributes configured for FC Arbitrated Loop 101 FC Port Attributes undefined 0 00 c ee eee eee 102 Progress window of changing the topology 000 102 Successful topology change cece eee 102 Open System Storage panel no shared volumes 103 Pla
32. 137 138 Planning and Implementing an IBM SAN Chapter 6 Implementing an IBM SAN Fibre Channel Switch A SAN is a dedicated high speed network of directly connected storage elements designed to move large amounts of data between host independent distributed devices It is a new infrastructure between servers and storage devices created to match the rapidly growing requirements of business solutions for example e business and mission critical applications In addition to ultra high performance and 24 x 7 availability the key requirements are improved control over storage resources and the maximum flexibility for the flow of information The SAN storage infrastructure is clearly an enabler for this flexible information flow As we look at the interconnect components in a SAN installation the Fibre Channel switch is one of its core elements In general they are used to implement a Fibre Channel fabric topology A fabric is an intelligent shared interconnected scheme of FC server and storage nodes A single switch can be used to build a basic fabric By interconnecting to other switches you can build much larger fabrics Interconnecting switches in this way is called cascading Cascading allows for higher availability and larger topologies Fibre Channel switches can be used in entry level enterprise heterogeneous implementations and also in the largest of enterprise environments Any Fibre Channel enabled device can be connected t
33. 2 In the Connection Description window type the name you want to use for your new session Select any icon from the icon menu shown and click OK The Connect to window appears In this window change the Connect using setting from the default to Direct to Com1 and click OK The COM1 Properties window appears as shown in Figure 114 on page 130 Set the following parameters in the Port Settings tab e 8 bit e No parity e One stop bit e 9600 baud e Flow Control None e Emulation Auto Detect 3 Click File gt Properties The Properties window appears Select the Settings tab set the Emulation field to Autodetect 4 After this start up the hub by inserting the power cord into electrical outlet and waiting for about two minutes for diagnostics to complete Make sure that the power cord is fully seated into the front of the unit and the green ready LED is on 5 Press Enter on your laptop Planning and Implementing an IBM SAN The hub responds with the prompt Admin gt 6 The HyperTerminal session is now running For each prompt type in the information as shown and press Enter at the end of each response 7 Admin gt ipAddrSet This is the command to set the IP address 8 Ethernet Subnetmask Current sub net mask will be shown or None new Subnetmask or press Enter This is the new Subnet mask from your system administrator or if none is required press Enter 9 Fibre Channel IP Address None press En
34. 2109 S16 One of the differences between the S08 and the S16 apart from the increase in port counts is the control panel that has been introduced There is no serial port on the S16 From the control panel it is possible to set the IP address The front panel of the IBM SAN Fibre Channel Switch 2109 S16 is shown in Figure 126 LE E sw sw u n s E a ie i M i 00 Figure 126 2109 816 The triangular buttons to the right of the picture just above port 12 represent the data entry buttons referred to in the following steps 1 Power up the switch Wait two minutes while the diagnostics complete The display panel will go blank after the diagnostics are complete 2 Press the button the switch will respond Configuration menu 146 Planning and Implementing an IBM SAN 3 Press the gt button the switch will respond Ethernet IP address 4 Press the gt button the switch will respond by Displaying the current Ethernet IP address in the form xxx xxx XXX XXX To change the current address to the preferred address do the following e Pressing the lt button will move the cursor the entry point from one field to the next If you go past a field continuing to press the lt button will cause the cursor to wrap around and return to the desired spot e Pressing the button or the v button will either increment or decrement the current field Holding the button down will make this happen rapidly The
35. 221 SCSI channel data Information pertaining to a particular disk device is shown in Figure 222 264 Planning and Implementing an IBM SAN PA Storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc File View Admin Tools Controls Help E 9 113 24 110 ITSO SAN Data Gateway Q amp SCSI Channel 1 TE Current SAN Data Gateway ITSO Disk Device 0 0 IP Address 9 113 24 110 B Disk Device 0 1 B Disk Device 1 0 o B Disk Device 1 1 Channel Description SCSI Channel 1 B Disk Device 2 0 B Disk Device 2 1 i I l l B Disk Device 3 0 Device Description Disk Device 5 0 iM Disk Device 3 1 Assigned LUN 11 J Disk Device 4 0 SCSI Channel 1 B Disk Device 4 1 Target ID 5 Device LUN 0 Disk Device 5 1 B Disk Device 6 0 vendor ID IBM B Disk Device 6 1 Product ID 2105E20 B Disk Device 8 0 Revision 0525 B Disk Device 8 1 Serial Number 124FC142 B Disk Device 9 0 Capacity 1 953 151 Blocks B Disk Device 9 1 A Block Size l 512 D Disk Device 10 0 Removable No B Disk Device 10 1 a B Disk Device 11 0 Width 16 B Disk Device 11 1 Speed Ultra 20MHz J Disk Device 1 2 0 Status Operational Up Time 4 days 23 32 33 Figure 222 Disk device data You can perform the same drill down on the Fibre Channel host as we did with the SCSI channel and disk devices Select one of the Fibre Channel port connections as shown in Figure 223 and its d
36. 288 UnMapped composite drives The existing composite drives ITSO1 and ITSO3 cannot be mapped or given a new LUN number at this point Each logical composite drive must be removed as well This will cause the composite drive to be removed and its member drives to become general spares Once they are general spares the drives can be used to recreate the composite drive or to create new logical drives 9 9 Expanding the SLIC system The SLIC storage system can be expanded to add more SSA disks or more SLIC Routers Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 337 Each SLIC storage system can support up to 64 SSA disks and have 16 Routers 9 9 1 Adding disk To add disk to an existing SLIC system is very easy because they are SSA disks and the rules for SSA disks apply here as well If there is a dummy drive in an existing SSA drawer then it can be replaced by a real drive If a new drawer has to be added the SSA cabling is changed to include this new drawer Because this is SSA this can be done on the fly As the SSA loop is broken the Router will still access all disks due to the structure of the SSA loop If possible we recommended that you stop host access and power down the loop In any case the rules regarding SSA disks and cabling must be adhered As disks are added to an existing loop the new disks will be recognized If all disks in the loop are used as single disks JBOD and have LUN nu
37. 7190 cfg is added in the c ibm7190 sdus directory This is a text file that can be viewed and edited by simple text editors such as Windows Wordpad Open up the 7190 cfg file and it will contain a sample of how the file should look Also note that on the left hand side the sign is entered in every line to mark it out as a comment This is shown in figure Figure 255 Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 305 2 7190 cfg WordPad Of x File Edit View Insert Format Help pela SIR al Hen S pu Service Utility Daemon Configuration File Syntax SLIC_name path SLIC_ Zone user logfile c ibm7190 cOevent log Quer yChange Interval 10 he Comment Lines are declared using Example co path J IBM7190 SFa K IBM7190 S5Fa QueryChangeInterval in seconds QueryChangeInterval 10 be Call Home Feature Support The Call Home Feature allows the user to be notified through the email when a user designated SRNs occur The designated email address the profile for the email program used and SRNs must be defined in this file Syntax system email userif femail addr com user2 email addr com profile email program name srn N srn number N 2nd srn number E For Help press F1 Figure 255 Sample configuration file 306 This file can now be edited to be used as
38. 80 15 12 2 LVD SCSI Sig Bus Max Max Max Ver nal Width DTR Num Cable sion Rate bits MB s Devic Lengt MHz es h m SCSI 5 8 5 7 6 1 SCSI 5 8 5 7 6 2 Wide 5 16 10 15 6 SCSI 2 360 Planning and Implementing an IBM SAN SCSI SCSI 3 SCSI 3 consists of a set of primary commands and additional specialized command sets to meet the needs of specific device types The SCSI 3 command sets are used not only for the SCSI 3 parallel interface but for additional parallel and serial protocols including Fibre Channel Serial Bus Protocol used with IEEE 1394 Firewire physical protocol and the Serial Storage Protocol SSP SCSI FCP The term used to refer to the ANSI Fibre Channel Protocol for SCSI document X3 269 199x that describes the FC 4 protocol mappings and the definition of how the SCSI protocol and command set are transported using a Fibre Channel interface Sequence A series of frames strung together in numbered order which can be transmitted over a Fibre Channel connection as a single operation See also Exchange SERDES Serializer Deserializer Server A computer which is dedicated to one task SES SCSI Enclosure Services ANSI SCSI 3 proposal that defines a command set for soliciting basic device status temperature fan speed power supply status etc from a storage enclosures Single Mode Fiber In optical fiber technology
39. Before you restart the SAN Data Gateway you must stop all input and output I O activity between the host and SCSI devices 1 SCSI devices Turn on the SCSI devices You must turn on all SCSI devices attached to the SAN Data Gateway before you initially turn on or restart the SAN Data Gateway 2 SAN Data Gateway The SAN Data Gateway scans the SCSI buses when it starts If you add or remove SCSI devices after the Gateway has started the Gateway will not detect the changes You can invoke an SCSI rescan or restart operation Chapter 8 Implementing the SAN Data Gateway 255 from either the StorWatch SAN Data Gateway Specialist client or the service terminal 3 Fibre Channel host Before you turn on or restart the hosts that are connected with Fibre Channel to the SAN Data Gateway you must wait until the SAN Data Gateway has finished starting You will know the Gateway has finished starting when the ready light on the front panel blinks at frequency intervals of one second e Some operating systems provide you with software methods that allow you to add or remove SCSI devices dynamically after the host has started To ensure reliable operation restart the host e If you update SAN Data Gateway firmware you must restart the Gateway to use the new firmware To ensure compatibility between the firmware features or functions and the host restart the host e If you update SCSI device firmware the SAN Data Gateway Explorer applicat
40. Channel Switch and IBM 3534 Managed Hub so they cannot yet be used together in a cascaded fabric This may be resolved in the future as standards are agreed and implemented The McDATA ED 5000 Director does support servers and devices attached using the IBM SAN Data Gateway and the IBM SAN Data Gateway Router Chapter 2 SAN planning and design considerations 37 2 3 2 Building a multiswitch fabric 38 A single switch or director is limited in the number of ports it can directly interconnect To increase connectivity in the fabric it is necessary to connect multiple switches or directors This is known as a cascaded fabric 2 3 2 1 Cascading Cascaded fabric is a cost effective reliable way to achieve very large port counts in the SAN It is also used as a means of delivering fault tolerant fabrics by eliminating single points of failure and to service applications requiring high availability Cascading also increases the maximum distance between interconnected devices Examples used in this chapter will be based on Inter Switch Links ISLs between multiple IBM 2109 Fibre Channel Switches the first cascadable Fibre Channel switch available on the market 2 3 2 2 Inter switch links When cascading IBM 2109 switches the ports used for ISL will automatically be designated as E Ports by the switch software E Ports reduce the number of ports available for device connection More switches can be added to the fabric non disruptively Multipl
41. Chapter 7 Implementing the McDATA ED 5000 197 McDATA EFC Management Download and Installation Netscape ile Edit View Go Communicator Help rE ee gt 3 3 2a sts F Back Foward Reload Home Search Netscape Print Security Stop Bookmarks 4 Location ANS McDATA Enterprise Fabric Connectivity Management This page allows you to download and install the McDATA EFC Manager remote client application on any of the supported platforms listed below Please follow the instructions for your particular platform to install the application Click the following button to begin the installation process You will receive a Java Security warning requesting permission to download and start the installer program and to read and write files on your system You must grant these permissions to Microsoft Windows 95 98 NT allow installation of the EFC Manager application Begin Windows Installation Sun Solaris requires specific patches to support the Java 2 platform used by the installer program and the EFC Manager application Please read the Solaris patch requirements prior to installing the EFC Manager application If the latest patches have not been applied the installer will not run Ifthe patches have been applied click the following button to begin the installation process You will receive a Java Sun Solaris SPARC Security warning requesting permission to download and start the installer program
42. Enterprise Storage Server Configuration Planner SC26 7353 e Provides work sheets for planning the logical configuration of the ESS This book is not available in hard copy This book is only available on the following Web site http www storage ibm com hardsoft products ess refinfo htm IBM Enterprise Storage Server Quick Configuration Guide SC26 7354 e Provides flow charts for using the StorWatch Enterprise Storage Server Specialist The flow charts provide a high level view of the tasks the IBM service support representative performs during initial logical configuration You can also use the flow charts for tasks that you might perform during modification of the logical configuration The hard copy of this booklet is a 9 inch by 4 inch fanfold Planning and Implementing an IBM SAN Chapter 5 Implementing an IBM managed and unmanaged hub In this chapter we show how an unmanaged hub and a managed hub can be implemented For the managed hub we also show how to use a feature called QuickLoop and QuickLoop zoning The products we describe are the e IBM Fibre Channel Storage Hub 2103 H07 e IBM SAN Fibre Channel Managed Hub 35341RU These are seen as entry level components of a SAN 5 1 ITSO environment This setup as shown in Figure 96 is mainly a low cost entry level solution for connecting one or more storage devices to one or more servers It is not a very scalable solution and should not be chosen if many devic
43. FCSCSI disk device hdisk4 Available 30 68 01 Other FCSCSI disk device hdisk5 Available 30 68 01 Other FCSCSI disk device When you use the 1sdev Cc disk grep 2105 command you see only display lines that contain the value immediately after it If you have not defined any 2105 devices none will be displayed Chapter 3 Implementing Fibre Channel host adapter cards 61 62 Planning and Implementing an IBM SAN Chapter 4 Configuring the ESS with native Fibre Channel One benefit of a SAN is to implement disk pooling To do this successfully we need an easy to manage storage server with a Fibre Channel attachment This is the IBM Enterprise Storage Server 2105 F20 with native Fibre Channel The prime focus of this redbook is directed towards the fabric rather than the devices connected to it We describe what we need to do to make the ESS part of our SAN The following hardware and software necessary for successful attachment should be installed before beginning to set up the SAN e Fibre Channel adapters installed in the ESS e Microcode level on the ESS 1 1 to support the Fibre Channel attachment e InfoServer must be available In the topics that follow we cover e 4 1 Defining the ESS on page 63 e 4 2 Related information on page 109 4 1 Defining the ESS As with the SCSI ESCON ESS we will use the StorWatch Enterprise Storage Server Specialist to configure the hosts ports and volumes to be accessed
44. IP Address 9 113 24 110 lt 3 SCSI Channel 3 SCSI Channel 4 Fibre Channel 1 Channel Description Fibre Channel 4 Z Fibre Channel 4 Fibre Channel Host 20fd0060 69201d4e Host Name unknown Host Type SWITCH Host Connection unknown Figure 246 Switch port information 8 6 1 4 Attaching hosts to the switch Any hosts that will attach to the switch should have the host registration software loaded and installed Refer to 8 5 2 2 Host registration on page 273 for details Plug in the Fibre Channel cable from their respective adapters and power on or restart the host The registration of the hosts will be completed through the switch Figure 247 shows a switch installed on port 4 and two hosts connected on the switch The host plymouth is now connected through the switch and an NT host was added 290 Planning and Implementing an IBM SAN lt gt Storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc Iof Xx File View Admin Tools Controls Help E 9 113 24 110 ITSO SAN Data Gateway lt SCSI Channel 1 Current SAN Data Gateway ITSO eg SCSI Channel 2 IP Address 9 113 24 110 gt SCSI Channel 3 lt S SCSI Channel 4 gt Fibre Channel 1 Fibre Channel 4 10000000 c9221802 plymouth AIX On Line Am 20fd0060 69201d4e unknown SWITCH OnLine gt Channel Description Fibre Channel 4 Fibre Channel Host 10000000 c920cae6 Host Name F
45. IP addr press Enter This would be a new address provided by the Network Administrator and in our example we used 9 113 24 19 6 Ethernet Subnetmask Current subnet mask or None new Subnetmask or press Enter This would be a new Subnetmask provided by the network administrator and we set ours to 255 255 255 0 7 Fibre Channel IP Address None press Enter 8 Fibre Channel Subnetmask None press Enter 9 Gateway Address Current Gateway address or None new Gateway address or press Enter This would be a new Gateway address provided by the network administrator 10 admin gt Logout press Enter This will end the Telnet session This completes setting the IP address Serial port settings In our test scenario we used an IBM laptop as a serial terminal emulator and linked it to the switch using a standard serial cable with two female 9 pin connectors The serial port settings are e 8 bit e No parity e One stop bit e 9600 baud e Flow Control None e Emulation Auto Detect It is important to change the Ethernet Subnetmask and the Gateway address You can see the entries on the front panel of the switch in Figure 125 Chapter 6 Implementing an IBM SAN Fibre Channel Switch 145 Lio 1 Lio 3 Lio a o Lio 2 fio 4 Lo Figure 125 2109 S08 switch Now the switch should be linked to the network and you should be able to access it from any server in the network 6 2 3 Setting the IP address on an IBM
46. Implementing an IBM SAN SAN components Can sufficient spare bandwidth be provided to continue to deliver acceptable performance 2 1 2 3 Scalability How much growth is expected Will the SAN you design be required to support additional applications If so in what time scale for instance within the next year or two years How fast is data growing and will you need to expand storage resources or add more servers Do you need to support legacy SCSI hardware What are the distances between server and storage resources and will this need to expand to include other departments and locations 2 1 2 4 Security How will you protect application data on the SAN from loss or corruption without losing access to the information Can you provide backup and recovery and disaster protection capabilities to meet your organizations policies for the data What failover facilities within the SAN design will be required to ensure continued accessibility in the event of errors or disasters If multiple servers are attached to the SAN can you ensure that each may only access the data or storage devices it is authorized to access This is particularly critical in a heterogeneous platform environment especially if Windows NT hosts are participating For instance Windows NT expects to see a SCSI bus attachment and it seeks to attach all the devices which are attached to the bus It does the same on a SAN so it is essential to provide security against this o
47. LAN free and server free data transfers largely eliminate the use of server cycles to manage housekeeping tasks such as backup and recovery and archive and recall The configuration of what might be termed thin servers therefore might be possible and this can result in significant hardware cost savings to offset against the costs of installing the SAN fabric 1 5 11 Storage resources match e business enterprise needs 24 By eliminating islands of information typical of the client server model of computing and introducing an integrated storage infrastructure SAN solutions match the strategic needs of today s e business This is shown in Figure 14 Planning and Implementing an IBM SAN Storage with a SAN Dynamic Storage Resource Management Automatic Data Management Windows 7 x 24 Connectivity E Server amp Storage NT 2000 Universal W Data Access Scalability UNIX G Flexibility DEC Integrated Enterprise Storage Resource Figure 14 SAN solutions match e business strategic needs 1 6 SAN market trends In view of SAN s potential to deliver valuable business benefits we should not be surprised at the substantial interest being shown by users vendors and analysts alike While early adopters have been installing limited SAN solutions since 1998 significant awareness among business users began to be generated during 1999 Many vendors announced SAN products and sol
48. Managed hub cee 121 107 IBM 3534 management ports 0 ee 123 108 IBM 3534 Managed Hub 0 eee ee 123 109 Start setting the IP address 1 0 ee 124 110 Current IP address 0 0 paiia cee 124 111 Changing the IP address 1 0 0 cee ee 125 1123534 BoP Ort esi gkeca ge eid eae eae ea el eR ee tes 129 113 Arbitrated loop eee 130 LTA Private lOO Peace ct tote a cissac atc aei a an ieii nest nin ahaa Hace neal tei dyed 130 115 PUbliGlOOps2 0 doe eee eee ene ee eae de ae ea 131 116 QuickLoop using managed hub 0 0 e eee 133 117 Quickloop spanning to switch 0 00 0 c ee ee 134 118 3534 hub and 2109 switch csao rki e Eka e ET ee 135 119 StorWatch QuickLoop panel nnana aaua 0 000 e ee eee 136 120 QuickLoop with switch partner 0 0 0 0 cee eee 137 121 IBM 2109 S08 Fibre Channel switch 0 000000 eae 140 122 SAN Fibre Channel Switch 2109 S08 Ethernet and serial connectors 141 123 2109 S08 front panel 2 ee 141 124 RJ 45 Ethernet Connector and Serial Port Connector 144 125 2109 S08 switch 0 0 ees 146 126 2109 S16 ic eatin hoagie acke ALE aa ee eee a hb be ew cbs 146 xi xii 127 Launeh VIEW 8 Aika clot ke eee Ee ace Sa Ge ars Baas 152 128 Port attachment ene new eae leek eee E de eh ee Ga OE pease 152 129 F Port storage connection 1 0 0 e ee 153 130 L Port host connection 0 0 eee 154 131 Zoni
49. Of course if a switch goes down all nodes connected to it will not be able to talk to each other or to a node connected to another switch However a node running a mission critical application can have multiple Fibre Channel interfaces each one connected to a different switch in the fabric to overcome this problem This is not possible in a single switch fabric In Figure 141 we show a six switch implementation of a fault tolerant fabric Figure 141 A fault tolerant fabric with six switches This is an example of a completely fault tolerant fabric If any link goes down every switch is still able to communicate with all the other switches If a switch goes down all the remaining switches are able to communicate with each other Another example of a fault tolerant fabric with shorter distances between the switches is shown in Figure 142 Chapter 6 Implementing an IBM SAN Fibre Channel Switch 173 Figure 142 A fault tolerant fabric with four switches Fabric fault tolerance distance or performance considerations determine the design of a multiswitch fabric the number of involved devices decide the number of switches 6 5 3 Cascading using IBM switches All IBM 2109 type switches can be interconnected and due to the switch software and embedded protocols this is a non disruptive topology change The 2109 ports are Fibre Channel G_Ports or FL_Ports G_Ports can connect to either an external device F_Port
50. PA Storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc File View Admin Tools Controls Help E 9 113 24 110 ITSO SAN Data Gateway lt 3 SCSI Channel 1 gt SCSI Channel 2 gt SCSI Channel 3 lt SCSI Channel 4 Z Fibre Channel 1 Z Fibre Channel 4 Ms Current SAN Data Gateway ITSO IP Address 9 113 24 110 Channel Description Fibre Channel 4 Fibre Channel Host 20fd0060 69201d4e Host Name unknown Host Type unknown Host Connection unknown Figure 241 Switch registration By selecting and highlighting the Fibre Channel port as shown in Figure 242 you will see in the right hand view pane that the port is now using a point to point or N_port connection that denotes a fabric login rather than a loop login 286 Planning and Implementing an IBM SAN PA storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc File View Admin Tools Controls Help E 9 113 24 110 ITSO SAN Data Gateway F SCSI Channel 1 gt SCSI Channel 2 gt SCSI Channel 3 amp SCSI Channel 4 Current SAN Data Gateway ITSO IP Address 9 113 24 110 Z Fibre Channel 1 Channel Description World Wide Port Name Fibre Channel 4 20040060 45160E08 Serial Number 00010072 Media Short Wave Optical Firmware Revision 2 01 11 Lm 20fd0060 69201d4e unknown SS SSH Port Type Loop ID Frame Size 2048 Port Mode Public Target
51. Switches Fully redundant or fault or Single Director tolerant interconnects 4 Failure tolerant 99 99 500 100 Switched Dual Directors Fully redundant paths Fabric Max of and interconnects Ports per backplane Fault tolerant backbone interconnects Fault tolerant 99 999 50 100 Switched Dual Directors 5 Fully redundant paths Fabric Max of and interconnects Ports per backplane All interconnects fault tolerant Source Strategic Research Corp Figure 22 SAN Quality of Connection For instance a single point to point topology has no redundancy and is classified as Class 1 QoC If a failure occurs there is no access so there is also 100 performance degradation Class 2 has some redundancy with multiple paths and interconnects but an outage can still occur for an extended period Dual switches or a single director provide full path and interconnect redundancy in Class 3 but a failure would imply degraded performance delivering variable service levels 46 Planning and Implementing an IBM SAN SRC defines Class 4 QoC as Failure Tolerant and Class 5 as Fault Tolerant The Class 4 network must be able to recover from an outage and not incur more than 500 path minutes of degraded operation per year and Class 5 must meet the five 9s measure of availability with only 5 minutes down time and 50 path minutes degradation This requires multiple fabric connections between all devices requiring director
52. System 000 5 GBE RAD amy Bay l Caral hod Cluster 1 Loop A SCSI Port A Action Assign selected yolume s to target hosts Use same ID Lun in source and target Unassign selected yolurne s from target hosts Perform Configuration Update Cancel Configuration Update Figure 86 Modify Volume Assignments panel The Volume Assignments table provides a comprehensive overview of all configured volumes with any associated information We can also sort the table specifying the column which we want to use as sort criteria There is an Action field which is grayed out when no volume is selected and a Target Hosts field Now we show their interaction Figure 87 shows volumes selected and ready for modification Chapter 4 Configuring the ESS with native Fibre Channel 105 Modify Volume Assignments Volume Assignments no sart gt no sort no sort no sort gt no sart no sort no sort no sort gt Array 2 Vol 102 067 FCA13 Device Adapter Pair 1 Open System 096 0GB RAD Amay Fibre Chamel Netfinity ITSO_1 Chister 1 Loop B D 00 LUN 0000 array 2 Vol 103 063 FCA13 Device Adapter Pair 1 Open System 096 0 GBE RAD Amay Fibre Chamel Netfinity ITSO_1 Cluster 1 Loop B D 00 LUN 0001 Array 2 Vol 104 100 FCA13 Device Adapter Pair 1 Open System 000 5 GBE RAD ary Bay l Caral fod Chaster 2 Loop A SCSI Port A Array 1 Vol 000 ID 05 LUN 0000 Action G Assign selected volume s to targe
53. Tape Frame Size 512 1024 2048 ISP2200 Port Mode Target Initiator Target and Initiator Connection Options Loop Loop Preferred Pointto Point Point to Point Preferred Identify Channel OK Cancel Figure 240 Fibre Channel port setting for switch attachment By default the setting in the Connection Options box will have Loop enabled For switch or fabric connection select Point to Point and then Planning and Implementing an IBM SAN select OK The SAN Data Gateway must be restarted for the change to take effect Note If a Connection Options box does not appear the Fibre Channel module installed will support loop connection only and will not support fabric connection The module must be replaced or select a port that will support fabric connection A Fibre Channel cable can be connected from the switch to the port on the Gateway The data to the Specialist must be refreshed by selecting and highlighting the Gateway and then select View gt Refresh SAN Data Gateway 8 6 1 2 Switch registration With VPS enabled the switch will register with the database the WWPN of the port on the IBM SAN Fibre Channel Switch Figure 241 shows a switch connected to port 4 of the Gateway and which has registered to the database Since we cannot load any host registration software onto the switch all other fields are left unknown Chapter 8 Implementing the SAN Data Gateway 285
54. Topology Point to Point BOY Dipruye Open System Storage Figure 71 Storage allocation with host FC port and ESS FC port selected To see the configured host Fibre Channel ports and the assigned volumes in the tabular view we click on the Open System Storage button Open Sysiem Storage This takes us to the Open System Storage panel as shown in Figure 72 Chapter 4 Configuring the ESS with native Fibre Channel 95 Open aA edit Storage Sas ome A or hig puma HBMRsiso00 a T SSSS S pomag2 BM RSi6000 re f0000000c920p773_ o S y O pomag3 emrRsaoo0 re f1ooooooocszos081 y O mags BM Rsi6000_ irc__ _ itogooooocsgopsep f Total 4 volumes S P Size orage Typ Location sd LSS 067 FCA13 Open System 96 0 GB RAID Array Device Adapter Pair 1 LSS 010 Chaster 1 Loop B 2 Vol 103 068 FCA13 Open System 96 0 GB RAID Array Device Adapter Pair 1 LSS 010 paat 1 Loop B y2 Wol La LS5 011 No Modify Host Systems Configure Host Adapter Ports Configure Disk Groups Modify Volume Ssignnent Figure 72 Host system port with associated volumes Now when we select the host Fibre Channel port that we configured in the Host Systems field the associated volumes appear in the Assigned Volume field with all the relevant details As a reminder of what we have done so far we have e Configured a Fibre Channel port of our host system e Configured disk groups e Assigned volumes to the Fibre Channel ho
55. Web site that the StorWatch SAN Data Gateway Specialist was downloaded from http www storage ibm com hardsoft products sangateway support forml htm After completing the registration and license agreement the Download Main page is displayed Select the operating system software subheading and look for HOSTSW View the read me file for the latest information Download the software and install it onto the host that will be attaching to the SAN Data Gateway Follow the instructions provided in the read me file For Windows NT the file is a self executing file so it can be executed or run as any exe file As it runs through the install make sure to select SAN Explorer and VPS Registration Service as shown in Figure 230 Select Components X In the options list below select the checkboxes for the options that you would like to have installed The disk space fields reflect the requirements of the options you have selected IV SAN Explorer Md Pathlight VPS Registration Service Disk Space Required 183k Disk Space Remaining 3101079 k lt Back Cancel Figure 230 Loading VPS Registration software on Windows NT Once installed the service runs automatically and does not require further administrator interaction 8 5 2 3 Communicating with the Gateway Once the host is attached to the Gateway and restarted the registration service will communicate to the Gateway The data shown on the Gateway
56. World Wide Node Name field Enter exactly 16 hexadecimal characters 0 9 a f and A F in the WWNN field This field is mandatory for Fibre Channel hosts it is enabled only when you select Fibre Channel in the Host Attachment field 6 Enter a description of the host in the Description field This is an optional field The description should not exceed 255 characters 7 Enter the remote ESS in the Remote ESS field in Import Export Options on the left side of the panel 8 Click Add to add the host to the Host Systems List on the right side of the panel Note The second host we brought to our basic SAN installation was an RS 6000 Fibre Channel AIX 6 4 2 Setting up zones in your SAN fabric As we stated earlier there are several ways that zoning and zone management can be performed e By using Telnet using either out of band or in band communication by logging into the IBM SAN Fibre Channel Switch e By using the StorWatch Fibre Channel Switch Specialist 6 4 2 1 Using the StorWatch Fibre Channel Switch Specialist We show how to implement zoning using the StorWatch Fibre Channel Switch Specialist We used the following steps 1 We started a Java enabled Web browser and entered a switch name or IP address in the Location Address field We got a fabric view similar to that shown in Figure 133 162 Planning and Implementing an IBM SAN File Edit View Go Communicator Help Z 3 teaeusth S F 2 Back Forward Reload
57. a SAN configuration you have already established the expected benefits But costs are always one of the most significant aspects of any investment decision What are the decision criteria which are most important in the solution you are contemplating For a mission critical enterprise application it may be that high availability is the overriding requirement In a campus wide application the dominating theme may be connectivity At an individual departmental level low cost may be the main objective As with most other choices you can make design trade offs Table 1 but each compromise usually involves giving up on something whether it is performance availability security scalability manageability or some other characteristic Table 1 Design trade offs Design Goal Trade off High availability Redundant components and higher costs High performance Higher cost circuits and more equipment High level of security More costly monitoring and reduced ease of use High scalability Higher costs with possible availability impacts High throughput for one application Lower throughput for another application Low cost Reduced availability and performance 2 2 Defining the infrastructure requirements 30 If you are starting from scratch with a totally new network in a green field site then you can go straight ahead with selection of the optimum SAN topology to meet your needs But in most situations it is likely
58. a structured approach to SAN design and implementation 2 1 Establishing the goals There is an old maxim which states If it isn t broken don t fix it This can easily be applied to discussions about implementing SANs When you look at your current storage infrastructure if you find that it meets all your expectations for data availability performance and connectivity then implementing a SAN will be difficult to cost justify Most IT executives will be reluctant to make investments in a new IT infrastructure unless they can be shown that real benefits will accrue 2 1 1 Business goals As we have seen in 1 5 The business benefits of a Fibre Channel SAN on page 17 there are numerous ways in which a SAN can benefit an organization Each company will have a unique set of circumstances and needs If you can identify specific applications which today suffer from lack of connectivity inability to share information or storage resources with others which cannot be backed up in a timely manner due to bandwidth constraints on your LAN or otherwise are limited in the way in which they provide service to the organization then a SAN can be the solution If users are always asking for more storage and your storage costs are growing rapidly and management of resources is becoming increasingly difficult then a SAN is a likely answer If your company is moving into e business supporting application operations 24 hours 7 days a w
59. and Implementing an IBM SAN Storage Area Network Utilizing a specialized network behind the servers Local Wide Area Network Messaging Protocol TCPAP Net BI Servers Storage Area Network VO Protocols SCSI ESCON FICON etc Enterprise Storage Magstar JBOD Figure 8 Storage Area Network the network behind the servers A SAN differs from traditional networks because it is constructed from storage interfaces SAN solutions utilize a dedicated network behind the servers based primarily though not necessarily on Fibre Channel architecture Fibre Channel provides a highly scalable bandwidth over long distances and with the ability to provide full redundancy including switched parallel data paths to deliver high availability and high performance Therefore a SAN can bypass traditional network bottlenecks It supports direct high speed transfers between servers and storage devices in the following ways e Server to storage This is the traditional method of interaction with storage devices The SAN advantage is that the same storage device may be accessed serially or concurrently by multiple servers e Server to server This is high speed high volume communications between servers e Storage to storage For example a disk array can back up its data direct to tape across the SAN without processor intervention Or a device can be mirrored remotely a
60. and to read and wnite files on your system You must grant these permissions to allow installation of the EFC Manager application Begin Solaris Installation Click the button below to download and save the installer program to your hard disk After the download is complete run the following command sh medataClientInstall bin then follow the install prompts You may delete the HP UX 11 x downloaded installer program once installation is complete Download HP UX Installer PAD Document Done Figure 156 Start page for remote EFC Manager installation We will install the EFC Manager software on a Microsoft Windows 95 98 NT system so we select that option After doing so we are taken to the next Web page to start the download and installation procedure Immediately we are prompted to grant additional privileges to the Java based installation 198 Planning and Implementing an IBM SAN software First we have to grant the right to start programs and then grant the right to read modify or delete files If we do not grant the additional privileges we will not be able to perform the installation The button to start the download is covered until we grant the rights as shown in Figure 157 Chapter 7 Implementing the McDATA ED 5000 199 etscape Biel x elp ome Search Netscape Print Security Stop t 79 113 24 119 EFC_ClientInstall install htm v M Java Security E JavaScript or a Java applet from Zero G Soft
61. be something completely different An example is shown in Figure 236 Pa Storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc File View Admin Tools Controls Help lt SCSI Channel 2 P Disk Device 0 0 J Disk Device 0 1 J Disk Device 1 0 9 Disk Device 1 1 J Disk Device 2 0 J Disk Device 2 1 B Disk Device 3 0 J Disk Device 3 1 9 Disk Device 4 0 B Disk Device 4 1 B Disk Device 5 0 B Disk Device 5 1 Current SAN Data Gateway ITSO IP Address 9 113 24 110 N NI Channel Description SCSI Channel 2 Device Description Disk Device 6 1 Assigned LUN 38 SCSI Channel 2 Target ID 6 Device LUN 1 J Disk Device 6 0 Vendor ID IBM Product ID 2105E20 Revision 0524 Serial Number 625FC142 Capacity 1 953 151 Blocks Block Size 512 Removable No Disk Device 8 0 9 Disk Device 8 1 B Disk Device 9 0 J Disk Device 9 1 f Disk Device 10 0 9 Disk Device 10 1 B Disk Device 11 0 B Disk Device 11 1 J Disk Device 12 0 P Disk Device 12 1 gt SCSI Channel 3 SSS SSS Width 16 Speed Ultra 20MHz Status Operational Figure 236 SCSI LUN assignment In this example the disk device attached has a SCSI target of six and LUN ID of 1 When it was recognized by the SAN Data Gateway the Gateway assigned LUN number 38 to this device This Gateway s assigned LUN number corresponds to the assigned LUN
62. be used are to be connected together to form a complete SSA loop with the Router included All dip switches in SW1 should be in the down position On SW2 dip switches 0 and 1 are set in the down position all other switches in dip switch 2 should be turned up This is Planning and Implementing an IBM SAN considered mode 3 Power on the Router the Status LED will begin to flash rapidly as it searches the SSA loop to recognize all the disk drives This may take approximately 1 minute to complete Once the Status LED has stopped flashing and is solid the process is complete and the Router is powered down 4 Router Subsystem Diagnostic test A Subsystem Diagnostic test is now run on the disk drives called mode 15 This tests the disk drives for spindle spin up read tests and nondestructive write tests The Router is set for mode 15 by setting switches 0 1 2 and 3 on SW2 to the down position and the rest turned up The Router is now powered on the Status LED will flash rapidly The test will be done on each disk drive in the SSA loop separately and will begin with the drive closest to the A1 port on the back of the Router As the test is completed on a drive the LED on the SSA drive will flash and then it will move to the next drive This test should continue until all drives have been tested The test runs in a continuous cycle so once all drives have been tested at least once the Router is powered off If a drive fails the test
63. button To change specific information we display it using this button As we want to change it we press the Clear button to get a cursor in the entry field With the Detail button we increment the value of the field we have activated Pressing the Clear button again gets us to the next field to change If all entries are correct we use the Entry button to save the changes This has to be done with the IP address the Subnetmask and the Gateway address 7 2 1 2 Suggested network setup We suggest for security and availability reasons using the optional Ethernet hub to establish a dedicated Ethernet segment to connect the EFC server and the switch which should look similar to that shown in Figure 149 Corporate Ethernet batt Gee EFC ea e ih Server Intranet Sen Dedicated Ethernet McDATA Remote EFC ED5000 Manager LN Figure 149 Suggested ED 5000 setup Chapter 7 Implementing the McDATA ED 5000 189 If you plan to use the dedicated Ethernet between the EFC server and the ED 5000 then you only need to set up the EFC server for IP 7 2 2 Logging on to the EFC server and the EFC Manager 190 We logon to Windows NT with the user ID Administrator and the password password both of which are case sensitive After logging on to Windows NT we get the login window for the Enterprise Fabric Connectivity Manager EFC Manager which
64. can be viewed more like a RAID subsystem in that it is designed to be highly fault tolerant Only the failure of the mother board would result in total failure of the director All other Chapter 2 SAN planning and design considerations 45 components are redundant with automatic failover Redundant field replaceable units are hot swappable and microcode updates can be made non disruptively Maintenance capabilities such as call home are supported According to tests run by CLAM Associates it can take more than an hour to replace and reconfigure a new switch and bring it back into operation For a 16 port switch this equates to 960 path minutes of degraded performance as defined by the SRC QoC methodology A path minute describes one user port being unavailable for one minute Using path minutes as a way of describing the impact of an outage SRC defines five levels of QoC as shown in Figure 22 SAN Quality of Connection QoC Fault Availability Performance Bandwidth Device Class Tolerance Annual uptime Degradation Scaleability Topology Path minutes year 1 Failure sensitive 90 Not applicable Single Point to Point Single Hub No redundancy or Loop 2 Failure resilient 99 50 000 Loops and or Single or Dual Partially redundant Switched Fabric Hubs or paths and interconnects Fabric Switches with single or dual paths 3 Failure resilient 99 9 5000 Switched Fabric Dual Fabric Fully redundant paths
65. capability is not yet supported due to a number of network management requirements imposed by the S 390 architecture IBM expects a transition period during which S 390 FICON SANs will develop separately from Fibre Channel Protocol FCP open systems SANs which use the SCSI protocol In the longer term FCP and FICON SANs will merge into a true Enterprise SAN IBM has published a number of redbooks on the subject of FICON and an example of this is ntroduction to IBM S 390 FICON G24 5176 Additional redbooks that describe FICON can be found at the IBM Redbooks site by using the search argument FICON http www ibm com redbooks For this reason this book will focus exclusively on FCP open systems elements of IBM s Enterprise SAN which are available today 1 4 What Fibre Channel is Fibre Channel is an open technical standard for networking which incorporates the channel transport characteristics of an I O bus with the flexible connectivity and distance characteristics of a traditional network Notice the European spelling of Fibre which is intended to distinguish it from fiber optics and fiber optic cabling which are physical hardware and media used to transmit data at high speed over long distances using light emitting diode LED and laser technology Because of its channel like qualities hosts and applications see storage devices attached to the SAN as if they are locally attached storage Because of its network chara
66. considerations 31 an increase in the frequency of backup activity by user groups Perhaps today they are deterred by the slow speed of backups across the LAN Could the current weekly backup cycle move to a daily cycle as a result of the improved service If so what would this do to SAN bandwidth requirements 2 2 3 Platforms and storage 32 How many servers and what are the operating platforms which will be attached to the SAN The majority of early SAN adopters have tended to implement homogeneous installations that is supporting a single operating platform type such as all Netfinity all HP or all Sun servers As SANs are maturing the trend is towards larger scale networks supporting multiple heterogeneous operating platforms combining AIX UNIX Windows NT and so on which has implications for security Fibre Channel capable servers require Fibre Channel HBAs to attach to the SAN fabric The choice of HBA is probably already decided by the server vendor Before you decide how many HBAs you require in your host to achieve optimal performance you need to evaluate the performance of the server Fibre Channel HBAs today transfer data at 100 MB s Can the system bus provide data at the same or higher speed If not the HBA will not be fully utilized The most common system bus in use today is the Peripheral Component Interconnect bus PCI which operates at either 132 MB s or 264 MB s Sun SBus operates at 50 MB s and HP HSC at only
67. defining a channel that transfers data between CPUs and from a CPU to disk arrays and other peripherals HMMP HyperMedia Management Protocol HMMS HyperMedia Management Schema the definition of an implementation independent extensible common data description schema allowing data from a variety of sources to be described and accessed in real time regardless of the source of the data See also WEBM HMMP HSM Hierarchical Storage Management A software and hardware system that moves files from disk to slower less expensive storage media based on rules and observation of file activity Modern HSM systems move files from magnetic disk to optical disk to magnetic tape HUB A Fibre Channel device that connects nodes into a logical loop by using a physical star topology Hubs will automatically recognize an active node and insert the node into the loop A node that fails or is powered off is automatically removed from the loop HUB Topology see Loop Topology Hunt Group A set of associated Node Ports N_Ports attached to a single node assigned a special identifier that allows any frames containing this identifier to be routed to any available Node Port N_Port in the set In Band Signaling Signaling that is carried in the same channel as the information Information Unit A unit of information defined by an FC 4 mapping Information Units are transferred as a Fibre Channel Sequence Intermix A mode of service defined by Fibre
68. delivery of the data Planning and Implementing an IBM SAN e Packaging data in packets frames in Fibre Channel terminology e Flexibility in terms of the types of information which can be transported in frames such as data video and audio e Use of existing device oriented command sets such as SCSI and FCP e A vast expansion in the number of devices which can be addressed when compared to I O interfaces a theoretical maximum of more than 16 million ports It is this high degree of flexibility availability and scalability the combination of multiple protocols at high speeds over long distances and the broad acceptance of the Fibre Channel standards by vendors throughout the IT industry which makes the Fibre Channel architecture ideal for the development of enterprise SANs For more details of the Fibre Channel architecture refer to Designing an IBM Storage Area Network SG24 5758 1 5 The business benefits of a Fibre Channel SAN Today s business environment creates many challenges for the enterprise IT planner SANs can provide solutions to many of their operational problems 1 5 1 Storage consolidation and sharing of resources By enabling storage capacity to be connected to servers at a greater distance and by disconnecting storage resource management from individual hosts a SAN enables disk storage capacity to be consolidated The results can be lower overall costs through better utilization of the storage low
69. eee 40 18 Load sharing on parallel paths 0 0 eee eee 41 19 A fully meshed topology 00 0 cee eee 42 20 Redundant fabrics ers darepada ERPAT NEEE ees 43 21 Fabric backbone interconnects SAN islands 000 44 22 SAN Quality of Connection 0 0 cee 46 23 SAN hierarchical design 0 00 e eee tees 48 24 The IBM Enterprise SAN vision 0 00 cee ee ee 50 25 Introduction panel of the native FC ESS 00 005 64 26 LOQINIWINKOW sca see ock a eama a aid pad e4 dae Oy aave dant da dae 65 27 Status panel graphical view s s s auaa aaaea 66 28 Status panel problem log 00 c eee eae 67 29 Problem notification panel 1 2 0 2 0 ee eee 68 30 Communications panel 0 0 00 eee 69 31 Remote support modification 20 0 cee eee 70 32 Storage Allocation panel 0 0c ec eee 71 33 User administration panel 0 0 0 eee ee 72 34 Modify users panel anuanua naana 73 35 Licensed Internal Code panel 0 000 e eee ee 74 36 Connecting to Copy Services Server 0000 0c eee 75 37 ESS Copy Services introduction panel 2 2 2000055 75 38 Storage Allocation ESS with FC and SCSI 2 76 39 SCSI adapter with two ports 0 6 ee ee 77 40 FC adapter with one port 0 2 0 eee ee 77 Copyright IBM Corp 2000 ix x 41 42 43 44
70. expanded using tiers of switches for redundancy and increased bandwidth In such an environment you will need to use zoning and LUN masking to provide appropriate security to application data Chapter 2 SAN planning and design considerations 43 Backbone SAN Switch Switch Switch Switch Switch Department B SAN Department C SAN Switch Figure 21 Fabric backbone interconnects SAN islands 2 3 3 Quality of service requirements 44 An important criterion for selection of SAN components relates to the level of service required from the SAN This includes all aspects of the technology hub switch or director the topology loop or fabric and the degree of redundancy including fault tolerance This is particularly relevant for organizations serving the global marketplace 24 hours per day seven days per week over the Internet In the e business economy of today continuous availability is not optional If you are not online you are not open for business and widely reported incidents of system outages in well known e business companies show that loss of revenue can be immense Strategic Research Corporation SRC has described the principle of Quality of Service QoS This is a framework used to establish appropriate performance and availability characteristics of a complex service such as a network Specifically SRC defines service levels for SANs which they
71. for view download or order hardcopy CD ROM Redbooks from the Redbooks Web site Also read redpieces and download additional materials code samples or diskette CD ROM images from this Redbooks site Redpieces are Redbooks in progress not all Redbooks become redpieces and sometimes just a few chapters will be published this way The intent is to get the information out much quicker than the formal publishing process allows e E mail Orders Send orders by e mail including information from the IBM Redbooks fax order form to e mail address In United States or Canada pubscan us ibm com Outside North America Contact information is in the How to Order section at this site http www elink ibmlink ibm com pbl pbl Telephone Orders United States toll free 1 800 879 2755 Canada toll free 1 800 IBM 4YOU Outside North America Country coordinator phone number is in the How to Order section at this site http www elink ibmlink ibm com pbl pbl e Fax Orders United States toll free 1 800 445 9269 Canada 1 403 267 4455 Outside North America Fax phone number is in the How to Order section at this site http www elink ibmlink ibm com pbl pbl This information was current at the time of publication but is continually subject to change The latest information may be found at the Redbooks Web site m IBM Intranet for Employees IBM employees may register for information on workshops residencies and Re
72. host The host will only see the storage devices defined in the software zoning table But the host can also make a direct connection to the storage device while doing device discovery without asking SNS for the information it has 6 3 3 4 Zone aliases Zone aliases simplify repetitive port numbers entries or World Wide Names A zone alias is a C style name for one or more port numbers or World Wide Names For example the name Production can be used as an alias for 10 00 00 60 69 00 00 8a 6 3 3 5 Zone configurations A zone configuration is a set of zones At any one time zoning may be disabled or one zone configuration may be in effect When a zone configuration is in effect all zones that are members of that configuration are in effect You select which zone configuration is currently in effect The set of zone configurations defined in a fabric may not be the same as the configuration that is currently in effect and may also not be the same as the configurations that are saved in the switches flash memory The following three terms are used to differentiate between these configurations e Defined configuration The defined configuration is the complete set of all zone objects that have been defined in the fabric There may be multiple zone configurations defined although only one can be in effect at a time There may be inconsistencies in the definitions there may be zones or aliases that are referenced but are not defined th
73. in this SAN is e 2 McDATA Enterprise Director ED 5000 32 ports 50 micron shortwave cable 9 micron longwave cable e 1 Netfinity NT Server 5500 1 FC HBA Emulex 7000 e 1 Netfinity NT Server 5500 1 FC HBA QLogic 2200 1 ESS 2105 F20 with Feature Code 3022 native FC adapter 10 FC adapter 10 ports 1 SCSI adapter 2 ports 3 RS 6000 AIX servers 1 FC HBA Emulex 7000 7 1 3 Software overview The software that we use in this SAN is e Netfinity Microsoft Windows NT 4 0 Service Pack 5 e RS 6000 AIX e Emulex 7000 e QLogic 2200 e ESS Microcode 1 1 7 1 4 ED 5000 installation steps The installation of the switch and the EFC server is straight forward The ED 5000 ships with all necessary components The McDATA SAN was implemented using the following steps we have included them for reference These steps are also provided with the ED 5000 Enterprise Fibre Channel Director Installation Manual which should be referred to if more detailed information is required Chapter 7 Implementing the McDATA ED 5000 183 184 In 7 2 Setting up an environment for using and managing a McDATA SAN on page 187 we show how to manage this environment once it is installed If you have an environment with the McDATA ED 5000 already installed you may skip this section We do not carry out every task for example setting up for SNMP nor do we describe every single step in its entirety However we show how we built and man
74. integrity 282 Planning and Implementing an IBM SAN As each Fibre Channel port can support up to 8 eight different hosts there can be up to 48 hosts attached to the SAN Data Gateway They can all share the same four SCSI channels By using the Virtual Private SAN feature you can ensure that only the LUNs you desire a certain host to use will be accessed and that no other host will access them 8 5 3 Combining Zoning and Virtual Private SAN If Virtual Private SAN is enabled and LUNs have been allocated to specific hosts then zoning is not necessary or required The Channel Zoning window can remain at the default settings with all Fibre Channel ports accessing all SCSI channels However they can work in combination to add an extra level of control and security If zoning is added then VPS can only control the LUNs that are included in its access zone Figure 239 shows a zoned SAN Data Gateway Channel Zoning Setting x SCSI Channel i 2 3 4 SAN Connection J v J Ww SAN Connection4 v y v J Check the SCSI Channel box to allow Access OK Cancel Figure 239 Combining channel zoning and VPS SAN connection 1 has access to the LUNs on SCSI channels 2 and 4 VPS will control access to LUNs on SCSI channels 2 and 4 for SAN connection 1 Since SAN 1 is not zoned for SCSI channel 1 and 3 it will not reach any LUNs on these channels Even if a LUN on SCSI 3 has been enabled for access to a ho
75. is installed and operational on the system The EFC Manager is a centralized tool for the McDATA network It is a Java based GUI that shows a graphical view of all managed devices and functions as an entry point for all common management and monitoring tasks For detailed information about the EFC Manager and how to use it refer to the Enterprise Fabric Connectivity Manager User Manual To start the EFC Manager we log on with the same user and password we used when logging on to Windows NT Administrator and password We are locally working on the EFC server and therefore we specify in the EFC server entry field localhost which is shown in Figure 150 Planning and Implementing an IBM SAN EFC Manager Login aros il X Ene Figure 150 Logging in to the EFC Manager on the EFC Server After logging on to the EFC Manager it opens with the Product View shown in Figure 151 On the left there is a panel that allows us to perform various configuration and monitoring tasks The main window is empty because there are no devices configured on the EFC Manager The same applies to the Fabric View of the EFC Manager We can switch to it by using the View button on the panel as shown in Figure 151 Chapter 7 Implementing the McDATA ED 5000 191 6 McDATA Enterprise Fabric Connectivity Manager Product View Fabric X O Session View be Configure i Logs xe Maintenance i Help Na LogoutE
76. level hardware or multiple inter switch links ISLs in a meshed fabric A consideration with ISLs using E port connections is that they do not add bandwidth to the total configuration only bandwidth and connectivity between switches For this reason SRC concludes that to meet Class 4 and Class 5 QoC requirements today for fault tolerance with scalable performance the maximum number of ports per backplane are required hence directors are favored for these mission critical applications This is due to their larger number of ports 32 and n 1 fault tolerant design Future switches with larger port counts would also address this effectively if configured in redundant meshed fabrics 2 3 4 Hierarchical design What we have seen is that a SAN can take numerous shapes When you start thinking about SAN design for your own organization you can learn from the experience gained in the design of other mature networks such as LANs and the Internet In these a hierarchical network structure has generally been adopted to facilitate change allow easy replication as the structure grows and minimize costs This hierarchy comprises three layers as shown in Figure 23 The core At the center is a high speed fault tolerant backbone which is designed to provide very high reliability QoC Class 4 or 5 as defined by SRC This is designed to minimize latency within the fabric and to optimize performance This core would normally be built around fault
77. more to consider but in our installation we will show how to install GBICs for those that are new to the Fibre Channel world To insert the GBIC follow these steps 1 Remove the plastic cover 1 from the GBIC 2 This is shown in Figure 101 Figure 101 Insert GBIC 2 Slide the GBIC 2 into the port 3 Connect the fiber optic cable 3 into the installed GBIC The GBIC housing has an integral guide key that is designed to prevent improper insertion and this is shown in Figure 102 Figure 102 Insert cable into GBIC Planning and Implementing an IBM SAN 4 Once you have installed the GBICs in your hub attach the hub ports using standard 50 125 or 62 5 125 FC cable to your Fibre Channel host and storage device The FC AL was now operational and we powered on the hub The Netfinity server and the ESS were already connected and running By looking at the Device Active LEDs on the FC Storage Hub ports as shown in Figure 103 and ensuring that they were green we knew that all the connections were working Device Active Green LEDs Figure 103 Device Activity LEDs in FC Storage Hub Now we checked to see whether we can see the same ESS logical volumes as though we were directly attached to the ESS 5 2 5 Checking disk access We used the Windows NT Disk Administrator to view the available disks or ESS LUNs In Figure 104 we show the Disk Administrator window from the Windows NT system T
78. or more networks See also Bridge Bridge Router SAF TE SCSI Accessed Fault Tolerant Enclosures SAN A Storage Area Network SAN is a dedicated centrally managed secure information infrastructure which enables any to any interconnection of servers and storage systems SAN System Area Network term originally used to describe a particular symmetric multiprocessing SMP architecture in which a switched interconnect is used in place of a shared bus Server Area Network refers to a switched interconnect between multiple SMPs SC Connector A fiber optic connector standardized by ANSI TIA EIA 568A for use in structured wiring installations Scalability The ability of a computer application or product hardware or software to continue to function well as it or its context is changed in size or volume For example the ability to retain performance levels when adding additional processors memory and or storage SCSI Small Computer System Interface A set of evolving ANSI standard electronic interfaces that allow personal computers to communicate with peripheral hardware such as disk drives tape drives CD_ROM drives printers and scanners faster and more flexibly than previous interfaces The table below identifies the major characteristics of the different SCSI version Fast 10 8 10 7 6 SCSI 2 Fast 10 16 20 15 6 Wide SCSI 2 Ultra 20 8 20 7 1 5 SCSI Ultra 20 16 40 7 12 SCSI 2 Ultra 40 16
79. proverbial paradigm shift in the way we organize and use the IT infrastructure of an enterprise In the chapter that follows we show the market forces that have driven the need for a new storage infrastructure coupled with the benefits that a SAN brings to the enterprise 1 1 The need for a new storage infrastructure Copyright IBM Corp 2000 The 1990 s witnessed a major shift away from the traditional mainframe host centric model of computing to the client server model Today many organizations have hundreds even thousands of distributed servers and client systems installed throughout the enterprise Many of these systems are powerful computers with more processing capability than many mainframe computers had only a few years ago Storage for the most part is directly connected by a dedicated channel to the server it supports Frequently the servers are interconnected using local area networks LAN and wide area networks WAN to communicate and exchange data This is illustrated in Figure 1 The amount of disk storage capacity attached to such systems has grown exponentially in recent years It is commonplace for a desktop Personal Computer PC today to have 5 gigabytes GB or 10 gigabytes and single disk drives with up to 75 GB are available There has been a move to disk arrays comprising a number of disk drives The arrays may be just a bunch of disks JBOD or various implementations of redundant array
80. reports of data growing by more than 10 times annually This is a nightmare for planners as it is increasingly difficult to predict storage requirements A finite amount of disk storage can be connected physically to an individual server due to adapter cabling and distance limitations With a SAN new capacity can be added as required without disrupting ongoing operations SANs enable disk storage to be scaled independently of servers 1 5 4 Improved backup and recovery 20 With data doubling every year what effect does this have on the backup window Back up to tape and recovery are operations which are problematic in the parallel SCSI or LAN based environments For disk subsystems attached to specific servers two options exist for tape backup Either it must be done to a server attached tape subsystem or by moving data across the LAN 1 5 4 1 Tape pooling Providing tape drives to each server is costly and it also involves the added administrative overhead of scheduling the tasks and managing the tape media SANs allow for greater connectivity of tape drives and tape libraries especially at greater distances Tape pooling is the ability for more than one server to logically share tape drives within an automated library This can be achieved by software management by using tools such as Tivoli Storage Planning and Implementing an IBM SAN Manager or with tape libraries with outboard management such as IBM s 3494 1 5 4
81. special fibre channel sequence that is used to start loop initialization Allows ports to establish their port addresses Local Area Network LAN A network covering a relatively small geographic area usually not larger than a floor or small building Transmissions within a Local Area Network are mostly digital carrying data among stations at rates usually above one megabit s Login Server Entity within the Fibre Channel fabric that receives and responds to login requests Loop Circuit A temporary point to point like path that allows bi directional communications between loop capable ports Loop Topology An interconnection structure in which each point has physical links to two neighbors resulting in a closed circuit In a loop topology the available bandwidth is shared LVD Low Voltage Differential Management Agent A process that exchanges a managed node s information with a management station Managed Node A managed node is a computer a storage system a gateway a media device such as a switch or hub a control instrument a software product such as an operating system or an accounting package or a machine ona factory floor such as a robot Managed Object A variable of a managed node This variable contains one piece of information about the node Each node can have several objects Management Station A host system that runs the management software 357 Meter 39 37 inches or just slightly larger than a yar
82. stock prices of specialist manufacturers and developers of fabric components As with any new technology it is up to the user to assess its relevance and value and to decide if and when to deploy appropriate solutions But buyers should also beware It is easy to assume that because other mature network solutions operate in a particular way for example in relation to the interoperability of solution components so does Fibre Channel This is not necessarily the case because Fibre Channel standards for storage networking are still emerging The purpose of this book is to discuss some of the factors which IT architects and planners should take into consideration as they begin to investigate and to design business oriented SAN solutions for their enterprise Planning and Implementing an IBM SAN Chapter 2 SAN planning and design considerations Once you have concluded that Storage Area Networks will be beneficial to your organization where do you go from here How do you plan to go about implementing a SAN Who should be involved What do you need to do before you call your vendor and order new equipment This chapter discusses some of the things you need to consider when you are at the planning and design stage It does not purport to be a detailed methodology It is intended only to cover some basic ideas and suggestions IBM s International Global Services division offers detailed planning design and consultant services which give
83. the B loop of the first adapter pair as RAID array To do this we select the first disk group we want to configure and select RAID array from the Storage Type pull down menu and this is also shown in Figure 52 Doing this also changes the Track Format to Fixed Block FB in the Track Format pull down menu After doing this we select the second disk group in the other cluster and also select RAID array The Modification column of the panel as shown in Figure 53 shows that we have made changes Defined and the Capacity column shows that our disk groups are still unformatted In this case formatted to the ESS means it is not formatted as either a RAID array nor as non RAID Chapter 4 Configuring the ESS with native Fibre Channel 85 Fixed Block Storage a cal ia Fixed Block FB Ufommatted 291 20 GB Fixed Block FB Urfomnatted 291 20 GB me Adapter Pair 2 mae ia Block FB Fonmatted 52 62 GB A Amay i ir ae Block FB Fomnatted 52 62 GB Disk Group Attributes ram Array Storage Type Fiea Block FB be Track Format Perform Configuration Update Cancel Configuration Update Figure 53 Changed disk group definition To apply the changes made we pressed the Perform Configuration Update button Perform Configuration Update A warning message appears stating that this will be a time consuming action as shown in Figure 54 Warning 1402 Time intensive Action The requested configuration update will take
84. the IBM 2109 S08 switch 175 6 6 Related information 0 0 000 ee es 180 Chapter 7 Implementing the McDATA ED 5000 181 7 1 Our ITSO environment and the installation steps 181 7 1 1 ED 5000 overview sio cotria tyre eia E ees 182 7 1 2 Hardware overview sasusaaanaa aaa 183 7 1 3 Software overview 0 ee 183 7 1 4 ED 5000 installation steps 00 0000 eee 183 7 2 Setting up an environment for using and managing a McDATA SAN 187 7 2 1 Setting up the network environment 005 187 7 2 2 Logging on to the EFC server and the EFC Manager 190 7 2 3 Defining users on the EFC Manager 192 7 2 4 Installing the EFC Manager on remote workstations 196 7 3 Managing the ED 5000 fabric 0 2 0000 eee eee 203 7 3 1 Identifying the ED 5000 to the EFC Manager 204 7 3 2 Assigning nicknames to WWN 2 00 0 ee eee 206 7 3 3 Configuring the ED 5000 with the Product Manager 208 7 3 4 Configuring the FC ports 0 0 0 0 eee eee eee 211 7 3 5 Configuring the FC operating parameters 214 7 4 Zoning with MCDATA 6 si ga leeward deed aian ea e de de pad 216 7 4 1 Zoning overview ee ees 216 7 4 2 Preparing to define zones 0 0 00 eee ee 217 7 4 3 McDATA zoning concept 00 e eee eee eee 219 7 4 4 Creating a zone set and establishing an NT zone
85. the Mirror Drive window This is shown in Figure 274 322 Planning and Implementing an IBM SAN T33282 ITSOm T33025 ITSOl TOOOOO 3SE68229 TOOOO6 3SE68462 TOOOO 35E684AE Figure 274 Control Center with Mirror Drive If you select and highlight the mirror drive and then click on the Properties button the Mirror Properties window opens and you can see the information in the mirror drive Figure 275 shows an example of the properties of the mirror drive Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 323 Mirror Instant Copy Drive Properties Eg Mirror Instant Copy Drive Selection Cancel Drive Members frargee SSA UID Function Add Member Identify Menber T00003 35E6848D A T00004 35E22D6C A Delete Member Identify mirror T00005 35E6840E s Drive Characteristics Tool Box Capacity MB 8689 PUBLIC FC LUN B Drive Name frrsom Number of 3 Change Members Figure 275 Mirror Drive Properties Clicking on the Identify Mirror button will cause the LED on the actual SSA drives that belong to the mirror to flash If a member drive is selected and the Identify Member button is chosen then the LED only on that drive will flash In both cases a dialog box appears to allow you to stop the flashing 9 6 Instant Copy drive Instant Copy is a feature that allows a drive to become part of a mirror synchronize to the latest data
86. the configuration file for your SLIC system Begin by deleting the sign on the lines that contain the sample configuration The rest of the information can be entered in as shown in Figure 256 A description of each entry field is also provided Planning and Implementing an IBM SAN 6 7190 cfg WordPad olx Fie Edit View Insert Format Help ian Haes SDU Service Utility Daemon Configuration File Syntax itso path F IBM7190 SFa userlogfile c ibm7190 itsoevent log QueryChangeInterval 10 Comment Lines are declared using Example co path J IBM7190 SFa K IBM7190 SFa QueryChangeInterval in seconds QueryChangeInterval 10 3 Call Home Feature Support The Call Home Feature allows the user to be notified through the email when a user designated SRNs occur The designated email address the profile for the email program used and SRNs must be defined in this file Syntax system email userif email addr com user2 email addr com profile email program name srn N srn number N 2nd srn number e For Help press F1 Figure 256 Edited configuration file 9 2 2 2 Configuration file information The SLIC_name can be any name that you would like to use to identify the Router Creating a SLIC Zone The path refers to the SLIC Zone file or partition used for the Manager to communicate to
87. we change the wiring to the devices but we can change devices connected to ports without having to redefine the zones Planning and Implementing an IBM SAN 7 4 2 Preparing to define zones To view and manage the zones in the McDATA fabric we must open the zoning view of the EFC Fabric Manager To do so we connect to the EFC Manager then select View gt Fabric as shown in Figure 178 Figure 178 Opening Fabric View of the EFC Manager Click on the fabric icon in the upper left of the Fabric View to open the EFC Fabric Manager Now the Fabric Manager opens as a second window with the Topology View In our case as there is only one McDATA installed and there is only one switch visible as shown in Figure 179 Chapter 7 Implementing the McDATA ED 5000 217 McDATA Fabric Manager Figure 179 Topology View of the Fabric Manager To change to the Zoning View we again select the View button and this is also shown in Figure 179 In Figure 180 we show the Zoning View of the Fabric Manager This is the entry point for viewing and managing zones within the selected McDATA fabric At this moment in time there are no active zone sets In the topics that follow we show how to set up zoning in our environment 218 Planning and Implementing an IBM SAN Figure 180 Zoning View of the Fabric Manager 7 4 3 McDATA zoning concept With the McDATA we can specify zone sets A zone set is a group of zones that can be a
88. window From the Available Drive Pool list click on a desired drive and then select the Add gt gt button The drive name will be added to the Member window An 314 Planning and Implementing an IBM SAN asterisk will appear on the left hand side of the drive that was selected in the Available Drive window to denote that the drive has been selected Each drive is added one at a time To remove a drive from the Member window select the desired drive and click on the Remove lt lt button Below each window there is a Drive Capacity box As a drive is selected its capacity in megabytes is shown As you add more member drives to the Member window the Capacity box will add all drive sizes together to provide a total capacity in megabytes This is shown in Figure 266 Composite Drive Setup Composite Drive Members Selection TO0000 LOOO T00001 L001 T00003 L003 T00004 L004 T00005 L005 TOO006 LODE T00007 LOO Available Drive Pool Drive Capacity MB 35E68229 JEZ 35E22D6C 35E6840E S5E68462 35E684AE Identify lt lt Remove e Reset Composite Drive Members T00001 35E22D0B T00002 35E69318 Capacity MB 17378 lt Back Next gt Cancel Figure 266 Creating Composite drive from available drives When all the desired drives are added select Next gt The Assigning Properties window opens as shown in Fi
89. 0 Serial Number 21081300724 Firmware Revision 3 40 10 Health Check Level 2 Health Check Interval 15 Log Size 2047 Receive Event Traps Enabled Serial Baud Rate 19200 E Figure 220 SCSI channel expanded view You can also select and highlight each SCSI Channel You will notice that as you do this the information window on the right side will provide data that is unique to that SCSI channel as shown in Figure 221 Chapter 8 Implementing the SAN Data Gateway 263 PA Storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc File View Admin Tools Controls Help WH 9 113 24 110 ITSO SAN Data Gateway Current SAN Data Gateway ITSO Disk Device 0 0 IP Address 9 113 24 110 M Disk Device 0 1 B Disk Device 1 0 n B Disk Device 1 1 Channel Description SCSI Channel 1 Disk Device 2 0 Channel Type Differential B Disk Device 2 1 Termination Enabled B Disk Device 3 0 Channel Mode Initiator B Disk Device 3 1 Hast ID 7 M Disk Device 4 0 re i Disk Device 4 1 f Device at Max Speed Ultra 20Mhz B Disk Device 5 1 Status Operational 9 Disk Device 6 0 l l 9 Disk Device 6 1 Max ID s per Bus 16 B Disk Device 8 0 Max LUN s per ID 32 9 Disk Device 8 1 B Disk Device 9 0 M Disk Device 9 1 B Disk Device 10 0 M Disk Device 10 1 9 Disk Device 11 0 M Disk Device 11 1 M Disk Device 12 0 Up Time 4 days 23 91 07 Figure
90. 0 00 24 1 5 11 Storage resources match e business enterprise needs 24 1 6 SAN market trends 0 0 00 ee 25 Chapter 2 SAN planning and design considerations 27 2 1 Establishing the goals 0 0 eee ee 27 2 1 1 Business goalS a a Taa eee 27 2 1 2 Technical goals 0 00 ee 28 2 2 Defining the infrastructure requirements 00 05 30 2 2 1 Use of existing fiber 2 2 2 2 ce ee 31 2 2 2 Application traffic characteristics 0000000 31 2 2 3 Platforms and storage 00 cee ee 32 2 3 Selecting the topology 0 eee eee 33 2 3 1 Assessing the components 0 0000 e eee eee 34 2 3 2 Building a multiswitch fabric 0 00 0 c eee eee ee 38 Copyright IBM Corp 2000 iii iv 2 3 3 Quality of service requirements 00 eee eee 44 2 3 4 Hierarchical design 0 2 00 e eee eee 47 24 TINGsNEXE STEPS aer boc A ee T E asec s ete a a a a lie 48 2 4 1 The planning team 1 ee ee 48 2 4 2 Equipment selection 2 00 eee ee 49 2 4 3 Interoperability testing 00 ee 49 2 4 4 Documentation 0 0 00 eee 49 2 5 Future developments 0 000 eee ee 50 Chapter 3 Implementing Fibre Channel host adapter cards 53 3 1 Installing QLogic host adapter cards on a Windows NT system 53 3 1 1 Installing the QLogic 2100F adapter card 53 3 1 2 Install
91. 0 Fe 1oo0o000C920B85A8 E pesi EH Rs 6000 Fe fu Jenne Lo O O P C920B56B Total 4 volumes Shared 067 FCA13 Open System 96 0 GB RAID Array Device Adapter Pair 1 LSS 010 Chaster 1 Loop B 2 Vol 103 068 FCA13 Open System 96 0 GB RAID Array Device Adapter Pair 1 LSS 010 Cluster 1 Loop B ya Vol Me LSS 011 No Modify Host Systems Configure Host Adapter Ports Configure Disk Groups Modify Volume Assignment Figure 83 Open System Storage panel no shared volumes From here we can continue to modify volume assignments What we want to do is to share the volumes we defined with another host Fibre Channel port In the Shared column of the Assigned Volumes table in Figure 83 we can see that the volumes we created are not shared at this moment in time To do this we first click on the Storage Allocation button to get a graphical representation of our ESS Fibre Channel port configuration This is shown in Figure 84 Chapter 4 Configuring the ESS with native Fibre Channel 103 Storage Allocation Graphical View Clear View Click on a Host or to see paths ddl_2N hudson itso moe2200 moe5100 25055 5 0 Topology Point to Point GY Dipriye Open System Storage Figure 84 Selected host FC adapter and ESS FC adapter In the Information field we can verify that the port we selected to plug into the switched fabric is configured correctly 4 1 8 Modifying volume assignments and sharing volumes At this t
92. 0 c921da0a plymouth AIX dewifsosi0 Oxef 0 Emulex 10000000 09221802 K gt New Host Data Original Host Data Close Host Name plymouth plymouth New Host Host Type AIX AIX Host Connection dewtfscsi0 Oxef 0 Emulex fdevifscsi0 Oxef 0 Emulex paige Sob Host WN 10000000 c9221802 10000000 c9221802 Undo Changes Delete a Host Take Host Off Line Hitt Figure 238 Setting LUN masking The host plymouth has access to LUNs 17 to 24 but does not have access to LUNs 25 to 30 Conversely the host at the top has access to LUNs 25 to 30 but does not have access to LUNs 17 to 24 Using the scroll bar on the bottom we can scroll through all 255 available LUNs in the SAN Data Gateway and enable or disable access by adding or clearing the check mark The scroll bar on the right hand side allows us to scroll through the different hosts In this example there are only two hosts but there can be several more Once completed select Apply Changes and then Close The host system may now have to be restarted or some other method used to rescan the bus for the host to detect that it now has access to new devices Any combination is allowed so if the same LUN is to be shared by two different hosts a check mark for that LUN must be set for both If this is the case the host systems must have a device sharing software installed to control access to the disk device for data
93. 0521 is a licensed product requiring a valid license key The RPQ 850521 License Agreement is covered by the IBM Agreement for the Licensed Internal Code and is linked to the serial number of the hub Chapter 5 Implementing an IBM managed and unmanaged hub 133 Figure 117 Quickloop spanning to switch Both configurations allow up to 126 NL_Port devices in one QuickLoop Note The IBM 3534 managed hub is an OEM product from Brocade In Brocade terminology this is a switch 5 7 4 1 Managing QuickLoop To manage QuickLoop for the IBM 3534 Managed Hub you can choose from two methods e Using Telnet commands e Using the managed hub StorWatch Specialist Figure 118 shows the hub switch combination we used in our ITSO scenario 134 Planning and Implementing an IBM SAN ee ie A Figure 118 3534 hub and 2109 switch We show the Storwatch Specialist as an example of how to administer the QuickLoop Figure 119 shows the 3534 QuickLoop entry panel Chapter 5 Implementing an IBM managed and unmanaged hub 135 9 113 25 254 0 0 0 0 Figure 119 StorWatch QuickLoop panel In Figure 120 we expanded the number of looplets ports in the private loop by bringing in a QuickLoop partner fcswitch1 You have two options e Select a QuickLoop partner and enter its WWN e Select None 136 Planning and Implementing an IBM SAN witch Admin for fchu
94. 1 Mirror drive member selection 000 cee eee ee eee 320 272 Adding a dedicated spare 1 0 eee 321 273 Mirror drive properties 2 2 ee 322 274 Control Center with Mirror Drive 0000 e eee 323 275 Mirror Drive Properties 0 00 cece ete 324 276 Instant Copy Drive Member Selection 0 2 00ers 325 277 Instant Copy Drive Properties 000 e eee eee 326 278 Control Center with Instant Copy Drive 2 00 eee 327 279 Add Mirror Member display 000 cece eee ee 328 280 Adding drive members toa Mirror 00 0 eee ee 328 281 Mirror drive properties with copy drive attached 329 282 Creating composite drive to be used ina mirror 330 283 Control Center with two composite drives 0 200 ee eee 331 284 Creating mirror drive from two composite drives 332 285 Control Center with mirror drive using two composite drives 333 286 Removing a logical drive 2 2 2 ee 335 287 Mapping a general spare anuanua aaau 336 288 UnMapped composite drives naaasar 337 289 Increasing storage capacity 0 eee 339 290 Increasing throughput es ssa sae aeae a E a ee 339 XV xvi Planning and Implementing an IBM SAN Tables 1 Desin tade dE eera ake ceed notte Ake et ast aa tee EEES 30 Copyright IBM Corp 2000 xvii xviii Planning and Implementing an IBM SAN Preface
95. 100 MB s Each port requires a gigabit interface converter to connect it to each attached node The Fibre Channel Storage Hub supports any combination of shortwave or longwave optical GBICs We show a GBIC in Figure 99 Planning and Implementing an IBM SAN Figure 99 Gigabit Interface Converter The GBICs are hot pluggable into the IBM Fibre Channel Storage Hub which means you can add host computers servers and storage modules to the arbitrated loop dynamically without powering off the Fibre Channel Storage Hub or any connected devices If you remove a GBIC from a Fibre Channel Storage Hub port that port is automatically bypassed The remaining hub ports continue to operate normally with no degradation of system performance Conversely if you plug a GBIC into the Fibre Channel Storage Hub it will automatically be inserted and become a node on the loop if valid Fibre Channel data is received from the device 5 2 4 Hub installation In Figure 100 we show the front panel of the IBM Fibre Channel Storage Hub 2103 H07 Figure 100 FC Storage Hub 2103 H07 front panel The picture shows the front panel with the seven slots for GBICs After GBIC installation they represent the FC Storage Hub ports 0 to 6 Chapter 5 Implementing an IBM managed and unmanaged hub 115 116 5 2 4 1 GBIC insertion The unmanaged hub is as close to plug and play as you are likely to encounter in a SAN installation Of course there is
96. 109 S08 switch We recommend that you utilize a pre installation checklist An example of a pre installation checklist and a description of the GBICs and how to insert them into the switch is detailed in the BM 2109 Model S08 User s Guide SC26 7349 and the IBM 2109 Model S08 Installation and Service Guide C26 7350 To install the SAN Fibre Channel Switch 2109 S08 we followed these steps 1 Power On Self Test The switch is designed for maintenance free operation and supports Power On Self Test POST and diagnostic tests The diagnostic tests determine the switch s status isolate problems and will take approximately two minutes As one of our first steps we powered on the switch and ensured that it ran its POST successfully After the switch has completed the POST the GBIC modules return to a steady state from the flashing state that is shown during the test Chapter 6 Implementing an IBM SAN Fibre Channel Switch 143 2 Setting the IP address The switch is shipped from the factory with a default IP address on the switch of 10 77 77 77 This IP address is noted on a label on the top front edge of the switch This address is for the external Ethernet connection In your environment you will want to utilize your own range of IP addresses and you will have to change the default address to establish a network connection to the switch Your LAN administrator should be able to provide a valid free IP address or a range o
97. 188 E E_Port 128 142 174 237 e business 3 ED 5000 181 EFC login 202 EFC Manager 185 186 EFC Manager Software 198 364 Planning and Implementing an IBM SAN EFC monitoring 204 EFC operational status 209 EFC port number 212 EFC Server 185 EFC server logon 190 effective configuration 160 enabling the Ethernet port 254 Enabling VPS 273 Enterprise Fabric Connectivity EFC server 182 Enterprise Resource Planning 27 Enterprise System Connection 12 E Ports 37 ERP 3 error detection time out value E_D_TOV 243 ESCON 12 16 37 79 ESCON directors 181 ESS 294 296 ESS Copy Server 74 ESS interface adapters 97 ESS Specialist 64 Ethernet 10 184 Ethernet port 122 ethernet port 141 event logs 186 Expansion Port E_Port 182 extended distance buffering 243 external storage consulting groups 25 F F_Port 142 174 182 fabric 139 Fabric backbone 43 fabric parameter 177 Fabric Shortest Path First 40 factory default settings 177 factory settings 140 fault tolerance 172 fault tolerant fabric 173 FC 112 FC AL 15 34 63 101 FCP 13 16 FCP protocols 50 FC PP 101 FC SL 15 FC SW 15 34 63 101 fiber optic 13 31 Fibre Channel 11 13 268 Fibre Channel adapters 76 Fibre Channel Arbitrated Loop 15 101 Fibre Channel Director 37 Fibre Channel HBAs 32 Fibre Channel host 265 Fibre Channel Managed Hub 36 Fibre Channel Point to Point 100 Fibre Channel port attributes 100 Fibre Channel Protocol 1
98. 2 L003 ITSO1 ITSO3 Figure 284 Creating mirror drive from two composite drives Continue through the Mirror Drive Setup Wizard to complete the process as described in 9 5 Mirror drive on page 319 9 7 3 Viewing mirror drive using composite drives 332 With the Mirror Drive Setup Wizard completed you can now view the Control Center window once again as shown in Figure 285 Planning and Implementing an IBM SAN T33Z282 ITSOm T33540 CompMir T33025 ITSOl CompMir 733027 ITSO3 CompMir TOOOOO 35E68229 Figure 285 Control Center with mirror drive using two composite drives In the Mirror Drive window you can see the new mirror drive that was created above and named CompMir In the Composite Drive window you can see that there are still the two composite drives but instead of having Fibre Channel LUN numbers assigned to them they are shown as belonging to a mirror with the name CompMir You can highlight the CompMir drive and click on the Properties button All the same functions that were described in 9 5 2 Mirror drive properties on page 322 are available Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 333 9 8 Reusing logical drives At some point the composite mirror and instant copy logical drives that have created may be no longer required The logical drive can be removed so that the member drives that made up the logical drive can then be used ind
99. 2 LAN free and server free data movement Backup using the LAN moves the administration to centralized tape drives or automated tape libraries However at the same time the LAN experiences very high traffic volume during the backup or recovery operations and this can be extremely disruptive to normal application access to the network Although backups can be scheduled during non peak periods this may not allow sufficient time Also it may not be practical in an enterprise which operates in multiple time zones We illustrate loading the IP network in Figure 13 LAN Backup Restore Today Existing IP Network for Client Server Communications Backup Restore Control and Data Movement E Need to offload busy LANs and servers E Need to reduce backup window m Need rapid recovery solutions Figure 13 LAN backup restore today loading the IP network SAN provides the solution by enabling the elimination of backup and recovery data movement across the LAN Fibre Channel s high bandwidth and multi path switched fabric capabilities enable multiple servers to stream backup data concurrently to high speed tape drives This frees the LAN for other application traffic IBM s Tivoli software solution for LAN free backup offers the capability for clients to move data directly to tape using the SAN A Chapter 1 Introduction to Storage Area Networks 21 future enhancement to be provid
100. 20 90 Largest block of contigu Bay 1 Adapter 2 Port Selected Ranks Cluster 1 Adapter 1 Cluster 2 Adapter 1 Figure 61 Add Volumes 1 of 2 panel with Not Allocated space selected There are different colors to indicate which state the disk groups are in In our case most of the storage is already configured only the two disk groups that we configured before have not been allocated This is indicated with a green color Chapter 4 Configuring the ESS with native Fibre Channel 89 To mark the storage for use in the next step we click on these which highlights these with a border as shown in Figure 61 Then if we press the Next button we are taken to the second step of defining the volumes This is the Add Volumes 2 of 2 panel as shown in Figure 62 Add Volumes 2 of 2 Hostname Netfinity ITSO_1 Port A Adapter 2 IO Bay 1 Free Storage Space Information Total Free Space all Fixed Block storage 442 62 GB Total Free Space selected storage 420 90 GB Largest possible volume size 210 45 GB Volume Definition Volumes to be added Select a Volume Size 0 1 GB 0 2 GB 03GB Number of Volumes O4GB fi Te juuye 0 5 GB Enter 1 to 4 0 6 GB NITAR zl O volumes 0 00 GB Total Remaining Free Space 420 90 GB GB Volume Placement Place volumes sequentially starting in first selected storage area Spread volumes across all selected storage areas Perform Configurati
101. 241 203 Setting domain ID 1 0 ee 243 204 Configure ISL poms recrear ien eree Recall Gai oe alee dhe oe 244 205 Two managed switches from within one EFC Manager 245 200 Fabri VICW soiin pekao Sea wl Rane eset lt een se Wales a bs ketal be 245 207 Topology View with two switches connected and configured 246 208 Changing icon text 0 0 eee 247 209 Interconnected ED 5000 one managed by another EFC Server 248 210 Port List View with two E_Ports 00 c ee eee eee 249 211 Active Zone Set corresponding to Figure 202 on page 241 250 212 SAN Data Gateway configuration 0 eee ee 251 xiii xiv 213 SAN connection port assignment 0000 ee 252 214 IBM Storage Area Network Data Gateway startup 253 215 StorWatch SAN Data Gateway Specialist startup 258 216 StorWatch SAN Data Gateway Specialist server 258 217 StorWatch SAN Data Gateway Specialist initial view 260 218 Selecting from multiple SAN Data Gateways 04 261 219 Expanded Gateway VieW auauua eaae 262 220 SCSI channel expanded viewW 000 e eee eee eee 263 221 SCSI channel data 0 cee ee 264 222 Disk device data 2 2 0 cee eee 265 223 Fibre Channel port data aaaea aaua auaa 266 224 Fibre Channel host data 0 0 0 eee eee 267 225 SCSI channel parameters aaua naana naaa e ee
102. 3 Fibre Channel Storage Hub 36 fibre channel topologies 129 Fibre Connection 12 FICON 12 16 field replacable units FRU 182 field replaceable unit 45 file systems 2 firmware 127 Fixed Block Storage 84 FL_Port 142 174 flash memory 160 flow control 14 FL Port 37 FL Ports 37 Format the Drives 302 F Port 152 frames 14 31 172 FRU 45 210 FSPF 40 G G_Port 210 G_Ports 174 237 Gateway address 189 gateways 10 GBIC 114 141 Generic Port G_Port 182 Get New Mapping 336 Gigabit Interface Converter 114 H HACMP 238 239 hardware zoning 157 166 heterogeneous 32 Heterogeneous Hosts 341 High availability considerations 295 HiPPI 16 homogeneous 32 Homogeneous Hosts 341 hops 37 Host Attributes 81 Host information 279 Host registration 273 host centric 1 Hosts Systems 80 HOSTSW 274 Hub 36 hub 111 hub cascading 128 Hub Specialist 121 hub zoning 127 hubs 10 l I O Priority Queuing 23 IBM 2108 47 IBM Enterprise Storage Server 2105 F20 63 IBM SAN Fibre Channel Switch 284 293 IBM SAN Fibre Channel Switch 2109 808 140 IBM SAN Fibre Channel Switch 2109 816 140 144 146 IBM Storage Area Network Data Gateway 2108 G07 251 increased addressability 36 infrastructure requirements 30 Initialization Method 316 initializeBox 253 in sequence delivery 40 InstallAnywhere 201 Installing additional Routers 339 Installing StorWatch Specialist 257 Installing the SLIC Manager 304 Installing the SLIC R
103. 40 MB s If the system bus delivers 132 MB s or less you will only need to attach one Fibre Channel HBA to the bus to achieve the required performance since two would over run the bus speed If you attach a second HBA it should only be for redundancy purposes Our recommendation is to install one adapter per system bus Another major component of your current assets are the storage systems You may have a variety of internally attached disk devices which will not be relevant in a SAN operation Also you may have externally attached JBODs or RAID disk subsystems and tape drives or libraries which can be utilized within the SAN These current assets have implications for the selection of interconnections to the SAN You may want to support existing hardware which are SCSI or SSA compatible and which will need to be provided with router or gateway connections for protocol conversion to Fibre Channel Planning and Implementing an IBM SAN 2 3 Selecting the topology The most fundamental choice in the design of your SAN is the selection of the most appropriate topology This selection may be colored by the overall approach to SAN planning that your organization wants to adopt The question is top down or bottom up design In other words should you try to design a corporate strategy with a view to implement an enterprise wide SAN or should you address the problem from the perspective of individual departments or user groups and imple
104. 45 46 47 48 49 50 51 52 53 54 55 56 57 58 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 FG host selected iria te skh leo eae eed Beebe DOTS ad aks 78 SCSI host selected 0 0 cece tenes 78 Storage Allocation tabular view 0 0 c eee eee 79 Entry for open system storage 1 2 6 eee 79 Open System Storage panel 0 0 0 cece ee eee 80 Entry to the Modify Hosts Systems panel 000 0 0a 80 Modify Host Systems panel 0000 cece eee ees 81 Adding a FC host adapter port 00 c cece eee 82 Performing the port definition 0 0 c eee ee ee 83 Successfully added host FC port 0 0 0 eee eee 83 Newly configured Fibre Channel host adapter port no volume assigned 84 Fixed Block Storage panel 0 cece ee 85 Changed disk group definition 2 000 cee eee eee 86 Time consuming action warning 0 ce eee ee 86 RAID definition progress window 0 00 eee eee ee 87 Successful disk group configuration 00002 eee eaee 87 Device adapter pair one with four disk groupS 000 87 Entry to the Add Volumes 1 of 2 panel 0 0 00 eae 88 Add Volumes 1 of 2 panel with one Fibre Channel host port selected 88 Selecting an ESS Fibre Channel port 0 0000 e ee eee 89 Add Volumes
105. 5 255 255 0 Role Principal FC IP none State Online FCNM none Firmware v2 1 3 Gateway 9 113 265 2654 Figure 108 IBM 3534 Managed Hub 5 4 1 1 Setting the IP address using the Ethernet port We show how to set the hub IP address by using the Ethernet port 1 Attach the LAN to the front panel of the hub by plugging an existing Ethernet 10 100BaseT LAN cable to the RJ 45 connector on the front of the hub 2 Turn on the hub by plugging it into an electrical outlet Make sure that the power cord is fully seated into the front of the unit and the green ready LED is on Wait two minutes for diagnostics to complete 3 From a LAN attached server type the Telnet IP address For example issue the command Telnet 9 113 25 6 Chapter 5 Implementing an IBM managed and unmanaged hub 123 124 If this is the initial installation use the default IP address found on the label on the top left corner of the 3534 Managed Hub If the 3534 Managed Hub has been installed before using the IP address on the label continue using the current address from the label If the IP address on the label was not used you will need to get the current IP address from the system administrator After the Telnet command has been issued the hub will respond as shown in Figure 109 At each prompt type in the information as shown and press Enter The default login is Admin and the default password is password ois Telnet 9 113 25 6 Connect Edit Te
106. 52 5E Emulex 10 00 00 00 C9 20 CA E6 Figure 188 Viewing zone members after creating a zone To assign the newly created zone NT_Zone_1 in the zone set we created before we drag and drop the zone from the Zone Library to the Zones in Set on the left side in the window This displays the zones assigned for this set as shown in Figure 189 Chapter 7 Implementing the McDATA ED 5000 227 New Zone Set ITSO_Zone_Set_1 _SaveZone Set Cancel R Figure 189 Assigning a zone to a zone set Now we save the zone set as a member in the Zone Set Library with the Save Zone Set button The Zone Set Library now looks like Figure 190 We have one zone set ITSO_Zone_Set_1 with one zone NT_Zone_1 We have one ESS port and two hosts within the zone one host attached with an Emulex adapter and one host detached with a QLogic adapter 228 Planning and Implementing an IBM SAN Zone Sets AITSO_Zone_Set_1 ANT_Zone_1 RPKAG3_C1B2A1 QLogic 20 00 00 E0 88 00 52 5E Emulex 10 00 00 00 C9 20 CA E6 Figure 190 Zone Set Library with one zone set and one zone with two hosts The ITSO_Zone_Set_1 zone set with NT_Zone_1 zone is now defined However it is not active We can define more zones in the zone set or create other zone sets To finish our zoning example we will activate the zone set now This is also done using the Action button and by clicking Activate With this action we are prompted to start or to cancel the
107. 53 Bad End of Frames 8 Frames Received 602593 Ened Errs Outside Frames 255 C2 Frames Received 765 C3 Frames Discarded 0 C3 Frames Received 84737 LIP Ins 0 Link Control Frames Received 258557 LIP Outs 0 Mcast Frames Received 0 Last LIP Receiwed 00 00 Mcast Timeouts 0 Frames Rejected 765 Mcast Frames Transmitted 0 Frames Busied 0 Time R_RDY Priority 0 Link Failure Ti Time BB_Credit Zero 0 Loss of Sync 292 Encd Errs Inside Frames 4 Loss of Signal 30 Frames with CRC Errs 8 Figure 112 3534 E_Port 5 7 QuickLoop Fibre Channel provides three distinct interconnection topologies The three fibre channel topologies are e Point to point e Arbitrated loop e Switched referred to as a fabric The IBM 3534 Managed Hub is a Fibre Channel Arbitrated Loop device A simple loop configured using a hub is shown in Figure 113 Chapter 5 Implementing an IBM managed and unmanaged hub 129 Figure 113 Arbitrated loop There are two different kinds of loops the private and the public loop 5 7 1 Private loop The private loop does not connect with a fabric only to other private nodes using attachment points called Node Loop Ports NL_Ports A private loop is enclosed and known only to itself NL_Ports can only attach to other NL_Ports or to Fabric Loop Port FL_Ports In Figure 114 we show a private loop Tape Drive Library Server Disk Storage Figure 114 Private lo
108. 55 time outs 41 Tivoli 21 Tivoli SAN Manager 24 Tivoli SANergy 341 Tivoli Storage Manager 24 topology 101 track format 85 transport topologies 14 U undefined state 102 Uniform Resource Locator 64 unmap 336 UnMapped 336 URL 64 196 user rights 194 Users 71 V Vicom Fibre Channel SLIC Router 36 299 virtual channel parameters 177 Virtual Private SAN 272 279 283 292 Virtual Private SAN VPS 271 Virtual Tape Server 23 volumes sequentially placed 91 VPS 275 285 295 VPS Registration Service 274 VTS 23 W Wall Street 26 WAN 1 World Wide Port Name 77 WWN 206 WWNN 65 WWPN 77 80 81 99 100 Z zone 155 Zone Alias Settings 164 zone aliases 160 169 zone configurations 160 zone members 159 Zone Name 165 zone objects 156 Zone Settings 164 Zoning 272 283 293 zoning 29 36 155 252 297 369 370 Planning and Implementing an IBM SAN IBM Redbooks review Your feedback is valued by the Redbook authors In particular we are interested in situations where a Redbook made the difference in a task or problem you encountered Using one of the following methods please review the Redbook addressing value subject matter structure depth and quality as appropriate e Use the online Contact us review redbook form found at ibm com redbooks e Fax this form to USA International Access Code 1 914 432 8264 e Send your comments in an Internet note to redbook us ibm com Document Number
109. 5E68229 35EZ2ZD0B 35E69318 35E6848D 3S5EZ2ZD6C 3S5E6840E arrears Figure 260 Control Center window 310 In the Physical Drive box the drives that are on the SSA loop can be seen This window will be useful as you start to create mirrors and composite drives because it provides a summary of all drives Planning and Implementing an IBM SAN 9 3 1 Drive properties You can get detailed information on each drive Select the drive so that it is highlighted and then select Properties A dialog box will appear with the drive s information as in Figure 261 Physical Drive Properties x Target rocooo Number SSA ID SSA Node EGI Capacity MB Vendor Model Number Product Type 35E68229 2 SSA LUN 0 33 FC LUN 0 8689 Vital Product Data BM Unit Serial Number oB DAVC Microcode OK Cancel Bhange tis 68050021 0016 Used As i is currently used as a public mapped physical drive Figure 261 Disk drive properties Here you can see its SSA attributes its Fibre Channel attributes and its model type and serial number By selecting the Identify button the LED on the selected drive will begin to flash 9 3 2 Router properties To view the information on the Router go to the toolbar and select Properties gt SLIC Properties As shown in Figure 262 you will see the serial number of the Router its ID that was set in SW1 and its supporte
110. 66 4516 8e 68 2 Ge 00066 45166e 68 2 OF 66066 45166e 68 21666066 4516 8e 68 211666066 45166e 68 2126606 0 4516 8e 68 2136606 0 4516 Ge 68 21466806 6 45166e 68 215 66066 45166e 68 21666066 45166e 68 21766066 45166e 68 218 66066 45166e 68 21966066 45166e 68 21266606 6 45166e 68 21b66066 45166e 68 21c 6606 0 4516 8e 68 2106606 6 45166e 68 21e 66066 45166e 68 Figure 237 Service terminal display of device map The numbers on the left are the assigned LUN numbers and note that number 0 has been assigned to the Gateway The other columns contain the device type the SCSI channel it is connected to on the SAN Data Gateway and the actual target and LUN ID 8 5 2 7 Setting host access to LUNs To view and set host access to particular LUNs access the Virtual Private SAN Access Settings by selecting Controls gt Access Options gt Virtual Private SAN This window will show all the hosts that have registered to the Gateway To allow a host to access a particular LUN place a check mark in the row that corresponds to the host To disable access the square must be clear without a check mark The check mark is toggled on and off by clicking in each square Figure 238 provides an example of LUN masking Chapter 8 Implementing the SAN Data Gateway 281 PA virtual Private SAN Access Settings ACTIVE Assigned LUNs Ml Off Line 47 18 19 20 21 22 23 24 25 26 27 28 29 30 unknown unknown unknown 1000000
111. A SAN and Storage Solutions Center team He joined IBM more than 28 years ago in the days of System 360 and is now enjoying the frenetic world of SAN He has 13 years experience in IBM s storage business He held a number of sales and marketing roles in different branch offices in Germany Since 1988 he has been based in the EMEA SSD Customer Executive Briefing Center in Mainz Germany Uwe holds a diploma in National Economics He is a regular speaker on SAN and storage related topics for IBM customer groups and external conferences in Europe Thomas Jahn is an IT Specialist within the IBM Storage Subsystems Division in Germany He has three years of experience providing technical support in IBM Thomas has provided technical support for networking and server consolidation on OS 390 UNIX for IBM and its customers He is currently engaged in providing support for open systems storage across multiple platforms and a wide customer base He holds a Dipl Ing degree in Computer Science from the Staatliche Studienakademie Sachsen Thanks to the following people for their invaluable contributions to this project Lisa Haut Mikkelsen IBM Storage Subsystems Division Robert Azevedo IBM Storage Subsystems Division Planning and Implementing an IBM SAN Jim Baldyga Brocade Mark Bruni IBM Storage Subsystems Division Joe Emery Vicom Systems Inc Glenda Fuller IBM Storage Subsystems Division Malkit Hayun IBM Storage Subsystems Division
112. C26 7294 e Introduces the product and lists the features you can order It also provides guidelines on planning for the installation and configuration of the ESS IBM Enterprise Storage Server User s Guide 2105 Models E10 E20 F10 and F20 SC26 7295 e Provides instructions for setting up and operating the ESS Chapter 4 Configuring the ESS with native Fibre Channel 109 110 IBM Enterprise Storage Server Host Systems Attachment Guide 2105 Models E10 E20 F10 and F20 SC26 7296 e Provides guidelines for attaching the ESS to your host system IBM Enterprise Storage Server SCSI Command Reference 2105 Models E10 E20 F10 and F20 SC26 7297 e Describes the functions of the ESS It provides reference information for UNIX and AS 400 hosts such as channel commands sense bytes and error recovery procedures IBM Enterprise Storage Server System 390 Command Reference 2105 Models E10 E20 F10 and F20 SC26 7298 e Describes the functions of the ESS and provides reference information for System 390 hosts such as channel commands sense bytes and error recovery procedures ESS Web Interface User s Guide for ESS Specialist and ESS Copy Services SC26 7346 e Provides instructions for using the IBM StorWatch Enterprise Storage Server Web interface ESS Specialist IBM Storage Solutions Safety Notices GC26 7229 e Provides translations of the danger notices and caution notices that IBM uses in ESS publications IBM
113. Composite drive 9 4 2 Composite drive properties If you view the Control Center again by selecting Tools gt Control Center as shown in Figure 269 the newly created Composite drive is listed in the Composite Drive box Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 317 Control Center T33025 ITSsol Tooooo T00003 T00004 TO000S T00006 T00007 35E68229 35E6848D 35EZZD6C 35E6840E 35E68462 3SE684AE Figure 269 Control Center with Composite drive 318 Select the Composite drive and then click on the Properties button the Composite Drive Properties dialog box opens as shown in Figure 270 Planning and Implementing an IBM SAN Composite Drive Properties Ea Composite Drive Selection Composite Drive Members Cancel frar get SSA UID Function ee m Composite Drive Tool Box TOOOOL 35EZZD0B A ToooOZ2 35869318 A Identify Member Identify Composite Composite Drive Characteristics Capacity MB 17378 PUBLIC FC LUN fa Drive Name itso Change Number of Members 2 Figure 270 Composite Drive Properties Here you can find information on the Composite Drive about the member drives that make up the composite as well as the Composite Drive characteristics Clicking on the Identify Composite button will cause the LED on the actual SSA drives that belong to the Composite to flash If a member drive is selected an
114. Creating a second composite drive 330 To provide an example of a mirror using only composite drives another composite drive is required The example shown in Figure 282 shows that drive 6 and 7 were used to create another composite drive Composite Drive Setup Composite Drive Members Selection TOOOOOLO00 35E68229 TOOO06 35E68462 TOO006 LOOG _ 35E684 T00007 35E6844E TOOO07 LOO 35E684AE Figure 282 Creating composite drive to be used in a mirror Follow the steps to create a composite drive as described in 9 4 Composite drive on page 314 Once created you can view the Control Center window by selecting Tools gt Control Center from the toolbar Figure 283 shows that there are now two composite drives Planning and Implementing an IBM SAN T33282 ITSOm T33025 ITSOl T33027 ITSO3 TOOOOO 35E68229 Figure 283 Control Center with two composite drives 9 7 2 Creating the mirror The mirror can now be created by selecting Tools gt Mirror Drive Setup Wizard When the Member Selection window appears select the composite drives as members of a mirror Figure 284 shows where composite drives ITSO1 and ITSO3 are selected as members of a mirror Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 331 700000 LOOO 733025 L001 wall 7 L003 35E68229 ITSO1 ITSO3 Mirror Drive Setup Wizard Mirror Drive Members Selection 733025 L001 7330
115. E7 C7 KATE_FCSO Emmulex 10 00 00 00 C9 20 E7 F3 Figure 198 Zone Sets with two zone sets and two zones in one set By activating or deactivating this zone set we activate or deactivate both zones at the same time The result of doing this after confirming that we want to start the activation and selecting OK on the resulting message means that we now have a Zoning View with an Active Zone Set which looks like Figure 198 236 Planning and Implementing an IBM SAN McDATA Fabric Manager olx rl Zoning View Default Zone Enabled View ctive Zone Set Ba _JITSO_Zone_Set_1 Configure INT_Zone_1 RPKAS3_C1B2A1 GLogic 20 00 00 E0 88 00 52 5E E Emulex 10 00 00 00 C9 20 CAE6 JAX Zone_1 Help Emulex10 00 00 00 C9 20 DC 71 Emulex 10 00 00 00 09 20 E7 C7 4i KATE_FCSO Emulex 10 00 00 00 C9 20 E7 F3 Close Save active zone set as N Deactivate zone set Figure 199 Zoning View with active zone set containing two zones We successfully created and activated a zone set with two zones 7 5 Cascading with McDATA building a multiswitch fabric The focus of McDATA was primarily high availability and reliability not connectivity and extended fabrics However times change and with the introduction of E_Port capability of the McDATA G_Ports it is now possible to cascade McDATA switches In the following sections we cover these topics e 7 5 1 Multiswitch fabric c
116. EG alate te aS aigtane gate Be a mareena tes ix Wables os ests dadte Pal Sener dors Meld 64 ead nada Stags xvii Prelate seriar emea ath keh EAEN e eae ERRE xix The team that wrote this redbook 00 cee eee eee XX Comments welcome 0 0 c eect ete eens xxii Chapter 1 Introduction to Storage Area Networks 1 1 1 The need for a new storage infrastructure 005 1 1 2 The Small Computer Systems Interface legacy 4 1 3 Storage network solutions 2 0 0 0 ee ee 8 1 3 1 What network attached storage is 0002 00 ee 9 1 3 2 What a Storage Area Network iS saasaa nanana aaran 10 1 3 3 What about ESCON and FICON 0000000 12 1 4 What Fibre Channel iS 0 00 00 cee eee 13 1 5 The business benefits of a Fibre Channel SAN 17 1 5 1 Storage consolidation and sharing of resources 17 1 5 2 Data sharing osion a a aa a a aia E ee 19 1 5 3 Non disruptive scalability for growth 005 20 1 5 4 Improved backup and recovery snanu naana aaa 20 1 5 5 High performance 0 0 eee 22 1 5 6 High availability server clustering 000000 22 1 5 7 Improved disaster tolerance 0 000 00 ee eeeaee 22 1 5 8 Allow selection of best of breed storage 23 1 5 9 Ease of data migration 0 0 00 eee ee 23 1 5 10 Reduced total costs of ownership 0
117. Enterprise Storage Server Virtual Tape Server FCSS Magstar Serial Disk Figure 24 The IBM Enterprise SAN vision 50 Planning and Implementing an IBM SAN In a fast moving world there is often a tendency to hold back and wait for something better which may be just around the corner But those who wait are sometimes left behind in the race for competitive advantage The race is on Now is the time to join in Chapter 2 SAN planning and design considerations 51 52 Planning and Implementing an IBM SAN Chapter 3 Implementing Fibre Channel host adapter cards In this chapter we will describe the steps involved in implementing Fibre Channel host adapter cards in a supported environment This chapter lists the steps necessary to implement Qlogic Fibre Channel cards in Windows NT and RS 6000 host systems e 3 1 Installing QLogic host adapter cards on a Windows NT system on page 53 e 3 2 Installing Fibre Channel host adapter cards on a RS 6000 system on page 58 For more general information or technical information on employment and application of these adapters refer to the appropriate Fibre Channel adapter manufacturer 3 1 Installing QLogic host adapter cards on a Windows NT system In the following sections we cover these topics e 3 1 1 Installing the QLogic 2100F adapter card on page 53 e 3 1 2 Installing the QLogic 2200F a
118. Figure 138 we show the SUNzone members Planning and Implementing an IBM SAN Zone Administration Netscape Zone Alias Settings Zone Settings Zone Config Settings Zone Name suNzone Create Zone Delete Zone Rename Zone SUNzone Members Add Member Remove Meme Add Other Apply Cancel Done Figure 138 Creating zone names 6 4 2 3 Zone aliases Zone aliases simplify repetitive port numbers entries or World Wide Names A zone alias is a C style name for one or more port numbers or World Wide Names For example the name ESS can be used as an alias for 10 00 00 00 c9 20 db 0c From the panel shown in Figure 139 click the Zone Alias Settings tab enter an Alias Name and click the Create Alias button to create an alias Use the Add Member field to add or use the Remove Member field to remove an alias In Figure 139 we show the alias window for ESS which we have defined as an alias for WWN 10 00 00 00 c9 20 db 0c Chapter 6 Implementing an IBM SAN Fibre Channel Switch 169 Zone Alias Settings Zone Settings Zone Config Settings Alias Name Ess Create Alias Delete Alias Rename Alias ESS Members Hose 10 00 00 00 9 20 db 0 lt Remove Member 10 00 00 00 c9 20 b5 6b 50 05 07 63 00 c0 0c 00 Aad Ofer 10 00 00 00 9 20 db 0 10 00 00 00 c9 21 23 48 1 0 00 00 00 c9 20 d4 9F 10 00 00 00 c9 20 04 9f B 20 00 00 e0 8b 00 52 5e g
119. G5 Server e Many connections e Fewer connections e Many channels ports cables Fewer channels ports cables patch patch panel ports etc panel ports etc e 20MB sec link rate half duplex e 100MB sec link rate full duplex Supports gt 500 I O s sec channel e Supports gt 4000 I O s sec channel Intermix of large amp small block e Intermix of large amp small block data data on one channel not advised on one channel is feasible e 1K unit addresses channel e 16K unit addresses channel e 1K unit addresses CU e 4K unit addresses CU e Performance degradation at9KM Little performance degradation at 100KM Figure 9 FICON enhances ESCON 12 Planning and Implementing an IBM SAN The S 390 FICON architecture retains ESCON topology and switch management characteristics FICON channels can deliver data rates up to 100 MB second full duplex and they extend channel distances up to 100 kilometers More storage controllers and devices can be supported per FICON link relieving channel constraints in configuring S 390 processors The FICON architecture is fully compatible with existing S 390 channel command words CCWs and programs But most importantly FICON uses Fibre Channel for transportation of data and therefore in principle is capable of participating with other platforms UNIX Windows NT Novell Netware and so on in a Fibre Channel enterprise SAN However this
120. Granting additional rights to the EFC Manager installation software 200 158 Starting the installation of the EFC Manager 5 201 159 EFC Manager version 0 anuanua ete 202 160 EFC Manager icon 0 0 ete 202 161 EFC Manager login window 0 naana aa 203 162 Product View with no devices installed 0 000 0 eee 204 163 Adding new product 0 ce eee 205 164 Adding new ED 5000 with its IP address 200055 205 165 New ED 5000 icon 0 eee eens 206 166 Configuring nicknames 000 0 cee ee 207 167 Nickname window with nicknames assigned 207 168 Node List View with some nicknames 00 0 02 eee eee 208 169 Hardware view of the ED 5000 0 eee ee 209 Planning and Implementing an IBM SAN 170 Setting the switch identification aaua aaae 210 171 Switch information 00 0000 cc ees 210 172 Port Card View with Port Properties 000 c ee eee eee 211 173 Configure Ports window 000 aaua 212 174 Link Incident Log oo idrica eb eee as ieee ae Fe Sa 213 175 Port list View is asepi ei nnana a KAALE eee 214 176 Set Online State uaaaaaaaa tees 215 177 Configure operating parameters 0 0 00 cee eee 215 178 Opening Fabric View of the EFC Manager 217 179 Topology View of the Fabric Manager 2 0000000 218 180 Zoning
121. Home Search Netscape Print Security Stop T Bookmarks A Location http 9 113 24 57 Instant Message WebMail E Contact People YellowPages Download J FindSites C4 Channels Figure 133 Fabric view 2 From this view we clicked the Zone administration button at the bottom Once this was done we were presented with the view shown in Figure 134 Chapter 6 Implementing an IBM SAN Fibre Channel Switch 163 Zone Administration Netscape Figure 134 Zone alias settings From this panel we can create zones in a direct way using Zone Settings or Zone Alias Settings to create aliases Zone aliases are optional When familiarizing yourself with zoning it may be easier to start with Zone Settings instead of Zone Alias Settings 3 We selected Zone Settings by clicking the middle tab We were presented with the view shown in Figure 135 164 Planning and Implementing an IBM SAN Zone Administration Netscape Figure 135 Zone settings view As you can see there are no names in the Zone Name field and we have no entries in the member field in the right hand panel The 8 ports of the switch are shown and the WWN field will contain at least one entry which is the WWN of the switch itself 4 We created the first zone and named it NTzone restricting storage access to the Netfinity host This is shown in Figure 136 Chapter 6 Implementing an IBM SAN Fibre Channel S
122. IBRE1 Host Type NT 4 0 SP4 Host Connection ScsiPortt 1 1 0 Emulex Figure 247 Hosts and a switch on a Gateway port By selecting and highlighting the new NT host we can see its information that was automatically sent by the host registration process The configuration with two different hosts connected to the switch as described previously is shown in Figure 248 Server Server Server EA 35 NT 33 AIX unknown FIBRE1 plymouth 3 noon 5 Fibre Channel 7 IBM i scsi Switch 1 4 Gateway H i2 i3 ESS Figure 248 Adding two heterogeneous hosts to the switch Chapter 8 Implementing the SAN Data Gateway 291 8 6 1 5 Device access To have the new NT host access some of the devices you need to set up the VPS Access parameters by selecting Controls gt Access Options gt Virtual Private SAN Figure 249 shows the switch and the two hosts in the VPS database As plymouth was previously connected direct to the Gateway its settings have been maintained but now that it is reconnected it is back online The NT host does not have any LUN access yet FA virtual Private SAN Access Settings ACTIVE Assigned LUNs fdevwitsesiO Oxef 0 Emulex 10000000 c9221802 unknown SWITCH unknown 20fd0060 69201d4e FIBRE1 NT SesiPort1 1 1 0 Emulex 10000000 c920cae6 New Host Data Host Na
123. Only Connection Point to Point Only Max Speed 100 Mb sec Status Ready point to point N_Port Sos SSS SSS SS Sos Figure 242 Switch port login You can also check the port on the switch by accessing the switch configuration from a Web browser to ensure that the port has registered and is communicating properly Figure 243 shows that in this example port 7 of the switch was used to connect to the Gateway Note that the WWPN is the same as in the Specialist window and that the port type is F_port For more information on the switch refer to Chapter 6 Implementing an IBM SAN Fibre Channel Switch on page 139 E Port Information for IBM_2109_FC_Switch2 o 1 2 3 4 sle7 Port World Wide Name 20 07 00 60 69 20 1d 4e Port Status Online Port Module for GBIC Module id Port Type F Port PortStats Figure 243 IBM SAN Fibre Channel Switch port settings Chapter 8 Implementing the SAN Data Gateway 287 8 6 1 3 Changing the switch information From the VPS Access Settings window it is possible to change the unknown information of the switch Select Controls gt Virtual Private SAN and select the entry that has the WWN of the switch You can now change the information to further describe the switch or other pertinent information if desired This is shown in Figure 244 PA virtual Private SAN Access Settings ACTIVE Assigned LUNs unknown unknown unknown 10000000 c921da0
124. Planning and Implementing an IBM SAN Review planning considerations associated with Fibre Channel products Learn how to build an IBM SAN environment Set up and administer zoning Jon Tate George DeBiasi Uwe Hofmann Thomas Jahn ibm com redbooks Redbooks International Technical Support Organization Planning and Implementing an IBM SAN November 2000 SG24 6116 00 Take Note Before using this information and the product it supports be sure to read the general information in Appendix A Special notices on page 343 First Edition November 2000 This edition applies to components programs architecture and connections between multiple platforms and storage systems and a diverse range of software and hardware Comments may be addressed to IBM Corporation International Technical Support Organization Dept QXXE Building 80 E2 650 Harry Road San Jose California 95120 6099 When you send information to IBM you grant IBM a non exclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you Copyright International Business Machines Corporation 2000 All rights reserved Note to U S Government Users Documentation related to restricted rights Use duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corp Contents Figures o Ss iaeaea eed chatat aerate R
125. Planning and Implementing an IBM SAN N IBM Copy Services olx Connecting to Copy Services Server please wait Jim Unsigned Java Applet Window Figure 36 Connecting to Copy Services Server Once connected we get the introduction panel for all copy service configurations as shown in Figure 37 Seascape Copy Services Netscape File Edit View Go Communicator Help 2 32geudcak Ps G7 Bookmarks At Location https 9 113 24 141 html sCopyServices html StorWatch Enterprise Storage Server Copy Services Solutions Welcome to IBM StorWatch Copy Services Volumes Storage Servers Configuration Exit Copy Services Copyright IBM Corporation 1998 2000 Licensed Materials Property of IBM All Rights Reserved IBMis a registered trademark of the IBM Corp U S Goverment Users Restricted Rights Use duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp Figure 37 ESS Copy Services introduction panel Copy Services will be covered in future IBM Redbooks 4 1 2 Viewing the Storage Allocation of the ESS Using the Storage Allocation button we get the Storage Allocation Graphical View panel as shown in Figure 38 Chapter 4 Configuring the ESS with native Fibre Channel 75 Storage Allocation Storage Allocation Graphical View Clear View View All Storage Tabular View 2 lick on a Host or Array to see paths YY Divroye Open System Siorag
126. Press the button or the v button to increment or decrement the current field Holding the button down will make this happen rapidly The numbers in the field will wrap to 0 from 255 or from 255 to 0 depending on whether you are incrementing or decrementing This can help you get to the desired value quickly When you have all fields set to the desired value continue with step 16 16 Press the gt button and the switch will respond Accept Y N Pressing the lt button will indicate Yes Pressing the gt button will indicate No e If you say no the switch will again respond Domain and you can restart the process e If you say yes the switch will respond Updating the Config This will cause the new address to be stored After the switch has made the change it will again display Domain 17 Respond by pressing the lt button You have completed installation of the 2109 S16 switch To perform a quick check of the switch s Fibre Channel ports follow the switch installation verification process Switch installation verification 1 Power down the switch Power up the switch if dual power power up both supplies Verify that the associated power supply LEDs are on Wait two minutes while POST diagnostics run a A OO N Verify that the switch ready LED is on Chapter 6 Implementing an IBM SAN Fibre Channel Switch 149 6 Plug the appropriate wrap connector black for shortwave and gray for longwave into ea
127. Redbook Title SG24 6116 00 Planning and Implementing an IBM SAN Review What other subjects would you like to see IBM Redbooks address Please rate your overall O Very Good OGood O Average O Poor satisfaction Please identify yourself as O Customer O Business Partner O Solution Developer belonging to one of the following groups O IBM Lotus or Tivoli Employee O None of the above Your email address The data you provide here may be used to provide you with information from IBM or our business partners about our products services or activities O Please do not use the information collected here for future marketing or promotional contacts or other communications beyond the scope of this transaction Questions about IBM s privacy policy The following link explains how we protect your personal information ibm com privacy yourprivacy Copyright IBM Corp 2000 371 D 5 D 2 D D r amp D z U gt e 0 5 spine 0 475 lt gt 0 875 250 lt gt 459 pages Planning and Implementing an IB SAN Review planning considerations associated with Fibre Channel products Learn how to build an IBM SAN environment Set up and administer zoning This book is a follow on from the IBM Redbook Designing an IBM Storage Area Network SG24 5758 In that book we introduced Fibre Channel basics
128. SCSI Channel 1 2s ae SAN Connection 1 y SAN Connections v v Check the SCSI Channel box to allow Access OK Cancel Figure 228 IBM Storage Area Network Data Gateway channel zoning 8 5 2 Virtual Private SAN 272 The IBM Storage Area Network Data Gateway also provides LUN masking through a feature called Virtual Private SAN VPS This provides a granular restriction of host access to specific LUNs while utilizing and sharing the same connectivity paths through the SAN Data Gateway VPS keeps track of Fibre Channel hosts by using their unique World Wide Name In this way if a switch is attached the Gateway will also keep track of it because the switch also has a unique World Wide Name VPS also includes a host registration service so that when a host is attached its unique WWN is registered to the Gateway to provide information on the host and its status Planning and Implementing an IBM SAN 8 5 2 1 Enabling VPS The Virtual Private SAN feature has to be enabled because it comes disabled from the factory To enable it make sure the desired SAN Data Gateway is highlighted if more than one appears in your Specialist window Select Controls gt Feature Enable gt Virtual Private SAN as shown in Figure 229 A dialog box appears requesting the entry of a license key number For units with serial numbers lower than 1300600 you are required to get a license key that is matched to the s
129. Switch 177 178 7 Make sure the two switches are working in different domains This is mandatory The complete configdefault command sequence is shown in Figure 145 Connect Edit Terminal Help route stickyRoutes rpc rstatd 6 rpc rusersd 8 xlativeModeDisable 6 IBM_2169 FC _Switch admin gt aliasshow There is no entry in the Local Alias Server IBM_2169 FC _Switch admin gt aliasShow There is no entry in the Local Alias Server IBM_2169 FC Switch admin gt switchdisable IBM_2169 FC _Switch admin gt Fabric OS tm Release v2 1 7 IBM_2169 FC Switch2 admin gt switchdisable IBM_2169 FC Switch2 admin gt configdefault Committing configuration done IBM_2169 FC_Switch2 admin gt switchenable IBM_2169 FC_Switch2 admin gt 16 9 8 7 6 5 4 3 2 41 fabric Principal switch fabric Domain 1 IBM_2169 FC Switch2 admin gt Figure 145 Setting default switch configuration Should you choose to enter your own configuration parameters be aware that these can significantly affect switch performance The default factory settings should not be changed unless you are an experienced SAN administrator and you have recorded the original settings before you make the changes The configDefault command is used to reset some of the switch configuration values to their factory default values In addition this command configures the switch to boot from its internal firmware if it has been previously configured to boot from the netw
130. Switch Specialist setting up zoning and methods for managing the IBM 2109 S16 switch remotely IBM 2109 S16 Switch Service Guide SC26 7352 e This guide introduces the product and lists the features you can order It also provides steps for the installation configuration and servicing of the IBM 2109 S16 switch Translated External Devices Safety Information SA26 7003 e This book provides translations of the danger and caution notices used in IBM 2109 Switch publications Electrical Safety for IBM Customer Engineers S229 8124 To get specific details on models and firmware that the switch supports refer to this Web site http www ibm com storage fcswitch Planning and Implementing an IBM SAN Chapter 7 Implementing the McDATA ED 5000 Since the early 1990 s McDATA has provided IBM with fiber switches for the S 390 world the ESCON directors They have built up experience in creating the highly available and reliable equipment necessary to meet the demands of S 390 environment Any SAN component needs to be as reliable and as available as the large systems to which they are attached With the McDATA Enterprise Director ED 5000 ED 5000 and its Fibre Channel interface McDATA has moved this proven technology towards the open market With open systems closing up on 8 390 systems choosing the McDATA switch allows for enterprise class availability and performance that is usually found in mission critical data centers In th
131. T with an EMULEX LP7000 FC card e Netfinity and the ESS with standard FC cables connected to the switch e StorWatch Enterprise Storage Server Specialist to add our Netfinity host to the other hosts already defined to the ESS Support matrix A host and storage support matrix for the IBM 2109 S08 S16 switches with host bus adapter information can be found on the Web at http www storage ibm com hardsoft products fcswitch supserver htm Planning and Implementing an IBM SAN The latest list of ESS supported servers also gives you host adapter card information This is found on the Web at http www storage ibm com hardsoft products ess supserver htm 6 2 4 2 Defining a host to the ESS This is described in 4 1 4 Defining a new host FC port with its WWPN on page 80 and the following topics assume that this has been performed The nickname that we used for the host is TSO 6 2 4 3 Configuring the host adapter This has been described in 4 1 7 Configuring an ESS Fibre Channel port on page 96 and we will assume that this has been performed and represents the topology that is employed in your environment Remember that for a SAN fabric solution you must select Fibre Channel Point to Point on the ESS 6 2 4 4 Assigning array capacity This is described in 4 1 5 Configuring disk groups on page 84 6 2 5 SAN installation verification If everything has been configured correctly and all the parameters are set appro
132. UNs should be assigned to each host Volumes within the ESS assigned to a particular SCSI port should be of the same host type 8 6 2 Connecting a McDATA Enterprise Fibre Channel Director 294 The McDATA Director can be used to attach to the SAN Data Gateway The connection of the McDATA Director to the SAN Data Gateway is similar to the description provided previously in 8 6 1 Connecting an IBM SAN Fibre Channel Switch on page 284 The difference that can be seen when using the McDATA Director is that once connected the Director does not register the WWPN of the port connected to the Gateway visually to the VPS database Figure 251 provides an example of where a McDATA Director was connected to the Gateway on Fibre Channel port 4 Planning and Implementing an IBM SAN PA Storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc File View Admin Tools Controls Help WB 9 113 24 110 ITSO SAN Data Gateway lt 3 SCSI Channel 1 lt 3 SCSI Channel 2 lt 3 SCSI Channel 3 lt SCSI Channel 4 ae o Fibre Channel 4 Channel Description Fibre Channel 4 World Wide Port Name 20040060 45160E08 Serial Number 00010072 Media Short Wave Optical Firmware Revision 2 01 11 Current SAN Data Gateway ITSO IP Address 9 113 24 110 Port Type point to point N_Port Loop ID Frame Size 2048 Port Mode Public Target Only Connection Point to Point Preferred Max Speed 100 Mb sec
133. View of the Fabric Manager 00 e eee eee 219 181 Example for McDATA Zoning 0 0 cee eee 220 182 NT zone with MCDATA 000 eet eee 221 183 Actions for Zone Sets 0 0 ee 222 184 Defining a new zone Set 0 anaa cette 223 185 Adding a detached Fibre Channel node 224 186 Defining the zone sars kegi aa a OE a RE eee 225 187 Members in zone based on WWN and port 00000 226 188 Viewing zone members after creating a zone 00 227 189 Assigning a zone to a zone Set 0 0000 eee 228 190 Zone Set Library with one zone set and one zone with two hosts 229 191 Activate Zone Set complete 0 eee eee 229 192 Active Zone Set shown in the Zoning View of the fabric manager 230 193 Saving zone set with differentname 0 0 000e0 eee 231 194 NT and AIX zones with MCDATA 0 000 cece eee 232 195 Modify zone set 00000 eee 233 196 Defining an AIX Zone 1 eee 234 197 Assigning the AIX zone to the zone set 000000 0c aee 235 198 Zone Sets with two zone sets and two zones in one set 236 199 Zoning View with active zone set containing two zones 237 200 High Availability and disaster recovery with MCDATA 239 201 Extended High Availability and disaster recovery with McDATA 240 202 Multi switch fabric with MCDATA 0000 eee eee eee
134. Wide Name Nickname 10 00 00 00 C9 20 48 CD RPKAS3_C1B3A1 10 00 00 00 09 20 CA 83 KATE_FCSO 10 00 00 00 C9 20 DA 2B RPKA93_C1B2A1 ttt wate ow cen Figure 167 Nickname window with nicknames assigned Chapter 7 Implementing the McDATA ED 5000 207 In our simple case it might be easier to work with the WWN and to skip the task of assigning nicknames However as more devices attach maintaining the fabric with names is more convenient than figuring out which WWN belongs to which machine After assigning the nicknames the Node List View shows the names of those that are currently attached as shown in Figure 168 There are not many devices attached but with a growing SAN it becomes more and more important to distinguish between the nodes W ED 5000 ED 5000 Sw Figure 168 Node List View with some nicknames 7 3 3 Configuring the ED 5000 with the Product Manager Now we can configure the ED 5000 We click on the ED 5000 icon in the Product View This opens the Product Manager in its own window with the Hardware View as shown in Figure 169 208 Planning and Implementing an IBM SAN i ED 5000 E3 e el OT rise OOOUOOOOOOOOOOOOOOOODOONINI 7 MM COTE s Men Figure 169 Hardware view of the ED 5000 On the main window we see the operational status of the switch itself The switch is fully operational and online Also there are fields for the switch
135. a plymouth AIX fdevifscsi0 Oxef 0 Emule 10000000 9221802 unknown SWITCH unknown 20fd0060 69201d4e RB New Host Data Original Host Data Close Host Name lunknown unknown New Host Host Type SWITCH SwiTcH Jea iesta A Changes Host Connection unknown unknown pply Chang Host VWWYN 20f40060 69201d4e 20fd0060 69201 d4e Undo Changes Delete a Host Take Host Off Line Figure 244 Changing switch information The information on the host plymouth that was attached using port 4 is still kept in the VPS database but it is now shown in blue to indicate that it is offline Figure 245 is a diagram to show the configuration with the switch 288 Planning and Implementing an IBM SAN Server Server EEFI EFH unknown plymouth Ee IBM Fibre Channel 7 Switch 1 4 Gateway OL scsi ESS Figure 245 Adding an IBM switch to the Gateway Once changed as shown in Figure 246 the information shown on the main window will reflect the change and the icon on the left hand side of the WWN changes to depict a switch Chapter 8 Implementing the SAN Data Gateway 289 FA Storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc File View Admin Tools Controls Help 9 113 24 110 ITSO SAN Data Gateway oS SCSI Channel 1 Current SAN Data Gateway ITSO oa Co SCSI Channel 2
136. activation of the zone set with the Activate Zone Set window We Start it and receive a message that the activation is complete as shown in Figure 191 Activate Zone Set Activate zone set complete Figure 191 Activate Zone Set complete Chapter 7 Implementing the McDATA ED 5000 229 After returning to the Zoning View window and expanding the zone set by clicking the small symbol to the left of ITSO_Zone_Set_1 and then NT_Zone_1 the Zoning View of our fabric manager looks like Figure 192 lt 2 McDATA Fabric Manager lolx m Zoning View Default Zone Enabled View ctive Zone Set A _JITSO_Zone_Set_1 NT_Zone_1 RPKAS3_C1B2A1 eb GLogic 20 00 00 E0 3B 00 52 5E Configure Emulex 10 00 00 00 09 20 CA E6 Save active zone set as N Deactivate zone set Figure 192 Active Zone Set shown in the Zoning View of the fabric manager We see the zone set and the associated zone by its name and the configured host adapters by their manufacturers name and their WWN Also we see the nickname for the ESS port which we configured earlier If we had not configured a nickname for the ESS port we would only see it as another Emulex adapter which are used as the Fibre Channel adapters in the ESS From within the Zoning View window we can also manipulate the zone sets for example deactivating a zone set or saving the zone set As an example we copy the same zone set but assign it a different n
137. aged Hub GBIC Port Ready LED Port 7 Figure 106 Managed hub The system board is enclosed in an air cooled chassis which may be either mounted in a standard rack or used as a stand alone unit The chassis includes a power supply an RJ 45 Ethernet connection for 3534 Managed Hub set up and management and a serial port If the default address is not known the serial port is used for recovering the factory settings and initial configuration of the IP address The hub can accommodate one GBIC module and can be connected to one other managed hub to expand the loop capabilities to 14 ports It can also be connected with a single port into a SAN fabric as a loop extension The hub may be managed using the serial port or the 10 100BaseT Ethernet port Management interfaces include Telnet or Web based management using the IBM StorWatch SAN Fibre Channel Managed Hub Specialist This is similar to the StorWatch Fibre Channel Switch Specialist All network cable connection is through 50 125 or 62 5 125 Short Wave Length SWL cable or through 9 125 Long Wave Length LWL cable Fibre Channel Storage Hubs are designed to provide a centralized point of connectivity to provide loop fault tolerance and to simplify configuration management Specifically designed for entry level workgroup FC AL applications the hubs provide considerable flexibility in configuring loops and segmenting them for performance or high profile availability applica
138. aged our McDATA fabric in a task oriented way 1 Verify installation requirements We know what equipment we want to use and how we want to configure it In our case we want to first build a single switch fabric with one ESS and one Windows NT server Later on we want to extend the setup with more hosts for example RS 6000 Once this is established we build zones and add another ED 5000 to build a cascaded fabric The hosts are already attached in our environment and so is the ESS What we need to ensure is that everything relating to the physical and logical installation of the ED 5000 is complete For instance we need to make sure we have access to power supplies and access to the local network What we need to make sure before attempting any installation is that the necessary resources whatever they may be are available to support us or can be scheduled to become available In Appendix A of the ED 5000 Enterprise Fibre Channel Director Planning Manuaf there is a checklist which is useful for keeping track of the project and its progress 2 Determine local area network addresses Note If you plan to connect both the EFC Server and the switch ona dedicated LAN you do not have to change the network information for the first switch Of course if you later add more switches you will need to change the network information for the new devices This is referred to in 7 2 1 Setting up the network environment on page 187
139. ailable Software panel displays with the name of the software you selected to install 6 Check the default option settings to ensure that they are what you need 7 Press Enter to install the software SMIT responds with the following question Are you sure 8 Press Enter to continue The installation process may take several minutes A message displays on your window when the installation process is complete 9 Verify whether or not the installation is successful 10 Press F10 when the installation process is complete 11 Exit from SMIT 12 Remove the compact disc 13 Shut down the host system 14 Turn on the host system 3 2 5 Locating the World Wide Port Name WWPN on RS 6000 Perform this step to locate the WWPN 1 Log in as root Type 1scfg v1 fcsx where x is the adapter number The network address is the Fibre Channel adapter port WWPN value 3 2 6 Verifying the IBM ESS configuration 60 To verify the configuration of the ESS on the AIX host system type the following command lsdev Cc disk grep 2105 The installation is successful if a list of all IBM ESS devices displays hdisk3 Available 30 68 01 IBM FC2105F20 hdisk4 Available 30 68 01 IBM FC2105F20 hdisk5 Available 30 68 01 IBM FC2105F20 Planning and Implementing an IBM SAN If a device is listed as another type of device this message displays This message indicates that the configuration was not successful hdisk3 Available 30 68 01 Other
140. ame as shown in Figure 193 230 Planning and Implementing an IBM SAN gt Save Active Zone Set As X ITSO_Zone_Set_2 et ee Figure 193 Saving zone set with different name 7 4 5 Adding an AIX zone to the existing zone set We also have AIX hosts and we want to add a zone with the AIX systems and another ESS as shown in Figure 194 The AIX hosts are already connected to the McDATA so all we need to do is to define the zone in the zone set which already includes the NT zone Chapter 7 Implementing the McDATA ED 5000 231 232 NT_Zone_1 __ Netfinity a NT Server Shortwave fiber Bt AIX Server Figure 194 NT and AIX zones with McDATA To add another zone to the zone set we basically follow the steps described previously We navigate to the Zone Set Library and mark the zone set in which we want the zone to be added In our case this is the ITTSO_Zone_Set_1 Then we use Modify from the Actions menu which allows us to change the content of the zone set as shown in Figure 195 Planning and Implementing an IBM SAN gt Zone Sets ANT_Zone_1 RPKAS3_C1B2A1 QLogic 20 00 00 E0 88 00 52 5E Emulex 10 00 00 00 09 20 CA E6 ITSO_Zone_Set_2 Figure 195 Modify zone set From the Modify Zone Set window we select Actions gt NewZone which opens the New Zone window We drag and drop the adapters of our AIX hosts to the Members in zone entry fie
141. anager Chapter 7 Implementing the McDATA ED 5000 201 202 After pressing the button to start the installation process the software starts downloading to our local machine and begins the installation We now follow the instructions to install the EFC Manager After confirming the licence agreement we get information about which version we are going to install as shown in Figure 159 The final step is to tell the installation program where to put the program files 4B McDATA Enterprise Fabric Connectivity Manager Iof x Important Information Please read before continuing This will install Version 02 00 00 Click Next to continue or Exit to cancel Exit Previous Figure 159 EFC Manager version After finishing the installation there will be a shortcut to the EFC Manager on the desktop as shown in Figure 160 Ny My Computer EFC Manager Figure 160 EFC Manager icon By double clicking on this icon we get the login window for the EFC Manager We have already defined the user on the EFC Manager on the EFC server Now we use our username and password and the IP address of the EFC server to login as shown in Figure 161 Planning and Implementing an IBM SAN EFC Manager Login x morsa E Louin Dy Exit Figure 161 EFC Manager login window After a successful login we can move on to managing the fabric 7 3 Managing the ED 5000 fabric In our environment we now have an ED 5000 an
142. and hold the Alt key and press the Q key to get to the FAST Util Restart the server command panel From the Configuration Settings menu select Host Adapter settings To set the parameters and values from the Host Adapter settings menu use the following a Host adapter BIOS Disabled b Frame size 2 048 c Loop reset delay 5 minimum d Adapter hard loop ID Disabled Some configurations might require hard IDs Consult the separate application note that addresses Fibre Channel address for more information or consult the adapter card vendor e Hard loop ID From the Advanced Adapter Settings menu use the down arrow key on your keyboard to highlight LUNs per target then press Enter Set the parameters and values from the Advanced adapter settings menu as follows a Execution throttle 240 b Fast command posting Enabled c gt 4 GB addressing Disabled for 32 bit systems d LUNs per target 0 Planning and Implementing an IBM SAN e Enable LIP reset No Enable LIP full login No Enable target reset Yes zs Q Login retry count 20 minimum i Port down retry count 20 minimum j Driver load RISC code Enabled k Enable database updates No l IOCB allocation 256 m Extended error logging Disabled might be enabled for debugging Note The enable LIP reset enable LIP full logon and enable target reset parameters control the behavior of the adapter when Windows NT tries t
143. and tested but every extra hop adds about 1 2 microseconds latency to the transmission The length of the fiber is another consideration since each kilometer between nodes adds a further five microseconds delay Traffic patterns need to be understood to avoid long paths and bottlenecks Ideally devices should be attached to the same switch if they exchange large amounts of data because this minimizes communication delays If this is not possible then more ISLs should be configured to increase the available bandwidth between switches Of course this also adds to the resiliency of the fabric The fabric design in Figure 19 illustrates a fully meshed fabric in which a switch is only one hop from any other switch This minimizes latency across the fabric Also if any link fails even with two link failures all switches can still communicate with each other Chapter 2 SAN planning and design considerations 41 42 Switch 1 Fully Meshed Fabric Switch 2 00000 oooooooo Switch 3 HOoOoooo00 00000 Switch 4 Figure 19 A fully meshed topology 2 3 2 5 Redundant fabrics We have been discussing redundant elements like paths and switches within a single fabric Another approach and one which gives many advantages is to use redundant fabrics The simplest version of this is two switches which are not interconnected If one switch fails data is automatically routed via the second
144. andles all communications StorWatch Expert These are StorWatch applications that employ a 3 tiered architecture that includes a management interface a StorWatch manager and agents that run on the storage resource s being managed Expert products employ a StorWatch data base that can be used for saving key management data e g capacity or performance metrics Expert products use the agents as well as analysis of storage data saved in the data base to perform higher value functions including reporting of capacity performance etc over time trends configuration of multiple devices based on policies monitoring of capacity and performance automated responses to events or conditions and storage related data mining StorWatch Specialist A StorWatch interface for managing an individual fibre Channel device or a limited number of like devices that can be viewed as a single group StorWatch specialists typically provide simple point in time management functions such as configuration reporting on asset and status information simple device and event monitoring and perhaps some service utilities Striping A method for achieving higher bandwidth using multiple N_Ports in parallel to transmit a single information unit across multiple levels STP Shielded Twisted Pair Storage Media The physical device itself onto which data is recorded Magnetic tape optical disks floppy disks are all storage media Switch A component
145. anned e SAN applications e Logical and physical design Chapter 2 SAN planning and design considerations 49 e Hardware and software plan e Training plan e Project implementation plan 2 5 Future developments We can all speculate on what the future will bring for SANs Fibre Channel version 2 will bring speeds of 200 MB s double what we have today SAN fabric vendors will most likely develop new more intelligent and faster gateways hubs and switches with more scaleability in port count bandwidth and greater fault tolerance Server and storage vendors will introduce more native Fibre Channel solutions faster processors more scalable and intelligent storage subsystems Fibre Channel industry standards will continue to be delivered through cooperation among the vendors creating greater ease of interoperability and rapid growth in the SAN marketplace S 390 FICON and FCP protocols will be enabled to operate on the same fabric Sophisticated software management tools will finally deliver full end to end SAN management The true Enterprise SAN will arrive as we show in Figure 24 Sa mme SS aces iscsi issall IBM Enterprise SAN Vision 5 390 i T HP a Intel Intel RS 6000 gi HP p i based Escon Ficon PFcP PFcp PFce FcP ssa scsi Scsil Lt a oe e Ee Fibre Channel SAN ea ESE a mm Wee RVA x83 Magstar
146. annel fabric components 0 283 8 6 1 Connecting an IBM SAN Fibre Channel Switch 284 8 6 2 Connecting a McDATA Enterprise Fibre Channel Director 294 8 7 High availability considerations 0000 c eee eee 295 8 7 1 Single host with multiple Fibre Channel adapters 295 8 7 2 Multiple SCSI connections 0 000 296 8 7 3 Adding Fibre Channel switches 2 0245 296 Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 299 9 1 Installing the SLIC Router 0 00 00 eee eee 300 9 2 SLIC Manager software 0 0 eee 303 9 2 1 Installing the SLIC Manager software 304 9 2 2 Communicating to the Router 00 0 0 eee 305 9 2 3 Starting the SLIC Manager 00 0000 ee eee 309 9 3 Using SLIC Manager 0000 es 310 9 3 1 Drive properties 0 ee ees 311 9 3 2 Router properties 0 0 eee ee 311 9 3 3 Setting Router to master eesse rakie eaa e eka e eee 312 9 3 4 The SignOn drive 2 2 eee 313 9 4 COMpPOSite Masseren ea be tae ed ba ad eke 314 9 4 1 Creating a composite drive 0 0 2 eee eee 314 9 4 2 Composite drive properties 0000 317 9 5 Miror OrIVGs 2625 5 sie tong Glenn a aha hele atte Bal a tiatalinns Aiea ga 319 9 5 1 Creating a mirror drive 2 2 ee 320 9 5 2 Mirror drive properties 2 0 0 0 0 ee 322 9 6 Instant Copy drive
147. apidly to rediscover the disks and the SSA drive LEDs should be on solid Once completed the status LED will be solid and the drives are now considered to be mapped physical drives The Router is powered off 7 Create general spares The mode on SW2 is changed to mode 12 to set all drives to be general spares Mode 12 is represented by setting switch 2 and 3 down and the rest turned up Power on the Router again the Status LED will flash After approximately one minute the LED will flash code 100 to indicate it has completed The Router is again powered off 8 Format the drives The Router is set to mode 14 switch 1 2 and 3 down on SW2 to format all disk drives Power on the Router depending on the number of drives the format process will take anywhere from 30 to 60 minutes During this time the Status LED will flash rapidly and the SSA drive LEDs will flash When it has completed the Status LED will flash code 100 Power off the Router 9 Clear node map Once completed the node map must be cleared This is done as described earlier by shorting ports A1 and A2 with an SSA cable Power on wait for code 060 to flash and then power off The drives are now ready to be assigned and used on a host You can also now setup mirror drives or composite drives within the Router This is done by setting the switches in SW2 to other modes For detailed information on setting the switches and selecting the other modes please refer to t
148. approximately 1 to 3 minutes to complete Would you like to continue and perform the update Ss Unsigned Java Applet Window Figure 54 Time consuming action warning 86 Planning and Implementing an IBM SAN How time consuming this will be depends on how many disk groups are to be configured at once Pressing the OK button executes the script which changes the configuration and gives us a progress window where we can see which arrays are initializing as shown in Figure 55 N Performing Configuration Update x Defining RAID Array ssa01 Red 2 zz E Estimated Remaining Time 30 seconds Ey Unsigned Java Applet Window Figure 55 RAID definition progress window After completing the configuration the Specialist informs us of the success as shown in Figure 56 N xi 2202 The requested disk group configuration has been performed successfully Unsigned Java Applet Window Figure 56 Successful disk group configuration We have to press the OK button here which takes us back to the Open System Storage panel From there we can continue with our configuration If you go back to the Storage Allocation window device adapter pair number one will now look like Figure 57 Figure 57 Device adapter pair one with four disk groups Chapter 4 Configuring the ESS with native Fibre Channel 87 4 1 6 Defining volumes for Fibre Channel host adapter ports If we return to the Open System Sto
149. aring of expensive hardware resources between multiple servers Most storage devices cannot just attach to a LAN NAS solutions are specialized file servers which are designed for this type of attachment NAS therefore offers a number of benefits which address some of the limitations of parallel SCSI However by moving storage transactions such as disk accesses and tasks such as backup and recovery of files to the LAN conflicts can occur with end user traffic on the network LANs are tuned to favor short burst transmissions for rapid response to messaging requests rather than large continuous data transmissions Significant overhead can be imposed to move large blocks of data over the LAN due to the small packet Chapter 1 Introduction to Storage Area Networks 9 size used by messaging protocols For instance the maximum packet size for Ethernet is about 1500 bytes A 10 MB file has to be segmented into more than 7000 individual packets each sent separately by the LAN access method if it is to be read from a NAS device Therefore a NAS solution is best suited to handle cross platform direct access applications not to deal with applications requiring high bandwidth NAS solutions are relatively low cost and straightforward to implement as they fit in to the existing LAN environment which is a mature technology However the LAN must have plenty of spare capacity to justify NAS implementations A number of vendors including IBM
150. at each site which adds even more availability bandwidth and ports This is shown in Figure 201 Chapter 7 Implementing the McDATA ED 5000 239 local site i remote site Open Systems Server Open Systems Server McDATA ED5000 Ree SELES ESSnativeFC ESSnative Fc Mirror 2 Figure 201 Extended High Availability and disaster recovery with McDATA The arrows indicate a possible route for the data to get to both parts of the mirrored sets In this setup there is no single point of failure at a device level and even if one site completely fails the other site will take over operation In our example for a multiswitch fabric shown in Figure 202 we are not focusing on clustering What we want to show is how to apply zoning in a multiswitch fabric 240 Planning and Implementing an IBM SAN A RPKA93_C1B2A1 NT Fs 7 Q09i0 20 00 00 E0 8B 00 52 5E _ NT_Zone_1 y Shortwave fiber McDATA ED5000 ee i is local site X FC LongWave ISL ko o Ee a s 7 remote site McDATA ED5000 _ oS 5s go ESS Ta l AlX Zone 1 Emulex 10 00 00 00 C9 20E7 F3 7 EEEH a is A Pa hei I a 5 J lt me RS6K eS pats Emulex 10 00 00 00 C9 20 DC 71 Shee eee v RSBK a Emulex 10 00 00 00 C9 20 E7 C7 eee on RS6K ar i chy KATE_FCSO ee Figure 202 Multi switch fabric with McDATA We have our NT zone with two ESSs On
151. ata will be shown on the right hand view pane Chapter 8 Implementing the SAN Data Gateway 265 Ef Storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc Fie View Admin Tools Controls Help E 93 113 24 110 ITSO SAN Data Gateway lt 3 SCSI Channel 1 Current SAN Data Gateway ITSO og SCSI Channel 2 IP Address 9 113 24 110 lt 3 SCSI Channel 3 gt SCSI Ch l 4 C REEN Channel Description Fibre Channel 1 World Wide Port Name 20010060 45160E08 4r Fibre Ch l 4 ore nanne Serial Number 00010072 Media Short Wave Optical Firmware Revision 2 01 11 Port Type node loop NL_Port Loop ID 0 Frame Size 2048 Port Mode Public Target Only Connection Loop Only Max Speed 100 Mb sec Status Ready Up Time 4 days 23 34 54 Figure 223 Fibre Channel port data By selecting the key to the left you can expand the tree and select the host system attached to that port Figure 224 shows the detail on the specific host 266 Planning and Implementing an IBM SAN FA storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc Iof x File View Admin Tools Controls Help E 9 113 24 110 ITSO SAN Data Gateway F SCSI Channel lt 3 SCSI Channel lt 3 SCSI Channel lt SCSI Channel Fibre Channel Current SAN Data Gateway ITSO IP Address 9 113 24 110 Channel Description Fibre Channel 1 gt Fibre Channel
152. atch Atmimistration DEFAULT ADMIN ID WILL BE REMOVED Figure 33 User administration panel In our case there is no user configured other than the default user that we logged on with In the Modify Users panel as shown in Figure 34 we can add users with different access levels 72 Planning and Implementing an IBM SAN zz fe iuye Perform Configuration Update Cancel Configuration Update Figure 34 Modify users panel 4 1 1 8 Licensed Internal Code Using the Licensed Internal Code button we get the Licensed Internal Code panel as shown in Figure 35 Licensed Internal Gode Chapter 4 Configuring the ESS with native Fibre Channel 73 Licensed Internal Code jis Readies A Licensed Internal Code LIC Levels Custer LIC Source Weston Cd Cister active LIC scoosun ft Chust Previous LIC scoo2a5_ zooms Custi Weenie coos ooo o Chusta Active LIC coos sft Chuster 2 Previous LIC scoozas_ fz S Custera Meere scons o d Licensed Feature Codes Figure 35 Licensed Internal Code panel 74 Here we can see the LIC levels and the licensed feature codes 4 1 1 9 Copy Services Using the Copy Services button we connect to the copy server Copy Services We have to login to gain access just like we did for the ESS Specialist This opens the Web based Java interface for the ESS Copy Server in a dedicated Navigator window and connects to the copy services server as shown in Figure 36
153. ationship can be established between a client and a group of storage devices that are not physically co located excluding devices which are internally attached to servers A logical view of the combined disk resources may allow available capacity to be allocated and reallocated between different applications running on distributed servers to achieve better utilization Consolidation is covered in greater depth in BM Storage Solutions for Server Consolidation SG24 5355 Figure 12 shows a logical consolidation of independent arrays 18 Planning and Implementing an IBM SAN Physically Independent Disk Arrays Logically Viewed as a Single Entity SAN Disk Array 1 Disk Array 2 Disk Array 3 Figure 12 Logical consolidation of dispersed disk subsystems 1 5 2 Data sharing The term data sharing is used somewhat loosely by users and some vendors It is sometimes interpreted to mean the replication of files or databases to enable two or more users or applications to concurrently use separate copies of the data The applications concerned may operate on different host platforms A SAN may ease the creation of such duplicated copies of data using facilities such as remote mirroring Data sharing may also be used to describe multiple users accessing a single copy of a file This can be called true data sharing In a homogeneous server environment with appropriate application software controls multiple servers may acc
154. ave been reviewed by IBM for accuracy in a specific situation there is no guarantee that the same or similar results will be obtained elsewhere Customers Copyright IBM Corp 2000 343 344 attempting to adapt these techniques to their own environments do so at their own risk Any pointers in this publication to external Web sites are provided for convenience only and do not in any manner serve as an endorsement of these Web sites The following terms are trademarks of the International Business Machines Corporation in the United States and or other countries IBM Redbooks Redbooks Logo ae AIX AS 400 AT DB2 DFSMSrmm ECKD Enterprise Storage Server ESCON FICON IBM Magstar Netfinity OS 390 OS 400 RS 6000 390 Seascape SP StorWatch Ultrastar Versatile Storage Server 400 The following terms are trademarks of other companies Tivoli Manage Anything Anywhere The Power To Manage Anything Anywhere TME NetView Cross Site Tivoli Ready Tivoli Certified Planet Tivoli and Tivoli Enterprise are trademarks or registered trademarks of Tivoli Systems Inc an IBM company in the United States other countries or both In Denmark Tivoli is a trademark licensed from Kj benhavns Sommer Tivoli A S C bus is a trademark of Corollary Inc in the United States and or other countries Java and all Java based trademarks and logos are trademarks or registered trademarks of Sun Microsystems Inc in the United States and
155. bl Netscape Switch User Firmware Reboot SNMP tel a Admin Admin Upgrade Switch Admin Admin Admin Quickloop Config for fehub1 WWN Name Domain Id Current QuickLoop Partner 10 00 00 60 69 10 10 8 feswitch1 1 Select a QuickLoop Partner for this Switch RESTENE Submit Reset AL_PA Bitmap in hexadecimal 00000000 00000000 00000000 00000000 Local AL_PAs Remote AL_PAs Port Port Id AL_PAs Port Port Id AL_PAs Not Available Not Available Figure 120 QuickLoop with switch partner Telnet QuickLoop commands are also supported and these are described in IBM 3534 SAN Fibre Channel Managed Hub User s Guide GC26 7391 5 7 5 IBM 3534 managed hub zoning The IBM SAN Fibre Channel Managed Hub 35341RU is able to use zoning for finer segmentation in the access of devices Also it is used to protect devices against LIP As the managed hub is always in QuickLoop mode this means that all devices on that QuickLoop are subject to LIP exposure By using zoning it is possible to limit the effect of LIP to QuickLoop zones That is to say a LIP in one zone does not cause a LIP in another zone Zoning in the IBM 3534 works exactly as zoning does in the IBM SAN Fibre Channel Switch and can be managed by using Telnet commands or by using the IBM Storwatch Specialist Zoning is described in 6 4 Implementing zoning on page 161 Chapter 5 Implementing an IBM managed and unmanaged hub
156. c element of a message in Fibre Channel communications consisting of a 24 byte header and zero to 2112 bytes of data See also Sequence FSP Fibre Channel Service Protocol The common FC 4 level protocol for all services transparent to the fabric type or topology Full Duplex A mode of communications allowing simultaneous transmission and reception of frames G_Port Generic Port a generic switch port that is either a Fabric Port F_Port or an Expansion Port E_Port The function is automatically determined during login 356 Planning and Implementing an IBM SAN Gateway A node on a network that interconnects two otherwise incompatible networks GBIC GigaBit Interface Converter Industry standard transceivers for connection of Fibre Channel nodes to arbitrated loop hubs and fabric switches Gigabit One billion bits or one thousand megabits GLM Gigabit Link Module a generic Fibre Channel transceiver unit that integrates the key functions necessary for installation of a Fibre channel media interface on most systems Half Duplex A mode of communications allowing either transmission or reception of frames at any point in time but not both other than link control frames which are always permitted Hardware The mechanical magnetic and electronic components of a system e g computers telephone switches terminals and the like HBA Host Bus Adapter HIPPI High Performance Parallel Interface An ANSI standard
157. call Quality of Connection QoC This is built on the concept of system availability commonly used when defining service levels These are normally described in terms of percentage systems availability Planning and Implementing an IBM SAN A 99 999 five 9s up time refers to achievement of less than five minutes systems downtime in one year A one 9 measure refers to a 90 availability less than 36 5 days systems downtime and a three Qs level is 99 9 uptime less than 8 hours 45 minutes systems downtime annually Downtime can be defined as any complete interruption of service for any reason whether planned or unplanned To meet the very high levels of uptime required by planners and administrators it is essential to design the correct network architecture It needs built in fault tolerance failover capabilities and available bandwidth to handle unplanned outages in a transparent manner SAN QoC measurements are determined by the network topology and the interconnect technologies used hubs switches These define how well the SAN can sustain operations in the event of an outage within the network from the perspective both of connection availability and maintaining performance High availability can be built in to the fabric by eliminating single points of failure This is achieved by deploying hardware components in redundant pairs and configuring redundant paths Redundant paths will be routed through different switches to prov
158. capability of the particular host system an access profile can contain up to 256 or up to 4096 volumes The setup of an access profile is transparent to the user when you use the ESS Specialist Web user interface to configure the hosts and volumes in the ESS Configuration actions that affect the access profile are Chapter 4 Configuring the ESS with native Fibre Channel 99 e When you locate the WWPN for a Fibre Channel attached host system you create the access profile for that host system Initially the profile is empty That is it contains no volumes The host cannot access any logical volumes that are already defined in the ESS e When you add new logical volumes to the ESS the new volumes go to the host that you select e When you assign volumes to Fibre Channel attached hosts you add to the access profile of the selected host system and to pre existing volumes e When you remove a Fibre Channel attached host system from the ESS you delete the host and its access profile Storage Allocation Graphical View Clear View View All Storage Tabular View id Click on a Host or Array to see SUNW 2_ kate fes kate fes shak20cl a Anonymous WER unknown Attachment Fc Description none Operating System DGB Civruye Open System Storage Figure 77 Anonymous host in access any mode 4 1 7 4 FC Port Attributes field The second pull down is for the Fibre Channel port attributes an
159. cascading 38 139 171 CCW 13 centralized management 24 channel access mode 98 command words 13 transport 13 CLAM Associates 46 Class 1 46 Class2 46 Class 3 46 Class 4 47 Class 5 47 classes of service 14 Client 257 client server 1 clock frequency 15 cluster 22 Combining Composite And Mirroring 329 common services 14 Communications panel 68 composite drive 314 329 as members of amirror 331 properties 317 configDefault 177 178 configdefault 177 configure disk groups 84 Configure Host Adapter Ports 96 consultant 27 Control Center 310 control panel 144 146 copper 13 Copy Services 74 Create Alias 169 363 Create General Spares 302 create user for telnet to San Data Gateway 255 Creating A Composite Drive 314 Creating A Mirror Drive 319 Creating a SLIC Zone 307 Creating an Instant Copy drive 324 credit based flow 16 D datacentric 11 data corruption 8 dataencoding 14 data entry buttons 146 dataformats 2 Data Gateway 35 data migration 23 data objects 31 Data Path Optimizer 42 data rate 5 data sharing 19 Data Warehouse 3 database software 1 default IP address 144 define user EFC 192 defined configuration 160 design 27 Detach Instant Copy Drive from a Mirror 329 Device discovery 270 Director 37 director 45 director offline 214 disaster tolerance 22 disk mirroring 36 disk pooling 63 distance extenders 36 Distributed Name Server 39 DNS 188 206 domain 242 domain ID 242 Domain Name System
160. ccurrence by means of zoning and or logical unit number LUN masking Decide which level of zoning hardware and or software and LUN masking is appropriate remembering that LUN masking at the storage device level including SAN Data Gateway provides the highest level of security because it logically binds storage volumes to specific servers This ensures that each server can only access its own data just as though the storage was directly attached to the server 2 1 2 5 Manageability Consider how the SAN will be managed in your environment You will need to have tools to handle a number of critical aspects A variety of software vendors offer tools to address some of these requirements Tivoli Systems has a set of software tools which provide complete SAN management These management tools include e Configuration Facilities to identify operate collect data from and control the devices in the SAN Chapter 2 SAN planning and design considerations 29 e Access Tools to allow configuration maintenance zoning and LUN masking to protect data and ensure only authorized access to information e Performance Managing performance to meet service levels analyze the traffic on the SAN and understand the behavior of applications in order to be able to optimize the network and plan for future requirements e Faults The ability to detect isolate correct and report on events within the SAN 2 1 2 6 Affordability When you are considering
161. ce environments with their production environment Within a fabric you can easily separate your test environment from your production bandwidth allocation on the same fabric using zoning We show an example of zoning in Figure 131 Chapter 6 Implementing an IBM SAN Fibre Channel Switch 155 Unix Windows NT 2000 Zone red Figure 131 Zoning example Zoning is a fabric management service used to create logical device subsets within a Storage Area Network and enables resource partitioning for management and access control A zone can also be defined as a set of devices that access one another All devices connected to a fabric may be configured into one or more zones Devices in the same zone can see each other devices that are in different zones cannot Before you can start using zoning and before we start setting up zones in our SAN scenario we have to explain some of the basic naming conventions and components necessary when working with zones in an IBM SAN Fibre Channel Switch environment using the StorWatch Fibre Channel Switch Specialist 6 3 2 Zoning components Zoning has several components besides the zones themselves such as e Zone members e Zone aliases e Zone configurations These components are generically referred to as zone objects 156 Planning and Implementing an IBM SAN Every zone has a name that begins with a letter and is followed by any number of letters digits and
162. ch GBIC Verify that each associated port LED shows a slow green every two seconds flash If any of the above checks have failed see Problem Determination Start Map on page If none fail the switch is ready for use 6 2 4 Attaching the switch to a network server and a storage device 150 Now that the switch is working attach one of the eight available ports using a Fibre Channel cable to your host server Fibre Channel adapter and another switch port again using a Fibre Channel cable to your storage device Depending on the GBIC in the port this can be either an SWL fiber optic link or an LWL fiber optic link The SWL fiber optic GBIC module with SC connector color coded black is based on shortwave length lasers supporting 1 0625 Gb s link speeds This GBIC module supports 50 micron multi mode fiber optic cables with cables up to 500 meters in length The LWL fiber optic GBIC module with SC connector color coded blue is based on longwave length 1300nm lasers supporting 1 0625 Gb s link speeds This GBIC module supports 9 micron single mode fiber Cables up to 10 kilometers in length with a maximum of five splices can be used 6 2 4 1 Attaching the server and the storage device To establish a SAN with fabric point to point connection between server and storage your server needs a Fibre Channel host bus adapter HBA card that supports fabric point to point In our environment we used the following e Netfinity 5500N
163. cross the SAN A SAN changes the server centric model of the typical open systems IT infrastructure replacing it with a data centric infrastructure Chapter 1 Introduction to Storage Area Networks 11 1 3 3 What about ESCON and FICON Sceptics might already be saying that the concept of SAN is not new Indeed for System 390 S 390 users the implementation of shared storage on a dedicated network has been common since the introduction of Enterprise System Connection ESCON in 1991 However for UNIX Windows NT and other open systems users the need for such capability is now extremely high As we have shown the traditional SCSI parallel bus architecture most commonly used in these environments is no longer capable of handling their growing data intensive application requirements These users are faced with many of the same problems which challenged mainframe users in the late 1980s and early 1990s and which largely were solved by ESCON But the ESCON architecture does not answer the open systems needs of today due to a number of critical limitations ESCON is primarily a S 390 solution which does not support open systems protocols for data movement and ESCON is limited in performance nominally 17 MB second relative to technologies available today An enhancement to ESCON is provided by Fibre Connection FICON Figure 9 shows how FICON enhances ESCON FICON enhances ESCON ESCON FICON Bridge S 390 Server S 390
164. ct SAN Data Gateway or SAN Data Gateway Router from the toolbar This can also be used to connect to several Gateways or Routers from a single client session In Figure 217 we show the initial view once a connection to a SAN Data Gateway is established and the data has been downloaded Chapter 8 Implementing the SAN Data Gateway 259 Ef Storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc Fie View Admin Tools Controls Help Current SAN Data Gateway ITSO World Wide Node Name 10000060 45160E08 IP Address 9 113 24 110 Serial Number 21081300724 Firmware Revision 3 40 10 Health Check Level 2 Health Check Interval 15 Log Size 2047 Receive Event Traps Enabled Serial Baud Rate 19200 Figure 217 StorWatch SAN Data Gateway Specialist initial view 260 The left side of the window shows the SAN Data Gateway unit we are connected to and the right side provides product data information You will also notice that the toolbar will have options available that were previously greyed out You can now connect to another SAN Data Gateway disconnect from a SAN Data Gateway enable and access the Zoning and VPS features restart the Gateway and also refresh the data to your window by downloading it again These options become available when a SAN Data Gateway is highlighted As you begin to add SAN Data Gateway systems or drill down into a particular Gateway by selecting and highlightin
165. cteristics it can support multiple protocols and a broad range of devices and it can be managed as a network Fibre Channel can use either optical fiber for distance or copper cable links for short distance at low cost Chapter 1 Introduction to Storage Area Networks 13 14 Fibre Channel is a multi layered network based on a series of American National Standards Institute ANSI standards which define characteristics and functions for moving data across the network These include definitions of physical interfaces such as cabling distances and signaling data encoding and link controls data delivery in terms of frames flow control and classes of service common services and protocol interfaces Like other networks information is sent in structured packets or frames and data is serialized before transmission But unlike other networks the Fibre Channel architecture includes a significant amount of hardware processing to deliver high performance The speed currently achieved is 100 MB per second with the potential for 200 MB and 400 MB and higher data rates in the future In all Fibre Channel topologies a single transmitter sends information to a single receiver In most multi user implementations this requires that routing information Source and target must be provided Transmission is defined in the Fibre Channel standards across three transport topologies e Point to point A bidirectional dedicated interconnection betwe
166. ctivated and deactivated at the same time This can be used to save different configurations for different tasks for example if we want to have different devices in the same zone for backup but not during normal operation Devices that are not configured in a zone within the active zone set are considered as members of the default zone Also if no zone sets are activated all devices are in the default zone With the default zone enabled it is possible for all devices in the default zone to communicate with each other in parallel to the currently active zone set You can also disable the default zone independently from the active zone An example of how zones and zone sets are related is shown in Figure 181 Chapter 7 Implementing the McDATA ED 5000 219 Amount of Devices wes _ a ee NS f a E Default Zone Figure 181 Example for McDATA Zoning 7 4 4 Creating a zone set and establishing an NT zone The next step in our setup of the McDATA SAN is shown in Figure 182 There are hosts already connected to it but we introduce the zoning concept first by establishing an NT zone and then later by the addition of an AIX zone 220 Planning and Implementing an IBM SAN Netfinity aera _ NT Server hess ESS 1 McDATA ED5000 Other Server ESS native F default zone default zone Figure 182 NT zone with McDATA
167. d 36 inches Media Plural of medium The physical environment through which transmission signals pass Common media include copper and fiber optic cable Media Access Rules MAR MIA Media Interface Adapter MIAs enable optic based adapters to interface to copper based devices including adapters hubs and switches MIB Management Information Block A formal description of a set of network objects that can be managed using the Simple Network Management Protocol SNMP The format of the MIB is defined as part of SNMP and is a hierarchical structure of information relevant to a specific device defined in object oriented terminology as a collection of objects relations and operations among objects Mirroring The process of writing data to two separate physical devices simultaneously MM Multi Mode See Multi Mode Fiber MMF See Multi Mode Fiber In optical fiber technology an optical fiber that is designed to carry multiple light rays or modes concurrently each at a slightly different reflection angle within the optical core Multi Mode fiber transmission is used for relatively short distances because the modes tend to disperse over longer distances See also Single Mode Fiber SMF Multicast Sending a copy of the same transmission from a single source device to multiple destination devices on a fabric This includes sending to all N_Ports on a fabric broadcast or to only a subset of the N_Ports on a fabric mult
168. d features Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 311 SLIC Info Figure 262 SLIC Router properties 9 3 3 Setting Router to master 312 As you move through the toolbar you may notice that most selections have been grayed out This is due to the fact that the Router is currently in a subordinate role and does not have access to create mirrors or composite drives This function is done by a Master Router There can be only one master in a SLIC loop This is used as more Routers and more disks can be added to the loop With several Routers in the same loop there needs to be a requirement where one system acts as the control and the others will follow and understand any configuration changes that may occur To set the Router into a master role select File gt Program Option from the top toolbar You will be presented a dialog box as shown in Figure 263 Program Option a mee Figure 263 Setting the Router to master Planning and Implementing an IBM SAN Click in the Always Master box so that a check mark appears Once you select OK the Router will then be set as the master You will notice that now all options in the toolbar are available and can start to use the features of the SLIC Router By placing a check mark in the Auto Start box the SLIC Manager will automatically connect to the Router defined in the Connection window as seen in Figure 258 on page 309 9 3 4 The SignO
169. d get detailed port information similar to that shown in Figure 172 210 Planning and Implementing an IBM SAN i Port Properties g Long wave laser single mode 000000 McDATA 20 17 08 00 88 40 68 29 Not logged in Unblocked a Zia poe Tels ojo dl es tQ i Figure 172 Port Card View with Port Properties 7 3 4 Configuring the FC ports To configure the options relating to each port we select Configure gt Ports We are now presented with the Configuration Ports window which is shown in Figure 173 Chapter 7 Implementing the McDATA ED 5000 211 i Configure Ports s RATE FOS le SsRPKAgacip2at CO OO OT 7s RPKAgaciB3at O O Mi RI CE 9 M5 RIRI o Figure 173 Configure Ports window The port number is automatically assigned and cannot be changed We can specify a port name here but this is only useful if the cabling on the port does not change that often We recommend using the name used as the WWN nickname To save the information use the Activate button Of more interest here is the ability to block specific ports to use extended distance buffering and disable link incident LIN alerts A link incident is a problem on a link which is visible in the Link Incident Log It is indicated with a little yellow triangle next to the port as shown in Figure 174 To view the LIN log go to Logs gt Link Incident Log as shown in Figure 174 212 Plannin
170. d the EFC server installed both have valid IP addresses and are accessible through the corporate intranet Now we are going to use the newly created user ID and the EFC Chapter 7 Implementing the McDATA ED 5000 203 Manager software on the remote workstation to perform these tasks as listed in the following sections e 7 3 1 Identifying the ED 5000 to the EFC Manager on page 204 e 7 3 3 Configuring the ED 5000 with the Product Manager on page 208 e 7 3 4 Configuring the FC ports on page 211 e 7 3 5 Configuring the FC operating parameters on page 214 7 3 1 Identifying the ED 5000 to the EFC Manager After logging on to the EFC server the Product View opens with no devices installed as shown in Figure 162 McDATA Enterprise Fabric Connectivity Manager Product Product View Fabric X view O Session Configure l Logs S Maintenance i Help Na LogoutExit Figure 162 Product View with no devices installed To the left there is a panel from where we can access the different management and monitoring functions To identify the ED 5000 to the EFC 204 Planning and Implementing an IBM SAN Manager to be managed we need to tell the EFC Manager the IP address of the ED 5000 This is accomplished by selecting Configure gt New product as shown in Figure 163 y McDATA E nterprise Fabric Connectivity Manager Figure 163 Adding new product Selecting this takes
171. d the Identify Member button is chosen then the LED only on that drive will flash In both cases a dialog box will appear to allow you to stop the flashing 9 5 Mirror drive A mirror is typically two drives a 2 way mirror that contains exactly the same information The SLIC Router can also support a 1 way or 3 way mirror A 3 way mirror consists of three drives with the same information A 1 way mirror is a single drive or single composite drive that is used with an Instant Copy Drive that can attach to the single drive mirror to synchronize the data The Instant Copy Drive can then be split off from the mirror to perform a backup or other action The Instant Copy Drive feature can be used with 2 way and 3 way mirrors as well Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 9 5 1 Creating a mirror drive 320 To create a mirror using physical drives from the toolbar select Tools gt Mirror Drive Setup Wizard You will see a dialog box Mirror Drive List that will be blank If there were mirror drives created then it would display the names of the drives Click on the Next gt gt button and the Mirror Drive Members Selection window opens The window on the left named Available Drive Pool contains a list off all drives that are candidates to participate ina mirror drive Select a drive by highlighting it and click on the Add gt gt button The drive name will be added to the Member window An asterisk will app
172. d the technology associated with the transmission of information as light pulses along a glass or plastic wire or fiber Ordered Set A Fibre Channel term referring to four 10 bit characters a combination of data and special characters providing low level link functions such as frame demarcation and signaling between two ends of a link Originator A Fibre Channel term referring to the initiating device Out of Band Signaling Signaling that is separated from the channel carrying the information Peripheral Any computer device that is not part of the essential computer the processor memory and data paths but is situated relatively close by A near synonym is input output I O device PLDA Private Loop Direct Attach A technical report which defines a subset of the relevant standards suitable for the operation of peripheral devices such as disks and tapes on a private loop PLOGI See N_Port Login Point to Point Topology An interconnection structure in which each point has physical links to only one neighbor resulting in a closed circuit In point to point topology the available bandwidth the is dedicated Port The hardware entity within a node that performs data communications over the Fibre Channel Port Bypass Circuit A circuit used in hubs and disk enclosures to automatically open or close the loop to add or remove nodes on the loop Private NL_Port An NL_Port which does not attempt login with the fabric and on
173. d these are shown in Figure 78 and Figure 79 In Figure 78 the port selected is already configured for Fibre Channel Point to Point 100 Planning and Implementing an IBM SAN Storage Server Atiributes Access Restricted Fibre Channel Access Mode FC Port Atiribuies Fibre Channel Point to Point be Undefined Fibre Channel Point to Point Fibre Channel Topology Figure 78 FC Port Attributes configured for Point to Point In Figure 79 the port is configured for Fibre Channel Arbitrated Loop Sto Server Attributes Access _Restricted Fibre Channel Access Mode FC Port Atiributes Fibre Channel Avbitrated Loop Undefined Fibre Channel Arbitrated Loop Fibre Channel Topology igure 79 FC Port Attributes configured for FC Arbitrated Loop Obviously it is here that we define the topology to which the port will be connected The possible choices are Fibre Channel Point to Point and Fibre Channel Arbitrated Loop We can choose between Fibre Channel Arbitrated Loop for FC AL and Fibre Channel Point to Point for FC PP and for FC SW If a port is already configured for one topology and it has to be changed to another the port must first be undefined Then it is possible to choose the appropriate topology We want to undefine the Fibre Channel port in bay 4 adapter 3 and define this port to Fibre Channel Point to Point To undefine the port we select it and highlight Undefined
174. d zone configurations and the administration of zones are the same for the IBM SAN Fibre Channel Managed Hub 35341RU as they are for the IBM SAN Fibre Channel Switch 5 6 Cascading 128 Cascading is a term used for interconnecting multiple switches When you start your SAN business you might begin working with a simple SAN fabric with perhaps only one switch and a few ports But as you start implementing more and more host and storage devices in your SAN you will reach limits where you have to add either more ports or expand your fabric by adding more switches Interconnecting your first switch with other switches allows you to build much larger fabrics Expanding the fabric in this manner is called cascading The hub can be connected to one other hub to expand the loop capabilities from 7 to 14 ports It can also be connected with a single port into a SAN fabric as a loop extension The attribute for interconnecting ports changes to an E_Port expansion port In our hub port 5 was used as an E_Port into an IBM SAN Fibre Channel Switch 2109 S08 We show this in Figure 112 Planning and Implementing an IBM SAN E Port Information for fchub1 olx ojil2 a a 5 el7 Port World Wide Name 20 05 00 60 69 30 08 0d Port Status Online Port Module or GBIC Module sw Port Type E Port PortStats 4 Byte Word Transmitted 248255050 Short Frames 6 4 Byte Word Received 18323749 Long Frames 0 Frames Transmitted 205995
175. dapter card on page 54 e 3 1 4 Installing the drivers on page 57 e 3 1 5 Locating the World Wide Port Name WWPN on Windows NT on page 57 3 1 1 Installing the QLogic 2100F adapter card The Qlogic QLA2100F adapter card only supports FC AL This section lists the steps you have to follow to attach an ESS to a Windows NT host system with the Qlogic QLA2100F adapter card 1 Install the QLogic QLA2100F adapter card in the host system 2 Connect the cable to the ESS port 3 Restart the server 4 Press and hold the Alt key and press the Q key to get to the FAST Util command panel 5 From the Configuration Settings menu select Host Adapter settings 6 From the Advanced Adapter Settings menu use the down arrow key on your keyboard to highlight LUNs per target then press Enter 7 Use the down arrow key on the keyboard to highlight LUNs per target Copyright IBM Corp 2000 53 8 9 Use the down arrow key on the keyboard to find and highlight 256 Press Enter Press the escape key 10 Save the changes Highlight Yes 11 Restart the server 3 1 2 Installing the QLogic 2200F adapter card This section tells you how to attach an ESS to a Windows NT host system with the Qlogic QLA220F adapter card Perform the following steps to install the QLogic QLA2200F adapter card 54 1 Install the QLogic QLA2200F adapter card in the host system 2 Connect the cable to the ESS port 3 4 Press
176. dbooks by accessing the IBM Intranet Web site at http w3 itso ibm com and clicking the ITSO Mailing List button Look in the Materials repository for workshops presentations papers and Web pages developed and written by the ITSO technical professionals click the Additional Materials button Employees may access MyNews at http w3 ibm com for redbook residency and workshop announcements Copyright IBM Corp 2000 351 IBM Redbooks fax order form Please send me the following Title Order Number Quantity First name Last name Company Address City Postal code Country Telephone number Telefax number VAT number Invoice to customer number Credit card number Card issued to Signature Credit card expiration date We accept American Express Diners Eurocard Master Card and Visa Payment by credit card not available in all countries Signature mandatory for credit card payment 352 Planning and Implementing an IBM SAN Glossary 8B 10B A data encoding scheme developed by IBM translating byte wide data to an encoded 10 bit format Fibre Channel s FC 1 level defines this as the method to be used to encode and decode data transmissions over the Fibre channel Adapter A hardware unit that aggregates other I O units devices or communications links to a system bus ADSM Adstar Distributed Storage Manager Agent 1 In the client server model th
177. decrementing This can help you get to the desired value quickly When you have all fields set to the desired value continue with step 9 Press the gt button The switch will respond Accept Y N Chapter 6 Implementing an IBM SAN Fibre Channel Switch 147 148 Pressing the lt button will indicate Yes Pressing the gt button will indicate No e If you say No the switch will again respond Ethernet Subnetmask and you can restart the process e f you say yes the switch will respond Updating the Config This will cause the new address to be stored After the switch has made the change it will again display Ethernet Subnetmask e If no other address is to be changed Gateway Subnet etc you may simply stop here by pressing the lt button At this point you are done setting up the switch e If you need to set other switch addresses Gateway Domain press the 4 button and the switch will respond Fibre Channel IP address will not be required at this time 10 Press the button and the switch will respond Fibre Channel Subnetmask will not be required at this time 11 Press the button and the switch will respond Gateway Address 12 Press the gt button and the switch will respond by displaying the current Gateway address in the form xxx XXX XXX XXX 13 To change the current Gateway address to the preferred address e Press the lt button to move the cursor the entry point from one field
178. described the technical topology of a SAN detailed Fibre Channel products and IBM SAN initiatives We also designed configurations that were able to maximize the benefits of Fibre Channel products that are currently supported by IBM and available in the marketplace today Where this IBM Redbook picks up the story is how to implement those products that are in the IBM product armory today It is not possible to duplicate each and every SAN installation that is possible feasible or practical What we want to achieve is a consolidated reference guide that details how the basic products can be swiftly and in some cases easily implemented We will show the various features that each of these products enjoys and how the most common and important benefits of each are taken advantage of We will show how they can be employed in some of the more commonly encountered environments and platforms With this in mind in this redbook we have two objectives The first is to show practical decisions to be considered when planning a SAN the second objective is to show how the following products can be installed configured and tailored IBM Enterprise Storage Server with native Fibre Channel IBM Fibre Channel Storage Hub IBM SAN Fibre Channel Managed Hub IBM SAN Fibre Channel Switch McDATA Enterprise Fibre Channel Director IBM Storage Area Network Data Gateway Vicom Fibre Channel SLIC Router Once these products are successfully installed and all thes
179. dwidth so the overall bandwidth is multiplied by the number of connections delivering a maximum of 200 Planning and Implementing an IBM SAN MB second x n nodes The fabric itself is responsible for controlling the routing of information It may be simply a single switch or it may comprise multiple interconnected switches which function as a single logical entity Complex fabrics must be managed by software which can exploit SAN management functions which are built into the fabric Switched fabric is the basis for enterprise wide SANs A mix of these three topologies can be implemented to meet specific needs Fibre Channel Arbitrated Loop FC AL and switched fabric FC SW are the two most commonly used topologies satisfying differing requirements for scalability distance cost and performance A fourth topology has been developed known as slotted loop FC SL But this appears to have limited application specifically in aerospace so it is not discussed in this book Fibre Channel uses a serial data transport scheme similar to other computer networks streaming packets frames of bits one behind the other in a single data line To achieve the high data rate of 100 MB second the transmission clock frequency is currently one gigabit or one bit per 0 94 nanoseconds Serial transfer of course does not suffer from the problem of skew so speed and distance is not restricted as with parallel data transfers as we show in Figure 10
180. e Figure 38 Storage Allocation ESS with FC and SCSI We can see that our ESS is equipped with nine Fibre Channel adapters and three SCSI adapters Graphically the representation of SCSI adapters with two ports per adapter is shown in Figure 39 76 Planning and Implementing an IBM SAN Figure 39 SCSI adapter with two ports The Fibre Channel adapter with one port is graphically represented as shown in Figure 40 Figure 40 FC adapter with one port In the ESS we do not tie a host to a Fibre Channel port in the ESS which was what we did with the SCSI adapters Every host adapter port within the Fibre Channel fabric will be identified by its World Wide Port Name WWPN and can access data through every ESS Fibre Channel WWPN unless other measures are taken To show the independence of the port the selected host is graphically connected to each Fibre Channel adapter as shown in Figure 41 Chapter 4 Configuring the ESS with native Fibre Channel 77 Storage Allocation Graphical View Clear View View All Click on a Host or Array to see paths Figure 41 FC host selected In contrast to this the SCSI attached hosts have an affinity to the SCSI ports on the ESS which is visible by selecting a SCSI host as shown in Figure 42 Clear View View All Storage Allocation Graphical View Click on a Host or Array to see paths Sane jas le Figure 42 SCSI host selected To get a d
181. e 267 226 Advanced SCSI parameters 0 0000 cece eee 268 227 Fibre Channel port parameters 0 0 e eee eee 269 228 IBM Storage Area Network Data Gateway channel zoning 272 229 Enabling Virtual Private SAN 0 anaana 273 230 Loading VPS Registration software on Windows NT 274 231 Specialist display without VPS enabled 0 005 275 232 Specialist after VPS enabled and host registration 276 233 Host system with no host registration software 277 234 SAN Data Gateway with two hosts 00 eee eee eee 278 235 VPS host settings i eiii iaca aaa eo ia a aeai eee 279 236 SCSI LUN assignment 0 00 cece ee 280 237 Service terminal display of device map eee eee eee 281 238 Setting LUN masking 0 0 ee 282 239 Combining channel zoning and VPS 0 000 e ee eee 283 240 Fibre Channel port setting for switch attachment 284 241 Switch registration 6 eee 286 242 Switch port login aaau anaana aana 287 243 IBM SAN Fibre Channel Switch port settings 0 287 244 Changing switch information 0000 cee ee ee 288 245 Adding an IBM switch to the Gateway 00 eee eee 289 246 Switch port information 0 000 cee ee 290 247 Hosts and a switch on a Gateway port 0 20 e eee eee 291 248 Adding two heterogeneous hosts to the
182. e 6 3 2 Zoning components on page 156 e 6 3 3 Methods of implementing zones on page 157 6 3 1 The role of zoning in a SAN Zoning allows for finer segmentation of a switched fabric Zoning can be used to establish a barrier between different environments Only the members of the same zone can communicate within that zone and all other attempts from outside are rejected In our SAN scenario the idea was to limit storage access from one of the servers to only part of the shared storage facility ESS Zoning may be used in your environment to create e Closed user groups e Simplify resource utilization e Facilitate time sensitive functions e Secure fabric areas It is obvious that a growing SAN fabric with increasing complexity needs a segmentation and control tool like zoning For example it may be desirable to separate a Windows NT 2000 environment from a UNIX environment This is very useful because of the manner in which Windows attempts to claim all available storage for itself Because not all storage devices are capable of protecting their resources from any host seeking its available resources it makes sound business sense to protect the environment in another manner By looking at zoning in this way it can also be considered as a security feature and not just for separating environments Zoning can also be used for test and maintenance purposes For example not many enterprises will mix their test and maintenan
183. e Channel Switch 2109 S08 is an 8 port Fibre Channel gigabit switch that consists of a motherboard with connectors for supporting up to eight ports The motherboard is enclosed in an air cooled chassis which may be a standard rack or used as a standalone unit The chassis includes one or two power supplies a fan tray an RJ 45 Ethernet connection for switch set up and management and a serial port Serial Port connection The serial port is used for recovering factory settings only and for the initial configuration of the IP address for the switch if the default address is not known It is not used during normal operation The IBM SAN Fibre Channel Switch 2109 S16 does not have a serial port 140 Planning and Implementing an IBM SAN Ethernet connection It is possible to connect an existing Ethernet 10 100BaseT LAN to the switch using the front panel RJ 45 Ethernet connector This allows access to the switch s internal SNMP agent and also allows remote Telnet and Web access for remote monitoring and testing The IP address may be changed using the Ethernet Port which is shown in Figure 122 RJ 45 Ethernet Connector Senal Port Connector Figure 122 SAN Fibre Channel Switch 2109 S08 Ethernet and serial connectors In Figure 123 we show how the ports are numbered sequentially starting with zero for the left most port The two optional power supplies are shown to the left and right of the switch ports 7 s Power Supp
184. e Channel Switch e McDATA Enterprise Fibre Channel Director e IBM Storage Area Network Data Gateway e Vicom Fibre Channel SLIC Router Once these products are successfully installed and all these configurations have been tested using a hands on environment we will show some of the benefits that we believe are fundamental to their application in a SAN Copyright IBM Corp 2000 xix The team that wrote this redbook XX This redbook was produced by a team of specialists from around the world working at the International Technical Support Organization San Jose Center Jon Tate is a project leader for SAN Storage Solutions at the International Technical Support Organization San Jose Center Before joining the ITSO in 1999 he worked in the IBM Technical Support Center providing Level 2 support for IBM storage products Jon has 14 years of experience in storage software and management services and support George DeBiasi is an IT Specialist in the Americas Storage Techline in Toronto Before joining the Techline in 1998 he provided pre sales and technical support on videoconferencing systems for IBM and its customers George has 14 years of experience as a Technical Specialist providing customer service support and project management He holds a diploma in Electronic Engineering Technology from Radio College of Canada Uwe Hofmann is a Senior Consultant and Sales Support Manager in IBM s Europe Middle East and Africa EME
185. e ESS is at the local site and the other is at the remote site Both sites are connected with three longwave ISLs between the two ED 5000 At the remote site we have the AIX zone where the remote ESS is also a member This example can be used to establish a mirrored set from within the Windows NT Disk Administrator with one local Chapter 7 Implementing the McDATA ED 5000 241 copy of the data and one remote Conversely the AIX zone is limited to the devices at their site 7 5 1 3 Limits for the McDATA multiswitch fabrics McDATA supports only a homogenous environment This means only McDATA ED 5000 are supported as switching elements in a McDATA SAN The McDATA fabric supports up to 31 interconnected ED 5000 Although we can connect many directors the hop count with today s Fibre Channel is limited to two due to the delay that is applied traversing every switch The hop count delay increases with every additional switch between the source and the destination 7 5 2 Setting up our McDATA multiswitch fabric 242 We will use two ED 5000s for our zoning example We configure both directors as we did before First we define one director with its EFC Server and then we configure the second director using the same EFC Server To include the second switch in the fabric of the first we basically need to connect the directors with longwave or shortwave Fibre Channel cables The fabric building process itself is transparent to us as the swi
186. e Ethernet port on the SAN Data Gateway comes from the factory disabled To enable it you must issue the ethEnable command This will not take effect until the unit is rebooted The reboot can occur from a power off or Planning and Implementing an IBM SAN by issuing the reboot command During the reboot you will see that the IP address is set and now enabled Gateway gt ethEnable Write complete Ethernet will be enabled on next boot value 0 0x0 Gateway gt _ 8 2 2 Using Telnet on the SAN Data Gateway If a user would prefer to telnet to the SAN Data Gateway rather than by using the service terminal port after initial setup this can be done First you must create a user from the service terminal by using the useradd command Enter the login name and password using the quotes and comma ITSO gt userfAdd itso residency value 6 6x6 Itso gt F You cannot telnet to the Gateway and use the service port at the same time When you telnet to the Gateway the service port on the rear of the unit will stop its communications After you end the telnet session then the service port will become available again 8 2 3 Startup sequence You must start up the SAN Data Gateway and the attached host and target devices in a specific order When you add or remove SCSI devices or update firmware you must restart The following procedures describe the situations and order of procedure when you restart the SAN Data Gateway
187. e RPKA93_C1B2A1 This means the ESS RPKAQ3 with the FC port in Cluster one Bay two and Adapter one We use the information to include specific adapters in a zone This does not mean that we are able to restrict access to volumes on the ESS by doing this On the ESS every volume is accessible through every FC port and other steps have to be carried out on the ESS itself to accomplish this We detail this in Chapter 4 Configuring the ESS with native Fibre Channel on page 63 Including ESS FC adapters within specific zones might be useful when we want to influence the bandwidth that a specific group of devices zone can get and through which bay we want the data to go For our example we configure a nickname for one RS6000 and two for the ESS This is done by selecting Configure gt Nicknames which opens the window without any nicknames configured We use the Add button to add some nicknames as shown in Figure 166 Planning and Implementing an IBM SAN 4 10 00 08 00 88 40 C0 AD 10 00 00 00 C9 20 DC 71 10 00 00 00 C9 20 E7 C7 10 00 00 00 C9 20 CA 83 10 00 00 00 C9 20 DA 2B 10 00 00 00 C9 20 48 CD Ca a KET Figure 166 Configuring nicknames After adding some nicknames the window looks like Figure 167 Doing this is similar to a hosts file on a TCP IP host The next step to a more user friendly naming of the WWNs can be a dedicated service similar to the DNS Configure Nicknames x World
188. e SLIC storage system can be expanded to include more hosts whether they are homogeneous or heterogeneous It is recommended that as hosts are added each host is connected to its own and separate SLIC Router If more than one host was connected to a single Router there will be arbitration and performance issues Also it would have a single point of failure with the possibility of losing data access to many systems 9 9 3 1 Homogeneous hosts If another host of is added and you would like to have both hosts access the same disks then some sort of access sharing software must be loaded onto both hosts If other hosts are added to the storage system and they will not share data but are connected for storage consolidation there are a few issues to be considered as the Router does not provide a LUN masking capability In UNIX systems the hosts will see all disk in the loop But if the specific volume is not mounted there will be no data integrity problems For Windows NT each host will write its own signature on all available disk Adding another Windows NT host to the loop will cause problems To allow a specific Router and host attached to that Router access to a specific disk or set of disks you can set Private Attributes on the disks Private Attributes is a setting within SLIC manager that can set a disk to only be accessed by a certain Router and in turn the host attached to that Router m Note For more information and o
189. e configurations have been tested using a hands on environment we will show some of the benefits that are fundamental to their application in a SAN SG24 6116 00 ISBN 073841829X Redbooks INTERNATIONAL TECHNICAL SUPPORT ORGANIZATION BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE IBM Redbooks are developed by the IBM International Technical Support Organization Experts from IBM Customers and Partners from around the world create timely technical information based on realistic scenarios Specific recommendations are provided to help you implement IT solutions more effectively in your environment For more information ibm com redbooks
190. e field for Copy Manufacturer s Files From is the drive letter you specified to save the 2 xxxxxxx exe file in step 8 on page 56 Type the name of the current driver file in the field for Copy Manufacturer s Files From after the drive letter prompt Click OK 10 Click OK to exit 11 Restart your host system 3 1 5 Locating the World Wide Port Name WWPN on Windows NT Perform these steps to locate the World Wide Port Name WWPN for the Qlogic 2100F adapter card and the Qlogic 2200F adapter card 1 2 Restart the server Press and hold the Alt key and press the Q key to get to the FAST Util command panel If you have more than one Fibre Channel adapter installed you will see a window that displays all the Fibre Channel adapters Scroll down to the adapter you want and press Enter a From the Fast Util menu scroll down and highlight Select Host Adapter b Scroll up and highlight Configuration Settings Press Enter Chapter 3 Implementing Fibre Channel host adapter cards 57 c From the Configuration Settings menu select Host Adapter settings d Write down the host adapter name for example 200000E08B00C2D5 e Scroll down and highlight Select Host Adapter settings 3 2 Installing Fibre Channel host adapter cards on a RS 6000 system In the following sections we cover these topics e 3 2 1 The 2105 host attachment package on page 58 e 3 2 2 Before you install the 2105 host attachme
191. e links can operate concurrently between any two switches in the fabric allowing multiple redundant paths to be defined All ISLs carry traffic In the event of a link failure the traffic it was carrying will be automatically and immediately transferred to other links Adding ISLs will automatically cause routing and zoning information to be updated across all ISLs Changing ISL configurations causes recalculation of routes within the fabric This task is a load on all switches in the fabric so numerous changes should be avoided The maximum number of ISLs from one switch to a single adjacent switch is eight but more than eight ISLs can be configured from a single switch if they attach to several other switches This is shown in Figure 16 Planning and Implementing an IBM SAN Switc Switc Switch Swite pe ee meo AT a Switch Switc Switch Switch Switc Switch Switch Switch Switch Switch Switch Switch Switch Valid Configuration using 16 ISLs Jooo0o00 Switch Switch Invalid Configuration using 9 ISLs between 2 switches Figure 16 Valid and invalid inter switch links 2 3 2 3 Distributed name server If a switch fails the other switches in the fabric and the nodes attached to them are unaffected Nodes attached to the failed switch are of course unable to talk to each other or to nodes on other switches However this probl
192. e most pragmatic Solve specific application needs now to deliver value to your organization This does not mean that you should not establish some common guidelines or standards regarding the purchase of equipment within the enterprise This will facilitate interoperability in the future and avoid dead end investments which cannot be integrated in a larger SAN environment as you expand the topology in the future You may decide that there are a number of discrete and independent operating environments within your organization and these will not need to be interlinked in the future If so you may choose to establish SAN islands which are configured with different topologies and components in which cross island interoperability is not required Chapter 2 SAN planning and design considerations 33 A strong trend is towards switched fabric environments This is because fabric offers greatest flexibility and scope for the future You may choose to install small FC AL topologies now for reasons of cost or because the application being addressed is small scale today If so there is good logic in selecting hardware such as the IBM 3534 Managed Hub This gives you flexibility for the future such as more sophisticated function manageability and upgrade ability and with compatibility within a family of fabric devices Remember that FC AL is not designed for high performance It does not scale As you add more devices on a loop performance will tend
193. e name as shown in 224 Planning and Implementing an IBM SAN Figure 186 This is not allowed so to circumvent this problem we changed it to NT_Zone_1 NT Zone 1 RPKA93_C1B2A1 Domain 1 QLogic 20 00 00 E0 88 00 52 5E Port 2 Emulex 10 00 00 00 C9 20 DC 71 Emulex 10 00 00 00 09 20 C4 E6 Port 3 Ermulex 10 00 00 00 C9 20 E7 C7 Port 4 Emulex 10 00 00 00 C9 20 E7 F3 Port5 KATE_FCSO Port 6 RPKA93_C1B2A1 Port RPKA93_C1B3A1 Figure 186 Defining the zone We illustrate that we can specify the members in one zone based on the WWN and other members based on the port they are connected to as shown in Figure 187 Chapter 7 Implementing the McDATA ED 5000 225 NT_Zone_1 RPKAS3_C1B2A1 3 Domain 1 Domain 1 Port 1 Port 0 Domain 1 Port 8 Port1 R Port 2 Emulex 10 00 00 00 C9 20 DC 7 Port 3 Emulex 10 00 00 00 C9 20 E7 C Port 4 Emulex 10 00 00 00 C9 20 E7 F Port 5 KATE_FCSO Port 6 RPKA93_C1B2A1 Port 7 RPKA93_C1B3A1 Port9 Port10 Port11 Figure 187 Members in zone based on WWN and port After selecting Save Zone we return to the New Zone Set window where we can view the zones in the Zone Library in the right half of the window To view the zone members highlight the zone in the Zone Library and then go to Actions gt View Zone Members as shown in Figure 188 226 Planning and Implementing an IBM SAN ITSO_Zone_Set_1 View Zone Members RPKAS3_C1B2A1 QLogic 20 00 00 E0 88 00
194. e part of the system that performs information preparation and exchange on behalf of a client or server application 2 In SNMP the word agent refers to the managed system See also Management Agent AIT Advanced Intelligent Tape A magnetic tape format by Sony that uses 8mm cassettes but is only used in specific drives AL See Arbitrated Loop ANSI American National Standards Institute The primary organization for fostering the development of technology standards in the United States The ANSI family of Fibre Channel documents provide the standards basis for the Fibre Channel architecture and technology See FC PH Arbitration The process of selecting one respondent from a collection of several candidates that request service concurrently Arbitrated Loop A Fibre Channel interconnection technology that allows up to 126 participating node ports and one participating fabric port to communicate ATL Automated Tape Library Large scale tape storage system which uses multiple tape drives and mechanisms to address 50 or more cassettes ATM Asynchronous Transfer Mode A type of packet switching that transmits fixed length units of data Copyright IBM Corp 2000 Backup A copy of computer data that is used to recreate data that has been lost mislaid corrupted or erased The act of creating a copy of computer data that can be used to recreate data that has been lost mislaid corrupted or erased Bandwidth Measure
195. e platforms subsystems and software tools included in the design Additionally you will need project management skills to coordinate the whole exercise 2 4 2 Equipment selection The detailed list of logical and physical connections required in the SAN should act as the basis for defining your fabric hardware requirements and for arriving at an estimated implementation cost Now you are ready to make the final selection with your vendor 2 4 3 Interoperability testing No doubt you will want to ensure that the SAN solution you are designing will operate correctly in practice As industry standards are still under development this is particularly pertinent You may select a pretested and certified solution in which case there is little or no risk Vendors throughout the industry are testing their hardware and software in many differing environments IBM Global Services has made major investments in laboratories in the USA and Europe to help you with such testing Details of tested and certified solutions are constantly being updated and are posted on the IBM SAN Web site 2 4 4 Documentation As with any project you will need to fully document the SAN plan This should include details about most of the topics already discussed in this chapter e Business goals e Technical requirements e Infrastructure current and planned e Cabling e Traffic characteristics today and expected e Platforms and storage devices current and pl
196. e that as the first dual Fibre Channel port on the Gateway is used it is numbered as input 1 and 4 A description on the Fibre Channel port numbering is shown in Figure 213 on page 252 Chapter 8 Implementing the SAN Data Gateway 277 Server Server Fibre i i unknown Channel plymouth a Gateway L ESS Figure 234 SAN Data Gateway with two hosts 8 5 2 4 Adding host and connection information The host name and host type and connection information can be added manually by selecting the VPS feature and modifying the data To do this select the desired SAN Data Gateway so that it is highlighted Select Controls gt Access Options gt Virtual Private SAN and you will enter into the VPS settings window The hosts and their information will appear on the left hand side Select the host to be modified and the information is transferred to the bottom part of the window where the modifications can take place as shown in Figure 235 278 Planning and Implementing an IBM SAN BA virtual Private SAN Access Settings ACTIVE Assigned LUNs EE On Line EE Off Line 0 1 2 3 4 5 6 7 8 9 10 11 12 13 unknown unknown unknown 10000000 c921da0a plymouth AIX dewifscsi0 Oxef 0 Emulex 10000000 09221802 KI gt New Host Data Original Host Data Close Host Name unknown unknown New Host H
197. ear on the left hand side of the drive just selected in the Available Drive window to denote that the drive has been selected A second or third drive can be added to create a 2 way or 3 way mirror respectively Each drive is added one at a time To remove a drive from the Member window select the desired drive and click on the Remove lt lt button An example of adding two drives to create a 2 way mirror is shown in Figure 271 Mirror Drive Setup Wizard Mirror Drive Members Selection Mirror Drive Members Available Drive Pool 700000 LOOO 35E68229 TO0003 35E6848D TOO0003L003 35 6848D TO0004 35E22D6C TO0004L004 35E22D6C 700005 L005 35E6840E TO0006 LOOG 35E 68462 Identify 700007 LOO 35E6844E 733025 L001 ITSO1 ass Reset Drive Capacity MB Mirror Capacity MB 8689 lt Back Next gt Cancel Figure 271 Mirror drive member selection Below each window there is a Capacity window that will display the size of the available drive or of the mirror drive Each drive that participates in a mirror should be of equal capacity If you select a drive that has a greater capacity Planning and Implementing an IBM SAN and add it to the mirror the mirror capacity will still be the smaller of the two and the rest of the capacity of the larger drive will be unused For example if you added a 18 GB drive to the mirror in Figure 271 the Mirror Capacity w
198. ecified by node name then all ports on that device are in the zone When a zone member is specified by port name then only that single device port is in the zone The type of zone members used to define a zone may be mixed and matched For example a zone defined with the following members 2 12 2 14 10 00 00 60 69 00 00 8a This will contain whichever devices are connected to switch 2 ports 12 and 14 and the device with either node name or port name of 10 00 00 60 69 00 00 8a to whichever port in the fabric it is connected The number of members possible in a zone is limited only by the amount of memory in the fabric switch A member can belong to multiple zones You can define multiple sets of zones for the fabric but only one set can be active at any time You can activate another zone set any time you want without the need to power down the switch With software zoning there is no need to worry about the physical connections to the switch If you use WWNs for the zone members even when a device is connected to another physical port it will still remain in the same zoning definition because the device s WWN remains the same There is a potential security leak with software zoning When a specific host logs into the fabric and asks for available storage devices the SNS will look into the software zoning table to see which storage devices are allowable for Chapter 6 Implementing an IBM SAN Fibre Channel Switch 159 160 that
199. ed To the right of the window in Figure 185 we are presented with all of the McDATA connected Fibre Channel adapters and their WWNs To the left we can view all the members currently defined for the zone There are two check Chapter 7 Implementing the McDATA ED 5000 223 boxes where we can choose if we want to assign a McDATA port to a zone or if we want to specify directly the machines based on the WWN of their Fibre Channel adapter In our example we chose to add the Windows NT servers regardless of which port they are connected to To assign a Fibre Channel adapter to the zone we drag and drop the WWN associated with the Fibre Channel adapter of our Windows NT server to the left part of the window This is what we will do with the ESS The NT host we want to define for this zone is currently detached but we can also define this host using this window To perform this we use the Actions button and select Add Detached Node We get a data entry field where we insert the WWN of the node as shown in Figure 185 NT Zone 1 RPKAS3_C1B2A1 3 Domain 1 Port 2 Emulex 10 00 00 00 C9 20 DC 71 Port 3 Emulex 10 00 00 00 C9 20 E7 C7 Port 4 Emulex 10 00 00 00 C9 20 E7 F3 PortS KATE_FCSO Port 6 RPKA93_C1B2A1 Port 7 RPKAG3_C1B3A1 Add Detached Node Figure 185 Adding a detached Fibre Channel node While trying to save the zone using the Save Zone button we got an error message This is because of the spaces in the zon
200. ed by IBM Tivoli will allow data to be read directly from disk to tape and tape to disk bypassing the server This solution is known as server free backup 1 5 5 High performance Applications benefit from the more efficient transport mechanism of Fibre Channel Currently Fibre Channel transfers data at 100 MB second several times faster than typical SCSI capabilities and many times faster than standard LAN data transfers Future implementations of Fibre Channel at 200 MB second and 400 MB second have been defined offering the promise of even greater performance benefits in the future Indeed prototypes of storage components which meet the two gigabit transport specification are already in existence and may be in production in 2001 The elimination of conflicts on LANs by removing storage data transfers from the LAN to the SAN may also significantly improve application performance on servers 1 5 6 High availability server clustering Reliable and continuous access to information is an essential prerequisite in any business As applications have shifted from robust mainframes to the less reliable client file server environment so have server and software vendors developed high availability solutions to address the exposure These are based on clusters of servers A cluster is a group of independent computers managed as a single system for higher availability easier manageability and greater scalability Server system components are
201. ee only one switch and not the complete fabric as depicted in Figure 144 176 Planning and Implementing an IBM SAN EEA AE Pact Figure 144 Fabric view panel To make the switch known in the fabric you must take the following actions 1 Telnet to the switch that you are adding for example telnet 9 113 24 57 and press Enter 2 Login enter your userid and password issue the command switchdisable and press Enter This is necessary to modify switch parameters 3 Issue the configdefault Command and press Enter to set the switch s configuration parameters back to the factory default settings Any zone information that was in this switch will be deleted 4 The configdefault command is navigated by entering a series of collapsible top level menus However there is another alternative that you can try before issuing the configdefault command If you know the configuration parameters that relate to your environment you can enter the configure command Then e Each menu divides up the various switch configuration parameters into logical groupings These are fabric parameter virtual channel parameters arbitrated loop parameters and system service parameters Enter the parameters that reflect your environment 5 Issue the switchenable command 6 Issue the reboot command After gt switch admin gt configDefault the switch will answer with committing configuration done Chapter 6 Implementing an IBM SAN Fibre Channel
202. eek implementing Enterprise Resource Planning and Business Intelligence and cannot tolerate outages of such mission critical applications then a SAN can solve your problems In other words you need to identify the pain levels associated with data movement data sharing data growth and so on in your own organization Then you can quantify how a SAN will contribute to your ability to achieve the Copyright IBM Corp 2000 27 levels of service demanded by your business managers This might be quantified in terms of improved communications within the business and externally with customers It can be a matter of improving the ability of managers and employees to make good business decisions due to better information availability You might be measuring the security and reliability of mission critical applications or reducing costs of storage hardware and skilled human management resources The need may be to establish flexible adaptable IT services to stay in the race with competitors who are not constrained by legacy applications or inflexible IT infrastructures The focus can be on the ability to increase revenue and profit with effective growth of new e business services Whatever the level of the pain you need to understand your existing IT infrastructure which are the mission critical applications and what are the business goals and directions 2 1 2 Technical goals 28 When you understand the business goals the
203. efines the communications between different fabric elements required for those elements to coordinate their operations and management address assignment 355 FC Storage Director See SAN Storage Director FCA Fibre Channel Association a Fibre Channel industry association that works to promote awareness and understanding of the Fibre Channel technology and its application and provides a means for implementers to support the standards committee activities FCLC Fibre Channel Loop Association an independent working group of the Fibre Channel Association focused on the marketing aspects of the Fibre Channel Loop technology FCP Fibre Channel Protocol the mapping of SCSI 3 operations to Fibre Channel Fiber Optic Refers to the medium and the technology associated with the transmission of information along a glass or plastic wire or fiber Fibre Channel A technology for transmitting data between computer devices at a data rate of up to 4 Gb s It is especially suited for connecting computer servers to shared storage devices and for interconnecting storage controllers and drives FICON Fibre Connection A next generation I O solution for IBM S 390 parallel enterprise server FL_Port Fabric Loop Port the access point of the fabric for physically connecting the user s Node Loop Port NL_Port FLOGI See Fabric Log In Frame A linear set of transmitted bits that define the basic transport unit The frame is the most basi
204. efines the concepts behavior and characteristics of the Fibre Channel Fabric along with suggested partitioning of the 24 bit address space to facilitate the routing of frames FC FP Fibre Channel HIPPI Framing Protocol A reference to the document ANSI X3 254 1994 defining how the HIPPI framing protocol is transported via the fibre channel FC GS Fibre Channel Generic Services A reference to the document ANSI X3 289 1996 describing a common transport protocol used to communicate with the server functions a full X500 based directory service mapping of the Simple Network Management Protocol SNMP directly to the Fibre Channel a time server and an alias server FC LE Fibre Channel Link Encapsulation A reference to the document ANSI X3 287 1996 which defines how IEEE 802 2 Logical Link Control LLC information is transported via the Fibre Channel FC PH A reference to the Fibre Channel Physical and Signaling standard ANSI X3 230 containing the definition of the three lower levels FC O FC 1 and FC 2 of the Fibre Channel FC PLDA Fibre Channel Private Loop Direct Attach See PLDA FC SB Fibre Channel Single Byte Command Code Set A reference to the document ANSI X 271 1996 which defines how the ESCON command set protocol is transported using the fibre channel FC SW Fibre Channel Switch Fabric A reference to the ANSI standard under development that further defines the fabric behavior described in FC FG and d
205. egy of sharing fewer highly reliable powerful tape solutions such as IBM s Magstar family of drives and automated libraries between multiple users and departments 1 5 9 Ease of data migration Data can be moved non disruptively from one storage subsystem to another using a SAN without server intervention This may greatly ease the migration of data associated with the introduction of new technology and the retirement of old devices Chapter 1 Introduction to Storage Area Networks 23 1 5 10 Reduced total costs of ownership Expenditure on storage today is estimated to be in the region of 50 of a typical IT hardware budget Some industry analysts expect this to grow to as much as 75 by the end of the year 2002 IT managers are becoming increasingly focused on controlling these growing costs 1 5 10 1 Consistent centralized management As we have shown consolidation of storage can reduce wasteful fragmentation of storage attached to multiple servers It also enables a single consistent data and storage resource management solution to be implemented such as IBM s StorWatch tools combined with software such as Tivoli Storage Manager and Tivoli SAN Manager which can reduce costs of software and human resources for storage management 1 5 10 2 Reduced hardware costs By moving data to SAN attached storage subsystems the servers themselves may no longer need to be configured with native storage In addition the introduction of
206. em can be overcome since any node can have several Fibre Channel interfaces each one attached to nodes on different switches in the fabric This is illustrated in Figure 17 If any link fails every switch can still communicate with all the other switches IBM 2109 switches use a distributed fabric wide name server This means that the name server is fully distributed to each switch therefore ensuring no single point of failure When end nodes attached to servers and devices want to communicate to other nodes across the fabric any switch provides information about the devices connected to the fabric by means of the distributed name server even in the event of a failed switch Chapter 2 SAN planning and design considerations 39 40 A Fault Tolerant Fabric Design Switch Switch Figure 17 A fault tolerant fabric design 2 3 2 4 Fabric Shortest Path First FSPF FSPF is the path selection protocol used by the IBM 2109 switch It automatically calculates the best path between any two switches in the fabric when switches are powered up It establishes all the routes across the fabric and these change only in the event of a failure or if a new ISL is created which offers an equal or better path to a given target FSPF is very resilient to failures of hardware and software automatically computing an alternate path around a failed link typically in less than one second If several equivalent paths are availab
207. en two nodes with full duplex bandwidth 100 MB second in each direction concurrently e Arbitrated loop A unidirectional ring topology similar to a token ring supporting up to 126 interconnected nodes Each node passes data to the next node in the loop until the data reaches the target node All nodes share the 100 MB second Fibre Channel bandwidth Devices must arbitrate for access to the loop Therefore with 100 active devices ona loop the effective data rate for each is 1 MB second which is further reduced by the overhead of arbitration A loop may also be connected to a Fibre Channel switch port therefore enabling attachment of the loop to a wider switched fabric environment In this case the loop may support up to 126 devices Many fewer devices are normally attached in practice because of arbitration overheads and shared bandwidth constraints Due to fault isolation issues inherent with arbitrated loops most Fibre Channel arbitrated loop FC AL SANs have been implemented with a maximum of two servers plus a number of peripheral storage devices So FC AL is suitable for small SAN configurations or SANlets e Switched fabric The term fabric describes an intelligent switching infrastructure which delivers data from any source to any destination The interconnection of up to 2 4 nodes is allowed with each node able to utilize the full 100 MB second duplex Fibre Channel bandwidth Each logical connection receives dedicated ban
208. enting an IBM managed and unmanaged hub 111 5 1 ITSO environment A E A T ees 111 5 1 1 Hardware overview 0 0 cee 112 5 1 2 Software overview 0 0 ee 112 5 1 3 Configuring the ESS 0 0 0 0 ee ee 112 5 1 4 Configuring the host bus adapters 4 112 5 2 Configuring the unmanaged hub 0 0000 eee eee 112 5 2 1 IBM Fibre Channel Storage Hub 0 113 Planning and Implementing an IBM SAN 5 2 2 IBM Fibre Channel Storage Hub 2103 HO7 114 5 2 3 Hub configuration 2 ee 114 5 2 4 Hub installation 0 0 0 0 eee 115 5 2 5 Checking disk acceSS 0 0 e eee eee 117 5 3 IBM SAN Fibre Channel Managed Hub 120 5 3 1 The ITSO environment 0000 ee 121 5 3 2 The hardware involved 0000 eee eee ees 121 5 3 3 The software involved 000 c eee es 122 5 4 Installing the IBM SAN Fibre Channel Managed Hub 35341RU 122 5 4 1 Setting the IP address 0 0 eee ee 122 5 5 ZONING ace dee eset aa Gels Sik fa aed GN ete God A Ghat elie a ek ey Ge 127 5 6 Cascading whats MAS 4a tade dear had eed cht halen Ohad EE 128 DT QUICKLOOPt sey stereo G4 Ware hea areiced 4 Ya aie hee als 129 SAI Private OOP diy bdacieert nadaeed aep eaa Aone ds 130 5 722 PubliGloop ais sndew el hee pha ed OS ea aed Bae tas 131 5 7 3 Private devices in public fabrics 0 0005
209. er management costs increased flexibility and increased control This can be achieved physically or logically 1 5 1 1 Physical consolidation Data from disparate storage subsystems can be combined on large enterprise class shared disk arrays which may be located at some distance from the servers The capacity of these disk arrays can be shared by multiple servers and users may also benefit from the advanced functions typically offered with such subsystems This may include RAID capabilities remote mirroring and instantaneous data replication functions which might not be available with smaller integrated disks The array capacity may be partitioned so that each server has an appropriate portion of the available GBs This is shown in Figure 11 Chapter 1 Introduction to Storage Area Networks 17 Consolidated Storage Server A Server B Free space Shared Disk Array Figure 11 Consolidated storage efficiently shared capacity Available capacity can be dynamically allocated to any server requiring additional space Capacity not required by a server application can be reallocated to other servers This avoids the inefficiency associated with free disk capacity attached to one server not being usable by other servers Extra capacity may be added in a non disruptive manner 1 5 1 2 Logical consolidation It is possible to achieve shared resource benefits from the SAN but without moving existing equipment A SAN rel
210. ere may be duplicate members The defined configuration is the current state of the administrator s input Effective configuration The effective configuration is a single zone configuration that is currently in effect The devices that an initiator sees are based on this configuration The effective configuration is built when you enable a specified zone configuration This configuration is compiled by checking for undefined zone names or zone alias names or other inconsistencies by expanding zone aliases removing duplicate entries and then building the effective configuration e Saved configuration The saved configuration is a copy of the defined configuration plus the name of the effective configuration which is saved in flash memory by the cfgSave command There may be differences between the saved Planning and Implementing an IBM SAN configuration and the defined configuration if you have modified any zone definitions and have not saved these The saved configuration is automatically reloaded by the switch during power up and if a configuration was in effect when it was saved the same configuration is reinstated with an automatic cfgEnable command 6 4 Implementing zoning In the following sections we cover these topics e 6 4 1 Adding a second host to the ESS on page 161 e 6 4 2 Setting up zones in your SAN fabric on page 162 6 4 1 Adding a second host to the ESS Once the 2109 S08 switch is cor
211. erial number of the SAN Data Gateway For units with serial numbers higher than 1300600 the VPS feature is bundled into the product Type the word enable and the feature will now be enabled FA Storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc File View Admin Tools Virtual Private Gateway ITSO lt 3 SCSI Channel x Access Options gt SCSI Channel pt Data Mover Name 10000060 45160E08 1 ia IP Address K SCSI Channel 3 SNMP Community Strings ress 9 113 24 110 4 S Serial Number 21081300724 Scal Channel SCSI Channel Firmware Revision 3 40 10 gt Fibre Channel 1 i 10000000 92 Fibre Channel Health Check Level 2 47 Fibre Channel 4 Device Parameters jaaa miara Lae Update Firmware Receive Event Traps Enabled Restart SAN Data Gateway Serial Baud Rate 19200 Identify SAN Data Gateway Figure 229 Enabling Virtual Private SAN 8 5 2 2 Host registration With VPS enabled the host registration service is also available This service is provided to simplify VPS configurations It provides the host name host type host connection and the unique WWN of the Fibre Channel host bus adapter Host information is sent over the Fibre Channel connection to the Gateway Chapter 8 Implementing the SAN Data Gateway 273 To obtain this information a program is loaded on each host This program is found and can be downloaded at the same
212. es Received 6178 Encd Errs Outside Frames 21 C2 Frames Received oO C3 Frames Discarded 0 C3 Frames Received 6176 LIP Ins Link Control Frames Received 1 LIP Outs Mcast Frames Received o Last LIP Received Mcast Timeouts o Frames Rejected Mcast Frames Transmitted 0 Frames Busied Time R_RDY Priority 737422675 Link Failure Time BB_Credit Zero 1024 Loss of Syne Encd Errs Inside Frames 0 Loss of Signal Frames with CRC Errs 0 Figure 130 L Port host connection Our Netfinity host is working in arbitrated loop mode and we need to modify this to reflect our environment Depending on the host fibre channel adapter cards used a similar operation to this may have to be performed The vendor will have to be consulted to select the correct steps for your installed adapter card In our case we were using the EMULEX LP7000 so we referred to that vendors documentation in conjunction with our switch installation instructions to configure our adapter to support F Port 6 3 Zoning in an IBM SAN Fibre Channel Switch environment At this stage of the chapter we have implemented a simple SAN consisting of one host one switch and an ESS connected through point to point Fibre Channel 154 Planning and Implementing an IBM SAN Now we use Zoning to allow for finer segmentation of the access to storage devices from different hosts In these sections we describe e 6 3 1 The role of zoning in a SAN on page 155
213. es are to be connected later A hub can also be used to connect to a remote location to extend the distance Netfinity NT Server Shortwave fiber Shortwave fiber Unmanaged Hub ESS native FC Figure 96 Simple loop setup Copyright IBM Corp 2000 111 5 1 1 Hardware overview Here is the hardware setup e 1 IBM Fibre Channel Storage Hub 2103 H07 7 ports 2 shortwave gigabit interface converters GBICs 2010 2 shortwave cables 9701 5m 9702 25m e 1 Netfinity NT Server 5500 1 Fibre Channel FC HBA QLogic 2100 FC_AL e 1 IBM ESS 2105 F20 with Feature Code 3022 native FC adapter 9 FC adapters 9 ports 3 SCSI adapters 6 ports 5 1 2 Software overview Here is the software setup e Netfinity Microsoft Windows NT 4 0 Service Pack 5 e QLogic 2100 e ESS Microcode 1 1 5 1 3 Configuring the ESS To configure the ESS refer to Chapter 4 Configuring the ESS with native Fibre Channel on page 63 5 1 4 Configuring the host bus adapters To configure the QLogic 2100 refer to 3 1 1 Installing the QLogic 2100F adapter card on page 53 5 2 Configuring the unmanaged hub Our first SAN scenario consists of an IBM Enterprise Storage Server 2105 F20 IBM ESS F20 with native Fibre Channel attachment linked to an IBM Netfinity 5500 server through an IBM 2103 H07 Fibre Channel Storage Hub Our planned configuratio
214. ess a single copy of data stored on a consolidated storage subsystem If attached servers are heterogeneous platforms for example a mix of UNIX and Windows NT sharing of data between such unlike operating system environments is complex This is due to differences in file systems data formats and encoding structures IBM however uniquely offers a true data Chapter 1 Introduction to Storage Area Networks 19 sharing capability with concurrent update for selected heterogeneous server environments using the Tivoli SANergy File Sharing solution The details can be found at http www sanergy com The SAN advantage in enabling enhanced data sharing may reduce the need to hold multiple copies of the same file or database This reduces duplication of hardware costs to store such copies It also enhances the ability to implement cross enterprise applications such as e business which may be inhibited when multiple data copies are stored 1 5 3 Non disruptive scalability for growth There is an explosion in the quantity of data stored by the majority of organizations This is fueled by the implementation of applications such as e business e mail Business Intelligence Data Warehouse and Enterprise Resource Planning Industry analysts such as external storage consulting groups estimate that electronically stored data is doubling every year In the case of e business applications opening the business to the Internet there have been
215. etailed view of the assigned volumes we use the Tabular View button to get to the view as shown in Figure 43 78 Planning and Implementing an IBM SAN Storage Allocation Tabular View a List of Assigned Volumes 000 5 GB 01D FCAI3 Open System 000 5 GB D 00 LUN 501D O1E FCAI3 Open System 000 5 GB Fibre Charmel ID 00 LUN 501E O1F FCA13 Open System 000 5 GB Fibre Charmel T 00 LUN 501F Figure 43 Storage Allocation tabular view To perform the configuration tasks that we need we press the Open System Storage button as shown in Figure 44 HEP DESOT Open System Storage gt Figure 44 Entry for open system storage The S 390 Storage button is grayed out because there are no ESCON cards installed 4 1 3 Accessing the Open System Storage panel This will lead us to the Open System Storage panel as shown in Figure 45 The buttons at the bottom of the panel are the entry points to all of our host and storage configuration tasks In this section we will indicate our use of those buttons by pointing the mouse on the button Chapter 4 Configuring the ESS with native Fibre Channel 79 Open System Storage a pee M or hig noe5100 PC Save WRT 40 aaa a or hig Assigned Volumes Total 1 volumes Votume Vol Type Size Storage Type Location 165 FCA13 Open System 01 0 GB RAID Array Device Adapter Pair 1 LSS 011 Yes Chister 2 Loop A Array 1 Vol 101 Modify Host Systems Configure Host Adap
216. everal benefits to bridge the legacy gap as storage products migrate from SCSI based attachments to Fibre Channel The IBM Storage Area Network Data Gateway allows you to e Protect legacy storage equipment while utilizing the latest host servers with Fibre Channel support Copyright IBM Corp 2000 251 Expand connectivity to storage devices with use of IBM SAN hubs switches and directors Perform channel zoning and LUN masking capability to allow access ata volume level Overcome the distance limitations of SCSI based host systems using longwave ports that support distances up to 10 km Utilize the StorWatch SAN Data Gateway Specialist which is an easy to use interface for managing and controlling access of host systems to storage devices The SAN Data Gateway is available as a rack mount unit or as a stand alone tabletop unit The gateway model provides two shortwave Fibre Channel ports and four Ultra SCSI Differential ports to attach disk or tape storage devices One or two Fibre Channel cards dual port shortwave and or single port longwave may be added for a maximum of six shortwave ports or two shortwave and two longwave ports If you are using the dual port shortwave cards Figure 213 depicts the port assignment numbers for the optical interfaces SAN Connection Port Number Assignments Figure 213 SAN connection port assignment 8 2 Installation 252 Before any server or device connection is made to
217. ew gt Refresh SAN Data Gateway Once a device has been assigned a LUN ID by the Gateway it will maintain that ID since it was written into the device map This is useful in case the device is lost or needs to be replaced Remove the old device set the new device to the same SCSI bus target and LUN as the old device and attach it to the same channel You must rescan the SAN Data Gateway SCSI bus for it to update its data You will notice that the replaced device has kept the same assigned LUN If a device is no longer attached or no longer required then the assigned LUN is also no longer required To free up this assigned LUN ID you can issue the mapWinnowDatabase Command from the service terminal The existing devices will maintain their assigned LUN IDs If a complete rebuild of the SCSI device map is desired or required this can be done from the service terminal only You would issue the mapRebuildDatabase command This command deletes the existing device map and the SAN Data Gateway will restart When it is rebooted a new scan of the SCSI buses is done as if the system was brand new and assign LUN IDs as described earlier When this command is issued you will also be given an option to clear the Virtual Private SAN VPS access settings Because this allows host access to specific LUNs and by issuing this command we may change the assigned LUN ID it is recommended that you always say yes to this option If not a host may access a vo
218. existing fabric e You can easily increase the distance between various SAN participants e By adding more switches to the fabric you increase connectivity by providing more available ports e Cascading provides high resilience in the fabric e With inter switch links ISL you can increase the bandwidth The frames between the switches are delivered over all available data paths So the more ISLs you create the faster the frame delivery will be However careful consideration must be employed to ensure that a bottleneck is not introduced e When the fabric grows the SNS is fully distributed across all the switches in fabric e With cascading you also provide greater fault tolerance within the fabric 6 5 2 Cascading examples A multiswitch fabric offers more flexibility to build a fault tolerant system 172 Planning and Implementing an IBM SAN The sophisticated path selection protocol allows multiple links to operate at the same time between any two switches in the fabric so multiple redundant paths can be created The extra links do not have to be reserved and maintained idle all links carry traffic If a link goes down the traffic that was carried over that link will be simply transferred to the other link s This transfer takes places automatically with no human intervention and in a very short time Even if a switch goes down all other switches in the fabric and the end nodes connected to them are not affected
219. f addresses We changed the default IP address to reflect the ITSO environment To change an existing IP address you can either use the Ethernet port and Telnet commands entered from a server which has access to the same LAN or you can use the switch serial port using a serial terminal emulator The IBM SAN Fibre Channel Switch 2109 S16 allows you to change the IP address using the control panel on the front of the switch We document this in 6 2 3 Setting the IP address on an IBM 2109 S16 on page 146 In our installation we will show how to use the serial port connector shown in Figure 124 to set the IP address RJ 45 Ethernet Connector Senal Port Connector Figure 124 RJ 45 Ethernet Connector and Serial Port Connector 6 2 2 Setting the IP address using the serial port 2109 S08 only 144 Here are the steps to set the IP address using the serial port 1 Connect a system with a serial terminal emulator to the serial port prior to powering up the switch 2 Start up a terminal emulation session hyperterm 3 Power up the switch If two supplies are present power up both As the switch goes through its diagnostics it will post messages to your service terminal When the switch is fully powered on the switch will respond Admin gt 4 Admin gt ipAddrSet this is the command to set the IP address press Enter Planning and Implementing an IBM SAN 5 Ethernet IP address Current ipaddress or None new
220. ficial If the hub with its arbitrated loop devices is part of a larger SAN fabric that includes multiple 2109 switches it is possible to expand the number of ports in the QuickLoop called looplets to the total of the hub ports plus the ports of one IBM SAN Fibre Channel Switch For example e 7 hub ports 7x 2109 S08 ports total of 14 NL_Ports e 7 hub ports 15x 2109 S16 ports total of 22 NL_Ports There are two options with regard to how a hub is employed in this manner e As a single hub all looplets of a QuickLoop reside in one hub e In a Hub Switch combination looplets of a QuickLoop span across hub plus switch Planning and Implementing an IBM SAN In Figure 116 we show high performance multi target connectivity to a single PLDA where all the ports of the hub operate in QuickLoop mode The hub serves as a concentrator Figure 116 QuickLoop using managed hub In Figure 117 we show how the QuickLoop feature supports legacy devices The legacy devices refer to devices that are in a PLDA environment The QuickLoop feature allows these devices to be attached to a Storage Area Network and operate no differently than if they were in a PLDA environment When QuickLoop mode is enabled all ports defined as QuickLoop ports on the switch behave as if they are one logical Fibre Channel Arbitrated Loop The switch needs the QuickLoop facility available from IBM as RPQ 8S0521 RPQ 8S
221. fied volumes have been successfully assigned to the specified target hosts Unsigned Java Applet Window Figure 92 Volume Assignments successfully changed So if we press OK we do not change back to the Open System Storage panel Instead we see the volume assigned to the host Fibre Channel port itso as shown in Figure 93 Chapter 4 Configuring the ESS with native Fibre Channel 107 Modify Volume Assignments a Volume Assignments no sort no sort no sort no sort gt no sort no sort no sort no sort gt W3F FCA13 Device Adapter Pair 17 Open System 000 5 GB RAD amy Bay3 Cadal fud ee SCSI Port B 1 Vol 063 TD 02 LUN 000F 068 FCAI3 Device Adapter Pair 1 Open System 096 0 GE RAD amy Fibre Charmel Chister 1 Loop B T 00 LUN 0002 2 Vol 104 067 FCA13 Device Adapter Pair 1 Open System 096 0 GBE RAD Amay Pitre Charmel Chister 1 Loop B D 00 LUN 0001 2 Vol 103 165 FCA13 ten Pair 1 Ope System 001 0GB RAD ara haster 2 Loop A D rogers LUN roi Action Assign selected volume s to target hosts J Use sare ID Lun in source and target ey Unassign selected volume s from target hosts Perform Configuration Update Cancel Configuration Update Figure 93 To itso assigned volume The same volume is also assigned to the host Fibre Channel port of our other Fibre Channel host Netfinity_ITSO_1 as shown in Figure 94 068 FCAl3 Device Adapter Pair 1 Open System 096 0 GB e
222. frastructure to meet the requirements of the business One of the first steps to improved control of computing resources throughout the enterprise is improved connectivity In the topics that follow we look at the advantages and disadvantages of the standard storage infrastructure of today 1 2 The Small Computer Systems Interface legacy 4 The Small Computer Systems Interface SCSI is the conventional server centric method of connecting peripheral devices disks tapes and printers in the open client server environment As its name indicates it was designed for the PC and small computer environment It is a bus architecture with dedicated parallel cabling between the host and storage devices such as disk arrays This is similar in implementation to the Original Equipment Manufacturer s Information OEMI bus and tag interface commonly used by mainframe computers until the early 1990 s SCSI shares a practical aspect Planning and Implementing an IBM SAN with bus and tag in that cables and connectors are bulky relatively expensive and are prone to failure The amount of data available to the server is determined by the number of devices which can attach to the bus and by the number of buses attached to the server Up to 15 devices can be attached to a server on a single SCSI bus In practice because of performance limitations due to arbitration it is common for no more than four or five devices to be attached in this way
223. g and Implementing an IBM SAN U Link Incident Log 6 5 00 10 09 AM 6 3 00 10 55 AM 6 3 00 10 55 AM 6 3 00 10 54 AM 6 3 00 10 05 AM 6 3 00 9 41 AM 6 3 00 9 41 AM 6 3 00 9 41 AM 6 3 00 9 18 AM 6 3 00 9 18 AM 6 2 00 4 32 PM 6 2 00 4 11 PM 6 2 00 4 11 PM 6 2 00 3 48 PM 6 2 00 3 48 PM 6 2 00 3 29 PM Loss of Signal or Loss of Synchronization Loss of Signal or Loss of Synchronization Loss of Signal or Loss of Synchronization Loss of Signal or Loss of Synchronization Loss of Signal or Lass of Synchronization Not Operational primitive sequence NOS received Not Operational primitive sequence NOS received Not Operational primitive sequence NOS received Loss of Signal or Loss of Synchronization Loss of Signal or Loss of Synchronization Loss of Signal or Loss of Synchronization Loss of Signal or Loss of Synchronization Not Operational primitive sequence NOS received Loss of Signal or Loss of Synchronization Not Operational primitive sequence NOS received Loss of Signal or Loss of Synchronization ANANDAN NUNN ODARAN pot clear Retesh cise Figure 174 Link Incident Log To see the changes that have been made to the ports we select the Port List View as shown in Figure 175 Chapter 7 Implementing the McDATA ED 5000 213 Figure 175 Port list view 7 3 5 Configuring the FC operating parameters The Fibre Channel operating parameters do not normally have to be changed The mo
224. g different channels or ports different options will become available and other options will become greyed out and unavailable Be aware of what system channel or port is highlighted as you move through the toolbar As we can connect to several SAN Data Gateway systems from one client session select the particular Gateway you want and it will be highlighted in blue as shown in Figure 218 Planning and Implementing an IBM SAN File View Admin Tools Controls Help HH 9 113 24 110 ITSO SAN Data Gateway PA Storwatche SAN Data Gateway Specialist 2000 Pathlight Technology Inc Current SAN Data Gateway fcbridge1 World Wide Node Name 10000060 450D00B8F IP Address 9 113 24 246 Serial Number 100012 Firmware Revision 3 40 10 Health Check Level 2 Health Check Interval 60 Log Size 214 Receive Event Traps Enabled Serial Baud Rate 19200 RID Tag 100012 Figure 218 Selecting from multiple SAN Data Gateways On the left hand side of the highlighted Gateway there is a small key and by selecting this it expands the view to show you all SCSI ports and installed Fibre Channel ports Figure 219 shows a Gateway with four SCSI ports and two Fibre Channel ports Chapter 8 Implementing the SAN Data Gateway 261 PA Storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc File View Admin Tools Controls Help Current SAN Data Gateway ITSO lt 8c Channel 1 Wor
225. g the EFC Manager on remote workstations on page 196 7 2 1 Setting up the network environment First we need to think about how we want to integrate the McDATA in our network environment Do we want the EFC server accessible through our corporate intranet or do we only want to access it locally Second do we want to establish a dedicated network segment to connect the switch with the EFC server or do we want to use the existing LAN for this 7 2 1 1 Our network setup We want to be able to reach the EFC server through the network which is also reachable from remote using IP so we need to install the EFC server in the LAN Chapter 7 Implementing the McDATA ED 5000 187 188 Also we chose to install the McDATA ED 5000 in our laboratory network as well This configuration is shown in Figure 148 EFC Server Remote EFC Manager a LN buc l l I Ethernet Intranet McDATA ED5000 Figure 148 Our ED 5000 setup To set up the switch and the EFC server we used the following network information For the EFC server e IP address 9 113 25 62 The unique address of the EFC Server in the corporate intranet We use it with a Web browser to install the EFC manager on a remote workstation and to access the EFC Server from the remote workstation Subnetmask 255 255 254 0 Our intranet is divided into several subnets The Subnetmask is u
226. g the same axis The inner channel carries the signal and the outer channel serves as a ground Controller A component that attaches to the system topology through a channel semantic protocol that includes some form of request response identification CRC Cyclic Redundancy Check An error correcting code used in Fibre Channel DASD Direct Access Storage Device any on line storage device a disc drive or CD ROM 354 Planning and Implementing an IBM SAN DAT Digital Audio Tape A tape media technology designed for very high quality audio recording and data backup DAT cartridges look like audio cassettes and are often used in mechanical auto loaders typically a DAT cartridge provides 2GB of storage But new DAT systems have much larger capacities Data Sharing A SAN solution in which files on a storage device are shared between multiple hosts Datagram Refers to the Class 3 Fibre Channel Service that allows data to be sent rapidly to multiple devices attached to the fabric with no confirmation of delivery dB Decibel a ratio measurement distinguishing the percentage of signal attenuation between the input and output power Attenuation loss is expressed as dB km Disk Mirroring A fault tolerant technique that writes data simultaneously to two hard disks using the same hard disk controller Disk Pooling A SAN solution in which disk storage resources are pooled across multiple hosts rather than be dedicated to a spec
227. ge Allocation button that we used in Figure 38 on page 76 Storage Allocation Figure 69 Storage Allocation button Chapter 4 Configuring the ESS with native Fibre Channel 93 This takes us back to the Storage Allocation Graphical View panel There we select the host Fibre Channel port that we configured as shown in Figure 70 Storage Allocation Graphical View Clear View Click on a Host or Array to see paths Seuss m Netfinity 200000E08 FC ED eT F 5 an 3 z 2 an H Figure 70 Storage allocation with host FC port selected What this shows is the volumes defined for our host Fibre Channel port Also we see details of the selected host Fibre Channel port to the right of the window in the Information field Clicking on an ESS Fibre Channel port will draw lines to the volumes configured for the selected host Fibre Channel port and the Information field now displays information of the configuration of the ESS Fibre Channel port This is shown in Figure 71 This information is useful to see which ESS Fibre Channel port is configured for the appropriate topology We show how to configure this in 4 1 7 Configuring an ESS Fibre Channel port on page 96 94 Planning and Implementing an IBM SAN Storage Allocation Graphical View Click on a Host or Array to see paths _IN ddl 2N hudsen itso moe 200 moe5100 zj a ee Clear View View All Storage Tabular View aa
228. ge Area Networks Nefinity Servers http www storage ibm com hardsoft products fcswitch fcswitch htm IBM SAN Fibre Channel Switch http www storage ibm com hardsoft products fchub fcmhub htm IBM SAN Fibre Channel Managed Hub http www storage ibm com hardsoft products sangateway supserver htm IBM SAN Data Gateway http www storage ibm com hardsoft products tape ro3superserver htm IBM SAN Data Gateway Router http www storage ibm com hardsoft products fcss fcss htm IBM Fibre Channel RAID Storage Server http www storage ibm com hardsoft products ess ess htm Enterprise Storage Server http www brocade com Brocade Communications Systems Inc http www fibrechannel com Fibre Channel Industry Association http www mcdata com McDATA Corporation http www pathlight com Pathlight http www sanergy com Tivoli SANergy http www snia org Storage Networking Industry Association http www tivoli com Tivoli Systems http www t11 org Technical Committee T11 http www vicom com Vicom Systems http www vixel com Vixel Appendix B Related publications 349 350 Planning and Implementing an IBM SAN How to get IBM Redbooks This section explains how both customers and IBM employees can find out about IBM Redbooks redpieces and CD ROMs A form for ordering books and CD ROMs by fax or e mail is also provided e Redbooks Web Site ibm com redbooks Search
229. ge Server Configuration Planner SC26 7353 IBM Enterprise Storage Server Quick Configuration Guide SC26 7354 IBM SAN Fibre Channel Managed Hub 3534 Service Guide SY27 7616 IBM SAN Fibre Channel Managed Hub 3534 User s Guide GC26 7391 IBM Enterprise Storage Server Introduction and Planning Guide 2105 Models E10 E20 F10 and F20 GC26 7294 IBM Enterprise Storage Server User s Guide 2105 Models E10 E20 F10 and F20 SC26 7295 IBM Enterprise Storage Server Host Systems Attachment Guide 2105 Models E10 E20 F10 and F20 SC26 7296 IBM Enterprise Storage Server SCSI Command Reference 2105 Models E10 E20 F10 and F20 SC26 7297 IBM Enterprise Storage Server System 390 Command Reference 2105 Models E10 E20 F10 and F20 SC26 7298 IBM Storage Solutions Safety Notices GC26 7229 Translated External Devices Safety Information SA26 7003 Electrical Safety for IBM Customer Engineers S229 8124 The following publications can be ordered at http www vicom com e SLIC Router FC SL Installation and User Guide 310 605759 SLIC Manager Installation and User Guide 310 605807 348 Planning and Implementing an IBM SAN B 4 Referenced Web sites These Web sites are also relevant as further information sources e http www storage ibm com ibmsan index htm IBM Enterprise SAN http www storage ibm com hardsoft products fchub fchub htm IBM Fibre Channel Storage HUB http www pc ibm com ww netfinity san IBM Stora
230. ggregation of interconnected nodes workstations file servers and or peripherals with its own protocol that supports interaction Network Topology Physical arrangement of nodes and interconnecting communications links in networks based on application requirements and geographical distribution of users NFS Network File System A distributed file system in UNIX developed by Sun Microsystems which allows a set of computers to cooperatively access each other s files in a transparent manner NL_Port Node Loop Port a node port that supports Arbitrated Loop devices NMS Network Management System A system responsible for managing at least part of a network NMSs communicate with agents to help keep track of network statistics and resources Node An entity with one or more N_Ports or NL_Ports Non Blocking A term used to indicate that the capabilities of a switch are such that the total number of available transmission paths is equal to the number of ports Therefore all ports can have simultaneous access through the switch Non L_Port A Node or Fabric port that is not capable of performing the Arbitrated Loop functions and protocols N_Ports and F_Ports are not loop capable ports Operation A term defined in FC 2 that refers to one of the Fibre Channel building blocks composed of one or more possibly concurrent exchanges Optical Disk A storage device that is written and read by laser light Optical Fiber A medium an
231. gure 214 IBM Storage Area Network Data Gateway startup If you type in help and then press Enter a list of available commands is provided The commands are case sensitive and must be entered as they appear Issue the initializeBox command to remove any configurations files that may be present The unit will restart automatically Chapter 8 Implementing the SAN Data Gateway 253 Note The initializeBox command will erase all configuration files It should be used only during initial SAN Data Gateway installation 8 2 1 Setting the Ethernet address 254 Once restarted the Ethernet port must be configured and attached using network information provided by the network administrator To set the IP address use the ethAddrSet command The address must contain the double quotes Gateway gt ethAddrSet 9 111 24 66 Network not Enabled Write complete Host Address set to 9 111 24 66 for Ethernet interface value 0 0x0 Gateway gt _ If a subnet is required then add it after the IP address and separate the two addresses using a comma For example Gateway gt ethAddrSet 9 111 24 66 255 255 255 0 If a gateway or standard router is to be specified then issue the gateaddrGet command to view if there is one set and the gateAddrSet command to set or change it For example Gateway gt gateAddrGet No current gateway address set value 0 0x0 Gateway gt gateAddrSet 193 1 1 11 Write complete Th
232. gure 267 Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 315 316 Composite Drive Setup Wizard Assigning Composite Drive Properties Figure 267 Assigning Composite Drive Properties window The properties that can be changed are the FC LUN and the Drive Name There will be a suggested LUN number in this field that can be accepted If not simply type in the desired LUN number The name can also be defined to the Composite drive for easier identification with a limit of up to eight characters The Initialization Method box refers to whether or not to allow the operating system to write its signature on the Composite drive Select the Next gt button and a dialog box will appear as shown in Figure 268 to allow you to create another Composite drive Select Yes If you would like to create another Composite drive and the Composite Drive List window opens and the steps described above can be repeated Planning and Implementing an IBM SAN Composite Drive Setup Wizard Figure 268 Completing the Composite Drive setup Select Finish when you have created all the desired Composite drives Up to this point the configuration has been kept within the SLIC Manager software When the Finish button is selected the SLIC Manager will now communicate to the Router to complete the process and update the Router to control the drives The Host system must rescan for devices or restart to be able to see the
233. he SLIC Router Installation and Users Guide 310 605759 10 Host attach and power up sequence For a host to now recognize and use the disks set the dip switches in SW2 back to mode 3 this is normal host operation mode The Fibre Channel cable from the host can be connected to the Router If the SSA drives are 302 Planning and Implementing an IBM SAN not powered on do this now and this should be done before the Router Next the Router is powered on wait for the Status LED to stop flashing and remain on solid At this point the host can be powered on A check can be done to see that the SLIC Router is being recognized by the host adapter card On a Windows NT with a QLogic Fibre Channel adapter during boot up look for a prompt to enter the QLogic bios by entering in aur Q At the bios window select Scan Fibre Devices A list of the Fibre Channel target IDs are presented scroll down to the ID that you set in SW1 You will see the WWN of the SLIC Router Exit the bios and the system will reboot Once the system has started you use a method to ensure that the host has access to all the drives This is different depending on the operating system of the computer For Windows NT select Start gt Programs gt Administrative Tools gt Disk Administrator This tool will report that new disks have been found and will be added to the system 9 2 SLIC Manager software Rather than using the dip switches to configure the features an
234. he system already had 44 disks available to it but the 1 GB volume we assigned to it from the ESS was not recognized at this stage Chapter 5 Implementing an IBM managed and unmanaged hub 117 a Disk Administrator Figure 104 Disk Administrator view before reboot We rebooted the Windows NT system to let the Windows operating system rescan for attached devices Once this was done an extra disk Disk 45 now became available and ready for use In Figure 105 we show that Disk 45 with a 953 MB capacity is now ready to be assigned and formatted for use as any regular disk 118 Planning and Implementing an IBM SAN a Disk Administrator Figure 105 Disk Administrator view after reboot A second Windows NT host was also attached to the hub so that there were two analogous host operating systems in a single loop We assigned the same volume in the ESS that was assigned to the first host to this second host and then rebooted the second host system It recognized the new volume and was able to access it To ensure data integrity clustering or shared access software must be installed in both hosts Chapter 5 Implementing an IBM managed and unmanaged hub 119 m Note Two hosts or initiators attached to a hub in a single loop to an ESS is not supported by IBM Although the two host example is fully working because of the loop initialization process we do not recommend it What we recommend is to have a seco
235. ibutes Figure 76 Storage Server Attributes field There are two Fibre Channel Access Modes to which the ESS can be configured e Access_Any In Access_Any mode any host system that is not defined in the ESS Specialist can access all non AS 400 open system logical volumes In Access Any mode the ESS Specialist Web interface displays an anonymous pseudo host which is shown in Figure 77 and which is taken from another ESS because the one we used here is configured for Access_Restricted An anonymous host is not a real host system in disguise and connected to the storage server It represents all Fibre Channel attached host systems if any that are connected to the ESS and that are not configured on the ESS The anonymous hosts do not have an access profile defined You can access specific logical volumes that are defined in the ESS by hosts that are not identified by the ESS Access_ Restricted In Access_Restricted mode a connected host which is not configured with the WWPN of its host FC adapter cannot access any volumes When this host gets configured by using the WWPN of its host FC adapter it will only see the volumes for which it the host FC adapter is configured Once a host is configured there are no differences 4 1 7 3 Access profiles Whichever access mode is chosen any Fibre Channel attached host system that has an access profile can access only those volumes that are defined in the profile Depending on the
236. icast Multi Mode Fiber MMF In optical fiber technology an optical fiber that is designed to carry multiple light rays or modes concurrently each at a slightly different reflection angle within the optical core Multi Mode fiber transmission is used for relatively short distances because the modes tend to disperse over longer distances See also Single Mode Fiber 358 Planning and Implementing an IBM SAN Multiplex The ability to intersperse data from multiple sources and destinations onto a single transmission medium Refers to delivering a single transmission to multiple destination Node Ports N_Ports N_Port Node Port A Fibre Channel defined hardware entity at the end of a link which provides the mechanisms necessary to transport information units to or from another node N_Port Login N_Port Login PLOGI allows two N_Ports to establish a session and exchange identities and service parameters It is performed following completion of the fabric login process and prior to the FC 4 level operations with the destination port N_Port Login may be either explicit or implicit Name Server Provides translation from a given node name to one or more associated N_Port identifiers NAS Network Attached Storage a term used to describe a technology where an integrated storage system is attached to a messaging network that uses common communications protocols such as TCP IP NDMP Network Data Management Protocol Network An a
237. ice between two ports or a multicast service between a multicast originator and one or more multicast recipients Class 4 A class of service providing a fractional bandwidth virtual circuit between two ports with confirmed delivery or notification of non deliverability Class 6 A class of service providing a multicast connection between a multicast originator and one or more multicast recipients with confirmed delivery or notification of non deliverability Client A software program used to contact and obtain data from a server software program on another computer often across a great distance Each client program is designed to work specifically with one or more kinds of server programs and each server requires a specific kind of client program Client Server The relationship between machines in a communications network The client is the requesting machine the server the supplying machine Also used to describe the information management relationship between software components in a processing system Cluster A type of parallel or distributed system that consists of a collection of interconnected whole computers and is used as a single unified computing resource Coaxial Cable A transmission media cable used for high speed transmission It is called coaxial because it includes one physical channel that carries the signal surrounded after a layer of insulation by another concentric physical channel both of which run alon
238. ide availability of connection In the event of a path failure for instance due to HBA port card fiber optic cable or storage adapter software running in the host servers initiates failover to a secondary path If the path failover malfunctions the application will fail Then the only choice is to repair the failed path or replace the failed device Both these actions potentially lead to outages of other applications on multiple heterogeneous servers if the device affected is the switch Switches like the IBM 2109 have redundant hot pluggable components including fans power supplies ASICs and GBICs which can be replaced during normal operation These hardware failures cause little or no noticeable loss of service However in the case of some failed components such as the mother board the switch itself will be treated as the field replaceable unit FRU Then all the ports and data paths are taken down Automatic path failover will occur to another switch so the network continues to operate but in degraded mode Here there is a distinction between a switch and a director Using the analogy of disk arrays an individual switch can be likened to a JBOD in that it is just a bunch of ports That is to say although it has redundant components in the event of certain component failures the total switch can fail or must be replaced as the FRU Cascading of multiple switches can achieve a higher level of fault tolerance A single director
239. ific host DLT Digital Linear Tape A magnetic tape technology originally developed by Digital Equipment Corporation DEC and now sold by Quantum DLT cartridges provide storage capacities from 10 to 35GB E_Port Expansion Port a port on a switch used to link multiple switches together into a Fibre Channel switch fabric ECL Emitter Coupled Logic The type of transmitter used to drive copper media such as Twinax Shielded Twisted Pair or Coax Enterprise Network A geographically dispersed network under the auspices of one organization Entity In general a real or existing thing from the Latin ens or being which makes the distinction between a thing s existence and it qualities In programming engineering and probably many other contexts the word is used to identify units whether concrete things or abstract ideas that have no ready name or label ESCON Enterprise System Connection Exchange A group of sequences which share a unique identifier All sequences within a given exchange use the same protocol Frames from multiple sequences can be multiplexed to prevent a single exchange from consuming all the bandwidth See also Sequence F_Node Fabric Node a fabric attached node F_Port Fabric Port a port used to attach a Node Port N_Port to a switch fabric Fabric Fibre Channel employs a fabric to connect devices A fabric can be as simple as a single cable connecting two devices The term is most often used t
240. igure 273 Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 321 9 5 2 Mirror Mirror Drive Setup Wizard Assigning Mirror Drive Properties Setup Summary Type 2 way Simple Mirror FC ID E Logical Drive ID 2 FC LUN 2 Target Number 733282 Drive Name fiTSOm Capacity 8689 MB m Initialization Method Primary Member None C DoNNa iniae Quick Initialize lt Back Cancel Figure 273 Mirror drive properties Select the Next gt button and a dialog box appears to allow you to create another mirror drive Select Yes if you would like to create another mirror drive and the Mirror Drive List window opens and the steps described above can be repeated Select Finish when you have created all the desired mirror drives Up to this point the configuration has been kept within the SLIC Manager software When the Finish button is selected the SLIC Manager will now communicate to the Router to complete the process and update the Router to control the drives If Quick Initialize in the Initialization Method box was selected the Router will take a short period of time to write the host signature and build the mirror During this time if you try to communicate to the Router you may experience a slower than normal response drive properties If you go to the Control Center window by selecting Tools gt Control Center you will see that the mirror drive is now displayed in
241. ime we have configured all that is necessary to enable the ESS to present volumes to a Fibre Channel host For further changes of the assignment of volumes to ports we use the Modify Volume Assignments panel For example we can share volumes between host Fibre Channel ports These host Fibre Channel ports can be in the same host which uses for example the Subsystem Device Driver SDD to increase the bandwidth and availability or in different hosts for example for clustering We clicked on the Modify Volume Assignments button as shown in Figure 85 104 Planning and Implementing an IBM SAN Modify Host Systems Configure Host Adapter Ports Configure Disk Groups Sy Whos omens Figure 85 Entry point for modifying volume assignments This takes us to the Modify Volume Assignments panel as shown in Figure 86 Modify Volume Assignments Volume Assignments Print Table no sent no sort no sort no sort no sort no sort no sort 7 no sort gt 000 FCA13 Device Adapter Pair 1 Open System 000 5 GBE RAD amy Bayl Caral fod Cluster 1 Loop A SCSI Port A Array 2 Vol 000 TD 06 LUN 0000 001 FCA13 Device Adapter Pair 1 Open System 000 5 GBE RAD ary Bay l Caral fod Chaster 1 Loop A SCSI Port A Array 2 Vol 001 TD 06 LUN 0001 Device Adapter Pair 1 Open System 000 5 GBE RAD Amay Bay l Caral fod Chacter 1 Loop A SCSI Port A Array 2 Vol 002 D 06 LUN 0002 003 FCA13 Device Adapter Pair 1 Open
242. in the FC Port Attributes field Once highlighted we press the Perform Configuration Update button Perform Configuration Update Chapter 4 Configuring the ESS with native Fibre Channel 101 This will perform the update As a result the Fibre Channel port is returned to an undefined state as shown in Figure 80 Storage Server Atiribuies Access Restricted Fibre Channel Access Mode FC Port Attributes Fibre Channel Topology Undefined Fibre Channel Point to Point Fibre Channel Arbitrated Loop Figure 80 FC Port Attributes undefined Now we can choose between Fibre Channel Point to Point and Fibre Channel Arbitrated Loop Again we need to press the Perform Configuration Update button As we are in the process of performing the update we get the progress window shown in Figure 81 N Performing Configuration Update x Modifying FibreChannel Port Topology Data E ee Estimated Remaining Time 2 minutes 37 seconds Ey Unsigned Java Applet Window Figure 81 Progress window of changing the topology Figure 82 shows a successful topology change of the port N x 2104 The specified port has been configured successfully Ki ey Unsigned Java Applet Window Figure 82 Successful topology change 102 Planning and Implementing an IBM SAN Pressing the OK button takes us back to the Open System Storage panel as shown in Figure 83 Open added Storage T ET RS 600
243. indow would still show the capacity of 8 696 MB Approximately half of the 18 GB drive will be unused After all drives have been added select Next gt and you will be able to adda dedicated spare drive to the mirror if desired Highlight one of the remaining available drives click on Add gt gt and its name will appear in the Mirror Drive Dedicated Spare window as shown in Figure 272 Mirror Drive Setup Wizard Mirror Drive Members Selection m Miror Drive Dedicated Spares TO0005 35E 6840E Available Drive Pool TOOOOOLO00 35E68229 TOOOO3LO03 35E6848D TOO0004L004 35E22D6C TOOOO5LOO5 35E6840E TOOO06 LOOG 35E68462 Identify T00007 LOO7 35E684AE lt lt Remove Reset Drive Capacity MB m Mirror Capacity MB 8689 8689 lt Back Next gt Cancel Figure 272 Adding a dedicated spare Click on the Next gt button and the properties of the mirror drive can be changed The properties that can be changed are the FC LUN and the Drive Name There will be a suggested LUN number in this field that can be accepted If not simply type in the desired LUN number A name can also be defined to the mirror drive for easier identification with a limit of up to eight characters The Initialization Method box refers to whether or not to allow the operating system to write its signature on the Mirror drive The Assigning Mirror Drive Properties window is shown in F
244. ing IBM 2109 switches As your SAN begins to grow you may want to add more switches At some stage you may want to connect them together This is known as cascading In the following sections we cover these topics e 6 5 1 Cascading overview on page 172 e 6 5 2 Cascading examples on page 172 showing cascaded fabric e 6 5 3 Cascading using IBM switches on page 174 showing 2109 capabilities Chapter 6 Implementing an IBM SAN Fibre Channel Switch 171 e 6 5 4 Implementing cascading with the IBM 2109 S08 switch on page 175 6 5 1 Cascading overview We have already stated that switches are among the core elements of a SAN fabric When you start your SAN business you might begin working with simple SAN fabric with perhaps only one switch and a few ports But as you start implementing more and more host and storage devices in your SAN you will reach limits where you have to add either more ports or expand your fabric by adding more switches Interconnecting your first switch with newly added or other switches allows you to build much larger fabrics Expanding the fabric in this manner is called cascading Cascading is a reliable scalable and cost effective way to build fabrics with hundreds or thousands of ports Cascading of switches provides the following benefits to a SAN environment e The fabric can be seamlessly extended Additional switches can be added to the fabric without powering down the
245. ing Parameters window shown in Figure 203 7 5 2 3 Other prerequisites for a multiswitch fabric To be able to successfully establish a multiswitch fabric some important prerequisites apply One is the zoning configuration which must be compatible What this means is that the active zone set name must be the same and the zones with the same name must have the same members Also the operating parameters resource allocation time out value R_A_TOV and error detection time out value E_D_TOV must be the same 7 5 2 4 Configuring the ports for the ISLs The ports for the ISLs can be configured like the other ports as we described in 7 3 4 Configuring the FC ports on page 211 From here we can check the checkbox for extended distance buffering Figure 204 Chapter 7 Implementing the McDATA ED 5000 243 pU Configure Ports csssssssssss R az Figure 204 Configure ISL ports 7 5 2 5 Connecting the directors Now we can connect the two switches with ISLs We are using two longwave ISLs between the two switches As a result we now have two switches in the EFC Manager s Product View as shown in Figure 205 244 Planning and Implementing an IBM SAN McDATA Enterprise Fabric Connectivity Manager Figure 205 Two managed switches from within one EFC Manager To see what the topology looks like now we navigate to the Topology View of the Fabric Manager Select View gt Fabric in the EFC Manager window and c
246. ing the QLogic 2200F adapter card 54 3 1 3 Loading the current Fibre Channel adapter driver 56 3 1 4 Installing the drivers 0 0 0 eee 57 3 1 5 Locating the World Wide Port Name WWPN on Windows NT 57 3 2 Installing Fibre Channel host adapter cards on a RS 6000 system 58 3 2 1 The 2105 host attachment package 0 58 3 2 2 Before you install the 2105 host attachment package 58 3 2 3 Replacing an older version of the 2105 installation package 59 3 2 4 Installing the 2105 host attachment package 59 3 2 5 Locating the World Wide Port Name WWPN on RS 6000 60 3 2 6 Verifying the IBM ESS configuration 0 5 60 Chapter 4 Configuring the ESS with native Fibre Channel 63 4 1 Detining the ESS e c 86 cb waded eed Gay heron de ends Gea 63 4 1 1 Introduction to the ESS Specialist 0 64 4 1 2 Viewing the Storage Allocation of the ESS 75 4 1 3 Accessing the Open System Storage panel 79 4 1 4 Defining a new host FC port with its WWPN 80 4 1 5 Configuring disk groupS 0 02 eee ee 84 4 1 6 Defining volumes for Fibre Channel host adapter ports 88 4 1 7 Configuring an ESS Fibre Channel port 96 4 1 8 Modifying volume assignments and sharing volumes 104 4 2 Related information 00000 ee 109 Chapter 5 Implem
247. interconnected using specialized cluster interconnects or open clustering technologies such as Fibre Channel Virtual Interface mapping Complex software is required to manage the failover of any component of the hardware the network or the application SCSI cabling tends to limit clusters to no more than two servers A Fibre Channel SAN allows clusters to scale to 4 8 16 and even to 100 or more servers as required to provide very large shared data configurations including redundant pathing RAID protection and so on Storage can be shared and can be easily switched from one server to another Just as storage capacity can be scaled non disruptively in a SAN so can the number of servers in a cluster be increased or decreased dynamically without impacting the storage environment 1 5 7 Improved disaster tolerance 22 Advanced disk arrays such as IBM s Enterprise Storage Server ESS provide sophisticated functions like Peer to Peer Remote Copy services to Planning and Implementing an IBM SAN address the need for secure and rapid recovery of data in the event of a disaster Failures may be due to natural occurrences such as fire flood or earthquake or to human error A SAN implementation allows multiple open servers to benefit from this type of disaster protection and the servers may even be located some distance up to 10 km from the disk array which holds the primary copy of the data The secondary site holding the m
248. ion does not display the new firmware version until the SAN Data Gateway has issued an SCSI inquiry The SCSI inquiry occurs when the Gateway rescans the SCSI buses The SCSI inquiry also occurs when the StorWatch SAN Data Gateway Specialist client application or the service terminal rescans the SCSI buses Currently up to eight different hosts can be attached to each Fibre Channel port If all six ports are installed then 48 different hosts can attach to the SAN Data Gateway 8 3 StorWatch SAN Data Gateway Specialist 256 The StorWatch SAN Data Gateway Specialistsoftware provides remote capability for all management configuration and event notification It is comprised of three parts e Agent e Server e Client Agent The agent is embedded in the operating system of each SAN Data Gateway to provide a stand alone manageable host The StorWatch SAN Data Gateway Specialistsoftware uses SNMP to set and retrieve information that controls the operation of the Agent The Specialist also uses SCSI over TCP to allow updates to the Gateway and target device Planning and Implementing an IBM SAN Server The server is a Java application that runs on a host and is used to maintain communication with the agents and acts as an intermediary between the agent and the client The server coordinates the request from multiple clients to manage multiple gateways or agents Multiple clients can share data the server already knows about and the ser
249. irror image of the data may be located up to a further 100 km from the primary site IBM has also announced Peer to Peer Copy capability for its Virtual Tape Server VTS This will allow VTS users to maintain local and remote copies of virtual tape volumes improving data availability by eliminating all single points of failure 1 5 8 Allow selection of best of breed storage Internal storage purchased as a feature of the associated server is often relatively costly A SAN implementation enables storage purchase decisions to be made independently of the server Buyers are free to choose the best of breed solution to meet their performance function and cost needs Large capacity external disk arrays may provide an extensive selection of advanced functions For instance the ESS includes cross platform functions such as high performance RAID 5 Peer to Peer Remote Copy Flash Copy and functions specific to S 390 such as Parallel Access Volumes PAV Multiple Allegiance and I O Priority Queuing This makes it an ideal SAN attached solution to consolidate enterprise data Client server backup solutions often include attachment of low capacity tape drives or small automated tape subsystems to individual PCs and departmental servers This introduces a significant administrative overhead as users or departmental storage administrators often have to control the backup and recovery processes manually A SAN allows the alternative strat
250. is chapter we show how to install and manage a McDATA ED 5000 This device is ordered through IBM as the McDATA Enterprise Fibre Channel Director 2032 001 The topics covered in this chapter are e 7 1 Our ITSO environment and the installation steps on page 181 e 7 2 Setting up an environment for using and managing a McDATA SAN on page 187 e 7 3 Managing the ED 5000 fabric on page 203 e 7 4 Zoning with McDATA on page 216 e 7 5 Cascading with McDATA building a multiswitch fabric on page 237 7 1 Our ITSO environment and the installation steps We configured the ESS as described in Chapter 4 Configuring the ESS with native Fibre Channel on page 63 to provide access through the McDATA ED 5000 to the host Our configuration is shown in Figure 147 This setup will be the starting point for our McDATA SAN In the following sections we cover these topics e 7 1 1 ED 5000 overview on page 182 e 7 1 2 Hardware overview on page 183 e 7 1 3 Software overview on page 183 e 7 1 4 ED 5000 installation steps on page 183 Copyright IBM Corp 2000 181 Netfinity NT Server Shortwave fiber McDATA ED5000 Shortwave fiber ESS native Figure 147 Simple McDATA switch setup 7 1 1 ED 5000 overview At the heart of our McDATA SAN is the ED 5000 The ED 5000 is a Fib
251. ivalent ports using fiber optic cables 3 Check the switch parameter settings You can find a list of all the possible switch parameters in Chapter 6 Implementing an IBM SAN Fibre Channel Switch 175 e IBM 2109 Model S08 Installation and Service Guide SC26 7350 e IBM 2109 Model S08 User s Guide SC26 7349 To change any or all of these settings you must Telnet to the 2109 Switch and enter the configure command 4 Power on the new switch 5 The IBM 2109 switch software detects the fabric expansion and automatically configures the related ports as E Ports 6 As an administrator you can see both switches in the fabric 7 The port LEDs on the switch are solid green 8 The extended fabric is working 6 5 4 2 Adding a switch that has been previously configured The new switch process works fine under the assumption that all parameters relating to the switches in the extended fabric are set properly and match each other However this will not be the case if the switch to be added has been used before and has specific parameter settings As aresult the interconnecting port LED will show a slow flashing green LED on your Switch View panel like that shown in Figure 143 Figure 143 Switch view panel The slow LED means the ports are online but are segmented and cannot establish a fabric The error message says Please check for loopback cable or incompatible switch From the administrators Fabric View panel you will s
252. ividually or reconfigured to make new logical drives 9 8 1 Remove a logical drive To remove a logical drive you access the Control Center by selecting Tools gt Control Center from the top toolbar At the Control Center window select the logical drive composite mirror or copy that you want to remove Select the Remove Logical Drive button on the right hand side and a dialog box appears that will ask you to confirm that you want to remove the logical drive Once it is removed the member drives will become general spares and will show up in the General Spare Drive window of the Control Center This is shown in Figure 286 334 Planning and Implementing an IBM SAN Control Center T33282 ITSOm T33025 ITSOl TOOOOO 35E68229 TOOOO 3SE684AE TOOOOS 35E68462 Figure 286 Removing a logical drive The logical drive that was created as an Instant Copy Drive in 9 6 1 Creating an Instant Copy drive on page 324 has been removed and is now a general spare 9 8 2 Mapping a general spare You will notice in Figure 286 that the general spare does not have a LUN number assigned to it To get a new LUN number for this drive you select the drive and click on the Properties button The Drive Properties window appears select the Change FC button A dialog box opens as shown in Figure 287 Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 335 Modify Fibre Channel Mapping Ea Original Mapping
253. l Done Figure 137 Choosing zone members The NTzone contains the devices which are connected to ports 0 and 1 of the switch Additionally to show how these can be used in combination with each other we also selected the WWN 40 05 07 63 00 c0 0c 5d as a member in our NTzone Now the NTzone also contains the device with the WWN node name or port name 40 05 07 63 00 c0 0c 5d to whichever port in the fabric it is connected The WWN 40 05 07 63 00 c0 0c 5d was actually the node WWN of a tape storage device which was not online at the time of its addition We have successfully defined a zone called NTzone Chapter 6 Implementing an IBM SAN Fibre Channel Switch 167 168 6 4 2 2 Adding additional zones To illustrate the zoning concept we will establish a second zone and name it AlXzone Again we will use hardware zoning and we will use ports 2 and 3 As a further illustration we will also add for the Sun host a zone named SUNzone and we will use ports 4 and 5 Ports 6 and 7 have been left free because we want them for future use as E Ports when we add cascade a second switch This is covered in 6 5 1 Cascading overview on page 172 Using the same process that we followed to add our initial NTzone now we have created e The Netfinity zone NTzone ports 0 1 and WWN e The RS 6000 zone AlXzone ports 2 3 e The SUN zone SUN zone ports 4 5 Our zone names are shown in the Zone Name field and in
254. l environments using an industry standard Fibre Channel Arbitrated Loop FC AL interface The SAN Data Gateway enables SCSI devices to benefit from distance extension to 500 meters increased bandwidth of Fibre Channel and increased addressability A wide range of IBM and non IBM SCSI based servers are supported including UNIX and Windows based plus many IBM and non IBM SCSI storage devices including IBM Magstar tape and the Enterprise Storage Server Because of its comprehensive zoning access control capabilities including persistent binding of hosts to LUNs LUN masking the SAN Data Gateway is an ideal solution to support attachment of a SCSI based ESS to multiple hosts in a Fibre Channel SAN for storage consolidation 2 3 1 2 When will you use a hub You will use a hub to implement a Fibre Channel Arbitrated Loop Hubs can also be used as distance extenders in connection with the IBM SAN Data Gateway Usually they are used for entry level homogeneous server implementations Some of the possible uses of these hubs are clustering LAN free backup storage consolidation and remote disk mirroring The IBM products available are the IBM Fibre Channel Storage Hub 2103 H07 and IBM Fibre Channel Managed Hub 3534 1RU The IBM 3534 1RU offers superior function due to its manageability with superior fault isolation planning and controlling It also has a non blocking architecture This means that any two pairs of ports can be active a
255. l of 256 available The first LUN LUN 0 is used for the Gateway for control and command purposes That leaves 255 allowable LUN addresses to be used for attaching SCSI devices to the four SCSI ports Chapter 8 Implementing the SAN Data Gateway 269 Note Do not attach more than 255 LUNs to the SAN Data Gateway Exceeding the 255 limit will cause unpredictable results The SCSI devices must be previously set up in your host prior to attaching to the SAN Data Gateway For example if an IBM Enterprise Storage Server is to be used as the target device then all the volume or LUN assignments must be completed Also the volumes should be assigned to the appropriate SCSI port in the ESS Attach the SCSI cable from the SCSI device to a SCSI port on the Gateway 8 4 2 Device discovery Once attached the SAN Data Gateway can be restarted or powered on to perform a discovery of the attached devices If this was done after the initalizeBox command from the service terminal then the initial discovery will be carried out in a specific order The buses are scanned in order from 1 to 4 and each bus is scanned from SCSI target 0 to 15 and LUN 0 to 32 for each ID As a device is discovered it is assigned a specific LUN number by the SAN Data Gateway This numbering will begin at LUN number 1 zero is reserved for the SAN Data Gateway control and continues sequentially as devices are discovered The SAN Data Gateway must add this extra laye
256. lanners need to link many users of multi vendor heterogeneous systems to multi vendor shared storage resources and they need to allow those users to access common data wherever it is located in the enterprise These requirements imply a network solution and two types of network storage solutions are now available e Network attached storage NAS e Storage Area Network SAN 8 Planning and Implementing an IBM SAN 1 3 1 What network attached storage is NAS solutions utilize the LAN in front of the server and transmit data over the LAN using messaging protocols such as TCP IP and Net BIOS We illustrate this in Figure 7 Network Attached Storage Utilizing the network in front of the servers Intelligent Disk Array Clients JBOD Clients Jill 5 m ma E E a i e EE Gan M Local Wide Area Network Messaging Protocol TCP IP Net TT Database Application Server Server Server r Network Backup Server Figure 7 Network attached storage utilizing the network in front of the servers By making storage devices LAN addressable the storage is freed from its direct attachment to a specific server In principle any user running any operating system can address the storage device by means of a common access protocol for example Network File System NFS In addition a task such as back up to tape can be performed across the LAN enabling sh
257. lanning and Implementing an IBM SAN Port Information for IBM_2109_FC_Switch o 1 2 3 4 s5 amp 7 Port World Wide Name 20 06 00 60 69 20 1d 74 Port Status Online Port Module or GBIC Module id PortStats 4 Byte Word Transmitted 2180915 4 Byte Word Received 2384557 Frames Transmitted 5763 Frames Receiwed 6896 C2 Frames Received 10 C3 Frames Received 6886 Link Control Frames Received Mcast Frames Received Meast Timeouts Mcast Frames Transmitted Time R_RDY Priority Time BB_Credit Zero Encd Errs Inside Frames 5 0 0 0 0 0 0 0 Frames with CRC Errs Figure 129 F Port storage connection In Figure 130 we show the operation mode of port 1 which is the host connection as L Port This indicates arbitrated loop and is not what we expect to see Chapter 6 Implementing an IBM SAN Fibre Channel Switch Port Type F Port Short Frames Long Frames Bad End of Frames Ened Errs Outside Frames C3 Frames Discarded LIP Ins LIP Outs Last LIP Received Frames Rejected Frames Busied Link Failure Loss of Syne Loss of Signal 153 E Port Information for IBM_2109_FC_Switch o 1 2 3 4 5 6 7 Port World Wide Name 20 01 00 60 69 20 1d 74 Port Status Online Port Module or GBIC Module id Port Type L Port PortStats 4 Byte Word Transmitted 36198353 Short Frames 4 Byte Word Received 2325826 Long Frames Frames Transmitted 2824608 Bad End of Frames 0 Fram
258. ld as shown in Figure 196 The last entry to the left of the window is another ESS but which is without a nickname Chapter 7 Implementing the McDATA ED 5000 233 New Zone AIX_Zone_1 Emulex 10 00 00 00 C9 20 DC 71 Domain 1 Emulex 10 00 00 00 C9 20 E7 C7 Port 2 Emulex 10 00 00 00 C9 20 DC 71 KATE FCS0 Port 3 Emulex 10 00 00 00 C9 20 E7 C7 Emulex 10 00 00 00 09 20 E7 F3 be Port5 KATE_FCSO Port6 RPKAS3_C1B2A1 Port RPKAS3_C1B3A1 Figure 196 Defining an AIX zone After selecting Save Zone we assign this zone to our zone set Just as we did previously with the NT zone we drag and drop the AIX zone in the Modify Zone Set window from the Zone Library to the Zones in Set on the left side in the window as shown in Figure 197 234 Planning and Implementing an IBM SAN Modify Zone Set ITSO_Zone_Set_1 NT_Zone_1 Figure 197 Assigning the AIX zone to the zone set After saving the zone set we have two zones the NT_Zone_1 and the AIX_Zone_1 Both of these are in our ITSO_Zone_Set_1 zone set as shown in Figure 198 To apply the changes in the zone set we must activate it by selecting Actions gt Activate as shown in Figure 198 Chapter 7 Implementing the McDATA ED 5000 235 Zone Sets C ITSO_Zone_Set_2 AITSO_Zone_Set_t ANT_Zone_1 RPKAS3_C1B2A1 QLogic 20 00 00 E0 88 00 52 5E Emulex 10 00 00 00 C9 20 CA E6 AAK_Zone_1 Emulex 10 00 00 00 C9 20 0C 71 Emulex 10 00 00 00 09 20
259. ld Wide Node Name 10000060 45160E08 gt SCSI Channel 2 oS SCSI Channel 3 IP Address 9 113 24 110 amp SCSI Channel 4 Serial Number 21081300724 Firmware Revision 3 40 10 r Fibre Channel 1 T Fibre Channel 4 Health Check Level 2 n Health Check Interval 15 Log Size 2047 Receive Event Traps Enabled Serial Baud Rate 19200 Figure 219 Expanded Gateway view 262 SCSI channels 1 2 and 3 and Fibre Channel ports 1 and 4 also have a key on the left hand side to depict that there are devices attached By selecting a key you will now expand the tree as seen in Figure 220 and view the different disk devices attached Planning and Implementing an IBM SAN File View Admin Tools Controls Help lt SCSI Channel 1 P Disk Device 0 0 M Disk Device 0 1 B Disk Device 1 0 M Disk Device 1 1 9 Disk Device 2 0 M Disk Device 2 1 J Disk Device 3 0 M Disk Device 3 1 9 Disk Device 4 0 M Disk Device 4 1 B Disk Device 5 0 B Disk Device 5 1 9 Disk Device 6 0 M Disk Device 6 1 9 Disk Device 8 0 M Disk Device 8 1 9 Disk Device 9 0 M Disk Device 9 1 9 Disk Device 10 0 B Disk Device 10 1 M Disk Device 11 0 B Disk Device 11 1 M Disk Device 12 0 Up Time 4 days 23 29 49 PA Storwatch SAN Data Gateway Specialist 2000 Pathlight Technology Inc Current SAN Data Gateway ITSO World Wide Node Name 10000060 45160E08 IP Address 9 113 24 11
260. le between two nodes FSPF will automatically share traffic between these paths This feature provides high bandwidth as well as fault tolerance because no paths are held idle as stand by redundant links This is quite different to LAN path redundancy which maintains idle paths for redundancy FSPF can guarantee in sequence delivery of frames even if the routing topology has changed during a failure by enforcing a hold down period before a new path is activated This allows all frames in transit to a specific destination to be delivered or discarded An example of shared traffic routing is shown in Figure 18 Four switches have two ISLs between any two of them Traffic between Switches 1 and 3 for example will be shared on two paths but traffic between Switches 1 and 4 can be shared on four paths Planning and Implementing an IBM SAN Shared Load on Parallel Paths Switch Traffic Switch 1 2 Switch 2 a Po Pe A Es esoseososceeseoseesoseecosses seesceses f PEPI PAAA A ARA EEE EAA i Traffic Switch lt 2 3 on 2 To Fe Traffic Switch eo i Sa ee Switch 3 Switch 4 Figure 18 Load sharing on parallel paths A maximum of seven hops is recommended between any two switches in the fabric to avoid time outs We say recommended because the actual hops are not monitored and restricted to seven More hops are possible and test beds up to 20 switches have been installed
261. lick on the icon that represents the fabric in the main window as shown in Figure 206 The number 2 in the pentagon fabric icon indicates that we have two directors installed in the fabric McDATA Enterprise Fabric Connectivity Manager Figure 206 Fabric View Clicking on this icon will open the Fabric Manager in a dedicated window with the Topology View as shown in Figure 207 Chapter 7 Implementing the McDATA ED 5000 245 lt 2 McDATA Fabric Manager rit Topology View Fabric WN Fabric Status Principal Switch View ED 5000 1 Operational Domain ID 1 Configure Help iil N 2 interswitch links o 9 113 25 61 9 113 25 63 Figure 207 Topology View with two switches connected and configured This indicates that the connection of the two directors consists of two ISLs Clicking on either icon will open the Product Manager for the associated director We can also change the description of the icon by right clicking on the ISL as shown in Figure 208 Currently we see the network address displayed 246 Planning and Implementing an IBM SAN Figure 208 Changing icon text If the other connected ED 5000 is accessed by means of its own EFC server the status cannot be determined by the first EFC Server Therefore the Topology View looks like Figure 209 Chapter 7 Implementing the McDATA ED 5000 247 gt McDATA Fabric Manager Figure 209 Interconnected ED 5000 one managed b
262. listsoftware file is a self extracting file Once it has been downloaded execute or run the file and it will automatically load onto your computer Chapter 8 Implementing the SAN Data Gateway 257 8 3 1 1 Starting the Specialist To start the Specialist the server must be started first and then the client can be launched Figure 215 provides an example of the StorWatch SAN Data Gateway Specialistwith server and client loaded onto the same Windows NT computer f IBM StorWatch SAN Data Gateway Specialist la lt gt Client LE Netscape Communicator LS Norton AntiVirus gt Figure 215 StorWatch SAN Data Gateway Specialist startup Once the Server has been launched you should see a window similar to Figure 216 Welcome to SAN Director Remote Management Copyright 2666 Pathlight Technology Inc All rights reserved Server Version 2 64 00 Server Start Centralized error handling initiated Log Filename SanngrServerLog 6064 txt SNMP Initialized NetSessionManager instantiated RMI registry started from within server Host 9 113 24 99 Port 1099 Checking for other applications bound with URL rmi 9 113 24 99 1699 starfish No other Server URL bound in registry OK to proceed Trap receiver started UDP Port 162 Server logging on Server to Gateway heartbeat started Server to Client heartbeat started commands rmi 9 113 24 99 1699 starfish Figure 216 StorWatch SAN Data Gateway S
263. lowing sections cover these topics e 7 4 1 Zoning overview on page 216 e 7 4 2 Preparing to define zones on page 217 e 7 4 3 McDATA zoning concept on page 219 e 7 4 4 Creating a zone set and establishing an NT zone on page 220 e 7 4 5 Adding an AIX zone to the existing zone set on page 231 7 4 1 Zoning overview 216 Zoning is a feature that is used to divide a fabric into groups of devices Members of a zone can only communicate with members in the same zone For instance zoning is used to separate groups of hosts with different operating systems from each other or to separate part of the fabric for testing or for separating user groups for security and management reasons There are two approaches to assign devices to a zone One is commonly called hard zoning and uses the FC ports and the devices connected to it to build a zone The other is called soft zoning and uses the WWN of the HBA of a host or the FC interface adapter of another FC device to build the zone The two approaches to configure devices for a zone can be mixed WWN based zoning has the advantage that we can rearrange the devices on the ports without affecting the definition of the zones On the other hand we have to change the zoning definition every time we change a member ina zone or if we have to replace an HBA or an interface adapter to a storage device Port based zoning requires us to change the zoning definition every time
264. lume that you do not want it to access and be restricted from a volume that it had access to previously The system administrator must rebuild the VPS settings if enabled to allow host access to the desired LUNs 8 5 IBM Storage Area Network Data Gateway access options The SAN Data Gateway includes two features that are helpful in providing control and security of host access to SCSI devices e Zoning e Virtual Private SAN or LUN masking Chapter 8 Implementing the SAN Data Gateway 271 8 5 1 Zoning Channel zoning is a feature included with the IBM Storage Area Network Data Gateway Zoning allows you to restrict access between SAN Fibre Channel connections and SCSI channels The default settings allow all SAN connections to access all SCSI channels 8 5 1 1 Configuring zoning To configure zoning and you must be an administrator select Control gt Access Options gt Channel Zoning from the toolbar Figure 228 shows a zoned SAN Data Gateway A check mark will allow access and in Figure 228 you can see that SAN connection 1 has access to SCSI channels 3 and 4 but not to SCSI channels 1 and 2 SAN connection 4 has access to SCSI channels 1 and 2 but not to SCSI channels 3 and 4 To change the settings click on any box and the check mark will toggle on and off All combinations are possible Once the desired settings are selected click OK For the new zone settings to take effect the SAN Data Gateway must be restarted
265. ly 2 oto Port Power Supply 1 Ethernet Port Serial Port Figure 123 2109 S08 front panel GBICs The switch accommodates up to eight GBIC modules All interfaces have status lights which are visible on the front panel giving a quick visual check of the GBICs status and activity The GBIC modules supported are the Short Wave Length SWL and Long Wave Length LWL fiber optic versions If you install less than eight GBIC modules the unused port positions are protected by a metal spring loaded door Fibre Channel connections The IBM SAN Fibre Channel Switch supports the following types of Fibre Channel connections Chapter 6 Implementing an IBM SAN Fibre Channel Switch 141 e Fabric F_Port e Arbitrated loop public and private FL_Port e Interswitch connection E_Port Currently only same vendor switch interconnection E_Port is supported Host attachment The IBM SAN Fibre Channel Switch supports attachments to multiple host systems e IBM Netfinity and Intel based servers running Microsoft s Windows NT or Novell Netware e IBM RS 6000 running AIX e SUN servers running Solaris The host system we used is a Netfinity 5500 running Windows NT Device attachment The SAN connectivity products and storage systems that can be attached are e IBM SAN Data Gateway with IBM Magstar and Magstar MP libraries and the IBM Versatile Storage Server IBM DLT tape libraries e IBM Fibre Channel Hub and Netfi
266. ly communicates with other NL Ports on the same loop Protocol A data transmission convention encompassing timing control formatting and data representation Public NL_Port An NL_Port that attempts login with the fabric and can observe the rules of either public or private loop behavior A public NL_Port may communicate with both private and public NL_Ports Quality of Service QoS A set of communications characteristics required by an application Each QoS defines a specific transmission priority level of route reliability and security level RAID Redundant Array of Inexpensive or Independent Disks A method of configuring multiple disk drives in a storage subsystem for high availability and high performance Raid 0 Level 0 RAID support Striping no redundancy Raid 1 Level 1 RAID support mirroring complete redundancy Raid 5 Level 5 RAID support Striping with parity Repeater A device that receives a signal on an electromagnetic or optical transmission medium amplifies the signal and then retransmits it along the next leg of the medium Responder A Fibre Channel term referring to the answering device Router 1 A device that can decide which of several paths network traffic will follow based on 359 some optimal metric Routers forward packets from one network to another based on network layer information 2 A dedicated computer hardware and or software package which manages the connection between two
267. mbers assigned the new disks added will have LUN numbers assigned to them automatically If there are any composite mirror instant copy or spare drives in the loop then the new disks will not have LUN numbers assigned and become general spares 9 9 2 Adding Routers 338 By adding Routers we can increase the amount of storage a host can access and increase throughput On the rear panel of the Router there are two Fibre Channel GBIC ports that are available and act as a mini hub You can add a Fibre Channel cable from the second port on the existing Router to one of the ports on the second Router You are basically daisy chaining the Routers But since the ports on the Router act as a hub an arbitrated loop is created However in this scenario there is only one Fibre Channel cable from the host to the Router and it is a single point of failure Another option is to add a second Fibre Channel host adapter that will connect to the other Router This provides a high availability feature because there are now two paths to the storage system Software must be used for automatic failover and load balancing between the two Fibre Channel host adapters Failover also can be done manually if so desired On the SSA side there are a few options available Each Router can have its own SSA loop so that each one can support 64 SSA disks In this way Planning and Implementing an IBM SAN storage capacity is scalable because it can be increased b
268. me FIBRE Host Type INT Host Connection ScsiPortt 1 1 0 Emulex Host WN i 0000000 920cae6 Original Host Data FIBRE1 NT 4 0 SP4 ScsiPort 1 1 0 Emulex 10000000 c920cae6 Figure 249 VPS Access window with switch and two hosts New Host Apply Changes Undo Changes Delete a Host EE Take Host Off Line A check mark in the box allows a host access to a LUN or clear the box if you want to restrict access as described in 8 5 2 5 Setting up a Virtual Private SAN on page 279 Figure 250 shows the host plymouth with access to assigned LUNS 17 to 24 and the NT host F BRE7 is now set with access to LUNs 25 to 30 292 Planning and Implementing an IBM SAN FA Virtual Private SAN Access Settings ACTIVE Assigned LUNs plymouth AIX fdevitsesiO Oxef 0 Emulex 10000000 c9221802 unknown SWITCH unknown 20fd0060 69201d4e FIBRE1 NT SesiPort1 1 1 0 Emulex 40000000 c920cae6 gt New Host Data Original Host Data Close Host Name New Host Host Type INT 4 Changes Host Connection Host VWWN Undo Changes Delete a Host Take Host Off Line Figure 250 Setting LUN access for the host FIBRE1 Once you select Apply Changes and then Close the new settings will be in effect For Windows NT to recognize the new devices it will have to be restarted so that it will do a rescan The SWITCH entry in the VPS da
269. member is specified by a physical fabric port number then any and all devices connected to that port are in the zone If this port is an arbitrated loop then all loop devices are in the zone One of the disadvantages of hardware zoning is that devices have to be connected to a specific port and the whole zoning configuration can become unusable when the device is connected to a different port In cases where the device connections are not permanent the use of software zoning is recommended The advantage of hardware zoning is that it can be implemented into a routing engine by filtering As a result this kind of zoning has a very low impact on the performance of the routing process Planning and Implementing an IBM SAN 6 3 3 2 Software zoning Software zoning is implemented within the Simple Name Server SNS running inside the fabric switch When using software zoning the members of the zone can be defined with e Node WWN e Port WWN 6 3 3 3 Zone members Usually zoning software also allows you to create symbolic names for the zone members and for the zones themselves A World Wide Name notation node and port is specified as an eight byte hexadecimal number separated by colons An example of this is 10 00 00 60 69 00 00 8a Zoning has no field knowledge within a World Wide Name the eight bytes are simply compared with the node and port names presented by a device ina login frame FLOGI or PLOGI When a zone member is sp
270. ment multiple SANlets Maybe these small SANs will later merge into an enterprise wide solution This is a difficult question to answer Probably it will be answered differently depending on the size of the organization the IT management philosophy the politics of the organization and the business objectives It is also colored by the degree of risk which you associate with the implementation of an enterprise wide SAN today The technologies are still relatively new Industry standards in some key areas are still to be agreed upon Not all server platforms can easily participate in Fibre Channel configurations and the rate of change is extremely rapid It is probable that in a years time things will look very different than they do today The fact is that the majority of SANs which have been implemented today are relatively small point solutions By this we mean that they were designed to address a specific pain or problem Many users have implemented simple point to point Fibre Channel solutions to solve distance or performance issues Many others have installed small clustered server solutions or shared storage capacity by means of FC ALs because this provides improved connectivity and better utilization of storage resources Others have designed switched fabric solutions for departments or have used FC directors to facilitate large scale storage consolidation in campus locations In practice then the bottom up approach seems to be th
271. mission critical applications Multiple copies of the same data may lead to inconsistencies between one copy and another Data spread on multiple small systems is difficult to coordinate and share for enterprise wide applications such as e business Enterprise Resource Planning ERP Data Warehouse and Business Intelligence BI Backup and recovery operations across a LAN may also cause serious disruption to normal application traffic Even using fast Gigabit Ethernet transport sustained throughput from a single server to tape is about 25 GB per hour It would take approximately 12 hours to fully backup a relatively moderate departmental database of 300 GBs This may exceed the available window of time in which this must be completed and it may not be a practical solution if business operations span multiple time zones It is increasingly evident to IT managers that these characteristics of client server computing are too costly and too inefficient The islands of information resulting from the Chapter 1 Introduction to Storage Area Networks 3 distributed model of computing do not match the needs of the e business enterprise We show this in Figure 3 Typical Client Server Storage Environment UNIX DEC Islands of information Figure 3 Distributed computing model tends to create islands of information New ways must be found to control costs to improve efficiency and to properly align the storage in
272. mode or to another switch E_Port mode FL_Ports allow a Fibre Channel Arbitrated Loop to be connected to a possibly cascaded fabric supporting communication between loop devices and fabric attached devices The IBM switch software automatically selects the port s mode of operation so that no configuration is necessary This feature is extremely useful in a cascaded environment because it allows a user to allocate more bandwidth between switches if necessary without having to plug in a new card or even to change a configuration parameter Even a port previously connected to an external device such as a workstation can be connected to another switch by simply unplugging the old cable and plugging in the new one connected to another switch The 2109 software will detect the state transition and automatically configure the port to be an E_Port 174 Planning and Implementing an IBM SAN Theoretically up to 239 switches are supported in an IBM SAN fabric This allows thousands of fabric and Fibre Channel Arbitrated Loop connections Practical implementations tend not to have taken advantage of this number as yet Considering the complexity and performance that cascading may introduce the recommendation is to limit the number of cascaded switches to seven It should be noted that in a fabric environment some of the ports are used for inter switch links ISL that is E_Ports This means that the maximum number of fabric attached de
273. n Time Between Failure 28 Mean Time To Repair 28 members 159 memory 159 messaging protocols 9 Mirror Capacity 320 mirror capacity 329 Mirror drive 329 mirror drive 319 Mirror Drive Dedicated Spare 321 Mirror Drive Properties 322 Modify Host Systems 80 Modify Users 72 modify volume assignments 103 mother board 45 motherboard 140 MTBF 28 multi drop configuration 7 Multiple Allegiance 23 multi switch 38 multi switch fabric 172 N naming convention 97 NAS 8 9 NetBIOS 9 16 Netscape 64 151 196 Network File System 9 new switch 175 new zone set 221 NFS 9 nicknames 207 Node WWN 159 nodes 31 non blocking architecture 36 non RAID 85 notification options 68 O OEMI 4 Open System Storage 79 operating systems 1 operation mode 153 P packets 10 pain levels 27 Parallel Access Volumes 23 path minute 46 path selection 40 PAV 23 PCI 32 Peer to Peer Remote Copy 22 Performance 28 performance 28 Peripheral Component Interconnect 32 physical consolidation 17 planning 27 Point to Point 284 point to point 14 port numbering 141 Pot WWN 159 power supplies 141 Power On Self Test POST 143 185 preferred domain ID 214 previously configured switch 175 principal switch 243 Private Attributes 341 private host 131 Private Loop 37 private loop 130 Private Loop Direct Attach PLDA 132 133 Problem Log 66 Problem Notification 67 Propagation delays 5 protocol conversion 32 protocol in
274. n drive When the SLIC zone was created to be used as the communication path a disk file or partition was created on a specific disk within the SSA loop As you begin to access the features of the SLIC Router it should be known which disk was used to create the SLIC zone This disk is considered to be the SignOn drive In the topics 9 4 Composite drive on page 314 and 9 5 Mirror drive on page 319 we describe creating composite and mirror drives and you will see that the properties of the individual physical drives may change As they become part of a logical drive they take on the properties of this logical drive If the SignOn drive is used to create a logical drive its attributes may change and you may lose the communication path that was created in the SLIC zone When you select the SignOn drive as a member of a logical drive a dialog box will be displayed as in Figure 264 to remind you that the attributes of this drive may be affected SLIC Manager 1 0 The selected device is a SignOn Drive Do you wish to continue Figure 264 Selecting SignOn drive dialog box As long as the LUN number of the SignOn drive becomes the LUN of the new logical drive the communications from the Manager to the Router will not be affected Another way to be certain that you do not lose your SignOn drive is not to use the SignOn drive to create logical drives However once some logical drives have been created a SLIC zone can be crea
275. n is shown in Figure 97 112 Planning and Implementing an IBM SAN Netfinity NT Server Shortwave fiber CZ Shortwave fiber Unmanaged Hub ESS native FC Figure 97 Simple SAN with hub 5 2 1 IBM Fibre Channel Storage Hub Fibre Channel Storage Hubs are seen as entry level components in SAN fabric installations The Fibre Channel Storage Hub is designed to provide a centralized point of connectivity to provide loop fault tolerance and to simplify configuration management Fibre Channel products that are commonly interconnected to the Fibre Channel Hub are Fibre Channel host bus adapters FC AL storage devices and FC AL storage arrays IBM today offers two kind of hub solutions e The unmanaged IBM Fibre Channel Storage Hub 2103 H07 e The managed IBM Fibre Channel Storage Hub 3534 1RU Chapter 5 Implementing an IBM managed and unmanaged hub 113 5 2 2 IBM Fibre Channel Storage Hub 2103 H07 The IBM Fibre Channel storage Hub 2103 H07 used in our first scenario is a 7 port central interconnection for Fibre Channel Arbitrated Loops that follow the ANSI FC AL standard In Figure 98 we show the hub Figure 98 Fibre Channel Hub and Interface Connector 5 2 3 Hub configuration 114 The IBM Fibre Channel Storage Hub interconnects multiple servers and storage systems over fiber optic media and transfers data at speeds up to
276. nd host with clustering software like the Microsoft Cluster Server in a loop to take over operation if one host fails However this is still not a high availability storage configuration because the loop itself provides no redundancy and is therefore potentially a single point of failure This problem could be solved by using two loops In terms of scalability of bandwidth one FC AL loop by itself is not scalable All devices share the bandwidth of 100 MB s rather than that offered by the managed hub 5 3 IBM SAN Fibre Channel Managed Hub 120 The IBM SAN Fibre Channel Managed Hub 35341RU is an 8 port fibre channel hub that consists of a system board with connectors for supporting up to eight ports This includes seven fixed short wavelength ports one pluggable GBIC port and an operating system for building and managing a switched loop architecture The hub is supported on IBM PC Netfinity servers and other Intel based servers The latest support matrix including adapters and operating system requirements can be found at the following Web page http www storage ibm com hardsoft products fchub msupserver htm In Figure 106 we show the faceplate of the IBM SAN Fibre Channel Managed Hub 35341RU The ports are numbered sequentially starting with zero for the left most port Planning and Implementing an IBM SAN RJ 45 i Ethernet Serial Port Port 0 Port 6 Port pay teeth ys we ee Ei Power Supply Man
277. nd transferring data without blocking the transfer of data from another pair of ports therefore guaranteeing full speed data delivery irrespective of traffic conditions This product technically is based on the IBM 2109 SAN Fibre Channel Switch Therefore it has the potential to be made upgradable in the future so possibly protecting your investment For these reasons we recommend that you normally select the IBM 3534 1RU in preference to the unmanaged hub 2 3 1 3 When will you use a switch You will use a switch as the basis for development of a full switched fabric SAN IBM s products today are the IBM SAN Fibre Channel Switch with two models an 8 port model and a 16 port model IBM 2109 S08 and S16 Planning and Implementing an IBM SAN Multiple switches can be interlinked cascaded to build a large SAN comprising many ports The ultimate limitation in the fabric design is 239 physical switches imposed by the maximum number of unique domain IDs that can be defined Today the practical tested limit is about 15 of this number and with no more than seven hops allowed from the source port to the destination port There are fabric designs in production today with between 10 and 20 switches in a single fabric This number will certainly grow significantly over time The IBM 2109 also supports attachment of FL Ports so it can interlink to Fibre Channel Arbitrated Loops In addition hosts which are not fabric aware and only opera
278. ng example 2 0 0 eee 156 182 Hardware ZONING 1 ee eee eee 158 133 Rabe Views vy accent te ghd cadet Ne Stee cits aN a Sth este Me ate sel 163 134 Zone alias settings 0 0 ce tee 164 135 Zone settings VieW 6 eee 165 136 NTzone creation oessa enan E U eee 166 137 Choosing zone members 000 eee 167 138 Creating zone names 0 eee eee 169 139 Zone Alias create example 00 ee 170 140 Software zoning representation 000 eee ee 171 141 A fault tolerant fabric with six switches 0 00 00 00 eee 173 142 A fault tolerant fabric with four switches 0000 0 ee ee 174 143 Switch view panel oange sorai e naaa cee ee eee 176 144 Fabric view panel 0 0 0 0c cee ee tee eens 177 145 Setting default switch configuration 000 e eee eee 178 146 Cascaded fabric View cacra 00 et ee 179 147 Simple McDATA switch setup 0 0 00 cee ee eee 182 148 Our ED 5000 setup 2 0 ee 188 149 Suggested ED 5000 setup 0 0 eee ee 189 150 Logging in to the EFC Manager on the EFC Server 191 151 EFC Manager product view with no devices configured 192 152 Configuring users 1 0 tee 193 153 Defining new user 2 0 eee 194 154 Modify users 0 0 eee 195 155 Netscape preferences disabling style sheets 197 156 Start page for remote EFC Manager installation 198 157
279. ng the Perform Configuration Update button Perform Configuration Update A progress bar informs us about the steps that are being executed as shown in Figure 49 82 Planning and Implementing an IBM SAN NW Performing Configuration Update Defining Volume Set 16 43 Estimated Remaining Time 6 seconds Continue cy Unsigned Java Applet Window Figure 49 Performing the port definition A window appears to indicate that the configuration update was successful as shown in Figure 50 If the update had failed for any reason we will get an error message N Ed A 1502 The Host System list was successfully modified Note To complete the configuration of newly added SCSI attached hosts they must be assigned to one or more SCSI ports using the Configure Host Ports screen Ey Unsigned Java Applet Window Figure 50 Successfully added host FC port Pressing the OK button on this window will immediately take us back to the Open System Storage panel where we now see the newly configured host Fibre Channel port in the Host Systems Table as shown in Figure 51 Chapter 4 Configuring the ESS with native Fibre Channel 83 Open System Storage Host Systems Nidmame Host Type Attachment WWPN HosmameTP Address Netimity ITSO_1 PC Sever Win NT 4 0 200000EISB00525D a cy or high Assigned Volumes Total 0 volumes There are no volumes assigned to the currently selected host systern
280. nity Channel Hub e IBM Fibre Channel RAID Storage Server and the Netfinity Fibre Channel RAID Controller Unit We attached to the IBM Enterprise Storage Server 2105 F20 with native FC attachment FC 3022 6 1 2 IBM 2109 software features You can manage the IBM Fibre Channel Switch in three different ways e By using the StorWatch Fibre Channel Switch Specialist This is a user friendly Web browser interface e By using Telnet commands e With SNMP We used the Fibre Channel Switch Specialist 142 Planning and Implementing an IBM SAN The IBM Fibre Channel Switch provides advanced management capabilities for e Automatic discovery and registration of host and storage devices e Intelligent rerouting of connection paths should a port problem occur e Cascading of switches for scaling to larger configurations and to provide resiliency for high data availability e Switch zoning for fabric segmentation e Configuration with hot pluggable port GBICs for shortwave or longwave optical connections of up to 10 kilometers Zoning is detailed in 6 3 Zoning in an IBM SAN Fibre Channel Switch environment on page 154 and cascading switches is detailed in 6 5 Cascading IBM 2109 switches on page 171 6 2 The ITSO environment The ITSO target SAN installation consists of e IBM Netfinity 5500 server running Windows NT 4 0 Service pack 5 e IBM ESS 2105 F20 with native FC adapter e IBM 2109 S08 Switch 6 2 1 Installing the 2
281. nning and Implementing an IBM SAN 84 Selected host FC adapter and ESS FC adapter 104 85 Entry point for modifying volume assignments 24 105 86 Modify Volume Assignments panel 0000 eee eee eee 105 87 Selected volumMeS 0 2 0 0c eee 106 88 Accessible Action box 0 ce ees 106 89 Checkbox to assign volumeS 0 eet tee 106 90 Field for target host Fibre Channel ports 2000 0 00 107 91 Applying volume assignment changes 000 ee eee eae 107 92 Volume Assignments successfully changed 00 107 93 To itso assigned volume 0 0 ees 108 94 To Netfinity_ITSO_1 assigned volume 2000 eae 108 95 Open System Storage panel with shared volumes 109 96 Simple loop setup 0 ee eens 111 97 Simple SAN with hub 0 0 0 0 eee 113 98 Fibre Channel Hub and Interface Connector 000 114 99 Gigabit Interface Converter 0 0 es 115 100 FC Storage Hub 2103 H07 front panel 000008 115 TOU INSEN GBIG 3 5 wasn e note Sawa iba hb ee owed Pade awed oe hee 116 102 Insert cable into GBIC 1 eee 116 103 Device Activity LEDs in FC Storage Hub 005 117 104 Disk Administrator view before reboot 00020 e eee eee 118 105 Disk Administrator view after reboot 0 00 e eee eee 119 106
282. nt we need to update the network information that was determined in step 2 Refer to 7 2 1 Setting up the network environment on page 187 for more information 8 Install the ED 5000 on the LAN We configured the switch as a member of the LAN and now we are going to connect it We use the supplied Ethernet cable and connect it to a Ethernet hub Then we route it to the cable access hole inside of the ED 5000 Going through the cable restraint mechanism we connect the cable to the left Ethernet plug We do the same with the second Ethernet cable on the backup card to the left 9 Unpack and inspect the EFC server Note As with the ED 5000 itself verify that the EFC Server is not damaged and that all parts are shipped For a complete list of items to be shipped refer to the ED 5000 Enterprise Fibre Channel Director Installation Manual 10 Install the EFC server on the LAN This consists in plugging together the PC and the peripherals shipped to set up the EFC server for example connecting the monitor the keyboard and the mouse to the PC We connect the inboard Ethernet interface to our LAN Note Don t forget to connect the power supply Chapter 7 Implementing the McDATA ED 5000 185 11 Configure EFC server network information Note Use the information determined in step 2 to install the EFC server in the LAN To log on to Windows NT use the user id Administrator and the password password both of which a
283. nt package on page 58 e 3 2 3 Replacing an older version of the 2105 installation package on page 59 e 3 2 4 Installing the 2105 host attachment package on page 59 e 3 2 5 Locating the World Wide Port Name WWPN on RS 6000 on page 60 e 3 2 6 Verifying the IBM ESS configuration on page 60 3 2 1 The 2105 host attachment package This section provides the instructions to install the host attachment package for the ESS on your host system IBM recommends that you run the host attachment package on each host system attached to the ESS for which an installation script is provided Note For an RS 6000 host system you can use either point to point switched fabric topology or an arbitrated loop hub topology 3 2 2 Before you install the 2105 host attachment package Before you install the host attachment package 1 Attach the ESS to your host system 2 Turn on the host system and all attachments 58 Planning and Implementing an IBM SAN Note Before you install the host attachment package you must have e Root access e AIX system administrator knowledge e Knowledge of the Software and Management Interface Tool SMIT to install the ESS host attachment package 3 2 3 Replacing an older version of the 2105 installation package If you want to replace the older version of the host attachment package tar version and have data that exists on all configured 2105 disks the code
284. number that appears on the top of the VPS settings window as shown Figure 238 on page 282 8 5 2 6 SCSI to LUN map Prior to setting the LUN masking it makes sense to prepare a list that has each LUN and shows which host is allocated to that LUN To assist in this process the Gateway has a map that provides a cross reference of the actual SCSI target and ID to the Gateway s assigned LUN number To access this map you must access the service terminal from the serial port as described earlier in 8 2 Installation on page 252 Once connected type in the command mapShowDatabase The database lists not only devices that are presently connected but also devices that have previously been connected If a previously attached device is later reattached it is assigned back to its 280 Planning and Implementing an IBM SAN previous address Figure 237 shows the output returned to the service terminal ITSO gt mapShowDatabase devid Type Chan tId 636 SNA SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI SCSI 662 662 661 6660666 6 4516 He 68 26166066 45166e 68 26266066 45166e 68 263 6606 6 45166e 68 26466066 4516 8e 68 265 66066 45166e 68 266 66066 45166e 68 267 66066 45166e 68 268 66066 45166e 68 269 66066 45166e 68 26266066 45166e 68 2 6b 66066 45166e 68 2 6c 6606 6 45166e 68 2 6d 660
285. numbers in the field will wrap to 0 from 255 or from 255 to 0 depending on whether you are incrementing or decrementing This will help you get to the desired value quickly When you have all fields set to the desired value continue with step 6 Press the gt button The switch will respond Accept Y N Pressing the lt button indicates Yes Pressing the gt button indicates No e If you respond no the switch will again respond Ethernet IP address You can now restart the process e If you respond yes the switch will respond updating the Config This will cause the new address to be stored After the switch has made the change it will again display Ethernet address If no other address is to be changed Gateway Subnet etc you may simply stop here by pressing the lt button At this point you are done setting up the switch If you need to set other switch addresses Gateway Subnetmask press the button and the switch will respond Ethernet Subnetmask Press the gt button the switch will respond by displaying the current Ethernet Subnetmask in the form xxx xxx XXX XXX To change the current Subnetmask to the preferred address do the following Pressing the button or the v button will either increment or decrement the current field Holding the button down will make this happen rapidly The numbers in the field will wrap to 0 from 255 or from 255 to 0 depending on if you are incrementing or
286. o do a SCSI bus reset You must perform a target reset to make cluster failovers work Use SCSI bus device reset to clear SCSI reservations The SAN Data Gateway does not support LIP reset and a full login is not necessary after the target reset 7 Press escape to return you to the Configuration Settings menu 8 From the Configurations setting menu scroll down to Extended Firmware Settings menu Press Enter 9 From the Extended Firmware Settings menu scroll down to Connection Options to open a window for Option and Type of Connection 10 Select the option 0 Loop only 1 Point to point 2 Loop preferred If you cannot use arbitrated loop then default to point to point 3 Point to point otherwise loop If you cannot use point to point default to arbitrated loop Chapter 3 Implementing Fibre Channel host adapter cards 55 Note The option you select must match the port connections on the ESS Use the StorWatch ESS Specialist to set the Fibre Channel topology on the ESS 11 Use the down arrow key on the keyboard to highlight LUNs per target 12 Use the down arrow key on the keyboard to find and highlight 256 Press Enter 13 Press the escape key 14 Save the changes Highlight Yes 15 Restart the server 3 1 3 Loading the current Fibre Channel adapter driver 56 Perform the following steps to load the current driver onto the QLogic adapter card Go to this Web site http www gqlc c
287. o any Fibre Channel switch For flexible administration and control of storage to server connections most switch vendors provide a set of management interfaces and management software In this chapter we show the practicalities associated with implementing an IBM SAN Fibre Channel Switch The following topics cover e 6 1 IBM SAN Fibre Channel switch on page 140 e 6 2 The ITSO environment on page 143 e 6 3 Zoning in an IBM SAN Fibre Channel Switch environment on page 154 e 6 4 Implementing zoning on page 161 e 6 5 Cascading IBM 2109 switches on page 171 Copyright IBM Corp 2000 139 6 1 IBM SAN Fibre Channel switch IBM offers three different types of switches e The SAN Fibre Channel Switch 2109 S08 which is an OEM product from the Brocade SilkWorm family and is an 8 port model e The IBM SAN Fibre Channel Switch 2109 S16 which is an OEM product from the Brocade SilkWorm family and is a 16 port model e The McDATA Enterprise Fibre Channel Director 2032 001 which is a 32 port director Implementation of this is discussed in Chapter 7 Implementing the McDATA ED 5000 on page 181 In the following sections we will show how to implement the SAN Fibre Channel Switch 2109 S08 Figure 121 shows a picture of the 2109 model S08 Me RO yf Co Po IP Eo Beer Figure 121 IBM 2109 S08 Fibre Channel switch 6 1 1 IBM 2109 S08 hardware components The IBM SAN Fibr
288. o describe a more complex network utilizing hubs switches and gateways Fabric Login Fabric Login FLOGI is used by an N_Port to determine if a fabric is present and if so to initiate a session with the fabric by exchanging service parameters with the fabric Fabric Login is performed by an N_Port following link initialization and before communication with other N_Ports is attempted FC Fibre Channel FC 0 Lowest level of the Fibre Channel Physical standard covering the physical characteristics of the interface and media FC 1 Middle level of the Fibre Channel Physical standard defining the 8B 10B encoding decoding and transmission protocol FC 2 Highest level of the Fibre Channel Physical standard defining the rules for signaling protocol and describing transfer of frame sequence and exchanges FC 3 The hierarchical level in the Fibre Channel standard that provides common services such as striping definition FC 4 The hierarchical level in the Fibre Channel standard that specifies the mapping of upper layer protocols to levels below FCA Fiber Channel Association FC AL Fibre Channel Arbitrated Loop A reference to the Fibre Channel Arbitrated Loop standard a shared gigabit media for up to 127 nodes one of which may be attached to a switch fabric See also Arbitrated Loop FC CT Fibre Channel common transport protocol FC FG Fibre Channel Fabric Generic A reference to the document ANSI X3 289 1996 which d
289. o the same volume As stated earlier SDD is not supported on the SAN Data Gateway connection to an ESS 8 7 3 Adding Fibre Channel switches 296 As switches are added to allow for more host attachments there are considerations about how many paths the host has to the SAN Data Gateway If we refer back to Figure 248 on page 291 there is a single path from the switch to the Gateway Planning and Implementing an IBM SAN If another Fibre Channel path from the switch to the Gateway was added each host now has two paths to access the same LUNs Each host will see the volumes twice Once again to prevent the host from recognizing the same volume twice software similar to SDD is required Another option here is to utilize the zoning and LUN masking capabilities of the SAN Data Gateway This would ensure that certain volumes can only be accessed on one Fibre Channel Gateway port and by a particular host Also available is to add zoning within the switch The switch Fibre Channel ports can be zoned so that the host only has one path to the SAN Data Gateway This would be used in combination with the zoning and LUN masking features of the SAN Data Gateway Chapter 8 Implementing the SAN Data Gateway 297 298 Planning and Implementing an IBM SAN Chapter 9 Implementing the Vicom Fibre Channel SLIC Router The Vicom Fibre Channel SLIC Router 7139 111 enables all IBM 7133 7131 and 3527 SSA Serial Disk Systems to attach to host sy
290. of the information capacity of a transmission channel Bridge 1 A component used to attach more than one I O unit to a port 2 A data communications device that connects two or more networks and forwards packets between them The bridge may use similar or dissimilar media and signaling systems It operates at the data link level of the OSI model Bridges read and filter data packets and frames Bridge Router A device that can provide the functions of a bridge router or both concurrently A bridge router can route one or more protocols such as TCP IP and bridge all other traffic See also Bridge Router Broadcast Sending a transmission to all N_Ports on a fabric Channel A point to point link the main task of which is to transport data from one point to another Channel I O A form of I O where request and response correlation is maintained through some form of source destination and request identification CIFS Common Internet File System Class of Service A Fibre Channel frame delivery scheme exhibiting a specified set of delivery characteristics and attributes Class 1 A class of service providing dedicated connection between two ports with confirmed delivery or notification of non deliverability Class 2 A class of service providing a frame switching service between two ports with confirmed delivery or notification of non deliverability 353 Class 3 A class of service providing frame switching datagram serv
291. offer a variety of NAS solutions These fall into two categories e File servers e Backup archive servers However it is not the purpose of this book to discuss these NAS can be used separately or together with a SAN as the technologies are complementary In general terms NAS offers lower cost solutions but with more limited benefits lower performance and less scalability than Fibre Channel SANs 1 3 2 What a Storage Area Network is 10 A SAN is a specialized high speed network attaching servers and storage devices It is sometimes called the network behind the servers A SAN allows any to any connection across the network using interconnect elements such as routers gateways hubs and switches It eliminates the traditional dedicated connection between a server and storage and the concept that the server effectively owns and manages the storage devices It also eliminates any restriction to the amount of data that a server can access currently limited by the number of storage devices which can be attached to the individual server Instead a SAN introduces the flexibility of networking to enable one server or many heterogeneous servers to share a common storage utility which may comprise many storage devices including disk tape and optical storage And the storage utility may be located far from the servers which use it We show what the network behind the servers may look like in Figure 8 Planning
292. ology you will need to understand the nature of the estimated traffic Which servers and storage devices will generate data movements Which are the sources and which are the targets Will data flow between servers as well as from servers to storage If you plan to implement LAN free or server free data movement what are the implications How much data will flow directly from storage device to storage device such as disk to tape and tape to disk What is the protocol For instance is this standard SCSI or are you including digital video or audio What are the sizes of data objects sent by differing applications Are there any overheads which are incurred by differing Fibre Channel frames What Fibre Channel class of service needs to be applied to the various applications Which departments or user groups generate the traffic Where are they located what applications does each community use and how many in the user group This information may point to opportunities for physical storage consolidation It will also help you to calculate the number of Fibre Channel nodes required the sum of all the data traffic which can be in transit at any time and potential peaks and bottlenecks Can you identify any latent demand for applications which are not carried out today because of constraints of the existing infrastructure If you introduce high speed backup and recovery capabilities across a SAN can this lead to Chapter 2 SAN planning and design
293. om From the home page click Driver Download Click Drivers Click Fibre Channel Adapter Drivers Click QLA2 xxxdrivers Click Windows NT 4 0 Click Driver In the Save As pop up window get the current driver file 2 sccccoxx exe oo N Oa FWD Note Save the file to an existing file folder or create a temporary directory where you can save the current driver file 9 Click Save 10 Close the Web site 11 From your Windows NT Start menu select Run 12 In the Run pop up window ensure the drive letter in the field is the same as the drive letter where you saved the 2 xxxxxxx exe file in step 8 If no drive letter appears type the letter of the drive where you saved the driver file Planning and Implementing an IBM SAN 13 Type the driver file name after x where x is the drive letter you specified to save the file 14 Type the directory name where you want to put the file Click on Zip 15 Click OK to unzip the current driver file 3 1 4 Installing the drivers Perform these steps to install the Fibre Channel adapter drivers N OORA WD 9 From your Windows NT desktop double click the icon for My Computer Double click the icon for Control Panel Double click the icon for SCSI Adapters In the SCSI Adapters window click the tab for Drivers Click Add In the Install Drivers window click Have Disk In the Install from Disk window ensure the drive letter in th
294. on Update Cancel Configuration Update Figure 62 Add Volumes 2 of 2 panel 90 At the top of the panel we see the selected Fibre Channel host and port We can see the total free storage the size of the storage we selected and the largest possible volume size In the left hand panel we can choose the size of the volumes that we want to configure We choose to define four volumes with 96 GB each This is shown in Figure 63 Planning and Implementing an IBM SAN Volume Definition Select a Volume Size 96 2 GB a Number of Volumes 96 3 GB 4 96 4GB Enter 1 to 4 96 5GB DAANA z Figure 63 Volume definition Using the Add button we place the selected amount of volumes in the Volumes to be added panel If we wanted to we can add other volumes We can also choose if we want to place the volumes sequentially or if we want to spread the volumes over all selected storage areas This is shown in Figure 64 Volume Placement Place vol cme Figure 64 Volume placement After all volumes to be defined are specified the window looks like Figure 65 We can now perform the update by pressing Perform Configuration Update Chapter 4 Configuring the ESS with native Fibre Channel 91 Li J Netfinity ITSO_1 _ Free Storage Space Information _ Total Free Space all Fixed Block storage 442 62 GB _ Total Free Space selected storage 420 90 GB _ Largest possible volume size 210 45 GB Port A Ada
295. onsiderations on page 238 e 7 5 2 Setting up our McDATA multiswitch fabric on page 242 Chapter 7 Implementing the McDATA ED 5000 237 7 5 1 Multiswitch fabric considerations 238 The planning of multiswitch fabrics depends on many things Are you going to have a local SAN in one site with up to 32 devices connected Then you may not want to consider cascading switches If you want to build a SAN between two sites that are far apart cascading becomes valued Also if you need more devices connected or if you are looking to introduce extra redundancy cascading is the only way to achieve this Nevertheless we still might think about whether or not or to what extent we want to cascade switches The reason for this is that by using E_Ports we will sacrifice F_Ports Also with an extended fabric the ISLs can possibly become a bottleneck This will lead to the use of more ISLs which means even fewer F_Ports What seems easy in the first instance can get more complicated once we add the zoning concept load balancing and any bandwidth issues that may appear 7 5 1 1 Where multiswitch fabrics are appropriate Certainly there are some possible solutions where a multiswitch fabric is needed For example disaster recovery solutions that are using a SAN can be built upon a McDATA SAN but only when using E_Ports to connect directors between two sites We need directors at both sites to back up one site completely Disaster recove
296. op Planning and Implementing an IBM SAN 5 7 2 Public loop A public loop requires a fabric and has at least one FL_Port connection to a fabric A public loop extends the reach of the loop topology by attaching the loop to a fabric FL_Ports can only attach to NL_Ports In Figure 115 we show a public loop Fabric Switch Disk Storage Server Tape Drive Library Figure 115 Public loop 5 7 3 Private devices in public fabrics The characteristic of a fabric is allowing communication between any host or storage device in the fabric In other words all communication is public Problems may arise when a private Fibre Channel device is attached to a switch Private Fibre Channel devices were designed to only work in private loops For more details about how node and port addressing works refer to Designing an IBM Storage Area Network SG24 5758 5 7 4 QuickLoop feature QuickLoop applies to both the IBM managed hub and the IBM 2109 switch On the managed hub QuickLoop is always in effect on the switch QuickLoop must be enabled at a port level If you have a private host server communication of a public nature is not possible To solve this issue switch vendors including IBM support a QuickLoop feature or a feature which allows public to private or private to public bridging QuickLoop creates a unique Fibre Channel topology that allows host bus adapter
297. or other countries Microsoft Windows Windows NT and the Windows logo are trademarks of Microsoft Corporation in the United States and or other countries PC Direct is a trademark of Ziff Communications Company in the United States and or other countries and is used by IBM Corporation under license Planning and Implementing an IBM SAN ActionMedia LANDesk MMX Pentium and ProShare are trademarks of Intel Corporation in the United States and or other countries UNIX is a registered trademark in the United States and other countries licensed exclusively through The Open Group SET SET Secure Electronic Transaction and the SET Logo are trademarks owned by SET Secure Electronic Transaction LLC Other company product and service names may be trademarks or service marks of others Appendix A Special notices 345 346 Planning and Implementing an IBM SAN Appendix B Related publications The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this redbook B 1 IBM Redbooks For information on ordering these publications see How to get IBM Redbooks on page 351 e Designing an IBM Storage Area Network SG24 5758 e Introduction to Storage Area Network SAN SG24 5470 e IBM Storage Solutions for Server Consolidation SG24 5355 e Implementing the Enterprise Storage Server in Your Environment G24 5420 e Storage Area Networks Ta
298. ork This command may not be executed on an enabled switch you must first disable the switch using the switchDisable command Planning and Implementing an IBM SAN Because some configuration parameters are cached by the switch it is recommended that the switch is rebooted immediately following the execution of the configDefault otherwise unexpected behavior may result With the exception of the following all configuration parameters are reset to their default values e World Wide Name e Ethernet MAC address e Ethernet IP address and subnetmask e IP Gateway address e OEM customization e SNMP configuration e Zoning configuration e System name Now cascading in our target SAN Fabric scenario will work On the Fabric View panel you will see both switches as shown in Figure 146 and all switch port LEDs are showing a steady solid green Figure 146 Cascaded fabric view We have now successfully implemented a cascaded fabric Chapter 6 Implementing an IBM SAN Fibre Channel Switch 179 6 6 Related information 180 IBM 2109 S08 Switch Service Guide SC26 7350 e This guide introduces the product and lists the features you can order It also provides steps for the installation configuration and servicing of the IBM 2109 S08 Switch IBM 2109 S16 Switch User s Guide SC26 7351 e This guide introduces the IBM 2109 S16 switch and its features It also provides information on using the IBM StorWatch SAN Fibre Channel
299. ost Type unknown v unknown _ NewHost Host Connection unknown unknown appi Changos Host WNN 10000000 921da0a 10000000 c921da0a Undo Changes Delete a Host Take Host Off Line Figure 235 VPS host settings The Host information can now be entered in the left hand column labeled New Host Data The Original Host Data column displays the values before any changes are made Once completed select Apply Changes and then Close This window can also be used to enter in host information for a host that has not yet connected to the SAN Data Gateway Select New Host and then enter in the WWN and other parameters that are available You must at minimum enter in a WWN and when entering in the number a colon must be used to separate the first four bytes from the last four bytes Also you will notice that the host description in the top can change color If the information is in blue then the host is offline If the host information is in black then it is online The VPS feature allows an administrator to quickly view the host information and status at a central location 8 5 2 5 Setting up a Virtual Private SAN Remember that the SAN Data Gateway assigns its own LUN numbers and in a sequential order An attached disk device may have a SCSI target of 1 with a LUN ID of 4 but when recognized by the SAN Data Gateway its LUN Chapter 8 Implementing the SAN Data Gateway 279 number assigned will
300. other option is to use the SLIC Manager software The SLIC Manager also provides configuration monitoring and management capabilities of the SLIC router and the SSA drive loop The SLIC Manager can be setup to allow remote access if desired The Manager software consists of a server and client portions The server includes a daemon service and a user interface The client has the user interface only The server portion must be loaded on the host that is directly attached to the Router as the daemon service is started from here The daemon must reside on the host that is directly connected to the Router This host can also be used to run the Manager software for local access The client software can be loaded on to any computer running a supported operating system that can communicate to the host with the daemon service running It must communicate to the server host using TCP IP This allows remote access to the Router and the storage loop This is depicted in Figure 254 Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 303 9 2 1 304 mar Remove Access ameer Bet gic TCP IP a g SSA Disks a a on oe of Figure 254 SLIC Manager access The SLIC Manager has a GUI and a command line interface available for Windows NT systems UNIX systems will only have the command line interface available The following installation and configuration examples will detail using the GUI
301. outer 300 Instant Copy 299 324 Instant Copy Drive 319 Instant Copy Drive Properties 326 inter switch link ISL 34 38 172 175 182 187 Internet Explorer 151 interoperability testing 49 introduction panel 65 IP address 81 122 140 188 setting S16 146 ISL 34 238 islands of information 3 24 ITSO International Technical Support Organization XX ITSO environment 121 143 365 installation steps 181 J Java Plug In 1 2 2 151 JBOD 1 85 JBODs 32 Just a Bunch Of Disks 85 L LAN 1 9 LAN free data movement 21 laser 13 LED 13 152 LIC 74 Licensed Internal Code 73 light emitting diode 13 link controls 14 link incident 212 LIP 34 logical consolidation 18 logical drives 299 334 logical unit number 98 Logical Volume Manager LVM 238 239 login 65 Long Wave Length LWL cable 121 long wave ports 252 Loop 284 loop private 130 public 130 loop initialization 120 LUN 98 LUN access 292 LUN masking 29 252 281 297 341 LUN support 269 M Magstar 23 mainframe 1 Manageability 29 manageability 28 Mapping a general spare 335 Mapping Physical Drives 302 mapRebuildDatabase 271 mapShowDatabase 280 mapWinnowDatabase 271 Master Failover 340 366 Planning and Implementing an IBM SAN Master Router 312 McDATA 181 McData ED5000 37 47 McDATA Enterprise Director ED 5000 181 McDATA Enterprise Fibre Channel Director 294 McDATA Enterprise Fibre Channel Director 2032 001 181 McDATA zoning 216 Mea
302. ow Entering Y installs the new value Typing N delays the change until the next hub restart ow Telnet 9 113 25 6 Connect Edit Terminal Help Fabric OS tm Release v2 1 3 login admin Password WARNING sensor 6 Fan 1 is below minimum fchub1 admin gt ipAddrSet Ethernet IP Address 9 113 25 6 9 113 25 7 Ethernet Subnetmask 255 255 255 0 Fibre Channel IP Address none Fibre Channel Subnetmask none Gateway Address 99 113 25 254 Set IP addresses now y set now n next reboot Figure 111 Changing the IP address Chapter 5 Implementing an IBM managed and unmanaged hub 125 126 9 Ipaddress admin gt logout This ends the Telnet session We have completed the installation of the 3534 Managed Hub 5 4 1 2 Setting the IP address using the serial port We also show how to set the hub IP address by using the serial port If you also choose to set up the IP address using the serial cable open a HyperTerminal session on your laptop and proceed with these steps Note Opening a HyperTerminal session varies depending on which version of Windows you are using With Windows NT to start a HyperTerminal session go to Start gt Programs gt Accessories Configure HyperTerminal as follows 1 We used a laptop with Windows NT 4 0 Prior to plugging the hub into the electrical outlet we attached the laptop to it with the serial cable female to female shipped with the hub
303. own in Figure 30 Communications 68 Planning and Implementing an IBM SAN Modify Remote Support Reset PE Password Figure 30 Communications panel Here we can view the network information for which the ESS is configured and verify if the call home feature and remote service access is enabled Also we can change the remote support features shown in Figure 31 and reset the PE password Chapter 4 Configuring the ESS with native Fibre Channel 69 Enable Perform Configuration Update Cancel Configuration Update Figure 31 Remote support modification 4 1 1 6 Storage allocation Using the Storage Allocation button we get the Storage Allocation Graphical View panel shown in Figure 32 Storage Allocation 70 Planning and Implementing an IBM SAN Clear View View All Storage Tabular View Figure 32 Storage Allocation panel This is the start panel for host and storage based configurations In 4 1 2 Viewing the Storage Allocation of the ESS on page 75 we will show how to configure the ESS for our SAN For completeness we will show the other panels 4 1 1 7 User Administration Using the Users button we get the User Administration panel as shown in Figure 33 Users Chapter 4 Configuring the ESS with native Fibre Channel 71 User Administration User Permissions Total Users 1 ine sort Y no sort x no sort nost Y Usemame Access Level IP Address Ra jtonm
304. pe Future In Fabrics SG24 5474 e IBM Enterprise Storage Server SG24 5465 e Introduction to IBM S 390 FICON SG24 5176 B 2 IBM Redbooks collections Redbooks are also available on the following CD ROMs Click the CD ROMs button at ibm com redbooks for information about all the CD ROMs offered updates and formats CD ROM Title IBM System 390 Redbooks Collection IBM Networking Redbooks Collection Collection Kit Number SK2T 2177 SK2T 6022 IBM Transaction Processing and Data Management Redbooks CollectionSK2T 8038 IBM Lotus Redbooks Collection Tivoli Redbooks Collection IBM AS 400 Redbooks Collection IBM Netfinity Hardware and Software Redbooks Collection IBM RS 6000 Redbooks Collection PDF Format IBM Application Development Redbooks Collection IBM Enterprise Storage and Systems Management Solutions Copyright IBM Corp 2000 SK2T 8039 SK2T 8044 SK2T 2849 SK2T 8046 SK2T 8043 SK2T 8037 SK3T 3694 347 B 3 Other resources These publications are also relevant as further information sources e e e e e e e e ESS Web Interface User s Guide for ESS Specialist and ESS Copy Services SC26 7346 IBM Storage Area Network Data Gateway Installation and User s Guide SC26 7304 IBM 2109 Model S08 User s Guide SC26 7349 IBM 2109 Model S08 Switch Service Guide SC26 7350 IBM 2109 S16 Switch User s Guide SC26 7351 IBM 2109 S16 Switch Service Guide SC26 7352 IBM Enterprise Stora
305. pecialist server The client software can now be launched If the server and client are not on the same PC then a dialog box will appear to allow you to enter in the IP address of the computer that has the server software loaded If the server and client are on the same computer you will be automatically connected to this server After connection to the server is complete a dialog box will appear in which you can enter in a user name and password 258 Planning and Implementing an IBM SAN The IBM Storage Area Network Data Gateway provides a default administrator e User Name StorWatch e Password StorWatch The fields are case sensitive so they must be entered in as shown above A new administrator account should be set up by selecting Admin gt Add User from the toolbar After a new administrator account is created then the default user StorWatch is deactivated m Note If a new administrator account has been created and the password is lost and no other account has administrator access a service representative must be contacted 8 3 2 Using the StorWatch SAN Data Gateway Specialist Once you are logged in to the Specialist you must now connect to the SAN Data Gateway A dialog box appears requesting the IP address of the SAN Data Gateway As it connects it will download the information from the SAN Data Gateway and be presented on your window If a dialog box does not appear automatically select Tools gt Conne
306. peration on the Private Attributes setting please refer to the SLIC Manager Installation and User Guide 310 605807 In all cases if extra control for disk access is required a third party software such as Tivoli SANergy must be used 9 9 3 2 Heterogeneous hosts As the Router does not provide for LUN masking you must use the SLIC Manager Private Attribute setting or a third party software such as Tivoli Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 341 SANergy to restrict and control host access to the disk The Private Attributes and Tivoli SANergy can be used together for added control 342 Planning and Implementing an IBM SAN Appendix A Special notices This publication is intended to help professionals plan and implement a Storage Area Network The information in this publication is not intended as the specification of any programming interfaces that are provided by the solutions or products mentioned See the PUBLICATIONS section of the IBM Programming Announcement for each described product for more information about what publications are considered to be product documentation References in this publication to IBM products programs or services do not imply that IBM intends to make these available in all countries in which IBM operates Any reference to an IBM product program or service is not intended to state or imply that only IBM s product program or service may be used Any functionally equi
307. priately this entry level SAN installation should now run as a Fibre Channel Point to Point fabric solution We used the StorWatch Fibre Channel Switch Specialist to verify our SAN solution 6 2 5 1 Launching the StorWatch Fibre Channel Switch Specialist Access to the IBM StorWatch Specialist is provided through one of the following Java enabled Web browsers e For Windows 95 98 or Windows NT Internet Explorer 4 0 or above Netscape 4 51 or above e For UNIX Netscape 4 51 or above In addition to the above Java Plug In 1 3 0 is recommended To Launch 1 Start the Web browser if it is not already active Chapter 6 Implementing an IBM SAN Fibre Channel Switch 151 2 Enter a switch name or IP address in the Location Address field 3 The Fabric View appears displaying all compatible switches in the fabric We show an example of a cascaded fabric in Figure 127 C4 News C4 Financial 4 Search Engines 4 Channels Figure 127 Launch view In our installation port 1 of the switch was connected to the Netfinity NT host and port 6 was connected to the ESS storage device The LED to the right of ports 1 and 6 is a steady green indicating link is working and shown in Figure 128 Figure 128 Port attachment In Figure 129 we indicate the F Port connection of port 6 which is attached to the storage device As our Port Type is F Port indicating fabric we know that our installation has been successful 152 P
308. prompts you to remove all ESS product related hdisk devices Perform the following steps to remove the devices 1 Run umount on the file system 2 Run varyoffvg 2105 volume group 3 Type rmdev a1 on the command line to unconfigure the 2105 devices After you install the ibm2105 rte file and all of the 2105 devices are reconfigured vary on the volume groups and remount the file systems The data on the file systems should now be available again Perform the following steps by using SMIT to install the IBM 2105 host attachment on your system 3 2 4 Installing the 2105 host attachment package Note The following procedure is an example The example uses dev cdO for the address of the compact disc Your address may vary Install the host attachment package from a compact disc or a diskette You must have superuser authority to complete the instructions 1 From your desktop window type smit install_update to go directly to the installation panel 2 Click Install and Update from the Latest Available Software and press Enter 3 Press F4 to display the Input Device Directory for Software window Chapter 3 Implementing Fibre Channel host adapter cards 59 4 Select the CD ROM drive that you are using for the installation for example dev cdO Press Enter The Install and Update from the Latest Available Software window displays 5 Click Software to Install and press F4 The Install and Update from the Latest Av
309. pter 2 I0 Bay 1 Volumes to be added Volume Definition 96 00 GB 96 00 GB 96 00 GB 96 00 GB zzi z teu l Perform Configuration Update Cancel Configuration Updatg Figure 65 Performing the volume definitions Again we get a window warning us about a time intensive action and we are asked if we want to continue with the update This is shown in Figure 66 A Warning 1402 Time iniensive Action The requested configuration update will take approximately 2 to 5 minutes to complete Would you like to continue and perform the update Figure 66 Warning window As with every run of a configuration update there is a progress window as shown in Figure 67 92 Planning and Implementing an IBM SAN NS Performing Configuration Update Modifying Logical Subsystem Configuration 1581 Estimated Remaining Time 36 seconds JE Unsigned Java Applet Window Figure 67 Progress window for volume definition After the update has finished successfully we are presented with the message shown in Figure 68 N Ea 2101 The specified volumes have been successfully added eB Unsigned Java Applet Window Figure 68 Successful volume update Pressing OK leads us back to the Adding Volumes 1 of 2 panel from where we can start the process again and define other volumes To see what we have done we press on the Storage Allocation button shown in Figure 69 This is the same Stora
310. quires the Subsystem Device Driver SDD on each host this is also true when using the Fibre Channel adapters Software similar to SDD is required in the host for it to recognize that it may have two or more paths to the same volume If this was not loaded on the host the host would recognize the volumes as being different and there will be contention problems and data corruption As SDD is currently only supported on SCSI adapters and not with Fibre Channel adapters it is also not supported when using the SAN Data Gateway to connect to an ESS 8 7 2 Multiple SCSI connections The SAN Data Gateway can have all four SCSI channels connected to a single ESS A volume in the ESS can be assigned to more than one SCSI port in the ESS If these SCSI ports that have the same volume assigned to it are connected to a SAN Data Gateway the Gateway will assign multiple LUN ID numbers to the same volume This is because the Gateway upon discovery or scanning of the SCSI buses will view the volumes on each SCSI channel as separate volumes For further explanation on this refer to 8 4 2 Device discovery on page 270 If each LUN ID was then masked and zoned to different host systems it is vital that the hosts have some access sharing software loaded to control access and avoid data corruption If the LUN IDs were assigned to the same host then again software similar to SDD is required for the host to recognize that it has multiple paths t
311. r Planning and Implementing an IBM SAN Click on the OK button and the Mirror Drive Properties will now reflect the change as shown in Figure 281 Mirror Instant Copy Drive Properties x Mirror Instant Copy Drive Selection T33282 ITSOm Looz bad Cancel ih Drive Members Tool Box frarget SSA UID Function i i Identify enber T00003 35E6848D T00004 35E22D6C Delete Member Identify Mirror T33283 copyl A Toooos 3SE6840E s Drive Characteristics Capacity MB 8689 PUBLIC FC LUN B Drive Name rrsom Number of 3 Change Members Figure 281 Mirror drive properties with copy drive attached Click OK to complete the process 9 6 4 Detach Instant Copy Drive from a mirror To detach or split off the copy drive from the mirror the procedure is similar except at the Mirror Drive Properties window select Delete Member A window will appear that displays all current members of the Mirror Select the Copy drive and then delete it from the Mirror The Copy drive can now be accessed by another host 9 7 Combining composite and mirroring The SLIC Manager can also be used to combine the two features of the Router You can create a mirror drive using composite drives A mirror can have drive members of different sizes but the actual mirror capacity will be the smaller of the drive sizes Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 329 9 7 1
312. r of addressing as the host is no longer directly attached to the disk devices but will see a single target ID the SAN Data Gateway In a regular SCSI environment with the host attached directly to the SCSI device the host is able to recognize the target and LUN ID of that device Since we add the SAN Data Gateway in between the host and the device the host is no longer directly attached to the SCSI device to recognize its target or LUN ID 8 4 3 Assigning LUN IDs The Gateway assigns LUN IDs to all the attached devices up to the maximum of 255 and it creates a map of the actual disk device target and LUN ID to its own assigned LUN ID This map is stored in nonvolatile memory within the Gateway 8 4 4 Adding SCSI devices 270 New SCSI devices can be added at any time If a new device is added for example another volume in the IBM Enterprise Storage Server has been Planning and Implementing an IBM SAN assigned to a SCSI port on a San Data Gateway the SCSI buses must be rescanned to detect the new device This can be done from the service terminal using the scsiRescan command or from the StorWatch SAN Data Gateway Specialist If using the Specialist select and highlight the SCSI bus that the device has been assigned to and select Controls gt SCSI Channel gt Re Scan SCSI Bus As it rescans the SAN Data Gateway will assign the next available LUN address to the new device Refresh the data on your Specialist window by selecting Vi
313. rage panel we use the Add Volumes button to define volumes on the newly configured disk groups as shown in Figure 58 Modify Host Systems Configure Host Adapter Ports Configure Disk Groups Add Volumes x Modify Volume Assignments Figure 58 Entry to the Add Volumes 1 of 2 panel This takes us to the Add Volumes 1 of 2 panel as shown in Figure 59 Add Volumes 1 of 2 Click on a Host or Array to see paths _IN ddl_2N hudson itso moe2200 moe5100 Jee Information Netfinity Figure 59 Add Volumes 1 of 2 panel with one Fibre Channel host port selected From here we select the Fibre Channel port that we configured Again this is not the host but is graphically shown as the host because our host has only one adapter built in 88 Planning and Implementing an IBM SAN We also have to select one of the ESS Fibre Channel ports Figure 60 before we can go to the next panel J s GS S Babes Figure 60 Selecting an ESS Fibre Channel port Logically there is no reason to have to select a specific Fibre Channel port because we can reach the volumes through every properly configured ESS Fibre Channel port However by clicking on an ESS Fibre Channel port we can now see the storage that we can use to define volumes on This is shown in Figure 61 Add Volumes 1 of 2 faa Click on a Host or to see paths adl 2N B itso moe 200 moe5100 D gt Total Free Space 4
314. rdinate Routers so that all systems in the loop are aware of what is happening 9 9 2 2 Using SLIC Manager on additional Routers Since the master Router does all the work it is not necessary to use SLIC Manager to view the new Router However depending on how the new Router is used this may become a requirement The same daemon used to connect to the first Router can be used to connect to several Routers The configuration file is edited again and the process to name and create a SLIC zone on this new Router can be added within the same file The SLIC Manager software can now be used to communicate to the new Router The SLIC Manager can only communicate to a single Router at a time Multiple SLIC Manager sessions can be started to communicate to each Router 9 9 2 3 Master failover If at some time the Router that is designated as the master within the SLIC storage system fails the master designation will failover to next nearest Router This is accomplished within the communications between the Routers and it is done automatically When the new Router accepts becoming master it will maintain the master role if even the failed Router is replaced and rejoins the storage system The master role can be changed back to the original Router or to another Router if desired using the SLIC Manager software 340 Planning and Implementing an IBM SAN There is no capability to select a specific failover Router 9 9 3 Adding hosts Th
315. re Channel switch with a maximum of 32 Generic Ports G_Port These ports when connected to another ED 5000 act as Expansion Ports E_Ports to setup inter switch links ISL Connected to hosts or storage devices they act as fabric ports F_Port The ports are on cards with four ports each Most parts of the switch are hot pluggable field replacable units FRU The design is geared towards high reliability and availability The switch supports features like internal failover to redundant FRUs and fault detection The maintenance firmware loads and upgrades can be done without downtime and a call home feature is also provided The management and monitoring software does not run directly on the switch It is provided through an Ethernet connected PC which is called the Enterprise Fabric Connectivity EFC server This PC is shipped as part of the ED 5000 package For in depth information on the McDATA also see the McDATA ED 5000 manuals provided with the switch e ED 5000 Enterprise Fibre Channel Director Planning Manual e Enterprise Fabric Connectivity Manager User Manual e ED 5000 Enterprise Fibre Channel Director User Manual e ED 5000 Enterprise Fibre Channel Director Installation Manual e ED 5000 Enterprise Fibre Channel Director Service Manual 182 Planning and Implementing an IBM SAN e ED 5000 Product Manager User Reference Guide e EFC and Fabric Manager User Reference Guide 7 1 2 Hardware overview The hardware that we will use
316. re case sensitive 12 Set date and time on the EFC server Note As the audit and event logs of the EFC Manager are time stamped with the date and time of the EFC Server make sure that the time and date on the EFC Server are correct 13 Configure Windows NT users on the EFC server Note If you plan to allow more users to access the EFC Server then we recommend that you create Windows NT users on the EFC server and change the Administrator password 14 Configure the call home feature Note The EFC Server has a built in modem which you can configure to report ED 5000 problems to remote service support facility 15 Assign EFC Manager user names and passwords The EFC Manager has the ability to manage different users with different rights Refer to 7 2 3 Defining users on the EFC Manager on page 192 for more information 16 ldentify the ED 5000 to the EFC Manager The ED 5000 has to be made known to the EFC Manager if it is to be managed from within the EFC Manager This is described in 7 3 Managing the ED 5000 fabric on page 203 17 Record information for EFC Server restore Note You will need to record some information in case you have to restore the EFC manager software 18 Verify EFC Server communication with the ED 5000 The operational state of the devices and the communication is indicated with different colors and symbols in the EFC Manager 19 Set date and time on the ED 5000 The logs of the P
317. re is 10 Ensure that at the end of every line a semi colon is used and that if several SLIC Zones are created a comma separates them Save and exit the file 9 2 2 3 Installing the SLIC Manager Daemon With the configuration file edited and a SLIC Zone created the daemon service can be installed and run To install the service in Windows NT open a DOS prompt and go to C ibm7190 sdus Type in slicd install and the daemon will be installed 9 2 2 4 Starting the SLIC Manager Daemon To start the daemon service select Start gt Settings gt Control panel from Windows NT Double click on the Services icon Scroll down until you see Vicom SLIC Manager select and highlight it You will see two columns to the right to indicate its status To start the service select the Start button and it will take a few moments to complete Once it is done you will see the word Started in the Status column If the Startup column contains the word Automatic no further action is required If not select the Startup button and change the Startup Type to automatic This will have the daemon service start automatically during a reboot This is shown in Figure 257 Planning and Implementing an IBM SAN Services Status Service Telephony Service TRAMM API Server TSM Serverl TSM Server2 TSM Server3 UPS Vicom SLIC Daemon service Started Windows Installer Workstation Started World Wide Web Publishing Service Started Start
318. rectly implemented adding a second host to the ESS is straight forward from the point of view of the switch To connect the new host to the switch you need at least one free switch port with the appropriate GBIC module SWL or LWL and a multi mode 50 125 or 62 5 125 type cable allowing distances up to 500 meters or a single mode 9 125 type cable with distances up to 10 kilometers You must use the StorWatch Enterprise Storage Server Specialist to add a new host to the ESS This has been described comprehensively in 4 1 4 Defining a new host FC port with its WWPN on page 80 and more detailed information about how to do this is found in the ESS Web Interface User s Guide SC26 7346 6 4 1 1 Adding a second host quick steps 1 From the Storage Allocation panel click Open Systems Storage 2 From the Open Systems Storage panel select Modify Host Systems 3 To add a host system enter the following information in the Host Attributes table on the left side of the panel a Enter the host name in the Nickname field A host nickname should not exceed 29 characters b Enter the host type in the Host Type field Use the drop down list box to select a host type 4 Enter the host attachment in the Host Attachment field Use the drop down list box to select either SCSI attached or Fibre Channel attached SCSI is the default Chapter 6 Implementing an IBM SAN Fibre Channel Switch 161 5 Enter the World Wide Node Name in the
319. rminal Help Fabric OS tm Release v2 1 3 login admin Password WARNING sensor 6 Fan 1 is below minimum Fchub1 admin gt Jj Figure 109 Start setting the IP address 4 At the prompt enter the command ipaddrset This is the command to set the IP address and the result is shown in Figure 110 The current Ethernet IP address is shown Now you can enter your new address Press Enter to keep the old one jc Telnet 3 015 206 Connect Edit Terminal Help Fabric OS tm Release v2 1 3 login admin Password WARNING sensor 6 Fan 1 is below minimum fchub1 admin gt ipAddrSet Ethernet IP Address 9 113 25 6 JJ Figure 110 Current IP address Planning and Implementing an IBM SAN 5 Ethernet Subnetmask Current Subnetmask will be shown or None new Subnetmask or press Enter This is the new Subnetmask from your system administrator or if none is required press Enter 6 Fibre Channel IP Address None press Enter Fibre Channel Subnetmask None press Enter 7 Gateway Address Current Gateway address or None enter new Gateway address or press Enter This is the Gateway address the system administrator provided or if none is required press Enter 8 lpaddress admin gt logout This process in step 5 through step 8 is shown in Figure 111 In our example we have changed the IP address from 9 113 25 6 to 9 113 25 7 The final prompt will ask if you want to set the IP address to the new value n
320. roduct Manager are stamped with the time and date of the ED 5000 Note To be able to use the logs efficiently set the time using the Product Manager Planning and Implementing an IBM SAN 20 Test remote notification Note If you want to be sure that remote notification works as expected test it using the Maintenance icon in the Product View of the EFC Manager 21 Configure the EFC Manager and ED 5000 Product Manager For example this consists of configuring the identification ports and the operating parameters 22 Back up the configuration The EFC Server ships with an internal Zip drive It is used to mirror the EFC Manager data every time it has changed 23 Cable the ED 5000 device ports Last but not least we connect the Fibre Channel cables to the Fibre Channel cards of the ED 5000 24 Configuring a multiswitch fabric This consists of connecting two switches together using a fiber optic connection known as an inter switch link Refer to 7 5 Cascading with McDATA building a multiswitch fabric on page 237 7 2 Setting up an environment for using and managing a McDATA SAN Now that we have installed the ED 5000 in a base configuration in the following sections we cover these topics e 7 2 1 Setting up the network environment on page 187 e 7 2 2 Logging on to the EFC server and the EFC Manager on page 190 e 7 2 3 Defining users on the EFC Manager on page 192 e 7 2 4 Installin
321. ry and high availability can be established together using a multiswitch fabric and open system hosts using Logical Volume Manager LVM mirroring together with clustering software such as HACMP for AIX or Veritas Cluster Server Due to the high availability and the many ports of the McDATA ED 5000 two McDATA ED 5000 may be enough 7 5 1 2 Solutions for high availability and disaster recovery An example of a solution that provides high availability with disaster recovery is shown in Figure 200 Planning and Implementing an IBM SAN local remote site site Open i Systems i Systems Server i Server Heartbeat Shortwave fiber McDATA i McDATA ED5000 e i ED5000 ESS E ESS native FC native FC Figure 200 High Availability and disaster recovery with McDATA This is a setup which consists of the same configuration at both the local and the remote site Both sites can be up to 10 km apart when using 9 micron fiber optic cable The Open Systems Server Cluster for instance can consist of two or more RS 6000 with HACMP The mirroring can be done with the native LVM on AIX Another solution can be SUN servers running for example the Veritas Cluster Server and the Veritas Volume Manager Due to the high availability of the McDATA ED 5000 one may be sufficient if that leaves enough ports to accommodate the specific environment When more ports are needed this solution can easily be extended with another director
322. s are grayed out except the Fibre Channel option By selecting this a dialog box will display the parameters that can be changed for the Fibre Channel port selected If any of the settings as shown in Figure 227 are changed the SAN Data Gateway must be restarted Planning and Implementing an IBM SAN PA Set Fibre Channel Parameters x Loop ID v Soft FC Tape Frame Size 512 21024 2048 ISP2200 Port Mode Target Initiator Port Mode Initiator Private Target and Initiator Public Connection Options Loop Loop Preferred Point to Paint Point to Point Preferred Identify Channel OK Cancel Figure 227 Fibre Channel port parameters A detailed description of the SCSI and Fibre Channel settings can be found in the IBM Storage Area Network Data Gateway Installation and User s Guide SC26 7304 8 4 SCSI Devices The four SCSI ports on the SAN Data Gateway support Differential Ultra Wide SCSI devices It will automatically negotiate speed for wide or narrow bus width devices as well as standard fast and ultra speeds The SAN Data Gateway provides a termination on each of its SCSI buses The termination can be separately disabled if so desired from the StorWatch SAN Data Gateway Specialist window 8 4 1 LUN support The SAN Data Gateway provides support for up to 256 LUNs Each SCSI channel supports up to 15 SCSI targets and up to 32 LUN per target This is subject to the overall tota
323. s of independent disks RAID The capacity of such arrays may be measured in tens or hundreds of GBs but I O bandwidth has not kept pace with the rapid growth in processor speeds and disk capacities Distributed clients and servers are frequently chosen to meet specific application needs They may therefore run different operating systems such as Windows NT UNIX of differing flavors Novell Netware VMS and so on and different database software for example DB2 Oracle Informix SQL 2 Server Consequently they have different file systems and different data formats Client Server Infrastructure Storage Storage aji Storage Storage Storage Storage Storage g Individual Workstations Local Area Network Figure 1 Typical distributed systems or client server infrastructure Managing this multi platform multi vendor networked environment has become increasingly complex and costly Multiple vendor s software tools and appropriately skilled human resources must be maintained to handle data and storage resource management on the many differing systems in the enterprise Surveys published by industry analysts consistently show that management costs associated with distributed storage are much greater up to 10 times more than the cost of managing consolidated or centralized s
324. s that use Fibre Channel Arbitrated Loop FC AL without knowledge Chapter 5 Implementing an IBM managed and unmanaged hub 131 132 of the SAN fabric that is to say an adapter that cannot perform fabric login commonly referred to as private loop initiators to communicate with Fibre Channel Arbitrated Loop public storage devices This can also include IBM 2109 Fibre Channel Switches QuickLoop allows individual switch ports to be designated as arbitrated loop ports allowing a private loop host initiator to communicate with arbitrated loop storage devices as though they were all contained in one logical loop The QuickLoop feature supports legacy devices Legacy devices are those that are in a Private Loop Direct Attach PLDA environment The QuickLoop feature allows these devices to be attached to a SAN and operate no differently than in a PLDA environment As opposed to the IBM SAN Fibre Channel Switch where you have the option of defining QuickLoop ports the IBM SAN Fibre Channel Managed Hub 35341RU ports work in QuickLoop mode as a default For a list of supported devices for the IBM 3534 see the Web site http www ibm com storage fchub In a simple scenario with one host a single hub and some storage devices it does not matter if we have a private or public loop or if there is a fabric Having a hub rather than a switch has an impact on the performance and integrating the hub with QuickLoop into a SAN fabric is bene
325. s to attach to your local area network to establish a network connection to the hub In Figure 107 we show the Ethernet and serial port locations You can change this IP address later using a Telnet command or by using the StorWatch Specialist issued from any server having access to the same LAN This is the easiest way to set the IP address Your system or network administrator will advise if the default address can be used If using the default IP address is not possible the IP address will have to be set using the Ethernet port or the serial port Set the IP address using the 122 Planning and Implementing an IBM SAN information provided by the system administrator and record this on the pre installation checklist 8990400904660 ane ETTET TE a nog tabeeeeee perros rre 0000see8 Serial Port Ethernet Port Figure 107 IBM 3534 management ports For a comprehensive description of how to set the IP address using either the serial or the Ethernet port refer to BM SAN Fibre Channel Managed Hub 3534 Service Guide SY27 7616 The hub we used in our test scenario had been installed before and we had to change the IP address In Figure 108 we show the front panel of the hub Port 5 is operating no GBIC is installed and the left fan has stopped working This is indicated by a red color and no movement of the fan Cir bb Cy oe Ce ae WN 10 00 00 60 69 30 08 0d Ether IP 9 113 25 6 Domain ID 2 Ether NM 25
326. se will lead you to evaluate the technical requirements placed on the supporting IT infrastructure and what a SAN must provide to meet these requirements These can be measured in terms of reliability availability and serviceability RAS performance scalability security manageability and affordability 2 1 2 1 RAS Assess the nature of the applications to be supported on the SAN Do they need to be available on a 24 hour 7 days a week basis If they are not mission critical how much downtime if any is acceptable What are the costs of downtime Non availability of some applications may be measured in hundreds of thousands or even millions of dollars per hour for others there may be very limited financial impact The answer may lead you to focus on hardware aspects such as mean time between failure MTBF mean time to repair and serviceability characteristics such as fault tolerance hot swappable components failover facilities error reporting and call home capabilities 2 1 2 2 Performance What are the performance characteristics required to support the various applications on the SAN How do you measure this With throughput MB second or I Os per second or response time What is the maximum capacity or bandwidth required for peak loads What percentage of the SAN capacity will be used on average and at what level of utilization would it become saturated What happens to performance in the event of failure of Planning and
327. sed to distinguish between the net part and the host part of the IP address The host needs this information to know how to treat the addresses it uses Domain Name System DNS host name EFC Server 1 Numbers for example IP addresses are sometimes difficult for people to remember Therefore a service called the Domain Name System is used to assign host names to IP addresses and it makes the destination transparent and reachable with an easy to remember host name DNS domain name sanjose ibm com The name says that the DNS is using a hierarchy of domains for addressing The host name itself is not unique to be able to reach the hosts given we need a fully qualified host name which consists of the host name and the domain it is in Planning and Implementing an IBM SAN e Gateway address 9 113 42 250 This is the default router which has to be known to the IP stack to route to a destination outside the current LAN using IP For information about how to set up the network on the Windows NT PC refer to the ED 5000 Enterprise Fibre Channel Director Installation Manual The director needs to have following information too to be accessible from the management software For ED 5000 1 this is e IP address 9 113 25 63 e Subnet mask 255 255 254 0 e Gateway address 9 113 42 250 To set the network information on the switch we use the operator panel We can scroll though the information displayed on the panel with the Advance
328. st important setting here is the Preferred Domain ID This has to be unique for each director within a multiswitch fabric If we add directors to our SAN we have to change this parameter Otherwise the fabric build process will fail To change the operating parameters we first have to set the ED 5000 offline To set the director offline which will terminate all FC operations we select Maintenance gt Set Online State which is shown in Figure 176 214 Planning and Implementing an IBM SAN Figure 176 Set Online State Now we can go to the Configure Operating Parameters window by selecting Configure gt Operating Parameters Here we can change some of the Fibre Channel parameters for the director for example the preferred domain ID This is shown in Figure 177 Figure 177 Configure operating parameters For more information regarding the other management and monitoring functions of the ED 5000 refer to the Enterprise Fabric Connectivity Manager User Manual Chapter 7 Implementing the McDATA ED 5000 215 Now the director is ready for use in the Fibre Channel network It can be connected to devices such as other switches storage or hosts 7 4 Zoning with McDATA The advantage of a SAN and the accessibility of any storage anywhere raises issues which must be solved For example do we really want any device to have the ability to access all storage Zoning helps to split the SAN into logical parts The fol
329. st on SAN 1 in the VPS settings window the host will not see that LUN because of the zoned settings The same is true for SAN connection 4 A host connected here will access LUNs only on SCSI channels 1 2 and 3 but not channel 4 regardless of whether the VPS settings will allow it 8 6 Adding Fibre Channel fabric components The SAN Data Gateway is an integral component in a storage network As such you can also attach other SAN fabric components to the Gateway to Chapter 8 Implementing the SAN Data Gateway 283 increase the connectivity options in a SAN Hubs and switches can be easily added and allow many more systems local or remote to access the SCSI devices on the Gateway As switches provide more flexibility and hubs are mainly used to extend distances the following discussion will focus on switch implementation rather than hubs 8 6 1 Connecting an IBM SAN Fibre Channel Switch 284 The IBM SAN Fibre Channel Switch provides either an 8 port or 16 port switch that can also be cascaded 8 6 1 1 Allowing fabric connection Before connecting a switch to the SAN Data Gateway there is a setting that must be checked to allow proper fabric connection From the SAN Data Gateway Specialist select the Fibre Channel port that will have the switch connected Select Control gt Fibre Channel and the Set Fibre Channel Parameters window appears as shown in Figure 240 Pf Set Fibre Channel Parameters xi LoopID v Soft FC
330. st port using the disk groups that we configured Now we will configure an ESS Fibre Channel port 4 1 7 Configuring an ESS Fibre Channel port Pressing the Configure Host Adapter Ports button takes us to the Configure Host Adapter Ports panel as shown in Figure 73 96 Planning and Implementing an IBM SAN Modify Host Systems Configure Host Adapter Ports Configure Disk Groups A Modi Volume assignments Figure 73 Entry point for configuring ESS FC ports Again the naming convention in the ESS specialist can be a little misleading The ESS has built in Fibre Channel adapters which when used in a host system would be host adapters An ESS in its simplest form is a Fibre Channel storage device not a host We will refer to these as an ESS Fibre Channel adapter to avoid any confusion The Configure Host Adapter Ports panel is shown in Figure 74 Configure Host Adapter Ports i Host Adapter Port By Adapter 1 Pot A SCSD 7 Second Bus Connection optional e o SCSI Bus Co tion IBMI_RS 6000 Available Hosis Unrelated Host or Device Map of SCSI IDs on Bus jo 16 Volumes Attached to SCSIIDO fi 16 Volumes Attached to SCSI ID 1 je IF 16 Volumes Attached to SCSI ID 2 E J 7 Volumes Attached to SCSI ID 3 7 Volumes Attached to SCSI ID 4 s 7 Volumes Attached to SCSI ID 5 xl z Perform Configuration Update Cancel Configuration Update Figure 74 Configuring ESS interface ports SCSI
331. stems using Fibre Channel host adapters and drivers This allows you to protect your investment in SSA disk while being able to create and build a SAN infrastructure The Vicom Fibre Channel SLIC Router replicates data across or within serial disk systems simultaneously mirroring two or three copies of data without host involvement With global hot disk sparing data is automatically rebuilt if a mirrored disk fails In this way the Vicom Fibre Channel SLIC Router improves performance and data availability while simplifying storage operations The Instant Copy function can create a separately addressable copy of mirrored data that can be used for tape backup After the backup has completed data is resynchronized with the primary copy The Vicom Fibre Channel SLIC Router also can create composite drives by concatenating up to 16 physical disks Using these functions physical drives become members of larger or more complex logical drives A diagram to depict a single host to router configuration is shown below in Figure 252 Server Fibre Channel m SLIC Router A2 Al z SSA Disk Loop is mmea Figure 252 SLIC Router with a single host Copyright IBM Corp 2000 299 In the following sections we cover these topics e e e Installing the SLIC Router on page 300 SLIC Manager software on page 303 Using SLIC Manager on page 310 Composi
332. switch 291 249 VPS Access window with switch and two hosts 292 250 Setting LUN access for the host FIBRE1 293 251 McDATA Director connection to a Gateway 20 005 295 252 SLIC Router with a single host 0 0 c ee eee eee 299 253 SW1dip SWitCheS ares ween aie el lenea lela a mbna eke BRE 301 254 SLIC Manager acceSS 0 0 cece tee 304 255 Sample configuration fille 2 00 eee 306 Planning and Implementing an IBM SAN 256 Edited configuration file 0 2 0 0 0 cece ete 307 257 SLIC daemon start up in Windows NT 00 0a ee eee 309 258 SLIC connection WindoW saasaa aeaee 309 259 SLIC Manager title bar 0 0 anaana 310 260 Control Center window 000 eee 310 261 Disk drive properties 0 0 cece eee 311 262 SLIC Router properties 0 0 0 ee tee 312 263 Setting the Router to master 0 0 0 eee eee 312 264 Selecting SignOn drive dialog box 0 cee eee 313 265 Composite Drive Member Selection window 045 314 266 Creating Composite drive from available drives 315 267 Assigning Composite Drive Properties window 4 316 268 Completing the Composite Drive setup 0 e eee eee 317 269 Control Center with Composite drive saaa aaaea aaa 318 270 Composite Drive Properties 0 0 c eee eee 319 27
333. switch This is initiated by host device driver software like IBM s Subsystem Device Driver formerly Data Path Optimizer which recognizes the failure of the path and fails over to the alternate path on the redundant switch This configuration illustrated in Figure 20 also allows for maintenance or repair actions to be made on one SAN while the other stays in operation More detailed examples of redundant fabric designs are described in Designing an IBM Storage Area Network SG24 5758 Planning and Implementing an IBM SAN Redundant Fabric Switch i Switch Disk storage Figure 20 Redundant fabrics 2 3 2 6 Fabric backbone Building on the concept of departmental SAN islands each designed with fault tolerant fabrics it is possible to link such islands together into an enterprise SAN SAN continent perhaps by providing a fault tolerant backbone of inter linked switches This concept is shown in Figure 21 The fabric design shown here provides a total of 186 nodes in total with up to 150 nodes at the department level and 36 nodes in the backbone fabric The backbone SAN can be used for shared devices such as a tape library which can be accessed from any node in the enterprise fabric In this manner you can begin to use building block SANIets and grow these to larger resilient cascaded fabrics with multiple alternate paths through the network Such designs can be easily
334. t 20 00 00 20 8b 00 52 5e Figure 139 Zone Alias create example 6 4 2 4 Software zoning As we have stated previously one disadvantage of hardware zoning is that devices have to be connected to a specific port If the device is connected to a different port which is outside of the zone the whole zoning configuration will become unusable The members of software zoning can be defined as either a node WWN ora port WWN They are unique With software zoning your zoning configuration is not affected if the device is moved off a specific port The WWNN or WWPN adds the consistency that port zoning does not offer This is because the zone will follow the WWN Software zoning also allows for the creation of symbolic names for the zone members An example of software zoning was illustrated when we defined the members of our NTzone step 4 on page 165 In our example 40 05 07 63 00 c0 0c 5d is a software zoning member Figure 140 shows this NTzone again 170 Planning and Implementing an IBM SAN Zone Administration Netscape Zone Alias Settings Zone Settings Zone Config Settings Zone Name NTzone Create Zone Delete Zone Rename Zone Member Selection List NTzone Members port_o Shor Add Member gt po Te if AL_PA s Apply Cancel Done Figure 140 Software zoning representation This process can be easily repeated for as many WWNN or WWPN that you require to be in a zone 6 5 Cascad
335. t it is clear that all devices cannot transfer data at the same time SCSI uses an arbitration protocol to determine which device can gain access to the bus Arbitration occurs before and after every data transfer on the bus While arbitration takes place no data movement can occur This represents an additional overhead which reduces bandwidth utilization substantially reducing the effective data rate achievable on the bus Actual rates are typically less than 50 of the rated speed of the SCSI bus In addition to being a physical transport SCSI is also a protocol which specifies commands and controls for sending blocks of data between the host and the attached devices SCSI commands are issued by the host operating system in response to user requests for data Some operating systems for example Windows NT treat all attached peripherals as SCSI devices and issue SCSI commands to deal with all read and write operations It is clear that the physical parallel SCSI bus architecture has a number of significant speed distance and availability limitations which make it increasingly less suitable for many applications in today s networked IT infrastructure However since the SCSI protocol is deeply embedded in the way that commonly encountered operating systems handle user requests for data it would be a major inhibitor to progress if we were obliged to move to new protocols 1 3 Storage network solutions Today s enterprise IT p
336. t hosts Use same ID Lun in source and target Unassign selected volume s from target hosts Figure 87 Selected volumes 106 The Action box is now accessible as shown in Figure 88 Action Assign selected volure s to target hosts Use same ID Lun in source and target Unassign selected volume s from target hosts Figure 88 Accessible Action box We Assign selected volume s to target hosts by selecting the radio button shown in Figure 89 From here we can also Unassign selected volume s from target hosts Use same ID Lun in source and target Unassign selected volume s from target hosts Figure 89 Checkbox to assign volumes Selecting this allows us to choose the Target Hosts Fibre Channel port from those displayed in the Target Hosts field as shown in Figure 90 Planning and Implementing an IBM SAN Figure 90 Field for target host Fibre Channel ports We choose the host Fibre Channel port named itso by highlighting its name To apply the changes we press the Perform Configuration Update button The progress bar shows the configuration update is taking place as shown in Figure 91 N Performing Configuration Update x Modifying Volume Set Configuration 15 Estimated Remaining Time 0 seconds i Unsigned Java Applet Window Figure 91 Applying volume assignment changes If successful we get the message shown in Figure 92 N Yolume Assignments Ea 2106 The speci
337. t remotely Now we have to install the EFC Manager on a remote workstation This can be a PC running Windows NT a SUN ora HP workstation In our case this will be a PC running Windows NT To use the EFC Manager from a remote workstation we need to download and install the code on our workstation The code is downloaded from within the EFC server In our case it is an already installed EFC server of another ED 5000 The download and installation of the EFC Manager is done using a Web and Java based installation procedure All we need is a Web browser and we will use Netscape Navigator In the Uniform Resource Locator field URL of the Navigator we point to the address of the EFC server 9 113 24 119 to access the initial page on the EFC server 7 2 4 1 Changing the Netscape Navigator preferences If you can access the page but only the background is visible it is likely that you may have to change the preferences of the Netscape Navigator To accomplish this go to the Preferences by selecting Edit gt Preferences On the Advanced tab uncheck the check box Enable Style Sheets as shown in Figure 155 Planning and Implementing an IBM SAN Preferences Figure 155 Netscape preferences disabling style sheets 7 2 4 2 Downloading and installing the EFC Manager This takes us to the start page for the remote EFC Manager installation as shown in Figure 156 where we can choose the operating system of our remote workstation
338. tabase does not require any check marks to be set Because the WWN of the host is known and the switch WWPN is known LUN access is accomplished through the switch as if it was transparent m Note LUN access is not only affected by the VPS and Zoning with the SAN Data Gateway there is also Zoning within the IBM SAN Fibre Channel Switch Be aware of the zone settings within the IBM SAN Fibre Channel Switch Chapter 8 Implementing the SAN Data Gateway 293 8 6 1 6 ESS considerations As you configure and allow access from host systems to volumes that are in the ESS through the Gateway you must consider how the volumes were created within the ESS Volumes in the ESS are assigned to a particular SCSI port in the ESS As they are assigned you also specify the host type that will access these volumes Through the SAN Data Gateway it is possible to assign volumes to a specific operating system in the ESS but then have a host with a different operating system access to these volumes As an example look at Figure 248 on page 291 There we have a Windows NT and an AIX host The ESS was configured and the Gateway attached so that SCSI 2 was assigned Windows NT volumes and SCSI 3 was assigned AIX volumes It would be possible in the SAN Data Gateway to allow the NT host to access a LUN on SCSI 3 and the AIX host access to volumes on SCSI 2 Here is where the device map would again be helpful in determining what Gateway assigned L
339. tch IP address 144 Setting the Ethernet Address 254 setting the IP address 254 share 103 Shared 103 109 shared bus 8 Short Wave Length SWL cable 121 short wave ports 252 signal interference 8 SignOn Drive 313 Simple Name Server SNS 159 172 Simple Network Management Protocol 68 skew 5 15 SLIC Manager 340 SLIC Manager daemon 303 308 340 SLIC Manager software 303 SLIC Zone 305 SLIC zone 313 340 Slotted Loop 15 Small Computer Systems Interface 4 SNMP 68 142 SNS 159 software zoning 159 170 sort criteria 105 speeds of 200MB s 50 SRC 44 SSA 16 299 311 338 Starting the SLIC Manager 309 Startup sequence 255 status lights 141 Storage Allocation 70 75 368 Planning and Implementing an IBM SAN Storage Area Networks 1 storage consolidation 33 Storage Server Attributes 98 StorWatch 24 StorWatch Enterprise Storage Server Expert 81 StorWatch Enterprise Storage Server Specialist 63 StorWatch Fibre Channel Switch Specialist 142 151 StorWatch SAN Data Gateway Specialist 256 259 Strategic Research Corporation 44 Subnet mask 188 Subsystem Device Driver 42 104 296 Switch 36 switch 139 pre installation 143 switch priority 243 Switch registration 285 switchdisable 177 Switched Fabric 15 switched fabric 14 34 36 switchenable 177 switches 10 symbolic names 159 system bus 32 system service parameters 177 T tag 4 tape pooling 20 Target Hosts 105 TCP IP 9 16 Telnet 122 142 144 162 177 2
340. tches will recognize the connection and automatically configure the G_Ports to be used as E_Ports However there are some configuration options that need to be set up or reviewed before connecting the switches 7 5 2 1 Setting the domain ID Each switch itself is recognized in the fabric as a domain and is identified with a domain ID Every domain ID in the fabric must be unique ranging from 1 to 31 To view and to change the domain ID we go to the Configure Operating Parameters window from within the Product Manager of the specific switch Then we select Configure gt Operating Parameters In the next window shown in Figure 203 we can change the preferred domain ID and other Fibre Channel parameters for the director for instance the Switch Priority Planning and Implementing an IBM SAN i Configure Operating Parameters Figure 203 Setting domain ID 7 5 2 2 Setting the switch priority In every multiswitch fabric one switch will act with responsibility for the domain address manager functionality This principal switch controls the allocation and distribution of the domain IDs for all connected switches in the fabric The principal switch is the one with the highest switch priority ranging from 1 highest priority to 255 lowest priority If switches share the same value for the priority the one with the lowest number in the WWN becomes the principal switch To change the switch priority we also use the Configure Operat
341. te drive on page 314 Mirror drive on page 319 Instant Copy drive on page 324 Combining composite and mirroring on page 329 9 1 300 Installing the SLIC Router To install and establish a SLIC storage system the Router and all the disks to be used must be setup in a proper sequence During this sequence only a single SLIC Router must be used to do the configuration Once configured other SLIC Routers can be added to the loop This sequence is described below Note For information and a description to understand the LED codes that will be discussed please refer to the SLIC Router Installation and User s Guide 310 605759 1 Power on SSA disks The SSA disks must be powered on to ensure that all disks spin up and pass the power sequence Make sure all drive LEDs are on solid to indicate a functioning device Any faulty or suspect disk drive should be replaced Clear Router Node Mapping To begin the Router installation the first step is to clear the Node Mapping table This is done by shorting the SSA ports on the rear of the router Plug in an SSA cable from port A1 to A2 on the back of the Router and power it on Clearing the table will take only seconds and when completed the Status LED on the front of the Router will flash a code 060 The router is now powered off the SSA shorting cable is removed and the SSA disks are attached Connect SSA disks to Router All the disks to
342. te on a Private Loop such as HP servers can be supported on the IBM 2109 using a feature known as QuickLoop QL The switch can be set up to create a logical private loop with the storage assigned to that server The whole switch can operate in QL mode or individual ports can be configured as QL In this way the IBM 2109 can be used instead of a hub to support such servers The IBM 2109 can also be cascaded with the IBM 3534 1RU Fibre Channel Managed Hub Therefore the switch can be used in a number of ways for cost availability and performance to satisfy differing SAN application and user group needs within the enterprise 2 3 1 4 When will you use a director You would select the McDATA ED 5000 Fibre Channel Director for high availability applications requiring extensive fault tolerance within the switch and high port count to support multiple node attachments and high switch bandwidth These applications might typically be found in large data centers supporting large numbers of heterogeneous open systems servers The McDATA ED 5000 is based on the design of the IBM ESCON Director which is widely used in S 390 Data Centers for core mission critical applications The McDATA ED 5000 Director will support cascading of directors using E Ports but it does not provide FL Port connectivity to arbitrated loops Also due to differences in implementation of name server and zoning techniques the director is incompatible with the IBM 2109 Fibre
343. tecture is extremely flexible in its potential application Fibre Channel transport layers are protocol independent enabling the transmission of multiple protocols It is possible therefore to transport storage I O protocols and commands such as SCSI 3 Fibre Channel Protocol or FCP the most common implementation today ESCON FICON SSA and HIPPI Network packets may also be sent using messaging protocols for instance TCP IP or Net BIOS over the same physical interface using the same adapters cables switches and other infrastructure hardware Theoretically then multiple protocols can move concurrently over the same fabric This capability is not in common use today and in any case currently excludes concurrent FICON transport Most Fibre Channel SAN installations today only use a single protocol Using a credit based flow control methodology Fibre Channel is able to deliver data as fast as the destination device buffer is able to receive it And low transmission overheads enable high sustained utilization rates without loss of data Therefore Fibre Channel combines the best characteristics of traditional I O channels with those of computer networks e High performance for large data transfers by using simple transport protocols and extensive hardware assists e Serial data transmission e A physical interface with a low error rate definition e Reliable transmission of data with the ability to guarantee or confirm error free
344. ted to one or more of the newly created logical drives This logical drive can now be used as the SignOn drive Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 313 and the previous drive is now available to be configured without any problems 9 4 Composite drive A Composite drive is a large drive that consists of two or more smaller drives The capacity of the composite drive is an aggregate of the capacities of all the smaller drives that are used to comprise this one large drive 9 4 1 Creating a composite drive To create a Composite drive from the SLIC Manager select Tools gt Composite Drive Setup Wizard A dialog box Composite Drive List will appear Currently the list will be blank because there are no Composite drives created Once there are Composite drives created you will see a list of the drives Click on the Next button and you will see the Members Selection window as shown in Figure 265 Composite Drive Setup Composite Drive Members Selection Available Drive Pool Tooo00LO00 35 68229 T00001 L001 35E22D0B T00002 L002 35E69318 TO0003 L003 35E6848D TO0004 L004 35E22D6C Ea TO0005S L005 35E6840E TOOOD6 L006 35E68462 T00007 LOO 35E6844E Add gt gt Composite Drive Members lt lt Remove iik Reset Drive Capacity MB Capacity MB T Ea lt Back Cancel Figure 265 Composite Drive Member Selection
345. ter Fibre Channel Subnetmask None press Enter 10 Gateway Address Current Gateway address or None enter new Gateway address or press Enter This is the Gateway address the system administrator provided or if none is required press Enter 11 lpaddress admin gt logout This ends the Serial port session You have completed the installation of the 3534 Managed Hub by using the serial port Remove the cable from the serial port connector 5 4 1 3 Downloading Firmware The hub is shipped with the latest level of code firmware available However new code is periodically released that you can easily download to the hub This task requires that you save data and executable software to your server The latest code can be obtained from the IBM SAN Fibre Channel Managed Hub 35341RU Web site http www ibm com storage fchub 5 5 Zoning Zoning is used to set up barriers between different operating environments to deploy logical fabric subsets by creating defined user groups or to create test or maintenance areas or both which are separated within the fabric Zoning gives you the flexibility to manage a Storage Area Network to meet different user groups objectives Chapter 5 Implementing an IBM managed and unmanaged hub 127 Zoning components are discussed in greater depth in 6 3 1 The role of zoning in a SAN on page 155 where the same principles apply The way in which to define zones zone members aliases an
346. ter The copy drive that was created above can now be seen in the Mirror Drive Instant Copy window as shown in Figure 278 326 Planning and Implementing an IBM SAN Control Center T33282 ITSOm PUBLIC T33283 copyl PUBLIC IC T330Z5 ITSOL TOOOOO 35E68229 TOOOO 35E684AE Figure 278 Control Center with Instant Copy Drive Notice that in the information provided for the copy drive there is an Ic included to distinguish between mirror drives and copy drives within this window 9 6 3 Adding an Instant Copy Drive to a mirror To add or detach the copy drive from a mirror you select and highlight the mirror drive and then click on the Properties button The Mirror Drive Properties window opens as shown in Figure 275 on page 324 Select the Add Member button and the Add Mirror Member window opens as shown in Figure 279 Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 327 328 Add Mirror Member TOOO00 LOOO 35E68229 TOOOO LOO 35E684AE T33025 ITsol T33283 copyl Figure 279 Add Mirror Member display Select and highlight the copy drive from the Available Drive Pool window click on the Add Member button and the name of the copy drive will appear in the New Mirror Member window This is shown in Figure 280 Add Mirror Member TOOOOO LOOO 35868229 TOOOO LOO 35E684AE T33025 ITS01 T33283 copyl Figure 280 Adding drive members to a mirro
347. ter Ports Configure Disk Groups Modify Volume Aviganend Figure 45 Open System Storage panel Here we can see the Hosts Systems table and the Assigned Volumes table We will see shortly that those two tables are showing the defined host FC adapter ports and to a selected port the assigned volumes 4 1 4 Defining a new host FC port with its WWPN We pressed the Modify Host Systems button as shown in Figure 46 Modify Host Systems Configure Host Adapter Ports Configure Disk Groups Modify Volume Assignments Figure 46 Entry to the Modify Hosts Systems panel We are taken to the Modify Host Systems panel as shown in Figure 47 This panel consists of the Host Attributes entry fields and The Host Systems List table The Host Systems List represents each hosts Fibre Channel port If a host has two Fibre Channel adapter cards this appears as two hosts 80 Planning and Implementing an IBM SAN Modify Host Systems Host Atiribuies Nickname Pey arso Host Type rc Server Win NT 4 0 or higher gt Host Attachment es i Fibre Charmel attached Hosiname IP Address Host Systems List Nidmame Host Type de INT kevin PC Server Win NT dell 2NT kevin PC Server Win NT Imdson l CIB SVG 7085100 PC Server Win NT es EM RS 6000 bommag 2 CI RSVGOOO komag3 CIB RS VGOOO m TEM RS 6000 World Wide Port Name 200000E08B00525E cote som list of novm WWPHs TEM R576000 pyratexd PC Server WinNT _ 4 a gt
348. terfaces 14 pseudo host 99 public loop 131 Q Qlogic 112 QoC 44 QoS 44 Quality of Connection 44 Quality of Service 44 Quick Initialize 322 Quick Loop 131 QuickLoop 37 111 R RAID 1 32 85 RAS 28 reboot 177 Redundant fabrics 42 reliability 28 remote notification 187 remote service support 186 remote support 69 remote workstation 196 Remove logical drive 334 repeaters 16 Re Scan SCSI Bus 271 resource allocation time out value R_A_TOV 243 RJ 45 140 Router 35 Router config file 305 Router LED codes 300 Router Node Mapping 300 Router power up sequence 302 Router Properties 311 Router Subsystem Diagnostic test 301 routers 10 S SAN 8 SAN Data Gateway 251 SAN Data Gateway Router 35 SAN Fibre Channel Switch 36 SAN islands 33 SANergy 20 SANlets 33 saved configuration 160 Scalability 29 scalability 28 SCSI 4 251 255 263 269 296 arbitration protocol 8 commands 8 protocol 8 unused ports 8 SCSI adapters 76 SCSI attached hosts 78 SCSI Channel 267 SCSI commands 8 SCSI device map 271 SCSI distance limitations 6 367 SCSI to LUN map 280 SCSI 3 16 SCSI ESCON 63 scsiRescan 271 SDD 104 296 security 28 155 159 segmentation 155 serial port 140 serial port settings 145 serial terminal emulator 144 Serial transfer 15 Server 257 server clustering 22 server free datamovement 21 Service port 252 serviceability 28 setting a Gateway address 254 setting a Subnetmask 254 setting swi
349. that you are replacing an existing infrastructure for storage You may even be planning to change or upgrade an existing SAN implementation So before selecting a design for Planning and Implementing an IBM SAN the new SAN it makes good sense to fully understand what it is that is being replaced The current storage configuration LAN or SAN network structure application uses traffic loads peak periods and performance as well as current constraints are all relevant information in determining realistic goals for the SAN This information will also help you to determine what if any of the existing components can be used in a new topology and what will be involved in migrating from today s environment to the new one 2 2 1 Use of existing fiber In many cases you may already have fiber optic cables laid in your organization IT budget holders will want to know if you can use the existing cabling This is discussed in greater depth in Designing an IBM Storage Area Network SG24 5758 If the existing cabling has been laid for some time the answer may well be that the high speeds and accuracy required of Fibre Channel requires new cable investments It is possible to test if installed fiber meets the necessary quality but this can also be a costly exercise If recent fiber cable has been laid you may need to decide what extensions need to be added to the configuration 2 2 2 Application traffic characteristics Before selecting a SAN top
350. the Router To edit this option it requires that a drive on the SSA loop has been recognized by the host and that the drive has been formatted In the example above a Windows NT host was used The Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 307 308 Windows NT Disk Administrator was used for the host to recognize the drives the first drive assigned the next drive letter F and it was formatted The file naming for a SLIC Zone depends on the type of operating system running For Windows NT the naming is lt drive letter gt IBM7190 SFA You can enter in many SLIC Zones but only one is required to get access at the beginning After the other drives have been configured as mirrors or composite drives then SLIC Zones can be created for these drives if desired Including many SLIC zones in the path statement will allow the Manager to access a zone on another drive This is helpful to protect against when a drive fails and that drive has a SLIC zone defined to it If the Manager cannot access the first SLIC zone it would try the next zone in the order it was entered in the path statement For the naming conventions used on other operating systems to create a SLIC Zone refer to the SLIC Manager Installation and User Guide 310 605807 The userlogfile will define a file with which you can view logged events The QueryChangelnterval sets the time in seconds that the daemon will poll the Router The recommended time set he
351. the SAN Data Gateway power on the unit and connect a service terminal to the 9 pin Service port located on the rear right hand side of the unit A PC running a terminal emulation program such as Windows HyperTerminal or NETTERM can be used The settings of the Service port are e 19200 baud e 8 data bits e No parity e 1 stop bit e X on X off flow control e VT 100 compatible terminal Planning and Implementing an IBM SAN If a PC with terminal emulation is used a 9 pin female to 9 pin female Null modem cable is required and is provided with the unit Once connected power on the SAN Data Gateway and the start up messages will appear and scroll across the window When the power on sequence has completed a prompt Gateway gt appears on the window as shown in Figure 214 fra sdg HyperT erminal ojx File Edit View Call Transfer Help value 1 Oxfffffffft a esSrveInit value 0 0x0 amemInit Testing DDF PCI 0 DMA PageX Testing DDF PCI 1 DMA PageX Test PASSED Amem interrupt handler installed value 0 0x0 scsintInit SAN Gateway Version 0340 Built Dee 23 1999 15 47 13 Gateway Oxclfce690 tShell VPS Feature NOT Enabled Data Mover is Disabled SCSI 3 DE Terminated SCSI 1 DE Terminated SCSI 4 DE Terminated SCSI 2 DE Terminated value 4444 Oxl1l5e lt Done executing startup script ffs0 sna re Service Port Reflected Signal Test Starting shell Gateway gt Fi
352. the testing will stop and the Router s Status LED will flash a diagnostic code A code map with a description of the errors can be found in the SLIC Router Installation and User s Guide 310 605759 5 Assign Fibre Channel target With the Router powered off you can now assign a Fibre Channel target ID number to the Router Any number can be selected however this number must be a unique ID No other device can have the same Fibre Channel target ID once it is set on the Router This is done by setting selected dip switches in SW1 to the down position The switch is set up in binary notation a switch that is down represents a 1 and a switch up represents a 0 Figure 253 shows the switch numbers and their corresponding value 128 64 32 16 8 4 2 fo Value 7 6 5 4 3 2 1 O Switen Number Figure 253 SW1 dip switches Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 301 For example by setting switch 2 and 0 down a value of 5 is represented All other switches would be in the up position By setting a switch down its value is added To change a number that was previously set on a Router power must be cycled to the Router for the change to take effect 6 Mapping physical drives Before powering on again SW2 must be set to mode 3 with switch 0 and 1 set down and all other switches set up The Router is powered on the Status LED will flash r
353. the underscore character _ Names are case sensitive for example Zone_1 and zone_1 are different zones Note that spaces are not allowed Every zone has a member list consisting of one or more members empty zones are not allowed The maximum number of zones and the maximum number of members in a zone are constrained by memory usage Since these limits are far larger than the number of devices connected to a fabric they are effectively unlimited Zone definitions are persistent That is the definition remains in effect across reboots and power cycles until it is deleted or changed A device may be a member of multiple zones 6 3 3 Methods of implementing zones Zoning can be implemented in two ways e Hardware zoning e Software zoning 6 3 3 1 Hardware zoning Hardware zoning is based on the physical fabric port number The members of a zone are physical ports on the fabric switch It can be implemented in the following configurations e One to one e One to many e Many to many A single port can also belong to multiple zones We show an example of hardware zoning in Figure 132 Chapter 6 Implementing an IBM SAN Fibre Channel Switch 157 158 p E Figure 132 Hardware zoning A physical fabric port number notation is specified as a pair of decimal numbers where is the switch number domain ID p is the switch port number For example 2 12 specifies port 12 on switch number 2 When a zone
354. timity ITSO_1 i erring Chaster 1 Loop B ne 00 pee i 2 Vol 104 zr FCA13 Device Adapter Pair 1 Open System 096 0 GB RAD Amay Fibre Charmel ii ee Chaster 1 Loop B D 00 LUN 0002 2 Vol 104 ai FCAI13 Device Adapter Pair 1 Open System 000 5 GB RAID Array Bay 1 Card 1 nd Chaster 1 Loop A SCSI Port A y2 Vol a ID 06 LUN 0001 Figure 94 To Netfinity_ITSO_1 assigned volume However if we use the back button of our browser we return to the Open System Storage panel as shown in Figure 95 108 Planning and Implementing an IBM SAN Open oe Storage fetfinity ITSO_1 PC Server NT 4 0 200000ENSB00525D a Imm or hig amyl femrsisooo rc e a pomag3 emerso re fno000000cp20BOBI i O bmg ie si6o00 c fooooooocsaoBssBs i Assigned Volumes Total 4 volumes Volume Vol Type Size Storage Type Location 067 FCA13 Open System 96 0 GB RAID Armay Device Adapter Pair 1 LSS 010 Yes Chister 1 Loop B A 2 Vol 103 068 FCA13 Open System 96 0 GB RAD Amay Device Adapter Pair 1 LSS 010 Yes 1 Loop B y2 Wol 104 Modify Host Systems Configure Host Adapter Poris Configure Disk Groups Modify Volume Assignments Figure 95 Open System Storage panel with shared volumes If we look at the Shared field we can see that this volume is now shared 4 2 Related information For more information on the ESS see IBM Enterprise Storage Server Introduction and Planning Guide 2105 Models E10 E20 F10 and F20 G
355. ting leads of coaxial cable Twisted Pair A transmission media cable consisting of two insulated copper wires twisted around each other to reduce the induction thus interference from one wire to another The twists or lays are varied in length to reduce the potential for signal interference between pairs Several sets of twisted pair wires may be enclosed in a single cable This is the most common type of transmission media ULP Upper Level Protocols UTC Under The Covers a term used to characterize a subsystem in which a small number of hard drives are mounted inside a higher function unit The power and cooling are obtained from the system unit Connection is by parallel copper ribbon cable or pluggable backplane using IDE or SCSI protocols UTP Unshielded Twisted Pair Virtual Circuit A unidirectional path between two communicating N_Ports that permits fractional bandwidth WAN Wide Area Network A network which encompasses inter connectivity between devices over a wide geographic area A wide area network may be privately owned or rented but the term usually connotes the inclusion of public shared networks 362 Planning and Implementing an IBM SAN WDM Wave Division Multiplexing A technology that puts data from different sources together on an optical fiber with each signal carried on its own separate light wavelength Using WDM up to 80 and theoretically more separate wavelengths or channels of data can be
356. tions 5 3 1 The ITSO environment From our ITSO environment we show how to implement the IBM SAN Fibre Channel Managed Hub 35341RU 5 3 2 The hardware involved We used this hardware in our configuration e 1 IBM PC with 1 FC HBA from QLogic Chapter 5 Implementing an IBM managed and unmanaged hub 121 e 1 Fibre Channel RAID Storage server IBM 2102 F10 with native FC adapter e 1 IBM 3534 managed hub 7 fixed optic ports 0 hot pluggable GBIC port empty 5 3 3 The software involved We used this software in our configuration e Microsoft Windows NT 4 0 Service pack 5 e IBM 3534 microcode level 2 1 3 5 4 Installing the IBM SAN Fibre Channel Managed Hub 35341RU We recommend that you use a pre installation checklist An example of a pre installation checklist is detailed in the BM SAN Fibre Channel Managed Hub 3534 Service Guide SY27 7616 and the IBM SAN Fibre Channel Managed Hub 3534 User s Guide GC26 7391 The checklist ensures a successful installation and includes checks on the host operating system host bus adapter storage devices cables and network parameters the most important of which is the hub IP address 5 4 1 Setting the IP address The 3534 Managed Hub is shipped from the factory with a default IP address 10 77 77 77 This IP address is printed on the label on the top front edge of the 3534 Managed Hub This address is for the external Ethernet connection If you can use this default addres
357. to the next If you go past a field continue to press the lt button to wrap the cursor around and return to the desired spot e Pressing the button or the v button will either increment or decrement the current field Holding the button down will make this happen rapidly The numbers in the field will wrap to 0 from 255 or from 255 to 0 depending on if you are incrementing or decrementing This can help you get to the desired value quickly When you have all fields set to the desired value continue with step 14 14 Press the gt button and the switch will respond Accept Y N Pressing the lt button will indicate Yes Pressing the gt button will indicate No e If you say no the switch will again respond Gateway Address and you can restart the process e If you say yes the switch will respond Updating the Config Planning and Implementing an IBM SAN This will cause the new address to be stored After the switch has made the change it will again display Gateway Address e If no other address is to be changed Gateway Subnet etc you may simply stop here by pressing the lt button At this point you are done setting up the switch e If you need to set other switch addresses Domain press the button The switch will respond Domain 15 Press the gt button the switch will respond by Displaying the current Domain in the form xxx XXX XXX XXX To change the current Domain to the preferred address e
358. to reduce because of the shared bandwidth and arbitration overheads A maximum of two servers is advisable on a FC AL If one server fails and has to reboot causing a new Loop Initialization Primitive sequence LIP it will bring down the whole loop Availability is therefore also a serious consideration FC SW on the other hand scales performance as you add nodes This does not apply to inter switch links ISLs which do not add bandwidth between end nodes because ISLs reduce the number of end to end connections Secure zones and masked LUNs can be created so that only authorized servers can access specific information FC SW provides comprehensive flexible growth options but is more expensive at the outset 2 3 1 Assessing the components 34 IBM provides a hierarchy of interconnect options with which to build Fibre Channel SANs to suit differing application characteristics These can range from FC AL topologies for work groups to departmental switched fabrics and to highly available topologies based on cascaded switches or fault tolerant directors In addition bridge solutions allow for attachment of legacy SCSI and SSA devices This hierarchy is illustrated in Figure 15 Planning and Implementing an IBM SAN Full Range of Fibre Channel SAN Offerings Workgroups Departmental Data Center Interdepartmental Cascaded Fault Switches Tolerant WETE Switches Hubs ce and A Va an performa s quirements Unmanaged Hubs
359. tolerant Fibre Channel directors like the McDATA ED 5000 or a fully redundant meshed topology of switches like the IBM 2109 The distribution layer The distribution layer of the hierarchy would comprise fault resistant fabric components designed to deliver QoC Class 3 or Class 4 depending on the applications Good connectivity and performance would be prime considerations The access layer Here are the entry point nodes to the fabric comprising host bus adapters routers gateways hubs and switches that are appropriate to service the number of servers and storage devices supported on the fabric Chapter 2 SAN planning and design considerations 47 Hierarchical Design Distribution Figure 23 SAN hierarchical design This hierarchy is analogous to the telephone switching system Each user has access to the network using an individual node the telephone these link to local area switches which in turn link to central core switches which serve a large national and international network with very high bandwidth A similar hierarchy has been built to serve the Internet with end users linked to local Web servers which in
360. torage This includes costs of backup recovery space management performance management and disaster recovery planning Disk storage is often purchased from the processor vendor as an integral feature and it is difficult to establish if the price you pay per gigabyte is competitive compared to the market price of disk storage Disks and tape drives directly attached to one client or server cannot be used by other systems leading to inefficient use of hardware resources Organizations often find that they have to purchase more storage capacity even though free capacity is available but is attached to other platforms This is illustrated in Figure 2 Planning and Implementing an IBM SAN Plenty of free space available but distributed Servers are out of space Free space Allocated space RS6000_1 NT_1 NT_2 Sun_1 Sun_2 Figure 2 Inefficient use of available disk capacity attached to individual servers Additionally it is difficult to scale capacity and performance to meet rapidly changing requirements such as the explosive growth in e business applications Data stored on one system cannot readily be made available to other users except by creating duplicate copies and moving the copy to storage that is attached to another server Movement of large files of data may result in significant degradation of performance of the LAN WAN causing conflicts with
361. trator Operator e Maintenance e View The System Administrator right grants access to every control and configuration task that needs to be performed and can be viewed as the highest level of authority All new users initially have view rights and this cannot be removed For a table of user rights of product manager functions refer to the Enterprise Fabric Connectivity Manager User Manual 194 Planning and Implementing an IBM SAN To change the settings for a user for instance to change the password we go to Configure gt Users With the Modify button we get a window similar to the New window where we can change our password and the user rights This is shown in Figure 154 User Description Z o Administrator System Administrator itso T50 Jon Tate TA m oooO O ne a aaoo remote administrator System Administrator Product Administrator Operator Maintenance Figure 154 Modify users Once the new user has been defined we can login to the EFC server with the newly created user ID and password To be able to connect from a remote workstation the EFC Manager has to be installed on this workstation This is described in 7 2 4 Installing the EFC Manager on remote workstations on page 196 Chapter 7 Implementing the McDATA ED 5000 195 7 2 4 Installing the EFC Manager on remote workstations 196 So we know that the EFC manager is running properly and we have defined users that can access i
362. turn communicate with large scale high performance core servers 2 4 The next steps Now that you are ready to design your SAN there are many things to do 2 4 1 The planning team 48 You will need to bring together the people with the appropriate skills to plan and implement the SAN Who should be in the team depends on the scale of the project This might range from installing a simple point to point connection to solve a distance issue in one location to a large scale SAN comprising multiple meshed fabrics interconnected across a large campus or linking between several locations in order to serve a large organization In the first case the team may just be one or two storage administrators Planning and Implementing an IBM SAN In the enterprise wide case you will probably need to include a number of skills Since you are planning a network it makes only common sense to include staff who have knowledge of complex networks You will want to have representatives who know about the various platforms UNIX Windows NT Novell Netware AS 400 systems Numa Q and so on since there are differing system requirements and quirks which must be understood Also consider the databases and applications you will be supporting and include advisors for these Knowledge of fiber cabling and data center planning may be necessary You will certainly of course need strong storage planning and management skills that are appropriate to th
363. up Parameters x Startup Manual Manual Start Manual Manual De Manual Pause Manual Automatic Eoninue Manual Automatic i Startup Automatic _ Hw Profiles Cs Help tee _ Figure 257 SLIC daemon start up in Windows NT 9 2 3 Starting the SLIC Manager To start the Vicom SLIC Manager software select Start gt Programs gt Vicom gt Vicom SLIC Manager The software will load and a dialog box will appear In the box with the heading Hostname enter in the name or IP address of the host the daemon service is running Enter in the SLIC name you entered in when editing the 7190 cfg file An example is shown in Figure 258 Cancel i SDU Connect x Hostname fe 1 150 96 SLIC Name Find itso Connection History Figure 258 SLIC connection window Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 309 Select OK and the software will begin to communicate to the Router You will notice that the top title bar of your window will now include the host name and SLIC name as in Figure 259 SLIC Manager 1 0 9 1 150 96 itso Figure 259 SLIC Manager title bar 9 3 Using SLIC Manager You can now look to see that all communications are working properly by going to the toolbar and selecting Tools gt Control Center A dialog box will appear as shown in Figure 260 Tooooo TooooL TooooZ T00003 T00004 T0000S TANANA 3
364. ure 277 Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 325 Instant Copy Drive Setup Wizard Assigning Instant Copy Drive Properties Setup Summary Type Instant Copy Drive with Simple Member FC ID 33 Logical Drive ID 3 FC LUN E Target Number 733283 Drive Name copy 1l Capacity 8689 MB Initialization Method Primary Member z C DoNot Initialize Guck Initialize cmoa Figure 277 Instant Copy Drive Properties The properties that can be changed are the FC LUN and the Drive Name There will be a suggested LUN number in this field that can be accepted If not simply type in the desired LUN number A name can also be defined to the copy drive for easier identification with a limit of up to eight characters Click on the Next gt button and a dialog box appears to allow you to create another copy drive Select Yes if you would like to create another copy drive and the Instant Copy Drive List window opens and the steps described above can be repeated Select Finish when you have created all the desired copy drives Up to this point the configuration has been kept within the SLIC Manager software When the Finish button is selected the SLIC Manager will now communicate to the Router to complete the process and update the Router to control the drives 9 6 2 Instant copy drive properties You can go to the Control Center window by selecting Tools gt Control Cen
365. us to the New Product entry field where we have to fill in the IP address of the director that we want to add This is shown in Figure 164 New Product 9 113 25 63 Figure 164 Adding new ED 5000 with its IP address The ED 5000 was properly installed in the network previously and now the EFC server can communicate with it Therefore the new director appears as an icon in the left area of the main window as shown in Figure 165 We can also see the IP address of the ED 5000 A green circle indicates that the switch is up and running with no known problems Chapter 7 Implementing the McDATA ED 5000 205 McDATA Enterprise Fabric Connectivity Manager Figure 165 New ED 5000 icon 7 3 2 Assigning nicknames to WWN 206 As with IP addresses and the DNS life can be made easier by defining nicknames for WWN The names can be the DNS host name in the case of only one adapter in one host If there are more adapters in one device we recommend that the nickname consists of the hostname and some extension to distinguish between them This helps later when we have to identify devices for instance when we setup zoning This is especially true because the ESS has Emulex adapters built in We have other Emulex adapters in our SAN so it would be useful to distinguish between the ones in workstations and the ones found in the ESS For the hosts we use the nickname that we used in the ESS configuration For the ESS we chose as an exampl
366. utions in 1999 and this trend is accelerating in the year 2000 Analysts now estimate that industry revenue for network attached storage both SAN and NAS will grow rapidly during the next two years Indeed by the year 2003 external storage consulting groups estimate that SAN attached disk arrays will reach 48 of the revenue for externally attached disk arrays NAS is expected to reach 23 while disk arrays attached in the traditional manner directly to servers will account for only 29 This is a dramatic shift in the IT infrastructure in a very short time frame It is also no surprise that the main drivers for SAN solutions are coming from the open systems environment In 1999 for the first time industry revenues for open systems disk arrays UNIX Windows NT and so on are estimated to have overtaken revenues for S 390 attached arrays By the year 2003 Chapter 1 Introduction to Storage Area Networks 25 26 external storage consulting groups estimate that disk array revenues from the open environment will be approximately six times greater while S 390 array revenues will remain relatively flat IBM s own estimates are that by the year 2003 some 70 of all medium and large enterprises will install Fibre Channel SAN solutions to address varying business needs Stock markets around the world especially Wall Street are excited by the opportunities offered by the emerging Fibre Channel technology and this is reflected in soaring
367. valent program that does not infringe any of IBM s intellectual property rights may be used instead of the IBM product program or service Information in this book was developed in conjunction with use of the equipment specified and is limited in application to those specific hardware and software products and levels IBM may have patents or pending patent applications covering subject matter in this document The furnishing of this document does not give you any license to these patents You can send license inquiries in writing to the IBM Director of Licensing IBM Corporation North Castle Drive Armonk NY 10504 1785 Licensees of this program who wish to have information about it for the purpose of enabling i the exchange of information between independently created programs and other programs including this one and ii the mutual use of the information which has been exchanged should contact IBM Corporation Dept 600A Mail Drop 1329 Somers NY 10589 USA Such information may be available subject to appropriate terms and conditions including in some cases payment of a fee The information contained in this document has not been submitted to any formal IBM test and is distributed AS IS The use of this information or the implementation of any of these techniques is a customer responsibility and depends on the customer s ability to evaluate and integrate them into the customer s operational environment While each item may h
368. ver receives all traps from the agent and forwards them to the clients that are registered to receive them Client The client is a Java application that operates from any compatible computer as long as it has a TCP IP connection to the server One or more clients can connect to a server The client provides the user interface to allow the management and configuration of the SAN Data Gateway The server and client can be installed on to the same computer The StorWatch SAN Data Gateway Specialistsupports the following operating systems Windows 95 98 2000 and NT 4 0 with SP5 or later e AIX ver 4 3 3 or later e Solaris 2 6 or later 8 3 1 Installing StorWatch Specialist The Specialist software is not bundled with the SAN Data Gateway The Specialist software is downloaded using a Web browser by going to the IBM Storage Area Network Data Gateway Web site http www storage ibm com hardsoft products sangateway support forml htm This will take you to a registration window Enter the required information and select Submit Information A license agreement window is shown and once reviewed select I agree The Download Main Page window will load Then select the specific operating system platform Review the readme txt file for the latest information and instructions before installing This Web site also contains all the latest firmware for the SAN Data Gateway and supported host bus adapters The StorWatch SAN Data Gateway Specia
369. vice connections is actually less than the total number of ports The number of ports used as ISLs depends on the fabric configuration This is dictated by the amount of bandwidth required between switches and by the number of desired redundant links to maintain connectivity in the case of link or switch failures 6 5 4 Implementing cascading with the IBM 2109 S08 switch We will use our basic scenario with one host a 2109_ S08 switch and an ESS storage device and add a new switch to the configuration 6 5 4 1 Adding a new switch In 6 2 1 Installing the 2109 S08 switch on page 143 we have described the process to successfully implement a new switch A new switch is a switch that has not been previously connected to a fabric with zoning configured and which has had no zone configuration data entered into it To return a switch to new switch state you must use the cfgClear command on the switch that you are adding before connecting it to the fabric The cfgclear command must be used with caution as it removes all zone information from the fabric If you are unsure of the consequence of this do not issue the command We show an alternative method to add a previously configured switch in 6 5 4 2 Adding a switch that has been previously configured on page 176 To implement the new switch make sure you follow these steps 1 Bring in the new switch 2 Establish one or more paths to the existing switch by connecting the equ
370. ware Incorporated N is requesting additional privileges Granting the following is high risk tarting programs stored on your computer I Remember this decision Identity verified by VeriSign Class 3 CA Commercial Content Software Publisher Copyright 1999 200 Certificate Grant Deny Help DATA Enterprise Fabric Connectivity Manager Available Installers Platform includes Java YM Instructions xl a Se ele 7 Figure 157 Granting additional rights to the EFC Manager installation software Due to the fact that for security reasons a Java applet is not allowed to perform the tasks mentioned above this warning message appears Only allow Java programs to perform like this from trusted sources 200 Planning and Implementing an IBM SAN After this the button to start the InstallAnywhere Web Installer appears and we are able to start the installation as shown in Figure 158 e Web Installer Netscape ho Communicator Help N a 2 ee E Home Search Netscape Print Security Stop As Location http 9 113 24 119 EFC_ClientInstall install htm McDATA Enterprise Fabric Connectivity Manager Recommended Installation for Your Platform Start Installer for Windows 95 98 NT I Installer created with InstailAnywhere by Zero G Software inc Copyright 1998 www ZeroGcom Available Installers Platform includes Java VM Instructions Figure 158 Starting the installation of the EFC M
371. witch 165 166 Zone Administration Netscape Zone Alias Settings Zone Settings Zone Config Settings zone Name NTzone 7 Delete Zone Rename Zone Member Selection List NTzone Members E aSwitchPorts B Domain 1BM_2109_FC_Switd Add Member Remove Meme Add Other Apply Cancel Done Figure 136 NTzone creation 5 As soon as we defined a zone the system asked for zone members because an empty zone is not allowed We decided to opt for hardware zoning and allocated switch ports 0 and 1 to the NTzone However these members can be switch ports WWNs or aliases We show our hardware zoning example in Figure 137 You will also notice that in the member selection list that there has been a new domain added called Domain_2 This is because the dynamic environment which we were working in had at the time of screen capture another switch connected to our switch Domain_1 This is a good opportunity for us to point out that if a switch is cascaded then we have the ability to zone that into our environment too Planning and Implementing an IBM SAN Zone Administration Netscape Zone Alias Settings Zone Settings Zone Config Settings Zone Name NTzone Create Zone Delete Zone Rename Zone Member Selection List NTzone Members E aSwitchPorts E Domain_1 IBM_2109_FC Add Member i S4 40 05 07 63 00 c0 0c 5d lt Remove Member Ga Apply Cance
372. with multiple entry exit points ports that provides dynamic connection between any two of these points Switch Topology An interconnection structure in which any entry point can be dynamically connected to any exit point In a switch topology the available bandwidth is scalable T11 A technical committee of the National Committee for Information Technology Standards titled T11 I O Interfaces It is tasked with developing standards for moving data in and out of computers Tape Backup Making magnetic tape copies of hard disk and optical disc files for disaster recovery Tape Pooling A SAN solution in which tape resources are pooled and shared across multiple hosts rather than being dedicated to a specific host 361 TCP Transmission Control Protocol a reliable full duplex connection oriented end to end transport protocol running on top of IP TCP IP Transmission Control Protocol Internet Protocol a set of communications protocols that support peer to peer connectivity functions for both local and wide area networks Time Server A Fibre Channel defined service function that allows for the management of all timers used within a Fibre Channel system Topology An interconnection scheme that allows multiple Fibre Channel ports to communicate For example point to point Arbitrated Loop and switched fabric are all Fibre Channel topologies Twinax A transmission media cable consisting of two insulated central conduc
373. xit Figure 151 EFC Manager product view with no devices configured 7 2 3 Defining users on the EFC Manager We define users on the EFC Manager because we do not want the Administrator user ID to be used remotely so we create a new user and use that for remote access We can define up to 16 users for the EFC Manager but only a maximum of four can log on concurrently This includes the user of the EFC Manager running locally on the EFC server From the Product View we go to Configure gt Users on the button panel on the left side of the window as shown in Figure 152 Planning and Implementing an IBM SAN 6 McDATA Enterprise Fabric Connectivity Manager Figure 152 Configuring users We are presented with a list of the defined users and the options to add users modify existing users view the rights of a user and delete users We will add another user by pressing the New button and then specifying the name password and description of the new user Also this window is used to specify the rights that the new user should have This is shown in Figure 153 Chapter 7 Implementing the McDATA ED 5000 193 re x user Jooo Description o O Administrator System Administrator New User view System Administrator Operator Maintenance Figure 153 Defining new user To assign rights to the user we click on one of the Rights Available and the Add button The rights are e System Administrator e Product Adminis
374. y adding more Routers This is shown in Figure 289 Server Fibre Channel A1 A2 A1 A2 SSA Disks SSA Disks Figure 289 Increasing storage capacity The other option is to have each additional Router added to the same SSA loop Throughput to the SSA loop will increase because each Router can access the disks for multiple simultaneous operations This configuration is shown in Figure 290 Server Fibre Channel A1 A2 A1 A2 SSA Disks SSA Disks Figure 290 Increasing throughput 9 9 2 1 Installing additional Routers With the existing Router and storage system powered on Chapter 9 Implementing the Vicom Fibre Channel SLIC Router 339 The node map on the new Router must be cleared first Connect the Router to the existing system with the power off Set SW2 to mode 3 and set SW1 to an unique Fibre Channel target ID Power on the new Router a A O N When the Status LED on the new Router is on solid lit the install is complete When the new Router is powered on communication between the two Routers will occur to query and update the new Router The new Router will be added as a subordinate so that the first Router will maintain its master status Any configuration changes to the storage system is always done on the master As this is done the changes are also communicated to the subo
375. y another EFC Server Changing to the Port List View of the cascaded McDATA without any other device connected we see the G_Ports of our two ISLs have turned into E_Ports We also see the names of the ports we configured as shown in Figure 210 248 Planning and Implementing an IBM SAN Figure 210 Port List View with two E_Ports To finish our example we configure the zoning and connect our devices to the fabric as shown in Figure 202 on page 241 After activating the zone set our Zoning View now looks like that shown in Figure 211 Chapter 7 Implementing the McDATA ED 5000 249 gt McDATA Fabric Manager OF x Figure 211 Active Zone Set corresponding to Figure 202 on page 241 We successfully completed all the steps necessary to cascade a McDATA director 250 Planning and Implementing an IBM SAN Chapter 8 Implementing the SAN Data Gateway In this chapter we describe the steps involved in planning and implementing the IBM Storage Area Network Data Gateway 2108 G07 The SAN Data Gateway is a hardware solution to allow connection of Fibre Channel ready host systems to attach to SCSI storage systems A diagram to show a SAN Data Gateway configuration using a single host is shown in Figure 212 ESS Figure 212 SAN Data Gateway configuration 8 1 SAN Data Gateway The IBM Storage Area Network Data Gateway is an essential component of the SAN infrastructure It provides s

Download Pdf Manuals

image

Related Search

Related Contents

DVR-LE-A-Guia-Rapida-web (1)  QUICK REFERENCE  Hi-Copy DMX- 8611e User Guide 多媒体储存系统使用手册  ComponentsServiceManualv2002-10  KEF PICOFORTE 3  Panas。m。 取扱説明書  Tecra M5 - Toshiba Europe  Morse Manufacturing 195A-1-115 User's Manual  Belzona® 1391  ネットワークボード POA-MD27NET 取扱説明書  

Copyright © All rights reserved.
Failed to retrieve file