Home

PCI Hot Plug User`s Guide I/O devices edition

1. 68 3 Connect PCI cards The cfgadm command is executed with the configure option to Ap Id or the push button of the PCI bus slot is pushed Then extended PCI card is connected In addition a push button is valid only in multiple user mode Please confirm that the slot status is connected configured by using the cfgadm 1M command after the above mentioned procedure cfgadm c configure pcipsy15 COM01 PCI slot02 Return cfgadm pcipsy15 COM01 PCI slot02 Return Ap_Id Type Receptacle Occupant Condition pcipsy15 COM01 PCI slot02 pci pci hp connected configured ok 4 Start the hardware monitoring of the Machine Administration Please execute the following commands in order to update configuration information and restart the hardware monitoring of the Machine Administration usr sbin FJSVmadm postphp lt Return gt 5 Connect equipment to PCI cards The extended PCI card is connected with network equipment by the cable 6 Setup driver The driver configuration is added by following operations a The interface name is investigated in order to configure drivers and high layer products Please save the result of the prtcicl 1M command and obtain the difference information between current result and the result taken at 2 a Then the driver instance number for the extended PCI card is obtained In the following example since the instance number is 1 it can be determined that the interface na
2. 4 1 2 Replacement of PCI cards on redundant system When exchanging PCI cards with redundant configuration software it can exchange PCI cards without stopping applications This subsection describes replacement procedure of PCI cards with the following redundant configuration software SynfinityLink 2 2 PRIMECLUSTER GLS 4 0 or later The replacement and extension of PCI cards are supported only on Redundant Line Control Function of SynfinityLink Notes Because of the dynamic replacement if the environment of SynfinityLink or PRIMECLUSTER GLS has been modified system reboot is not required When using other redundant software please refer to manuals of each product l 66 Stop the hardware monitoring of the Machine Administration The hardware monitoring of the Machine Administration is stopped by using the following command usr sbin FJSVmadm prephp lt Return gt Specify the replacing PCI card The interfaces e g hmel which stop applications for replacement of PCI cards are specified from warning messages outputted the console Disconnect from redundant system In order to disconnect the PCI card for exchange from a redundant system please execute the following command with the interface name specified by procedure 2 Specify the replacing PCI card On the redundant system of standby mode when an online communication path is disconnected a standby communication path will be online comm
3. z a p E eis ai ie PCI Hot Plug User s Guide I O devices edition FUJITSU Preface Purpose This manual describes the procedure of exchanging or extending file PCI cards and network PCI cards using the PCI Hot Plug feature Intended Readers This manual is intended for the following readers System software developers and test engineers System support engineers System administrators who uses this function Hardware engineers who uses this function Organization This manual is organized as follows Chapter 1 PCI cards with support for PCI Hot Plug describes the lists of PCI cards supporting PCI Hot Plug Chapter 2 Overview of replacement and extension of PCI cards describes the outline of a procedure of replacement and extension of PCI cards Chapter 3 File devices describes the detailed procedure of PCI Hot Plug about file devices Chapter 4 Network devices describes the detailed procedure of PCI Hot Plug about network devices Chapter 5 Notes for PCI Hot Plug describes the notes when using PCI Hot Plug Notation Used The following shows the notation conventions used in this manual In this manual the Solaris TM Operating Environment is abbreviated to Solaris OE the Solaris TM 8 Operating Environment is abbreviated to Solaris 8 OE the Solaris TM 9 Operating Environment is abbreviated to Solaris 9 OE the Solaris TM 10 Operating Environment is abbreviated to Solaris 10 OE The
4. WARNING Ipfc0 INe Adapter Hardware Error WARNING pci 89 4000 fibre channel 2 sd 3 0 sd20 SCSI transport failed reason reset giving up Exec the following command and check the physical path name grep Ipfc etc path to inst Return pci 89 4000 fibre channel 92 0 Ipfc The second value is instance number of each Ipfc instance Determine the logical path name under dev dsk corresponding to the disk device sd20 connected to the target PCI card In the example below c2t3d0 is the logical path corresponding to sd20 Is 1 dev dsk grep pci 89 4000 scsi 2 sd 9 3 0 Return Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s0 gt devices pci 89 4000 scsi 2 sd 3 0 a Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s1 gt devices pci 89 4000 scsi 2 sd 3 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s2 gt devices pci 89 4000 scsi 2 sd 9 3 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s3 11 12 gt devices pci 89 4000 scsi 2 sd 3 0 d Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s4 gt devices pci 89 4000 scsi 2 sd 3 0 e Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s5 gt devices pci 89 4000 scsi 2 sd 3 0 f Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s6 gt devices pci 89 4000 scsi 2 sd 9 3 0 g Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s7 gt devices pci 89 4000 s
5. automap 1 set to 1 3 2 1 1 Extension of PCI cards on non redundant system Procedures to add PCI cards without making them redundant on a Solaris 8 OE system are explained here 1 Setup I O devices which will be added to system Pre define devices to be added with the following procedures according to the device type Extension using PCI Hot Plug cannot be done without definition so devices must be defined before the system goes into operation Disk Tape devices Solaris OE sd st This is an example where a disk device at target 2 lun 0 is pre defined If the device is a tape device re read sd in the following as st and perform the same operations a Confirm that target 2 lun 0 is defined in kernel drv sd conf If this definition does not exist add the definition If it is commented out remove the comments Disk device vi kernel drv sd conf Return name sd class scsi target 0 lun 0 name sd class scsi In case the definition does not exist target 2 lun 0 add or remove comments Tape device vi kernel drv st conf lt Return gt name st class scsi target 0 lun 0 name st class scsi In case the definition does not exist target 5 lun 0 add or remove comments b In case kernel drv sd conf was updated in 1 a reconfigure the kernel restart the system and then start system operation If it was not updated this procedure is not required touch reconfigu
6. efgadm pcipsy3 COM00 PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy3 COMO00 PCI slot02 pci pci hp connected configured ok c Please disconnect the PCI card by executing the cfgadn 1M command with Ap Id and confirm that the slot status is disconnected unconfigured cfgadm c disconnect pcipsy3 COM00 PCI slot02 Return cfgadm pcipsy3 COM00 PCI slot02 Return Ap_Id Type Receptacle Occupant Condition pcipsy3 COM00 PCI slot02 pci pci hp disconnected unconfigured unknown d To confirm the slot position at replacement operation blink the ALARM LED of Ap Id displayed in procedure 4 a cfgadm x led fault mode blink pcipsy3 COM00 PCI slot02 Return 5 Replace the PCI card The PCI card disconnected in procedure 4 Disconnect the PCI card is exchanged for a new PCI card Our customer 65 service works about this procedure Connect the PCI card The cfgadm command is executed with the configure option to Ap Id or the push button of the PCI bus slot is pushed Then a new PCI card is connected In addition a push button is valid only in multiple user mode Please confirm that the slot status is connected configured by using the cfgadm 1M command after the above mentioned procedure Notes Ifthe setup of driver conf 4 file exist please refer to 5 1 1 Network driver configuration changing cfgadm c configure pcipsy3 COM00 PCI
7. etc rc2 d S99hbanywareDisc start Return Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJSV madm postphp Return Preparation to recognize devices for Fibre Channel When the Host Affinity function is being used with Fibre Channel SN200 series Host Affinity must be configured for the device added for PCI Fibre Channel PW008FC3U PW008FC2U GP7B8FC10U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility WWPN zoning of SN200 series Host affinity of Array disk device ETERNUS3000 ETERNUS6000 GR700 800 series and binding setting for fjpfca conf sd conf of host can be added from SAN management window automatically See Operation guide or User s Manual of each product for details Before using this function the following procedure 8 a 1 and 8 a 2 except fcp bind target setting are necessary Moreover begin from procedure 8 a 3 after executing this function If you don t use SAN management function of the above products No procedure is necessary Go to step 8 for Fibre Channel Card PW028FC4 PW028FC5 No procedure is necessary Go to step 8 Note When changing Affinity configuration on SN200
8. devices pci 89 2000 fibre channel 1 hddv 3 0 e Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s5 gt J devices pci 89 2000 fibre channel 1 hddv 3 0 f Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s6 gt devices pci 89 2000 fibre channel 1 hddv 3 0 g Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s7 gt devices pci 89 2000 fibre channel 1 hddv 9 3 0 h 4 Start the format 1M command and configure labeling and partition slice information calculated in 9 b 3 to the logical path name of each extended LUN c Add configure of PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver No procedure is necessary Go to step 10 Fibre Channel driver PW008FCZU W008FC2X GP7B8FC1U GP7B8FC1X No procedure is necessary Go to step 10 Fibre Channel Card PW028FC4 PW028FC5 Bind the auto mapped target connected with the card and modify the entries of lpfc conf to match the requirements of the extended Fibre Channel Controller These procedures are reflected on next boot time In the following indicate to bind the auto mapped target with the card usr sbin Ipfc Iputil Return LightPulse Common Utility for Solaris SPARC Version 1 6a8 03 03 2004 Copyright c 2003 Emulex Network Systems Inc Emulex Fibre Channel Host Adapters Detected 2 Host Adapter 0 Ipfc11 is an LP9802 Ready Mode Host Adapter 1 Ipfc9 is an LP9802 Ready Mode MAIN MENU 1 List Adap
9. usr sbin FJSVpfca fc_config v Return Fibre Channel driver PW028FC4 PW028FC5 Configuration is not required Compare the output of prtpicl 1M command with the output saved at 4 b and find the instance number of the 39 40 added card Add device driver configuration Add device driver configuration with the following procedures according to the I O device type Disk Tape devices Solaris OE sd st This example is a case where a disk device of target 2 lun 0 is pre defined in procedure 1 and added in procedure 5 If the device is a tape device re read sd as st in this procedure and perform the same operation Execute the following command to make the sd driver set up the added disk device usr sbin devfsadm Return 2 After saving the execution result of the prtpicl 1M command take the difference between the result and the information obtained in 4 a and get the physical path name of additional disk device then get the logical path name under dev dsk In the following example c2t2d0 is the logical path name of the disk device added prtpicl v gt tmp prtpicl post Return Disk device diff tmp prtpicl pre tmp prtpicl post grep devfs path grep sd Return gt devfs path pci 89 4000 scsi 2 sd 92 0 Is 1 dev dsk grep pci 89 4000 scsi 2 sd 2 0 Return Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s0 gt devices pci 89 4000
10. 2 Execute the following command to make the hard disk driver set up the LUN usr sbin hddvadmdr lt Return gt 3 After saving execution result of the prtpicl command get the physical path name of added LUN by taking the difference between the result and the information obtained in 4 a and calculate the corresponding logical path name under dev dsk In the following example c3t3d0 is logical path name of added lun 0 The logical path name of lun 1 2 is set to c3t3d1 c3t3d2 respectively because cXtYdZmeans X PCI card number Y target number Z lun number prtpicl v gt tmp prtpicl post lt Return gt diff tmp prtpicl pre tmp prtpicl post grep devfs path grep hddv lt Return gt gt devfs path pci 89 2000 fibre channel 1 hddv 3 0 gt devfs path pci 89 2000 fibre channel 1 hddv 93 1 gt devfs path pci 89 2000 fibre channel 1 hddv 3 2 Is 1 dev dsk grep pci 89 2000 fibre channel 1 hddv 3 0 lt Return gt Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s0 gt devices pci 89 2000 fibre channel 1 hddv 3 0 a Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s1 gt devices pci 89 2000 fibre channel 1 hddv 3 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s2 gt devices pci 89 2000 fibre channel 1 hddv 3 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s3 gt devices pci 89 2000 fibre channel 1 hddv 9 3 0 d Irwxrwxrwx 1 root ro
11. 12 Patch 913294 03 or later is needed 13 Redundant software is not supported on Solaris 10 OE 14 In case of using Solaris 10 OE FUJITSU PCI ATM 1 3 or later is needed Chapter 2 Overview of replacement and extension of PCI cards This chapter describes the outline of a procedure of replacement and extension of PCI cards by using PCI Hot Plug 2 4 Replacement of PCI card This section describes outline to exchange PCI cards using PCI Hot Plug feature About software which makes PCI card redundantly please refer to each manual of the software 2 1 1 Replacement of PCI cards on non redundant system Before you exchange PCI cards without redundant software you must stop applications which using the PCI cards The replacement procedure of PCI cards is as follows Please refer to Chapter 3 File devices or Chapter 4 Network devices about detailed replacing procedure of PCI cards File devices procedures 1 Stop the hardware monitoring of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration 2 Stop the daemons If you use the Fibre Channel Card PW028FC4 PW028FCS stop the daemons 3 Specify the replacing PCI card Specify the devices and interfaces that are influenced by replacing PCI cards 4 Stop applications Stop services and high layer products that are influenced by replacing PCI cards 5 Disconnect the PCI card It changes into the state where the PCI card is exchangeabl
12. 89 4000 scsi 2 st 5 0 cbn Irwxrwxrwx 1 root root 41 Sep 20 22 53 1cn gt devices pci 89 4000 scsi 2 st 9 5 0 cn Irwxrwxrwx 1 root root 41 Sep 20 22 53 1h gt devices pci 89 4000 scsi Q 2 st 5 0 h Irwxrwxrwx 1 root root 41 Sep 20 22 53 1hb gt devices pci 89 4000 scsi 2 st 5 0 hb Irwxrwxrwx 1 root root 41 Sep 20 22 53 1hbn gt devices pci 89 4000 scsi 2 st 5 0 hbn Irwxrwxrwx 1 root root 41 Sep 20 22 53 Ihn gt devices pci 89 4000 scsi 2 st 5 0 hn 3 If the device is a disk device start the format 1M command and configure labeling and partition slice information of the logical path name of added disks If the device is a tape device this procedure is not needed Disk array device hard disk driver hddv This example is a case where a disk array device of target 3 lun 0 2 is pre defined in procedure 1 and added in procedure 5 1 There may be a change of status depending on the system operating state after pre definition in procedure 1 so check the LUN definition conflict between sd conf and hddv conf again In case of Check NG the definitions are conflicting so go back to procedure 1 e Check OK to procedure 9 b 2 opt FJSVhddv bin hddvchk conf lt Return gt kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 1 e opt FJS Vhddv bin hddvchk_conf lt Return gt NGID 3 2 Execute the following comm
13. Delete the swap device swap d dev dsk c2t3d0s4 Return 4 Disconnect the PCI card Disconnect the target PCI card with the following procedure a By using inst2comp 1M command determine the slot position of the PCI card called an attachment point identifier Ap Id from the interface name of the path connecting the target PCI card and I O devices determined in procedure 2 a glm2 Please refer to PCI Hot Plug User s Guide about the detail of inst2comp 1M command In this example the Ap Id is pcipsy3 C0M00 PCEfslot02 usr sbin F JS Vmadm inst2comp glm2 Return pcipsy3 COMOO0 PCTZslot02 b Specify Ap Id determined from 4 a as a parameter and confirm that the slot status of the PCI card to disconnect is connected configured cfgadm pcipsy3 COM00 PCI slot02 Return Ap Id Type Receptacle Occupant Condition pcipsy3 COMO00 PCI slot02 mult hp connected configured ok c After executing the command to disconnect the PCI card specifying the Ap Id from 4 a confirm that the slot status has changed to disconnected unconfigured cfgadm c disconnect pcipsy3 COM00 PCI slot02 Return cfgadm pcipsy3 COM00 PCI slot02 Return Ap Id Type Receptacle Occupant Condition pcipsy3 COM00 PCI slot02 unknown disconnected unconfigured unknown Note When error occurs during disconnect cfgadm command unusually fail with following message If cfgadm command fails execute the command once again cfgadm Compon
14. Please refer to the online manual etc about the details of the ifconfig 1M command Moreover when high layer programs which using the TCP IP interface needs to be stopped please stop each high layer program Please refer to each manual about the stop procedure of high layer programs 1 When using Ethernet devices usr sbin ifconfig hmel down Return usr sbin ifconfig hmel unplumb lt Return gt 2 When using ATM devices opt FJS Vatm bin atmunlink adapter number lt Return gt b When using FNA SNA communication protocol over Ethernet When not using the redundant configuration software it is necessary to stop all high layer applications that are using Netcompo FNA LAN and all interfaces of Netcompo FNA LAN For details please refer to the manual of Netcompo FNA LAN and high layer applications 4 Disconnect the PCI card a The Ap Id is displayed using the following command based on the interface e g hmel specified in Procedure 2 Specify the replacing PCI card The attachment pint identifier Ap Id is a value for the cfgadm 1M command specifying the slot position of the PCI card Please refer to PCI Hot Plug user s guide about the details of cfgadm 1M and Ap Id usr sbin FJSVmadm inst2comp hmel Return pcipsy3 COM00 PCI slot02 b Obtained Ap Id is specified as an argument of the cfgadm 1M command and it confirms that the slot status of the PCI card that disconnects is connected configured
15. Return Fibre Channel driver PW028FCA4 PW028FC5 Configuration is not required Compare the output of prtpicl 1M command with the output saved at 4 b and find the instance number of the added card Add device driver configuration Add device driver configuration with the following procedures according to the I O device type Disk Tape devices Solaris OE sd st This example is a case where a disk device of target 2 lun 0 is pre defined in procedure 1 and added in procedure 6 If the device is a tape device re read sd as st in this procedure and perform the same operation Execute the following command to make the sd driver set up the added disk device usr sbin devfsadm Return 2 After saving the execution result of the prtpicl 1M command take the difference between the result and the information obtained in 4 a and get the physical path name of additional disk device then get the logical path name under dev dsk In the following example c2t2d0 is the logical path name of the disk device added prtpicl v gt tmp prtpicl post Return Disk device diff tmp prtpicl pre tmp prtpicl post grep devfs path grep sd Return gt devfs path pci 89 4000 scsi Q2 sd 92 0 Is 1 dev dsk grep pci 89 4000 scsi 2 sd 9 2 0 Return Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s0 gt devices pci 89 4000 scsi 9 2 sd 9 2 0 a 23 Irwxrwxrw
16. Return vi etc vfstab Return dev FJS V mphd dsk mphd2s6 dev FJS Vmphd rdsk mphd256 mnt ufs 1 yes raw access operation Configure applications For details see the manual of each application swap device Add swap device To make valid at system start add an entry in the etc vfstab file swap a dev FJSV mphd dsk mphd2s4 Return vi etc vfstab Return dev FJS V mphd dsk mphd2s4 swap no swap l Return swapfile dev swaplo blocks free dev FJS V mphd dsk mphd2s4 32 164 16 788384 683680 11 Start applications Use the logical path name determined in 8 to start applications filesystem operation Mount filesystem mount F ufs dev F JS Vmphd dsk mphd2s6 mnt Return raw access operation Start applications For details see the manual of each application swap device This procedure is not required 3 2 2 3 Extension of PCI cards to redundant system The procedure to add PCI cards to a redundant system in order to extend redundancy on a Solaris 9 OE or later system is explained with an example with GR multipath driver GRMPD There is no additional pattern for Multipath Disk Control MPHD If multipath software products other than GRMPD are being used see the manual of each product No configuration is needed for applications when adding PCI cards to an existing redundant system 1 Stop the machine administration hardware
17. Return kernel drv hddv conf vi kernel drv hddv conf lt Return gt name hddv class scsi target 0 lun 0 name hddv class scsi target 0 lun 7 name hddv class scsi target 1 lun 0 name hddv class scsi target 1 lun 1 Remove comments of the LUN in use name hddv class scsi target 1 lun 2 name hddv class scsi target 3 lun 0 name hddv class scsi target 3 lunz1 Remove comments of extended LUN name hddv class scsi target 3 lun 2 name hddv class scsi target 3 lun 3 f Check the LUN definition overlap between sd conf and hddv conf by executing hddvchk_conf command The message example for Check NG means that target 3 is defined in both files and the procedures must be performed again from procedure 1 d The followings are examples of execution of the FJSVhddv package installation to opt The path of the hddvchk_conf command depends on the installed directory Check OK to procedure 1 g opt FJS Vhddv bin hddvchk conf Return kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 1 d opt FJS Vhddv bin hddvchk_conf lt Return gt NGID 3 g Reconfigure the kernel restart the system and then start system operation t touch reconfigure Return usr sbin shutdown y i6 g0 Return 2 Stop the machine administration hardware monitoring daemon Execute the following commands to stop the machin
18. Setup driver The driver configuration is added by following operations a The interface name is investigated in order to configure drivers and high layer products Please save the result of the prtcicl 1M command and obtain the difference information between current result and the result taken at 2 a Then the driver instance number for the extended PCI card is obtained In the following example since the instance number is 1 it can be determined that the interface name of the extended PCI card is hmel prtpicl v gt tmp prtpic post lt Return gt diff tmp prtpicl pre tmp prtpicl post more lt Return gt status okay devfs path pci 8d 2000 network 1 driver name hme binding name SUNW hme bus addr 1 instance 1 class obp device name network UU VVVVV VV V b It confirms that the interface name that obtained above mentioned procedure is an interface name that corresponds to the PCI bus slot of the added PCI card Please carry out this procedure when it cannot identify relationship between an interface name and PCI bus slot by expanding two or more PCI cards especially usr sbin FJSVmadm inst2comp hmel lt Return gt pcicsy18 ROBO1 PCI slot02 c The configuration of each driver is added Please refer to each driver manual for details 7 Setup redundant system When establishing the redundant configuration system it is necessary to configure the environmental configuration of the Redunda
19. kernel drv sd conf Ensure target 3 lun 0 target 3 lun 1 target 3 lun 3 are defined in kernel drv sd conf If they are not defined or commented modify kernel drv sd conf vi kernel drv sd conf Return name sd class scsi target 0 lun 0 name sd class scsi target 3 lun 0 name sd class scsi target 3 lun 1 name sd class scsi target 3 lun 2 b Boot the system and reconfigure the kernel touch reconfigure lt Return gt usr sbin shutdown y i6 g0 Return If you use hddv If this is the first time to pre define a disk array device begin with procedure 1 a If this is not the first time to pre define a disk array device begin with procedure 1 e a Check if kernel drv hddv conf exists Check if kernel drv hddv conf exists If it exists the hard disk driver is being installed so start from procedure l c If it does not exist the system is being operated with none active extension procedures so start from procedure 1 b Not exist new install to procedure 1 c Is kernel drv hddv conf lt Return gt kernel drv hddv conf No such file or directory Exists operation with non active extension procedures to procedure 1 b Is kernel drv hddv conf lt Return gt kernel drv hddv conf b When operating with non active extension procedures execute the following commands and confirm that kernel drv hddv conf is deleted hddvadm i Return Is kernel drv
20. lt Return gt GS SURE linkage mode in Redundant Line Control Function lopt FJSVhanet usr sbin stphanet n sha0 lt Return gt lopt FJSVhanet usr sbin hanetconfig create n sha2 m n i 192 168 20 10 t hme2 lt Return gt lopt FJSVhanet usr sbin hanetnic modify n sha0 t sha2 sha3 sha4 lt Return gt opt FJS Vhanet usr sbin strhanet n sha0 lt Return gt Multipath Function usr opt FJS Viomp bin iompadm c mpnet add mpnet0 hme2 lt Return gt usr opt FJS Viomp bin iompadm c mpnet start mpnet0 hme2 lt Return gt Notes It is necessary to configure the IP address specified by ifconfig 1M command in etc inet hosts and etc hostname hmel Unless these are configured the virtual interface for fast switching mode can not be activated after rebooting the system In addition in order to use IPv6 address please refer to the PRIMECLUSTER GLS 4 1A20 or later manual 8 Switch the redundant path Please switch an online communication path for extended communication path if needed Please refer to manuals of each redundant configuration software about how to switch a communication path 73 74 Chapter 5 Notes Restrictions This chapter describes notes and restrictions in case a PCI card exchanges or extends by using PCI Hot Plug feature 5 1 Notes This section describes notes when exchanging and extending PCI cards by using PCI Hot Plug feature 5 1 1 Network driver configuration changing In case of confi
21. prtpicl v gt tmp prtpic post Return diff tmp prtpicl pre tmp prtpicl post more Return status okay devfs path pci 8d 2000 network 1 driver name hme binding name SUNW hme bus addr 1 instance 1 class obp device name network VVVVV VV VO b It confirms that the interface name that obtained above mentioned procedure is an interface name that corresponds to the PCI bus slot of the added PCI card Please carry out this procedure when it cannot identify relationship between an interface name and PCI bus slot by expanding two or more PCI cards especially usr sbin FJSVmadm inst2comp hmel Return pcipsy18 ROBO1 PCI slot02 c The configuration of each driver is added Please refer to each driver manual for details 7 Connect to redundant system Please execute following commands in order to connect the extended PCI card to the existing redundant configuration system Notes On the NIC switching mode in Redundant Line Control Function it is impossible to extend a new interface to the existing redundant configuration system Moreover when extending an interface on the GS SURE linkage mode it is necessary to extend an interface after deactivating the virtual interface e g sha0 used by the GS SURE linkage mode Fast switching mode in Redundant Line Control Function usr sbin ifconfig hme2 plumb lt Return gt usr sbin ifconfig hme2 IP address netmask broadcast trailers up
22. usr sbin FJSVpfca fc_config v Return Fibre Channel driver PW028FC4 PW028FC5 Configuration is not required Compare the output of prtpicl 1M command with the output saved at 3 b and find the instance number of the added card b Add configuration of device driver Disk Tape devices Solaris OE sd st This example is a case where a disk device of target 2 lun 0 is pre defined in procedure 1 and added in procedure 4 If the device is a tape device re read sd as st in this procedure and perform the same operation Execute the following command to make the sd driver set up the added disk device usr sbin devfsadm Return 2 After saving the execution result of the prtpicl 1M command take the difference between the result and the information obtained in 3 a and get the physical path name of additional disk device then get the logical path name under dev dsk In the following example c3t3d0 c4t3d0 is the logical path name of the disk device added prtpicl v gt tmp prtpicl post lt Return gt diff tmp prtpicl pre tmp prtpicl post grep devfs path grep hddv lt Return gt devfs path pci 8d 2000 fibre channel 1 sd 3 0 devfs path pci 9 8d 2000 fibre channelQ 1 sd 3 1 devfs path pci 9 8d 2000 fibre channelQ 1 sd 3 2 devfs path pci 89 2000 fibre channel 1 sd 3 0 devfs path pci 89 2000 fibre channel 1 sd 3 1 devfs path pci 89 2000 fibre
23. 1O0hbanywareDisc stop Return 2 Specify the replacing PCI card If you use the Fibre Channel Card PW028FC4 PW028FCS execute the following The daemons will be stopped Follow the instructions below and determine the interface name of the path connecting the target PCI card and I O devices and the connected I O device a Determine from the WARNING messages output on the console the interface name of the path connecting the target PCI card and I O devices and the connected I O device In the example below glm2 is the interface name of the path connecting the broken PCI card and I O devices and sd20 is the disk device connected to glm2 When exchanging a PCI card which has two ports the physical path name of the connection path between the I O device corresponding to the other port and the I O device connected to that path must be determined in the same manner The physical path name corresponding to the other port can be determined by the fact that the name of the two ports on the same PCI card have a relation like pci 289 4000 scsi 92 and pci 89 4000 scsi 2 1 The I O device connected to that path resides under the directory of the physical path name determined WARNING pci 89 4000 scsi 2 glm2 invalid intcode fe00 WARNING pci 89 4000 scsi 2 sd 3 0 sd20 SCSI transport failed reason reset giving up If you use the Fibre Channel Card PW028FC4 PW028FCS the messages are shown as follows
24. 20 22 53 1cbn gt devices pci 89 4000 scsiQ 2 st 5 0 cbn Irwxrwxrwx 1 root root 41 Sep 20 22 53 lcn devices pci 9 89 4000 scsi 9 2 st 9 5 0 cn Irwxrwxrwx 1 root root 41 Sep 20 22 53 1h gt devices pci 89 4000 scsi Q 2 st 5 0 h Irwxrwxrwx 1 root root 41 Sep 20 22 53 1hb gt devices pci 89 4000 scsi Q 2 st 5 0 hb Irwxrwxrwx 1 root root 41 Sep 20 22 53 1hbn gt devices pci 89 4000 scsi 2 st 9 5 0 hbn Irwxrwxrwx 1 root root 41 Sep 20 22 53 Ihn gt devices pci 89 4000 scsi 2 st 9 5 0 hn 3 If the device is a disk device start the format 1M command and configure labeling and partition slice information of the logical path name of added disks If the device is a tape device this procedure is not needed Disk array device hard disk driver hddv This example is a case where a disk array device of target 3 lun 0 2 is pre defined in procedure 1 and added in procedure 5 1 There may be a change of status depending on the system operating state after pre definition in procedure 1 so check the LUN definition conflict between sd conf and hddv conf again In case of Check NG the definitions are conflicting so go back to procedure 1 e Check OK to procedure 9 b 2 opt FJSVhddv bin hddvchk conf Return kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 1 e opt FJSVhddv bin hddvchk conf Return NGID 3
25. 4 FUJITSU PCI Gigabit dap ME 22 Ethernet 2 0 or later 33MHz FWOUBGES enclosed with a PCI 66MHz Of OK PRIMECLUSTER card 8 9 GLS 4 0 or later 4 FUJITSU PCI Gigabit e a 2 2 Ethernet 2 0 or later 33MHz PWOO8GES enclosed with a PCI 66MHz OF Oe PRIMECLUSTER card 8 9 GLS 4 0 or later 4 oprpeariy FUJITSU PCLATM REO 22 ATM 1 2 or later enclosed 33MHz OK NG 10 GP7BSATIX With a PCI card PRIMECLUSTER 14 GLS 4 0 or later 4 BLASTBAND 1 0 or PRIMECLUSTER Li InfiniBand 11 PWO28SYI inter 66MHz OK OK GLS 4 1A20 or later 1 Please refer to 2 1 1 2 PCI Hot Plug configure operation of the PCI Hot Plug User s Guide about the operating clock of PCI cards and PCI slots 2 In case of replacing PCI card the PCI card with which the network interface is not once activated from the time of system booting cannot be exchanged 3 The replacement and expansion are supported only on the Redundant Line Control Function 4 When using the PRIMECLUSTER GLS 4 0 the replacement and expansion are supported only on the Redundant Line Control Function 5 It is necessary to apply the patch 912785 05 or later Please contact our customer service about the latest patch 6 When using the FUJITSU PCI Quad 10 100 Ethernet 1 0 it is necessary to apply the patch 912216 09 or later Please contact our customer service about the latest patch 7 It is necessary to apply the driver version 1
26. an entry in the etc vfstab file swap a dev dsk c2t2d0s4 Return vi etc vfstab Return dev dsk c2t2d0s4 swap no swap l Return swapfile dev swaplo blocks free dev dsk c2t2d0s4 32 164 16 788384 683680 Start applications Use the logical path name determined in 9 b to start applications Disk device filesystem operation Mount filesystem mount F ufs dev dsk c2t2d0s6 mnt Return Disk device raw access operation Tape device Start applications For details see the manual of each application Disk device swap device This procedure is not required 3 2 1 2 Extension of PCI cards on new redundant system The procedure to add PCI cards to a new redundant system on a Solaris 8 OE system is explained with examples for the multipath software below Multipath Disk Control MPHD GR Multipath Driver GRMPD If other multipath software is being used see the manual of each product 26 1 Setup devices which will be added to system This is an example where an disk array device at target 3 lun 0 2 is pre defined Extension using PCI Hot Plug is impossible without definition so definition must be done before the system goes into operation If you use Solaris OE sd If this is the first time to pre define a disk array device begin with procedure 1 a If this is not the first time to pre define a disk array device go to step 2 a Modify
27. array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration 43 a Before extension of PCI cards save the current execution result of prtpicl IM command composition information at present prtpicl v gt tmp prtpicl pre Return b Check the status of the slot The relationship of the position of a PCI card and format of Ap Id is shown below Location of PCI slot Element Format of Ap Id Solaris 8 OE and Solaris 9 OE Cabinet of Cabinet Number X i PRIMEPOWER System Board Number Y POPS OY TIBIQUE 900 1500 2500 Physical Slot Number ZZ DAS CXMOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN RXBOY PCI slotZZ Solaris 10 OE RXBOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN PCI slotZZ Solaris 10 OE PCI ZZ Below an example of adding a PCI card to chassis number 0 system board number 1 physical slot number 02 slot in chassis COMO1 PCIZslot02 is explained Check the status of the target slot is empty unconfigured cfgadm grep COM01 PCT slot02 Return pceipsy15 COM01 PCI slot02 unknown empty unconfigured unknown T O Cabinet Number X PCI Disk Box PCI Disk Box Number Y Physical Slot Number ZZ other than those above Physical Slot Number ZZ c To confirm the slot positio
28. be started etc rc2 d S99hbanyware start Return Jetc rc2 d S99hbanywareDisc start Return Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJSVmadm postphp Return Preparation to recognize devices for Fibre Channel When the Host Affinity function is being used with Fibre Channel SN200 series Host Affinity must be configured for the device added for PCI Fibre Channel PW008FC3U PW008FC2U GP7B8FC10U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility WWPN zoning of SN200 series Host affinity of Array disk device ETERNUS3000 ETERNUS6000 GR700 800 series and binding setting for fjpfca conf sd conf of host can be added from SAN management window automatically See Operation guide or User s Manual of each product for details Before using this function the following procedure 8 a 1 and 8 a 2 except fcp bind target setting are necessary Moreover begin from procedure 8 a 3 after executing this function If you don t use SAN management function of the above products No procedure is necessary Go to step 8 for Fibre Channel Card PW028FC4 PW028FC5 No procedure is necessary Go to
29. channel 1 sd 3 2 Is 1 dev dsk grep pci 8d 2000 fibre channel 1 sd 9 3 0 Return Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s0 gt J devices pci 8d 2000 fibre channel 1 sd 9 3 0 a Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s1 gt devices pci 8d 2000 fibre channel 1 sd 9 3 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s2 gt devices pci 8d 2000 fibre channel 1 sd 9 3 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s3 gt devices pci 8d 2000 fibre channel 1 sd 9 3 0 d Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s4 gt devices pci 8d 2000 fibre channel 1 sd 9 3 0 e Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s5 gt devices pci 8d 2000 fibre channel 1 sd 3 0 f Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s6 gt devices pci 8d 2000 fibre channel 1 sd 9 3 0 g Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s7 gt devices pci 8d 2000 fibre channel 1 sd 9 3 0 h Is 1 dev dsk grep pci 89 2000 fibre channel 1 sd 9 3 0 Return Irwxrwxrwx 1 root root 41 Sep 20 22 53 c4t3d0s0 gt J devices pci 89 2000 fibre channel 1 sd 9 3 0 a Disk array device hard disk driver hddv This example is a case where a disk array device of target 3 lunz0 2 is pre defined in procedure 1 and added in procedure 4 1 Check if kernel drv hddv conf exists Check if kernel drv hddv conf exists If it exists the hard disk driver is being ins
30. driver of the PCI card added In the following example instance number is 1 indicating that the interface name of the PCI card added is fjpfcal Making the LED of the PCI card blink using the fc adm command can specify the relation between the instance number and the slot position The relation between the WWPN and instance number of the PCI card added can be known from the fc info a command prtpicl v gt tmp prtpicl pfca Return diff tmp prtpicl pre tmp prtpicl pfca more Return status okay devfs path pci 8d 2000 fibre channelQ 1 driver name fjpfca binding name pcilOcf 105c bus addr 1 instance 1 _class obp device name fibre channel VVVVVVVV usr sbin FJSVpfca fc_adm l fjpfcal Return 2 Edit kernel drv fjpfca conf and add the following property definitions to the instance number from 9 a 1 port topology classification link speed Define this property when PCI cards using Fabric connection are added This definition is not needed with FC AL connection fcp bind target connection of target ID and WWN Define this property when disk array equipment using Fabric connections are added This definition is not needed with FC AL connection max throttle number of maximum SCSI command issue This property is defined for restriction on simultaneous issue of SCSI commands The necessity and value of this definition depends on the specification of the disk array device so refe
31. hardware monitoring daemon Use the following command to stop the hardware monitoring daemon of machine administration usr sbin FJSVmadm prephp Return 50 2 If you use the Fibre Channel Card PW0O28FC4 PW028FCS execute the following The daemons will be stopped etc rc0 d K10hbanyware stop Return etc rc0 d K10hbanywareDisc stop Return 3 Add PCI cards Add PCI cards according to the following procedure for PCI Fibre Channel PW008FC3U PW008FC2U GP7BSFCIU If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility No procedure is necessary Go to step 4 If you don t use SAN management function of the above products To add PCI cards with the following configurations Fibre Channel switch SN200 series and disk array device ETERNUS3000 ETERNUS6000 GR700 800 series need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on SN200 series If the Host Affinity function of ETERNUS3000 ETERNUS6000 GR700 800 series is used For details see SN200 Series Affinity User s Guide or ETERNUSmgr User Guide GRmgr User Guide If you use the Fibre Channel switch or disk array device other than described above see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be known from the eight ch
32. hddv conf Return kernel drv hddv conf No such file or directory c Use the following command to create hddv conf file for the pre defined extension function The message examples for operation with non active extension procedures means that disk array device GR720 currently used with taget 1 lun 0 is registered twice on the sd hddv driver New installation to procedure 1 e hddvadm M lt Return gt Operation with non active extension procedures to procedure 1 d hddvadm M lt Return gt already ID 1 exists devices pci 1d 2000 fibre channel 1 sd 9 1 0 a raw GR720 d When performing pre defined extension the same target number cannot be registered in sd conf and hddv conf so delete or comment out the conflicting definition parts in sd conf file when definition in the two files conflict with each other The following example shows the case that target 1 is repeated vi kernel drv sd conf Return name sd class scsi target 0 lun 0 name sd class scsi target 1 lun 0 name sd class scsi Delete or comment out target 1 lun 1 conflicting definition parts in sd hddv name sd class scsi target 1 lun 2 A target number that is already used by the sd driver of the system disk or CD ROM cannot be deleted In this 27 28 case change the target number in hddv conf and of the disk array device so that it does not conflict with sd conf e Edit th
33. is necessary Go to step 8 Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration 8 Set up driver Add driver configurations with the operation below With file related drivers configuration for the PCI card driver and the device driver is needed a Add configuration for PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver Configuration is not required Fibre Channel driver PW008FC3U PW008FC2U GP7B8FC1U 1 After saving execution result of prtpicl IM command take the difference between the result and the information obtained in procedure 3 a to calculate the instance number of the fjpfca driver of the PCI card added 59 60 In the following example instance number is 1 indicating that the interface name of the PCI card added is fjpfcal Making the LED of the PCI card blink using the fc adm command can specify the relation between the instance number and the slot position The relation between the WWPN and instance number of the PCI card added can be known from the fc info a command prtpicl v gt tmp prtpicl pfca Return diff tmp prtpicl pre tmp pr
34. monitoring daemon Use the following command to stop the hardware monitoring daemon of machine administration usr sbin FJSVmadm prephp lt Return gt 57 58 2 If you use the Fibre Channel Card PW0O28FC4 PW028FCS execute the following The daemons will be stopped etc rc0 d K 1Ohbanyware stop Return ete rc0 d K10hbanywareDisc stop Return Add PCI cards Add PCI cards according to the following procedure for PCI Fibre Channel PW008FC3U PW008FC2U GP7B8FC1U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility No procedure is necessary Go to step 4 If you don t use SAN management function of the above products To add PCI cards with the following configurations Fibre Channel switch SN200 series and disk array device ETERNUS3000 ETERNUS6000 GR700 800 series need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on SN200 series If the Host Affinity function of ETERNUS3000 ETERNUS6000 GR700 800 series is used For details see SN200 Series Affinity User s Guide or ETERNUSmgr User Guide GRmgr User Guide If you use the Fibre Channel switch or disk array device other than described above see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be known fro
35. mplb rdsk mplb8s3 dev FJS V mplb rdsk mplb8s4 dev FJS V mplb rdsk mplb8s5 dev FJS V mplb rdsk mplb8s6 dev FJS V mplb rdsk mplb8s7 Chapter 4 Network Devices This chapter describes the detailed procedure about replacement or extension of network related PCI cards by PCI Hot Plug Please refer to manuals of each software about the details of the redundant software 4 1 Replacement of PCI card This section describes replacement procedure of the PCI card by using PCI Hot Plug on non redundant system and redundant system 4 1 1 Replacement of PCI cards on non redundant system When exchanging PCI cards without redundant configuration software youmust stop applications which using PCI cards The replacement procedure of PCI cards is as follows 1 Stop the hardware monitoring of the Machine Administration The hardware monitoring of the Machine Administration is stopped by using the following command Please refer to PCI Hot Plug User s Guide about the details of prephp 1M command usr sbin FJSV madm prephp Return 2 Specify the replacing PCI card The interfaces e g hmel which stop applications for replacement of PCI cards are specified from warning messages outputted the console 3 Stop applications By following operations the high layer product e g service application which uses PCI cards for exchange is stopped a When using TCP IP interface The example in the case of using TCP IP interface is shown below
36. on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration a Before extension of PCI cards save the current execution result of the prtpicl IM command current configuration at present prtpicl v gt tmp prtpicl pre Return b Check the status of the slot The relationship of the position of a PCI card and format of Ap Id is shown below Location of PCI slot Element Format of Ap Id Solaris 8 OE and Solaris 9 OE Cabinet of Cabinet Number X PRIMEPOWER System Board Number Y POPS TTO 900 1500 2500 Physical Slot Number ZZ apni CXMOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN RXBOY PCI slotZZ Solaris 10 OE RXBOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN PCI slotZZ Solaris 10 OE PCI ZZ T O Cabinet Number X PCI Disk Box PCI Disk Box Number Y Physical Slot Number ZZ other than those above Physical Slot Number ZZ 51 52 Below an example of adding a PCI card to chassis number 0 PCI Disk BOX number 1 physical slot number 02 slot in PCI Disk BOX ROBO01 PCI slot02 is explained Check the status of the target slot is empty unconfigured cfgadm grep ROB01 PCI slot02 Return pcipsy18 ROBO1 PClI slot02 unknown empty unconfigured unknown c To confirm the slot position at extension operation blink the ALARM LED of Ap Id displayed in proc
37. physical path name of additional disk device then get the logical path name under dev dsk In the following example c2t2d0 is the logical path name of the disk device added prtpicl v gt tmp prtpicl post lt Return gt Disk device diff tmp prtpicl pre tmp prtpicl post grep devfs path grep sd lt Return gt gt devfs path pci 89 4000 scsi 2 sd 2 0 Is 1 dev dsk grep pci 89 4000 scsi 2 sd 2 0 Return Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s0 gt devices pci 89 4000 scsi 2 sd 9 2 0 a Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s1 gt devices pci 89 4000 scsi 2 sd 2 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s2 gt devices pci 89 4000 scsi 9 2 sd 9 2 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s3 gt devices pci 89 4000 scsi 2 sd 2 0 d Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s4 gt devices pci 89 4000 scsi 2 sd 9 2 0 e Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s5 gt devices pci 89 4000 scsi 2 sd 9 2 0 f Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s6 gt devices pci 89 4000 scsi 2 sd 9 2 0 g Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s7 gt devices pci 89 4000 scsi 2 sd 2 0 h Tape device diff tmp prtpicl pre tmp prtpicl post grep devfs path grep st Return 31 32 gt devfs path pci 2 89 4000 scsi Q2 st 9 5 0 Is 1 dev rmt grep pci
38. the PCI card can be known from the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digit of the WWPN in hexadecimal form The top eight digit are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the replacement card is 100000000e244061 0e24 4061 for Fibre Channel Card PW028FC4 PW028FC5 To replace PCI cards with the following configurations Fibre Channel switch and disk array device need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on Fibre Channel switch If the Host Zoning function of disk array device is used For details see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card These characters represent the bottom twelve digit of the WWPN in hexadecimal form The top four digit are fixed to 1000 in hexadecimal form For example if the following label is shown on the back of the card the WWPN of the replacement card is 10000000c9366037 IEEE 0000c9366037 Note When changing Affinity configuration on SN200 series or other Fibre Channel switch I O to other devices is effected by the change and may result in temporal errors I O to disk array devices reco
39. usr sbin ifconfig hmel inet 192 168 10 10 netmask broadcast trailers up lt Return gt b When using FNA SNA communication protocol over Ethernet When not using the redundant configuration software it is necessary to start all interfaces of Netcompo FNA LAN and all high layer applications that are using Netcompo FNA LAN For details please refer to the manual of Netcompo FNA LAN and high layer applications 4 2 2 Extension of PCI cards with new redundant system This subsection describes extension procedure of a new redundant system with the following redundant configuration software lt SynfinityLink 2 2 69 PRIMECLUSTER GLS 4 0 or later The replacement and extension of PCI cards are supported only on Redundant Line Control Function of SynfinityLink Notes Because of the dynamic exteinsion if the environment of SynfinityLink or PRIMECLUSTER GLS was newly added system reboot is not required When using other redundant software please refer to manuals of each product 1 Stop the hardware monitoring of the Machine Administration The hardware monitoring of the Machine Administration is stopped by using the following command usr sbin FJSV madm prephp Return 2 Add PCI cards a Before extending the PCI card please save the result current configuration information of the prtpicl 1M command prtpicl v gt tmp prtpicl pre Return b Check the status of the slot Subsequent examples describe e
40. you don t use SAN management function of the above products No procedure is necessary Go to step 7 for Fibre Channel Card PW028FCA4 PW028FC5 No procedure is necessary Go to step 7 7 Connect to redundant system The following procedure is described for MPHD GRMPHD If you use the redundancy software other than MPHD GRMPHD see the document for each product Execute the following command specifying the logical path name from 3 b If the operating path switched in procedure 3 the operating path will be automatically switched back If both MPHD GRMPD controlled devices are connected to the PCI card to replace the commands for both products need to be executed MPHD usr opt F JS Viomp bin iompadm c mphd restart adapter connect dev rdsk c3t1d0s2 Return GRMPD usr opt F JS Viomp bin iompadm c mplb restart adapter connect dev rdsk c3t1d0s2 Return 8 If you use the Fibre Channel Card PWO28FC4 PW028FCS execute the following The daemons will be started etc rc2 d S99hbanyware start Return etc rc2 d S99hbanywareDisc start Return 9 Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJS Vmadm postphp Return 3 2 Extension of PCI cards This section describes the procedu
41. 0B01 PCI slot03 fibre hp connected configured ok c After executing the command to disconnect the PCI card specifying the Ap_Id from 4 b confirm that the slot status has changed to disconnected unconfigured cfgadm c disconnect pcipsy21 R0B01 PCI slot03 lt Return gt cfgadm pcipsy21 R0B01 PCI slot03 lt Return gt Ap Id Type Receptacle Occupant Condition pcipsy21 R0OB01 PCI slot03 unknown disconnected unconfigured unknown Note When error occurs during disconnect cfgadm command unusually fail with following message If cfgadm command fails execute the command once again cfgadm Component system is busy try again disconnect failed d To confirm the slot position at replacement operation blink the ALARM LED of Ap Id displayed in procedure 4 a cfgadm x led fault mode blink pcipsy21 ROB01 PCI slot03 Return 5 Replace the PCI card Replace the PCI card disconnected in 4 with a replacement card and connect cable to devices This operation is performed by our customer support When exchanging Fibre Channel cards the following operations are also required for PCI Fibre Channel PW008FC3U PW008FC2U GP7B8FC1U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility No procedure is necessary Go to step 7 If you don t use SAN management function of the above products To replace PCI cards with the following c
42. 1 dev dsk grep pci 89 2000 fibre channel 1 hddv 3 0 lt Return gt Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s0 gt devices pci 89 2000 fibre channel 1 hddv 3 0 a Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s1 gt devices pci 89 2000 fibre channel 1 hddv 3 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s2 gt devices pci 89 2000 fibre channel 1 hddv 3 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s3 gt devices pci 89 2000 fibre channel 1 hddv 3 0 d Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s4 gt devices pci 89 2000 fibre channel 1 hddv 3 0 e Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s5 gt devices pci 89 2000 fibre channel 1 hddv 3 0 f Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s6 gt devices pci 89 2000 fibre channel 1 hddv 3 0 g Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s7 gt devices pci 89 2000 fibre channel 1 hddv 3 0 h 4 Start the format 1M command and configure labeling and partition slice information calculated in 9 b 3 to the logical path name of each extended LUN c Add configure of PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver No procedure is necessary Go to step 10 Fibre Channel driver PW008FC2U W008FC2X GP7B8FC1U GP7B8FC1X No procedure is necessary Go to step 10 Fibre Channel Card PW028FC4 PW028FC5 Bind the
43. 1X GR Multipath Driver 1 0 1 or later PPO28SCIU FUJITSU PCI SCSI Adapter Driver 1 0 or later enclosed 33MHz OK OK none PPO28SCIX with a PCI card 33MHz x i Solaris OE fjulsa 66MHz NG 3 NG 3 none PWO08SC3 FUJITSU ULTRA LVD SCSI Host Bus Adapter 33MHz x Driver 1 0 or later enclosed 66MHz BO NO Os none with a PCI card Multipath Disk Control 2 2 1 or Fibre FUJITSU PCI Fibre Channel later 5 Channel GP7B8FC1U 2 2 or later enclosed with a 33MHz OK OK PCI card 6 7 9 GR Multipath Driver 1 0 1 or later Multipath Disk Control 2 2 1 or FUJITSU PCI Fibre Channel 33MHz later 5 PWOO8FC2U 2 2 1 or later enclosed with 66MH OK OK i a PCI card 7 9 GR Multipath Driver 1 0 1 or later Supporting Functions Software of driver 1 Functions Product ID Operating Hot Hot Redundant driver name clock 1 Software replacement expansion Multipath Disk FUJITSU PCI Fibre Channel err dd or ater PWOOSFC3U 2 2 1 Update2 or later 33MHz OK OK enclosed with a PCI card 66MHz GR Multipath 9 Driver 1 0 1 or later PWO028FC4S Emulex LightPulse FC 33MHz OK OK 8 PW028FC4U SCSI IP 6 01f or later 13 66MHZ PWO028FCSS Emulex LightPulse FC 33MHZ OK OK 8 PWO28FCSU SCSI IP 6 01f or later 13 66MHZ 1 Please refer to 2 1 1 2 PCI Hot Plug configure operation of the PCI Hot Plug User s Guide about
44. 289 4000 scsi 2 st 2 5 0 Return Irwxrwxrwx root root 41 Sep 20 22 53 1 gt devices pci 89 4000 scsi Q 2 st 5 0 Irwxrwxrwx 1 root root 41 Sep 20 22 53 1b gt devices pci 89 4000 scsi Q 2 st 5 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 Ibn gt devices pci 89 4000 scsi 2 st 9 5 0 bn Irwxrwxrwx 1 root root 41 Sep 20 22 53 1c gt J devices pci 89 4000 scsi Q 2 st 5 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 Icb gt devices pci 89 4000 scsi 2 st 5 0 cb Irwxrwxrwx 1 root root 41 Sep 20 22 53 1cbn gt devices pci 89 4000 scsiQ 2 st 5 0 cbn Irwxrwxrwx 1 root root 41 Sep 20 22 53 1cn gt devices pci 89 4000 scsi 2 st 9 5 0 cn Irwxrwxrwx 1 root root 41 Sep 20 22 53 1h gt devices pci 89 4000 scsi Q 2 st 5 0 h Irwxrwxrwx 1 root root 41 Sep 20 22 53 1hb gt J devices pci 89 4000 scsi 2 st 5 0 hb Irwxrwxrwx 1 root root 41 Sep 20 22 53 1hbn gt devices pci 89 4000 scsi 2 st 5 0 hbn Irwxrwxrwx 1 root root 41 Sep 20 22 53 Ihn gt J devices pci 89 4000 scsi 2 st 5 0 hn 3 If the device is a disk device start the format 1M command and configure labeling and partition slice information of the logical path name of added disks If the device is a tape device this procedure is not needed Disk array device hard disk driver hddv This example is a case where a disk array device of target 3 lun 0 2 is pre defined in proced
45. 28FC4 PW028FCS execute the following The daemons will be started ete re2 d S99hbanyware start Return ete rc2 d S99hbanywareDisc start Return 6 Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJSVmadm postphp lt Return gt 7 Preparation to recognize devices for Fibre Channel When the Host Affinity function is being used with Fibre Channel SN200 series Host Affinity must be configured for the device added for PCI Fibre Channel PW008FC3U PW008FC2U GP7B8FC10U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility WWPN zoning of SN200 series Host affinity of Array disk device ETERNUS3000 ETERNUS6000 GR700 800 series and binding setting for fjpfca conf sd conf of host can be added from SAN management window automatically See Operation guide or User s Manual of each product for details Before using this function the following procedure 8 a 1 and 8 a 2 except fcp bind target setting are necessary Moreover begin from procedure 8 a 3 after executing this function If you don t use SAN management function of the above products No procedure is necessary Go to step 8 for Fibre Channel Card PW028FC4 PW028FC5 No procedure
46. 3000 ETERNUS6000 GR700 800 series need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on SN200 series If the Host Affinity function of ETERNUS3000 ETERNUS6000 GR700 800 series is used For details see SN200 Series Affinity User s Guide or ETERNUSmgr User Guide GRmgr User Guide If you use the Fibre Channel switch or disk array device other than described above see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digit of the WWPN in hexadecimal form The top eight digit are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the replacement card is 100000000e244061 0e24 4061 for Fibre Channel Card PW028FCA4 PW028FC5 To replace PCI cards with the following configurations Fibre Channel switch and disk array device need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on Fibre Channel switch If the Host Zoning function of disk array device is used For details see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be
47. 4 1 or later and the patch 912214 03 or later Please contact our customer service about the latest patch 8 When using Solaris 8 OE it is necessary to apply the patch 114536 03 or later Please contact our customer service about the latest patch When applying the patch 114536 03 or later if the patch 912314 01 or later has applied it is necessary to remove the patch When removing the patch 912314 01 or 912314 02 the platform sun4us kernel drv fjgi conf file which defines fjgi interface settings will be replaced If you have edited fjgi conf please make a backup copy of fjgi conf beforehand and pick up your own settings and add them to the new one after removing the patch if necessary 9 When using Solaris 9 OE it is necessary to apply the patch 114994 03 or later Please contact our customer service about the latest patch When applying the patch 114994 03 or later if the patch 912660 01 or later has applied it is necessary to remove the patch When removing the patch 912660 01 or 912660 02 the platform sun4us kernel drv fjgi conf file which defines fjgi interface settings will be replaced If you have edited fjgi conf please make a backup copy of fjgi conf beforehand and pick up your own settings and add them to the new one after removing the patch if necessary 10 System rebooting is needed for hot extension 11 When PCI card is InfiniBand please refer to BLASTBAND GUIDE for the procedure for replacement and expansion
48. I Hot Plug feature Please do not exchange or expand PCI cards by using PCI Hot Plug feature during executing Dynamic Reconfiguration DR 76
49. I slot03 pci pci hp connected configured ok c Please disconnect the PCI card by executing the cfgadn 1M command with Ap Id and confirm that the slot status is disconnected unconfigured cfeadm c disconnect pcipsy21 ROB01 PCI slot03 Return cfead m pcipsy21 R0B01 PCTZslot03 Return Ap Id Type Receptacle Occupant Condition pcipsy21 ROB01 PCI slot03 pci pci hp disconnected unconfigured unknown d To confirm the slot position at replacement operation blink the ALARM LED of Ap Id d 4 a isplayed in procedure cfgadm x led fault mode blink pcipsy21 R0B01 PCI slot03 lt Return gt 5 Replace the PCI card The PCI card disconnected in procedure 4 Disconnect the PCI card is exchanged for a new PCI card Out customer support works about this procedure Notes When using the PRIMECLUSTER GLS Multipath Function please exchange the PCI card after waiting 10 seconds or more after Procedure 4 Disconnect the PCI card 6 Connect the PCI card The cfgadm command is executed with the configure option to Ap_Id or the push button of the PCI bus slot is pushed Then a new PCI card is connected In addition a push button is valid only in multiple user mode Please confirm that the slot status is connected configured by using the cfgadm 1M command after the above mentioned procedure Notes If the setup of driver conf 4 file exist please refer to 5 1 1 Network driver configuration
50. JS Vpfca fc config v Return 45 Fibre Channel driver PW028FCA4 PW028FC5 Configuration is not required Compare the output of prtpicl 1M command with the output saved at 3 b and find the instance number of the added card Add device driver configuration Add device driver configuration with the following procedures according to the I O device type Disk Tape devices Solaris OE sd st This example is a case where a disk device of target 2 lun 0 is pre defined in procedure 1 and added in procedure 4 If the device is a tape device re read sd as st in this procedure and perform the same operation 1 See kernel drv sd conf and check that target 2 and lun 0 are defined When this definition does not exist add the definition If it is commented out remove comments Disk equipment vi kernel drv sd conf Return name sd class scsi target 0 lun 0 name sd class scsi In case the definition does not exist target 2 lun 0 add or remove comments Tape equipment vi kernel drv st conf lt Return gt name st class scsi target 0 lun 0 name st class scsi In case the definition does not exist target 5 lun 0 add or remove comments 2 To make the sd driver set up the disk device added execute the following commands When the commands are executed the following messages are displayed but do not effect performance Similar messages are displayed when a tape device alread
51. PCI slot02 unknown empty unconfigured unknown c To confirm the slot position at extension operation blink the ALARM LED of Ap Id displayed in procedure 3 b cfgadm x led fault mode blink pcipsy18 ROB01 PCI slot02 Return d After checking that the READY LED on the PCI slot to add the PCI card is off add PCI cards and connect cable to devices This operation is performed by our customer support e Use the cfgadm 1M command to check that the slot status of the added PCI card changed to disconnected unconfigured cfgadm pcipsy18 ROBO1 PCI slot02 Return Ap Id Type Receptacle Occupant Condition pcipsy18 ROB0O1 PCI slot02 unknown disconnected unconfigured unknown 4 Connect the PCI card Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the extension slot position Note that the push button is only effective in multiuser mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy18 R0B01 PCI slot02 Return cfgadm pcipsy18 ROB01 PCI slot02 Return Ap Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCI slot02 mult hp connected configured ok 5 If you use the Fibre Channel Card PW0
52. UN Storage Zone mplbO exist c1t4d0s2 c2t3d0s2 0 GR720464f 12340001 mplb8 add c3t3d0s2 c4t3d0s2 c5t3d0s2 0 GR8404550 000213 usr sbin mplbconfig av Return IOMP dev FJS Vmplb fiomp adm8 Node dev FJS V mplb rdsk mplb8s0 dev FJS V mplb rdsk mplb8s1 dev FJS V mplb rdsk mplb8s2 dev FJS V mplb rdsk mplb8s3 dev FJS V mplb rdsk mplb8s4 dev FJS V mplb rdsk mplb8s5 dev FJS V mplb rdsk mplb8s6 dev FJS V mplb rdsk mplb8s7 3 2 2 Solaris 9 OE or later This section describes the procedures to extend PCI cards on Solaris 9 OE or later systems If you use Fibre Channel Card PW028FCA4 PWO28FC5 ensure the automap function is effective Boot the system after 42 modifying Ipfc conf vi kernel drv Ipfc conf lt Return gt If automap is set SCSI IDs for all FCP nodes without persistent bindings will be automatically generated If new FCP devices are added to the network when the system is down there is no guarantee that these SCSI IDs will remain the same when the system is booted again The bind method of the port is used as the binding method of automap devices to preserve SCSI IDs between link down and link up If automap is 0 only devices with persistent bindings will be recognized by the system automap 1 lt set to 1 3 2 2 1 Extension of PCI cards on non redundant system Procedures to add PCI cards without making them redundant on a Solar
53. ample of adding a PCI card to chassis number 0 PCI Disk BOX number 1 physical slot number 02 slot in PCI Disk BOX ROBO1 PCI slot02 is explained Check the status of the target slot is empty unconfigured cfgadm grep ROBO1 PCEislot02 Return pcipsy18 ROBO1 PCI slot02 unknown empty unconfigured unknown T O Cabinet Number X PCI Disk Box PCI Disk Box Number Y Physical Slot Number ZZ other than those above Physical Slot Number ZZ c To confirm the slot position at extension operation blink the ALARM LED of Ap_Id displayed in procedure 4 b cfgadm x led fault mode blink pcipsy18 ROB01 PCI slot02 Return d After checking that the READY LED on the PCI slot to add the PCI card is off add PCI cards and connect cable to devices This operation is performed by our customer support e Use the cfgadm 1M command to check that the slot status of the added PCI card changed to disconnected unconfigured cfgadm pcipsy18 ROBO1 PCI slot02 Return Ap Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PClI slot02 unknown disconnected unconfigured unknown 5 Connect the PCI card Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the extension slot position Note that the push button is only effective in multiuser mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status ha
54. and calculate the corresponding logical path name under dev dsk In the following example c3t3d0 is logical path name of added lun 0 The logical path name of lun 1 2 is set to c3t3d1 c3t3d2 respectively because cXtYdZmeans X PCI card number Y target number Z lun number prtpicl v gt tmp prtpicl post Return diff tmp prtpicl pre tmp prtpicl post grep devfs path grep hddv Return gt devfs path pci 89 2000 fibre channel 1 hddv 9 3 0 gt devfs path pci 89 2000 fibre channel 1 hddv 93 1 gt devfs path pci 89 2000 fibre channel 1 hddv 3 2 Is 1 dev dsk grep pci 89 2000 fibre channel 1 hddv 9 3 0 lt Return gt Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s gt devices pci 89 2000 fibre channel 1 hddv 3 0 a Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s1 gt devices pci 89 2000 fibre channel 1 hddv 3 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s2 gt devices pci 89 2000 fibre channel 1 hddv 3 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s3 gt devices pci 89 2000 fibre channel 1 hddv 3 0 d Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s4 gt J devices pci 89 2000 fibre channel 1 hddv 3 0 e Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s5 gt devices pci 89 2000 fibre channel 1 hddv 3 0 f Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s6 gt devices pci 89 2000 fibre channel 1 hd
55. and timeout for Target 0 0 Or scsi WARNING pci9 MN M000 scsi 9 M N xxxx Disconnected tagged cmd s x timeout for Target x x Example scsi WARNING pci 80 4000 scsi 9 2 1 glm2 Disconnected tagged cmd s 1 timeout for Target 0 0 Or scsi WARNING pci 9 MN M000 xxxx 9 M N xxxx 9 M N xxxx SCSI transport failed reason reset xxxx Example scsi WARNING pci 2 80 4000 scsi 92 1 sd 9 3 0 sd20 SCSI transport failed reason reset retrying command When these WARNING messages are displayed the access to the shared device is temporarily interrupted But the access is retried and restored automatically by the system However there is a possibility that the access is not restored according to some conditions Please stop the access to the shared device connected with the PCI card during its exchange 5 1 4 Others When expanding PCI cards by using PCI Hot Plug feature it is necessary to install the device driver case of file devices and the driver for using the added PCI card When extending PCI cards each driver consumes memory capacity to control PCI cards Please refer to the installation guide attached to each PCI card about memory capacity required in order to extend PCI cards and installation guide of the redundant configuration software about memory capacity required 5 2 Restrictions This section describes restrictions when exchanging and extending PCI cards by using PC
56. and to make the hard disk driver set up the LUN usr sbin hddvadmdr lt Return gt 3 After saving execution result of the prtpicl command get the physical path name of added LUN by taking the difference between the result and the information obtained in 4 a and calculate the corresponding logical path name under dev dsk In the following example c3t3d0 is logical path name of added lun 0 The logical path name of lun 1 2 is set to c3t3d1 c3t3d2 respectively because cXtYdZmeans X PCI card number Y target number Z lun number 24 prtpicl v gt tmp prtpicl post Return diff tmp prtpicl pre tmp prtpicl post grep devfs path grep hddv Return gt devfs path pci 89 2000 fibre channel 1 hddv 9 3 0 gt devfs path pci 89 2000 fibre channel 1 hddv 93 1 gt devfs path pci 89 2000 fibre channel 1 hddv 3 2 Is 1 dev dsk grep pci 89 2000 fibre channel 1 hddv 3 0 lt Return gt Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s0 gt devices pci 89 2000 fibre channel 1 hddv 3 0 a Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s1 gt devices pci 89 2000 fibre channel 1 hddv 3 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s2 gt devices pci 89 2000 fibre channel 1 hddv 3 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s3 gt devices pci 89 2000 fibre channel 1 hddv 3 0 d Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s4 gt
57. aracters shown on a label on the front plate of the card These characters represent the bottom eight digit of the WWPN in hexadecimal form The top eight digit are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the replacement card is 100000000e244061 0e24 4061 for Fibre Channel Card PW028FCA4 PW028FC5 To replace PCI cards with the following configurations Fibre Channel switch and disk array device need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on Fibre Channel switch If the Host Zoning function of disk array device is used For details see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card These characters represent the bottom twelve digit of the WWPN in hexadecimal form The top four digit are fixed to 1000 in hexadecimal form For example if the following label is shown on the back of the card the WWPN of the replacement card is 10000000c9366037 IEEE 0000c9366037 Note When changing Affinity configuration on SN200 series or other Fibre Channel switch I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but
58. ard file PWO08GE1X 1000BASE T PWO008GE4 1000BASE SX FUJITSU PCI Gigabit Ethernet Please make a setting by the ndd command 2 0 or later Or please reboot a system after changing the PWO08GE5 10 100 1000BASE T enclosed with a PCI card platform sun4us kernel drv fjgi conf file 1 In case of X1033A U X please refer to Sun Microsystems Inc web page about Platform Notes The hme FastEthernet Device Driver and ndd 1M man page etc 5 1 2 Using tape device while PCI Hot Plug operation By using the PCI Hot Plug feature a job which uses tape devices may be terminated abnormally While using the PCI Hot Plug feature avoid using tape devices 5 1 3 Exchanging while using shared device SCSI interface for cluster system If a shared device of non multipath configuration is connected with a cluster system there is a possibility that the following WARNING messages are displayed in the node to which the PCI card is not exchanged while exchanging it with which the shared device is connected by the PCI Hot Plug function scsi WARNING pci 9 MN M000 scsi M N xxxx Connected command timeout for Target x x Example scsi WARNING pci 80 4000 scsi 2 2 1 glm2 Connected command timeout for Target 0 0 75 Or scsi WARNING pci 9 MN M000 scsi 9 M N xxxx Disconnected command timeout for Target x x Example scsi WARNING pci 80 4000 scsi 9 2 1 glm2 Disconnected comm
59. auto mapped target connected with the card and modify the entries of lpfc conf to match the requirements of the extended Fibre Channel Controller These procedures are reflected on next boot time In the following indicate to bind the auto mapped target with the card usr sbin Ipfc Iputil Return LightPulse Common Utility for Solaris SPARC Version 1 6a8 03 03 2004 Copyright c 2003 Emulex Network Systems Inc Emulex Fibre Channel Host Adapters Detected 2 Host Adapter 0 Ipfc11 is an LP9802 Ready Mode Host Adapter 1 Ipfc9 is an LP9802 Ready Mode MAIN MENU 1 List Adapters 2 Adapter Information 3 Firmware Maintenance 4 Reset Adapter 5 Persistent Bindings 0 Exit Enter choice gt 5 lt Return gt PERSISTENT BINDINGS MENU 1 Display Current Bindings 2 Display All Nodes 3 Duplicate Adapter Bindings 4 Bind Target Manually 5 Bind Automapped Targets 6 Delete Binding s 0 Return to Main Menu Enter choice gt 5 lt Return gt O Ipfc11 1 Ipfc9 Select an adapter gt 0 lt Return gt Automapped FCP Nodes Node Target WWPN WWNN Automapped FCP Node 0 50 06 0e 80 00 10 04 92 50 06 0e 80 00 10 04 92 Automapped FCP Node 1 50 06 0e 80 00 10 04 90 50 06 0e 80 00 10 04 90 Press the Enter or Return key to continue Bind all auto mapped targets Y N gt Y lt Return gt Select Binding Method 1 By Node Name 2 By Port Name 3 ByD ID 0 Cancel Enter Choice gt 2 Retur
60. ay not be copied by any means without the expressed written permission of FUJITSU LIMITED FUJITSU LIMITED reserves the right to make changes to this manual without prior notice Copyright 1995 GLOBEtrotter Software Inc Copyright 1995 Software Research Associates Inc All Rights Reserved All Rights Reserved Copyright C FUJITSU LIMITED 2005 Contents Chapter 1 PCI cards with support for PCI Hot Plug sseeeeeeeeeeee 1 Val Pelcards for fil deVIGCeS s cient trie rie et Pede terne eee nee a P e Ree Eo duds 1 1 2 PCL cards for network QeviceS arooda terea etx aer o Tea an ua a een E 2 Chapter 2 Overview of replacement and extension of PCI cards 5 2 1 Replacement of PCI card sessssssssssssessssses entente r einen enn sss intet en nene 5 2 2 Extension of P Cl cards iia dota cde dc Lidia da iae oe de bed baee e co dU ee ede do eere 6 Chapter S File DeViC8S 3 ta ctr ia xe OEIL e ER o Ee t eL OU SLE DL ect d et Ave 3 11 34 Replacement of PCI Garde p ER e Ca duc d ee ue doc Lee ee ee en oos 11 32 Extension o uec 18 Chapter 4 Network Devices sssssssssssssssssseeeeeeeeenenen nne nnnnn nnn 65 4 1 Replacement of PCI card ssssssssssssssssssssseseenene eese entente rint intet entere enne 65 4 2 Extension ol POLCards ecol eet eter Ka AA PA EE OEA RAA CAAS NET PE LAAR RE EARRAS 68 Chapter 5 Notes Restrictions i suse ca ae n inina ni
61. c2 d S99hbanyware start Return Jetc rc2 d S99hbanywareDisc start Return Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJSV madm postphp Return Preparation to recognize devices for Fibre Channel When the Host Affinity function is being used with Fibre Channel SN200 series Host Affinity must be configured for the device added for PCI Fibre Channel PW008FC3U PW008FC2U GP7BS8FCI1U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility WWPN zoning of SN200 series Host affinity of Array disk device ETERNUS3000 ETERNUS6000 GR700 800 series and binding setting for fjpfca conf sd conf of host can be added from SAN management window automatically See Operation guide or User s Manual of each product for details Before using this function the following procedure 9 a 1 and 9 a 2 except fcp bind target setting are necessary Moreover begin from procedure 9 a 3 after executing this function If you don t use SAN management function of the above products No procedure is necessary Go to step 9 for Fibre Channel Card PW028FC4 PW028FC5 No procedure is necessary Go to step 9 Note When changin
62. card can be known from the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digit of the WWPN in hexadecimal form The top eight digit are fixed to 10000000 in hexadecimal form 13 14 For example if the following label is shown on the front plate of the card the WWPN of the replacement card is 100000000e244061 0e244061 for Fibre Channel Card PW028FC4 PW028FC5 To replace PCI cards with the following configurations Fibre Channel switch and disk array device need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on Fibre Channel switch If the Host Zoning function of disk array device is used For details see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card These characters represent the bottom twelve digit of the WWPN in hexadecimal form The top four digit are fixed to 1000 in hexadecimal form For example if the following label is shown on the back of the card the WWPN of the replacement card is 10000000c9366037 IEEE 0000c9366037 Note When changing Affinity configuration on SN200 series or other Fibre Channel switch I O to other devices is effected by the chang
63. changing cfgadm c configure pcipsy21 R0B01 PCIZslot03 lt Return gt cfgad m pcipsy21 ROB01 PCI slot03 Return Ap Id Type Receptacle Occupant Condition pcipsy21 ROBO1 PCI slot03 pci pci hp connected configured ok 7 Connect to redundant system In order to connect the exchanged PCI card into a redundant system please execute the following command with interface name specified at procedure 2 Specify the replacing PCI card Fast switching mode in Redundant Line Control Function usr sbin ifconfig hmel plumb Return usr sbin ifconfig hmel IP address netmask broadcast trailers up Return lopt FJSVhanet usr sbin hanetnic add n sha0 i hmel Return NIC switching mode in Redundant Line Control Function lopt FJSVhanet usr sbin hanetnic change n sha0 Return lopt FJSVhanet usr sbin strptl n shal Return GS SURE linkage mode in Redundant Line Control Function 67 lopt FJSVhanet usr sbin strhanet n sha0 Return lopt FJSVhanet usr sbin hanetpoll on Return Multipath Function usr opt FJS Viomp bin iompadm c mpnet connect mpnet0 hmel lt Return gt usr opt FJS Viomp bin iompadm c mpnet start mpnet0 hmel lt Return gt Notes It is necessary to configure the IP address specified by ifconfig 1M command in etc inet hosts and etc hostname hmel Unless these are configured the virtual interface for fast switching mode can not be activated after rebooting the system I
64. cl 1M command with the output saved at 3 b and find the instance number of the added card b Add configuration of device driver This is an example where an disk array device in redundant configuration at target 3 lun 0 2 is added in procedure 4 If this is the first time to pre define a disk array device begin with procedure 8 b 1 If this is not the first time to pre define a disk array device begin with procedure 8 b 5 1 Check if kernel drv hddv conf exists Check if kernel drv hddv conf exists If it exists the hard disk driver is being installed so start from procedure 8 b 3 If it does not exist the system is being operated with none active extension procedures so start from procedure 8 b 2 Not exist new install to Procedure 8 b 3 Is kernel drv hddv conf Return kernel drv hddv conf No such file or directory 53 54 Exist operation with non active extension procedures to Procedure 8 b 2 Is kernel drv hddv conf Return kernel drv hddv conf When operating with non active extension procedures execute the following commands and confirm that kernel drv hddv conf is deleted hddvadm i Return Is kernel drv hddv conf Return kernel drv hddv conf No such file or directory Use the following command to create hddv conf file for the pre defined extension function The message examples for operation with non active extension procedures mea
65. conf file for the active extension function The message examples for operation with non active extension procedures means that disk array device GR720 currently used with taget 1 lun 0 is registered twice on the sd hddv driver New installation to procedure 8 b 5 hddvadm M Return Operation with non active extension procedures to procedure 8 b 4 hddvadm M Return already ID 1 exists devices pci 1d 2000 fibre channel 1 sd 9 1 0 a raw GR720 4 When performing pre defined extension the same target number cannot be registered in sd conf and hddv conf so delete or comment out the conflicting definition parts in sd conf file when definition in the two files conflict with each other The following example shows the case that target 1 is repeated 47 48 5 6 vi kernel drv sd conf Return name sd class scsi target 0 lun 0 name sd class scsi target 1 lun 0 name sd class scsi Delete or comment out target 1 lun 1 the duplication definition parts in sd hddv name sd class scsi target 1 lun 2 A target number that is already used by the sd driver of the system disk or CD ROM cannot be deleted In this case change the target number in hddv conf and of the disk array device so that it does not conflict with sd conf Edit the kernel drv hddv conf file created automatically by the hddvadm M command and remove comments of the LUN definition to pre
66. csi 2 sd Q 3 0 h 3 Stop applications Stop applications with the following operations a Stop vold sh etc init d volmgt stop lt Return gt b Stop the all I O devices connected to the target PCI card determined in procedure 3 If the device is a disk unit file system operation 1 Determine the mountpoint of the disk from the logical path name determined in procedure 3 b c2t3d0 mount grep c2t3d0 Return export home on dev dsk c2t3d0s3 setuid read write largefiles on Mon Sep 30 01 00 51 2002 develop firm on dev dsk c2t3d0s0 setuid read write largefiles on Mon Sep 30 01 00 51 2002 develop drv on dev dsk c2t3d0s1 setuid read write largefiles on Mon Sep 30 01 00 51 2002 2002 pub on dev dsk c2t3d0s6 setuid read write largefiles on Mon Sep 30 01 00 50 2 Stop access to the disk To check which process is using the target file system command as follows fuser c export home Return export home 14967c 14571c 14493ctm 14020c 13828tm 13803c 13575c 13133c 13125tm 13107c 12682ctm 12066tm 12048c 11971ctm 11952ctm 11937c 11867c 11846c 349m 3 Unmount the disk umount export home Return umount develop firm Return umount develop drv Return umount pub Return If the device is a disk unit raw access operation or a tape device use the fuser 1M 1 Check the access statistics of the target disk determined in pr
67. d and format of Ap Id is shown below N an integral number Location of PCI slot Element Format of Ap Id Cabinet of Cabinet Number X ei a A CR BOR coe PRIMEPOWER System Board Number Y gi n Ok d 900 1500 2500 Physical Slot Number ZZ Mou CXMOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN RXBOY PCI slotZZ T O Cabinet Number X PCI Disk Box PCI Disk Box Number Y Solaris 10 OE i Physical Slot Number ZZ RXBOY PCI ZZ Solaris 8 OE and Solaris 9 OE other than those above Physical Slot Number ZZ m d d PCI ZZ Please confirm that the PCI slot status which extended PCI cards is empty unconfigured by using cfgadm 1M command cfeadm grep COM01 PCI slot02 lt Return gt pceicsy15 COM01 PClI slot02 unknown empty unconfigured unknown c To confirm the slot position at extension operation blink the ALARM LED of Ap Id displayed in procedure 2 b cfgadm x led fault mode blink pcipsy15 COM01 PCI slot02 lt Return gt d Please confirm that READY LED of the PCI bus slot that extends a PCI card has gone out and extend a PCI card This operation is performed by our customer support e Please confirm that the PCI slot status which extended PCI cards is disconnected unconfigured by using cfgadm 1M command cfeadm pcipsy15 COM01 PCIZslot02 lt Return gt Ap Id Type Receptacle Occupant Condition peicsy15 COM01 PCI slot02 unknown disconnected unconfigured unknown
68. d disk array device GR720 target 1 in use with non active extension procedures being registered Is kernel drv hddv conf Return kernel drv hddv conf vi kernel drv hddv conf Return name hddv class scsi target 0 lun 0 name hddv class scsi target 0 lun 7 name hddv class scsi target 1 lun 0 name hddv class scsi target 1 lun 1 Remove comments of the LUN in use name hddv class scsi target 1 lun 2 namez hddv class scsi target 3 lun 0 name hddv class scsi target 3 lunz1 Remove comments of extended LUN name hddv class scsi target 3 lun 2 name hddv class scsi target 3 lun 3 f Check for LUN definition conflict between sd conf and hddv conf by executing the hddvchk_conf command The message example for Check NG means that target 3 is defined in both files and the procedures must be performed again from procedure 1 d The followings are examples of execution of the FJSVhddv package installation to opt The path of the hddvchk_conf command depends on the installed directory Check OK to procedure 1 g opt FJS Vhddv bin hddvchk conf Return kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 1 d opt FJS Vhddv bin hddvchk_conf lt Return gt NGID 3 g Reconfigure the kernel restart the system and then start system operation touch reconfigure lt Return gt usr sbin shutdown y i6 g0 Re
69. define and the LUN definition currently being used with non active extension procedures If case active extension has been performed before and the LUN definition to extend the next procedures 1 f 1 g are not needed The following example shows target 3 lunz0 2 for pre defined extension and disk array device GR720 target 1 in use with non active extension procedures being registered Is kernel drv hddv conf lt Return gt kernel drv hddv conf vi kernel drv hddv conf lt Return gt name hddv class scsi target 0 lun 0 name hddv class scsi target 0 lun 7 name hddv class scsi target 1 lun 0 name hddv class scsi target 1 lun 1 Remove comments of the LUN in use name hddv class scsi target 1 lun 2 name hddv class scsi target 3 lun 0 name hddv class scsi target 3 lun 1 Remove comments of extended LUN name hddv class scsi target 3 lun 2 name hddv class scsi target 3 lun 3 Check the LUN definition overlap between sd conf and hddv conf by executing hddvchk_conf command The message example for Check NG means that target 3 is defined in both files and the procedures must be performed again from procedure 8 b 4 The followings are examples of execution of the FJS Vhddv package installation to opt The path of the hddvchk conf command depends on the installed directory Check OK to procedure 8 b 7 opt FJSVhddv bin hddvchk conf Return kernel drv sd conf and ke
70. device ETERNUS3000 ETERNUS6000 GR700 800 series with new WWN of replaced PCI card See Operation guide or User s Manual of each product for details Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration b Reconnect the replaced PCI card with changed zoning setting by using the following commands cfgadm c disconnect pcipsy3 COM00 PCI slot02 Return cfgadm c configure pcipsy3 COM00 PCI slot02 lt Return gt cfgadm pcipsy3 COM00 PCI slot02 lt Return gt Ap Id Type Receptacle Occupant Condition pcipsy3 COMO00 PCI slot02 mult hp connected configured ok If you don t use SAN management function of the above products No procedure is necessary Go to step 8 for Fibre Channel Card PW028FC4 PW028FC5 No procedure is necessary Go to step 8 7 Start applications Restart the stopped applications with the following operations a Start vold sh etc init d volmgt start lt Return gt b Restart the usage of the device stopped in procedure 4 b If the device is a disk unit file system operation Mount the unmounted filesystem and resume usage mount F ufs dev dsk c2t3d0s3 export home Return mount F
71. ds It changes into the state where the PCI card is usable Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration Connect equipment to PCI cards Connect equipment to PCI cards Setup driver a Add the driver configuration of PCI cards Add the driver configuration of PCI cards b Add the configuration of device driver Add the configuration of device driver in the case of file devices Setup redundant system Configure the environmental configuration of a new redundant system and activate Setup applications Configure the high layer products and services about a new redundant system Start applications Start high layer products and services used by the new redundant system Extension of PCI cards to redundant system When extending PCI cards to the existing redundant system it is possible to extend PCI cards without stopping services and applications The extension procedure of PCI card to existing redundant system is as follows File devices procedures l 2 10 Setup I O devices which will be added to system Setup devices which will be added to system in the case of file devices on Solaris 8 OE Stop the hardware monitoring of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration Stop the daemons If you use the Fibre Channel Card PW028FC4 PW028FCS s
72. dv 3 0 g Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s7 gt devices pci 89 2000 fibre channel 1 hddv 9 3 0 h 9 Start format command and configure labeling and partition slice information calculated at 8 b 8 to the logical path name of each extended LUN c Add configure of PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver No procedure is necessary Go to step 9 Fibre Channel driver PW008FC3U PW008FC2U GP7B8FC1U No procedure is necessary Go to step 9 Fibre Channel Card PW028FC4 PW028FC5 Bind the auto mapped target connected with the card and modify the entries of lpfc conf to match the requirements of the extended Fibre Channel Controller These procedures are reflected on next boot time In the following indicate to bind the auto mapped target with the card usr sbin Ipfc lputil lt Return gt LightPulse Common Utility for Solaris SPARC Version 1 6a8 03 03 2004 Copyright c 2003 Emulex Network Systems Inc Emulex Fibre Channel Host Adapters Detected 2 Host Adapter 0 Ipfc11 is an LP9802 Ready Mode Host Adapter 1 Ipfc9 is an LP9802 Ready Mode MAIN MENU 1 List Adapters 2 Adapter Information 3 Firmware Maintenance 4 Reset Adapter 5 Persistent Bindings 0 Exit Enter choice gt 5 Return PERSISTENT BINDINGS MENU 1 Display Current Bindings 2 Display All Nodes 3 Duplicate Adapter Bindings 4 Bind Target Manually 5 Bind Auto
73. dvadm M Return Operation with non active extension procedures to procedure 1 d hddvadm M Return already ID 1 exists devices pci 1d 2000 fibre channel 1 sd 9 1 0 a raw GR720 19 d When performing pre defined extension the same target number cannot be registered in sd conf and hddv conf so delete or comment out the conflicting definition parts in sd conf file when definition in the two files conflict with each other The following example shows the case that target 1 is repeated vi kernel drv sd conf lt Return gt name sd class scsi target 0 lun 0 name sd class scsi target 1 lun 0 name sd class scsi Delete or comment out target 1 lun 1 conflicting definition parts in sd hddv name sd class scsi target 1 lun 2 A target number that is already used by the sd driver of the system disk or CD ROM cannot be deleted In this case change the target number in hddv conf and of the disk array device so that it does not conflict with sd conf e Edit the kernel drv hddv conf file created automatically by the hddvadm M command and remove comments of the LUN definition to pre define and the LUN definition currently being used with non active extension procedures If case active extension has been performed before and the LUN definition to extend the next procedures 1 f 1 g are not needed The following example shows target 3 lun 0 2 for pre defined extension an
74. dy Mode MAIN MENU 1 List Adapters 2 Adapter Information 3 Firmware Maintenance 41 4 Reset Adapter 5 Persistent Bindings 0 Exit Enter choice gt 5 Return PERSISTENT BINDINGS MENU 1 Display Current Bindings 2 Display All Nodes 3 Duplicate Adapter Bindings 4 Bind Target Manually 5 Bind Automapped Targets 6 Delete Binding s 0 Return to Main Menu Enter choice gt 5 lt Return gt 0 Ipfc11 1 Ipfc9 Select an adapter gt 0 lt Return gt Automapped FCP Nodes Node Target WWPN WWNN Automapped FCP Node 0 50 06 0e 80 00 10 04 92 50 06 0e 80 00 10 04 92 Automapped FCP Node 1 50 06 0e 80 00 10 04 90 50 06 0e 80 00 10 04 90 Press the Enter or Return key to continue Bind all auto mapped targets Y N 2 Y Return Select Binding Method 1 By Node Name 2 By Port Name 3 ByD ID 0 Cancel Enter Choice gt 2 Return 10 Connect to redundant system The following procedure is described for MPHD GRMPHD If you use the redundancy software other than MPHD GRMPHD see the document for each product To connect the added PCI card to the redundant system execute the following command and confirm that the path name determined in procedure 9 b 3 in underlined letters is added usr sbin mplbconfig nv Return Phase 1 checking dev rdsk Phase 2 checking multi path Phase 3 analyzing path Path Action Element path L
75. e 6 Replace the PCI card Replace the PCI card 7 Connect the PCI card It changes into the state where the PCI card is usable 8 Start applications Start high layer products and services used by the replaced PCI card 9 Start the daemons If you use the Fibre Channel Card PWO28FCA4 PWO28FCS5 start the daemons 10 Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration Network devices procedures 1 Stop the hardware monitoring of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration 2 Specify the replacing PCI card Specify the devices and interfaces that are influenced by replacing PCI cards 3 Stop applications Stop services and high layer products that are influenced by replacing PCI cards 4 Disconnect the PCI card It changes into the state where the PCI card is exchangeable 5 Replace the PCI card Replace the PCI card 6 Connect the PCI card It changes into the state where the PCI card is usable 7 Start applications Start high layer products and services used by the replaced PCI card 8 Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration 2 1 2 Replacement of PCI cards on redundant system When exchanging PCI cards with redunda
76. e and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration Connect the PCI card Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the replacement slot position Note that the push button is only effective in multiuser mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy3 COM00 PCEslot02 Return cfgadm pcipsy3 COM00 PCI slot02 Return Ap Id Type Receptacle Occupant Condition pcipsy3 COMO00 PCI slot02 mult hp connected configured ok When exchanging Fibre Channel cards the following operations are also required for PCI Fibre Channel PW008FC3U PW008FC2U GP7B8FC10U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility a Refresh current condition on management window SAN Storage Area Network management function searches and indicates replaced PCI card This function will adjust switch zoning and host affinity function of disk array
77. e kernel drv hddv conf file created automatically by the hddvadm M command and remove comments of the LUN definition to pre define and the LUN definition currently being used with non active extension procedures If case active extension has been performed before and the LUN definition to extend the next procedures 1 f 1 g are not needed If the target number of each disk array controller of a redundant system differs add definition for each target number The following example shows target 3 lun 0 2 for pre defined extension and disk array device GR720 target 1 in use with non active extension procedures being registered Is kernel drv hddv conf Return kernel drv hddv conf vi kernel drv hddv conf lt Return gt name hddv class scsi target 0 lun 0 name hddv class scsi target 0 lun 7 name hddv class scsi target 1 lun 0 name hddv class scsi target 1 lun 1 Remove comments of the LUN in use name hddv class scsi target 1 lun 2 name hddv class scsi target 3 lun 0 name hddv class scsi target 3 lunz1 Remove comments of extended LUN name hddv class scsi target 3 lun 2 name hddv class scsi target 3 lun 3 f Check the LUN definition overlap between sd conf and hddv conf by executing hddvchk_conf command The message example for Check NG means that target 3 is defined in both files and the procedures must be performed again from procedure 1 d The following
78. e administration hardware monitoring daemon usr sbin FJSV madm prephp Return 3 If you use the Fibre Channel Card PW028FC4 PW028FCS execute the following The daemons will be stopped etce rc0 d K10hbanyware stop Return ete rc0 d K10hbanywareDisc stop Return 4 Add PCI cards Add PCI cards according to the following procedure for PCI Fibre Channel PW008FC3U PW008FC2U GP7BSFCIU If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility No procedure is necessary Go to step 5 If you don t use SAN management function of the above products To add PCI cards with the following configurations Fibre Channel switch SN200 series and disk array device ETERNUS3000 ETERNUS6000 GR700 800 series need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on SN200 series If the Host Affinity function of ETERNUS3000 ETERNUS6000 GR700 800 series is used For details see SN200 Series Affinity User s Guide or ETERNUSmgr User Guide GRmgr User Guide If you use the Fibre Channel switch or disk array device other than described above see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digi
79. e administration hardware monitoring daemon Use the following command to stop the hardware monitoring daemon of machine administration usr sbin FJSVmadm prephp Return 2 If you use the Fibre Channel Card PW0O28FC4 PW028FCS execute the following The daemons will be stopped etce rc0 d K10hbanyware stop Return ete rc0 d K10hbanywareDisc stop Return 3 Specify the replacing PCI card Follow the instructions below and determine the interface name of the path connecting the target PCI card and I O devices and the connected I O device The following procedure is described for MPHD GRMPHD If you use the redundancy software other than MPHD GRMPHD see the document for each product a Determine from the WARNING messages output on the console the interface name of the path connecting the target PCI card and I O devices and the connected I O device In the example below fjpfca3 interface name of the path connecting the target PCI card and I O devices and hddvlis the LUN Logical Unit Number of the disk array device connected to fjpfca3 WARNING pci 8d 2000 fibre channel 1 fjpfca3 Hard Error PCI DMA error WARNING pci 8d 2000 fibre channel 1 hddv 1 0 hddv1 SCSI transport failed reason reset giving up NOTICE mphd0 I O path switchover succeeded pci 8d 2000 fibre channel 1 hddv 1 0 gt pci 89 2000 fibre channel 1 hddv 92 0 The following procedures are also e
80. e and the LUN definition to extend the next procedures 1 f 1 g are not needed The following example shows target 3 lun 0 2 for pre defined extension and disk array device GR720 target 1 in use with non active extension procedures being registered Is kernel drv hddv conf Return kernel drv hddv conf vi kernel drv hddv conf lt Return gt name hddv class scsi target 0 lun 0 name hddv class scsi target 0 lun 7 name hddv class scsi target 1 lun 0 name hddv class scsi target 1 lun 1 remove comments of the LUN in use name hddv class scsi target 1 lun 2 name hddv class scsi target 3 lun 0 name hddv class scsi target 3 lun 1 remove comments of extended LUN name hddv class scsi target 3 lun 2 name hddv class scsi target 3 lun 3 Check the LUN definition overlap between sd conf and hddv conf by executing hddvchk_conf command The message example for Check NG means that target 3 is defined in both files and the procedures must be performed again from procedure 8 b 4 The followings are examples of execution of the FJSVhddv package installation to opt The path of the hddvchk conf command depends on the installed directory Check OK to procedure 8 b 7 opt FJSVhddv bin hddvchk conf Return kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 8 b 4 opt FJSVhddv bin hddvchk conf Return NGID 3 7 Execute th
81. e channel 1 driver name fjpfca binding name pcilOcf 105c bus addr 1 instance 1 class obp device name fibre channel VVVVVVVV usr sbin FJSVpfca fc_adm l fjpfcal Return 2 Edit kernel drv fjpfca conf and add the following property definitions to the instance number from 9 a 1 port topology classification link speed Define this property when PCI cards using Fabric connection are added This definition is not needed with FC AL connection fcp bind target connection of target ID and WWN Define this property when disk array equipment using Fabric connections are added This definition is not needed with FC AL connection max throttle number of maximum SCSI command issue This property is defined for restriction on simultaneous issue of SCSI commands The necessity and value of this definition depends on the specification of the disk array device so refer to the user s manual of each equipment The following example shows the definition of a fjpfcal PCI card instance number 1 and disk array equipment target 3 being added vi kernel drv fjpfca conf Return port fjpfca0 nport fjpfcal nport spl1 lt added fcp bind target fjpfca0t0 0x50000e 1000002501 fjpfca1t3 0x50000e1000002ab0 lt added max throttle fjpfca0t0 128 fjpfca1t3 240 lt added 3 Execute the following command to make the fjpfca driver take in the added part usr sbin FJS Vpfca fc config v
82. e following command in order to make the hard disk driver set up the LUN added update_drv f hddv lt Return gt In case disk array equipment currently used exists the following messages are displayed but do not effect performance update drv f hddv Return Cannot unload module hddv Will be unloaded upon reboot Forcing update of hddv conf 8 After saving execution result of the prtpicl 1M command get the physical path name of added LUN by taking the difference between the result and the information obtained in 3 a and calculate the corresponding logical path name under dev dsk In the following example c3t3d0 is logical path name of added lun 0 The logical path name of lun 1 2 is set to c3t3d1 c3t3d2 respectively because cXtYdZmeans X PCI card number Y target number Z lun number prtpicl v gt tmp prtpicl post Return diff tmp prtpicl pre tmp prtpicl post grep devfs path grep hddv Return gt devfs path pci289 2000 fibre channelQ 1 hddv Q9 3 0 gt devfs path pci 89 2000 fibre channel 1 hddv 93 1 gt devfs path pci 89 2000 fibre channel 1 hddv 9 3 2 Is 1 dev dsk grep pci 89 2000 fibre channel 1 hddv 3 0 Return Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s0 gt devices pci 89 2000 fibre channel 1 hddv 3 0 a Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s1 gt devices pci 89 2000 fibre channel 1 hddv 3 0 b Irwxrwxrwx 1 ro
83. eded The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digit of the WWPN in hexadecimal form The top eight digit are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the replacement card is 100000000e244061 0e24 4061 for Fibre Channel Card PW028FCA4 PW028FC5 To replace PCI cards with the following configurations Fibre Channel switch and disk array device need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on Fibre Channel switch If the Host Zoning function of disk array device is used For details see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card These characters represent the bottom twelve digit of the WWPN in hexadecimal form The top four digit are fixed to 1000 in hexadecimal form For example if the following label is shown on the back of the card the WWPN of the replacement card is 10000000c9366037 IEEE 0000c9366037 Note When changing Affinity configuration on SN200 series or other Fibre Channel switch I O to other devices is effected by the change and may result in temporal errors I O to disk
84. edure 3 b cfgadm x led fault mode blink pcipsy18 ROB01 PCI slot02 Return d After checking that the READY LED on the PCI slot to add the PCI card is off add PCI cards and connect cable to devices This operation is performed by our customer support e Use the cfgadm 1M command to check that the slot status of the added PCI card changed to disconnected unconfigured cfgadm pcipsy18 ROBO1 PCI slot02 Return Ap Id Type Receptacle Occupant Condition pcipsy18 ROB0O1 PCI slot02 unknown disconnected unconfigured unknown 4 Connect the PCI card Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the extension slot position Note that the push button is only effective in multiuser mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy18 R0B01 PCI slot02 Return cfgadm pcipsy18 ROBO1 PCIZslot02 Return Ap Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCI slot02 mult hp connected configured ok If you use the Fibre Channel Card PW028FC4 PW028FC5S execute the following The daemons will be started etc rc2 d S99hbanyware start Return
85. enn UAE 75 5 NOTES i t up eaten Iac n LR tea esi ho teres bobo da erre pede tit ove wate 75 5 2 testrictions i eH pae Nae ee a ae an inel cec sad ieaie ae Peu ecd aee lasse dts 76 Chapter 1 PCI cards with support for PCI Hot Plug This chapter shows the list of PCI cards supporting PCI Hot Plug feature enclosed with a PCI card in this chapter can be downloaded from the following URLs The driver patches http software fujitsu com en security index html The option driver packages enclosed with a PCI card http primepower fujitsu com en driver e index html The driver patches and the option driver packages Please contact our customer service about the latest information of the driver patches and the option driver packages 1 1 PCI cards for file devices The following list describes the PCI cards that support PCI Hot Plug for file devices Please ask the sales staff about the latest information of PCI Hot Plug supporting cards Software of driver Operating Supporting Functions Redundant Functions Product ID clock 1 Hot Hot Software driver name replacement expansion X6540A U Solaris OE glm 2 33MHz NG 3 NG 3 none Multipath Disk Control 2 2 1 or X6541A U later 5 Solaris OE glm 2 33MHz NG 4 OK 10 X6541A X GR Multipath Driver 1 0 1 or later Multipath Disk Control 2 2 1 or GP7B8SC1U later 5 Solaris OE glm 2 33MHz OK 11 OK SCSI GP7B8SC
86. ension the same target number cannot be registered in sd conf and hddv conf so delete or comment out the conflicting definition parts in sd conf file when definition in the two files conflict with each other The following example shows the case that target 1 is repeated vi kernel drv sd conf Return name sd class scsi target 0 lun 0 name sd class scsi target 1 lun 0 name sd class scsi Delete or comment out target 1 lun 1 conflicting definition parts in sd hddv name sd class scsi target 1 lun 2 A target number that is already used by the sd driver of the system disk or CD ROM cannot be deleted In this case change the target number in hddv conf and of the disk array device so that it does not conflict with sd conf e Edit the kernel drv hddv conf file created automatically by the hddvadm M command and remove comments of the LUN definition to pre define and the LUN definition currently being used with non active extension procedures If case active extension has been performed before and the LUN definition to extend the next procedures 1 f 1 g are not needed If the target number of each disk array controller of a redundant system differs add definition for each target number The following example shows target 3 lun 0 2 for pre defined extension and disk array device GR720 target 1 in use with non active extension procedures being registered Is kernel drv hddv conf
87. ent system is busy try again disconnect failed d To confirm the slot position at replacement operation blink the ALARM LED of Ap Id displayed in procedure 4 a cfgadm x led fault mode blink pcipsy3 COM00 PCI slot02 Return 5 Replace the PCI card Replace the PCI card disconnected in 4 with a replacement card and connect cable to devices This operation is performed by our customer support When exchanging Fibre Channel cards the following operations are also required for PCI Fibre Channel PW008FC3U PW008FC2U GP7B8FC1U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility No procedure is necessary Go to step 7 If you don t use SAN management function of the above products To replace PCI cards with the following configurations Fibre Channel switch SN200 series and disk array device ETERNUS3000 ETERNUS6000 GR700 800 series need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on SN200 series If the Host Affinity function of ETERNUS3000 ETERNUS6000 GR700 800 series is used For details see SN200 Series Affinity User s Guide or ETERNUSmgr User Guide GRmgr User Guide If you use the Fibre Channel switch or disk array device other than described above see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI
88. ep 9 Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration Set up driver Add the configuration of driver according to the following operation As for file related drivers additional configuration of the PCI card driver and the device driver is needed a Add configurations of PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver Configuration is not required Fibre Channel driver PW008FCZU W008FC2X GP7B8FC1U GP7B8FC1X 1 After saving execution result of prtpicl 1M command take the difference between the result and the information obtained in procedure 4 a to calculate the instance number of the fjpfca driver of the PCI card added In the following example instance number is 1 indicating that the interface name of the PCI card added is fjpfcal Making the LED of the PCI card blink using the fc adm command can specify the relation between the instance number and the slot position The relation between the WWPN and instance number of the PCI card added can be known from the fc info a command prtpicl v gt tmp prtpicl pfca Return diff tmp prtpicl pre tmp prtpicl pfca more Return status okay d
89. ermined in 8 b 8 as a parameter The italic letters below are the new multipath names and the underlined letters are the path names that compose the multipath MPHD usr opt F JS Viomp bin iompadm c mphd new dev rdsk c3t3d0s2 dev rdsk c4t3d0s2 Return IOMP dev FJS Vmphd fiomp adm2 Node dev FJS V mphd rdsk mphd2s0 dev FJS V mphd rdsk mphd2s1 dev FJS V mphd rdsk mphd2s2 dev FJS V mphd rdsk mphd2s3 dev FJS V mphd rdsk mphd2s4 dev FJS V mphd rdsk mphd2s5 dev FJS V mphd rdsk mphd2s6 dev FJS V mphd rdsk mphd2s7 GRMPD usr sbin mplbconfig nv Return Phase 1 checking dev rdsk Phase 2 checking multi path Phase 3 analyzing path Path Action Element path LUN Storage Zone mplbO exist c1t4d0s2 c2t3d0s2 0 GR720464f 12340001 mplb8 new c3t3d0s2 c4t3d0s2 0 GR8404550 000213 usr sbin mplbconfig av Return IOMP dev FJS Vmplb fiomp adm8 Node dev FJS V mplb rdsk mplb8s0 dev FJSV mplb rdsk mplb8s1 dev FJS V mplb rdsk mplb8s2 dev FJS V mplb rdsk mplb8s3 dev FJS V mplb rdsk mplb8s4 dev FJS V mplb rdsk mplb8s5 dev FJS V mplb rdsk mplb8s6 dev FJS V mplb rdsk mplb8s7 10 Setup applications Add configuration for the application using the logical path name determined in 8 filesystem operation Create file system To mount at system start add an entry in the etc vfstab file newfs dev FJSV mphd rdsk mphd2s6
90. es pci 89 4000 scsi 2 st 5 0 hn 4 If the device is a disk device start the format 1M command and configure labeling and partition slice information of the logical path name of added disks If the device is a tape device this procedure is not needed Disk array devices hard disk driver hddv This is an example where an disk array device at target 3 lun 0 2 is pre defined in procedure 4 If this is the first time to pre define a disk array device begin with procedure 8 b 1 If this is not the first time to pre define a disk array device begin with procedure 8 b 5 1 Check if kernel drv hddv conf exists Check if kernel drv hddv conf exists If it exists the hard disk driver is being installed so start from procedure 8 b 3 If it does not exist the system is being operated with none active extension procedures so start from procedure 8 b 2 Not exist new install to procedure 8 b 3 Is kernel drv hddv conf Return kernel drv hddv conf No such file or directory Exists operation with non active extension procedures to procedure 8 b 2 Is kernel drv hddv conf Return kernel drv hddv conf 2 When operating with non active extension procedures execute the following commands and confirm that kernel drv hddv conf is deleted hddvadm i Return Is kernel drv hddv conf Return kernel drv hddv conf No such file or directory 3 Use the following command to create hddv
91. evfs path pci 8d 2000 fibre channel 1 driver name fjpfca binding name pcilOcf 105c bus addr 1 instance 1 class obp device gt name fibre channel usr sbin FJSV pfca fc adm l fjpfcal Return 2 Edit kernel drv fjpfca conf and add the following property definitions to the instance number from 9 a 1 port topology classification link speed Define this property when PCI cards using Fabric connection are added This definition is not needed with FC AL connection fcp bind target connection of target ID and WWN Define this property when disk array equipment using Fabric connections are added This definition is not needed with FC AL connection max throttle number of maximum SCSI command issue This property is defined for restriction on simultaneous issue of SCSI commands The necessity and value of this definition depends on the specification of the disk array device so refer to the user s manual of each equipment The following example shows the definition of a fjpfcal PCI card instance number 1 and disk array equipment target 3 being added VVV N Y vi kernel drv fjpfca conf lt Return gt port fjpfca0 nport fjpfca1 nport sp1 lt added fcp bind target fjpfca0t0 0x50000e 1000002501 fjpfca1t3 0x50000e1000002ab0 lt added max throttle fjpfca0t0 128 fjpfca1t3 240 lt added 3 Execute the following command to make the fjpfca driver take in the added part
92. for fjpfca conf sd conf of host can be added from SAN management window automatically See Operation guide or User s Manual of each product for details Before using this function the following procedure 9 a 1 and 9 a 2 except fcp bind target setting are necessary Moreover begin from procedure 9 a 3 after executing this function If you don t use SAN management function of the above products No procedure is necessary Go to step 9 for Fibre Channel Card PW028FC4 PW028FC5 No procedure is necessary Go to step 9 Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration Setup driver Add the configuration of driver according to the following operation As for file related drivers additional configuration of the PCI card driver and the device driver is needed a Add configurations of PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver Configuration is not required Fibre Channel driver PW008FCZU W008FC2X GP7B8FC1U GP7B8FC1X 1 After saving execution result of prtpicl 1M command take the difference between the result and the information obtained in procedure 4 a to calculate the instance number of the fjpfca
93. g Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration Setup driver Add the configuration of driver according to the following operation As for file related drivers additional configuration of the PCI card driver and the device driver is needed a Add configurations of PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver Configuration is not required Fibre Channel driver PW008FCZU W008FC2X GP7B8FC1U GP7B8FC1X 1 After saving execution result of prtpicl 1M command take the difference between the result and the information obtained in procedure 4 a to calculate the instance number of the fjpfca driver of the PCI card added In the following example instance number is 1 indicating that the interface name of the PCI card added is fjpfcal Making the LED of the PCI card blink using the fc adm command can specify the relation between the instance number and the slot position The relation between the WWPN and instance number of the PCI card added can be known from the fc info a command prtpicl v gt tmp prtpicl pfca Return diff tmp prtpicl pre tmp prtpicl pfca more Return status okay devfs path pci 8d 2000 fibr
94. guring parameters in driver conf 4 file and replacing or extending the PCI card by using the PCI Hot Plug feature parameters which configured in driver conf 4 file may not be valid To change the initial parameters after the PCI card being replaced or extended by using PCI Hot Plug feature please refer to the solution of each PCI card shown below Each PCI card manual 1 has the description of setting by the ndd command Please refer it Product D Software of driver Resolution Functions X1033A U Please make a setting by the ndd command X1033A X Solaris OE hme Or please reboot a system after changing the kernel drv hme conf 10 100BASE T file PWOOSFEI 10 100BASE T FUJITSU PCI 10 100 Ethernet 1 0 or later enclosed with a PCI card Please make a setting by the ndd command Or please reboot a system after changing the kernel drv fjfe conf file PWOOSQEI 10 100B ASE T x 4 FUJITSU PCI Quad 10 100 Ethernet 1 0 or later enclosed with a PCI card PWO08QE2 10 100BASE T x 4 FUJITSU PCI Quad 10 100 Ethernet 1 1 or later enclosed with a PCI card Please make a setting by the ndd command Or please reboot a system after changing the kernel drv fjqe conf file PP028GE1U PPO28GEIX 1000BASE SX FUJITSU PCI Gigabit Ethernet The ndd command is not supported 1 4 1 or later Please reboot a system after changing the kernel drv fjge conf PWO08GEIU enclosed with a PCI c
95. h pci 8d 2000 fibre channel 1 driver name fjpfca binding name pcilOcf 105c bus addr 1 instance 1 class obp device name fibre channel VVVVVVVV usr sbin FJSVpfca fc_adm l fjpfcal Return 2 Edit kernel drv fjpfca conf and add the following property definitions to the instance number from 8 a 1 port topology classification link speed Define this property when PCI cards using Fabric connection are added This definition is not needed with FC AL connection fcp bind target connection of target ID and WWN Define this property when disk array equipment using Fabric connections are added This definition is not needed with FC AL connection max throttle number of maximum SCSI command issue This property is defined for restriction on simultaneous issue of SCSI commands The necessity and value of this definition depends on the specification of the disk array device so refer to the user s manual of each equipment The following example shows the definition of a fjpfcal PCI card instance number 1 and disk array equipment target 3 being added vi kernel drv fjpfca conf Return port fjpfcaO nport fjpfcal nport sp1 lt added fcp bind target fjpfca0t0 0x50000e 1000002501 fjpfca1t3 0x50000e1000002ab0 lt added max throttle fjpfca0t0 128 fjpfcalt3 240 lt added 3 Execute the following command to make the fjpfca driver take in the added part usr sbin F
96. h the configure option to Ap Id or the push button of the PCI bus slot is pushed Then a extended PCI card is connected In addition a push button is valid only in multiple user mode Please confirm that the slot status is connected configured by using the cfgadm 1M command after the above mentioned procedure cfgadm c configure pcipsy18 ROB01 PCI slot02 Return cfgad m pcipsy18 ROBO1 PCEIZslot02 Return Ap Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCI slot02 pci pci hp connected configured ok 4 Start the hardware monitoring of the Machine Administration Please execute the following command in order to update configuration information and restart the hardware monitoring of the Machine Administration usr sbin FJSV madm postphp Return 5 Connect equipment to PCI cards The extended PCI card is connected with network equipment by the cable 6 Setup driver The driver configuration is added by following operations a The interface name is investigated in order to configure drivers and high layer products Please save the result of the prtcicl 1M command and obtain the difference information between current result 72 and the result taken at 2 a Then the driver instance number for the extended PCI card is obtained In the following example since the instance number is 1 it can be determined that the interface name of the extended PCI card is hmel
97. hannel 1 driver name fjpfca binding name pcilOcf 105c bus addr 1 instance 1 class obp device name fibre channel VVVVVV VV VV 2 Edit the kernel drv fjpfca conf file and add the following property definitions to the instance number determined in 8 a 1 port type of topology link speed Defined when adding PCI cards with Fabric connection This definition is not needed with FC AL connection fcp bind target connection between target ID and WWN Defined when adding disk array with Fabric connection This definition is not needed with FC AL connection max throttle maximum issued SCSI commands Defined to limit the number of SCSI commands issued at the same time The need and value of this definition depends on the disk array device so refer to the manuals of each disk array device Below is an example where definitions for a PCI card at fjpfcal instance number 1 and a disk array device at target 3 is added vi kernel drv fjpfca conf lt Return gt port fjpfcaO nport fjpfcal nport sp1 lt add fcp bind target fjpfca0t0 0x50000e1000002501 fjpfcalt3 0x50000e1000002ab0 lt add max throttle fjpfca0t0 128 fjpfcalt3 240 lt add 3 Execute the following command to make the fjpfca driver set up the additional part usr sbin F JS Vpfca fc config v Return Fibre Channel driver PW028FCA4 PW028FC5 Configuration is not required Compare the output of prtpi
98. he ALARM LED of Ap Id displayed in procedure 4 b cfgadm x led fault mode blink pcipsy15 COM01 PCI slot02 Return d After checking that the READY LED on the PCI slot to add the PCI card is off add PCI cards and connect cable to devices This operation is performed by our customer support e Use the cfgadm 1M command to check that the slot status of the added PCI card changed to disconnected unconfigured cfgadm pcipsy15 COM01 PCI slot02 Return Ap Id Type Receptacle Occupant Condition pcipsy15 COMO01 PCI slot02 unknown disconnected unconfigured unknown 5 Connect PCI cards Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the extension slot position Note that the push button is only effective in multiuser mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy15 COM01 PCIT slot02 Return cfgadm pcipsy15 COMO1 PCEslot02 Return Ap Id Type Receptacle Occupant Condition pcipsy15 COMOI PCTIZslot02 mult hp connected configured ok If you use the Fibre Channel Card PWO28FCA PWO028FC5 execute the following The daemons will be started Jetc r
99. hich will be added to system Setup devices which will be added to system in the case of file devices on Solaris 8 OE Stop the hardware monitoring of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration Stop the daemons If you use the Fibre Channel Card PW028FC4 PW028FCS stop the daemons Add PCI cards Extend PCI cards Connect PCI cards It changes into the state where the PCI card is usable 10 11 Start the daemons If you use the Fibre Channel Card PW028FC4 PW028FC5 start the daemons Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration Preparation to recognize devices for Fibre Channel If you use SAN management function preparation to recognize devices Set up driver a Add the driver configuration of PCI cards Add the driver configuration of PCI cards b Add the configuration of device driver Add the configuration of device driver in the case of file devices Setup applications Configure the high layer products and services about extended PCI cards Start applications Start high layer products and services used by the replaced PCI card Network devices procedures 1 2 2 2 2 Setup I O devices which will be added to system Setup devices which will be added to system in the case of file devices on Solaris 8 OE Stop the hardware monitor
100. ibre Channel Card PW028FC4 PW028FCS stop the daemons Add PCI cards Extend PCI cards Connect PCI cards It changes into the state where the PCI card is usable Start the daemons If you use the Fibre Channel Card PWO28FCA4 PWO28FCS5 start the daemons Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration Preparation to recognize devices for Fibre Channel If you use SAN management function preparation to recognize devices Setup driver a Add the driver configuration of PCI cards Add the driver configuration of PCI cards 10 11 12 b Add the configuration of device driver Add the configuration of device driver in the case of file devices Setup redundant system Configure the environmental configuration of a new redundant system and activate Setup applications Configure the high layer products and services about a new redundant system Start applications Start high layer products and services used by the new redundant system Network devices procedures l 2 9 10 22 3 Setup I O devices which will be added to system Setup devices which will be added to system in the case of file devices on Solaris 8 OE Stop the hardware monitoring of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration Add PCI cards Extend PCI cards Connect PCI car
101. ill be started Jetc rc2 d S99hbanyware start Return etc rc2 d S99hbanywareDisc start Return Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJS Vmadm postphp Return Preparation to recognize devices for Fibre Channel When the Host Affinity function is being used with Fibre Channel SN200 series Host Affinity must be configured for the device added for PCI Fibre Channel PW008FC3U PW008FC2U GP7B8FC1U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility WWPN zoning of SN200 series Host affinity of Array disk device ETERNUS3000 ETERNUS6000 GR700 800 series and binding setting for fjpfca conf sd conf of host can be added from SAN management window automatically See Operation guide or User s Manual of each product for details Before using this function the following procedure 9 a 1 and 9 a 2 except fcp bind target setting are necessary Moreover begin from procedure 9 a 3 after executing this function If you don t use SAN management function of the above products No procedure is necessary Go to step 9 for Fibre Channel Card PW028FC4 PW028FC5 No procedure is necessary Go to st
102. ine Control Function of SynfinityLink Notes Because of the dynamic extension if the environment of SynfinityLink or PRIMECLUSTER GLS has been modified system reboot is not required When using other redundant software please refer to manuals of each product Moreover when extending PCI cards to the existing redundant configuration system it is unnecessary to operate about the high layer products Consequently PCI cards can be extended without stopping the high layer products 1 Stop the hardware monitoring of the Machine Administration The hardware monitoring of the Machine Administration is stopped by using the following command 71 usr sbin FJSV madm prephp Return 2 Add PCI cards a Before extending the PCI card please save the result current configuration information of the prtpicl 1M command prtpicl v gt tmp prtpicl pre Return b Check the status of the slot Subsequent examples describe extension of PCI cards to the slot ROBO1 PCI slot02 The ROBO1 PCI slot02 indicates the I O cabinet number 0 the PCI Disk Box number 1 physical slot number 02 In the extension of a new redundant system although two or more PCI cards are extended whenever you add a PCI card please perform from procedure 2 b to 3 Connect PCI cards The relationship of the position of a PCI card and format of Ap Id is shown below N an integral number Locati
103. ing of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration Add PCI cards Extend PCI cards Connect PCI cards It changes into the state where the PCI card is usable Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration Connect equipment to PCI cards Connect equipment to PCI cards Set up driver a Add the driver configuration of PCI cards Add the driver configuration of PCI cards b Add the configuration of device driver Add the configuration of device driver in the case of file devices Setup applications Configure the high layer products and services about extended PCI cards Start applications Start high layer products and services used by the replaced PCI card Extension of PCI cards with new redundant system The procedure of using the extended PCI card and establishing a new redundant configuration is as follows Please refer to Chapter 3 File devices or Chapter 4 Network devices about detailed extending procedure of PCI cards File devices procedures l 2 Setup I O devices which will be added to system Setup devices which will be added to system in the case of file devices on Solaris 8 OE Stop the hardware monitoring of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration Stop the daemons If you use the F
104. is 9 OE or later system are explained here 1 Stop the machine administration hardware monitoring daemon Execute the following commands to stop the machine administration hardware error monitoring daemon usr sbin FJSVmadm prephp lt Return gt 2 If you use the Fibre Channel Card PWO28FC4 PW028FCS execute the following The daemons will be stopped 3t etc rc0 d K10hbanyware stop Return letc rc0 d K 10hbanywareDisc stop Return 3 Add PCI cards Add PCI cards according to the following procedure for PCI Fibre Channel PW008FC3U PW008FC2U GP7B8FC10U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility No procedure is necessary Go to step 7 If you don t use SAN management function of the above products To add PCI cards with the following configurations Fibre Channel switch SN200 series and disk array device ETERNUS3000 ETERNUS6000 GR700 800 series need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on SN200 series If the Host Affinity function of ETERNUS3000 ETERNUS6000 GR700 800 series is used For details see SN200 Series Affinity User s Guide or ETERNUSmgr User Guide GRmgr User Guide If you use the Fibre Channel switch or disk array device other than described above see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is ne
105. known from the twelve characters shown on a label on the back of the card These characters represent the bottom twelve digit of the WWPN in hexadecimal form The top four digit are fixed to 1000 in hexadecimal form For example if the following label is shown on the back of the card the WWPN of the replacement card is 10000000c9366037 IEEE 0000c9366037 Note When changing Affinity configuration on SN200 series or other Fibre Channel switch I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration a Before extension of PCI cards save the current execution result of the prtpicl IM command current configuration at present prtpicl v gt tmp prtpicl pre Return b Check the status of the slot The relationship of the position of a PCI card and format of Ap Id is shown below Location of PCI slot Element Format of Ap Id Solaris 8 OE and Solaris 9 OE Cabinet of Cabinet Number X j PRIMEPOWER System Board Number Y Sol E oen P 900 1500 2500 Physical Slot Number ZZ PS CXMOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN RXBOY PCI slotZZ Solaris 10 OE RXBOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN PCI slotZZ Solaris 10 OE PCI ZZ Below an ex
106. llowing list shows the PCI cards that support PCI Hot Plug for network devices When using below PCI cards please refer to 5 1 1 Network driver configuration changing Please ask the sales staff about the latest information of PCI Hot Plug supporting cards Supporting Functions Software of driver Functions Product ID puis Hot Hot Redundan au driver name clock 1 replacement 13 2 extension SynfinityLink 2 2 X1033A U 3 Ethernet Solaris OE hme 33MHz OK OK X1033A X PRIMECLUSTER GLS 4 0 or later 4 FUJITSU PCI 10 100 gt 2 2 PWOOSFEI Ethernet 1 0 or later 33MHz OK OK Enclosed with a PCI PRIMECLUSTER card 5 GLS 4 0 or later 4 FUJITSU PCI Quad E 22 PwoosQE 10 100 EE 10 07 33Mpz OK OK later enclose with a PRIMECLUSTER PCI card 6 GLS 4 0 or later 4 FUJITSU PCI Quad d UE 22 PWOOSQE2 on 100 a i F 33MHz OK OK ater enc jn with a PRIMECLUSTER PCI card 12 GLS 4 0 or later 4 Supporting Functions functa Product ID Software of driver Operating Hot ai Redundant Software driver name clock 1 replacement e 13 2 extension FUJITSU PCI Gigabit ao de 22 Ethernet 1 4 1 orlater 33MHz PPOZ8GETU enclosed with a PCI 66MHz QR OK PRIMECLUSTER card 7 GLS 4 0 or later 4 FUJITSU PCI Gigabit FPE m 22 Ethernet 1 4 1 orlater 33MHz PWOOSGEI OK OK enclosed with a PCI 66MHz PRIMECLUSTER card 7 GLS 4 0 or later
107. m the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digit of the WWPN in hexadecimal form The top eight digit are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the replacement card is 100000000e244061 0e24 4061 for Fibre Channel Card PW028FCA4 PW028FC5 To replace PCI cards with the following configurations Fibre Channel switch and disk array device need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on Fibre Channel switch If the Host Zoning function of disk array device is used For details see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card These characters represent the bottom twelve digit of the WWPN in hexadecimal form The top four digit are fixed to 1000 in hexadecimal form For example if the following label is shown on the back of the card the WWPN of the replacement card is 10000000c9366037 IEEE 0000c9366037 Note When changing Affinity configuration on SN200 series or other Fibre Channel switch I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers n
108. mapped Targets 6 Delete Binding s 0 Return to Main Menu Enter choice gt 5 lt Return gt 0 Ipfc11 1 Ipfc9 Select an adapter gt 0 lt Return gt Automapped FCP Nodes Node Target WWPN WWNN Automapped FCP Node 0 50 06 0e 80 00 10 04 92 50 06 0e 80 00 10 04 92 Automapped FCP Node 1 50 06 0e 80 00 10 04 90 50 06 0e 80 00 10 04 90 Press the Enter or Return key to continue Bind all auto mapped targets Y N 2 Y Return Select Binding Method 1 By Node Name 2 By Port Name 63 3 By D ID 0 Cancel Enter Choice gt 2 Return 9 Connect to redundant system The following procedure is described for MPHD GRMPHD If you use the redundancy software other than MPHD GRMPHD see the document for each product To connect the added PCI card to the redundant system execute the following command and confirm that the path name determined in procedure 8 b 8 in underlined letters is added usr sbin mplbconfig nv Return Phase 1 checking dev rdsk Phase 2 checking multi path Phase 3 analyzing path Path Action Element path LUN Storage Zone mplbO exist c1t4d0s2 c2t3d0s2 0 GR720464f 12340001 mplb8 add c3t3d0s2 c4t3d0s2 c5t3d0s2 0 GR8404550 000213 3t usr sbin mplbconfig av Return IOMP dev FJS Vmplb fiomp adm8 Node dev FJS V mplb rdsk mplb8s0 dev FJSV mplb rdsk mplb8s1 dev FJS V mplb rdsk mplb8s2 dev FJS V
109. me of the extended PCI card is hmel prtpicl v gt tmp prtpic post lt Return gt diff tmp prtpicl pre tmp prtpicl post more lt Return gt status okay devfs path pci 8d 2000 network 1 driver name hme binding name SUNW hme bus addr 1 instance 1 class obp device name network VVVVV VV V b It confirms that the interface name that obtained above mentioned procedure is an interface name that corresponds to the PCI bus slot of the added PCI card Please carry out this procedure when it cannot identify relationship between an interface name and PCI bus slot by expanding two or more PCI cards especially usr sbin FJSVmadm inst2comp hmel Return pcipsy15 COM01 PCI slot02 c The configuration of each driver is added Please refer to each driver manual for details 7 Setup applications The configuration of extended PCI card used by high layer products services applications is added Please refer to each high layer product manual for details 8 Start applications By the following operations please start the high layer product service application used by the extended PCI card a When using TCP IP interface The example in the case of using TCP IP interface is shown below Furthermore when the high layer program which is using the PCI card needs to restart please refer to the manual of each high layer program usr sbin ifconfig hmel plumb lt Return gt
110. me to pre define a disk array device begin with procedure 1 e a Check if kernel drv hddv conf exists Check if kernel drv hddv conf exists If it exists the hard disk driver is being installed so start from procedure l c If it does not exist the system is being operated with none active extension procedures so start from procedure 1 b Not exist new install to procedure 1 c Is kernel drv hddv conf Return kernel drv hddv conf No such file or directory Exists operation with non active extension procedures to procedure 1 b Is kernel drv hddv conf Return kernel drv hddv conf b When operating with non active extension procedures execute the following commands and confirm that kernel drv hddv conf is deleted hddvadm i Return Is kernel drv hddv conf Return kernel drv hddv conf No such file or directory 35 c Use the following command to create hddv conf file for the pre defined extension function The message examples for operation with non active extension procedures means that disk array device GR720 currently used with taget 1 lun 0 is registered twice on the sd hddv driver New installation to procedure 1 e 3t hddvadm M Return Operation with non active extension procedures to procedure 1 d hddvadm M Return already ID 1 exists devices pci 1d 2000 fibre channel 1 sd 9 1 0 a raw GR720 d When performing pre defined ext
111. monitoring of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration Specify the replacing PCI card Specify the devices and interfaces that are influenced by replacing PCI cards Disconnect from redundant system Disconnect the devices path and interfaces of PCI cards from the redundant system Disconnect the PCI card It changes into the state where the PCI card is exchangeable Replace the PCI card Replace the PCI card Connect the PCI card It changes into the state where the PCI card is usable Connect to redundant system Connect device paths and interfaces of the replaced PCI card to the redundant system Switch back the redundant path Switch back the online path if needed Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration 2 20 Extension of PCI cards This section describes outline to extend PCI cards using PCI Hot Plug feature About software which makes PCI card redundantly please refer to each manual of the software 2 2 1 Extension of PCI cards on non redundant system When not using the redundant configuration software the extension procedure of PCI cards is as follows Please refer to Chapter 3 File devices or Chapter 4 Network devices about detailed extending procedure of PCI cards File devices procedures l 2 Setup I O devices w
112. n 10 Setup redundant system Execute the following commands to create a new redundant system The following procedure is described for MPHD GRMPHD If you use the redundancy software other than MPHD GRMPHD see the document for each product 33 34 In case of MPHD the commands need to be executed for each LUN specifying the path name determined in 9 b 3 as a parameter The italic letters below are the new multipath names and the underlined letters are the path names that compose the multipath MPHD usr opt F JS Viomp bin iompadm c mphd new dev rdsk c3t3d0s2 dev rdsk c4t3d0s2 Return IOMP dev FJS Vmphd fiomp adm2 Node dev FJS V mphd rdsk mphd2s0 dev FJS V mphd rdsk mphd2s1 dev FJS V mphd rdsk mphd2s2 dev FJS V mphd rdsk mphd2s3 dev FJS V mphd rdsk mphd2s4 dev FJS V mphd rdsk mphd2s5 dev FJS V mphd rdsk mphd2s6 dev FJS V mphd rdsk mphd2s7 GRMPD usr sbin mplbconfig nv Return Phase 1 checking dev rdsk Phase 2 checking multi path Phase 3 analyzing path Path Action Element path LUN Storage Zone mplbO exist c1t4d0s2 c2t3d0s2 0 GR720464f 12340001 mplb8 new c3t3d0s2 c4t3d0s2 0 GR8404550 000213 usr sbin mplbconfig av Return IOMP dev FJS Vmplb fiomp adm8 Node dev FJS V mplb rdsk mplb8s0 dev FJSV mplb rdsk mplb8s1 dev FJS V mplb rdsk mplb8s2 dev FJS V mplb rdsk mplb8s3 dev FJS V mplb rdsk mplb8s4 de
113. n addition in order to use IPv6 address please refer to the PRIMECLUSTER GLS 4 1A20 or later manual 8 Switch back the redundant path Please switch back an online communication path if needed 9 Start the hardware monitoring of the Machine Administration Please execute the following commands in order to update configuration information and restart the hardware monitoring of the Machine Administration usr sbin FJSV madm postphp Return 4 2 Extension of PCI cards This section describes extension procedure of the PCI card by using PCI Hot Plug on non redundant system and redundant system 4 2 1 Extension of PCI cards on non redundant system This subsection describes extension procedure of PCI cards without the redundant configuration software 1 Stop the hardware monitoring of the Machine Administration The hardware monitoring of the Machine Administration is stopped by using the following command usr sbin FJSVmadm prephp Return 2 Add PCI cards a Before extending the PCI card please save the result current configuration information of the prtpicl 1M command prtpicl v gt tmp prtpicl pre lt Return gt b Check the status of the slot Subsequent examples describe extension of PCI cards to the slot COMOI PCIZslot02 The COM01 PCI slot02 indicates the cabinet number 0 the system board number 1 physical slot number 02 The relationship of the position of a PCI car
114. n at extension operation blink the ALARM LED of Ap Id displayed in procedure 3 b cfgadm x led fault mode blink pcipsy15 COM01 PCI slot02 Return d After checking that the READY LED on the PCI slot to add the PCI card is off add PCI cards and connect cable to devices This operation is performed by our customer support e Use the cfgadm 1M command to check that the slot status of the added PCI card changed to disconnected unconfigured cfgadm pcipsy15 COM01 PCI slot02 Return Ap Id Type Receptacle Occupant Condition pceipsy15 COM01 PCI slot02 unknown disconnected unconfigured unknown Connect PCI cards Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the extension slot position Note that the push button is only effective in multiuser mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy15 COM01 PCI slot02 Return cfgadm pcipsy15 CO0MO01 P CEslot02 Return Ap Id Type Receptacle Occupant Condition pceipsy15 COMO01 PCI slot02 mult hp connected configured ok If you use the Fibre Channel Card PW028FC4 PW028FC5S execute the following The daemons will
115. nfiguration a Before extension of PCI cards save the current execution result of the prtpicl IM command current configuration at present prtpicl v gt tmp prtpicl pre Return b Check the status of the slot When adding PCI cards to a redundant configuration several PCI cards may be added to the system In this case perform to procedures 4 on each card The relationship of the position of a PCI card and format of Ap Id is shown below 37 38 Location of PCI slot Element Format of Ap Id Solaris 8 OE and Solaris 9 OE Cabinet of Cabinet Number X PRIMEPOWER System Board Number Y gi ae ANS 900 1500 2500 Physical Slot Number ZZ Mee CXMOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN RXBOY PCI slotZZ Solaris 10 OE RXBOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN PCI slotZZ Solaris 10 OE PCI ZZ Below an example of adding a PCI card to chassis number 0 PCI Disk BOX number 1 physical slot number 02 slot in PCI Disk BOX ROBO1 PCI slot02 is explained Check the status of the target slot is empty unconfigured cfgadm grep ROBO1 PCI slot02 Return pcipsy18 ROBO1 PCI slot02 unknown empty unconfigured unknown T O Cabinet Number X PCI Disk Box PCI Disk Box Number Y Physical Slot Number ZZ other than those above Physical Slot Number ZZ c To confirm the slot position at ex
116. nnel 1 hddv 3 0 a Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s1 gt devices pci 89 2000 fibre channel 1 hddv 3 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s2 gt devices pci 89 2000 fibre channel 1 hddv 3 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s3 gt devices pci 89 2000 fibre channel 1 hddv 3 0 d Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s4 gt devices pci 89 2000 fibre channel 1 hddv 3 0 e Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s5 gt J devices pci 89 2000 fibre channel 1 hddv 3 0 f Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s6 gt devices pci 89 2000 fibre channel 1 hddv 3 0 g Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s7 gt devices pci 89 2000 fibre channel 1 hddv 9 3 0 h 9 Start the format 1M command and configure labeling and partition slice information calculated in 8 b 8 to the logical path name of each extended LUN c Add configure of PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver No procedure is necessary Go to step 9 Fibre Channel driver PW00SFC3U PW008FC2U GP7BSFCIU No procedure is necessary Go to step 9 Fibre Channel Card PW028FC4 PW028FC5 Bind the auto mapped target connected with the card and modify the entries of lpfc conf to match the requirements of the extended Fibre Channel Controller These procedures are reflected on next boot time In the follo
117. ns that disk array device GR720 currently used with taget 1 lun 0 is registered twice on the sd hddv driver New installation to procedure 8 b 5 hddvadm M Return Operation with non active extension procedures to procedure 8 b 4 hddvadm M Return already ID 1 exists devices pci 1d 2000 fibre channel 1 sd 9 1 0 a raw GR720 4 When performing active extension the same target number cannot be registered in sd conf and hddv conf 5 6 so delete or comment out the conflicting definition parts in sd conf file when definition in the two files conflict with each other The following example shows the case that target 1 is repeated vi kernel drv sd conf Return name sd class scsi target 0 lun 0 name sd class scsi target 1 lun 0 name sd class scsi delete or comment out target 1 lun 1 the duplication definition parts in sd hddv name sd class scsi target 1 lun 2 A target number that is already used by the sd driver of the system disk or CD ROM cannot be deleted In this case change the target number in hddv conf and of the disk array device so that it does not conflict with sd conf Edit the kernel drv hddv conf file created automatically by the hddvadm M command and remove comments of the LUN definition to pre define and the LUN definition currently being used with non active extension procedures If case active extension has been performed befor
118. nt Line Control Function or the Multipath Function and to activate the redundant configuration system For details of environmental configuration and activation please refer to each function manual of SynfinityLink or PRIMECLUSTER GLS 8 Setup applications The configuration of high layer product services applications is added Please refer to the manual of each high layer product for details 9 Start applications By following operations please start the high layer product a When using TCP IP interface The high layer product services applications using the TCP IP interface is started For details of starting the high layer products please refer to the manual of each product b When using FNA SNA communication protocol over Ethernet It is necessary to start all interfaces of Netcompo FNA LAN and all high layer applications that are using Netcompo FNA LAN When using the Multipath Function the FNA Fujitsu Network Architecture SNA System Network Architecture communication protocol over Ethernet is valid For details please refer to the manual of Netcompo FNA LAN and high layer applications 4 2 3 Extension of PCI cards to redundant system This subsection describes the procedure extending PCI cards to the existing redundant configuration system with the following redundant configuration software SynfinityLink 2 2 PRIMECLUSTER GLS 4 0 or later The replacement and extension of PCI cards are supported only on Redundant L
119. nt software it is possible to exchange PCI cards without stopping services and applications The replacement procedure of PCI cards is as follows Please refer to Chapter 3 File devices or Chapter 4 Network devices about detailed replacing procedure of PCI cards File devices procedures l 25 10 Stop the hardware monitoring of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration Stop the daemons If you use the Fibre Channel Card PW028FC4 PW028FCS stop the daemons Specify the replacing PCI card Specify the devices and interfaces that are influenced by replacing PCI cards Disconnect from redundant system Disconnect the devices path and interfaces of PCI cards from the redundant system Disconnect the PCI card It changes into the state where the PCI card is exchangeable Replace the PCI card Replace the PCI card Connect the PCI card It changes into the state where the PCI card is usable Connect to redundant system Connect device paths and interfaces of the replaced PCI card to the redundant system Switch back the online path if needed Start the daemons If you use the Fibre Channel Card PWO28FCA4 PWO28FCS5 start the daemons Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration Network devices procedures l 2 Stop the hardware
120. ocedure 3 a Do the same with a tape device iostat xc Return extended device statistics cpu device r s w s kr s kw s wait actv svc t w 96b us sy wt id sdO 59 7 7 5 474 5 45 0 0 0 3 9 58 6 0 41 3 7 23 67 sd1 0 1 0 3 1 0 2 5 0 00 0 16 000 sd20 0 0 0 1 0 3 0 7 0 00 0 14 700 st82 0 0 0 0 0 0 0 0 0 0 0 0 0 000 nfs1 0 0 0 0 0 0 0 0 0 0 0 0 0 000 2 Stop disk access of applications Disk access statistics of the target disk can be checked as below sar d 60 1 Return SunOS machine0 5 8 Generic 108528 05 sun4u 10 02 02 17 56 00 device busy avque r w s blks s avwait avserv 17 57 00 nfs1 0 0 0 0 0 0 0 0 0 sd0 2 0 3 2 37 0 0 145 5 sd0 a 1 0 10 5 0 0 301 4 sd0 b 0 0 0 0 9 0 0 31 5 sd0 c 0 0 0 0 0 0 0 0 0 sd0 d 1 0 0 0 4 0 0 126 8 sd0 e 0 0 0 0 0 0 0 0 0 sd0 f 1 0 1 1 60 0 120 9 sd0 g 1 0 1 1 14 0 0 111 7 It is the same with tape devices The example below shows access statistics of one minute sd20 0 0 0 0 0 0 0 0 0 sd20 a 0 0 0 0 0 0 0 0 0 sd20 c 0 0 0 0 0 0 0 0 0 sd20 g 0 0 0 0 0 0 0 0 0 st82 0 0 0 0 0 0 0 0 0 3 Stop deactivate applications See the manuals of each software for details If the device is a disk unit swap device 1 Show a list of swap devices and check that the disk of the logical path determined in procedure 3 b c2t3d0 is a swap device swap l Return swapfile dev swaplo blocks free dev dsk c2t3d0s4 32 164 16 788384 683680 2
121. ollowing example shows the case that target 1 is repeated vi kernel drv sd conf Return name sd class scsi target 0 lun 0 name sd class scsi target 1 lun 0 61 62 5 6 8 name sd class scsi delete or comment out target 1 lun 1 the duplication definition parts in sd hddv name sd class scsi target 1 lun 2 A target number that is already used by the sd driver of the system disk or CD ROM cannot be deleted In this case change the target number in hddv conf and of the disk array device so that it does not conflict with sd conf Edit the kernel drv hddv conf file created automatically by the hddvadm M command and remove comments of the LUN definition to pre define and the LUN definition currently being used with non active extension procedures If case active extension has been performed before and the LUN definition to extend the next procedures 1 f 1 g are not needed The following example shows target 3 lun 0 2 for pre defined extension and disk array device GR720 target 1 in use with non active extension procedures being registered Is kernel drv hddv conf Return kernel drv hddv conf vi kernel drv hddv conf lt Return gt name hddv class scsi target 0 lun 0 name hddv class scsi target 0 lun 7 name hddv class scsi target 1 lun 0 name hddv class scsi target 1 lun 1 remove comments of the LUN in use name hdd
122. on of PCI slot Element Format of Ap Id Cabinet Number X Solaris 8 OE and Solaris 9 Cabinet of System Board Number OE PRIMEPOWER Y pcipsyN CXMOY PCI slotZZ 900 1500 2500 Physical Slot Number Solaris 10 OE ZL CXMOY PCI ZZ I O Cabinet Number X Solaris 8 OE and Solaris 9 PCI Disk Box OE PCI Disk Box Number Y pcipsyN RXBOY PCI slotZZ Physical Slot Number Solaris 10 OE ZL RXBOY PCI ZZ Solaris 8 OE and Solaris 9 i OE other than those above eee Slot NUMER pcipsyN PCI slotZZ Solaris 10 OE PCI ZZ Please confirm that the PCI slot status which extended PCI cards is empty unconfigured by using cfgadm 1M command cfgadm grep ROBO1 PCI slot02 lt Return gt pcicsy18 ROBO1 PCI slot02 unknown empty unconfigured unknown c To confirm the slot position at extension operation blink the ALARM LED of Ap Id displayed in procedure 2 b cfgadm x led fault mode blink pcipsy18 ROB01 PCI slot02 lt Return gt d Please confirm that READY LED of the PCI bus slot that extends a PCI card has gone out and extend a PCI card This operation is performed by our customer support e Please confirm that the PCI slot status which extended PCI cards is disconnected unconfigured by using cfgadm 1M command cfgadm pcipsy18 ROB01 PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcicsy18 ROBO1 PCI slot02 unknown disconnected unconfigured unknown 3 Connect PCI cards The cfgadm command is executed wit
123. ondition pcipsy3 COMOO0 PCTZslot02 mult hp connected configured ok When exchanging Fibre Channel cards the following operations are also required for PCI Fibre Channel PW008FC3U PW008FC2U GP7B8FC1U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility a Refresh current condition on management window SAN Storage Area Network management function searches and indicates replaced PCI card This function will adjust switch zoning and host affinity function of disk array device ETERNUS3000 ETERNUS6000 GR700 800 series with new WWN of replaced PCI card 17 See Operation guide or User s Manual of each product for details Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration b Reconnect the replaced PCI card with changed zoning setting by using the following commands cfgadm c disconnect pcipsy3 COM00 PCI slot02 Return cfgadm c configure pcipsy3 COM 00 PCEslot02 Return cfgadm pcipsy3 COM00 PCI slot02 Return Ap Id Type Receptacle Occupant Condition pcipsy3 COMOO0 PCTZslot02 mult hp connected configured ok If
124. onfigurations Fibre Channel switch SN200 series and disk array device ETERNUS3000 ETERNUS6000 GR700 800 series need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on SN200 series If the Host Affinity function of ETERNUS3000 ETERNUS6000 GR700 800 series is used For details see SN200 Series Affinity User s Guide or ETERNUSmgr User Guide GRmgr User Guide If you use the Fibre Channel switch or disk array device other than described above see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digit of the WWPN in hexadecimal form The top eight digit are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the replacement card is 100000000e244061 0e24 4061 for Fibre Channel Card PW028FC4 PW028FC5 To replace PCI cards with the following configurations Fibre Channel switch and disk array device need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on Fibre Channel switch If the Host Zoning function of disk array device is used For details see the document of each product To perform the above reconfiguration the WWPN a 16 digit n
125. ormally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration a Before extension of PCI cards save the current execution result of the prtpicl IM command current configuration at present prtpicl v gt tmp prtpicl pre Return b Check the status of the slot When adding PCI cards to a redundant configuration several PCI cards may be added to the system In this case perform procedures to 3 on each card The relationship of the position of a PCI card and format of Ap Id is shown below Location of PCI slot Element Format of Ap Id Solaris 8 OE and Solaris 9 OE Cabinet of Cabinet Number X PRIMEPOWER System Board Number Y Sol e NUUS 900 1500 2500 Physical Slot Number ZZ oS CXMOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN RXBOY PCI slotZZ Solaris 10 OE RXBOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN PCI slotZZ Solaris 10 OE PCI ZZ T O Cabinet Number X PCI Disk Box PCI Disk Box Number Y Physical Slot Number ZZ other than those above Physical Slot Number ZZ Below an example of adding a PCI card to chassis number 0 PCI Disk BOX number 1 physical slot number 02 slot in PCI Disk BOX ROBO01 PCI slot02 is explained Check the status of the target slot is empty unconfigured cfgadm grep ROBO1 PCI slot02 Return pcipsy18 ROBO1
126. ot 41 Sep 20 22 53 c3t3d0s4 gt J devices pci 89 2000 fibre channel 1 hddv 3 0 e Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s5 gt J devices pci 89 2000 fibre channel 1 hddv 3 0 f Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s6 gt devices pci 89 2000 fibre channel 1 hddv 3 0 g Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s7 gt devices pci 89 2000 fibre channel 1 hddv 3 0 h 4 Start the format 1M command and configure labeling and partition slice information calculated in 9 b 3 to the logical path name of each extended LUN c Add configure of PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver No procedure is necessary Go to step 10 Fibre Channel driver PW008FCZU W008FC2X GP7B8FC1U GP7B8FC1X No procedure is necessary Go to step 10 Fibre Channel Card PW028FC4 PW028FC5 Bind the auto mapped target connected with the card and modify the entries of lpfc conf to match the requirements of the extended Fibre Channel Controller These procedures are reflected on next boot time In the following indicate to bind the auto mapped target with the card usr sbin Ipfc Aputil Return LightPulse Common Utility for Solaris SPARC Version 1 6a8 03 03 2004 Copyright c 2003 Emulex Network Systems Inc Emulex Fibre Channel Host Adapters Detected 2 Host Adapter 0 Ipfc11 is an LP9802 Ready Mode Host Adapter 1 Ipfc9 is an LP9802 Rea
127. ot root 41 Sep 20 22 53 c3t3d0s2 gt devices pci 89 2000 fibre channel 1 hddv 3 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s3 gt devices pci 89 2000 fibre channel 1 hddv 3 0 d Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s4 gt devices pci 89 2000 fibre channel 1 hddv 3 0 e Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s5 gt devices pci 89 2000 fibre channel 1 hddv 3 0 f Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s6 gt devices pci 89 2000 fibre channel 1 hddv 3 0 g Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s7 gt devices pci 89 2000 fibre channel 1 hddv 9 3 0 h 9 Start the format 1M command and configure labeling and partition slice information calculated in 8 b 8 to the logical path name of each extended LUN c Add configure of PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver No procedure is necessary Go to step 9 Fibre Channel driver PW008FC3U PWO008FC2U GP7B8FC1U No procedure is necessary Go to step 9 Fibre Channel Card PW028FC4 PW028FC5 Bind the auto mapped target connected with the card and modify the entries of lpfc conf to match the requirements of the extended Fibre Channel Controller These procedures are reflected on next boot time In the following indicate to bind the auto mapped target with the card usr sbin Ipfc lputil lt Return gt LightPulse Common Utility for Solaris SPARC Ver
128. ple with GR multipath driver GRMPD There are no additional patterns for Multipath Disk Control MPHD If multipath software products other than GRMPD are being used see the manual of each product No configuration is needed for applications when adding PCI cards to an existing redundant system i Setup devices which will be added to system This is an example where a disk array controller of target 3 lun 0 is added with pre definition to an existing disk array device Extension using PCI Hot Plug cannot be done without definition so devices must be defined before the system goes into operation If you use Solaris OE sd If this is the first time to pre define a disk array device begin with procedure 1 a If this is not the first time to pre define a disk array device go to step 2 a Modify kernel drv sd conf Ensure target 3 lun 0 target 3 lun 1 target 3 lun 3 are defined in kernel drv sd conf If they are not defined or commented modify kernel drv sd conf vi kernel drv sd conf Return name sd class scsi target 0 lun 0 name sd class scsi target 3 lun 0 name sd class scsi target 3 lun 1 name sd class scsi target 3 lun 2 b Boot the system and reconfigure the kernel touch reconfigure lt Return gt usr sbin shutdown y i6 g0 lt Return gt If you use hddv If this is the first time to pre define a disk array device begin with procedure 1 a If this is not the first ti
129. r to the user s manual of each equipment The following example shows the definition of a fjpfcal PCI card instance number 1 and disk array equipment target 3 being added vi kernel drv fjpfca conf Return port fjpfcaO nport fjpfcal nport sp1 lt added fcp bind target fjpfca0t0 0x50000e1000002501 fjpfcalt3 0x50000e1000002ab0 lt added max throttle fjpfca0t0 128 fjpfcalt3 240 added 3 Execute the following command to make the fjpfca driver take in the added part usr sbin F JS Vpfca fc config v Return Fibre Channel driver PW028FCA4 PW028FC5 Configuration is not required Compare the output of prtpicl 1M command with the output saved at 4 b and find the instance number of the added card Add device driver configuration Add device driver configuration with the following procedures according to the I O device type Disk Tape devices Solaris OE sd st This example is a case where a disk device of target 2 lun 0 is pre defined in procedure 1 and added in procedure 5 If the device is a tape device re read sd as st in this procedure and perform the same operation Execute the following command to make the sd driver set up the added disk device usr sbin devfsadm Return 2 After saving the execution result of the prtpicl 1M command take the difference between the result and the information obtained in 4 a and get the
130. re lt Return gt shutdown y i6 g0 lt Return gt Disk Array device hard disk driver hddv This is an example where a disk array device at target 3 lun 0 2 is pre defined If this is the first time to pre define a disk array device start from procedure 1 a If this is not the first time to pre define a disk array device start from procedure 1 e a Check if kernel drv hddv conf exists If it does not exist the hard disk driver is being installed so start from procedure 1 c If it does exist the system is being operated with non active extension procedures so start from procedure 1 b Not exist new install to procedure 1 c Is kernel drv hddv conf Return kernel drv hddv conf No such file or directory Exists operation with non active extension procedures to procedure 1 b Is kernel drv hddv conf Return kernel drv hddv conf b When operating with non active extension procedures execute the following commands and confirm that kernel drv hddv conf is deleted hddvadm i Return Is kernel drv hddv conf Return kernel drv hddv conf No such file or directory c Use the following command to create an hddv conf file for the pre defined extension function The message examples for operation with non active extension procedures means that GR720 currently used with taget 1 lun 0 is registered twice on the sd hddv driver New installation to procedure 1 e hd
131. res to add PCI cards using the PCI Hot Plug feature on non redundant and redundant systems with Solaris 8 OE and Solaris 9 OE or later The difference between the procedures for the two OEs is that I O devices to be added must be pre defined before system operation with Solaris 8 OE but pre definition is not needed with Solaris 9 OE or later 3 2 1 Solaris 8 OE The procedures to add PCI cards on a Solaris 8 OE system are explained here To use the PCI Hot Plug function for extension on Solaris OE I O devices to be added must be pre defined and the system must be rebooted before system operation If the I O devices to be added are already defined pre definition is not needed and procedure 1 below can be skipped If you use Fibre Channel Card PWO28FC4 PW028FC5 ensure the automap function is effective Boot the system after modifying Ipfc conf vi kernel drv Ipfc conf lt Return gt If automap is set SCSI IDs for all FCP nodes without persistent bindings will be automatically generated If new FCP devices are added to the network when the system is down there is no guarantee that these SCSI IDs will remain the same when the system is booted again The bind method of the port is used as the binding method of automap devices to preserve SCSI IDs between link down and link up If automap is 0 only devices with persistent bindings will be recognized by the system 18
132. ring of the Machine Administration Connect equipment to PCI cards Connect equipment to PCI cards Set up driver a Add the driver configuration of PCI cards Add the driver configuration of PCI cards b Add the configuration of device driver Add the configuration of device driver in the case of file devices Connect to redundant system Connect device paths and interfaces of the replaced PCI card to the redundant system Switch the redundant path Switch the online path for extended path if needed 10 Chapter 3 File Devices This chapter describes in detail the procedures of replacement and extension of file PCI cards by using PCI Hot Plug For details about software to use PCI cards in redundant configuration see the manuals of each software 3 1 Replacement of PCI card This section describes the procedures to exchange PCI cards using the PCI Hot Plug feature on non redundant and redundant systems 3 1 1 Replacement of PCI cards on non redundant system Before exchanging PCI cards without using redundancy software such as multipath control applications using the PCI cards must be stopped Follow the procedures below to exchange PCI cards on a non redundant system 1 Stop the machine administration hardware monitoring daemon Execute the following command to stop the hardware monitoring daemon of machine administration usr sbin FJS Vmadm prephp Return etce rc0 d K10hbanyware stop Return etc rc0 d K
133. rnel drv hddv conf seems to be correct Check NG to procedure 8 b 4 opt FJSVhddv bin hddvchk conf Return NGID 2 3 Execute the following command in order to make the hard disk driver set up the LUN added update drv f hddv Return In case disk array equipment currently used exists the following messages are displayed but do not effect performance update drv f hddv Return Cannot unload module hddv Will be unloaded upon reboot Forcing update of hddv conf After saving execution result of the prtpicl 1M command get the physical path name of added LUN by taking the difference between the result and the information obtained in 3 a and calculate the corresponding logical path name under dev dsk In the following example c3t3d0 is logical path name of added lun 0 The logical path name of lun 1 2 is set to c3t3d1 c3t3d2 respectively because cXtYdZmeans X PCI card number Y target number Z lun number prtpicl v gt tmp prtpicl post Return diff tmp prtpicl pre tmp prtpicl post grep devfs path grep hddv Return gt devfs path pci 89 2000 fibre channel 1 hddv 9 3 0 gt devfs path pci 89 2000 fibre channel 1 hddv 93 1 gt devfs path pci 89 2000 fibre channel 1 hddv 3 2 Is 1 dev dsk grep pci 89 2000 fibre channel 1 hddv 9 3 0 lt Return gt Irwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s0 gt devices pci 89 2000 fibre cha
134. rwx 1 root root 41 Sep 20 22 53 c2t2d0s6 gt devices pci 89 4000 scsi 2 sd 9 2 0 g Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s7 gt devices pci 89 4000 scsi 2 sd 2 0 h Tape equipment diff tmp prtpicl pre tmp prtpicl post grep devfs path grep st lt Return gt gt devfs path pci 2 89 4000 scsi Q2 st 9 5 0 Is 1 dev rmt grep pci 289 4000 scsi 2 st 2 5 0 Return Irwxrwxrwx root root 41 Sep 20 22 53 1 gt J devices pci 89 4000 scsi Q 2 st 5 0 Irwxrwxrwx 1 root root 41 Sep 20 22 53 1b gt devices pci 89 4000 scsi Q 2 st 5 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 Ibn gt devices pci 89 4000 scsi 2 st 5 0 bn Irwxrwxrwx 1 root root 41 Sep 20 22 53 1c gt devices pci 89 4000 scsi 2 st 5 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 1cb gt devices pci 89 4000 scsi 2 st 5 0 cb Irwxrwxrwx 1 root root 41 Sep 20 22 53 1cbn gt devices pci 89 4000 scsi 2 st 5 0 cbn Irwxrwxrwx 1 root root 41 Sep 20 22 53 1cn gt devices pci 89 4000 scsi 2 st 5 0 cn Irwxrwxrwx 1 root root 41 Sep 20 22 53 1h gt devices pci 89 4000 scsi Q 2 st 5 0 h Irwxrwxrwx 1 root root 41 Sep 20 22 53 1hb gt devices pci 89 4000 scsi 2 st 5 0 hb Irwxrwxrwx 1 root root 41 Sep 20 22 53 1hbn gt devices pci 89 4000 scsi 2 st 9 5 0 hbn Irwxrwxrwx 1 root root 41 Sep 20 22 53 Ihn gt devic
135. s are examples of execution of the FJSVhddv package installation to opt The path of the hddvchk_conf command depends on the installed directory Check OK to procedure 1 g opt FJS Vhddv bin hddvchk conf Return kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 1 d opt FJS Vhddv bin hddvchk_conf lt Return gt NGID 3 g Reconfigure the kernel restart the system and then start system operation touch reconfigure lt Return gt usr sbin shutdown y i6 g0 lt Return gt 2 Stop the machine administration monitoring daemon Use the following command to stop the hardware disorder monitoring daemon of machine administration usr sbin FJSVmadm prephp lt Return gt If you use the Fibre Channel Card PW028FC4 PW028FCS execute the following The daemons will be stopped ete rc0 d K10hbanyware stop Return etce rc0 d K10hbanywareDisc stop Return Add PCI cards Add PCI cards according to the following procedure for PCI Fibre Channel PW008FC3U PW008FC2U GP7B8FC1U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility No procedure is necessary Go to step 5 If you don t use SAN management function of the above products To add PCI cards with the following configurations Fibre Channel switch SN200 series and disk array device ETERNUS
136. s changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status 29 30 confirmation may take time cfgadm c configure pcipsy18 R0OB01 PCI slot02 Return cfgadm pcipsy18 ROBO1 PCIslot02 Return Ap Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCI slot02 mult hp connected configured ok If you use the Fibre Channel Card PW028FC4 PW028FC5 execute the following The daemons will be started etc rc2 d S99hbanyware start Return letc rc2 d S99hbanywareDisc start Return Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJSVmadm postphp Return Preparation to recognize devices for Fibre Channel When the Host Affinity function is being used with Fibre Channel SN200 series Host Affinity must be configured for the device added for PCI Fibre Channel PW008FC3U PW008FC2U GP7B8FC1U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility WWPN zoning of SN200 series Host affinity of Array disk device ETERNUS3000 ETERNUS6000 GR700 800 series and binding setting
137. s described for MPHD GRMPHD If you use the redundancy software other than MPHD GRMPD see the document for each product Execute the following command specifying the logical path name of the connected LUN determined in procedure 3 b This command only needs to be executed on the representation LUN and does not need to be executed on each LUN under the same PCI card If the operation path is disconnected on a redundant system the standby path will automatically be switched to operation If both MPHD GRMPD controlled devices are connected to the target PCI card the commands for both products need to be executed MPHD usr opt F JS Viomp bin iompadm c mphd change adapter disconnect dev rdsk c3t1d0s2 Return GRMPD usr opt F JS Viomp bin iompadm c mplb change adapter disconnect dev rdsk c3t1d0s2 Return 4 Disconnect the PCI card Disconnect the defected PCI card with the following procedures a Determine the slot position of the PCI card from the interface name of the path connecting the target PCI card and I O devices determined in procedure 3 a fjpfca3 In this example the Ap Id is pcipsy21 R0B01 PCI slot03 usr sbin FJSVmadm inst2comp fjpfca3 Return pcipsy21 R0B01 PCI slot03 b Specify the Ap_Id from 4 a as a parameter and confirm that the slot status of the PCI card to disconnect is connected configured cfgadm pcipsy21 R0B01 PCI slot03 lt Return gt Ap Id Type Receptacle Occupant Condition pcipsy21 R
138. scsi 9 2 sd 9 2 0 a Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s1 gt devices pci 89 4000 scsi 2 sd 2 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s2 gt devices pci 89 4000 scsi 2 sd 9 2 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s3 gt devices pci 89 4000 scsi 2 sd 2 0 d Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s4 gt devices pci 89 4000 scsi 2 sd 9 2 0 e Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s5 gt devices pci 89 4000 scsi 9 2 sd 9 2 0 f Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s6 gt devices pci 89 4000 scsi 2 sd 99 2 0 g Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s7 gt devices pci 89 4000 scsi 2 sd 2 0 h Tape device diff tmp prtpicl pre tmp prtpicl post grep devfs path grep st lt Return gt gt devfs path pci 2 89 4000 scsi Q2 st 9 5 0 Is 1 dev rmt grep pci 289 4000 scsi 2 st 2 5 0 Return Irwxrwxrwx 1 root root 41 Sep 20 22 53 1 gt devices pci 89 4000 scsi Q 2 st 5 0 Irwxrwxrwx 1 root root 41 Sep 20 22 53 1b gt devices pci 89 4000 scsi Q 2 st 5 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 Ibn gt devices pci 89 4000 scsi 2 st 9 5 0 bn Irwxrwxrwx 1 root root 41 Sep 20 22 53 1c gt devices pci 89 4000 scsi Q 2 st 5 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 1cb gt devices pci 89 4000 scsi 2 st 5 0 cb Irwxrwxrwx 1 root root 41 Sep
139. series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration Setup driver Add driver configurations with the operation below With file related drivers configuration for the PCI card driver and the device driver is needed a Add configuration for PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver Configuration is not required Fibre Channel driver PW008FC3U PW008FC2U GP7BSFCIU 1 After saving execution result of prtpicl 1M command take the difference between the result and the information obtained in procedure 3 a to calculate the instance number of the fjpfca driver of the PCI card added In the following example instance number is 1 indicating that the interface name of the PCI card added is fjpfcal Making the LED of the PCI card blink using the fc adm command can specify the relation between the instance number and the slot position The relation between the WWPN and instance number of the PCI card added can be known from the fc info a command prtpicl v gt tmp prtpicl pfca Return diff tmp prtpicl pre tmp prtpicl pfca more Return status okay fru PCI slot PCI 02 at Rack 0 IO_BOX 1 component name ROBO1 PCI 02 devfs path pci 8d 2000 fibre c
140. sion 1 6a8 03 03 2004 Copyright c 2003 Emulex Network Systems Inc Emulex Fibre Channel Host Adapters Detected 2 Host Adapter 0 Ipfc11 is an LP9802 Ready Mode Host Adapter 1 Ipfc9 is an LP9802 Ready Mode 55 MAIN MENU 1 List Adapters 2 Adapter Information 3 Firmware Maintenance 4 Reset Adapter 5 Persistent Bindings 0 Exit Enter choice gt 5 Return PERSISTENT BINDINGS MENU 1 Display Current Bindings 2 Display All Nodes 3 Duplicate Adapter Bindings 4 Bind Target Manually 5 Bind Automapped Targets 6 Delete Binding s 0 Return to Main Menu Enter choice 2 5 Return O Ipfc11 1 Ipfc9 Select an adapter gt 0 lt Return gt Automapped FCP Nodes Node Target WWPN WWNN Automapped FCP Node 0 50 06 0e 80 00 10 04 92 50 06 0e 80 00 10 04 92 Automapped FCP Node 1 50 06 0e 80 00 10 04 90 50 06 0e 80 00 10 04 90 Press the Enter or Return key to continue Bind all auto mapped targets Y N gt Y Return Select Binding Method 1 By Node Name 2 By Port Name 3 ByD ID 0 Cancel Enter Choice gt 2 Return 9 Setup redundant system Execute the following commands to create a new redundant system The following procedure is described for MPHD GRMPHD If you use the redundancy software other than MPHD GRMPHD see the document for each product In case of MPHD the commands need to be executed for each LUN specifying the path name det
141. slot02 Return cfgadm pcipsy3 COM00 PCI slot02 Return Ap_Id Type Receptacle Occupant Condition pcipsy3 COM00 PCl slot02 pci pci hp connected configured ok Start applications Please restart the stopped high layer product service application by the following operations a When using TCP IP interface The example in the case of using TCP IP interface is shown below Furthermore when the high layer program which is using the PCI card needs to restart please refer to the manual of each high layer program 1 When using Ethernet devices usr sbin ifconfig hmel plumb lt Return gt usr sbin ifconfig hme1 inet 192 168 10 10 netmask broadcast trailers up lt Return gt 2 When using ATM devices opt FJS Vatm bin atmlink adapter number lt Return gt b When using FNA SNA communication protocol over Ethernet When not using the redundant configuration software it is necessary to start all interfaces of Netcompo FNA LAN and all high layer applications that are using Netcompo FNA LAN For details please refer to the manual of Netcompo FNA LAN and high layer applications Start the hardware monitoring of the Machine Administration Please execute the following commands in order to update configuration information and restart the hardware monitoring of the Machine Administration Please refer to PCI Hot Plug User s Guide about details of postphp 1M command usr sbin FJSVmadm postphp lt Return gt
142. step 8 Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration Setup driver Add the configuration of driver according to the following operation As for file related drivers additional configuration of the PCI card driver and the device driver is needed a Add the configuration of PCI cards driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver Configuration is not required Fibre Channel driver PW00SFC3U PW008FC2U GP7BSFCIU 1 After saving execution result of prtpicl 1M command take the difference between the result and the information obtained in procedure 3 a to calculate the instance number of the fjpfca driver of the PCI card added In the following example instance number is 1 indicating that the interface name of the PCI card added is fjpfcal Making the LED of the PCI card blink using the fc adm command can specify the relation between the instance number and the slot position The relation between the WWPN and instance number of the PCI card added can be known from the fc info a command prtpicl v gt tmp prtpicl pfca Return diff tmp prtpicl pre tmp prtpicl pfca more Return status okay devfs pat
143. t of the WWPN in hexadecimal form The top eight digit are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the replacement card is 100000000e244061 0e24 4061 for Fibre Channel Card PW028FC4 PW028FC5 To replace PCI cards with the following configurations Fibre Channel switch and disk array device need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on Fibre Channel switch If the Host Zoning function of disk array device is used For details see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card These characters represent the bottom twelve digit of the WWPN in hexadecimal form The top four digit are fixed to 1000 in hexadecimal form For example if the following label is shown on the back of the card the WWPN of the replacement card is 10000000c9366037 IEEE 0000c9366037 Note When changing Affinity configuration on SN200 series or other Fibre Channel switch I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity co
144. talled so start from procedure 8 b 3 If it does not exist the system is being operated with none active extension procedures so start from procedure 8 b 2 V VV VM V Not exist new install to procedure 8 b 3 Is kernel drv hddv conf Return kernel drv hddv conf No such file or directory Exist operation with non active extension procedures to procedure 8 b 2 Is kernel drv hddv conf lt Return gt kernel drv hddv conf 2 When operating with non active extension procedures execute the following commands and confirm that kernel drv hddv conf is deleted hddvadm i Return Is kernel drv hddv conf Return kernel drv hddv conf No such file or directory 3 Use the following command to create hddv conf file for the pre defined extension function The message examples for operation with non active extension procedures means that disk array device GR720 currently used with taget 1 lun 0 is registered twice on the sd hddv driver New installation to procedure 8 b 5 hddvadm M Return Set up with a non active extension procedure to procedure 8 b 4 hddvadm M Return already ID 1 exists devices pci 1d 2000 fibre channel 1 sd 9 1 0 a raw GR720 4 In activity extension because same target number can t be registered in sd conf and hddv conf delete or comment out the duplication definition parts in sd conf file when both definition are overlapped The f
145. tem start add an entry in the etc vfstab file newfs dev rdsk c2t2d0s6 Return vi etc vfstab Return dev dsk c2t2d0s6 dev rdsk c2t2d0s6 mnt ufs 1 yes Disk device raw access operation Tape device Configure applications For details see the manual of each application Disk device swap device Add swap device To make valid at system start add an entry in the etc vfstab file swap a dev dsk c2t2d0s4 Return vi etc vfstab Return dev dsk c2t2d0s4 swap no swap l Return swapfile dev swaplo blocks free dev dsk c2t2d0s4 32 164 16 788384 683680 10 Start applications Use the logical path name determined in 8 b to start applications Disk device filesystem operation Mount filesystem mount F ufs dev dsk c2t2d0s6 mnt Return Disk device raw access operation Tape device Start applications For details see the manual of each application Disk device swap device This procedure is not required 3 2 2 2 Extension of PCI cards with new redundant system The procedure to add PCI cards to a new redundant system on a Solaris 9 OE or later system is explained with examples for the multipath software below Multipath Disk Control MPHD GR Multipath Driver GRMPD an example with GR multipath driver GRMPD If other redundant software is used see the manual of each product 1 Stop the machine administration
146. tension operation blink the ALARM LED of Ap Id displayed in procedure 4 b cfgadm x led fault mode blink pcipsy18 ROB01 PCI slot02 Return d After checking that the READY LED on the PCI slot to add the PCI card is off add PCI cards and connect cable to devices This operation is performed by our customer support e Use the cfgadm 1M command to check that the slot status of the added PCI card changed to disconnected unconfigured cfgadm pcipsy18 ROBO1 PCI slot02 Return Ap Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PClI slot02 unknown disconnected unconfigured unknown Connect PCI cards Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the extension slot position Note that the push button is only effective in multi user mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy18 R0B01 PCI slot02 Return cfgadm pcipsy18 ROBO1 PCEislot02 Return Ap Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCI slot02 mult hp connected configured ok If you use the Fibre Channel Card PW028FC4 PW028FC5S execute the following The daemons w
147. ters 2 Adapter Information 3 Firmware Maintenance 4 Reset Adapter 5 Persistent Bindings 0 Exit Enter choice gt 5 Return PERSISTENT BINDINGS MENU 1 Display Current Bindings 2 Display All Nodes 3 Duplicate Adapter Bindings 4 Bind Target Manually 5 Bind Automapped Targets 6 Delete Binding s 0 Return to Main Menu Enter choice 2 5 Return O Ipfc11 1 Ipfc9 Select an adapter gt 0 lt Return gt Automapped FCP Nodes Node Target WWPN WWNN 25 Automapped FCP Node 0 50 06 0e 80 00 10 04 92 50 06 0e 80 00 10 04 92 Automapped FCP Node 1 50 06 0e 80 00 10 04 90 50 06 0e 80 00 10 04 90 Press the Enter or Return key to continue Bind all auto mapped targets Y N gt Y Return Select Binding Method 1 By Node Name 2 By Port Name 3 ByD ID 0 Cancel Enter Choice gt 2 Return 10 Setup applications 11 Add configuration for the application using the logical path name determined in 9 b Disk device filesystem operation Create file system To mount at system start add an entry in the etc vfstab file newfs dev rdsk c2t2d0s6 lt Return gt vi etc vfstab Return dev dsk c2t2d0s6 dev rdsk c2t2d0s6 mnt ufs 1 yes Disk device raw access operation Tape device Configure applications For details see the manual of each application Disk device swap device Add swap device To make valid at system start add
148. the operating clock of PCI cards and PCI slots 2 In case of using Solaris 8 OE patch 109885 09 or later is needed 3 Replacement and expansion are not supported due to hardware specifications 4 Replacement is not supported due to hardware specifications 5 Multipath Disk Control load balance option is not supporting PCI Hot Plug 6 If using Solaris 9 OE it is necessary to apply the FUJITSU PCI Fibre Channel 2 2 patch level 08 or later 7 In case of using FUJITSU PCI Fibre Channel 2 2 patch 910936 13 or later is needed And in case of using FUJITSU PCI Fibre Channel 2 2 1 patch 912069 09 or later is needed 8 Redundant software should support Dynamic Reconfiguration function 9 In case of using Solaris 10 OE FUJITSU PCI Fibre Channel 3 0 or later is needed 10 When connecting a shared device with a cluster system please attach the shared device to the system after connecting the PCI card and the shared device with each node of the cluster system 11 When a shared device is connected with a cluster system it is necessary to stop the use of the shared device from the operation node of the cluster system 12 When using Solaris 8 OE the patch 114632 04 or later is needed When applying the patch 114632 04 or later if the patch 912773 02 or later has applied it is necessary to remove the patch 13 This product is not supporting Solaris 10 OE in March 2005 1 2 PCI cards for network devices The fo
149. titles of chapters are enclosed in parentheses Example See Chapterl PCI cards supporting PCI Hot Plug Commands and other input use the following prompts C shell prompt prompt Bourne and korn shell prompt prompt Super user prompt Commands entered by the user is shown in bold cfgadm c configure pcipsy2 ROB00 PCI slot01 lt Return gt Key combinations are represented for example by CTL C which means to simultaneously press the Control and C key Reference Manuals If necessary refer to the following manuals PCI Hot Plug User s Guide Dynamic Reconfiguration User s Guide Trademarks Sun Sun Microsystems the Sun Logo Solaris and all Solaris based marks and logos are trademarks or registered trademarks of Sun Microsystems Inc in the U S and other countries and are used under license All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International Inc in the U S and other countries Products with the SPARC trademark are based on architecture developed by Sun Microsystems All other products or company names mentioned in this document are claimed as trademark and trade names by their respective companies FUJITSU LIMITED March 2005 January 2003 First Edition April 2003 Second Edition June 2003 Third Edition July 2003 Fourth Edition November 2003 Fifth Edition May 2004 Sixth Edition March 2005 Seventh Edition Notes This manual m
150. top the daemons Add PCI cards Extend PCI cards Connect PCI cards It changes into the state where the PCI card is usable Start the daemons If you use the Fibre Channel Card PWO28FCA4 PWO28FCS5 start the daemons Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration Preparation to recognize devices for Fibre Channel If you use SAN management function preparation to recognize devices Set up driver a Add the driver configuration of PCI cards Add the driver configuration of PCI cards b Add the configuration of device driver Add the configuration of device driver in the case of file devices Connect to redundant system Connect device paths and interfaces of the replaced PCI card to the redundant system Switch the online path for extended path if needed Network devices procedures l 2 Setup I O devices which will be added to system Setup devices which will be added to system in the case of file devices on Solaris 8 OE Stop the hardware monitoring of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration Add PCI cards Extend PCI cards Connect PCI cards It changes into the state where the PCI card is usable Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monito
151. tpicl pfca more Return status okay fru PCI slot PCI 02 at Rack 0 IO_BOX 1 component name ROBO1 PCI 02 devfs path pci 8d 2000 fibre channel 1 driver name fjpfca binding name pcilOcf 105c bus addr 1 instance 1 class obp device name fibre channel VVVVVV VV VV 2 Edit the kernel drv fjpfca conf file and add the following property definitions to the instance number determined in 8 a 1 port type of topology link speed Defined when adding PCI cards with Fabric connection This definition is not needed with FC AL connection fcp bind target connection between target ID and WWN Defined when adding disk array with Fabric connection This definition is not needed with FC AL connection max throttle maximum issued SCSI commands Defined to limit the number of SCSI commands issued at the same time The need and value of this definition depends on the disk array device so refer to the manuals of each disk array device Below is an example where definitions for a PCI card at fjpfcal instance number 1 and a disk array device at target 3 is added vi kernel drv fjpfca conf lt Return gt port fjpfca0 nport fjpfcal nport sp1 lt add fcp bind target fjpfca0t0 0x50000e 1000002501 fjpfca1t3 0x50000e1000002ab0 lt add max throttle fjpfca0t0 128 fjpfca1t3 240 lt add 3 Execute the following command to make the fjpfca driver set up the additional part
152. turn Stop the machine administration hardware monitoring daemon Execute the following commands to stop the machine administration hardware monitoring daemon usr sbin FJSVmadm prephp lt Return gt If you use the Fibre Channel Card PW028FC4 PW028FC5 execute the following The daemons will be stopped ete rc0 d K10hbanyware stop Return ete rc0 d K10hbanywareDisc stop Return 4 Add PCI cards Add PCI cards according to the following procedure for PCI Fibre Channel PW008FC3U PW008FC2U GP7B8FC10U If you use SAN management function of Systemwalker StorageMGR Softek SANView for ETERNUS except for Vixel SP5000 SRM Facility No procedure is necessary Go to step 5 If you don t use SAN management function of the above products To add PCI cards with the following configurations Fibre Channel switch SN200 series and disk array device ETERNUS3000 ETERNUS6000 GR700 800 series need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on SN200 series If the Host Affinity function of ETERNUS3000 ETERNUS6000 GR700 800 series is used For details see SN200 Series Affinity User s Guide or ETERNUSmgr User Guide GRmgr User Guide If you use the Fibre Channel switch or disk array device other than described above see the document of each product To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of
153. turn gt d Please confirm that READY LED of the PCI bus slot that extends a PCI card has gone out and extend a PCI card This operation is performed by our customer support e Please confirm that the PCI slot status which extended PCI cards is disconnected unconfigured by using cfgadm 1M command cfgadm pcipsy15 ROB01 PCIZslot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcicsy18 ROBO1 PCI slot02 unknown disconnected unconfigured unknown 3 Connect PCI cards The cfgadm command is executed with the configure option to Ap Id or the push button of the PCI bus slot is pushed Then a extended PCI card is connected In addition a push button is valid only in multiple user mode Please confirm that the slot status is connected configured by using the cfgadm 1M command after the above mentioned procedure cfgadm c configure pcipsy18 R0B01 PCIZslot02 Return cfgadm pcipsy18 ROB01 PCIZslot02 Return Ap Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCI slot02 pci pci hp connected configured ok 4 Start the hardware monitoring of the Machine Administration Please execute the following command in order to update configuration information and restart the hardware monitoring ofthe Machine Administration usr sbin FJSVmadm postphp Return 70 5 Connect equipment to PCI cards The extended PCI card is connected with network equipment by the cable 6
154. ufs dev dsk c2t3d0s0 develop firm Return mount F ufs dev dsk c2t3d0s1 develop drv Return mount F ufs dev dsk c2t3d0s6 pub Return If the device is a disk unit raw access operation or a tape device Restart applications and resume usage Refer to the manual of each application for details If the device is a disk unit swap device Add swap device and resume usage sbin swapadd 2 Return 8 If you use the Fibre Channel Card PW028FC4 PWO028FC5 execute the following The daemons will be started etc rc2 d S99hbanyware start Return Jetc rc2 d S99hbanywareDisc start Return 9 Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJS Vmadm postphp Return 3 1 2 Replacement of PCI cards on redundant system PCI cards can be replaced without stopping services such as user applications on a redundant system using software such as multipath control Here the procedure to replace PCI cards on a redundant system using the following redundancy software products is explained Multipath Disk Control MPHD GR Multipath Driver GRMPD If other redundancy software products are used see the manual of each product 1 Stop the machin
155. umber of the replacement card is needed The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card These characters represent the bottom twelve digit of the WWPN in hexadecimal form The top four digit are fixed to 1000 in hexadecimal form For example if the following label is shown on the back of the card the WWPN of the replacement card is 10000000c9366037 IEEE 0000c9366037 Note When changing Affinity configuration on SN200 series or other Fibre Channel switch I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration 6 Connect the PCI card Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the replacement slot position Note that the push button is only effective in multiuser mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy3 COM00 PCI slot02 Return cfgadm pcipsy3 COM00 PCEslot02 Return Ap Id Type Receptacle Occupant C
156. unication path automatically When using the GS SURE linkage mode in Redundant Line Control Function of SynfinityLink or PRIMECLUSTER GLS it is necessary to deactivate virtual interfaces e g sha0 used by GS SURE linkage mode Fast switching mode in Redundant Line Control Function opt FJS Vhanet usr sbin hanetnic delete n sha0 i hmel Return usr sbin ifconfig hmel unplumb Return NIC switching mode in Redundant Line Control Function opt FJSVhanet usr sbin hanetpoll off lt Return gt lopt FJSVhanet usr sbin stpptl n shal lt Return gt opt FJSVhanet usr sbin hanetnic change n sha0 lt Return gt GS SURE linkage mode in Redundant Line Control Function lopt FJSVhanet usr sbin stphanet n sha0 lt Return gt Multipath Function usr opt FJSViomp bin iompadm c mpnet stop mpnet0 hmel lt Return gt usr opt FJS Viomp bin iompadm c mpnet disconnect mpnet0 hmel lt Return gt 4 Disconnect the PCI card a Please specify the PCI bus slot position by the interface e g hmel in procedure 2 Specify the replacing PCI card usr sbin FJSVmadm inst2comp hmel lt Return gt peipsy21 R0B01 PCIZslot03 b Obtained Ap Id is specified as an argument of the cfgadm 1M command and it confirms that the slot status of the PCI card that disconnects is connected configured cfgadm pcipsy21 ROB01 PCIZslot03 Return Ap Id Type Receptacle Occupant Condition pcipsy21 ROB01 PC
157. ure 1 and added in procedure 5 1 There may be a change of status depending on the system operating state after pre definition in procedure 1 so check the LUN definition conflict between sd conf and hddv conf again In case of Check NG the definitions are conflicting so go back to procedure 1 e Check OK to procedure 9 b 2 opt FJSVhddv bin hddvchk conf Return kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 1 e opt FJSVhddv bin hddvchk conf Return NGID 3 2 Execute the following command to make the hard disk driver set up the LUN usr sbin hddvadmdr lt Return gt 3 After saving execution result of the prtpicl command get the physical path name of added LUN by taking the difference between the result and the information obtained in 4 a and calculate the corresponding logical path name under dev dsk In the following example c3t3d0 is logical path name of added lun 0 The logical path name of lun 1 2 is set to c3t3d1 c3t3d2 respectively because cXtYdZmeans X PCI card number Y target number Z lun number prtpicl v gt tmp prtpicl post lt Return gt diff tmp prtpicl pre tmp prtpicl post grep devfs path grep hddv lt Return gt gt devfs path pci 89 2000 fibre channel 1 hddv 3 0 gt devfs path pci 89 2000 fibre channel 1 hddv 93 1 gt devfs path pci 89 2000 fibre channel 1 hddv 3 2 Is
158. v class scsi target 1 lun 2 name hddv class scsi target 3 lun 0 name hddv class scsi target 3 lun 1 remove comments of extended LUN name hddv class scsi target 3 lun 2 name hddv class scsi target 3 lun 3 Check the LUN definition overlap between sd conf and hddv conf by executing hddvchk_conf command The message example for Check NG means that target 3 is defined in both files and the procedures must be performed again from procedure 8 b 4 The followings are examples of execution of the FJSVhddv package installation to opt The path of the hddvchk_conf command depends on the installed directory Check OK to procedure 8 b 7 opt FJSVhddv bin hddvchk conf lt Return gt kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 8 b 4 opt FJSVhddv bin hddvchk conf lt Return gt NGID 3 Execute the following command in order to make the hard disk driver set up the LUN added update_drv f hddv lt Return gt In case disk array equipment currently used exists the following messages are displayed but do not effect performance update_drv f hddv lt Return gt Cannot unload module hddv Will be unloaded upon reboot Forcing update of hddv conf After saving execution result of the prtpicl 1M command get the physical path name of added LUN by taking the difference between the result and the information obtained in 3 a
159. v FJS V mplb rdsk mplb8s5 dev FJS V mplb rdsk mplb8s6 dev FJS V mplb rdsk mplb8s7 11 Setup applications Add configuration for the application using the logical path name determined in 9 filesystem operation Create file system To mount at system start add an entry in the etc vfstab file newfs dev F JS V mphd rdsk mphd2s6 Return vi etc vfstab Return dev FJS V mphd dsk mphd2s6 dev FJS Vmphd rdsk mphd2s6 mnt ufs 1 yes raw access operation Configure applications For details see the manual of each application swap device Add swap device To make valid at system start add an entry in the etc vfstab file swap a dev FJSV mphd dsk mphd2s4 Return vi etc vfstab Return dev FJS V mphd dsk mphd2s4 swap no swap l Return swapfile dev swaplo blocks free dev FJS V mphd dsk mphd2s4 32 164 16 788384 683680 12 Start applications Use the logical path name determined in 9 to start applications filesystem operation Mount filesystem mount F ufs dev F JS Vmphd dsk mphd2s6 mnt Return raw access operation Start applications For details see the manual of each application swap device This procedure is not required 3 2 1 3 Extension of PCI cards to redundant system The procedure to add PCI cards to a redundant system in order to extend redundancy on a Solaris 8 OE system is explained with an exam
160. vers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration a Before extension of PCI cards save the current execution result of the prtpicl IM command current configuration at present prtpicl v gt tmp prtpicl pre Return b Check the status of the slot The relationship of the position of a PCI card and format of Ap Id is shown below Location of PCI slot Element Format of Ap Id Solaris 8 OE and Solaris 9 OE Cabinet of Cabinet Number X PRIMEPOWER System Board Number Y g E Md 900 1500 2500 Physical Slot Number ZZ as CXMOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN RXBOY PCI slotZZ Solaris 10 OE RXBOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN PCI slotZZ Solaris 10 OE PCI ZZ Below an example of adding a PCI card to chassis number 0 system board number 1 physical slot number 02 slot in chassis COMO1 PCI slot02 is explained Check the status of the target slot is empty unconfigured cfgadm grep COM01 PCI slot02 Return pcipsy15 COMO1 PCIZslot02 unknown empty unconfigured unknown T O Cabinet Number X PCI Disk Box PCI Disk Box Number Y Physical Slot Number ZZ other than those above Physical Slot Number ZZ 21 22 c To confirm the slot position at extension operation blink t
161. wing indicate to bind the auto mapped target with the card usr sbin Ipfc lputil lt Return gt LightPulse Common Utility for Solaris SPARC Version 1 6a8 03 03 2004 Copyright c 2003 Emulex Network Systems Inc Emulex Fibre Channel Host Adapters Detected 2 Host Adapter 0 Ipfc11 is an LP9802 Ready Mode Host Adapter 1 Ipfc9 is an LP9802 Ready Mode MAIN MENU 1 List Adapters 2 Adapter Information 3 Firmware Maintenance 4 Reset Adapter 5 Persistent Bindings 0 Exit Enter choice gt 5 Return PERSISTENT BINDINGS MENU 1 Display Current Bindings 2 Display All Nodes 3 Duplicate Adapter Bindings 4 Bind Target Manually 5 Bind Automapped Targets 6 Delete Binding s 0 Return to Main Menu Enter choice 2 5 Return O Ipfc11 1 Ipfc9 Select an adapter gt 0 lt Return gt Automapped FCP Nodes Node Target WWPN WWNN 49 Automapped FCP Node 0 50 06 0e 80 00 10 04 92 50 06 0e 80 00 10 04 92 Automapped FCP Node 1 50 06 0e 80 00 10 04 90 50 06 0e 80 00 10 04 90 Press the Enter or Return key to continue Bind all auto mapped targets Y N gt Y Return Select Binding Method 1 By Node Name 2 By Port Name 3 ByD ID 0 Cancel Enter Choice gt 2 Return 9 Setup applications Add configuration for the application using the logical path name determined in 8 b Disk device filesystem operation Create file system To mount at sys
162. x 1 root root 41 Sep 20 22 53 c2t2d0s1 gt devices pci 89 4000 scsi 9 2 sd 2 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s2 gt devices pci 89 4000 scsi 9 2 sd 9 2 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s3 gt devices pci 89 4000 scsi 2 sd 2 0 d Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s4 gt devices pci 89 4000 scsi 2 sd 9 2 0 e Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s5 gt devices pci 89 4000 scsiQ 2 sd 9 2 0 f Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s6 gt devices pci 89 4000 scsi 2 sd 9 2 0 g Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s7 gt devices pci 89 4000 scsi 2 sd 2 0 h Tape device diff tmp prtpicl pre tmp prtpicl post grep devfs path grep st lt Return gt gt devfs path pci 89 4000 scsi Q2 st 9 5 0 Is 1 dev rmt grep pci 289 4000 scsi 2 st 5 0 lt Return gt Irwxrwxrwx root root 41 Sep 20 22 53 1 gt devices pci 89 4000 scsi 2 st 5 0 Irwxrwxrwx 1 root root 41 Sep 20 22 53 1b gt devices pci 89 4000 scsi Q 2 st 5 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 Ibn gt devices pci 89 4000 scsi 2 st 9 5 0 bn Irwxrwxrwx 1 root root 41 Sep 20 22 53 1c gt devices pci 89 4000 scsi Q 2 st 5 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 Icb gt devices pci 89 4000 scsi 2 st 5 0 cb Irwxrwxrwx 1 root root 41 Sep 20 22 53 1cbn gt devices pci
163. xplained assuming the console messages above b Find the hddv1 with the status offline fail from the results of the iompadm command and determine the logical path name of the LUN connected to the target PCI card The following is an example of MPHD and dev rdsk c3t1d0s2 is the logical path name corresponding to hddvl In case of GRMPD specify mplb in the c option parameter of the iompadm command In case of updated 15 GRMPD from MPHD specify the mphd as c option parameter of the iompadm command usr opt FJS Viomp bin iompadm c mphd p info Return IOMP dev FJS Vmphd fiomp adm2 gt devices pseudo mphd 2 adm Element dev rdsk c3t1d0s2 offline fail block target completed hard reset sequence GR7104546 010000 00 00 30 hddv1 gt devices pci 8d 2000 fibre channel 1 hddv 1 0 c raw dev rdsk c2t2d0s2 online active block good status with active GR7104546 010000 01 01 32 hddv15 gt devices pci 89 2000 fibre channel 1 hddv 9 2 0 c raw Node dev FJS V mphd rdsk mphd2s0 dev FJS V mphd rdsk mphd2s1 dev FJS V mphd rdsk mphd2s2 dev FJS V mphd rdsk mphd2s3 dev FJS V mphd rdsk mphd2s4 dev FJS V mphd rdsk mphd2s5 dev FJS V mphd rdsk mphd2s6 dev FJS V mphd rdsk mphd2s7 Function MPmode false AutoPath true Block true NeedSync false c Disconnect from redundant system Disconnect the path between the target PCI card and I O devices The following procedure i
164. xtension of PCI cards to the slot ROBOI PCIZslot02 The ROBO1 PCI slot02 indicates the I O cabinet number 0 the PCI Disk Box number 1 physical slot number 02 In the extension of a new redundant system although two or more PCI cards are extended whenever you add a PCI card please perform from procedure 2 b to 3 Connect PCI cards The relationship of the position of a PCI card and format of Ap Id is shown below N an integral number Location of PCI slot Element Format of Ap Id Solaris 8 OE and Solaris 9 OE Cabinet of Cabinet Number X PRIMEPOWER System Board Number Y gi genre pce 900 1500 2500 Physical Slot Number ZZ ALIR CXMOY PCI ZZ Solaris 8 OE and Solaris 9 OE pcipsyN RXBOY PCI slotZZ Solaris 10 OE RXBOY PCI ZZ Solaris 8 OE and Solaris 9 OE I O Cabinet Number X PCI Disk Box PCI Disk Box Number Y Physical Slot Number ZZ ipsyN PCIZslotZZ ther than th Physical Slot N ZZ PERY other than those above ysical Slot Number Solaris 10 OE PCI ZZ Please confirm that the PCI slot status which extended PCI cards is empty unconfigured by using cfgadm 1M command cfgadm grep ROBO01 PCI slot02 lt Return gt pcicsy18 ROBO1 PCI slot02 unknown empty unconfigured unknown c To confirm the slot position at extension operation blink the ALARM LED of Ap Id displayed in procedure 2 b cfgadm x led fault mode blink pcipsy18 ROB01 PCI slot02 lt Re
165. y exists when a tape device is added but does not effect performance Disk device update drv f sd Return Cannot unload module sd Will be unloaded upon reboot Forcing update of sd conf Tape device update drv f st Return 3 After saving the execution result of the prtpicl 1M command take the difference between the result and the information obtained in 3 a and get the physical path name of additional disk device then get the logical path name under dev dsk In the following example c2t2d0 is the logical path name of the disk device added prtpicl v gt tmp prtpicl post Return Disk equipment diff tmp prtpicl pre tmp prtpicl post grep devfs path grep sd Return gt devfs path pci 89 4000 scsi Q2 sd 92 0 Is 1 dev dsk grep pci 89 4000 scsi 2 sd 2 0 Return Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s0 gt devices pci 89 4000 scsi 2 sd 9 2 0 a Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s1 gt devices pci 89 4000 scsi 9 2 sd 2 0 b Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s2 gt devices pci 89 4000 scsi 2 sd 9 2 0 c Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s3 gt devices pci 89 4000 scsi 9 2 sd 2 0 d Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s4 gt devices pci 89 4000 scsi 9 2 sd 9 2 0 e Irwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s5 gt devices pci 89 4000 scsi 9 2 sd 9 2 0 f Irwxrwx

PCI Hot Plug User`s Guide I/O devices edition

Contents

Download Pdf Manuals

Related Search

Related Contents