Home

PCI Hot Plug User`s Guide I/O device edition

1. prtpicl v gt tmp prtpicl post lt Return gt diff tmp prtpicl pre tmp prtpicl post grep devfs path grep hddv lt Return gt gt devfs path pci 8d 2000 fibre channel 1 hddv 3 0 gt devfs path pci 8d 2000 fibre channel 1 hddv 3 1 gt devfs path pci 8d 2000 fibre channel 1 hddv 3 2 gt devfs path pci 89 2000 fibre channel 1 hddv 3 0 gt devfs path pci 89 2000 fibre channel 1 hddv 3 1 gt devfs path pci 89 2000 fibre channel 1 hddv 3 2 ls 1 dev dsk grep pci 8d 2000 fibre channel 1 hddv 3 0 lt Return gt lrwxrwxrwx root root 41 Sep 20 22 53 c3t3d0s0 gt devices pci 8d 2000 fibre channel 1 hddv 3 0 a lrwxrwxrwx root root 41 Sep 20 22 53 c3t3d0sl gt devices pci 8d 2000 fibre channel 1 hddv 3 0 b lrwxrwxrwx root root 41 Sep 20 22 53 c3t3d0s2 gt devices pci 8d 2000 fibre channel 1 hddv 3 0 c lrwxrwxrwx root root 41 Sep 20 22 53 c3t3d0s3 gt devices pci 8d 2000 fibre channel 1 hddv 3 0 d lrwxrwxrwx root root 41 Sep 20 22 53 c3t3d0s4 gt devices pci 8d 2000 fibre channel 1 hddv 3 0 e lrwxrwxrwx root root 41 Sep 20 22 53 c3t3d0s5 gt devices pci 8d 2000 fibre channel 1 hddv 3 0 f lrwxrwxrwx root root 41 Sep 20 22 53 c3t3d0s6 gt devices pci 8d 2000 fibre channel 1 hddv 3 0 g lrwxrwxrwx root root 41 Sep 20 22 53 c3t3d0s7 gt devices pci 8d 200
2. ccccccccssscceeeseeeeesneeeesenseeeseneeeensseeeesseeeeens 65 4 1 2 Replacement of PCI cards on redundant system cccccccescccsessseceeseneeecseseeeesenseeeesseeeeseeeeeeeseesenes 67 4 2 Extension of P C Ica ds ELSE iia aa DE eda Debesesuidacee sabes E AE ed eres 69 4 2 1 Extension of PCI cards on non redundant SYStem cccccccesccesssseeeessnsecessneeeecseeeeeseneeeeessaeeeseneeeens 69 4 2 2 Extension of PCI cards with new redundant System ccccccccccceseesssseeeccccesessseeeeeccesensssseeeeeeeeees 71 4 2 3 Extension of PCI cards to redundant SyStem ccccccccccccccsssssseeecceeecesssseeeeecceseesnsseeeseeeeeesssseeeeeecensea 73 Chapter 5 Notes ReStrictions cccccccccssseeecsseececcaeeeeeecsseeeceeesseeeeeessaeeeeessnaaes 1T 51 NOLS iit anene ra a aee a aE e a aa aa aa a ala aai a e A aea E 77 5 1 1 Network driver configuration changing ccccccccccccscssssssceeceeceeeesssseecccceeseessseeeeeceeseesnsseeeseseeesaeeees 77 5 1 2 Using tape device while PCI Hot Plug operation ccccccccssscccssssecesssseeeesssseeessseeecseseesesssteeeesaes 78 DB LS OIA oter A NENTA E ail ea esas T EE fe edt cast T dec stee a ER ER ESKE KENDES ERR RE REDER NESS 78 57 RESUICUONS cree EA E a a ae a ce ee teas cetaceans 78 iii iv Chapter 1 PCI cards with support for PCI Hot Plug This chapter shows the list of PCI cards supporting PCI Hot Plug feature The driver patches and the option driver packages
3. cfgadm pcipsy21 R0B01 PCI slot03 lt Return gt Ap Id Type Receptacle Occupant Condition pcipsy21 ROBO0O1 PCI slot03 pci pci hp connected configured ok c Please disconnect the PCI card by executing the cfgadn 1M command with Ap_Id and confirm that the slot status is disconnected unconfigured cfgadm c disconnect pcipsy21 ROB01 PCI slot03 lt Return gt cfgadm pcipsy21 ROB01 PCI slot03 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy21 ROBO1 PCI slot03 pci pci hp disconnected unconfigured unknown d To confirm the slot position at replacement operation blink the ALARM LED of Ap_Id displayed in procedure 4 a cfgadm x led fault mode blink pcipsy21 ROB01 PCI slot03 lt Return gt 5 Replace the PCI card The PCI card disconnected in procedure 4 Disconnect the PCI card is exchanged for a new PCI card Out customer support works about this procedure Notes When using the PRIMECLUSTER GLS Multipath Function please exchange the PCI card after waiting 10 seconds or more after Procedure 4 Disconnect the PCI card Connect the PCI card The cfgadm command is executed with the configure option to Ap Id or the push button of the PCI bus slot is pushed Then a new PCI card is connected In addition a push button is valid only in multiple user mode Please confirm that the slot status is connected configured by using the cfgadm 1M command after the
4. pcipsy18 ROBO1 PCI slot02 mult hp connected configured ok 4 Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJSVmadm postphp lt Return gt 5 Connect equipment to PCI cards Connect new disk array device in redundant configuration to the extended PCI card When the Host Affinity function is being used with Fibre Channel switch SN200 series Host Affinity must be configured for the device added When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility WWPN zoning of SN200 series Host affinity of Array disk device ETERNUS3000 GR700 800 series and binding setting for fjpfca conf sd conf of host can be added from SAN management window automatically See Operation guide or User s Manual of each product for details Before using this function the following procedure 6 a i and 6 a ii except fep bind target setting are necessary Moreover begin from procedure 6 a iii after executing this function When not using SAN management function of the above products SN200 series need to be reconfigured individually For details see SN200 Series Affinity User s Guide or ETERNUS3000 GR700 800 series GRmgr User s Guide
5. 3 Execute the following command in order to make the hard disk driver set up the LUN added update drv f hddv lt Return gt In case disk array equipment currently used exists the following messages are displayed but do not effect performance update drv f hddv lt Return gt Cannot unload module hddv Will be unloaded upon reboot Forcing update of hddv conf After saving execution result of the prtpicl 1M command get the physical path name of added LUN by taking the difference between the result and the information obtained in 2 a and calculate the corresponding logical path name under dev dsk In the following example c3t3d0 is logical path name of added lun 0 The logical path name of lun 1 2 is set to c8t3d1 c8t8d2 respectively because cXtYdZmeans X PCI card number Y target number Z lun number prtpicl v gt tmp prtpicl post lt Return gt diff tmp prtpicl pre tmp prtpicl post grep devfs path grep hddv lt Return gt gt devfs path pci 89 2000 fibre channel 1 hddv 3 0 gt devfs path pci 89 2000 fibre channel 1 hddv 3 1 gt devfs path pci 89 2000 fibre channel 1 hddv 3 2 ls 1 dev dsk grep pci 89 2000 fibre channel 1 hddv 3 0 lt Return gt lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s gt devices pci 89 2000 fibre channel 1 hddv 3 0 a lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0sl gt devices pci 89 2
6. Commands and other input use the following prompts C shell prompt prompt Bourne and korn shell prompt prompt Super user prompt Commands entered by the user is shown in bold it cfgadm c configure pcipsy2 ROB00 PCI slot01 lt Return gt Key combinations are represented for example by CTL C which means to simultaneously press the Control and C key Reference Manuals If necessary refer to the following manuals PCI Hot Plug User s Guide Dynamic Reconfiguration User s Guide Trademarks Sun Sun Microsystems the Sun Logo Solaris and all Solaris based marks and logos are trademarks or registered trademarks of Sun Microsystems Inc in the U S and other countries and are used under license All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International Inc in the U S and other countries Products with the SPARC trademark are based on architecture developed by Sun Microsystems All other products or company names mentioned in this document are claimed as trademark and trade names by their respective companies FUJITSU LIMITED April 2003 January 2003 First Edition April 2003 Second Edition Notes This manual may not be copied by any means without the expressed written permission of FUJITSU LIMITED FUJITSU LIMITED reserves the right to make changes to this manual without prior notice Copyright 1995 GLOBEtrotter Software
7. If the Host Affinity function of ETERNUS3000 GR700 800 series is used When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility go to procedure 3 a because these products get WWPNs from 35 36 PCI cards automatically The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digits of the WWPN in hexadecimal form The top eight digits are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the adding card is 100000000e44061 0e24 4061 a Before extension of PCI cards save the current execution result of the prtpicl 1M command current configuration at present prtpicl v gt tmp prtpicl pre lt Return gt b Check the status of the slot When adding PCI cards to a redundant configuration several PCI cards may be added to the system In this case perform to procedures 4 on each card The relationship of the position of a PCI card and format of Ap_Id is shown below Location of PCI slot Definition of slot Format of Ap_Id Chassis No X System board No Y pcipsyN CXMOY PCI slotZZ Physical slot No ZZ I O Chassis No X Inside chassis of PRIMEPOWER 900 1500 2500 PCI Disk BOX PCI Disk BOX No Y pcipsyN RXBOY PCI
8. a Add configuration for PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver Configuration is not required Fibre Channel driver FUJITSU PCI Fibre Channel i After saving execution result of prtpicl 1M command take the difference between the result and the information obtained in procedure 2 a to calculate the instance number of the fjpfca driver of the PCI card added In the following example instance number is 1 indicating that the interface name of the PCI card added is fjpfcal Making the LED of the PCI card blink using the fe_adm command can specify the relation between the instance number and the slot position The relation between the WWPN and instance number of the PCI card added can be known from the fc info a command prtpicl v gt tmp prtpicl pfca lt Return gt diff tmp prtpicl pre tmp prtpicl pfca more lt Return gt gt status okay gt fru PCI slot PCI 02 at Rack 0 IO _BOX 1 gt component name ROBO1 PCI 02 gt devfs path pci 8d 2000 fibre channel l gt driver name fjpfca gt binding name pcilOcf 105c 51 52 gt bus addr 1 gt instance 1 gt _class obp device gt inam fibre channel ii Edit the kernel drv fjpfca conf file and add the following property definitions to the instance number determined in 6 a i port type of topology link speed Defined when adding PCI cards with Fabric connection
9. cfgadm c configure pcipsy21 ROB01 PCI slot03 lt Return gt it cfgadm pcipsy21 ROB01 PCI slot03 lt Return gt Ap Id Type Receptacle Occupant Condition pcipsy21 ROBO1 PCI slot03 fibre hp connected configured ok When not using SAN management function of the above products This procedure is not required 7 Connect to redundant system Execute the following command specifying the logical path name from 2 b to connect to the redundant system If the operating path switched in procedure 3 the operating path will be automatically switched back If both MPHD GRMPD controlled devices are connected to the PCI card to replace the commands for both products need to be executed MPHD usr opt FJSViomp bin iompadm c mphd restart adapter connect dev rdsk c3t1d0s2 lt Return gt Q RMPD usr opt FJSViomp bin iompadm c mplb restart adapter connect dev rdsk c3t1d0s2 Return gt A 8 Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJSVmadm postphp lt Return gt 3 2 Extension of PCI cards This section describes the procedures to add PCI cards using the PCI Hot Plug feature on non redundant and 15 redundant systems with Solaris 8 OE and Solaris 9 OE The difference
10. Not exist new install to procedure 1 c ls kernel drv hddv conf lt Return gt 33 34 kernel drv hddv conf No such file or directory Exists operation with non active extension procedures to procedure 1 b ls kernel drv hddv conf lt Return gt kernel drv hddv conf When operating with non active extension procedures execute the following commands and confirm that kernel drv hddv conf is deleted hddvadm i lt Return gt ls kernel drv hddv conf lt Return gt kernel drv hddv conf No such file or directory Use the following command to create an hddv conf file for the pre defined extension function The message examples for operation with non active extension procedures means that GR720 currently used with taget 1 lun 0 is registered twice on the sd hddv driver New installation to procedure 1l e hddvadm M lt Return gt Set up with a non active extension procedure to procedure 1 d hddvadm M lt Return gt already ID 1 exists devices pci ld 2000 fibre channel 1 sd 1 0 a raw GR720 When performing pre defined extension the same target number cannot be registered in sd conf and hddv conf so delete or comment out the conflicting definition parts in sd conf file when definition in the two files conflict with each other The following example shows the case that target 1 is repeated vi kernel drv sd conf lt Return gt
11. Please switch an online communication path for extended communication path if needed Please refer to manuals of each redundant configuration software about how to switch a communication path Chapter 5 Notes Restrictions This chapter describes notes and restrictions in case a PCI card exchanges or extends by using PCI Hot Plug feature 5 1 Notes This section describes notes when exchanging and extending PCI cards by using PCI Hot Plug feature 5 1 1 Network driver configuration changing When configuring parameters in driver conf 4 file and replacing the PCI card by using the PCI Hot Plug feature parameters which configured in driver conf 4 file are not valid except for following conditions In the Solaris 9 OE system by which only one PCI card that is supported by one driver is mounted please execute the following command after the replacement procedure 6 Connect the PCI card when replacing the PCI card by using the PCI Hot Plug feature Consequently parameters which configured in driver conf 4 file are reflected update drv driver name lt Return gt In the configuration of the driver conf 4 file when not specifying name parent unit address parameters configured parameters are reflected However all the PCI cards that the driver supports become the same configuration When not corresponding to the above mentioned conditions and configuring parameters after the PCI card is replaced or extended
12. above mentioned procedure Notes If the setup of driver conf 4 file exist please refer to 5 1 1 Network driver configuration changing cfgadm c configure pcipsy21 ROBO1 PCI slot03 lt Return gt cfgadm pcipsy21 ROB01 PCI slot03 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy21 ROBO01 PCI slot03 pci pci hp connected configured ok Connect to redundant system In order to connect the exchanged PCI card into a redundant system please execute the following command with interface name specified at procedure 2 Specify the replacing PCI card Fast switching mode in Redundant Line Control Function usr sbin ifconfig hmel plumb lt Return gt usr sbin ifconfig hmel 192 168 10 10 netmask broadcast trailers up lt Return gt opt FJSVhanet usr sbin hanetnic add n sha0 i hmel lt Return gt NIC switching mode in Redundant Line Control Function opt FJSVhanet usr sbin hanetnic change n sha0 lt Return gt opt FJSVhanet usr sbin strptl n shal lt Return gt 4 2 GS SURE linkage mode in Redundant Line Control Function opt FJSVhanet usr sbin strhanet n sha0 lt Return gt Multipath Function usr opt FJSViomp bin iompadm c mpnet connect mpnet0 hmel lt Return gt usr opt FJSViomp bin iompadm c mpnet start mpnet0 hmel lt Return gt Switch back the redundant path Please switch back an online communication path if needed Start the hardware
13. name sd class scsi target 0 lun 0 name sd class scsi target 1 lun 0 name sd class scsi Delete or comment out target 1 lun 1 the duplication definition parts in sd hddv name sd class scsi target 1 lun 2 A target number that is already used by the sd driver of the system disk or CD ROM cannot be deleted In this case change the target number in hddv conf and of the disk array device so that it does not conflict with sd conf Edit the kernel drv hddv conf file created automatically by the hddvadm M command and remove comments of the LUN definition to pre define and the LUN definition currently being used with non active extension procedures If case active extension has been performed before and the LUN definition to extend the next procedures 1 f 1 g are not needed The following example shows target 3 lun 0 2 for pre defined extension and disk array device GR720 target 1 in use with non active extension procedures being registered ls kernel drv hddv conf lt Return gt kernel drv hddv conf vi kernel drv hddv conf lt Return gt name hddv class scsi target 0 lun 0 name hddv class scsi target 0 lun 7 name hddv class scsi target 1 lun 0 name hddv class scsi target 1 lun 1 remove comments of the LUN in use name hddv class scsi target 1 lun 2 name hddv class scsi target 3 lun 0 name hddv class scsi target 3 lun 1 remove
14. respectively because cXtYdZmeans X PCI card number Y target number Z lun number prtpicl v gt tmp prtpicl post lt Return gt lt Return gt gt devfs path pci 89 2000 fibre channel 1 hddv 3 0 gt devfs path pci 89 2000 fibre channel 1 hddv 3 1 gt devfs path pci 89 2000 fibre channel 1 hddv 3 2 ls 1 dev dsk grep pci 89 2000 fibre channel 1 hddv 3 0 lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s0 gt devices pci 89 2000 fibre channel 1 hddv 3 0 a lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0sl gt devices pci 89 2000 fibre channel 1 hddv 3 0 b lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s2 gt devices pci 89 2000 fibre channel 1 hddv 3 0 c lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s3 gt devices pci 89 2000 fibre channel 1 hddv 3 0 d lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s4 gt devices pci 89 2000 fibre channel 1 hddv 3 0 e lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s5 gt devices pci 89 2000 fibre channel 1 hddv 3 0 f lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s6 gt devices pci 89 2000 fibre channel 1 hddv 3 0 g lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s7 v 255 gt devices pci 89 2000 fibre channel hddv 3 0 h diff tmp prtpicl pre tmp prtpicl post grep devfs path grep hddv lt Return gt ix Start the format 1M command and
15. scsi target 0 lun 0 name sd class scsi In case the definition does not exist target 2 lun 0 add or remove comments Tape device vi kernel drv st conf lt Return gt name st class scsi target 0 lun 0 name st class scsi In case the definition does not exist target 5 lun 0 add or remove comments In case kernel drv sd conf was updated in 1 a reconfigure the kernel restart the system and then start system operation If it was not updated this procedure is not required touch reconfigure lt Return gt shutdown y i6 g0 lt Return gt Disk Array device hard disk driver hddv This is an example where a disk array device at target 3 lun 0 2 is pre defined 16 If this is the first time to pre define a disk array device start from procedure 1 a If this is not the first time to pre define a disk array device start from procedure 1 e a Check if kernel drv hddv conf exists If it does not exist the hard disk driver is being installed so start from procedure 1 c If it does exist the system is being operated with non active extension procedures so start from procedure 1 b Not exist new install to procedure 1 c ls kernel drv hddv conf lt Return gt kernel drv hddv conf No such file or directory Exists operation with non active extension procedures to procedure 1 b ls kernel drv hddv conf lt Return gt kernel drv hddv
16. 20 22 53 1hb gt devices pcil89 4000 scsi 2 st 5 0 hb lrwxrwxrwx 1 root root 41 Sep 20 22 53 1hbn gt devices pci 89 4000 scsi 2 st 5 0 hbn lrwxrwxrwx 1 root root 41 Sep 20 22 53 ihn gt devices pcit89 4000 scsi 2 st 5 0 hn If the device is a disk device start the format 1M command and configure labeling and partition slice information of the logical path name of added disks If the device is a tape device this procedure is not needed Disk array device hard disk driver hddv This example is a case where a disk array device of target 3 lun 0 2 is pre defined in procedure 1 and added in procedure 6 i ii iii There may be a change of status depending on the system operating state after pre definition in procedure 1 so check the LUN definition conflict between sd conf and hddv conf again In case of Check NG the definitions are conflicting so go back to procedure 1 d Check OK to procedure 7 b ii opt FJSVhddv bin hddvchk conf lt Return gt kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 1 d opt FISVhddv bin hddvchk_conf lt Return gt NG ID 3 Execute the following command to make the hard disk driver set up the LUN usr sbin hddvadmdr lt Return gt After saving execution result of the prtpicl command get the physical path name of added LUN by
17. 41 Sep 20 22 53 c2t2d0s2 gt devices pci 89 4000 scsi 2 sd 2 0 c lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s3 gt devices pcit89 4000 scsi 2 sd 2 0 d lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s4 gt devices pci 89 4000 scsi 2 sd 2 0 e lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s5 gt devices pcit89 4000 scsi 2 sd 2 0 f lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s6 gt devices pcit89 4000 scsi 2 sd 2 0 g lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s7 gt devices pcit89 4000 scsi 2 sd 2 0 h Tape equipment diff tmp prtpicl pre tmp prtpicl post grep devfs path grep st lt Return gt gt devfs path pci 89 4000 scsi 2 st 5 0 ls l dev rmt grep pci 89 4000 scsi 2 st 5 0 lt Return gt lrwxrwxrwx 1 root root 41 Sep 20 22 53 gt devices pcit89 4000 scsi 2 st 5 0 lrwxrwxrwx 1 root root 41 Sep 20 22 53 1b gt devices pcit89 4000 scsi 2 st 5 0 b lrwxrwxrwx 1 root root 41 Sep 20 22 53 1bn gt devices pcil89 4000 scsi 2 st 5 0 bn lrwxrwxrwx 1 root root 41 Sep 20 22 53 lc gt devices pci 89 4000 scsi 2 st 5 0 c lrwxrwxrwx 1 root root 41 Sep 20 22 53 1cb gt devices pcil89 4000 scsi 2 st 5 0 cb lrwxrwxrwx 1 root root 41 Sep 20 22 53 1cbn gt devices pci 89 4000 scsi 2 st 5 0 cbn lrwxrwxrwx 1 root root 41 Sep 20 22 53 Icn gt devices pcit89 4000 scsi 2 st 5 0 cn
18. Host Affinity function is being used with Fibre Channel switch SN200 series Host Affinity must be configured for the device added When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility WWPN zoning of SN200 series Host affinity of Array disk device ETERNUS3000 GR700 800 series and binding setting for fjpfca conf sd conf of host can be added from SAN management window automatically See Operation guide or User s Manual of each product for details Before using this function the following procedure 6 a i and 6 a ii except fcep bind target setting are necessary Moreover begin from procedure 6 a iii after executing this function When not using SAN management function of the above products SN200 series need to be reconfigured individually For details see SN200 Series Affinity User s Guide or ETERNUS3000 GR700 800 series GRmgr User s Guide Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration 6 Set up driver Add driver configurations with the operation below With file related drivers configuration for the PCI card driver and the device driver is needed
19. Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration Set up driver Add driver configurations with the operation below With file related drivers configuration for the PCI card driver and the device driver is needed a Add configuration for PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver Configuration is not required Fibre Channel driver FUJITSU PCI Fibre Channel i After saving execution result of prtpicl 1M command take the difference between the result and the information obtained in procedure 2 a to calculate the instance number of the fjpfca driver of the PCI card added In the following example instance number is 1 indicating that the interface name of the PCI card added is fjpfcal Making the LED of the PCI card blink using the fc adm command can specify the relation between the instance number and the slot position The relation between the WWPN and instance number of the PCI card added can be known from the fc info a command prtpicl v gt tmp prtpicl pfca lt Return gt diff tmp prtpicl pre tmp prtpicl pfca more lt Return gt gt status okay gt eol PCI slot PCI 02 at Rack 0 I
20. SN200 series I O to other devices is effected by the change and may result in temporal errors T O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration b Reconnect the replaced PCI card with changed zonig setting by using the following commands cfgadm c disconnect pcipsy3 COM00 PCT slot02 lt Return gt cfgadm c configure pcipsy3 COM00 PCI slot02 lt Return gt it cfgadm pcipsy3 COM00 PCI slot02 lt Return gt Ap Id Type Receptacle Occupant Condition pcipsy3 COMO00 PCI slot02 mult hp connected configured ok When not using SAN management function of the above products This procedure is not required Start applications Restart the stopped applications with the following operations a Start vold sh etc init d volmgt start lt Return gt b Restart the usage of the device stopped in procedure 3 b If the device is a disk unit file system operation Mount the unmounted filesystem and resume usage mount export home lt Return gt mount develop firm lt Return gt mount develop drv lt Return gt mount pub lt Return gt 11 If the device is a disk unit raw access operation or a tape device Restart applications and resume usage Refer to the manual of each application for details If the device is a d
21. between the procedures for the two OEs is that I O devices to be added must be pre defined before system operation with Solaris 8 OE but pre definition is not needed with Solaris 9 OE 3 2 1 Solaris 8 OE The procedures to add PCI cards on a Solaris 8 OE system are explained here To use the PCI Hot Plug function for extension on Solaris OE I O devices to be added must be pre defined and the system must be rebooted before system operation If the I O devices to be added are already defined pre definition is not needed and procedure 1 below can be skipped 3 2 1 1 Extension of PCI cards on non redundant system Procedures to add PCI cards without making them redundant on a Solaris 8 OE system are explained here 1 Setup I O devices which will be added to system Pre define devices to be added with the following procedures according to the device type Extension using PCI Hot Plug cannot be done without definition so devices must be defined before the system goes into operation Disk Tape devices Solaris OE sd st This is an example where a disk device at target 2 lun 0 is pre defined If the device is a tape device re read sd in the following as st and perform the same operations a Confirm that target 2 lun 0 is defined in kernel drv sd conf If this definition does not exist add the definition If it is commented out remove the comments Disk device vi kernel drv sd conf lt Return gt name sd class
22. conf When operating with non active extension procedures execute the following commands and confirm that kernel drv hddv conf is deleted hddvadm i lt Return gt ls kernel drv hddv conf lt Return gt kernel drv hddv conf No such file or directory Use the following command to create an hddv conf file for the pre defined extension function The message examples for operation with non active extension procedures means that GR720 currently used with taget 1 lun 0 is registered twice on the sd hddv driver New installation to procedure 1 e hddvadm M lt Return gt Operation with non active extension procedures to procedure 1 d hddvadm M lt Return gt already ID 1 exists devices pci ld 2000 fibre channel 1 sd 1 0 a raw GR720 When performing pre defined extension the same target number cannot be registered in sd conf and hddv conf so delete or comment out the conflicting definition parts in sd conf file when definition in the two files conflict with each other The following example shows the case that target 1 is repeated vi kernel drv sd conf lt Return gt name sd class scsi target 0 lun 0 name sd class scsi target 1 lun 0 name sd class scsi Delete or comment out target 1 lun 1 conflicting definition parts in sd hddv name sd class scsi target 1 lun 2 A target number that is already used by the sd drive
23. corresponding to the disk device sd20 connected to the target PCI card In the example below c2t3d0 is the logical path corresponding to sd20 ls 1 dev dsk grep pci 89 4000 scsi 2 sd 3 0 lt Return gt lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s0 gt devices pcit89 4000 scsi 2 sd 3 0 a lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0sl gt devices pcit89 4000 scsi 2 sd 3 0 b lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s2 gt devices pcit89 4000 scsi 2 sd 3 0 c lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s3 gt devices pcit89 4000 scsi 2 sd 3 0 d lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s4 gt devices pcit89 4000 scsi 2 sd 3 0 e lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s5 gt devices pcit89 4000 scsi 2 sd 3 0 f lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s6 gt devices pcil89 4000 scsi 2 sd 3 0 g lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t3d0s7 gt devices pcit89 4000 scsi 2 sd 3 0 h 3 Stop applications Stop applications with the following operations a Stop vold sh etc init d volmgt stop lt Return gt b Stop the all I O devices connected to the target PCI card determined in procedure 2 If the device is a disk unit file system operation i Determine the mountpoint of the disk from the logical path name determined in procedure 2 b c2t3d0 mou
24. defined in both files and the procedures must be performed again from procedure 1 d The followings are examples of execution of the FJSVhddv package installation to opt The path of the hddvchk_conf command depends on the installed directory Check OK to procedure 1 g opt FISVhddv bin hddvchk_conf lt Return gt kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 1 d opt FISVhddv bin hddvchk_conf lt Return gt NG ID 3 g Reconfigure the kernel restart the system and then start system operation touch reconfigure lt Return gt usr sbin shutdown y i6 g0 lt Return gt 2 Stop the machine administration hardware monitoring daemon Execute the following commands to stop the machine administration hardware monitoring daemon usr sbin FJSVmadm prephp lt Return gt 3 Add PCI cards Add PCI cards according to the following procedure To add PCI cards with the following configurations the WWPN World Wide Port Name 16 digit number of the card to add is needed If zone configuration by WWPN is done with Fibre Channel switch SN200 series If the Host Affinity function of ETERNUS3000 GR700 800 series is used When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility go to procedure 3 a because these products get WWPNs from PCI cards automatically The WWP
25. example the Ap Id is pcipsy21 R0B01 PCI slot03 usr sbin FJSVmadm inst2comp fjpfca3 lt Return gt pcipsy21 ROBO1 PCI slot03 b Specify the Ap_Id from 4 a as a parameter and confirm that the slot status of the PCI card to disconnect is connected configured cfgadm pcipsy21 ROBO1 PCI slot03 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy21 ROBO1 PCI slot03 fibre hp connected configured ok c After executing the command to disconnect the PCI card specifying the Ap_Id from 4 b confirm that the slot status has changed to disconnected unconfigured cfgadm c disconnect pcipsy21 ROB01 PCI slot03 lt Return gt cfgadm pcipsy21 ROBO1 PCI slot03 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy21 ROBO1 PCI slot03 unknown disconnected unconfigured unknown d To confirm the slot position at replacement operation blink the ALARM LED of Ap_Id displayed in procedure 4 a ik cfgadm x led fault mode blink pcipsy21 ROB01 PCI slot03 lt Return gt 5 Replace the PCI card Replace the PCI card disconnected in 4 with a replacement card This operation is performed by our customer support When exchanging Fibre Channel cards the following operations are also required When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility Go to procedure 6 When not using SA
26. extension operation blink the ALARM LED of Ap_Id displayed in procedure 3 b cfgadm x led fault mode blink pcipsy15 COM01 PCI slot02 lt Return gt d After checking that the READY LED on the PCI slot to add the PCI card is off add PCI cards This operation is performed by our customer support e Use the cfgadm 1M command to check that the slot status of the added PCI card changed to disconnected unconfigured cfgadm pcipsy15 COM01 PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy15 COM0O1 PCI slot02 unknown disconnected unconfigured unknown 4 Connect PCI cards Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the extension slot position Note that the push button is only effective in 19 20 multiuser mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy15 COM01 PCI slot02 lt Return gt cfgadm pcipsy15 COM01 PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy15 COM01 PCI slot02 mult hp connected configured ok Update hardware configuration information of machine administration Start the hard
27. following example shows the definition of a fjpfeal PCI card instance number 1 and disk array equipment target 3 being added vi kernel drv fjpfca conf lt Return gt port fjpfca0 nport fjpfcal nport spl lt added fcp bind target fjpfca0t0 0x50000e1000002501 fjpfcalt3 0x50000e1000002ab0 lt added max throttle fj pfca0t0 128 fjpfcalt3 240 lt added iii Execute the following command to make the fjpfca driver take in the added part usr sbin FJSVpfca fc_config v lt Return gt b Add device driver configuration Add device driver configuration with the following procedures according to the I O device type Disk Tape devices Solaris OE sd st This example is a case where a disk device of target 2 lun 0 is pre defined in procedure 1 and added in procedure 5 If the device is a tape device re read sd as st in this procedure and perform the same operation i See kernel drv sd conf and check that target 2 and lun 0 are defined When this definition does not exist add the definition If it is commented out remove comments Disk equipment vi kernel drv sd conf lt Return gt name sd class scsi target 0 lun 0 name sd class scsi In case the definition does not exist 43 44 ii iii target 2 lun 0 add or remove comments Tape equipment vi kernel drv st conf lt Return gt name st class s
28. inst2comp hmel lt Return gt pcipsy18 ROBO1 PCI slot02 c The configuration of each driver is added Please refer to each driver manual for details 7 Connect to redundant system Please execute following commands in order to connect the extended PCI card to the existing redundant configuration system Notes On the NIC switching mode in Redundant Line Control Function it is impossible to extend a new interface to the existing redundant configuration system Moreover when extending an interface on the GS SURE linkage mode it is necessary to extend an interface after deactivating the virtual interface e g shaO used by the GS SURE linkage mode Fast switching mode in Redundant Line Control Function 75 76 usr sbin ifconfig hme2 plumb lt Return gt usr sbin ifconfig hme2 192 168 20 10 netmask broadcast trailers up lt Return gt GS SURE linkage mode in Redundant Line Control Function opt FJSVhanet usr sbin stphanet n sha0O lt Return gt opt FJSVhanet usr sbin hanetconfig create n sha2 m n i 192 168 20 10 t hme2 lt Return gt opt FJSVhanet usr sbin hanetnic modify n sha0 t sha2 sha3 sha4 lt Return gt opt FJSVhanet usr sbin strhanet n sha0 lt Return gt Multipath Function usr opt FJSViomp bin iompadm c mpnet add mpnet0 hme2 lt Return gt usr opt FJSViomp bin iompadm c mpnet start mpnet0 hme2 lt Return gt Switch the redundant path
29. interfaces of Netcompo FNA LAN and all high layer applications that are using Netcompo FNA LAN For details please refer to the manual of Netcompo FNA LAN and high layer applications 4 2 2 Extension of PCI cards with new redundant system This subsection describes extension procedure of a new redundant system with the following redundant configuration software SynfinityLink 2 2 PRIMECLUSTER GLS 4 0 or later The replacement and extension of PCI cards are supported only on Redundant Line Control Function of SynfinityLink When using other redundant software please refer to manuals of each product 1 Stop the hardware monitoring of the Machine Administration The hardware monitoring of the Machine Administration is stopped by using the following command usr sbin FJSVmadm prephp lt Return gt 2 Add PCI cards a Before extending the PCI card please save the result current configuration information of the prtpicl 1M command prtpicl v gt tmp prtpicl pre lt Return gt b Check the status of the slot Subsequent examples describe extension of PCI cards to the slot ROBO1 PCI slot02 The ROBO1 PCI slot02 indicates the I O cabinet number 0 the PCI Disk Box number 1 physical slot number 02 In the extension of a new redundant system although two or more PCI cards are extended whenever you add a PCI card please perform from procedure 2 b to 3 Connect PCI cards The relationsh
30. mplbs8 new c3t3d0s2 c4t3d0s2 0 GR8404550 usr sbin mplbconfig av lt Return gt IOMP dev FJISVmplb fiomp adm8 Node dev FJSVmplb rdsk mp1b8s0 dev FJSVmplb rdsk mplb8s1 dev FJSVmplb rdsk mplb8s2 dev FJISVmplb rdsk mp1b8s3 dev FJSVmplb rdsk mpl1b8s4 dev FJSVmplb rdsk mp1b8s5 dev FJSVmplb rdsk mpl1b8s6 dev FJISVmplb rdsk mpl1b8s7 Zone mp1lb0 exist clt4d0s2 c2t3d0s2 0 GR720464f 12340001 000213 9 Setup applications Add configuration for the application using the logical path name determined in 8 filesystem operation Create file system To mount at system start add an entry in the etc vfstab file newfs dev FJSVmphd rdsk mphd2s6 lt Return gt vi etc vfstab lt Return gt dev FISVmphd dsk mphd2s6 dev FJSVmphd rdsk mphd2s6 mnt ufs 1 yes raw access operation Configure applications For details see the manual of each application swap device Add swap device To make valid at system start add an entry in the etc vfstab file swap a dev FJSVmphd dsk mphd2s4 lt Return gt vi etc vfstab lt Return gt dev FISVmphd dsk mphd2s4 swap no swap l lt Return gt swapfile dev swaplo blocks free dev FJISVmphd dsk mphd2s4 32 164 16 788384 683680 10 Start applications Use the logical path name determined in 8 to start applications filesystem operation Mount filesystem mount F
31. order to update configuration information and restart the hardware monitoring of the Machine Administration usr sbin FJSVmadm postphp lt Return gt 5 Connect equipment to PCI cards The extended PCI card is connected with network equipment by the cable 6 Setup driver The driver configuration is added by following operations a The interface name is investigated in order to configure drivers and high layer products Please save the result of the prtcicl 1M command and obtain the difference information between current result and the result taken at 2 a Then the driver instance number for the extended PCI card is obtained In the following example since the instance number is 1 it can be determined that the interface name of the extended PCI card is hmel1 prtpicl v gt tmp prtpic post lt Return gt diff tmp prtpicl pre tmp prtpicl post more lt Return gt gt status okay gt devfs path pci 8d 2000 network l gt driver name hme gt binding name SUNW hme gt bus addr 1 gt instance il gt _class obp device gt name network b It confirms that the interface name that obtained above mentioned procedure is an interface name that corresponds to the PCI bus slot of the added PCI card Please carry out this procedure when it cannot identify relationship between an interface name and PCI bus slot by expanding two or more PCI cards especially usr sbin FJSVmadm
32. procedures to procedure 1 b ls kernel drv hddv conf lt Return gt kernel drv hddv conf b When operating with non active extension procedures execute the following commands and confirm that kernel drv hddv conf is deleted hddvadm i lt Return gt ls kernel drv hddv conf lt Return gt kernel drv hddv conf No such file or directory c Use the following command to create hddv conf file for the pre defined extension function The 25 26 message examples for operation with non active extension procedures means that disk array device GR720 currently used with taget 1 lun 0 is registered twice on the sd hddv driver New installation to procedure 1 e hddvadm M lt Return gt Operation with non active extension procedures to procedure 1 d hddvadm M lt Return gt already ID 1 exists devices pci ld 2000 fibre channel 1 sd 1 0 a raw GR720 When performing pre defined extension the same target number cannot be registered in sd conf and hddv conf so delete or comment out the conflicting definition parts in sd conf file when definition in the two files conflict with each other The following example shows the case that target 1 is repeated vi kernel drv sd conf lt Return gt name sd class scsi target 0 lun 0 name sd class scsi target 1 lun 0 name sd class scsi Delete or comment out target 1 lun 1 conflicting definit
33. prtpicl 1M command get the physical path name of added LUN by taking the difference between the result and the information obtained in 2 a and calculate the corresponding logical path name under dev dsk In the following example c3t3d0 is logical path name of added lun 0 The logical path name of lun 1 2 is set to c8t8d1 c3t3d2 respectively because cXtYdZmeans X PCI card number Y target number Z lun number LYwXrwxrwx LYwXrwxrwXx gt lrwxrwxrwx gt lrwxrwxrwx gt lrwxrwxrwx gt lrwxrwxrwx gt lrwxrwxrwx gt lrwxrwxrwx gt lt Return gt gt devfs path gt devfs path gt devfs path prtpicl v gt tmp prtpicl post lt Return gt diff tmp prtpicl pre tmp prtpicl post grep devfs path grep hddv pci 89 2000 fibre channel 1 hddv 3 0 pci 89 2000 fibre channel 1 hddv 3 1 pci 89 2000 fibre channel 1 hddv 3 2 root devices pci root devices pci root devices pci root devices pci root devices pci root devices pci root devices pci root devices pci root 89 2000 root 89 2000 root 89 2000 root 89 2000 root 89 2000 root 89 2000 root 89 2000 root 89 2000 41 fibre c fibre c fibre c fibre c fibre c fibre c fibre c 41 fibre c Sep 20 22 53 Sep Sep Sep Sep Sep Sep Sep nannetl nannel nan
34. slotZZ Physical slot No ZZ Other Physical slot No ZZ pcipsyN PCI slotZZ Below an example of adding a PCI card to chassis number 0 PCI Disk BOX number 1 physical slot number 02 slot in PCI Disk BOX ROBO01 PCI slot02 is explained Check the status of the target slot is empty unconfigured cfgadm grep ROBO1 PCI slot02 lt Return gt pcipsy18 ROBO1 PCI slot02 unknown empty unconfigured unknown c To confirm the slot position at extension operation blink the ALARM LED of Ap_Id displayed in procedure 3 b ik cfgadm x led fault mode blink pcipsy18 ROB01 PCI slot02 lt Return gt d After checking that the READY LED on the PCI slot to add the PCI card is off add PCI cards This operation is performed by our customer support e Use the cfgadm 1M command to check that the slot status of the added PCI card changed to disconnected unconfigured cfgadm pcipsy18 ROBO1 PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCI slot02 unknown disconnected unconfigured unknown 4 Connect PCI cards Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the extension slot position Note that the push button is only effective in multi user mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a larg
35. status is connected configured by using the cfgadm 1M command after the above mentioned procedure ik cfgadm c configure pcipsy15 COM01 PCI slot02 lt Return gt cfgadm pcipsy15 COM01 PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy15 COM01 PCI slot02 pci pci hp connected configured ok Start the hardware monitoring of the Machine Administration Please execute the following commands in order to update configuration information and restart the hardware monitoring of the Machine Administration usr sbin FJSVmadm postphp lt Return gt Connect equipment to PCI cards The extended PCI card is connected with network equipment by the cable Setup driver The driver configuration is added by following operations a The interface name is investigated in order to configure drivers and high layer products Please save the result of the prtcicl 1M command and obtain the difference information between current result and the result taken at 2 a Then the driver instance number for the extended PCI card is obtained In the following example since the instance number is 1 it can be determined that the interface name of the extended PCI card is hmel1 prtpicl v gt tmp prtpic post lt Return gt diff tmp prtpicl pre tmp prtpicl post more lt Return gt gt status okay gt devfs path pci 8d 2000 network l gt driver name hme gt binding name SUNW
36. ufs dev FJSVmphd dsk mphd2s6 mnt lt Return gt raw access operation Start applications For details see the manual of each application swap device This procedure is not required 3 2 1 3 Extension of PCI cards to redundant system The procedure to add PCI cards to a redundant system in order to extend redundancy on a Solaris 8 OE system is explained with an example with GR multipath driver GRMPD There are no additional patterns for Multipath Disk Control MPHD If multipath software products other than GRMPD are being used see the manual of each product No configuration is needed for applications when adding PCI cards to an existing redundant system 1 Setup devices which will be added to system This is an example where a disk array controller of target 3 lun 0 is added with pre definition to an existing disk array device Extension using PCI Hot Plug cannot be done without definition so devices must be defined before the system goes into operation If this is the first time to pre define a disk array device begin with procedure 1 a If this is not the first time to pre define a disk array device begin with procedure 1 e a Check if kernel drv hddv conf exists Check if kernel drv hddv conf exists If it does not exist the hard disk driver is being installed so start from procedure 1 c If it does exist the system is being operated with non active extension procedures so start from procedure 1 b
37. 0 fibre channel 1 hddv 3 0 h ls 1 dev dsk grep pci 89 2000 fibre channel 1 hddv 3 0 lt Return gt lrwxrwxrwx 1 root root 41 Sep 20 22 53 c4t3d0s0 gt devices pci 89 2000 fibre channel 1 hddv 3 0 a Start formats 1M command and configure labeling and partition slice information calculated at 6 b iii to the logical path name of each extended LUN This operation is only needed for each LUN on the representative additional path and is not needed to be performed on each additional PCI card 8 Setup redundant system Execute the following commands to create a new redundant system In case of MPHD the commands need to be executed for each LUN specifying the path name determined in 7 b iii as a parameter The italic letters below are the new multipath names and the underlined letters are the path names that compose the multipath MPHD lt Return gt usr opt FJSViomp bin iompadm c mphd new dev rdsk c3t3d0s2 dev rdsk c4t3d0s2 31 32 IOMP dev FJSVmphd fiomp adm2 Node dev FJSVmphd rdsk mphd2s0 dev FJSVmphd rdsk mphd2s1 dev FJSVmphd rdsk mphd2s2 dev FJSVmphd rdsk mphd2s3 dev FJSVmphd rdsk mphd2s4 dev FJSVmphd rdsk mphd2s5 dev FJSVmphd rdsk mphd2s6 dev FJSVmphd rdsk mphd2s7 GRMPD usr sbin mplbconfig nv lt Return gt x Phase 1 checking dev rdsk xxx Phase 2 checking multi path Phase 3 analyzing path Path Action Element path LUN Storage
38. 000 fibre channel 1 hddv 3 0 b lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s2 O Bias gt devices pci 89 2000 fibre channel hddv 3 0 c lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s3 gt devices pci 89 2000 fibre channel 1 hddv 3 0 d lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s4 gt devices pci 89 2000 fibre channel 1 hddv 3 0 e lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s5 gt devices pci 89 2000 fibre channel 1 hddv 3 0 f lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s6 gt devices pci 89 2000 fibre channel 1 hddv 3 0 g lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s7 O A gt devices pci 89 2000 fibre channel hddv 3 0 h ix Start format command and configure labeling and partition slice information calculated at 6 b viii to the logical path name of each extended LUN 7 Connect to redundant system To connect the added PCI card to the redundant system execute the following command and confirm that the path name determined in procedure 6 b viii in underlined letters is added usr sbin mplbconfig nv lt Return gt kkk Phase 1 checking dev rdsk xxx xxx Phase 2 checking multi path Phase 3 analyzing path Path Action Element path LUN Storage Zone mp1b0 exist clt4d0s2 c2t3d0s2 0 GR720464f 12340001 mplb8 add c3t3d0s2 0 GR8404550 000213 usr sb
39. 0s6 mnt lt Return gt Disk device raw access operation Tape device Start applications For details see the manual of each application Disk device swap device This procedure is not required 3 2 1 2 Extension of PCI cards on new redundant system The procedure to add PCI cards to a new redundant system on a Solaris 8 OE system is explained with examples for the multipath software below Multipath Disk Control MPHD GR Multipath Driver GRMPD If other multipath software is being used see the manual of each product 1 Setup devices which will be added to system This is an example where an disk array device at target 3 lun 0 2 is pre defined Extension using PCI Hot Plug is impossible without definition so definition must be done before the system goes into operation If this is the first time to pre define a disk array device begin with procedure 1 a If this is not the first time to pre define a disk array device begin with procedure 1 e a Check if kernel drv hddv conf exists Check if kernel drv hddv conf exists If it exists the hard disk driver is being installed so start from procedure 1 c If it does not exist the system is being operated with none active extension procedures so start from procedure 1 b Not exist new install to procedure 1 c ls kernel drv hddv conf lt Return gt kernel drv hddv conf No such file or directory Exists operation with non active extension
40. 1 root root 41 Sep 20 22 53 c3t3d0s7 gt devices pci 8d 2000 fibre channel 1 hddv 3 0 h 8 Connect to redundant system To connect the added PCI card to the redundant system execute the following command and confirm that the path name determined in procedure 7 b iii in underlined letters is added usr sbin mplbconfig nv lt Return gt Phase 1 checking dev rdsk xxx x Phase 2 checking multi path Phase 3 analyzing path 39 Path Action Element path LUN Storage Zone mp1b0 exist clt4d0s2 c2t3d0s2 0 GR720464f 12340001 mplb8 add gt c3t3d0s2 0 GR8404550 000213 usr sbin mplbconfig av lt Return gt IOMP dev FJISVmplb fiomp adms Node dev FISVmplb rdsk mp1b8s0 dev FISVmplb rdsk mplb8s1 dev FISVmplb rdsk mp1b8s2 dev FISVmplb rdsk mp1b8s3 dev FISVmplb rdsk mp1b8s4 dev FISVmplb rdsk mplb8s5 dev FISVmplb rdsk mplb8s6 dev EISVmplb rdsk mplb8s7 3 2 2 Solaris 9 OK This section describes the procedures to extend PCI cards on Solaris 9 OE systems 3 2 2 1 Extension of PCI cards on non redundant system Procedures to add PCI cards without making them redundant on a Solaris 8 OE system are explained here 1 Stop the machine administration hardware monitoring daemon Execute the following commands to stop the machine administration hardware error monitoring daemon usr sb
41. 3 240 lt add Execute the following command to make the fjpfca driver set up the additional part usr sbin FJSVpfca fc_config v lt Return gt b Add configuration of device driver An example where a path with target 3 lun 0 2 is added to a disk array device with procedure 6 is shown ii iii 30 The status may have changed after the pre definition in 1 so check again the LUN definition overlap between sd conf and hddv conf by executing hddvchk conf command Check NG means that definitions are repeated in the two files so go back to 1 d Check OK to procedure 7 b ii opt FJSVhddv bin hddvchk conf lt Return gt kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 1 d opt FISVhddv bin hddvchk_conf lt Return gt NG ID 3 Execute the following command in order to indicate connecting extension LUN to hard disk driver usr sbin hddvadmdr lt Return gt After saving executing result of prtpicl command get the physical path name of added LUN by taking difference between that result and extracted information of 3 a and calculate the corresponding logical path name under dev dsk iv The following examples shows that c3t3d0 is logical path name of extended lun 0 The logical path name of lun 1 2 is set to c3t3dl c3t3d2 respectively because cXtYdZshow X PCI card number Y target number Z lun number
42. B8FC1U Channel 2 2 or later 2 2 1 or later 5 GP7B8FC1X enclosed with a Pci 22MHz OK on GR Multipath Driver Fibre card 7 8 1 0 1 or later 6 Channel FUJITSU PCI Fibre Multipath Disk Control Channel 2 2 1 or 33MHz 2 2 1 or later 5 EWO0eRC CH later enclosed with a 66MHz Ok ok GR Multipath Driver PCI card 1 0 1 or later 76 April 2003 1 Please refer to 2 1 1 2 PCI Hot Plug configure operation of the PCI Hot Plug User s Guide about the operating clock of PCI cards and PCI slots 2 In case of using Solaris 8 OE patch 109885 09 or later is needed 3 Replacement and expansion are not supported due to hardware specifications 4 Replacement is not supported due to hardware specifications 5 Multipath Disk Control load balance option is not supporting PCI Hot Plug 6 Please contact our customer service about support to Solaris 9 OE of GR Multipath Driver 7 If using Solaris 9 OE it is necessary to apply the FUJITSU PCI Fibre Channel 2 2 patch level 08 or later 78 If using FUJITSU PCI Fibre Channel 2 2 patch 910936 08 or later is needed to hot expansion 1 2 PCI cards for network devices The following list shows the PCI cards that support PCI Hot Plug for network devices Please ask the sales staff about the latest information of PCI Hot Plug supporting cards Supporting Functions 2 Software of driver Operating Fu
43. Disk BOX ROB01 PCI slot02 is explained Check the status of the target slot is empty unconfigured cfgadm grep R0B01 PCI slot02 lt Return gt pcipsy18 R0B01 PCI slot02 unknown empty unconfigured unknown c To confirm the slot position at extension operation blink the ALARM LED of Ap_Id displayed in procedure 3 b cfgadm x led fault mode blink pcipsy18 ROBO1 PCI slot02 lt Return gt d After checking that the READY LED on the PCI slot to add the PCI card is off add PCI cards This operation is performed by our customer support e Use the cfgadm 1M command to check that the slot status of the added PCI card changed to disconnected unconfigured cfgadm pcipsy18 ROBO1 PCI slot02 lt Return gt Ap Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCI slot02 unknown disconnected unconfigured unknown 4 Connect the PCI card Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the extension slot position Note that the push button is only effective in multiuser mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy18 ROBO1 PCI slot02 lt Re
44. I card Specify the devices and interfaces that are influenced by replacing PCI cards 3 Disconnect from redundant system Disconnect the devices path and interfaces of PCI cards from the redundant system 4 Disconnect the PCI card It changes into the state where the PCI card is exchangeable 5 Replace the PCI card Replace the PCI card 6 Connect the PCI card It changes into the state where the PCI card is usable 7 Connect to redundant system Connect device paths and interfaces of the replaced PCI card to the redundant system 8 Switch back the redundant path Switch back the online path if needed 9 Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration 2 2 Extension of PCI cards This section describes outline to extend PCI cards using PCI Hot Plug feature About software which makes PCI card redundantly please refer to each manual of the software 2 2 1 Extension of PCI cards on non redundant system When not using the redundant configuration software the extension procedure of PCI cards is as follows Please refer to Chapter 3 File devices or Chapter 4 Network devices about detailed extending procedure of PCI cards 1 Setup I O devices which will be added to system Setup devices which will be added to system in the case of file devices on Solaris 8 OE 2 Stop the hardware monitoring of the Machin
45. Inc Copyright 1995 Software Research Associates Inc All Rights Reserved All Rights Reserved Copyright C FUJITSU LIMITED 2003 ii Contents Chapter 1 PCI cards with support for PCI Hot Plug cccccccceeeeeeeeeeeeeeeeeeeeeeees 1 Tal PCl cards for Tile devices 3 32 12 22 iss Sothink ees ek aed eee ea ee 1 1 2 PCI cards for network GeVICES cccccccce cece eeeeeeeeece cnet ee seaaeaececeeeeeeaeaeeeceeeseseeaaeaeeeeeeeseeeenensaees 2 Chapter 2 Overview of replacement and extension of PCI Cards cccceeeeeees 3 2 1 Replacementof PCS da En a a aaa ies 3 2 1 1 Replacement of PCI cards on non redundant system u u sssssseeererererereren erne n erne nerne ener nnnee 3 2 1 2 Replacement of PCI cards on redundant system ccccccscccceessecesesseeecsseeeeseeeeesssaeeecseseesessseseesaaes 3 22 Extensionof PCI cards escce iin esa ett aan oe ee eae tee aoe acta Me ke ee a held 4 2 2 1 Extension of PCI cards on non redundant System cccccccccsssscccceceeessssssceeeccecessssseeeececeeeeessssseeeeeeeses 4 2 2 2 Extension of PCI cards with new redundant System ccccccccccccssssssssceecceeesenssseeeceeeesesssseeececeeeeenaes 4 2 2 3 Extension of PCI cards to redundant System cccccccsscccsesseceessseeeessseeeecseeeeeseeesessseesecsuseeeeseseesenees 5 Chapters File devies cs rcs an tc tos tetas Has ue tats ce tas Wee facet teeta face 3 ls 7 3 1 Replacement of PO pearde e a e
46. It changes into the state where the PCI card is usable 5 Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration 6 Connect equipment to PCI cards Connect equipment to PCI cards 7 Setup driver a Add the driver configuration of PCI cards Add the driver configuration of PCI cards b Add the configuration of device driver Add the configuration of device driver in the case of file devices 8 Setup redundant system Configure the environmental configuration of a new redundant system and activate 9 Setup applications Configure the high layer products and services about a new redundant system 10 Start applications Start high layer products and services used by the new redundant system 2 2 3 Extension of PCI cards to redundant system When extending PCI cards to the existing redundant system it is possible to extend PCI cards without stopping services and applications The extension procedure of PCI card to existing redundant system is as follows 1 2 Setup I O devices which will be added to system Setup devices which will be added to system in the case of file devices on Solaris 8 OE Stop the hardware monitoring of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration Add PCI cards Extend PCI cards Connect PCI cards It changes into the state where the PCI c
47. LUN definition overlap between sd conf and hddv conf by executing hddvchk conf command The message example for Check NG means that target 3 is defined in both files and the procedures must be performed again from procedure 6 b iv The followings are examples of execution of the FJSVhddv package installation to opt The path of the hddvchk_conf command depends on the installed directory Check OK to procedure 6 b vii opt FISVhddv bin hddvchk_conf lt Return gt kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 6 b iv opt FISVhddv bin hddvchk_conf lt Return gt NG ID 3 Execute the following command in order to make the hard disk driver set up the LUN added update drv f hddv lt Return gt In case disk array equipment currently used exists the following messages are displayed but do not effect performance update drv f hddv lt Return gt 47 48 viii Cannot unload module hddv Will be unloaded upon reboot Forcing update of hddv conf After saving execution result of the prtpicl 1M command get the physical path name of added LUN by taking the difference between the result and the information obtained in 3 a and calculate the corresponding logical path name under dev dsk In the following example c3t3d0 is logical path name of added lun 0 The logical path name of lun 1 2 is set to c8t3d1 c8t8d2
48. N management function of the above products To replace PCI cards with the following configurations Fibre Channel switch SN200 series and disk array device ETERNUS3000 GR700 800 series need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on SN200 series If the Host Affinity function of ETERNUS3000 GR700 800 series is used For details see SN200 Series Affinity User s Guide or ETERNUS3000 GR700 800 series GRmer User s Guide To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digits of the WWPN in hexadecimal form The top eight digits are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the replacement card is 100000000e44061 0e24 4061 Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration 6 Connect the PCI card Connect the new PCI card with the cfgadm command with the configure option or by pushing the button c
49. N of the PCI card can be known from the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digits of the WWPN in hexadecimal form The top eight digits are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the adding card is 100000000e44061 0e24 4061 a Before extension of PCI cards save the current execution result of the prtpicl 1M command current configuration at present 57 prtpicl v gt tmp prtpicl pre lt Return gt b Check the status of the slot When adding PCI cards to a redundant configuration several PCI cards may be added to the system In this case perform procedures to 3 on each card The relationship of the position of a PCI card and format of Ap Id is shown below Location of PCI slot Definition of slot Format of Ap Id Chassis No X System board No Y pceipsyN CXMOY PCI slotZZ Physical slot No ZZ I O Chassis No X Inside chassis of PRIMEPOWER 900 1500 2500 PCI Disk BOX PCI Disk BOX No Y pcipsyN RXBOY PCT slotZZ Physical slot No ZZ Other Physical slot No ZZ pcipsyN PCI slotZZ Below an example of adding a PCI card to chassis number 0 PCI Disk BOX number 1 physical slot number 02 slot in PCI Disk BOX ROB01 PCI slot02 is explained Check the status of the target slot is empty unconfigure
50. N of the PCI card can be known from the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digits of the WWPN in hexadecimal form The top eight digits are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the adding card is 100000000e44061 0e24 4061 a Before extension of PCI cards save the current execution result of the prtpicl 1M command current configuration at present prtpicl v gt tmp prtpicl pre lt Return gt b Check the status of the slot The relationship of the position of a PCI card and format of Ap_Id is shown below Location of PCI slot Definition of slot Format of Ap_Id Chassis No X System board No Y pcipsyN CXMOY PCI slotZZ Physical slot No ZZ I O Chassis No X Inside chassis of PRIMEPOWER 900 1500 2500 PCI Disk BOX PCI Disk BOX No Y pcipsyN RXBOY PCI slotZZ Physical slot No ZZ Other Physical slot No ZZ pcipsyN PCI slotZZ Below an example of adding a PCI card to chassis number 0 system board number 1 physical slot number 02 slot in chassis COM01 PCI slot02 is explained Check the status of the target slot is empty unconfigured cfgadm grep COM01 PCI slot02 lt Return gt pcipsy15 COM01 PCI slot02 unknown empty unconfigured unknown c To confirm the slot position at
51. OBO1 PCI slot02 unknown empty unconfigured unknown c To confirm the slot position at extension operation blink the ALARM LED of Ap_Id displayed in procedure 2 b ik cfgadm x led fault mode blink pcipsy18 ROB01 PCI slot02 lt Return gt d Please confirm that READY LED of the PCI bus slot that extends a PCI card has gone out and extend a PCI card This operation is performed by our customer support e Please confirm that the PCI slot status which extended PCI cards is disconnected unconfigured by using cfgadm 1M command cfgadm pcipsy18 ROBO1 PCI slot02 lt Return gt Ap_Id Typ Receptacl Occupant Condition pcicsy18 ROBO1 PCI slot02 unknown disconnected unconfigured unknown 3 Connect PCI cards The cfgadm command is executed with the configure option to Ap_Id or the push button of the PCI bus slot is pushed Then a extended PCI card is connected In addition a push button is valid only in multiple user mode Please confirm that the slot status is connected configured by using the cfgadm 1M command after the above mentioned procedure cfgadm c configure pcipsy18 ROBO1 PCI slot02 lt Return gt cfgadm pcipsy18 ROBO1 PCI slot02 lt Return gt 74 Ap Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCI slot02 pci pci hp connected configured ok 4 Start the hardware monitoring of the Machine Administration Please execute the following command in
52. O_BOX 1 gt component name ROBO1 PCI 02 gt devfs path pci 8d 2000 fibre channel l gt driver name fjpfca gt binding name pcil0cf 105c gt bus addr 1 gt instance 1 gt class obp device gt inam fibre channel ii Edit the kernel drv fjpfca conf file and add the following property definitions to the 59 60 instance number determined in 6 a i port type of topology link speed Defined when adding PCI cards with Fabric connection This definition is not needed with FC AL connection fcp bind target connection between target ID and WWN Defined when adding disk array with Fabric connection This definition is not needed with FC AL connection max throttle maximum issued SCSI commands Defined to limit the number of SCSI commands issued at the same time The need and value of this definition depends on the disk array device so refer to the manuals of each disk array device Below is an example where definitions for a PCI card at fjpfcal instance number 1 and a disk array device at target 3 is added vi kernel drv fjpfca conf lt Return gt port fjpfca0 nport fjpfcal nport spl lt add fcp bind target fjpfca0t0 0x50000e1000002501 fjpfcalt3 0x50000e1000002ab0 lt add max throttle fjpfca0t0 128 fjpfcalt3 240 lt add iii Execute the following command to make the fjpfca driver set up the additional part usr sbin FJSVpfca fc_config v
53. PCI Hot Plug User s Guide I O device edition Preface Purpose This manual describes the procedure of exchanging or extending file PCI cards and network PCI cards using the PCI Hot Plug feature Intended Readers This manual is intended for the following readers System software developers and test engineers System support engineers System administrators who uses this function Hardware engineers who uses this function Organization This manual is organized as follows Chapter 1 PCI cards with support for PCI Hot Plug describes the lists of PCI cards supporting PCI Hot Plug Chapter 2 Overview of replacement and extension of PCI cards describes the outline of a procedure of replacement and extension of PCI cards Chapter 3 File devices describes the detailed procedure of PCI Hot Plug about file devices Chapter 4 Network devices describes the detailed procedure of PCI Hot Plug about network devices Chapter 5 Notes for PCI Hot Plug describes the notes when using PCI Hot Plug Notation Used The following shows the notation conventions used in this manual In this manual the Solaris TM Operating Environment is abbreviated to Solaris OE the Solaris TM 8 Operating Environment is abbreviated to Solaris 8 OE the Solaris TM 9 Operating Environment is abbreviated to Solaris 9 OE The titles of chapters are enclosed in parentheses Example See Chapter1 PCI cards supporting PCI Hot Plug
54. Please make a setting by the ndd command 1000BASE SX Ethernet 2 0 or later In case of reflecting the setting of PW008GE5 enclosed with a PCI card platform sun4us kernel drv fjgi conf file using name parent and unit address please reboot a system 77 5 1 2 Using tape device while PCI Hot Plug operation By using the PCI Hot Plug feature a job which uses tape devices may be terminated abnormally While using the PCI Hot Plug feature avoid using tape devices 5 1 3 Others When expanding PCI cards by using PCI Hot Plug feature it is necessary to install the device driver case of file devices and the driver for using the added PCI card When extending PCI cards each driver consumes memory capacity to control PCI cards Please refer to the installation guide attached to each PCI card about memory capacity required in order to extend PCI cards and installation guide of the redundant configuration software about memory capacity required 5 2 Restrictions This section describes restrictions when exchanging and extending PCI cards by using PCI Hot Plug feature Please do not exchange or expand PCI cards by using PCI Hot Plug feature during executing Dynamic Reconfiguration DR 78
55. R700 800 series is used When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility go to procedure 3 a because these products get WWPNs from PCI cards automatically The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digits of the WWPN in hexadecimal form The top eight digits are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the adding card is 100000000e44061 0e24 4061 a Before extension of PCI cards save the current execution result of the prtpicl 1M command current configuration at present prtpicl v gt tmp prtpicl pre lt Return gt 27 28 b Check the status of the slot The relationship of the position of a PCI card and format of Ap Id is shown below Location of PCI slot Definition of slot Format of Ap Id Inside chassis of Chassis No X System board No Y pcipsyN CXM0Y PCI slotZZ PRIMEPOWER 900 1500 2500 Physical slot No ZZ I O Chassis No X PCI Disk BOX PCI Disk BOX No Y pcipsyN RXBOY PCI slotZZ Physical slot No ZZ Other Physical slot No ZZ pcipsyN PCI slotZZ Below an example of adding a PCI card to chassis number 0 PCI Disk BOX number 1 physical slot number 02 slot in PCI
56. This definition is not needed with FC AL connection fcp bind target connection between target ID and WWN Defined when adding disk array with Fabric connection This definition is not needed with FC AL connection max throttle maximum issued SCSI commands Defined to limit the number of SCSI commands issued at the same time The need and value of this definition depends on the disk array device so refer to the manuals of each disk array device Below is an example where definitions for a PCI card at fjpfcal instance number 1 and a disk array device at target 3 is added vi kernel drv fjpfca conf lt Return gt port fjpfca0 nport fjpfcal nport sp1 lt add fcp bind target fjpfca0t0 0x50000e1000002501 fjpfcalt3 0x50000e1000002ab0 lt add max throttle fjpfca0t0 128 fjpfcalt3 240 lt add iii Execute the following command to make the fjpfca driver set up the additional part usr sbin FJSVpfca fc_config v lt Return gt b Add configuration of device driver This is an example where an disk array device in redundant configuration at target 3 lun 0 2 is added in procedure 5 If this is the first time to pre define a disk array device begin with procedure 6 b i If this is not the first time to pre define a disk array device begin with procedure 6 b v i Check if kernel drv hddv conf exists Check if kernel drv hddv conf exists If it exists the hard disk driver is being ins
57. a lt Return gt diff tmp prtpicl pre tmp prtpicl pfca more lt Return gt gt status okay gt fru PCI slot PCI 02 at Rack 0 IO _BOX 1 gt component name ROBO1 PCI 02 gt devfs path pci 8d 2000 fibre channel l gt driver name fjpfca gt binding name pcilOcf 105c gt bus addr 1 gt instance 1 gt _class obp device gt inam fibre channel ii Edit the kernel drv fjpfca conf file and add the following property definitions to the instance number determined in 7 a i port type of topology link speed Defined when adding PCI cards with Fabric connection This definition is not needed with FC AL connection 29 iii fcp bind target connection between target ID and WWN Defined when adding disk array with Fabric connection This definition is not needed with FC AL connection max throttle maximum issued SCSI commands Defined to limit the number of SCSI commands issued at the same time The need and value of this definition depends on the disk array device so refer to the manuals of each disk array device Below is an example where definitions for a PCI card at fjpfca1 instance number 1 and a disk array device at target 3 is added vi kernel drv fjpfca conf lt Return gt Port fjpfca0 nport fjpfcal nport spl lt add fcp bind target fjpfca0t0 0x50000e1000002501 fjpfcalt3 0x50000e1000002ab0 lt add max throttle fjpfca0t0 128 fj pfcalt
58. aa e a a e a e Sai ra a ph a eE 7 3 1 1 Replacement of PCI cards on non redundant system ssssssessseersstssressetsstssetsstesteestesssesstesseessesees 7 3 1 2 Replacement of PCI cards on redundant system W u u u sssseseesererererererenerren erne renere reen erne rr renee 12 3 2 Extension oT PEC ads FE E E og cas ead th E Rn eN erne ke 15 3 2 1 Solaris SOE 553 ske let a Eae ee ace nel aad hk eae E E ean es eee ee eee 16 3 2 1 1 Extension of PCI cards on non redundant SyStem cccccccccccccsssssseceeceesessssnsceeeceeeeessssseeeeeeeeeens 16 3 2 1 2 Extension of PCI cards on new redundant system cccccccccssssssseececceeceesssseeeeceeeecessssseeeeeeesens 25 3 2 1 3 Extension of PCI cards to redundant system cccccccessscccssseceesesseeeeessseeeesseeeeeeeeeessneeeesesees 33 3 22 Solaris 9 OI alton ES RESET EL add elancs lt cae dastatg ote DEERE BES PELSE REE EET Mahesh A IEEE TEE ET 40 3 2 2 1 Extension of PCI cards on non redundant System ccccccccccccsessssseceeccecsssssnsceeceeeeeessssseeeeeeeeeees 40 3 2 2 2 Extension of PCI cards with new redundant system cccccessssccccceeessssssseeececeeessssseeeeeceeeess 49 3 2 2 3 Extension of PCI cards to redundant system cccccessssscccceessessseeeecceecessssseeecceeeeestssseeeeeeeeens 57 Chapter 4 Network devies sranna a e a Sa 65 4 1 Replacement of PCI card ssssss seerne eee erne 65 4 1 1 Replacement of PCI cards on non redundant system
59. ally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration Set up driver Add the configuration of driver according to the following operation As for file related drivers additional configuration of the PCI card driver and the device driver is needed a Add configurations of PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver Configuration is not required Fibre Channel driver FUJITSU PCI Fibre Channel i After saving execution result of prtpicl 1M command take the difference between the result and the information obtained in procedure 3 a to calculate the instance number of the fjpfca driver of the PCI card added In the following example instance number is 1 indicating that the interface name of the PCI card added is fjpfcal Making the LED of the PCI card blink using the fc adm command can specify the relation between the instance number and the slot position The relation between the WWPN and instance number of the PCI card added can be known from the fc_info a command prtpicl v gt tmp prtpicl pfca lt Return gt diff tmp prtpicl pre tmp prtpicl pfca more lt Return gt ii gt status okay gt devfs path pci 8d 2000 fibre channel l gt driver name fjpfca gt binding name pcilOcf 105c gt bus addr 1 gt instance 1 gt _class
60. ard On the redundant system of standby mode when an online communication path is disconnected a standby communication path will be online communication path automatically When using the GS SURE linkage mode in Redundant Line Control Function of SynfinityLink or PRIMECLUSTER GLS it is necessary to deactivate virtual interfaces e g sha0 used by GS SURE linkage mode Fast switching mode in Redundant Line Control Function opt FJSVhanet usr sbin hanetnic delete n sha0 i hmel lt Return gt usr sbin ifconfig hmel unplumb lt Return gt NIC switching mode in Redundant Line Control Function opt FJSVhanet usr sbin stpptl n shal lt Return gt opt FJSVhanet usr sbin hanetnic change n sha0 lt Return gt GS SURE linkage mode in Redundant Line Control Function opt FJSVhanet usr sbin stphanet n sha0 lt Return gt Multipath Function usr opt FISViomp bin iompadm c mpnet stop mpnetO hmel lt Return gt usr opt FISViomp bin iompadm c mpnet disconnect mpnetO hmel lt Return gt 4 Disconnect the PCI card a Please specify the PCI bus slot position by the interface e g hmel in procedure 2 Specify the replacing PCI card usr sbin FJSVmadm inst2comp hmel lt Return gt pcipsy21 ROBO0O1 PCI slot03 67 68 b Obtained Ap Id is specified as an argument of the cfgadm 1M command and it confirms that the slot status of the PCI card that disconnects is connected configured
61. ard is usable Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration Connect equipment to PCI cards Connect equipment to PCI cards Set up driver a Add the driver configuration of PCI cards Add the driver configuration of PCI cards b Add the configuration of device driver Add the configuration of device driver in the case of file devices Connect to redundant system Connect device paths and interfaces of the replaced PCI card to the redundant system Switch the redundant path Switch the online path for extended path if needed Chapter 3 File devices This chapter describes in detail the procedures of replacement and extension of file PCI cards by using PCI Hot Plug For details about software to use PCI cards in redundant configuration see the manuals of each software 3 1 Replacement of PCI card This section describes the procedures to exchange PCI cards using the PCI Hot Plug feature on non redundant and redundant systems 3 1 1 Replacement of PCI cards on non redundant system Before exchanging PCI cards without using redundancy software such as multipath control applications using the PCI cards must be stopped Follow the procedures below to exchange PCI cards on a non redundant system 1 Stop the machine administration hardware monitoring daemon Execute the following command to stop
62. ards to a new redundant system on a Solaris 9 OE system is explained with examples for the multipath software below Multipath Disk Control MPHD GR Multipath Driver GRMPD an example with GR multipath driver GRMPD If multipath other multipath software are being used see the manual of each product 1 Stop the machine administration hardware monitoring daemon Use the following command to stop the hardware monitoring daemon of machine administration usr sbin FJSVmadm prephp lt Return gt 2 Add PCI cards Add PCI cards according to the following procedure To add PCI cards with the following configurations the WWPN World Wide Port Name 16 digit number of the card to add is needed If zone configuration by WWPN is done with Fibre Channel switch SN200 series If the Host Affinity function of ETERNUS3000 GR700 800 series is used When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility go to procedure 2 a because these products get WWPNs from PCI cards automatically The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of 49 50 the card These characters represent the bottom eight digits of the WWPN in hexadecimal form The top eight digits are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the adding ca
63. by using PCI Hot Plug feature please refer to the solution of each PCI card show below 10 100BASE T Product ID Software of driver Resolution Functions X1033A U Solaris OE hme Please make a setting by the ndd command X1033A X In case of reflecting the setting of kernel drv hme conf file using name parent and unit address parameters please reboot a system PWOO8FE1 10 100BASE T FUJITSU PCI 10 100 Ethernet 1 0 or later enclosed with a PCI card Please make a setting by the ndd command In case of reflecting the setting of kernel drv fjfe conf file using name parent and unit address parameters please reboot a system PWO08QE1 10 100BASE T x 4 FUJITSU PCI Quad 10 100 Ethernet 1 0 or later enclosed with a PCI card PWO08QE2 10 100BASE T x 4 FUJITSU PCI Quad 10 100 Ethernet 1 1 or later enclosed with a PCI card Please make a setting by the ndd command In case of reflecting the setting of kernel drv fjqe conf file using name parent and unit address parameters please reboot a system 10 100 1000BASE T PP028GE1U FUJITSU PCI Gigabit ndd command is not supported PP028GE1X Ethernet 1 4 1 or later In case of reflecting the setting of 1000BASE SX enclosed with a PCI card kernel drv fjge conf file using name parent and it add t l boot PWO08GE1U unit address parameters please reboot a system PWO08GE1X 1000BASE T PWO008GE4 FUJITSU PCI Gigabit
64. check that the slot status of the added PCI card changed to disconnected unconfigured cfgadm pcipsy18 ROBO1 PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCI slot02 unknown disconnected unconfigured unknown 3 Connect the PCI card Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the extension slot position Note that the push button is only effective in multiuser mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy18 ROBO1 PCI slot02 lt Return gt cfgadm pcipsy18 ROBO1 PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCI slot02 mult hp connected configured ok 4 Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJSVmadm postphp lt Return gt 5 Connect equipment to PCI cards Connect new disk array device in redundant configuration to the extended PCI card When the
65. comments of extended LUN name hddv class scsi target 3 lun 2 name hddv class scsi target 3 lun 3 f Check for LUN definition conflict between sd conf and hddv conf by executing the hddvchk_conf command The message example for Check NG means that target 3 is defined in both files and the procedures must be performed again from procedure 1 d The followings are examples of execution of the FJSVhddv package installation to opt The path of the hddvchk_conf command depends on the installed directory Check OK to procedure 1 g opt FISVhddv bin hddvchk_conf lt Return gt kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 1 d opt FISVhddv bin hddvchk_conf lt Return gt NG ID 3 g Reconfigure the kernel restart the system and then start system operation touch reconfigure lt Return gt usr sbin shutdown y i6 g0 lt Return gt 2 Stop the machine administration hardware monitoring daemon Execute the following commands to stop the machine administration hardware monitoring daemon usr sbin FJSVmadm prephp lt Return gt 3 Add PCI cards Add PCI cards according to the following procedure To add PCI cards with the following configurations the WWPN World Wide Port Name 16 digit number of the card to add is needed If zone configuration by WWPN is done with Fibre Channel switch SN200 series
66. configure labeling and partition slice information calculated in 6 b viii to the logical path name of each extended LUN 7 Setup applications Add configuration for the application using the logical path name determined in 6 b Disk device filesystem operation Create file system To mount at system start add an entry in the etc vfstab file newfs dev rdsk c2t2d0s6 lt Return gt vi etc vfstab lt Return gt dev dsk c2t2d0s6 dev rdsk c2t2d0s6 mnt ufs 1 yes Disk device raw access operation Tape device Configure applications For details see the manual of each application Disk device swap device Add swap device To make valid at system start add an entry in the etc vfstab file swap a dev dsk c2t2d0s4 lt Return gt vi etc vfstab lt Return gt dev dsk c2t2d0s4 swap no swap l lt Return gt swapfile dev swaplo blocks free dev dsk c2t2d0s4 32 164 16 788384 683680 8 Start applications Use the logical path name determined in 6 b to start applications Disk device filesystem operation Mount filesystem mount F ufs dev dsk c2t2d0s6 mnt lt Return gt Disk device raw access operation Tape device Start applications For details see the manual of each application Disk device swap device This procedure is not required 3 2 2 2 Extension of PCI cards with new redundant system The procedure to add PCI c
67. csi target 0 lun 0 name st class scsi In case the definition does not exist target 5 lun 0 add or remove comments To make the sd driver set up the disk device added execute the following commands When the commands are executed the following messages are displayed but do not effect performance Similar messages are displayed when a tape device already exists when a tape device is added but does not effect performance Disk device update_drv f sd lt Return gt Cannot unload module sd Will be unloaded upon reboot Forcing update of sd conf Tape device update_drv f st lt Return gt After saving the execution result of the prtpicl 1M command take the difference between the result and the information obtained in 2 a and get the physical path name of additional disk device then get the logical path name under dev dsk In the following example c2t2d0 is the logical path name of the disk device added prtpicl v gt tmp prtpicl post lt Return gt Disk equipment diff tmp prtpicl pre tmp prtpicl post grep devfs path grep sd lt Return gt gt devfs path pci 89 4000 scsi 2 sd 2 0 ls 1 dev dsk grep pci 89 4000 scsi 2 sd 2 0 lt Return gt lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s0 gt devices pcit89 4000 scsi 2 sd 2 0 a lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0sl gt devices pcit89 4000 scsi 2 sd 2 0 b lrwxrwxrwx 1 root root
68. csi target 1 lun 0 name hddv class scsi target 1 lun 1 remove comments of the LUN in use name hddv class scsi target 1 lun 2 name hddv class scsi target 3 lun 0 name hddv class scsi target 3 lun 1 remove comments of extended LUN name hddv class scsi target 3 lun 2 name hddv class scsi target 3 lun 3 Check the LUN definition overlap between sd conf and hddv conf by executing hddvchk conf command The message example for Check NG means that target 3 is defined in both files and the procedures must be performed again from procedure 6 b iv The followings are examples of execution of the FJSVhddv package installation to opt The path of the hddvchk_conf command depends on the installed directory Check OK to procedure 6 b vii opt FISVhddv bin hddvchk_conf lt Return gt kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 6 b iv opt FISVhddv bin hddvchk_conf lt Return gt NG ID 3 Execute the following command in order to make the hard disk driver set up the LUN added update_drv f hddv lt Return gt In case disk array equipment currently used exists the following messages are displayed but do not effect performance update drv f hddv lt Return gt Cannot unload module hddv Will be unloaded upon reboot Forcing update of hddv conf After saving execution result of the
69. d cfgadm grep ROBO1 PCI slot02 lt Return gt pcipsy18 ROBO1 PCI slot02 unknown empty unconfigured unknown c To confirm the slot position at extension operation blink the ALARM LED of Ap_Id displayed in procedure 2 b cfgadm x led fault mode blink pcipsy18 ROBO1 PCI slot02 lt Return gt d After checking that the READY LED on the PCI slot to add the PCI card is off add PCI cards This operation is performed by our customer support e Use the cfgadm 1M command to check that the slot status of the added PCI card changed to disconnected unconfigured cfgadm pcipsy18 ROBO1 PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCI slot02 unknown disconnected unconfigured unknown 3 Connect the PCI card Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the extension slot position Note that the push button is only effective in multiuser mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy18 ROBO1 PCI slot02 lt Return gt cfgadm pcipsy18 ROBO1 PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition
70. dded vi kernel drv fjpfca conf lt Return gt port fjpfca0 nport fjpfcal nport spl lt added fcp bind target fjpfca0t0 0x50000e1000002501 fjpfcalt3 0x50000e1000002ab0 lt added max throttle fjpfca0t0 128 fj pfcalt3 240 lt added iii Execute the following command to make the fjpfca driver take in the added part usr sbin FJSVpfca fc config v lt Return gt b Add device driver configuration An example where a path with target 3 lun 0 2 is added to a disk array device with procedure 6 is shown i The status may have changed after the pre definition in 1 so check again the LUN definition overlap between sd conf and hddv conf by executing hddvchk_conf command Check NG means that definitions are repeated in the two files so go back to 1 d Check OK to procedure 7 b ii opt FJSVhddv bin hddvchk conf lt Return gt 38 kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 1 d opt FISVhddv bin hddvchk_conf lt Return gt NG ID 3 ii Execute the following command in order to indicate connecting extension LUN to hard disk driver usr sbin hddvadmdr lt Return gt iii After saving executing result of prtpicl command get the physical path name of added LUN by taking difference between that result and extracted information of 3 a and calculate the correspondi
71. e In the example below fjpfea3 interface name of the path connecting the target PCI card and I O devices and hddvlis the LUN Logical Unit Number of the disk array device connected to fjpfca3 WARNING pci 8d 2000 fibre channel 1l fjpfca3 Hard Error PCI DMA error WARNING pci 8d 2000 fibre channel l1 hddv l1 0 hddv1 SCSI transport failed reason reset giving up NOTICE mphd0 I O path switchover succeeded pci 8d 2000 fibre channel l1 hddv l1 0 gt pci 89 2000 fibre channel l1 hddv 2 0 The following procedures are also explained assuming the console messages above Find the hddvl with the status offline fail from the results of the iompadm command and determine the logical path name of the LUN connected to the target PCI card The following is an example of MPHD and dev rdsk c3t1d0s2 is the logical path name corresponding to hddv1 In case of GRMPD specify mplb in the c option parameter of the iompadm command usr opt FJISViomp bin iompadm c mphd p info lt Return gt IOMP dev FJISVmphd fiomp adm2 gt devices pseudo mphd 2 adm Element dev rdsk c3t1d0s2 offline fail block target completed hard reset sequence GR7104546 010000 00 00 30 hddv1l gt devices pci 8d 2000 fibre channel 1 hddv 1 0 c raw dev rdsk c2t2d0s2 online active block good status with active GR7104546 010000 01 01 32 hddv15 gt devices pci 89 2000 fibre chann
72. e Administration Stop the hardware abnormal monitoring of the Machine Administration 3 Add PCI cards Extend PCI cards 4 Connect PCI cards It changes into the state where the PCI card is usable 5 Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration 6 Connect equipment to PCI cards Connect equipment to PCI cards 7 Set up driver a Add the driver configuration of PCI cards Add the driver configuration of PCI cards b Add the configuration of device driver Add the configuration of device driver in the case of file devices 8 Setup applications Configure the high layer products and services about extended PCI cards 9 Start applications Start high layer products and services used by the replaced PCI card 2 2 2 Extension of PCI cards with new redundant system The procedure of using the extended PCI card and establishing a new redundant configuration is as follows Please refer to Chapter 3 File devices or Chapter 4 Network devices about detailed extending procedure of PCI cards 1 Setup I O devices which will be added to system Setup devices which will be added to system in the case of file devices on Solaris 8 OE 2 Stop the hardware monitoring of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration 3 Add PCI cards Extend PCI cards 4 Connect PCI cards
73. e Machine Administration Please execute the following commands in order to update configuration information and restart the hardware monitoring of the Machine Administration Please refer to PCI Hot Plug User s Guide about details of postphp 1M command usr sbin FJSVmadm postphp lt Return gt 4 1 2 Replacement of PCI cards on redundant system When exchanging PCI cards with redundant configuration software it can exchange PCI cards without stopping applications This subsection describes replacement procedure of PCI cards with the following redundant configuration software SynfinityLink 2 2 PRIMECLUSTER GLS 4 0 or later The replacement and extension of PCI cards are supported only on Redundant Line Control Function of SynfinityLink When using other redundant software please refer to manuals of each product 1 Stop the hardware monitoring of the Machine Administration The hardware monitoring of the Machine Administration is stopped by using the following command usr sbin FJSVmadm prephp lt Return gt 2 Specify the replacing PCI card The interfaces e g hme1 which stop applications for replacement of PCI cards are specified from warning messages outputted the console 3 Disconnect from redundant system In order to disconnect the PCI card for exchange from a redundant system please execute the following command with the interface name specified by procedure 2 Specify the replacing PCI c
74. e duplication definition parts in sd hddv name sd class scsi target 1 lun 2 A target number that is already used by the sd driver of the system disk or CD ROM cannot be deleted In this case change the target number in hddv conf and of the disk array device vi vii so that it does not conflict with sd conf Edit the kernel drv hddv conf file created automatically by the hddvadm M command and remove comments of the LUN definition to pre define and the LUN definition currently being used with non active extension procedures If case active extension has been performed before and the LUN definition to extend the next procedures 1 f 1 g are not needed The following example shows target 3 lun 0 2 for pre defined extension and disk array device GR720 target 1 in use with non active extension procedures being registered ls kernel drv hddv conf lt Return gt kernel drv hddv conf vi kernel drv hddv conf lt Return gt name hddv class scsi target 0 lun 0 name hddv class scsi target 0 lun 7 name hddv class scsi target 1 lun 0 name hddv class scsi target 1 lun 1 Remove comments of the LUN in use name hddv class scsi target 1 lun 2 name hddv class scsi target 3 lun 0 name hddv class scsi target 3 lun 1 Remove comments of extended LUN name hddv class scsi target 3 lun 2 name hddv class scsi target 3 lun 3 Check the
75. e scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy18 ROBO1 PCI slot02 lt Return gt cfgadm pcipsy18 ROBO1 PCI slot02 lt Return gt Ap Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCIl slot02 mult hp connected configured ok 5 Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJSVmadm postphp lt Return gt 6 Connect equipment to PCI cards Add controllers to an existing disk array device and connect to the PCI card added When the Host Affinity function is being used with Fibre Channel SN200 series Host Affinity must be configured for the device added When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility WWPN zoning of SN200 series Host affinity of Array disk device ETERNUS3000 GR700 800 series and binding setting for fjpfca conf sd conf of host can be added from SAN management window automatically See Operation guide or User s Manual of each product for details Before using this function the following procedure 7 a i and 7 a ii except fep bind target settin
76. el 1 hddv 2 0 c raw Node dev FISVmphd rdsk mphd2s0 dev FISVmphd rdsk mphd2s1 dev FISVmphd rdsk mphd2s2 dev FISVmphd rdsk mphd2s3 dev FISVmphd rdsk mphd2s4 dev FISVmphd rdsk mphd2s5 dev FISVmphd rdsk mphd2s6 dev FISVmphd rdsk mphd2s7 Function MPmode false AutoPath true Block true NeedSync false 3 Disconnect from redundant system To disconnect the path between the target PCI card and I O devices execute the following command specifying the logical path name of the connected LUN determined in procedure 2 b This command only needs to be executed on the representation LUN and does not need to be executed on each LUN under the same PCI card If the operation path is disconnected on a redundant system the standby path will automatically be switched to operation If both MPHD GRMPD controlled devices are connected to the target PCI card the commands for both products need to be executed MPHD usr opt FJSViomp bin iompadm c mphd change adapter disconnect dev rdsk c3t1d0s2 lt Return gt RMPD usr opt FJISViomp bin iompadm c mplb change adapter disconnect dev rdsk c3t1d0s2 Return gt A Q 4 Disconnect the PCI card Disconnect the defected PCI card with the following procedures 13 a Determine the slot position of the PCI card from the interface name of the path connecting the target PCI card and I O devices determined in procedure 2 a fjpfca3 In this
77. enclosed with a PCI card in this chapter can be downloaded from the following URLs The driver patches http software fujitsu com en security index html The option driver packages enclosed with a PCI card http primepower fujitsu com en driver e_index html Please contact our customer service about the latest information of the driver patches and the option driver packages 1 1 PCI cards for file devices The following list describes the PCI cards that support PCI Hot Plug for file devices Please ask the sales staff about the latest information of PCI Hot Plug supporting cards es FE o ti Supporting Functions oftware of driver perating Functions Product ID driver name clock 1 Hot Hot Redundant Software replacement expansion X6540A U Solaris OE glm 2 33MHz NG 73 NG 8 none Multipath Disk Control X6541A U i A 2 2 1 or later 5 X6541A X Solaris OE glm 2 33MHz NG 4 OK GR Multipath Driver 1 0 1 or later 6 Multipath Disk Control GP7B8SC1U i PA 2 2 1 or later 75 GP7B8SC1x Solaris OE elm 2 33MHz OK on GR Multipath Driver 1 0 1 or later 6 eel FUJITSU PCI SCSI PP028SC1U Adapter Driver 1 0 or PP028SC1X later enclosed with a Hone Ok ee a PCI card FUJITSU ULTRA LVD SCSI Host Bus 33MHz PW008SC3 Adapter Driver 1 0 or NG 8 NG 8 none 66MHz later enclosed with a PCI card FUJITSU PCI Fibre Multipath Disk Control GP7
78. ermined that the interface name of the extended PCI card is hmel prtpicl v gt tmp prtpic post lt Return gt diff tmp prtpicl pre tmp prtpicl post more lt Return gt gt status okay gt devfs path pci 8d 2000 network l gt driver name hme gt binding name SUNW hme gt bus addr 1 gt instance i gt class obp device gt name network b It confirms that the interface name that obtained above mentioned procedure is an interface name that corresponds to the PCI bus slot of the added PCI card Please carry out this procedure when it cannot identify relationship between an interface name and PCI bus slot by expanding two or more PCI cards especially usr sbin FJSVmadm inst2comp hmel lt Return gt pcicsy18 ROBO1 PCIl slot02 c The configuration of each driver is added Please refer to each driver manual for details 7 Setup redundant system When establishing the redundant configuration system it is necessary to configure the environmental configuration of the Redundant Line Control Function or the Multipath Function and to activate the redundant configuration system For details of environmental configuration and activation please refer to each function manual of SynfinityLink or PRIMECLUSTER GLS 8 Setup applications The configuration of high layer product services applications is added Please refer to the manual of each high layer product for details 9 S
79. following operations the high layer product e g service application which uses PCI cards for exchange is stopped a When using TCP IP interface The example in the case of using TCP IP interface is shown below Please refer to the online manual etc about the details of the ifconfig 1M command Moreover when high layer programs which using the TCP IP interface needs to be stopped please stop each high layer program Please refer to each manual about the stop procedure of high layer programs i When using Ethernet devices usr sbin ifconfig hmel down lt Return gt usr sbin ifconfig hmel unplumb lt Return gt ii When using ATM devices opt FJSVatm bin atmunlink adapter number lt Return gt When using FNA SNA communication protocol over Ethernet When not using the redundant configuration software it is necessary to stop all high layer applications that are using Netcompo FNA LAN and all interfaces of Netcompo FNA LAN For details please refer to the manual of Netcompo FNA LAN and high layer applications 4 Disconnect the PCI card a The Ap_Id is displayed using the following command based on the interface e g hme1 specified in Procedure 2 Specify the replacing PCI card The attachment pint identifier Ap_Id is a value for the cfgadm 1M command specifying the slot position of the PCI card Please refer to PCI Hot Plug user s guide about the details of cfgadm 1M a
80. g are necessary Moreover begin from procedure 7 a iii after executing this function When not using SAN management function of the above products SN200 series need to be reconfigured individually For details see SN200 Series Affinity User s Guide or ETERNUS3000 GR700 800 series GRmgr User s Guide Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration 7 Set up driver Add the configuration of driver according to the following operation As to file related driver additional configuration of PCI cards driver and device driver is needed a Add the configuration of PCI cards driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver Configuration is not required Fibre Channel driver FUJITSU PCI Fibre Channel i After saving execution result of prtpicl 1M command take the difference between the result and the information obtained in procedure 3 a to calculate the instance number of the fjpfca driver of the PCI card added In the following example instance number is 1 indicating that the interface name of the PCI card added is fjpfcal Making the LED of the PCI card blink using the fe_adm command can specify the relatio
81. g that the interface name of the PCI card added is fjpfcal Making the LED of the PCI card blink using the fc adm command can specify the relation between the instance number and the slot position The relation between the WWPN and instance number of the PCI card added can be known from the fc info a command prtpicl v gt tmp prtpicl pfca lt Return gt diff tmp prtpicl pre tmp prtpicl pfca more lt Return gt gt status okay gt devfs path pci 8d 2000 fibre channel l gt driver name fjpfca gt binding name pcilOcf 105c gt bus addr 1 gt instance 1 gt class obp device gt inam fibre channel usr sbin FJSVpfca fc_adm 1 fjpfcal lt Return gt ii Edit kernel drv fjpfca conf and add the following property definitions to the instance number from 6 a i port topology classification link speed Define this property when PCI cards using Fabric connection are added This definition is not needed with FC AL connection fep bind target connection of target ID and WWN Define this property when disk array equipment using Fabric connection are added This definition is not needed with FC AL connection max throttlemumber of maximum SCSI command issue This property is defined for restriction on simultaneous issue of SCSI commands The necessity and value of this definition depends on the specification of the disk array device so refer to the user s manual of each equipment The
82. hddv class scsi target 3 lun 2 name hddv class scsi target 3 lun 3 f Check the LUN definition overlap between sd conf and hddv conf by executing hddvchk_conf command The message example for Check NG means that target 3 is defined in both files and the procedures must be performed again from procedure 1 d The followings are examples of execution of the FJSVhddv package installation to opt The path of the hddvchk_conf command depends on the installed directory Check OK to procedure 1 g opt FISVhddv bin hddvchk_conf lt Return gt kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 1 d opt FISVhddv bin hddvchk_conf lt Return gt NG ID 3 g Reconfigure the kernel restart the system and then start system operation touch reconfigure lt Return gt usr sbin shutdown y i6 g0 lt Return gt 2 Stop the machine administration monitoring daemon Use the following command to stop the hardware disorder monitoring daemon of machine administration usr sbin FJSVmadm prephp lt Return gt 3 Add PCI cards Add PCI cards according to the following procedure To add PCI cards with the following configurations the WWPN World Wide Port Name 16 digit number of the card to add is needed If zone configuration by WWPN is done with Fibre Channel switch SN200 series If the Host Affinity function of ETERNUS3000 G
83. he interface name of the path connecting the target PCI card and I O devices determined in procedure 2 a glm2 Please refer to PCI Hot Plug User s Guide about the detail of inst2comp 1M command In this example the Ap Id is peipsy3 COM00 PCI slot02 usr sbin FJSVmadm inst2comp glm2 lt Return gt pcipsy3 COM00 PCI slot02 b Specify Ap_Id determined from 4 a as a parameter and confirm that the slot status of the PCI card to disconnect is connected configured cfgadm pcipsy3 COMOO PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy3 COMO00 PCI slot02 mult hp connected configured ok c After executing the command to disconnect the PCI card specifying the Ap_Id from 4 a confirm that the slot status has changed to disconnected unconfigured cfgadm c disconnect pcipsy3 COMO0 PCI slot02 lt Return gt cfgadm pcipsy3 COMOO PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy3 COMO00 PCI slot02 unknown disconnected unconfigured unknown d To confirm the slot position at replacement operation blink the ALARM LED of Ap_Id displayed in procedure 4 a cfgadm x led fault mode blink pcipsy3 COMOO PCI slot02 lt Return gt 5 Replace the PCI card Replace the PCI card disconnected in 4 with a replacement card This operation is performed by our customer support When exchanging Fibre Channel cards the following opera
84. hme gt bus addr 1 gt tinstance 1 gt class obp device gt name network b It confirms that the interface name that obtained above mentioned procedure is an interface name that corresponds to the PCI bus slot of the added PCI card Please carry out this procedure when it cannot identify relationship between an interface name and PCI bus slot by expanding two or more PCI cards especially usr sbin FJSVmadm inst2comp hmel lt Return gt pcipsy15 C0M01 PCI slot02 c The configuration of each driver is added Please refer to each driver manual for details 7 Setup applications The configuration of extended PCI card used by high layer products services applications is added Please refer to each high layer product manual for details 8 Start applications By the following operations please start the high layer product service application used by the extended PCI card a When using TCP IP interface The example in the case of using TCP IP interface is shown below Furthermore when the high layer program which is using the PCI card needs to restart please refer to the manual of each high layer program usr sbin ifconfig hmel plumb lt Return gt usr sbin ifconfig hmel inet 192 168 10 10 netmask broadcast trailers up lt Return gt b When using FNA SNA communication protocol over Ethernet When not using the redundant configuration software it is necessary to start all
85. in FJSVmadm prephp lt Return gt 2 Add PCI cards Add PCI cards according to the following procedure To add PCI cards with the following configurations the WWPN World Wide Port Name 16 digit number of the card to add is needed If zone configuration by WWPN is done with Fibre Channel switch SN200 series If the Host Affinity function of ETERNUS3000 GR700 800 series is used When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility go to procedure 2 a because these products get WWPNs from PCI cards automatically The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digits of the WWPN in hexadecimal form The top eight digits are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the adding card is 100000000e44061 0e24 4061 40 a Before extension of PCI cards save the current execution result of prtpicl 1M command composition information at present prtpicl v gt tmp prtpicl pre lt Return gt b Check the status of the slot The relationship of the position of a PCI card and format of Ap_Id is shown below Location of PCI slot Definition of slot Format of Ap_Id Chassis No X System board No Y pcipsyN CXMOY PCI slo
86. in mplbconfig av lt Return gt IOMP dev FJISVmplb fiomp adm8s Node dev FISVmplb rdsk mp1b8s0 dev FISVmplb rdsk mp1lb8s1 dev FISVmplb rdsk mp1b8s2 dev FISVmplb rdsk mp1b8s3 dev FISVmplb rdsk mp1b8s4 63 64 dev FJSVmplb rdsk mplb8s5 dev FJSVmplb rdsk mplb8s6 dev FJSVmplb rdsk mplb8s7 Chapter 4 Network devices This chapter describes the detailed procedure about replacement or extension of network related PCI cards by PCI Hot Plug Please refer to manuals of each software about the details of the redundant software 4 1 Replacement of PCI card This section describes replacement procedure of the PCI card by using PCI Hot Plug on non redundant system and redundant system 4 1 1 Replacement of PCI cards on non redundant system When exchanging PCI cards without redundant configuration software you must stop applications which using PCI cards The replacement procedure of PCI cards is as follows 1 Stop the hardware monitoring of the Machine Administration The hardware monitoring of the Machine Administration is stopped by using the following command Please refer to PCI Hot Plug User s Guide about the details of prephp 1M command usr sbin FJSVmadm prephp lt Return gt 2 Specify the replacing PCI card The interfaces e g hme1 which stop applications for replacement of PCI cards are specified from warning messages outputted the console 3 Stop applications By
87. ing definition parts in sd conf file when definition in the two files conflict with each other The following example shows the case that target 1 is repeated vi kernel drv sd conf lt Return gt name sd class scsi target 0 lun 0 name sd class scsi target 1 lun 0 name sd class scsi delete or comment out target 1 lun 1 the duplication definition parts in sd hddv name sd class scsi target 1 lun 2 A target number that is already used by the sd driver of the system disk or CD ROM cannot be deleted In this case change the target number in hddv conf and of the disk array device so that it does not conflict with sd conf Edit the kernel drv hddv conf file created automatically by the hddvadm M command and remove comments of the LUN definition to pre define and the LUN definition currently being used with non active extension procedures If case active extension has been performed before and the LUN definition to extend the 53 54 vi vii viii next procedures 1 f 1 g are not needed The following example shows target 3 lun 0 2 for pre defined extension and disk array device GR720 target 1 in use with non active extension procedures being registered ls kernel drv hddv conf lt Return gt kernel drv hddv conf vi kernel drv hddv conf lt Return gt name hddv class scsi target 0 lun 0 name hddv class scsi target 0 lun 7 name hddv class s
88. ion Element path LUN Storage O mplbo exist clt4d0s2 c2t3d0s2 mplb8 new c3t3d0s2 c4t3d0s2 0 GR8404550 usr sbin mplbconfig av lt Return gt IOMP dev FJISVmplb fiomp adm8s Node dev FJSVmplb rdsk mp1b8s0 dev FJSVmplb rdsk mplb8s1 dev FJISVmplb rdsk mplb8s2 dev FJSVmplb rdsk mp1b8s3 dev FJSVmplb rdsk mpl1b8s4 dev FJSVmplb rdsk mplb8s5 dev FJSVmplb rdsk mp1b8s6 dev FJSVmplb rdsk mp1b8s7 Zone GR720464f 12340001 000213 8 Setup redundant system Add configuration for the application using the logical path name determined in 7 filesystem operation Create file system To mount at system start add an entry in the etc vfstab file newfs dev FJSVmphd rdsk mphd2s6 lt Return gt vi etc vfstab lt Return gt dev FISVmphd dsk mphd2s6 dev FUISVmphd rdsk mphd2s6 mnt ufs MK yes raw access operation Configure applications For details see the manual of each application swap device Add swap device To make valid at system start add an entry in the etc vfstab file 9 swap a dev FJSVmphd dsk mphd2s4 lt Return gt vi etc vfstab lt Return gt dev FISVmphd dsk mphd2s4 swap no swap l lt Return gt swapfile dev swaplo blocks free dev FISVmphd dsk mphd2s4 32 164 16 788384 683680 Start applications Use the logical path name determined in 7 to start applications filesystem operation M
89. ion parts in sd hddv name sd class scsi target 1 lun 2 A target number that is already used by the sd driver of the system disk or CD ROM cannot be deleted In this case change the target number in hddv conf and of the disk array device so that it does not conflict with sd conf Edit the kernel drv hddv conf file created automatically by the hddvadm M command and remove comments of the LUN definition to pre define and the LUN definition currently being used with non active extension procedures If case active extension has been performed before and the LUN definition to extend the next procedures 1 f 1 g are not needed If the target number of each disk array controller of a redundant system differs add definition for each target number The following example shows target 3 lun 0 2 for pre defined extension and disk array device GR720 target 1 in use with non active extension procedures being registered ls kernel drv hddv conf lt Return gt kernel drv hddv conf vi kernel drv hddv conf lt Return gt name hddv class scsi target 0 lun 0 name hddv class scsi target 0 lun 7 name hddv class scsi target 1 lun 0 name hddv class scsi target 1 lun 1 Remove comments of the LUN in use name hddv class scsi target 1 lun 2 name hddv class scsi target 3 lun 0 name hddv class scsi target 3 lun 1 Remove comments of extended LUN name
90. ip of the position of a PCI card and format of Ap_Id is shown below N an integral number Location of PCI slot Element Format of Ap_Id 71 72 Location of PCI slot Element Format of Ap_Id Cabinet Number X System Board Number Y peipsyN CXMOY PCI slotZZ Physical Slot Number ZZ I O Cabinet Number X PCI Disk Box PCI Disk Box Number Y pcipsyN RXBOY PCI slotZZ Physical Slot Number ZZ Cabinet of PRIMEPOWER 900 1500 2500 other than those above Physical Slot Number ZZ pceipsyN PCI slotZZ Please confirm that the PCI slot status which extended PCI cards is empty unconfigured by using cfgadm 1M command cfgadm grep ROBO1 PCI slot02 lt Return gt pcicsy18 ROBO1 PCI slot02 unknown empty unconfigured unknown c To confirm the slot position at extension operation blink the ALARM LED of Ap_Id displayed in procedure 2 b cfgadm x led fault mode blink pcipsy18 ROBO1 PCI slot02 lt Return gt d Please confirm that READY LED of the PCI bus slot that extends a PCI card has gone out and extend a PCI card This operation is performed by our customer support e Please confirm that the PCI slot status which extended PCI cards is disconnected unconfigured by using cfgadm 1M command cfgadm pcipsy15 ROBO1 PCI slot02 lt Return gt Ap Id Typ Receptacl Occupant Condition pcicsy18 R0B01 PCI slot02 unknown disconnected unco
91. irm that the slot status is connected configured by using the cfgadm 1M command after the above mentioned procedure Notes If the setup of driver conf 4 file exist please refer to 5 1 1 Network driver configuration changing cfgadm c configure pcipsy3 COM00 PCI slot02 lt Return gt cfgadm pcipsy3 COMOO PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy3 COMO0 PCI slot02 pci pci hp connected configured ok 7 Start applications Please restart the stopped high layer product service application by the following operations a When using TCP IP interface The example in the case of using TCP IP interface is shown below Furthermore when the high layer program which is using the PCI card needs to restart please refer to the manual of each high layer program i When using Ethernet devices usr sbin ifconfig hmel plumb lt Return gt usr sbin ifconfig hmel inet 192 168 10 10 netmask broadcast trailers up lt Return gt ii When using ATM devices opt FJSVatm bin atmlink adapter number lt Return gt b When using FNA SNA communication protocol over Ethernet When not using the redundant configuration software it is necessary to start all interfaces of Netcompo FNA LAN and all high layer applications that are using Netcompo FNA LAN For details please refer to the manual of Netcompo FNA LAN and high layer applications 8 Start the hardware monitoring of th
92. isk unit swap device Add swap device and resume usage sbin swapadd 2 lt Return gt 8 Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJSVmadm postphp lt Return gt 3 1 2 Replacement of PCI cards on redundant system PCI cards can be replaced without stopping services such as user applications on a redundant system using software such as mutipath control Here the procedure to replace PCI cards on a redundant system using the following redundancy software products is explained Multipath Disk Control MPHD GR Multipath Driver GRMPD If other redundancy software products are used see the manual of each product 12 1 Stop the machine administration hardware monitoring daemon Use the following command to stop the hardware monitoring daemon of machine administration usr sbin FJSVmadm prephp lt Return gt 2 Specify the replacing PCI card Follow the instructions below and determine the interface name of the path connecting the target PCI card and I O devices and the connected I O device a Determine from the WARNING messages output on the console the interface name of the path connecting the target PCI card and I O devices and the connected I O devic
93. ith a PCI card GLS 4 0 or later 73 April 2003 1 Please refer to 2 1 1 2 PCI Hot Plug configure operation of the PCI Hot Plug User s Guide about the operating clock of PCI cards and PCI slots 2 The replacement and expansion are supported only on the Redundant Line Control Function 3 When using the PRIMECLUSTER GLS 4 0 the replacement and expansion are supported only on the Redundant Line Control Function 4 When using the FUJITSU PCI Quad 10 100 Ethernet 1 0 it is necessary to apply the patch 912216 03 or later Please contact our customer service about the latest patch 5 If using the FUJITSU PCI Gigabit Ethernet 1 4 or earlier it is necessary to apply the driver version 1 4 1 or later Please contact our customer service about the latest driver 6 System rebooting is needed for hot extension Chapter 2 Overview of replacement and extensi on of PCI cards This chapter describes the outline of a procedure of replacement and extension of PCI cards by using PCI Hot Plug 2 1 Replacement of PCI card This section describes outline to exchange PCI cards using PCI Hot Plug feature About software which makes PCI card redundantly please refer to each manual of the software 2 1 1 Replacement of PCI cards on non redundant system Before you exchange PCI cards without redundant software you must stop applications which using the PCI cards The replacement procedure of PCI cards is as follows Please refer
94. l of each product for details Before using this function the following procedure 6 a i and 6 a ii except fep bind target setting are necessary Moreover begin from procedure 6 a iii after executing this function When not using SAN management function of the above products SN200 series need to be reconfigured individually For details see SN200 Series Affinity User s Guide or ETERNUS3000 GR700 800 series GRmgr User s Guide Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration Set up driver Add the configuration of driver according to the following operation As for file related drivers additional configuration of the PCI card driver and the device driver is needed a Add the configuration of PCI cards driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver Configuration is not required Fibre Channel driver FUJITSU PCI Fibre Channel i After saving execution result of prtpicl 1M command take the difference between the result and the information obtained in procedure 2 a to calculate the instance number of the fjpfca driver of the PCI card added In the following example instance number is 1 indicatin
95. lrwxrwxrwx 1 root root 41 Sep 20 22 53 1h gt devices pcit89 4000 scsi 2 st 5 0 h lrwxrwxrwx 1 root root 41 Sep 20 22 53 lhb gt devices pcil89 4000 scsi 2 st 5 0 hb lrwxrwxrwx 1 root root 41 Sep 20 22 53 1hbn gt devices pci 89 4000 scsi 2 st 5 0 hbn lrwxrwxrwx 1 root root 41 Sep 20 22 53 ihn gt devices pcit89 4000 scsi 2 st 5 0 hn iv If the device is a disk device start the format 1M command and configure labeling and partition slice information of the logical path name of added disks If the device is a tape device this procedure is not needed Disk array devices hard disk driver hddv This is an example where an disk array device at target 3 lun 0 2 is pre defined in procedure 5 If this is the first time to pre define a disk array device begin with procedure 6 b i If this is not the first time to pre define a disk array device begin with procedure 6 b v i Check if kernel drv hddv conf exists 45 46 ii iii iv Check if kernel drv hddv conf exists If it exists the hard disk driver is being installed so start from procedure 6 b iii If it does not exist the system is being operated with none active extension procedures so start from procedure 6 b ii Not exist new install to procedure 6 b iii ls kernel drv hddv conf lt Return gt kernel drv hddv conf No such file or directory Exists
96. lt Return gt b Add configuration of device driver This is an example where a disk array device in redundant configuration at target 3 lun 0 2 is added in procedure 5 If this is the first time to pre define a disk array device begin with procedure 6 b i If this is not the first time to pre define a disk array device begin with procedure 6 b v i Check if kernel drv hddv conf exists Check if kernel drv hddv conf exists If it exists the hard disk driver is being installed so start from procedure 6 b iii If it does not exist the system is being operated with none active extension procedures so start from procedure 6 b ii Not exist new install to procedure 6 b iii ls kernel drv hddv conf lt Return gt kernel drv hddv conf No such file or directory Exist operation with non active extension procedures to procedure 6 b ii ls kernel drv hddv conf lt Return gt kernel drv hddv conf ii iii iv When operating with non active extension procedures execute the following commands and confirm that kernel drv hddv conf is deleted hddvadm i lt Return gt ls kernel drv hddv conf lt Return gt kernel drv hddv conf No such file or directory Use the following command to create hddv conf file for the pre defined extension function The message examples for operation with non active extension procedures means that disk array device GR720 currently u
97. monitoring of the Machine Administration Please execute the following commands in order to update configuration information and restart the hardware monitoring of the Machine Administration usr sbin FJSVmadm postphp lt Return gt Extension of PCI cards This section describes extension procedure of the PCI card by using PCI Hot Plug on non redundant system and redundant system 4 2 1 Extension of PCI cards on non redundant system This subsection describes extension procedure of PCI cards without the redundant configuration software 1 2 Stop the hardware monitoring of the Machine Administration The hardware monitoring of the Machine Administration is stopped by using the following command usr sbin FJSVmadm prephp lt Return gt Add PCI cards a Before extending the PCI card please save the result current configuration information of the prtpicl 1M command prtpicl v gt tmp prtpicl pre lt Return gt b Check the status of the slot Subsequent examples describe extension of PCI cards to the slot COM01 PCI slot02 The COM01 PCI slot02 indicates the cabinet number 0 the system board number 1 physical slot number 02 The relationship of the position of a PCI card and format of Ap_Id is shown below N an integral number Location of PCI slot Element Format of Ap_Id Cabinet of Cabinet Number X System Board Number Y pceipsyN CXMOY PCI slotZZ Ph
98. n between the instance number and the slot position The relation between the WWPN and instance number of the PCI card added can be known from the fc info a command prtpicl v gt tmp prtpicl pfca lt Return gt diff tmp prtpicl pre tmp prtpicl pfca more lt Return gt gt status okay gt devfs path pci 8d 2000 fibre channel l gt driver name fjpfca gt binding name pcil0cf 105c 37 gt bus addr 1 gt sinstance 1 gt class obp device gt inam fibre channel usr sbin FJSVpfca fc adm 1 fjpfcal lt Return gt ii Edit kernel drv fjpfca conf definitions to the instance number from 7 a i and add the following property port topology classification link speed Define this property when PCI cards using Fabric connection are added This definition is not needed with FC AL connection fcp bind target connection of target ID and WWN Define this property when disk array equipment using Fabric connection is added This definition is not needed with FC AL connection max throttle number of maximum SCSI command issue This property is defined for restriction on simultaneous issue of SCSI commands The necessity and value of this definition depends on the specification of the disk array device so refer to the user s manual of each equipment The following example shows the definition of a fjpfca1 PCI card instance number 1 and disk array equipment target 3 being a
99. nctions Product ID driver name clock 1 Hot Hot Redundant Software replacement extension SynfinityLink 2 2 2 x locate Solaris OE hme 83MHz OK OK PRIMECLUSTER GLS 4 0 or later 73 Te A t10 SynfinityLink 2 2 2 PWO08FE1 emer 33MHz OK OK PRIMECLUSTER or later enclosed GLS 4 0 or later 8 with a PCI card i P ET Ei SynfinityLink 2 2 2 PWO08QE1 33MHz OK OK PRIMECLUSTER or later enclosed GLS 4 0 or later 3 with a PCI card 4 Qor later C3 A E 33MHz SynfinityLink 2 2 2 PW008QE2 ees 2 OK OK PRIMECLUSTER or later enclosed 66MHz GLS 4 0 or later 8 with a PCI card cua FUJITSU PCI Ethernet Gigabit Ethernet 33MHz SynfinityLink 2 2 2 PPO28GE1U 1 4 1 or later 66MHz OK OK PRIMECLUSTER enclosed with a PCI GLS 4 0 or later 73 card 5 FUJITSU PCI Gigabit Ethernet 33MHz SynfinityLink 2 2 2 PWOO08GE1 1 4 1 or later CNR OK OK PRIMECLUSTER enclosed with a PCI GLS 4 0 or later 3 card 5 FUJITSU PCI ids acta SynfinityLink 2 2 2 PW008GE4 Gigabit Ethernet 2 0 38MHz ok OK PRIMECLUSTER or later enclosed 66MHz GLS 4 0 or later 8 with a PCI card i aaa FUJITSU PCI es Fates SynfinityLink 2 2 2 pwogsaks ERNE Eaheret 2 0 oN sor OK PRIMECLUSTER or later enclosed 66MHz GLS 4 0 or later 3 with a PCI card 0 or later 73 GP7B8AT1U FUJITSU PCI ATM SynfinityLink 2 2 2 ATM GP7B8AT1X 1 2 or later enclosed 33MHz OK NG 6 PRIMECLUSTER w
100. nd Ap Id usr sbin FJSVmadm inst2comp hmel lt Return gt pcipsy3 COM00 PCI slot02 b Obtained Ap_Id is specified as an argument of the cfgadm 1M command and it confirms that the slot status of the PCI card that disconnects is connected configured cfgadm pcipsy3 COMO0 PCI slot02 lt Return gt Ap_Id Typ Receptacl Occupant Condition 65 pcipsy3 COM00 PCI slot02 pci pci hp connected configured ok c Please disconnect the PCI card by executing the cfgadn 1M command with Ap_Id and confirm that the slot status is disconnected unconfigured ik cfgadm c disconnect pcipsy3 COMO0 PCI slot02 lt Return gt cfgadm pcipsy3 COMOO PCI slot02 lt Return gt Ap Id Typ Receptacl Occupant Condition pcipsy3 COMO0 PCIT slot02 pci pci hp disconnected unconfigured unknown d To confirm the slot position at replacement operation blink the ALARM LED of Ap_Id displayed in procedure 4 a cfgadm x led fault mode blink pcipsy3 COMO0O PCI slot02 lt Return gt 5 Replace the PCI card The PCI card disconnected in procedure 4 Disconnect the PCI card is exchanged for a new PCI card Our customer service works about this procedure 6 Connect the PCI card The cfgadm command is executed with the configure option to Ap_Id or the push button of the PCI bus slot is pushed Then a new PCI card is connected In addition a push button is valid only in multiple user mode Please conf
101. needed The following example shows target 3 lun 0 2 for pre defined extension and disk array device GR720 target 1 in use with non active extension procedures being registered ls kernel drv hddv conf lt Return gt kernel drv hddv conf vi kernel drv hddv conf lt Return gt 61 62 vi vii viii name hddv class scsi target 0 lun 0 name hddv class scsi target 0 lun 7 name hddv class scsi target 1 lun 0 name hddv class scsi target 1 lun 1 remove comments of the LUN in use name hddv class scsi target 1 lun 2 name hddv class scsi target 3 lun 0 name hddv class scsi target 3 lun 1 remove comments of extended LUN name hddv class scsi target 3 lun 2 name hddv class scsi target 3 lun 3 Check the LUN definition overlap between sd conf and hddv conf by executing hddvchk conf command The message example for Check NG means that target 3 is defined in both files and the procedures must be performed again from procedure 6 b iv The followings are examples of execution of the FJSVhddv package installation to opt The path of the hddvchk_conf command depends on the installed directory Check OK to procedure 6 b vii opt FISVhddv bin hddvchk_conf lt Return gt kernel drv sd conf and kernel drv hddv conf seems to be correct Check NG to procedure 6 b iv opt FISVhddv bin hddvchk_conf lt Return gt NG ID
102. nel nannel nannel nannel nannel nannel 1 20 1 20 O SY O ion hddv hddv hddv hddv hddv hddv hddv hddv 22203 22 53 22753 22 53 22 53 22 55 3 22 753 ls 1 dev dsk grep pci 89 2000 fibre channel 1 hddv 3 0 c3t3d0s0 3 0 a c3t3d0sl 3 0 h lt Return gt ix Start the format 1M command and configure labeling and partition slice information calculated in 6 b viii to the logical path name of each extended LUN 7 Setup redundant system Execute the following commands to create a new redundant system In case of MPHD the commands need to be executed for each LUN specifying the path name determined in 6 b vili as a parameter The italic letters below are the new multipath names and the underlined letters are the path names that compose the multipath MP HD IOMP Node dev FJSVmphd fiomp adm2 dev FJSVmphd rdsk mphd2s0 dev FJSVmphd rdsk mphd2s1 dev FJSVmphd rdsk mphd2s2 dev FJSVmphd rdsk mphd2s3 dev FJSVmphd rdsk mphd2s4 usr opt FJSViomp bin iompadm c mphd new dev rdsk c3t3d0s2 dev rdsk c4t3d0s2 lt Return gt 55 56 dev FJSVmphd rdsk mphd2s5 dev FJSVmphd rdsk mphd2s6 dev FJSVmphd rdsk mphd2s7 GRMPD usr sbin mplbconfig nv lt Return gt kkk Phase 1 checking dev rdsk xxx Phase 2 checking multi path Phase 3 analyzing path Path Act
103. nfigured unknown Connect PCI cards The cfgadm command is executed with the configure option to Ap Id or the push button of the PCI bus slot is pushed Then a extended PCI card is connected In addition a push button is valid only in multiple user mode Please confirm that the slot status is connected configured by using the cfgadm 1M command after the above mentioned procedure cfgadm c configure pcipsy18 ROBO1 PCI slot02 lt Return gt cfgadm pcipsy18 ROBO1 PCI slot02 lt Return gt Ap Id Type Receptacle Occupant Condition pcipsy18 R0B01 PCI slot02 pci pci hp connected configured ok Start the hardware monitoring of the Machine Administration Please execute the following command in order to update configuration information and restart the hardware monitoring of the Machine Administration usr sbin FJSVmadm postphp lt Return gt Connect equipment to PCI cards The extended PCI card is connected with network equipment by the cable Setup driver The driver configuration is added by following operations a The interface name is investigated in order to configure drivers and high layer products Please save the result of the prtcicl 1M command and obtain the difference information between current result and the result taken at 2 a Then the driver instance number for the extended PCI card is obtained In the following example since the instance number is 1 it can be det
104. ng logical path name under dev dsk The following examples shows that c3t3d0 is logical path name of extended lun 0 The logical path name of lun 1 2 is set to e8t38d1 c3t3d2 respectively because cXtYdZshow X PCI card number Y target number Z lun number prtpicl v gt tmp prtpicl post lt Return gt diff tmp prtpicl pre tmp prtpicl post grep devfs path grep hddv lt Return gt gt devfs path pci 8d 2000 fibre channel 1 hddv 3 0 gt devfs path pci 8d 2000 fibre channel 1 hddv 3 1 gt devfs path pci 8d 2000 fibre channel 1 hddv 3 2 ls 1 dev dsk grep pci 8d 2000 fibre channel 1 hddv 3 0 lt Return gt lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s0 gt devices pci 8d 2000 fibre channel 1 hddv 3 0 a lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0sl gt devices pci 8d 2000 fibre channel 1 hddv 3 0 b lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s2 O SS gt devices pci 8d 2000 fibre channel hddv 3 0 c lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s3 gt devices pci 8d 2000 fibre channel 1 hddv 3 0 d lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s4 gt devices pci 8d 2000 fibre channel 1 hddv 3 0 e lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s5 gt devices pci 8d 2000 fibre channel 1 hddv 3 0 f lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s6 gt devices pci 8d 2000 fibre channel 1 hddv 3 0 g lrwxrwxrwx
105. nt grep c2t3d0 lt Return gt Sep 30 01 00 51 2002 Sep 30 01 00 51 2002 Sep 30 01 00 51 2002 01 00 50 2002 export home on dev dsk c2t3d0s3 setuid read write largefiles on Mon develop firm on dev dsk c2t3d0s0 setuid read write largefiles on Mon develop drv on dev dsk c2t3d0sl setuid read write largefiles on Mon pub on dev dsk c2t3d0s6 setuid read write largefiles on Mon Sep 30 ii Stop access to the disk To check which process is using the target file system use the fuser 1M command as follows fuser c export home lt Return gt 12048c 11971ctm 11952ctm 11937c 11867c 11846c export home 14967c 14571c 14493ctm 14020c 13828tm 13803c 13575c 13133c 13125tm 13107c 12682ctm 12066tm iii Unmount the disk umount export home lt Return gt umount develop firm lt Return gt umount develop drv lt Return gt umount pub lt Return gt If the device is a disk unit raw access operation or a tape device i Check the access statistics of the target disk determined in procedure 2 a Do the same with a tape device iostat xc lt Return gt xtended devic statistics cpu device r s w s kr s kw s wait actv svc_t w b us sy wt id sdo 59 7 7 5 474 5 45 0 0 0 349 58 46 0 41 3 Th 3 267 sdl GT 03 1 0 2 0 0 0 0 16 0 0 0 sd20 0 0 0 03 0 7 0 0 0 14 7 0 0 st82 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 nfsl 0 0 0 0 0 0 0 0 0 0 O 0 0 0 0 ii St
106. obp device gt inam fibre channel usr sbin FJSVpfca fc_adm 1l fjpfcal lt Return gt Edit kernel drv fjpfca conf and add the following property definitions to the instance number from 7 a i port topology classification link speed Define this property when PCI cards using Fabric connection are added This definition is not needed with FC AL connection fcp bind target connection of target ID and WWN Define this property when disk array equipment using Fabric connection are added This definition is not needed with FC AL connection max throttle number of maximum SCSI command issue This property is defined for restriction on simultaneous issue of SCSI commands The necessity and value of this definition depends on the specification of the disk array device so refer to the user s manual of each equipment The following example shows the definition of a fjpfca1 PCI card instance number 1 and disk array equipment target 3 being added vi kernel drv fjpfca conf lt Return gt port fjpfca0 nport fjpfcal nport sp1 lt added fcp bind target fjpfca0t0 0x50000e1000002501 fjpfcalt3 0x50000e1000002ab0 lt added max throttle fjpfca0t0 128 fjpfcalt3 240 lt added iii Execute the following command to make the fjpfca driver take in the added part usr sbin FJSVpfca fc_config v lt Return gt b Add device driver configuration Add device driver configura
107. on The hardware monitoring of the Machine Administration is stopped by using the following command usr sbin FJSVmadm prephp lt Return gt 2 Add PCI cards a Before extending the PCI card please save the result current configuration information of the prtpicl 1M command prtpicl v gt tmp prtpicl pre lt Return gt b Check the status of the slot Subsequent examples describe extension of PCI cards to the slot ROBO1 PCI slot02 The ROBO1 PCI slot02 indicates the I O cabinet number 0 the PCI Disk Box number 1 physical slot number 02 In the extension of a new redundant system although two or more PCI cards are extended whenever you add a PCI card please perform from procedure 2 b to 3 Connect PCI cards The relationship of the position of a PCI card and format of Ap_Id is shown below N an integral number Location of PCI slot Element Format of Ap_Id Cabinet of Cabinet Number X System Board Number Y pceipsyN CXMOY PCI slotZZ Physical Slot Number ZZ I O Cabinet Number X PRIMEPOWER 900 1500 2500 PCI Disk Box PCI Disk Box Number Y pcipsyN RXBOY PCI slotZZ Physical Slot Number ZZ other than those above Physical Slot Number ZZ pcipsyN PCI slotZZ Please confirm that the PCI slot status which extended PCI cards is empty unconfigured by using cfgadm 1M command cfgadm grep ROBO1 PCI slot02 lt Return gt pcicsy18 R
108. op disk access of applications Disk access statistics of the target disk can be checked as below It is the same with tape devices The example below shows access statistics of one minute rtw s blks s avwait avserv sar d 60 1 lt Return gt SunOS machineO 5 8 Generic 108528 05 sun4u 10 02 02 17 56 00 device busy avque 17 57 00 nfsl 0 0 0 sd0 2 0 3 2 sd0 a O 1 0 sd0 b 0 0 0 0 sco c 0 0 0 0 sd0 d i 0 0 0 sd0 e 0 0 0 0 sd0 T 0 1 I sd0 g 1 Osl J sd20 0 0 0 0 sd20 a 0 0 0 0 sd20 c 0 0 0 0 sd20 g 0 00 0 st82 0 0 0 0 0 0 0 0 0 w AQ 145 5 301 4 3155 0 0 126 8 0 0 120 9 Bn A CO KR CO oO WwW C4 O OC 304 OF On SS Oo OO OO OO ER pe OQ Oo 2 So e Or 2 Oo e lt e fo fo iii Stop deactivate applications See the manuals of each software for details If the device is a disk unit swap device i Show a list of swap devices and check that the disk of the logical path determined in procedure 2 b c2t3d0 is a swap device swap l lt Return gt dev dsk c2t3d0s4 32 164 16 788384 swapfile dev swaplo blocks free 683680 ii Delete the swap device swap d dev dsk c2t3d0s4 lt Return gt 4 Disconnect the PCI card Disconnect the target PCI card with the following procedure a By using inst2comp 1M command determine the slot position of the PCI card called an attachment point identifier Ap Id from t
109. operation with non active extension procedures to procedure 6 b ii ls kernel drv hddv conf lt Return gt kernel drv hddv conf When operating with non active extension procedures execute the following commands and confirm that kernel drv hddv conf is deleted hddvadm i lt Return gt ls kernel drv hddv conf lt Return gt kernel drv hddv conf No such file or directory Use the following command to create hddv conf file for the active extension function The message examples for operation with non active extension procedures means that disk array device GR720 currently used with taget 1 lun 0 is registered twice on the sd hddv driver New installation to procedure 6 b v hddvadm M lt Return gt Operation with non active extension procedures to procedure 6 b iv hddvadm M lt Return gt already ID 1 exists devices pci ld 2000 fibre channel 1 sd 1 0 a raw GR720 When performing pre defined extension the same target number cannot be registered in sd conf and hddv conf so delete or comment out the conflicting definition parts in sd conf file when definition in the two files conflict with each other The following example shows the case that target 1 is repeated vi kernel drv sd conf lt Return gt name sd class scsi target 0 lun 0 name sd class scsi target 1 lun 0 name sd class scsi Delete or comment out target 1 lun 1 th
110. or details see SN200 Series Affinity User s Guide or ETERNUS3000 GR700 800 series GRmgr User s Guide Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration Set up driver Add the configuration of driver according to the following operation As for file related drivers additional configuration of the PCI card driver and the device driver is needed a Add configuration for PCI card driver SCSI driver Solaris OE glm FUJITSU PCI SCSI Adapter Driver Configuration is not required Fibre Channel driver FUJITSU PCI Fibre Channel i After saving execution result of prtpicl 1M command take the difference between the result and the information obtained in procedure 3 a to calculate the instance number of the fjpfca driver of the PCI card added In the following example instance number is 1 indicating that the interface name of the PCI card added is fjpfcal Making the LED of the PCI card blink using the fc adm command can specify the relation between the instance number and the slot position The relation between the WWPN and instance number of the PCI card added can be known from the fc info a command prtpicl v gt tmp prtpicl pfc
111. orresponding to the replacement slot position After the new PCI card is connected confirm that the slot status has changed to connected configured cfgadm c configure pcipsy21 R0B01 PCI slot03 lt Return gt cfgadm pcipsy21 ROBO1 PCI slot03 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy21 ROBO1 PCI slot03 fibre hp connected configured ok When exchanging Fibre Channel cards the following operations are also required When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility a Refresh current condition on management window SAN Storage Area Network management function searchs and indicates replaced PCI card This function will adjust switch zoning and host affinity function of disk array device ETERNUS3000 GR700 800 series with new WWN of replaced PCI card See Operation guide or User s Manual of each product for details Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors T O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup processes may end in errors Stop backup before changing Affinity configuration b Reconnect the replaced PCI card with changed zonig setting by using the following commands cfgadm c disconnect pcipsy21 ROB01 PCI slot03 lt Return gt
112. ount filesystem mount F ufs dev FJSVmphd dsk mphd2s6 mnt lt Return gt raw access operation Start applications For details see the manual of each application swap device This procedure is not required 3 2 2 3 Extension of PCI cards to redundant system The procedure to add PCI cards to a redundant system in order to extend redundancy on a Solaris 9 OE system is explained with an example with GR multipath driver GRMPD There is no additional pattern for Multipath Disk Control MPHD If multipath software products other than GRMPD are being used see the manual of each product No configuration is needed for applications when adding PCI cards to an existing redundant system 1 2 Stop the machine administration hardware monitoring daemon Use the following command to stop the hardware monitoring daemon of machine administration usr sbin FJSVmadm prephp lt Return gt Add PCI cards To add PCI cards with the following configurations the WWPN World Wide Port Name 16 digit number of the card to add is needed If zone configuration by WWPN is done with Fibre Channel switch SN200 series If the Host Affinity function of ETERNUS3000 GR700 800 series is used When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility go to procedure 2 a because these products get WWPNs from PCI cards automatically The WWP
113. r of the system disk or CD ROM cannot be 17 deleted In this case change the target number in hddv conf and of the disk array device so that it does not conflict with sd conf e Edit the kernel drv hddv conf file created automatically by the hddvadm M command and remove comments of the LUN definition to pre define and the LUN definition currently being used with non active extension procedures If case active extension has been performed before and the LUN definition to extend the next procedures 1 f 1 g are not needed The following example shows target 3 lun 0 gt 2 for pre defined extension and disk array device GR720 target 1 in use with non active extension procedures being registered ls kernel drv hddv conf lt Return gt kernel drv hddv conf vi kernel drv hddv conf lt Return gt name hddv class scsi target 0 lun 0 name hddv class scsi target 0 lun 7 name hddv class scsi target 1 lun 0 name hddv class scsi target 1 lun 1 Remove comments of the LUN in use name hddv class scsi target 1 lun 2 name hddv class scsi target 3 lun 0 name hddv class scsi target 3 lun 1 Remove comments of extended LUN name hddv class scsi target 3 lun 2 name hddv class scsi target 3 lun 3 f Check for LUN definition conflict between sd conf and hddv conf by executing the hddvchk_conf command The message example for Check NG means that target 3 is
114. rd is 100000000e44061 0e24 4061 a Before extension of PCI cards save the current execution result of the prtpicl 1M command current configuration at present prtpicl v gt tmp prtpicl pre lt Return gt b Check the status of the slot The relationship of the position of a PCI card and format of Ap_Id is shown below Location of PCI slot Definition of slot Format of Ap_Id Inside chassis of Chassis Nox X System board No Y pcipsyN CXMOY PCI slotZZ PRIMEPOWER 900 1500 2500 Physical slot No ZZ I O Chassis No X PCI Disk BOX PCI Disk BOX No Y__ pcipsyN RXBOY PCI slotZZ Physical slot No ZZ Other Physical slot No ZZ pcipsyN PCI slotZZ Below an example of adding a PCI card to chassis number 0 PCI Disk BOX number 1 physical slot number 02 slot in PCI Disk BOX ROBO01 PCI slot02 is explained Check the status of the target slot is empty unconfigured cfgadm grep ROBO1 PCI slot02 lt Return gt pcipsy18 ROBO1 PCI slot02 unknown empty unconfigured unknown c To confirm the slot position at extension operation blink the ALARM LED of Ap_Id displayed in procedure 2 b cfgadm x led fault mode blink pcipsy18 ROBO1 PCI slot02 lt Return gt d After checking that the READY LED on the PCI slot to add the PCI card is off add PCI cards This operation is performed by our customer support e Use the cfgadm 1M command to
115. scsi 2 sd 2 0 d lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s4 gt devices pcit89 4000 scsi 2 sd 2 0 e lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s5 gt devices pcit89 4000 scsi 2 sd 2 0 f lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s6 gt devices pcit89 4000 scsi 2 sd 2 0 g lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s7 gt devices pci 89 4000 scsig2 sde2 0 h Tape device diff tmp prtpicl pre tmp prtpicl post grep devfs path grep st lt Return gt gt devfs path pci 89 4000 scsi 2 st 5 0 ls 1 dev rmt grep pci 89 4000 scsi 2 st 5 0 lt Return gt lrwxrwxrwx 1 root root 41 Sep 20 22 53 1 gt devices pcit89 4000 scsi 2 st 5 0 lrwxrwxrwx 1 root root 41 Sep 20 22 53 1b gt devices pcit89 4000 scsi 2 st 5 0 b lrwxrwxrwx 1 root root 41 Sep 20 22 53 1bn gt devices pcil89 4000 scsi 2 st 5 0 bn iii od lrwxrwxrwx 1 root root Sep 20 22 53 ic gt devices pci 89 4000 scsi 2 st 5 0 c lrwxrwxrwx 1 root root 41 Sep 20 22 53 1cb gt devices pcil89 4000 scsi 2 st 5 0 cb lrwxrwxrwx 1 root root 41 Sep 20 22 53 I1cbn gt devices pci 89 4000 scsi 2 st 5 0 cbn lrwxrwxrwx 1 root root 41 Sep 20 22 53 Ilcn gt devices pcil89 4000 scsi 2 st 5 0 cn lrwxrwxrwx 1 root root 41 Sep 20 22 53 1h gt devices pcit89 4000 scsi 2 st 5 0 h lrwxrwxrwx 1 root root 41 Sep
116. sed with taget 1 lun 0 is registered twice on the sd hddv driver New installation to procedure 6 b v hddvadm M lt Return gt Set up with a non active extension procedure to procedure 6 b iv hddvadm M lt Return gt already ID 1 exists devices pci ld 2000 fibre channel 1 sd 1 0 a raw GR720 In activity extension because same target number can t be registered in sd conf and hddv conf delete or comment out the duplication definition parts in sd conf file when both definition are overlapped The following example shows the case that target 1 is repeated vi kernel drv sd conf lt Return gt name sd class scsi target 0 lun 0 name sd class scsi target 1 lun 0 name sd class scsi delete or comment out target 1 lun 1 the duplication definition parts in sd hddv name sd class scsi target 1 lun 2 A target number that is already used by the sd driver of the system disk or CD ROM cannot be deleted In this case change the target number in hddv conf and of the disk array device so that it does not conflict with sd conf Edit the kernel drv hddv conf file created automatically by the hddvadm M command and remove comments of the LUN definition to pre define and the LUN definition currently being used with non active extension procedures If case active extension has been performed before and the LUN definition to extend the next procedures 1 f 1 g are not
117. ses may end in errors Stop backup before changing Affinity configuration Connect the PCI card Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the replacement slot position Note that the push button is only effective in multiuser mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy3 COM0O0 PCI slot02 lt Return gt cfgadm pcipsy3 COMOO PCI slot02 lt Return gt Ap Id Type Receptacle Occupant Condition pcipsy3 COMO00 PCI slot02 mult hp connected configured ok When exchanging Fibre Channel cards the following operations are also required When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility a Refresh current condition on management window SAN Storage Area Network management function searchs and indicates replaced PCI card This function will adjust switch zoning and host affinity function of disk array device ETERNUS3000 GR700 800 series with new WWN of replaced PCI card See Operation guide or User s Manual of each product for details Note When changing Affinity configuration on
118. t root 41 Sep 20 22 53 c3t3d0s6 gt devices pci 89 2000 fibre channel 1 hddv 3 0 g lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s7 O SS gt devices pci 89 2000 fibre channel hddv 3 0 h iv Start the format 1M command and configure labeling and partition slice information calculated in 7 b iii to the logical path name of each extended LUN 8 Setup applications Add configuration for the application using the logical path name determined in 7 b Disk device filesystem operation Create file system To mount at system start add an entry in the etc vfstab file newfs dev rdsk c2t2d0s6 lt Return gt vi etc vfstab lt Return gt dev dsk c2t2d0s6 dev rdsk c2t2d0s6 mnt ufs 1 yes Disk device raw access operation Tape device Configure applications For details see the manual of each application Disk device swap device Add swap device To make valid at system start add an entry in the etc vfstab file swap a dev dsk c2t2d0s4 lt Return gt vi etc vfstab lt Return gt dev dsk c2t2d0s4 swap no swap l lt Return gt swapfile dev swaplo blocks free dev dsk c2t2d0s4 32 164 16 788384 683680 9 Start applications Use the logical path name determined in 7 b to start applications Disk device filesystem operation Mount filesystem mount F ufs dev dsk c2t2d
119. tZZ Physical slot No ZZ I O Chassis No X Inside chassis of PRIMEPOWER 900 1500 2500 PCI Disk BOX PCI Disk BOX No Y pcipsyN RXBOY PCT slotZZ Physical slot No ZZ Other Physical slot No ZZ pceipsyN PCI slotZZ Below an example of adding a PCI card to chassis number 0 system board number 1 physical slot number 02 slot in chassis COM01 PCI slot02 is explained Check the status of the target slot is empty unconfigured cfgadm grep COM01 PCI slot02 lt Return gt pcipsy15 COM01 PCI slot02 unknown empty unconfigured unknown c To confirm the slot position at extension operation blink the ALARM LED of Ap_Id displayed in procedure 2 b cfgadm x led fault mode blink pcipsy15 COM01 PCI slot02 lt Return gt d After checking that the READY LED on the PCI slot to add the PCI card is off add PCI cards This operation is performed by our customer support e Use the cfgadm 1M command to check that the slot status of the added PCI card changed to disconnected unconfigured ik cfgadm pcipsy15 COM01 PCI slot02 lt Return gt Ap Id Type Receptacle Occupant Condition pcipsy15 C0M01 PCI slot02 unknown disconnected unconfigured unknown 3 Connect PCI cards Connect the replaced PCI card using the cfgadm 1M command with the configure option or by pushing the button corresponding to the extension slot position Note that the push button is only effecti
120. taking the difference between the result and the information obtained in 3 a and calculate the corresponding logical path name under dev dsk In the following example c3t3d0 is logical path name of added lun 0 The logical path name of lun 1 2 is set to c8t3d1 c8t8d2 respectively because cXtYdZmeans X PCI card number Y target number Z lun number prtpicl v gt tmp prtpicl post lt Return gt diff tmp prtpicl pre tmp prtpicl post grep devfs path grep hddv lt Return gt 23 gt devfs path pci 89 2000 fibre channel 1 hddv 3 0 gt devfs path pci 89 2000 fibre channel 1 hddv 3 1 gt devfs path pci 89 2000 fibre channel 1 hddv 3 2 ls 1 dev dsk grep pci 89 2000 fibre channel 1 hddv 3 0 lt Return gt lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s0 gt devices pci 89 2000 fibre channel 1 hddv 3 0 a lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0sl gt devices pci 89 2000 fibre channel 1 hddv 3 0 b lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s2 O Bias gt devices pci 89 2000 fibre channel hddv 3 0 c lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s3 gt devices pci 89 2000 fibre channel 1 hddv 3 0 d lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s4 gt devices pci 89 2000 fibre channel 1 hddv 3 0 e lrwxrwxrwx 1 root root 41 Sep 20 22 53 c3t3d0s5 gt devices pci 89 2000 fibre channel 1 hddv 3 0 f lrwxrwxrwx 1 roo
121. talled so start from procedure 6 b iii If it does not exist the system is being operated with none active extension procedures so start from procedure 6 b ii Not exist new install to Procedure 6 b iii ls kernel drv hddv conf lt Return gt ii iii iv kernel drv hddv conf No such file or directory Exist operation with non active extension procedures to Procedure bs briis ls kernel drv hddv conf lt Return gt kernel drv hddv conf When operating with non active extension procedures execute the following commands and confirm that kernel drv hddv conf is deleted hddvadm i lt Return gt ls kernel drv hddv conf lt Return gt kernel drv hddv conf No such file or directory Use the following command to create hddv conf file for the pre defined extension function The message examples for operation with non active extension procedures means that disk array device GR720 currently used with taget 1 lun 0 is registered twice on the sd hddv driver New installation to procedure 6 b v hddvadm M lt Return gt Operation with non active extension procedures to procedure 6 b iv hddvadm M lt Return gt already ID 1 exists devices pci ld 2000 fibre channel 1 sd 1 0 a raw GR720 When performing active extension the same target number cannot be registered in sd conf and hddv conf so delete or comment out the conflict
122. tart applications By following operations please start the high layer product a When using TCP IP interface The high layer product services applications using the TCP IP interface is started For details of starting the high layer products please refer to the manual of each product b When using FNA SNA communication protocol over Ethernet It is necessary to start all interfaces of Netcompo FNA LAN and all high layer applications that are using Netcompo FNA LAN When using the Multipath Function the FNA Fujitsu Network Architecture SNA System Network Architecture communication protocol over Ethernet is valid For details please refer to the manual of Netcompo FNA LAN and high layer applications 4 2 3 Extension of PCI cards to redundant system This subsection describes the procedure extending PCI cards to the existing redundant configuration system with the following redundant configuration software SynfinityLink 2 2 PRIMECLUSTER GLS 4 0 or later The replacement and extension of PCI cards are supported only on Redundant Line Control Function of SynfinityLink When using other redundant software please refer to manuals of each product 73 Moreover when extending PCI cards to the existing redundant configuration system it is unnecessary to operate about the high layer products Consequently PCI cards can be extended without stopping the high layer products 1 Stop the hardware monitoring of the Machine Administrati
123. the hardware monitoring daemon of machine administration usr sbin FJSVmadm prephp lt Return gt 2 Specify the replacing PCI card Follow the instructions below and determine the interface name of the path connecting the target PCI card and I O devices and the connected I O device a Determine from the WARNING messages output on the console the interface name of the path connecting the target PCI card and I O devices and the connected I O device In the example below glm2 is the interface name of the path connecting the broken PCI card and I O devices and sd20 is the disk device connected to glm2 When exchanging a PCI card which has two ports the physical path name of the connection path between the I O device corresponding to the other port and the I O device connected to that path must be determined in the same manner The physical path name corresponding to the other port can be determined by the fact that the name of the two ports on the same PCI card have a relation like pci 89 4000 scsi 2 and pci 89 4000 scsi 2 1 The I O device connected to that path resides under the directory of the physical path name determined WARNING pci 89 4000 scsi 2 glm2 invalid intcode fe00 WARNING pci 89 4000 scsi 2 sd 3 0 sd20 SCSI transport failed reason reset giving up The following procedures are also explained assuming the console messages above b Determine the logical path name under dev dsk
124. tion with the following procedures according to the I O device type 21 22 Disk Tape devices Solaris OE sd st This example is a case where a disk device of target 2 lun 0 is pre defined in procedure 1 and added in procedure 6 If the device is a tape device re read sd as st in this procedure and perform the same operation i ii Execute the following command to make the sd driver set up the added disk device usr sbin devfsadm lt Return gt After saving the execution result of the prtpicl 1M command take the difference between the result and the information obtained in 3 a and get the physical path name of additional disk device then get the logical path name under dev dsk In the following example c2t2d0 is the logical path name of the disk device added prtpicl v gt tmp prtpicl post lt Return gt Disk device diff tmp prtpicl pre tmp prtpicl post grep devfs path grep sd lt Return gt gt devfs path pci 89 4000 scsi 2 sd 2 0 ls 1 dev dsk grep pci 89 4000 scsi 2 sd 2 0 lt Return gt lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s0 gt devices pcit89 4000 scsi 2 sd 2 0 a lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0sl gt devices pcit89 4000 scsi 2 sd 2 0 b lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s2 gt devices pcit89 4000 scsi 2 sd 2 0 c lrwxrwxrwx 1 root root 41 Sep 20 22 53 c2t2d0s3 gt devices pcit89 4000
125. tions are also required When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility Go to procedure 6 When not using SAN management function of the above products To replace PCI cards with the following configurations Fibre Channel switch SN200 series and disk array device ETERNUS3000 GR700 800 series need to be reconfigured individually If zone configuration by WWPN World Wide Port Name is done on SN200 series If the Host Affinity function of ETERNUS3000 GR700 800 series is used For details see SN200 Series Affinity User s Guide or ETERNUS3000 GR700 800 series GRmgr User s Guide To perform the above reconfiguration the WWPN a 16 digit number of the replacement card is needed The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card These characters represent the bottom eight digits of the WWPN in hexadecimal form The top eight digits are fixed to 10000000 in hexadecimal form For example if the following label is shown on the front plate of the card the WWPN of the replacement card is 100000000e44061 0e24 4061 Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers normally because of retry processes but on Fibre Channel tape devices backup proces
126. to Chapter 3 File devices or Chapter 4 Network devices about detailed replacing procedure of PCI cards 1 Stop the hardware monitoring of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration 2 Specify the replacing PCI card Specify the devices and interfaces that are influenced by replacing PCI cards 3 Stop applications Stop services and high layer products that are influenced by replacing PCI cards 4 Disconnect the PCI card It changes into the state where the PCI card is exchangeable 5 Replace the PCI card Replace the PCI card 6 Connect the PCI card It changes into the state where the PCI card is usable 7 Start applications Start high layer products and services used by the replaced PCI card 8 Start the hardware monitoring of the Machine Administration Update the hardware configuration information and restart the hardware abnormal monitoring of the Machine Administration 2 1 2 Replacement of PCI cards on redundant system When exchanging PCI cards with redundant software it is possible to exchange PCI cards without stopping services and applications The replacement procedure of PCI cards is as follows Please refer to Chapter 3 File devices or Chapter 4 Network devices about detailed replacing procedure of PCI cards 1 Stop the hardware monitoring of the Machine Administration Stop the hardware abnormal monitoring of the Machine Administration 2 Specify the replacing PC
127. turn gt cfgadm pcipsy18 ROB01 PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy18 ROBO1 PCIl slot02 mult hp connected configured ok Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJSVmadm postphp lt Return gt Connect devices to PCI cards Connect new disk array device in redundant configuration to the extended PCI card When the Host Affinity function is being used with Fibre Channel switch SN200 series Host Affinity must be configured for the device added When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility WWPN zoning of SN200 series Host affinity of Array disk device ETERNUS3000 GR700 800 series and binding setting for fjpfca conf sd conf of host can be added from SAN management window automatically See Operation guide or User s Manual of each product for details Before using this function the following procedure 7 a i and 7 a ii except fep bind target setting are necessary Moreover begin from procedure 7 a iii after executing this function When not using SAN management function of the above products SN200 series need to be reconfigured individually F
128. ve in multiuser mode After the new PCI card is connected use the cfgadm 1M command and confirm that the slot status has changed to connected configured If a large scale configuration of I O devices is connected to the PCI card in the target slot command execution for status confirmation may take time cfgadm c configure pcipsy15 C0M01 PCI slot02 lt Return gt cfgadm pcipsy15 C0M01 PCI slot02 lt Return gt Ap_Id Type Receptacle Occupant Condition pcipsy15 C0M01 PCI slot02 mult hp connected configured ok 4 Update hardware configuration information of machine administration Start the hardware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJSVmadm postphp lt Return gt 41 42 5 Connect equipment to PCI cards Connect I O devices to add to the extended PCI card When the Host Affinity function is being used with Fibre Channel SN200 series Host Affinity must be configured for the device added When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility WWPN zoning of SN200 series Host affinity of Array disk device ETERNUS3000 GR700 800 series and binding setting for fjpfea conf sd conf of host can be added from SAN management window automatically See Operation guide or User s Manua
129. ware monitoring daemon Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon usr sbin FJSVmadm postphp lt Return gt Connect devices to PCI cards Connect I O devices to add to the extended PCI card When the Host Affinity function is being used with Fibre Channel SN200 series Host Affinity must be configured for the device added When using any of SAN management function of Systemwalker StorageMGR SANView for ETERNUS except for Vixel and SP5000 SRM Facility WWPN zoning of SN200 series Host affinity of Array disk device ETERNUS3000 GR700 800 series and binding setting for fjpfca conf sd conf of host can be added from SAN management window automatically See Operation guide or User s Manual of each product for details Before using this function the following procedure 7 a i and 7 a ii except fep bind target setting are necessary Moreover begin from procedure 7 a iii after executing this function When not using SAN management function of the above products SN200 series need to be reconfigured individually For details see SN200 Series Affinity User s Guide or ETERNUS3000 GR700 800 series GRmgr User s Guide Note When changing Affinity configuration on SN200 series I O to other devices is effected by the change and may result in temporal errors I O to disk array devices recovers norm
130. ysical Slot Number ZZ I O Cabinet Number X PRIMEPOWER 900 1500 2500 PCI Disk Box PCI Disk Box Number Y pcipsyN RXBOY PCI slotZZ Physical Slot Number ZZ other than those above Physical Slot Number ZZ pceipsyN PCI slotZZ Please confirm that the PCI slot status which extended PCI cards is empty unconfigured by using cfgadm 1M command cfgadm grep COM01 PCI slot02 lt Return gt pcicsy15 COM0O1 PCI slot02 unknown empty unconfigured unknown c To confirm the slot position at extension operation blink the ALARM LED of Ap_Id displayed in 69 70 procedure 2 b cfgadm x led fault mode blink pcipsy15 COM01 PCI slot02 lt Return gt d Please confirm that READY LED of the PCI bus slot that extends a PCI card has gone out and extend a PCI card This operation is performed by our customer support e Please confirm that the PCI slot status which extended PCI cards is disconnected unconfigured by using cfgadm 1M command cfgadm pcipsy15 COM01 PCI slot02 lt Return gt Ap Id Typ Receptacl Occupant Condition pcicsy15 COM01 PCI slot02 unknown disconnected unconfigured unknown 3 Connect PCI cards The cfgadm command is executed with the configure option to Ap_Id or the push button of the PCI bus slot is pushed Then extended PCI card is connected In addition a push button is valid only in multiple user mode Please confirm that the slot

PCI Hot Plug User`s Guide I/O device edition

Contents

Download Pdf Manuals

Related Search

Related Contents