Home

PCI Expansion Unit for Fujitsu M10/SPARC M10 Systems Service

1. Caution Before you handle any components wear a wrist strap to ground any static electricity If you perform this procedure without a wrist strap individual components or the overall system may be damaged For details see 1 4 Notes Regarding Static Electricity Accessing the PCI tray 1 Pull out the following components halfway Power supply unit For details see 13 3 Removing a Power Supply Unit PCI Express PCIe card cassette For details see 8 3 2 Removing a PCI Express card cassette 212 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 2 Remove the front cover For details see 5 7 3 Removing the front cover 3 Remove all the fan units For details see 14 3 2 Removing a fan unit 15 3 2 Removing the PCI tray 1 Loosen the two screws securing the PCI tray and then slightly pull it out Figure 15 2 Screw for PCI tray 2 Hold the PCI tray with both hands and carefully remove it from the chassis Note Place the removed PCI tray on the grounded ESD mat to ground any static electricity Chapter 15 Maintaining the PCI Tray 213 Figure 15 3 PCI tray removal 15 3 3 Removing the components of the PCI tray 1 Remove the I O board from the PCI tray For details see 16 3 2 Removing the I O board 2 Remove the fan backplane from the PCI tray For details see 17 3 2 Removing the fan backplane 15 4 Installing the PCI Tray This section describe
2. 2 The LEDs on the link card and the statuses indicated by those LEDs are listed below Chapter 2 Understanding the PCI Expansion Unit Components 25 26 Table 2 7 Location Name number 1 LINK STATUS PCI Ex 2 LINK STATUS Management Color Green Green 1 The blink interval is 1 second 1 Hz LEDs and status of link card Status On Blinking 1 Off On Blinking 1 Off Description The link is established with PCI Express Gen3 x8 The link is established with other than PCI Express Gen3 x8 degradation condition The link is down The management link is established The management link is disconnected Indicates that power is not being supplied PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 3 Types of Maintenance This chapter describes the maintenance types of the PCI expansion unit Maintenance Types of the PCI Expansion Unit Types of Maintenance for the Link Card Maintenance Types of the PCIe Card 3 1 Maintenance Types of the PCI Expansion Unit This section describes the maintenance types of the PCI expansion unit Maintenance of the PCI expansion unit is divided into two types depending on the input power status of the PCI expansion unit hot maintenance and cold maintenance The terms are defined as follows Hot maintenance Maintenance can be performed with the power cord of the PCI expansion unit connected
3. When mounting a component in the chassis confirm that the connectors on the chassis and components do not have any bent pins and that the pins are aligned If you attempt to mount a component while any of the connectors have bent pins the chassis or component may be damaged When mounting a component perform the work carefully so as not to bend any pins If you cannot reach the latch lock of the connector when removing the management cable or other cables depress the latch with the tip of a flat bladed screwdriver and then remove the cable If you attempt to remove the cable forcibly the link board and or the PCI Express PCle card may be damaged Do not use a power cord other than the specified type Chapter 1 Before Starting Maintenance Work 5 1 6 Emergency Power Off This section explains the procedure for powering off the system in the case of an emergency Caution Immediately shut down the product in the event of an emergency for example when the chassis emits smoke or flame and then disconnect the input power Prevention of fire must always be your highest priority regardless of the task that you are currently performing 1 Remove all the power cords from the power supply unit installed at the rear of the chassis Figure 1 4 Removing power cords 1 7 Important Information about the XCP Firmware This section describes important information that you should know about the XCP firmware version and when
4. 10 maintenance by causing the CHECK LED locator of the unit to blink Checking the operating condition of the system and the I O device usage status Stopping the power supply to the FRU requiring maintenance Enabling access to the FRU requiring maintenance Replacing a faulty FRU Restoring the chassis requiring maintenance Confirming that there is no problem with the replacement FRU Turning off the CHECK LED locator of the PCI expansion unit requiring maintenance Closing the rack door Hot replacement work procedure for a power supply unit or fan unit Reference 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 4 1 Checking the operating condition of the physical partition or logical domain 5 6 Stopping the FRU Requiring Maintenance For a power supply unit 5 7 1 Lowering the cable support For a fan unit 5 7 3 Removing the front cover Chapter 13 Maintaining the Power Supply Units Chapter 14 Maintaining the Fan Units For a power supply unit 6 1 3 Fixing the cable support For a fan unit 6 1 1 Installing the front cover 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 7 Checking the FRU Status after Maintenance in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 5 Turning off the CHECK LED Locator of the PCI Expansion Unit Chapter 7 Maintenance Flo
5. Closing the rack door Cold removal work procedure removing the link card after stopping the system continued Reference 5 2 Switching the Mode Switch to Service Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual or Fujitsu M10 1 SPARC M10 1 Service Manual 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 4 1 Checking the operating condition of the physical partition or logical domain 5 6 Stopping the Entire System in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 5 Stopping the Entire System in the Fujitsu M10 1 SPARC M10 1 Service Manual 5 9 2 Lowering the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 8 1 Removing the power cords in the Fujitsu M10 1 SPARC M10 1 Service Manual Chapter 8 Maintaining the PCI Express Cards in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual or Fujitsu M10 1 SPARC M10 1 Service Manual 6 1 2 Fixing the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 1 6 Installing the power cords in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 3 1 Diagnosing the system board in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 4 Diagnosing a Replacement FRU in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 3 3 Checking the FRU status
6. The following example shows that test is set as the current configuration information ldm 1ls spconfig factory default testl current test2 test3 Execute the Idm Is constraints x command to save the current configuration information If necessary backup XML In the following example the current configuration information is saved in test1 xml ldm ls constraints x gt var tmp test1 xml PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 6 Execute the more command to confirm that the configuration information is correct more var tmp test1 xml lt xml version 1 0 gt lt LDM_interface version 1 3 xmlns xsi http www w3 org 2001 XMLSchema instancce 7 If there are multiple pieces of configuration information to be saved save all of them Repeat from step 2 to step 6 to save the configuration information 8 Execute the Idm set spconfig factory default command to set the system to the factory default ldm set spconfig factory default ldm 1ls spconfig factory default next poweron testl current test2 test3 9 Execute the Idm rm spconfig command to delete all the configuration information ldm rm spconfig testl ldm rm spconfig test2 ldm rm spconfig test3 ldm 1ls config factory default next poweron 10 Stop the logical domain to set it to the OpenBoot PROM state Before turning off the power to the system stop the logical
7. 134 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 7 15 Item il 2 10 11 12 Task Opening the rack door Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the operating condition of the system and the I O device usage status Releasing the link card mounted on the server from the system using physical partition dynamic reconfiguration DR Lowering the cable support Removing the power cords of the PCI expansion unit Removing the PCIe card Installing the power cords of the PCI expansion unit Fixing the cable support Mounting the released link card on the system using physical partition dynamic reconfiguration DR Confirming that there is no problem with the replacement FRU Closing the rack door 1 4 7 4 1 Cold removal work procedure Releasing the link card using DR Reference 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 4 Checking the Operating Condition and Resource Usage Status 5 5 2 Releasing the link card or PCIe card using physical partition dynamic reconfiguration DR 5 7 1 Lowering the cable support 5 7 2 Removing the power cord Chapter 8 Maintaining the PCI Express Cards 6 1 2 Installing the power cords 6 1 3 Fixing the cable support 6 3 2 Mounting the link card on the server using ph
8. 2 Setting the root domain in delayed reconfiguration mode 3 Deleting the physical I O from the root domain 4 Restarting the root domain 5 Assigning the physical I O to the root domain 6 Starting the I O domain 7 Disabling the hotplug service Command svcadm enable hotplug ldm start reconf ldm remove io shutdown i6 g0 y Idm add io ldm start domain svcadm disable hotplug Dynamic reconfiguration is used 1 Perform Perform Perform Perform Differences in the procedures according to whether the dynamic reconfiguration function for PCle Unnecessary Dynamic reconfiguration is not used Perform 2 Perform Perform Perform Perform Perform Perform 2 1 Dynamic reconfiguration for PCIe end point devices is supported by XCP 2230 or later and Oracle VM Server for SPARC 3 1 1 1 or later 2 If dynamic reconfiguration for PCIe end point devices is not used the hotplug service operates whether it is enabled or disabled 90 1 If the hotplug service is disabled for the I O and root domains execute the svcadm enable hotplug command to enable the hotplug service svcadm enable hotplug 2 Transition the root domain to delayed reconfiguration mode This step is unnecessary if the dynamic reconfiguration function for PCIe end point devices is used Perform step 5 ldm start reconf lt root domain name gt 3 Delete the physical I O device from the root domain
9. 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 4 1 Checking the operating condition of the physical partition or logical domain 5 5 Powering Off the Physical Partition Requiring Maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 9 2 Lowering the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Chapter 8 Maintaining the PCI Express Cards in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 1 2 Fixing the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 1 Diagnosing the system board in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 6 Updating the Firmware of the PCI Expansion Unit 6 6 Returning the Mode Switch to Locked Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 7 Powering On the Physical Partition Requiring Maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 1 For details on the work for installing a PCI expansion unit see the Fujitsu M10 SPARC M10 Systems Installation Guide Chapter 7 Maintenance Flow 157 Inactive removal link card removal with the physical partition stopp
10. Checking the firmware version and matching it with the pre replacement one Returning the mode switch on the operation panel to Locked mode Cold replacement work procedure replacing the link card after stopping the system Reference 5 2 Switching the Mode Switch to Service Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual or Fujitsu M10 1 SPARC M10 1 Service Manual 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 3 Checking the Firmware Version of the PCI Expansion Unit 5 4 1 Checking the operating condition of the physical partition or logical domain 5 6 Stopping the Entire System in the Fujitsu M10 4 Fujitsu M10 45 SPARC M10 4 SPARC M10 4S Service Manual 5 5 Stopping the Entire System in the Fujitsu M10 1 SPARC M10 1 Service Manual 5 9 2 Lowering the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 8 1 Removing the power cords in the Fujitsu M10 1 SPARC M10 1 Service Manual Chapter 8 Maintaining the PCI Express Cards in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual or Fujitsu M10 1 SPARC M10 1 Service Manual 6 1 2 Fixing the cable support in the Fujitsu M10 4 Fujitsu M10 45 SPARC M10 4 SPARC M10 4S Service Manual 6 1 6 Installing the power cords in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 3 1 Diagnosing the system board in the Fujitsu M
11. 6 3 1 Diagnosing the system board in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 2 Mounting the link card on the server using physical partition dynamic reconfiguration DR 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Inactive maintenance You can perform inactive maintenance only for a system that uses the SPARC M10 4S in a building block configuration This section describes the flows of the inactive replacement inactive addition and inactive removal for the link card References to detailed descriptions are written in the work procedure tables See any of them as required The following patterns are available for inactive maintenance of the link card Chapter 7 Maintenance Flow 151 Inactive replacement link card replacement with the physical partition stopped Inactive addition link card installation with the physical partition stopped Inactive removal link card removal with the physical partition stopped Inactive replacement link card replacement with the physical partition stopped Stop the physical partition in which the link card requiring maintenance is mounted and then replace the link card 152 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 7 20 Inactive replacement flow link card replacement with the physical partition stopped 1 Openi
12. For details see Chapter 7 Maintenance Flow Caution Before you handle any components wear a wrist strap to ground any static electricity If you perform this procedure without a wrist strap individual components or the overall system may be damaged For details see 1 4 Notes Regarding Static Electricity 1 Push the latch lock of the management cable on the PCI expansion unit and then remove the management cable Note If you cannot not reach the latch lock of the connector depress the latch with a flat head screwdriver and then remove the management cable Do not use excessive force Otherwise the link board could be damaged 200 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 12 2 Management cable removal 2 Remove the management cable from the SPARC M10 1 or SPARC M10 4 M10 4S in the same way 12 4 Installing the Management Cable This section describes the procedure for installing the management cable 1 Hold the connector of the management cable and connect the management cable to the link board mounted on the PCI expansion unit Note Confirm that the management cable is securely connected 2 Connect the management cable to the link card mounted in the SPARC M10 1 or SPARC M10 4 M10 4S in the same way Note Confirm that the management cable is securely connected Now the work for mounting the FRU is complete While referencing Chapter 7 Maintenance
13. Hot maintenance can be performed for the FRUs to be maintained by using the ioxadm or cfgadm command Cold maintenance Maintenance is performed with the power cord of the PCI expansion unit disconnected Cold maintenance can be performed for all the FRUs of the PCI expansion unit Cold maintenance work varies depending on the method of releasing the link card mounted in the destination server Releasing the link card after stopping the system This can be done when the connection destination of the PCI expansion unit is a SPARC M10 1 M10 4 M10 4S Cold maintenance of the PCI expansion unit can be performed after stopping the system to which the PCI expansion unit is connected 27 28 Releasing the link card using PHP This can be done when the connection destination of the PCI expansion unit is a SPARC M10 4 M10 4S and the setting for the direct I O function is disabled Cold maintenance of the PCI expansion unit can be performed after releasing the link card mounted in the destination server from the system using the PCI hot plug PHP Releasing the link card using DR This can be done only when the connection destination of the PCI expansion unit has the building block configuration of the SPARC M10 4S Cold maintenance of the PCI expansion unit can be performed after releasing the chassis in which the link card is mounted from the system using physical partition dynamic reconfiguration DR Table 3 1 to Table 3 3 are list
14. NRTL C Electromagnetic wave VCCI A FCC A DOC A KCC and C Tick Security and electromagnetic wave CE and GOST R Figure 1 1 Positions of system nameplate label standard label 2 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 The RFID tag contains the Asset ID The RFID tag is attached to the front cover of the chassis Figure 1 2 RFID tag 341 A583DAAAAAAAAAAAAAAAA Asset ID 1 3 Safety Precautions Observe the following precautions to protect yourself when performing maintenance Observe all the precautions warnings and instructions described on the chassis Do not insert foreign objects into the openings in the chassis Any such foreign object could come into contact with high voltage circuitry or could short circuit Chapter 1 Before Starting Maintenance Work 3 the components causing a fire or an electric shock Contact a service engineer and request inspection of the chassis Safety precautions on electricity Confirm that the voltage and frequency of your input power supply match those shown on the electric rating label affixed on the chassis Wear a wrist strap when handling the I O board PCI tray or other print boards Use a grounded power outlet Do not make any mechanical or electrical alterations We do not take any responsibility for issues arising from non authorized modifications to the chassis 1 4 Notes Regarding
15. duplication disclosure modification and adaptation of the programs including any operating system integrated software any programs installed on the hardware and or documentation shall be subject to license terms and license restrictions applicable to the programs No other rights are granted to the U S Government Disclaimer The only warranties granted by Oracle and Fujitsu Limited and or any affiliate in connection with this document or any product or technology described herein are those expressly set forth in the license agreement pursuant to which the product or technology is provided EXCEPT AS EXPRESSLY SET FORTH IN SUCH AGREEMENT ORACLE OR FUJITSU LIMITED AND OR THEIR AFFILIATES MAKE NO REPRESENTATIONS OR WARRANTIE S OF ANY KIND EXPRESS OR IMPLIED REGARDING SUCH PRODUCT OR TECHNOLOGY OR THIS DOCUMENT WHICH ARE ALL PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS REPRESENTATIONS AND WARRANTIES INCLUDING WITHOUT LIMITATION ANY IMPLIED WARRANTY OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE OR NONINFRINGEMENT ARE DISCLAIMED EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID Unless otherwise expressly set forth in such agreement to the extent allowed by applicable law in no event shall Oracle or Fujitsu Limited and or any of their affiliates have any liability to any third party under any legal theory for any loss of revenues or profits loss of use or data or business interruptions or for any indirec
16. of the logical domain In the following example Status of the logical domains of PPAR ID 00 displays Solaris running which indicates that Oracle Solaris is operating XSCF gt showdomainstatus p 0 Logical Domain Name Status primary Solaris running guest0 Solaris running guestl Solaris running iodom0 Solaris running iodoml Solaris running sdiodomain Solaris running sr iodomain Solaris running 54 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 5 4 2 Checking the assignment status of I O devices In the following execution example the operating condition of the logical domain the resource usage status and the assignment status of I O devices are checked from Oracle Solaris on the logical domain For details see 3 2 Operations and Commands Related to Logical Domain Configurations in the Fujitsu M10 SPARC M10 Systems Domain Configuration Guide 1 Login to Oracle Solaris on the logical domain 2 Execute the following Idm command to check the operating condition of the logical domain ldm list domain 3 Execute the following Idm command to check the resource usage status ldm list devices a 4 Execute the following Idm command to check the assignment status of I O devices ldm list io For maintenance on a PCIe card retain the output data of the Idm list io command since you will need it later when restoring the original configuration 6 4 Restoring the Logical Domai
17. t fourni ne vous conf re aucun droit et aucune licence expr s ou tacites concernant le produit ou la technologie auxquels il se rapporte Par ailleurs il ne contient ni ne repr sente aucun engagement de quelque type que ce soit de la part d Oracle ou de Fujitsu Limited ou des soci t s affili es de l une ou l autre entit Ce document ainsi que les produits et technologies qu il d crit peuvent inclure des droits de propri t intellectuelle de parties tierces prot g s par le droit d auteur et ou c d s sous licence par des fournisseurs a Oracle et ou ses soci t s affili es et Fujitsu Limited y compris des logiciels et des technologies relatives aux polices de caract res Conform ment aux conditions de la licence GPL ou LGPL une copie du code source r gi par la licence GPL ou LGPL selon le cas est disponible sur demande par I Utilisateur Final Veuillez contacter Oracle et ou ses affili s ou Fujitsu Limited Cette distribution peut comprendre des composants d velopp s par des parties tierces Des parties de ce produit pourront tre d riv es des syst mes Berkeley BSD licenci s par l Universit de Californie UNIX est une marque d pos e de The OpenGroup Oracle et Java sont des marques d pos es d Oracle Corporation et ou de ses affili s Fujitsu et le logo Fujitsu sont des marques d pos es de Fujitsu Limited SPARC Enterprise SPARC64 le logo SPARC64 et toutes les marques SPARC sont utilis es sous licence et so
18. 1 1 1 or later Table 3 6 Maintenance types of the PCle card of the PCI expansion unit when the destination server has a building block configuration Not maintainable Domain to which the PCI expansion PCle card requiring maintenance Maintenance types of the PCle card unit belongs Direct I O 1 PHP support SR IOV 2 Hot use of PHP Hot useofDR Cold Control domain Unsupported Supported Unsupporte Enabled 3 Enabled d supported 8 Unsupported Unsupported Unsupporte Enabled d supported Supported Unsupported Unsupported Enabled Supported Supported Unsupported Enabled 9 Enabled Root domain Unsupported Supported Unsupported Enabled 4 Enabled 5 Enabled Unsupported Supported Supported Enabled 4 Enabled 6 Enabled 8 Unsupported Unsupported Unsupported Enabled 5 Enabled Unsupported Unsupported Supported Enabled 6 Enabled Supported Unsupported Unsupported Enabled 7 Enabled 32 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 3 6 Maintenance types of the PCle card of the PCI expansion unit when the destination server has a building block configuration continued Not maintainable Domain to which PCle card requiring maintenance Maintenance types of the PCle card the PCI expansion unit belongs Direct I O 1 PHP support SR IOV 2 Hot use of PHP Hot useofDR Cold Supported Supported Unsupported Enabled 9 Enabled 7 Enabled 1
19. 17 component specifications 239 F failure identifying 38 fan backplane 227 fan unit 207 I O board 217 L labels 2 link board 181 link cable 195 link card 189 M management cable 199 P PCI Express card 171 PCI tray 211 power supply unit 203 S system restoring 75 tags 2 247 248 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015
20. 3 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 4 Checking the firmware version of the PCI expansion unit 5 Checking the operating condition of the system and the I O device usage status 6 Stopping the system 7 Enabling access to the FRU requiring maintenance 8 Replacing a faulty link card 9 Restoring the System 10 Performing a diagnosis test i 11 Confirming that there is no problem with the replacement FRU 12 Checking the firmware version and matching it with the pre replacement one 13 Returning the mode switch on the operation panel to Locked mode 14 Starting the system 15 Closing the rack door End 162 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 7 25 Item Task 1 Opening the rack door 2 Switching the mode switch on the 10 11 12 13 operation panel of the destination server to Service mode Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the firmware version of the PCI expansion unit Checking the operating condition of the system and the I O device usage status Stopping the system Enabling access to the FRU requiring maintenance Replacing a faulty link card Restoring the System Performing a diagnosis test Confirming that there is no problem with the replacement FRU
21. 4S Service Manual 6 7 Checking the FRU Status after Maintenance in the Fujitsu M10 1 SPARC M10 1 Service Manual PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Cold removal releasing the link card using DR Cold removal of a PCI Express card from the PCI expansion unit can be performed after releasing the chassis in which the link card is mounted from the system using physical partition dynamic reconfiguration DR This can be done only when the connection destination of the PCI expansion unit has the building block configuration of the SPARC M10 4S Chapter 7 Maintenance Flow 133 Figure 7 13 Cold removal flow releasing the link card using DR 1 Opening the rack door 2 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 3 Checking the operating condition of the system and the I O device usage status 4 Releasing the link card mounted on the server from the system using physical partition dynamic reconfiguration DR 5 Lowering the cable support 6 Removing the power cords of the PCI expansion unit 7 Removing the PCle card 8 Installing the power cords of the PCI expansion unit 9 Fixing the cable support 10 Mounting the released link card on the system using physical partition dynamic reconfiguration DR 11 Confirming that there is no problem with the replacement FRU 12 Closing the rack door End
22. 5 Cold replacement flow releasing the link card using DR 1 Opening the rack door 2 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 3 Checking the firmware version of the PCI expansion unit if you are replacing an I O board 4 Checking the operating condition of the system and the I O device usage status 5 Releasing the link card mounted on the server from the system using physical partition dynamic reconfiguration DR 6 Enabling access to the FRU requiring maintenance 7 Replacing a faulty FRU 8 Restoring the PCI expansion unit 9 Mounting the released link card on the system using physical partition dynamic reconfiguration DR 10 Confirming that there is no problem with the replacement FRU 11 Closing the rack door 114 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 7 7 Item Task 1 Opening the rack door 2 Locating the PCI expansion unit requiring 10 11 maintenance by causing the CHECK LED locator of the unit to blink Checking the firmware version of the PCI expansion unit if you are replacing an I O board Checking the operating condition of the system and the I O device usage status Releasing the link card mounted on the server from the system using physical partition dynamic reconfiguration DR Enabling access to the FRU requiring maintenance Replacing a faulty FRU Resto
23. 8 in the execution example 5 Assignment of both BB0 PCIO and PCIE1 to primary 6 Assignment of BBO PCI4 to iodom1 and PCIE2 to primary 7 Assignment of both BB0 PCI8 and PCIE3 to rootdom 8 Assignment of BBO PCI5 to iodom2 and PCIE6 to rootdom Logical domain to which the PCle root complex to which the PCle card belongs is assigned Name PCIEx indicated in BUS in the line confirmed above indicates the name of the PCIe root complex to which the PCIe card belongs The name indicated in DOMAIN of the same PCIe root complex line where NAME indicates PCIEx indicates the logical domain to which the PCle root complex to which the PCIe card belongs is assigned 1 to 4 in the execution example 1 Assignment of PCIE1 to primary 2 Assignment of PCIE2 to primary 3 Assignment of PCIE3 to rootdom 4 Assignment of PCIE6 to rootdom 5 Based on the check results in step 4 determine whether PHP can be used according to the combination of above two pieces of information Suppose that the logical domains to which the PCIe slot and PCIe root complex are assigned are both the control domains primary Active hot maintenance can be performed using PHP from the control domain See 5 in the execution example of the ldm list io 1 command Suppose that the logical domains to which the PCIe slot and PCIe root complex are assigned are not control domains and are the same logical domains root domains Active hot maintenance can be perf
24. Board Configuration of the I O Board Removing the I O Board Installing the I O board 16 1 Before Maintaining the I O Board This chapter describes only the configuration removal work and mounting work of an I O board Before removal work of an FRU perform necessary work items while referencing Chapter 7 Maintenance Flow For maintenance types of the I O board see I O board in Table 7 2 List of available maintenance types by FRU 16 2 Configuration of the I O Board This section describes the configuration and location of the I O board Eleven PCI Express PCIe slots and one slot for the link board are provided on the T O board 217 Figure 16 1 Location of the I O board Location number Component 1 T O board IOB 16 3 Removing the I O Board This section describes the procedure for removing the I O board Enable the removal of the I O board before attempting to remove it For details see Chapter 7 Maintenance Flow Caution Before you handle any components wear a wrist strap to ground any static electricity If you perform this procedure without a wrist strap individual components or the overall system may be damaged For details see 1 4 Notes Regarding Static Electricity 16 3 1 Accessing the I O board 1 Pull out the following components halfway Power supply unit For details see 13 3 Removing a Power Supply Unit PCle card cassette For details see 8 3 2 Rem
25. Disabling the hotplug service svcadm disable hotplug Perform Perform 2 1 Dynamic reconfiguration for the virtual function of SR IOV is supported by XCP 2210 or later and Oracle VM Server for SPARC 3 1 or later 2 If static reconfiguration for the virtual function of SR IOV is used the hotplug service operates whether it is enabled or disabled 1 If the hotplug service is disabled for the I O and root domains execute the svcadm enable hotplug command to enable the hotplug service svcadm enable hotplug 2 Stop the I O domain This step is unnecessary if the dynamic SR IOV function is used Perform step 3 ldm stop domain lt I O domain name gt p Remove the virtual function VF assigned to the I O domain from the I O domain Execute the Idm remove io command to delete the virtual function VF assigned to the I O domain from the I O domain If multiple VFs are assigned from the physical function PF of a single PCIe card delete all the VFs corresponding to that PF from the I O domain ldm remove io lt VF name gt lt I O domain name gt 4 Transition the root domain that has the assigned PF to delayed reconfiguration mode This step is unnecessary if the dynamic SR IOV function is used Perform step 5 ldm start reconf lt root domain name gt 5 Discard the virtual function VF Execute the Idm destroy vf command to discard the virtual function VF However the VF needs to be discarded in the r
26. Enabled Enabled Chapter3 Types of Maintenance 31 Table 3 5 Maintenance types of the PCle card of the PCI expansion unit when the destination server has a single unit configuration continued Not maintainable Domain to which the PCI expansion PCle card requiring maintenance PCI card maintenance type unit belongs Direct I O 1 PHP support SR IOV 2 Hot use of PHP Cold Supported Supported Unsupported Enabled 6 Enabled Root domain Unsupported Supported Unsupported Enabled 4 Enabled Unsupported Supported Supported Enabled 4 5 Enabled Unsupported Unsupported Unsupported Enabled supported Supported Unsupported Unsupported Enabled supported Supported Supported Unsupported Enabled 4 6 Enabled 1 This is a status in which the PCIe card of the PCI expansion unit is assigned to a logical domain using direct I O 2 For details on whether to use the SR IOV function see 3 3 3 Checking whether the SR IOV function is used 3 Execute PHP from the control domain 4 Execute PHP from the root domain 5 The PHP function provided when the SR IOV function is used is supported by Oracle VM Server for SPARC 3 1 and later 6 Using the dynamic reconfiguration function for PCIe end point devices return the card from the I O domain with direct I O to the control domain or root and then execute PHP The dynamic reconfiguration function for PCle end point devices is supported by Oracle VM Server for SPARC 3
27. FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S5 SPARC M10 4 SPARC M10 4S Service Manual 6 6 Returning the Mode Switch to Locked Mode in the Fujitsu M10 4 Fujitsu M10 4S5 SPARC M10 4 SPARC M10 4S Service Manual 6 7 Powering On the Physical Partition Requiring Maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Maintenance with the system stopped This section describes the flows of the system stopped replacement system stopped addition and system stopped removal for the link card References to detailed descriptions are written in the work procedure tables See any of them as required The following patterns are available for system stopped replacement of the link card Cold replacement replacing the link card after stopping the system PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Cold addition installing the link card after stopping the system Cold removal removing the link card after stopping the system Cold replacement replacing the link card after stopping the system Stop the system in which the link card requiring maintenance is mounted and then replace the link card Chapter 7 Maintenance Flow 161 Figure 7 23 Cold replacement flow replacing the link card after stopping the system 1 Opening the rack door 2 Switching the mode switch on the operation panel of the destination server to Service mode
28. Flow 147 Table 7 20 Item 1 2 148 Task Opening the rack door Checking the operating condition of the system and the I O device usage status Releasing the link card mounted on the server from the system using PCI hot plug PHP Lowering the cable support Removing the link card Fixing the cable support Confirming that there is no problem with the chassis after the removal Closing the rack door Active removal work procedure removing the link card using PHP Reference 5 4 1 Checking the operating condition of the physical partition or logical domain 5 4 2 Checking the assignment status of I O devices 5 5 1 Releasing the link card or PCIe card using PCI hot plug PHP 5 9 2 Lowering the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Chapter 8 Maintaining the PCI Express Cards in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 1 2 Fixing the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Active removal removing the link card using DR Release the link card mounted on the destination server from the system using physical partition dynamic reconfiguration DR This can be
29. Hold the lever and carefully pull the PCIe card cassette out of the PCIe slot 2 installing the link card in a PCle cassette The procedure for installing a link card in a PCIe cassette is the same as that for installing a PCIe card of each server For details on the SPARC M10 1 see 8 4 1 Installing a PCI Express card in the Fujitsu M10 1 SPARC M10 1 Service Manual For details on the SPARC M10 4 M10 4S see 8 5 1 Installing a PCI Express card in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 192 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 3 Connect the link cable and the management cable to the link card 4 Install the PCle cassette containing the link card in the server The installation procedure is the same as that for a PCle cassette of each server For details on the SPARC M10 1 see 8 4 2 Installing a PCIe riser in the Fujitsu M10 1 SPARC M10 1 Service Manual For details on the SPARC M10 4 M10 4S see 8 5 3 Installing a PCI Express card cassette in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Now the work for mounting the FRU is complete While referencing Chapter 7 Maintenance Flow continue the maintenance work Chapter 10 Maintaining the Link Cards 193 194 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 11 11 1 Maintaining the Link Cables This chapter describes the
30. IOV function Unnecessary Item Task Command Dynamic SR IOV Static SR IOV function function is used 1 function is used 1 Enabling the hotplug service svcadm enable hotplug Perform Perform 2 2 Setting the root domain of the PF in ldm start reconf Perform delayed reconfiguration mode 3 Creating the VF Idm create vf Perform Perform 4 Assigning the VF to the I O domain ldm add io Perform Perform 5 Restarting the root domain shutdown i6 g0 y 7 Perform 6 Starting the I O domain start domain s Perform Chapter 6 Understanding the Preparations for Restoring the System 91 Table 6 4 Differences in the releasing procedures between the dynamic SR IOV function and static SR IOV function continued Unnecessary Item Task Command Dynamic SR IOV Static SR IOV function function is used 1 function is used 7 Disabling the hotplug service svcadm disable hotplug Perform Perform 2 1 Dynamic reconfiguration for the virtual function of SR IOV is supported by XCP 2210 or later and Oracle VM Server for SPARC 3 1 or later 2 If static reconfiguration for the virtual function of SR IOV is used the hotplug service operates whether it is enabled or disabled 1 If the hotplug service is disabled for the I O and root domains execute the svcadm enable hotplug command to enable the hotplug service svcadm enable hotplug 2 Transition the root domain that has the assigned physical function PF to delayed reconfiguration mode This st
31. Manual March 2015 5 9 Checking the Firmware Version of the PCI Expansion Unit This section describes the procedure for checking the firmware version of the PCI expansion unit The link card and I O board contain the PCI expansion unit firmware The firmware version cannot be automatically matched to the pre replacement firmware version when the link card or I O board is replaced Before proceeding with the replacement check the firmware version 1 Login to the XSCF shell 2 If multiple PCI expansion units are connected execute the ioxadm list command to check the combination of the link card and the I O board XSCF gt ioxadm list PCIBOX Link PCIBOX 2007 BB 00 PCI 10 PCIBOX 2006 BB 00 PCI 09 PCIBOX 2005 BB 00 PCI 08 PCIBOX 2004 BB 00 PCIL 07 3 Execute the ioxadm v list command to check the serial number and the firmware version of the PCI expansion unit Location displays the last four digits of the serial number of the PCI expansion unit and FW Ver displays the firmware version of the link card CARD shown on Type and that of the I O board IOBOARD shown on Type The following example shows that 2007 is the last four digits of the serial number of the PCI expansion unit and 1080 is the firmware version of the link card and the I O board XSCF gt ioxadm v list Location Type FW Ver Serial Num Part Num PCIBOX 2007 PCIBOX PZ21242007 PCIBOX 2007 PSU 0 PSU FEJD1212000521 CA01022 0750 D 7060
32. March 2015 LEDs on front of PCI expansion unit The status of the PCI expansion unit or fan unit can be determined by checking the LED or CHECK LED on the respective devices For details see 2 2 1 Front LEDs on the PCI expansion unit LEDs on rear of PCI expansion unit By checking the LEDs mounted on the components that can be maintained the status of each component or the error location can be determined For details see 2 2 2 LEDs on rear of PCI expansion unit LEDs on link card Check the LED on the link card of the server to which the PCI expansion unit is connected For details see 2 2 3 LEDs on link card Checking error messages Display the error messages to check the log information and obtain an error overview You can use either of the following two methods to check the error messages Checking the error log information with the XSCF shell For details see 12 1 Checking Log Saved with XSCF in the SPARC M10 Systems System Operation and Administration Guide Checking messages on Oracle Solaris For details see 12 2 Checking Warning and Notification Messages in the SPARC M10 Systems System Operation and Administration Guide Checking the status Execute the command to check the status of the PCI expansion unit PCI Express PCIe card or the system Table 4 1 lists the commands for checking the status Table 4 1 Status check command Type Command Description XSCF ioxadm Displays information on the PC
33. O device usage status Powering Off the Physical Partition Requiring Maintenance Enabling access to the FRU requiring maintenance Removing the link card Restoring the System Performing a diagnosis test Confirming that there is no problem with the replacement FRU Returning the mode switch on the operation panel to Locked mode Powering On the Physical Partition Requiring Maintenance Closing the rack door Inactive removal work procedure link card removal with the physical partition stopped Reference 5 2 Switching the Mode Switch to Service Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 4 1 Checking the operating condition of the physical partition or logical domain 5 5 Powering Off the Physical Partition Requiring Maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 9 2 Lowering the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Chapter 8 Maintaining the PCI Express Cards in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 1 2 Fixing the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 1 Diagnosing the system board in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 3 Checking the
34. Opening the rack door Checking the firmware version of the PCI expansion unit Checking the operating condition of the system and the I O device usage status Releasing the link card mounted on the server from the system using physical partition dynamic reconfiguration DR Releasing the chassis requiring maintenance from the system Lowering the cable support of the chassis requiring maintenance and removing the power cord Replacing a faulty link card Installing the power cord and fixing the cable support Incorporating the chassis requiring maintenance into the system Diagnosing the chassis requiring maintenance Mounting the released link card on the system using physical partition dynamic reconfiguration DR Confirming that there is no problem with the replacement FRU Closing the rack door Active replacement work procedure replacing the link card using DR Reference 5 3 Checking the Firmware Version of the PCI Expansion Unit 5 4 Checking the Operating Condition and Resource Usage Status 5 5 2 Releasing the link card or PCIe card using physical partition dynamic reconfiguration DR 5 8 1 Releasing of the SPARC M10 4S chassis possible only in a system with a building block configuration in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 9 Accessing an FRU in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Chapter 8 Maintaining the P
35. PCI expansion unit and updating the firmware as required Returning the mode switch on the operation panel to Locked mode Starting the system Cold addition work procedure installing the link card after stopping the system Reference 5 2 Switching the Mode Switch to Service Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual or Fujitsu M10 1 SPARC M10 1 Service Manual 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 4 1 Checking the operating condition of the physical partition or logical domain 5 6 Stopping the Entire System in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 5 Stopping the Entire System in the Fujitsu M10 1 SPARC M10 1 Service Manual 5 9 2 Lowering the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 8 1 Removing the power cords in the Fujitsu M10 1 SPARC M10 1 Service Manual Chapter 8 Maintaining the PCI Express Cards in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual or Fujitsu M10 1 SPARC M10 1 Service Manual 6 1 2 Fixing the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 1 6 Installing the power cords in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 3 1 Diagnosing the system board in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual
36. PHP confirm that the card is PHP enabled To do so check Appendix A Cards that Support PCI Hot Plug and Dynamic Reconfiguration in the Fujitsu M10 SPARC M10 Systems PCI Card Installation Guide 101 Table 7 2 List of available maintenance types by FRU FRU SPARC SPARC M10 4 M10 4S M10 1 single unit configuration Hot Cold Hot Cold maintenance maintenance maintenance maintenance PCI Express card 1 OK OK OK OK Link board 2 OK OK OK Link card mounted on server OK OK OK Link cable OK OK OK Management cable OK OK OK Power supply unit 3 OK OK OK OK Fan unit OK OK OK OK PCI tray OK OK I O board OK OK Fan backplane OK z OK Not maintainable SPARC M10 4S building block configuration Hot maintenance OK OK OK OK OK OK OK Cold maintenance OK OK OK OK OK OK OK OK OK OK 1 Maintenance work is performed using PCI hot plug PHP While referring to 3 3 2 How to check whether PCI hot plug PHP can be used check whether PHP is usable If the setting for the direct I O function is enabled then maintenance with the system stopped must be performed because the PCIe card cannot use PHP 2 It is necessary to use the PCI Hot Plug PHP to release the link card mounted on a destination server from the system If the setting for the direct I O function is enabled then maintenance with the system stopped must be performed because the link card cannot use PHP 3 This is supported on
37. Static Electricity Observe the precautions concerning the electrostatic discharge ESD as described in Table 1 1 to ensure the safety of personnel and the system Table 1 1 ESD precautions Item Note Wrist strap Wear an antistatic wrist strap when handling printed boards ESD mat An approved ESD mat provides protection from static damage when used together with a wrist strap The mat also acts as a cushion to protect the small parts that are attached to printed boards Antistatic bag After removing a printed board or component place it in the ESD safe packaging box antistatic bag or ESD safe packaging box Using a wrist strap Wear a wrist strap in such a way that the inner metal surface A in the figure of the wrist strap band is in contact with your skin Connect the clip B in the figure directly to the chassis strap clip to the chassis the operator and components assume the same electrical AN Caution Do not connect the wrist strap clip to the ESD mat By connecting the wrist potential thus eliminating the danger of damage from static discharge 4 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 1 3 Wrist strap connection destination 1 5 Other Precautions The printed boards of the PCI expansion unit are susceptible to static damage To prevent damage to the printed boards therefore wear a wrist strap and ground it to the chassis prior to starting maintenance
38. This is a status in which the PCIe card of the PCI expansion unit is assigned to a logical domain using direct I O 2 For details on whether to use the SR IOV function see 3 3 3 Checking whether the SR IOV function is used 3 Execute PHP from the control domain 4 Execute PHP from the root domain 5 Stop the root domain and then execute physical partition dynamic reconfiguration DR 6 Stop the I O domain of the virtual function VF of the SR IOV function stop the root domain delete and discard the VF of the SR IOV function and then execute physical partition dynamic reconfiguration DR 7 Stop the I O domain with direct I O stop the root domain delete the I O device and then execute physical partition dynamic reconfiguration DR 8 The PHP function provided when the SR IOV function is used is supported by Oracle VM Server for SPARC 3 1 and later 9 Using the dynamic reconfiguration function for PCIe end point devices return the card from the I O domain with direct I O to the control domain or root and then execute PHP The dynamic reconfiguration function for PCIe end point devices is supported by Oracle VM Server for SPARC 3 1 1 1 or later 3 3 2 How to check whether PCI hot plug PHP can be used Using PHP maintenance for the PCIe card mounted on the PCI expansion unit or the link card mounted on the server can be performed during Oracle Solaris operation This section enables you to check whether the PCIe card or link
39. Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 221 Front LEDs on the PCI expansion unit The following LEDs are mounted on the front panel of the PCI expansion unit to indicate the status Fan unit A in the figure PCI expansion unit B in the figure Figure 2 5 LEDs on front of the chassis The LEDs on each component and the statuses those LEDs indicate are listed below Table 2 1 LEDs and status of fan unit Name Color Status Description CHECK Amber On Indicates that an error has occurred A Blinking Indicates that the component requires maintenance 1 This function is also referred to as the locator Off Indicates the normal state Or the breaker is open or the power supply is otherwise off 1 The blink interval is 1 second 1 Hz Chapter 2 Understanding the PCI Expansion Unit Components 21 222 22 Table 2 2 Name READY CHECK LEDs and status of PCI expansion unit Color Green Amber Status On Blinking 1 Off On Blinking 1 Off 1 The blink interval is 1 second 1 Hz Description Indicates that the component is operating Power is supplied but the system is not running During this time all of the I O boards will be in the standby state The system is stopped Indicates that an error has occurred Indicates that the PCI expansion unit chassis requires maintenance This function is also referred to as the locator I
40. after maintenance in the Fujitsu M10 4 Fujitsu M10 4S5 SPARC M10 4 SPARC M10 4S Service Manual 6 7 Checking the FRU Status after Maintenance in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 6 Returning the Mode Switch to Locked Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 8 Returning the Mode Switch to Locked Mode in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 8 Starting the System in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 9 Starting the System in the Fujitsu M10 1 SPARC M10 1 Service Manual PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 8 Maintaining the PCI Express Cards This chapter describes the procedure for maintaining a PCI Express PCle card mounted in the PCI expansion unit PCIe cards can be replaced installed or removed Before Maintaining a PCI Express Card Configuration of the PCIe Cards Removing a PCI Express Card Installing a PCI Express Card 8 1 Before Maintaining a PCI Express Card This chapter describes only the configuration removal work and mounting work of a PCI Express card Before removal work of an FRU perform necessary work items while referencing Chapter 7 Maintenance Flow For maintenance types of the PCI Express card see PCI Express card in Table 7 2 List of available maintenance types by FRU 8 2 Configuration of the PCle Card
41. and restart Oracle Solaris This step is unnecessary if the dynamic SR IOV function is used Perform step 7 telnet localhost lt port number gt shutdown i6 g0 y 7 Ifyou need to disable the hotplug service for the I O and root domains disable the hotplug service svcadm disable hotplug 3 Releasing the link card or PCle card using PHP The following describes the procedure for releasing the link card or PCIe card requiring maintenance from the system using PHP 60 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Releasing the link card also releases the PCI expansion unit requiring maintenance from the system 1 If the link card or PCle card requiring maintenance belongs to the root domain connect to the console of the root domain If it belongs to the control domain this step is not necessary since the control domain console is being used 2 Login as an Oracle Solaris super user 3 If the hotplug service has been disabled execute the svcadm enable hotplug command to enable the hotplug service svcadm enable hotplug Pe Execute the cfgadm a command to check the Ap_ID of the link card or PCle card requiring maintenance and make a note of it cfgadm a a Execute the cfgadm c command to release the link card or PCle card requiring maintenance from the logical domain Enter the Ap_ID of the link card or PCIe card that you noted in advance cfgadm c unconfi
42. card from the PCIe card cassette Note Do not forcibly remove the PCIe card from the PCIe card cassette If excessive force is used to remove the PCle card it may be damaged Note Place the removed PCle card on a grounded antistatic ESD mat 176 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 8 7 PCIe card removal 8 4 8 4 1 Installing a PCI Express Card This section describes the procedure for installing a PCle card Perform expansion using the same procedure For removal install a filler unit for a PCIe card rather than a PCle card itself Installing a PCI Express card 1 Install the PCle card by inserting it into the connector of the PCle card cassette For removal you do not have to perform this step Mount the filler for a PCIe card in the PCle card cassette Note Confirm that the tab of the PCIe card A in the figure or the tab of the filler for the PCIe card is inserted into the notch B in the figure of the PCIe card cassette Chapter 8 Maintaining the PCI Express Cards 177 Figure 8 8 Installing a PCIe card 2 Install the bracket for the PCle card and then secure it with the screw 178 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 8 4 2 Figure 8 9 Installing the bracket 3 Mount the cover on the PCle card cassette Installing a PCI Express card cassette Caution When installing a PCIe card cassette confi
43. card requiring maintenance can use PHP Note Some PCIe cards do not support PHP For details see Appendix A Cards that Support PCI Hot Plug and Dynamic Reconfiguration in the Fujitsu M10 SPARC M10 Systems PCI Card Installation Guide Note If the setting of the direct I O function is enabled the link card cannot use PHP 1 Log in to the XSCF shell 2 Execute the showlogs command to identify the PCle card requiring maintenance XSCF gt showlogs error For details see Checking log information 3 Check the hardware and software configurations For details see Confirming the System Configuration in the Fujitsu M10 1 SPARC M10 1 Service Manual or in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Chapter 3 Types of Maintenance 33 4 Execute the Idm list io I command to check the logical domain to which the PCle card requiring maintenance is assigned and the logical domain to which the PCle root complex to which that PCle card belongs is assigned ldm list io 1 NAME TYPE BUS DOMAIN STATUS PCIEO BUS PCIEO primary pci s8000 PCIE1 BUS PCIE1 primary lt 1 pci 8100 PCIE2 BUS PCIE2 primary IOV lt 2 pci 8200 PCIE3 BUS PCIE3 rootdom lt 3 pci 8300 PCIE4 BUS PCIE4 primary pci s8400 PCIES BUS PCIES primary IOV pci 8500 PCIE6 BUS PCIE6 rootdom IOV lt 4 pci s8600 PCIE BUS PCIET7 primary IOV pci 8700 BBO CMUL NETO PCIE PCIEO primary OCC
44. cards Location Rear of chassis Active hot maintenance Supported Active cold maintenance Supported with SPARC M10 4 M10 45 connected Inactive hot maintenance Supported Inactive cold maintenance Supported System stopped maintenance Supported For the maintenance procedure see Chapter 8 Maintaining the PCI Express Cards 239 B 2 Link Board The link board is mounted in the slot that is dedicated to the link board for the PCIe card slot The link board is connected to the link card mounted on the SPARC M10 1 M10 4 M10 4S through the link cable and the management cable Table B 2 lists the specifications of the link board Table B 2 Link board specifications Item Description Number of link boards 1 Location Rear of chassis Active hot maintenance Supported with SPARC M10 4 M10 45 connected Active cold maintenance Supported with SPARC M10 4 M10 4S connected Inactive hot maintenance Supported Inactive cold maintenance Supported System stopped maintenance Supported For the maintenance procedure see Chapter 9 Maintaining the Link Board B 3 Power Supply Unit The power supply units take input power and supply it to the system The redundant configuration of the power supply units allows the system to continue operating even if one of the units fails Table B 3 lists the specifications of the power supply units Table B 3 Power supply unit specifications Item Description Number of power supply uni
45. cassette and then attach the filler that has been stored in a safe place Chapter 4 Preparation and Precautions for Maintenance 49 50 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 5 Preparations Required for Maintenance This chapter describes the preparations that must be completed before you can physically remove a component See this chapter as required when performing the maintenance work on each component described in Chapter 7 onwards Preparing Tools Required for Maintenance Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition Releasing the Link Card or PCIe Card from the System Stopping the FRU Requiring Maintenance Accessing an FRU Caution Do not operate a Field Replaceable Unit FRU requiring maintenance while OpenBoot PROM is running the ok prompt is displayed Turn off the power to the physical partition or start Oracle Solaris to release the PCI expansion unit from the server and then operate the FRU requiring maintenance 5 1 Preparing Tools Required for Maintenance This section explains the tools required for maintenance Table 5 1 lists the tools required for maintenance Table 5 1 Maintenance tools Item Use Phillips screwdriver No 2 Removing or replacing screws Wrist strap For grounding static electricity ESD mat For grounding static electricity 51 5 2 Checking the PCI Expansion Unit R
46. connecting the PCI expansion unit 6 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 12 1 1 7 2 Precautions for updating the XCP firmware SPARC M10 1 The logical domain configuration of the physical partition will return to the factory default at the next startup when any of the following conditions are met If the logical domain configuration contains a guest domain the OpenBoot PROM environment variable of the control domain will also be initialized When an update is performed from firmware version XCP2043 or earlier to firmware version XCP2044 or later in a configuration with a PCI expansion unit connected When a PCI expansion unit is installed or removed from a system with a firmware version that is XCP2044 or later Save the logical domain configuration information from Oracle Solaris to XML in advance To save it to XML execute the Idm list constraints x command and to restore it from XML execute the ldm init system i command Also save the setting information for the OpenBoot PROM environment variable of the control domain to the notepad and then reset it To display it execute the printenv command while the ok prompt is displayed For details on these procedures see 1 7 3 How to save restore the logical domain configuration information and the OpenBoot PROM environment variable The processes to be performed when it is necessary to save restore each item of information relat
47. done only when the link card is mounted on the M10 4S and the system is in a building block configuration of the SPARC M10 4S Chapter 7 Maintenance Flow 149 Figure 7 19 Active replacement flow replacing the link card using DR 1 Opening the rack door 2 Checking the operating condition of the system and the I O device usage status 3 Releasing the link card mounted on the server from the system using physical partition dynamic reconfiguration DR 4 Releasing the chassis requiring maintenance from the system 5 Lowering the cable support of the chassis requiring maintenance and removing the power cord 6 Removing the link card 7 Installing the power cord and fixing the cable support 8 Incorporating the chassis requiring maintenance into the system 9 Diagnosing the chassis requiring maintenance 10 Mounting the released link card on the system using physical partition dynamic reconfiguration DR 11 Confirming that there is no problem with the replacement FRU 12 Closing the rack door End 150 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 7 21 Item 1 2 10 11 12 7 4 2 Task Opening the rack door Checking the operating condition of the system and the I O device usage status Releasing the link card mounted on the server from the system using physical partition dynamic reconfiguration DR Releasing the chassis requiring maint
48. door Checking the firmware version of the PCI expansion unit Checking the operating condition of the system and the I O device usage status Releasing the link card mounted on the server from the system using PCI hot plug PHP Lowering the cable support Replacing a faulty link card Mounting the released link card on the system using PCI hot plug PHP Fixing the cable support Confirming that there is no problem with the replacement FRU Closing the rack door Active replacement work procedure replacing the link card using PHP Reference 5 3 Checking the Firmware Version of the PCI Expansion Unit 5 4 1 Checking the operating condition of the physical partition or logical domain 5 4 2 Checking the assignment status of I O devices 5 5 1 Releasing the link card or PCIe card using PCI hot plug PHP 5 9 2 Lowering the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Chapter 8 Maintaining the PCI Express Cards in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 1 Mounting a link card using PCI hot plug PHP 6 1 2 Fixing the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 45 SPARC M10 4 SPARC M10 4S Service Manual PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Act
49. link card on the server using physical partition dynamic reconfiguration DR This can be done only when the server on which the link card is to be installed has the building block configuration of the SPARC M10 4S 144 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 7 17 Active addition flow installing the link card using DR 1 Opening the rack door 2 Checking the operating condition of the system and the I O device usage status 3 Using physical partition dynamic reconfiguration DR to release the server on which the link card is to be installed from the system 4 Releasing the chassis requiring maintenance from the system 5 Lowering the cable support of the chassis requiring maintenance and removing the power cord 6 Installing the link card 7 Installing the power cord and fixing the cable support 8 Incorporating the chassis requiring maintenance into the system 9 Diagnosing the chassis requiring maintenance 10 Incorporating the released server into the system using physical partition dynamic reconfiguration DR 11 Confirming that there is no problem with the installed FRU 12 Closing the rack door End Chapter 7 Maintenance Flow 145 Table 7 19 Item Task 1 Opening the rack door 2 Checking the operating condition of the 10 11 12 system and the I O device usage status Using physical partition dynamic reconfiguration DR to
50. link card on the system using PCI hot plug PHP 7 Confirming that there is no problem with the installed FRU 8 Closing the rack door End 142 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 7 18 Item Task 1 Opening the rack door 2 Checking the operating condition of the system and the I O device usage status 3 Lowering the cable support 4 Installing the link card 1 5 Fixing the cable support 6 Mounting the installed link card on the system using PCI hot plug PHP 7 Confirming that there is no problem with the installed FRU 8 Closing the rack door Active addition work procedure installing the link card using PHP Reference 5 4 1 Checking the operating condition of the physical partition or logical domain 5 9 2 Lowering the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 10 4 Installing the Link Card 6 1 2 Fixing the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 1 Mounting a link card using PCI hot plug PHP 6 7 Checking the FRU Status after Maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 1 For details on the work for installing a PCI expansion unit see the Fujitsu M10 SPARC M10 Systems Installation Guide Chapter 7 Maintenance Flow 143 Active addition installing the link card using DR Installing the
51. maintenance procedure for the link cables Before Maintaining a Link Cable Configuration of the Link Cable Connection Ports Removing a Link Cable Installing a Link Cable Before Maintaining a Link Cable This chapter describes only the configuration removal work and mounting work of a link cable Before removal work of an FRU perform necessary work items while referencing Chapter 7 Maintenance Flow For maintenance types of the link cable see Link cable in Table 7 2 List of available maintenance types by FRU 11 2 Configuration of the Link Cable Connection Ports This section describes the configuration and location of the link cable connection ports The link cards of the SPARC M10 1 or SPARC M10 4 M10 4S are connected to the link board of the PCI expansion unit with the link cables and the management cable 195 Figure 11 1 Location of link cable connection ports Location number Component 1 Link cable LINKCBL 0 2 Link cable LINKCBL 1 11 3 Removing a Link Cable This section describes the procedure for removing a link cable Enable removal of the link cable before attempting to remove it For details see Chapter 7 Maintenance Flow Caution Before you handle any components wear a wrist strap to ground any static electricity If you perform this procedure without a wrist strap individual components or the overall system may be damaged For details see 1 4 Notes Regarding
52. mounted in the chassis requiring maintenance stop the use of the I O device to keep it unused Releasing the I O device in the case of a redundant configuration With the duplication function release the target I O device so that it will not be used For details on the work see the manual for the application being used Stop the use of the I O device in the case of a non redundant configuration Place the I O device in the unused state such as by deactivating the network or unmounting the disk For details on the work see the document of Oracle Solaris or the manual for the application being used 2 Deleting the assignment of the I O In the case that a virtual I O is used If the virtual I O service is used delete the virtual I O device For details see the Oracle VM Server for SPARC Administration Guide released by Oracle In the case that the PCle end point assignment is used If the PCIe end point device of the chassis requiring maintenance is assigned to an T O domain delete the PCIe end point device from the I O domain and return it to the root domain including the control domain The work procedure depends on whether the dynamic reconfiguration function for PCle end point devices is used For details see Table 5 2 Table 5 2 Differences in the procedures according to whether the dynamic reconfiguration function for PCle end point devices is used Item Task 1 Enabling the hotplug service 2 Stopping the I O do
53. of the fan unit before attempting to remove it For details see Chapter 7 Maintenance Flow Caution Before you handle any components wear a wrist strap to ground any static electricity If you perform this procedure without a wrist strap individual components or the overall system may be damaged For details see 1 4 Notes Regarding Static Electricity 14 3 1 Accessing a fan unit 1 Remove the front cover For details see 5 7 3 Removing the front cover 208 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 14 3 2 Removing a fan unit 1 Press the handle A in the figure of the unit requiring maintenance to the left and then remove the fan unit from its slot Note Place the removed fan unit on the grounded ESD mat to ground any static electricity Figure 14 2 Fan unit removal Chapter 14 Maintaining the Fan Units 209 14 4 14 4 1 14 4 2 Installing a Fan Unit This section describes the procedure for installing a fan unit Installing a fan unit 1 Insert the fan unit into its slot Note Do not force the fan unit into the slot Using excessive force may damage the component or the chassis 2 Firmly push the fan unit into its mounting location Note Confirm that the fan unit is inserted securely Restoring the chassis 1 Install the front cover For details see 6 1 1 Installing the front cover Now the work for mounting the FRU is complete While refer
54. ou la technologie fournis SAUF MENTION CONTRAIRE EXPRESSEMENT STIPULEE AU DIT CONTRAT ORACLE OU FUJITSU LIMITED ET OU LES SOCIETES AFFILIEES A L UNE OU L AUTRE ENTITE DECLINENT TOUT ENGAGEMENT OU GARANTIE QUELLE QU EN SOIT LA NATURE EXPRESSE OU IMPLICITE CONCERNANT CE PRODUIT CETTE TECHNOLOGIE OU CE DOCUMENT LESQUELS SONT FOURNIS EN L ETAT EN OUTRE TOUTES LES CONDITIONS DECLARATIONS ET GARANTIES EXPRESSES OU TACITES Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE A L APTITUDE A UNE UTILISATION PARTICULIERE OU A L ABSENCE DE CONTREFACON SONT EXCLUES DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE Sauf mention contraire express ment stipul e dans ce contrat dans la mesure autoris e par la loi applicable en aucun cas Oracle ou Fujitsu Limited et ou l une ou l autre de leurs soci t s affili es ne sauraient tre tenues responsables envers une quelconque partie tierce sous quelque th orie juridique que ce soit de tout manque a gagner ou de perte de profit de probl mes d utilisation ou de perte de donn es ou d interruptions d activit s ou de tout dommage indirect sp cial secondaire ou cons cutif m me si ces entit s ont t pr alablement inform es d une telle ventualit LA DOCUMENTATION EST FOURNIE EN L ETAT ET TOUTE AUTRE CONDITION DECLARATION ET GARANTIE EXPRESSE OU TACITE EST FORMELLEMENT EXCLUE DANS LA MESURE AUTORISEE PAR LA LOI EN VIGUEUR Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE R
55. pci pci pci pci pci pci pci pci pci pci sen kages 8100 pci 4 8100 pci 4 8100 pcie4 8100 pci 4 8100 pcie4 8100 pcie4 8100 pcie4 8000 pci 4 8000 pcie4 8000 pci 4 8000 pcit4 8000 pcie4 8000 pcie4 8000 pcie4 8000 pci 4 8000 pci 4 8000 pci 4 8000 pci 4 8000 pcit4 pci pci pci pci pci pci pci pci pci pci pci pci pci pci pci pci pci eO 0 pcie9 0 pci l 0 pci O0 0 pci 0 network 0 1 0 pci 0 network 0 eO 0 pci 8 0 pci 2 0 pci l 0 pci d 0 pci 8 pci d 0 pci 8 pci 0 pci 0 0 pci 8 pci 0 pci 0 pci d 0 pci 8 pci 0 pci 0 pci 0 pci l 0 pci 8 pci 0 pci 0 pci 0 pci l pci d 0 pci 8 pci 0 pci 0 pci 0 pci l pci 0 pci l1l Chapter4 Preparation and Precautions for Maintenance 41 42 pci 8000 pci 4 pci 0 pci 8 pci pci 8000 pci 4 pci 0 pcie 8 pci pci 8000 pci 4 pci 0 pci 8 pci pci 8000 pci 4 pci 0 pci 8 pci pci 8000 pci 4 pci 0 pci 8 pci pci 8000 pci 4 pci 0 pci 8 pci pci 8000 pci 4 pci 0 pci 8 pci pci 8000 pci 4 pci 0 pci 8 pci pci 8000 pci 4 pci 0 pci 8 pci pci 8000 pci 4 pci 0 pci 8 pci pci 8000 pci 4 pci 0 pci 8 pci pci 8000 pci 4 pci 0 pci 8 pci pci 8000 pci 4 pci 0 pci 8 pci pci 8000 pci 4 pci 0 pci 8 pci pci 8000 pci 4 pci 0 pci 8 pci FJSV eulsa 0 pci 8000 pci 4 pci 0 pcie 8 pci FISV eulsa 0 tape pci 8000 pci 4 pci 0 pci 8 pci FJSV eulsa 0 disk pci 8000 pci 4 pci 0 pci 2 pci pci 8000
56. pci 4 pci 0 pci 2 pci pci 8000 pci 4 pci 0 pci 2 pci 0 pci 0 pci 0 pei 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 peci 0 pci 0 pci 0 pei 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 pei 0 pci 0 pci 0 pci 0 pci 0 pci 0 pci 0 O0 usbe 4 1 0 usbe4 pci 8000 pci 4 pci 0 pci l network 0 1 pci 8000 pci 4 pci 0 pci l network 0d pci 8000 pci 4 pci 0 pci 0 scsi 0 pci 8000 pci 4 pci 0 pci 0 scsi 0 disk pci 8000 pci 4 pci 0 pci 0 scsi 0 tape virtual devices 100 channel devices 200 virtual devices 100 flashprom 0 virtual devices 100 rtc 5 virtual devices 100 console l virtual devices virtual devices virtual devices virtual devices virtual devices iscsi hba disk openprom client services packages obp tftp packages kbd translator packages SUNW asr packages dropins packages terminal emulator packages disk label packages deblocker pac pac kages SUNW builtin drivers 0 ok prtdiag command kages SUNW probe error handler 1 pci 0 pci 1 pci 0 pci s 1 pci 0 pci l 1 pci 0 pci d 1 pci 0 pci 1 pci 0 pci 1 pci 0 pci 1 pci 0 pci 1 pci 0 pci 1 pci 0 pci 1 pci 0 pci 1 pci 0 pci 1 pci 0 pci 1 pci 0 pci 1 pci 0 pci 1 pci 0 pci 1 pci 0 pci LO Ll pci o L1 pci 0 pci L
57. reconfiguration function for PCle end point devices confirm that the card supports the relevant function Check Appendix B Cards that Support SR IOV and Appendix D Cards that Support the Dynamic Reassignment Function for PCIe End Point Devices PCIe Cards in the same manual If you are performing PCIe card expansion remove the filler unit from the PCIe card cassette Store the removed filler in a safe place because it will be needed if you subsequently remove the PCIe card For details on the work for installing a PCI expansion unit see the Fujitsu M10 SPARC M10 Systems Installation Guide 4 2 3 Precautions for removal The following describes precautions for removal For PCIe card removal by PHP release of the multipath is necessary depending on the use of the PCle cards For active removal by PHP confirm that the card is PHP supported To do so check Appendix A Cards that Support PCI Hot Plug and Dynamic Reconfiguration in the Fujitsu M10 SPARC M10 Systems PCI Card Installation Guide If you perform active removal by combining PHP with dynamic SR IOV or the dynamic reconfiguration function for PCle end point devices confirm that the card supports the relevant function Check Appendix B Cards that Support SR IOV and Appendix D Cards that Support the Dynamic Reassignment Function for PCIe End Point Devices PCIe Cards in the same manual When performing PCIe card removal remove the PCIe card from the PCIe card
58. release the server on which the link card is to be installed from the system Releasing the chassis requiring maintenance from the system Lowering the cable support of the chassis requiring maintenance and removing the power cord Installing the link card 1 Installing the power cord and fixing the cable support Incorporating the chassis requiring maintenance into the system Diagnosing the chassis requiring maintenance Incorporating the released server into the system using physical partition dynamic reconfiguration DR Confirming that there is no problem with the installed FRU Closing the rack door Active addition work procedure installing the link card using DR Reference 5 4 Checking the Operating Condition and Resource Usage Status 5 5 2 Releasing the link card or PCIe card using physical partition dynamic reconfiguration DR 5 8 1 Releasing of the SPARC M10 4S chassis possible only in a system with a building block configuration in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 9 Accessing an FRU in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Chapter 8 Maintaining the PCI Express Cards in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 1 Restoring the Chassis in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 2 1 Incorporation of the SPARC M10 4S chassis possible only i
59. svcadm enable hotplug 3 Execute the cfgadm 1M command to check the mounting position Ap_lId of the target PCle card cfgadm a Pe Execute the cfgadm 1M command to supply power to the PCle card Enter the Ap_ID of the PCIe card that you noted in advance cfgadm c connect Ap ID 5 Execute the cfgadm 1M command to incorporate the PCle card into the logical domain For the Ap_ID enter the Ap_ID of the link card you noted down before the start of maintenance cfgadm c configure Ap ID 6 Execute the cfgadm 1M command to confirm that the PCle card requiring maintenance has been incorporated into the logical domain In the following example the Receptacle is displayed as connected and the Occupant as configured indicating that the PCIe card requiring maintenance is incorporated into the logical domain cfgadm a Ap_Id Type Receptacle Occupant Condition BB 0 PCL 0 iobE7a77 pcie4 etherne hp connected configured 7 Assign the I O device of the incorporated PCle card to the logical domain a If you have released the I O device assigned to an I O domain before maintenance restore the physical I O device PCle end point device to its original status For details see 6 4 3 Work to return the physical I O device to the I O domain b Suppose that the virtual function VF using the SR IOV function has been deleted or discarded before maintenance Based on the retained output result of the idm list io
60. the chassis 225 Chapter17 Maintaining the Fan Backplane 227 17 1 Before Maintaining the Fan Backplane 227 17 2 Location of the Fan Backplane 227 17 3 Removing the Fan Backplane 229 17 3 1 Accessing the fan backplane 229 17 3 2 Removing the fan backplane 229 17 4 Installing the Fan Backplane 232 17 4 1 Installing the fan backplane 232 17 4 2 Restoring the chassis 235 Appendix A Component List 237 viii PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Appendix B Component Specifications 239 B 1 B 2 B 3 B 4 B 5 B 6 B 7 PCI Express Card 239 Link Board 240 Power Supply Unit 240 Fan Unit 241 PCI Tray 241 I O Board 242 Fan Backplane 242 Appendix C Problems during Firmware Update 245 Index 247 Contents ix x PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Preface This document describes the maintenance procedure for the PCI expansion unit for Oracle or Fujitsu M10 SPARC M10 Systems The maintenance work should be performed by service engineers and or field engineers Fujitsu M10 is sold as SPARC M10 Systems by Fujitsu in Japan Fujitsu M10 and SPARC M10 Systems are identical products The preface includes the following sections Audience Related Documentation Text Conventions Notes on Safety Syntax of the Command Line Interface CLI Document Feedback Audience This document is intended for service engineers and field engineers who pe
61. without the crossbar box The master chassis is the chassis where the MASTER LED green is on on the rear of the chassis of BB ID 00 and 01 For the SPARC M10 45 with the crossbar box The master chassis is the chassis where the MASTER LED green is on on the rear of the chassis of BB ID 80 and 81 2 Connect the USB memory to the USB port of the server Note Connect the USB memory to the USB port shown as MAINTENANCE ONLY 3 Execute the getflashimage command to import the downloaded firmware into the system Specify the downloaded firmware file tar gz In the following example the PCIBOX1100 tar gz file in the images folder of the USB memory is specified XSCF gt getflashimage file media usb_msd images PCIBOX1100 tar gz PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 If the procedure ends successfully Download successful will be displayed first and then MDS will be displayed e Execute the getflashimage command and confirm that the PCI expansion unit firmware has been imported XSCF gt getflashimage 1 Updating the firmware Note Perform this task with a user account that has the fieldeng privilege 1 Execute the ioxadm command to check the version of the firmware that is currently running XSCF gt ioxadm v list For details see 6 6 1 How to check the serial number and firmware version of the PCI expansion unit N Execute the ioxadm
62. work Chapter 11 Maintaining the Link Cables 197 198 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 12 Maintaining the Management Cable This chapter describes the maintenance procedure for the management cable Before Maintaining a Management Cable Configuration of the Management Cable Connection Port Removing the Management Cable Installing the Management Cable 12 1 Before Maintaining a Management Cable This chapter describes only the configuration removal work and mounting work of a management cable Before removal work of an FRU perform necessary work items while referencing Chapter 7 Maintenance Flow For maintenance types of the management cable see Management cable in Table 7 2 List of available maintenance types by FRU 12 2 Configuration of the Management Cable Connection Port This section describes the configuration and location of the management cable connection port The link cards of the SPARC M10 1 or SPARC M10 4 M10 4S are connected to the link board of the PCI expansion unit with the link cables and the management cable 199 Figure 12 1 Location of management cable connection port Location number Component 1 Management cable MGCBL 12 3 Removing the Management Cable This section describes the procedure for removing the management cable Enable removal of the management cable before attempting to remove it
63. 0 Suppose that the virtual function VF using the SR IOV function has been deleted or discarded before maintenance Based on the retained output result of the idm list io command re perform the virtual function creation and assignment to the domain For details see 6 4 4 Work to restore the SR IOV function 11 Ifa virtual device vnet vdisk of a guest domain has been kept unused before maintenance return it to its original status For details see 6 4 5 Work to restore a virtual device 12 If you have enabled the hotplug service execute the svcadm command to disable the hotplug service svcadm disable hotplug 6 3 2 Mounting the link card on the server using physical partition dynamic reconfiguration DR This section describes the procedure for mounting the link card on the server using physical partition dynamic reconfiguration DR For details see 6 5 1 Incorporating a chassis into a physical partition in the Fujitsu 82 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Note For Oracle VM Server for SPARC 3 2 or later to mount the link card on the server pull all the PCIe cassettes containing PCIe cards out of the PCI expansion unit in advance Mount the link card on the server before installing the pulled out PCle cassettes in the PCI expansion unit Then use PHP for the PCIe cards in the PCI expansion unit to incorporat
64. 1 Checking the firmware version of the PCI expansion unit and updating the firmware as required 12 Returning the mode switch on the operation panel to Locked mode 13 Powering On the Physical Partition Requiring Maintenance 14 Closing the rack door End 156 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 7 23 Item Task 1 Opening the rack door 2 Switching the mode switch on the 10 11 12 13 14 operation panel of the destination server to Service mode Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the operating condition of the system and the I O device usage status Powering Off the Physical Partition Requiring Maintenance Enabling access to the FRU requiring maintenance Installing the link card 1 Restoring the System Performing a diagnosis test Confirming that there is no problem with the installed FRU Checking the firmware version of the PCI expansion unit and updating the firmware as required Returning the mode switch on the operation panel to Locked mode Powering On the Physical Partition Requiring Maintenance Closing the rack door Inactive addition work procedure link card installation with the physical partition stopped Reference 5 2 Switching the Mode Switch to Service Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual
65. 1 pci 0 pci L1 pci 0 pci L1 pci 0 pci LO pci o LO pci 0 pci LO pci 0 pci LO pci 0 pci LO pci 0 pci LO pci 0 pci LO pci 0 pci LO pci 0 pci 100 channel devices 200 virtual domain service 0 100 channel devices 200 virtual channel 3 100 channel devices 200 virtual channel client 2 100 channel devices 200 virtual channel client l 100 channel devices 200 virtual channel 0 Execute the prtdiag 1M command of Oracle Solaris to determine the system configuration and any failed FRUs PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Qe oe Qa Q 1 Display the Oracle Solaris super user prompt 2 Execute the prtdiag 1 command to determine the system configuration and any failed FRUs prtdiag v System Configuration Oracle Corporation sun4v SPARC M10 4 Memory size 64000 Megabytes Status L SASHBA scsi pciex1000 87 LSI 2308 2 pci 8000 pci 4 pci 0 pci 0 scsi 0 L N network pciexl4e4 1656 pci 8000 pci 4 pci 0 pci 9 networked L N network pciexl4e4 1656 pci 8000 pci 4 pci 0 pci 9 network 0 1 network pciexl4e4 1656 pci 8000 pci 4 pci 0 pci a networke d network pciexl4e4 1656 pci 8000 pci 4 pci 0 pci a network 0 1 2081203001 Checking log information Execute the showlogs command to check the error log information 1 Log in to the XSCF shell 2 Execute the showlogs command to determ
66. 10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 4 Diagnosing a Replacement FRU in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 7 Checking the FRU Status after Maintenance in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 6 Updating the Firmware of the PCI Expansion Unit 6 6 Returning the Mode Switch to Locked Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 8 Returning the Mode Switch to Locked Mode in the Fujitsu M10 1 SPARC M10 1 Service Manual Chapter 7 Maintenance Flow 163 Table 7 25 Cold replacement work procedure replacing the link card after stopping the system continued Item Task Reference 14 Starting the system 6 8 Starting the System in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 9 Starting the System in the Fujitsu M10 1 SPARC M10 1 Service Manual 15 Closing the rack door 164 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Cold addition installing the link card after stopping the system Stop the system in which the link card is to be installed and then mount the link card Chapter 7 Maintenance Flow 165 Figure 7 24 Cold addition flow installing the link card after stopping the system 1 Opening the rack door 2 Switching the mode switch on
67. 10 4 M10 4S connected 242 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table B 7 Fan backplane specifications continued Item Description Inactive hot maintenance Unsupported Inactive cold maintenance Supported System stopped maintenance Supported 1 For the maintenance procedure see Chapter 17 Maintaining the Fan Backplane Appendix B Component Specifications 243 244 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Appendix C Problems during Firmware Update This section explains the problems that may occur during firmware update The following errors may occur A hang up occurs in the XSCF while the firmware is being updated Action Update the firmware again A hang up occurs in the PCI expansion unit while the firmware is being updated Action Either the link card or the I O board has a failure Replace the failed part Anerror message related to a write failure FROM write error appears If an error occurs during firmware update operation continues using the previous version of the firmware Action Update the firmware again The firmware update may be completed successfully If an error occurs again during the firmware update the I O board has a failure Replace the failed part 245 246 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Index C component list 237 component name and location
68. 2 Checking the assignment status of I O devices 5 5 1 Releasing the link card or PCIe card using PCI hot plug PHP 5 7 1 Lowering the cable support Chapter 8 Maintaining the PCI Express Cards 6 1 3 Fixing the cable support 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S5 SPARC M10 4 SPARC M10 4S Service Manual 6 7 Checking the FRU Status after Maintenance in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 5 Turning off the CHECK LED Locator of the PCI Expansion Unit This section describes the cold removal workflow References to detailed descriptions are written in the work procedure tables See any of them as required Cold removal work varies depending on the method of releasing the link card mounted on the destination server The following are the work patterns Cold removal releasing the link card after stopping the system Cold removal releasing the link card using PHP Cold removal releasing the link card using DR Cold removal releasing the link card after stopping the system Cold removal of a PCI Express card from the PCI expansion unit can be performed after stopping the system to which the PCI expansion unit is connected This can be done when the connection destination of the PCI expansion unit is a SPARC M10 1 M10 4 M10 4S PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 7 11 Cold removal flow releasi
69. 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 8 Returning the Mode Switch to Locked Mode in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 8 Starting the Entire System in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 9 Starting the System in the Fujitsu M10 1 SPARC M10 1 Service Manual PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Cold removal releasing the link card using PHP Cold removal of a PCI Express card from the PCI expansion unit can be performed after releasing the link card mounted on the destination server from the system using the PCI hot plug PHP This can be done only when the connection destination of the PCI expansion unit is a SPARC M10 4 M10 4S and the setting for the direct I O function is disabled Figure 7 12 Cold removal flow releasing the link card using PHP 1 Opening the rack door 2 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 3 Checking the operating condition of the system and the I O device usage status 4 Releasing the link card mounted on the server from the system using PCI hot plug PHP 5 Lowering the cable support 6 Removing the power cords of the PCI expansion unit 7 Removing the PCle card 8 Mounting the released link card on the system using PCI hot plug PHP 9 Fixing the cable support 10 Confirming that there is no pr
70. 5 lists the specifications of the PCI tray Table B 5 Item Number of PCI trays Location Active hot maintenance Active cold maintenance Inactive hot maintenance PCI tray specifications Description 1 Inside the chassis Unsupported Supported with SPARC M10 4 M10 45S connected Unsupported Appendix B Component Specifications 241 Table B 5 PCI tray specifications continued Item Description Inactive cold maintenance Supported System stopped maintenance Supported For the maintenance procedure see Chapter 15 Maintaining the PCI Tray B 6 I O Board Eleven PCle slots and one slot for the link board are provided on the I O board Table B 6 lists the specifications of the I O board Table B 6 1 O board specifications Item Description Number of I O boards 1 Location Rear of chassis Active hot maintenance Unsupported Active cold maintenance Supported with SPARC M10 4 M10 45 connected Inactive hot maintenance Unsupported Inactive cold maintenance Supported System stopped maintenance Supported For the maintenance procedure see Chapter 16 Maintaining the I O Board B 7 Fan Backplane The fan backplane connects the three fan units Table B 7 lists the specifications of the fan backplane Table B 7 Fan backplane specifications Item Description Number of fan backplanes 1 Location Inside the chassis Active hot maintenance Unsupported Active cold maintenance Supported with SPARC M
71. 6 4 Diagnosing a Replacement FRU in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 7 Checking the FRU Status after Maintenance in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 6 Updating the Firmware of the PCI Expansion Unit 6 6 Returning the Mode Switch to Locked Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 8 Returning the Mode Switch to Locked Mode in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 8 Starting the System in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 9 Starting the System in the Fujitsu M10 1 SPARC M10 1 Service Manual Chapter 7 Maintenance Flow 167 Table 7 26 Cold addition work procedure installing the link card after stopping the system continued Item Task Reference 14 Closing the rack door 1 For details on the work for installing a PCI expansion unit see the Fujitsu M10 SPARC M10 Systems Installation Guide Cold removal removing the link card after stopping the system Stop the system from which the link card is to be removed and then mount the link card 168 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 7 25 Cold removal flow removing the link card after stopping the system 1 Opening the rack door 2 Switching the mode switch on the o
72. 9 Item 1 2 10 11 12 13 14 15 120 Task Opening the rack door Switching the mode switch on the operation panel of the destination server to Service mode Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the operating condition of the system and the I O device usage status Stopping the system Lowering the cable support Removing the power cords of the PCI expansion unit Installing the target FRU Installing the power cords of the PCI expansion unit Fixing the cable support Diagnosing the installed FRU Confirming that there is no problem with the installed FRU Returning the mode switch on the operation panel to Locked mode Starting the system Closing the rack door Cold addition work procedure releasing the link card after stopping the system Reference 5 2 Switching the Mode Switch to Service Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual or Fujitsu M10 1 SPARC M10 1 Service Manual 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 4 1 Checking the operating condition of the physical partition or logical domain 5 5 3 Releasing the link card or PCIe card after stopping the system 5 7 1 Lowering the cable support 5 7 2 Removing the power cord Chapter 8 Maintaining the PCI Express Cards 6 1 2 Installing the p
73. 988 PCIBOX 2007 PSU 1 PSU FEJD1201000738 CA01022 0750 D 7060988 PCIBOX 2007 IOB IOBOARD 1080 PP12470297 CA20365 B66X 010AJ 7061033 PCIBOX 2007 LINKBD BOARD PP1244027P CA20365 B60X 001AA 7061035 PCIBOX 2007 FANBP FANBP PP12470298 CA20365 B68X 005AD 7061025 BB 00 PCI 10 CARD 1080 PP124401L2 CA20365 B59X 001AA 7061040 A firmware version is displayed as a four digit release number in xyyz format Each release number is indicated in xyyz format as follows x Major release number yy Minor release number Chapter5 Preparations Required for Maintenance 53 z Micro release number 5 4 5 4 1 Checking the Operating Condition and Resource Usage Status This section describes how to check the operating condition of the logical domain and the resource usage status before maintenance is performed on each FRU Checking the operating condition of the physical partition or logical domain In the following execution example the operating condition of the physical partition and logical domain is checked from firmware 1 Log in to the XSCF shell 2 Execute the showpparstatus command to confirm the operating condition of the physical partition In the following example PPAR Status of PPAR ID 00 displays Running which indicates that the physical partition is operating XSCF gt showpparstatus p 0 PPAR ID PPAR Status 00 Running 3 Execute the showdomainstatus command to confirm the operating condition
74. ARC M10 Systems Service Manual March 2015 Figure 16 5 I O board stoppers 2 Tighten the two screws securing the I O board Chapter 16 Maintaining the I O Board 223 Figure 16 6 I O board screws 3 Connect all the six cables to the I O board Note Connect the cables in their original positions by referring to the record that you made before the start of maintenance 224 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 16 7 I O board cables 16 4 2 Restoring the chassis I Install the PCI tray For details see 15 4 2 Installing the PCI tray Install all the fan units For details see 14 4 1 Installing a fan unit Install the front cover For details see 6 1 1 Installing the front cover Install all the components listed below Power supply unit For details see 13 4 Installing a Power Supply Unit Chapter 16 Maintaining the I O Board 225 PCle card cassette For details see 8 4 2 Installing a PCI Express card cassette Now the work for mounting the FRU is complete While referencing Chapter 7 Maintenance Flow continue the maintenance work 226 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 17 Maintaining the Fan Backplane This chapter describes the procedure for maintaining the fan backplane mounted in the PCI expansion unit Before Maintaining the Fan Backplane Location of the Fan
75. B60X 001AA 7061035 On PCIBOX 2007 FANBP FANBP PP12470298 CA20365 B68X 005AD 7061025 On BB 00 PCL 10 CARD 1080 PP124401L2 CA20365 B59X 001AA 7061040 On A firmware version is displayed as a four digit release number in xyyz format Each release number is indicated in xyyz format as follows x Major release number yy Minor release number z Micro release number Check method when the system has been started 1 Log in to the XSCF shell in the master chassis For the SPARC M10 1 M10 4 The chassis is always a master chassis because this is a single chassis configuration For the SPARC M10 4S without the crossbar box The master chassis is the chassis where the MASTER LED green is on on the rear of the chassis of BB ID 00 and 01 For the SPARC M10 45 with the crossbar box The master chassis is the chassis where the MASTER LED green is on on the rear of the chassis of BB ID 80 and 81 2 If multiple PCI expansion units are connected execute the ioxadm list command to check the combination of the link card and the I O board XSCF gt ioxadm list PCIBOX Link PCIBOX 1007 BB OO PCI 10 PCIBOX 1006 BB O0 PCI 09 PCIBOX 1005 BB 00 PCI 08 PCIBOX 1004 BB O0O PCI 07 3 Execute the ioxadm v list command to check the serial number and the firmware version of the PCI expansion unit Location displays the last four digits of the serial number of the PCI expansion unit and FW Ver displays the firmware ve
76. Backplane Removing the Fan Backplane Installing the Fan Backplane 17 1 Before Maintaining the Fan Backplane This chapter describes only the configuration removal work and mounting work of the fan backplane Before removal work of an FRU perform necessary work items while referencing Chapter 7 Maintenance Flow For maintenance types of the fan backplane see Fan backplane in Table 7 2 List of available maintenance types by FRU 17 2 Location of the Fan Backplane This section describes the location of the fan backplane The fan backplane connects the individual fan units 227 Caution Do not replace a fan backplane together with an I O board If you replace a fan backplane together with an I O board the PCI expansion unit serial number changes to all 0 s 0000000000 If the serial number changes to all 0 s 0000000000 you need to restore the original serial number by using the ioxadm command of the XSCF firmware To replace both a fan backplane and an I O board replace either of the two start the physical partition connected to the PCI expansion unit and check for error log information related to the PCI expansion unit After confirming that no such error log information is displayed replace the other Field Replaceable Unit FRU Figure 17 1 Location of the fan backplane 1 Location number Component 1 Fan backplane FANBP 228 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Servi
77. CI Express Cards in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 1 Restoring the Chassis in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 2 1 Incorporation of the SPARC M10 4S chassis possible only in a system with a building block configuration in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 1 Diagnosing the system board in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 2 Mounting the link card on the server using physical partition dynamic reconfiguration DR 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Chapter 7 Maintenance Flow 141 Active addition installing the link card using PHP Installing a link card to the server using PCI hot plug PHP This can be done only when the server on which the link card is to be installed is a SPARC M10 4 M10 4S and the setting for the direct I O function is disabled Note Active addition of the link card using PHP is supported by Oracle Solaris 11 2 SRU11 2 2 8 0 or later Figure 7 16 Active addition flow installing the link card using PHP 1 Opening the rack door 2 Checking the operating condition of the system and the I O device usage status 3 Lowering the cable support 4 Installing the link card 5 Fixing the cable support 6 Mounting the installed
78. Cle card 1 The link board can be mounted only in its dedicated slot Connecting the link board to the link card Connect the link board mounted in the PCI expansion unit to the link card mounted in the SPARC M10 1 M10 4 M10 45 with the link cables and the management cable Figure 2 3 shows an example of the connection of the link board and the link card Figure 2 3 Example of connection PCI expansion unit r Link board Lt Mounted as standard PCI Express slot PCI Express slot Link cable PCI Express slot PCI Express slot Link card PCI Express slot PCI Express slot Management cable PCI Express slot PCI Express slot PCI Express slot PCI Express slot PCI Express slot PCI Express slot Internal components Note To access internal components remove the PCI tray from the PCI expansion unit For details on the procedure for removing the PCI tray see 15 3 Removing the PCI Tray Chapter 2 Understanding the PCI Expansion Unit Components 19 Figure 2 4 Locations of internal components Location Component number 1 PCI tray 2 T O board 3 PSU backplane 4 Fan backplane 2 2 Checking the Status with the LED This section describes how to identify the state of each component from its LED indication LEDs are mounted on the front panel of the PCI expansion unit and on each component that can be maintained If an error occurs check the LEDs to see which component requires maintenance 20 PCI Expansion
79. ELATIVE A LA QUALITE MARCHANDE AL APTITUDE A UNE UTILISATION PARTICULIERE OU A L ABSENCE DE CONTREFACON Contents Preface xi Chapter1 Before Starting Maintenance Work 1 1 1 Warning Caution Indications 1 1 2 Labels Tags 2 1 3 Safety Precautions 3 1 4 Notes Regarding Static Electricity 4 1 5 Other Precautions 5 1 6 Emergency Power Off 6 1 7 Important Information about the XCP Firmware 6 1 7 1 Precautions for updating the XCP firmware 7 1 7 2 Notes on using the direct I O function 7 1 7 3 How to save restore the logical domain configuration information and the OpenBoot PROM environment variable 9 Chapter 2 Understanding the PCI Expansion Unit Components 17 2 1 Identifying the Names and Locations of Components 17 2 2 Checking the Status with the LED 20 2 2 1 Front LEDs on the PCI expansion unit 21 2 2 2 LEDs on rear of PCI expansion unit 22 2 2 3 LEDs on link card 24 Chapter3 Types of Maintenance 27 3 1 Maintenance Types of the PCI Expansion Unit 27 3 2 Types of Maintenance for the Link Card 30 3 3 Maintenance Types of the PCIe Card 31 3 3 1 Understanding the maintenance types of the PCIe card 31 3 3 2 How to check whether PCI hot plug PHP canbe used 33 3 3 3 Checking whether the SR IOV function is used 35 Chapter 4 Preparation and Precautions for Maintenance 37 4 1 Troubleshooting 37 4 1 1 Determining the causes of individual failures 37 4 1 2 Identifying a failure 38 4 1 3 Locating the PCI expansion unit requiring maintenance 44 4 2 Pr
80. Expansion Unit 6 6 Returning the Mode Switch to Locked Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 8 Returning the Mode Switch to Locked Mode in the Fujitsu M10 1 SPARC M10 1 Service Manual PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 7 5 Cold replacement work procedure Releasing the link card after stopping the system continued Item Task Reference 14 Starting the system 6 8 Starting the Entire System in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 9 Starting the System in the Fujitsu M10 1 SPARC M10 1 Service Manual 15 Closing the rack door Cold replacement releasing the link card using PHP Cold replacement can be performed for all the FRUs mounted on the PCI expansion unit after releasing the link card mounted on the destination server from the system using the PCI hot plug PHP This can be done only when the connection destination of the PCI expansion unit is a SPARC M10 4 M10 4S and the setting for the direct I O function is disabled Chapter 7 Maintenance Flow 111 Figure 7 4 Cold replacement flow releasing the link card using PHP 1 Opening the rack door 2 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 3 Checking the firmware version of the PCI expansion unit if you are replacing an I O board 4 Checking the operating co
81. Flow continue the maintenance work Chapter 12 Maintaining the Management Cable 201 202 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 13 Maintaining the Power Supply Units This chapter describes the procedure for maintaining power supply units mounted in a PCI expansion unit Before Maintaining Power Supply Units Configuration of the Power Supply Units Removing a Power Supply Unit Installing a Power Supply Unit 13 1 Before Maintaining Power Supply Units This chapter describes only the configuration removal work and mounting work of power supply units Before removal work of an FRU perform necessary work items while referencing Chapter 7 Maintenance Flow For maintenance types of the power supply unit see Power supply unit in Table 7 2 List of available maintenance types by FRU 13 2 Configuration of the Power Supply Units This section describes the configuration and the locations of the power supply units The power supply units supply power to each component To provide redundancy two power supply units are mounted in the PCI expansion unit If one power supply unit fails while the system is running the 1 1 redundant configuration allows the system to continue operating 203 Figure 13 1 Location of the power supply units Location number Component 1 Power supply unit PSU 0 2 Power supply unit PSU 1 13 3 Removi
82. I expansion unit The system administrator and service engineer can manage the PCI expansion unit by executing the ioxadm command OpenBoot PROM show devs Displays device tree information on the PCIe card and other devices connected to the host Execute this command from the ok prompt Oracle Solaris prtdiag Displays the system configuration and all Field Replaceable Units FRUs that have experienced failures Execute this command from the Oracle Solaris super user prompt FRU refers to any component that can be replaced by a field engineer Chapter 4 Preparation and Precautions for Maintenance 39 The command execution examples shown here are based on the assumption that PCle cards are inserted into all of the slots ioxadm command Execute the ioxadm command to determine the environmental conditions temperature voltage etc or LED indications of the PCI expansion unit 1 Login to the XSCF shell 2 Execute the ioxadm command to check the environmental conditions of the specified PCI expansion unit The following example shows the environmental conditions for the PCIBOX 2008 2008 is the last four digits of the serial number of the PCI expansion unit XSCF gt ioxadm env te PCIBOX 2008 Location Sensor Value Resolution PCIBOX 2008 AIRFLOW 180 000 0 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 PCIBOX 2008 P_CONSUMPTION 68 000 PCIBOX 2008 PSU 0 F 3936 000 PCIB
83. Logical Domain Name Status primary OpenBoot Running 18 Execute the console command to switch to the control domain console XSCF gt console p 0 y Console contents may be logged Connect to PPAR ID 0 f yln 3y 19 Based on the record in step 11 restore the OpenBoot PROM environment variable In the following example auto boot is restored to true 0 ok setenv auto boot true auto boot true 0 ok printenv auto boot auto boot true 20 Boot Oracle Solaris 0 ok boot 21 With the Oracle Solaris super user prompt confirm that the system has PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 started with the factory default ldm 1ls spconfig factory default current 22 Execute the Idm init system command and the shutdown command to restart the control domain ldm init system i var tmp test1 xml Initiating a delayed reconfiguration operation on the primary domain All configuration changes for other domains are disabled until the primary domain reboots at which time the new configuration for the primary domain will also take effect shutdown y g0 i6 23 After restarting the control domain bind and start other logical domains For logical domains with dependency start them in the correct order In the following example both root domain and guest domain are bound and started ldm bind root domain ldm start domain root domain ldm bind guest doma
84. OX 2008 PSU 1 F 3584 000 PCIBOX 2008 FAN 0 F 3374 000 PCIBOX 2008 FAN 1 F 3374 000 PCIBOX 2008 FAN 2 F 3374 000 PCIBOX 2008 I B T_INTAKE 26 000 T_PART_NO0 31 500 T_PART_NO1 30 750 T_PART_NO2 31 500 12 069 2293 e295 291 300 804 900 PCIBOX 2008 I PCIBOX 2008 I PCIBOX 2008 I PCIBOX 2008 I PCIBOX 2008 I PCIBOX 2008 I PCIBOX 2008 I PCIBOX 2008 I PCIBOX 2008 I PCIBOX 2008 I O oO Oo OO OO 0 OO OC O OC Oo eoooooo0 000 08 U UDDUDD W e O O O O show devs command Execute the show devs command of OpenBoot PROM to display the path from the host server to the PCIe card on the I O board 1 Display the ok prompt 2 Execute the show devs command to check the device tree information 0 ok show devs pci performance counters 8100 pci performance counters 8000 pci 8100 pci 8000 cpu lft cpu le cpu ld cpu lc cpu lb 40 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu cpu la 19 18 17 16 15 14 13 12 11 10 ef aa QO coag RQ Qooaoaeoaoae e ec OrRNWHRUD A AO W virtual devices 100 isc virtual memory si hba memory m7e00 60000000 aliases options openprom cho pac pci pci pci pci pci pci pci pci pci
85. P 2210 or later and Oracle VM Server for SPARC 3 1 or later 2 If static reconfiguration for the virtual function of SR IOV is used the hotplug service operates whether it is enabled or disabled 1 If the hotplug service is disabled for the I O and root domains execute the svcadm enable hotplug command to enable the hotplug service svcadm enable hotplug Chapter 5 Preparations Required for Maintenance 59 2 Stop the I O domain This step is unnecessary if the dynamic SR IOV function is used Perform step 3 ldm stop domain lt I O domain name gt 3 Remove the virtual function VF assigned to the I O domain from the I O domain Execute the Idm remove io command to delete the virtual function VF assigned to the I O domain from the I O domain If multiple VFs are assigned from the physical function PF of a single PCle card delete all the VFs corresponding to that PF from the I O domain ldm remove io lt VF name gt lt I O domain name gt Pe Transition the root domain that has the assigned PF to delayed reconfiguration mode This step is unnecessary if the dynamic SR IOV function is used Perform step 5 ldm start reconf lt root domain name gt 5 Discard the virtual function VF Execute the Idm destroy vf command to discard the virtual function VF However the VF needs to be discarded in the reverse order to that in which it was created ldm destroy vf lt VF name gt 6 Log in to the root domain
86. PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual fo FUJITSU Manual Code C120 E683 10EN ORACLE March 2015 Copyright 2007 2015 Fujitsu Limited All rights reserved Oracle and or its affiliates provided technical input and review on portions of this material Oracle and or its affiliates and Fujitsu Limited each own or control intellectual property rights relating to products and technology described in this document and such products technology and this document are protected by copyright laws patents and other intellectual property laws and international treaties This document and the product and technology to which it pertains are distributed under licenses restricting their use copying distribution and decompilation No part of such product or technology or of this document may be reproduced in any form by any means without prior written authorization of Oracle and or its affiliates and Fujitsu Limited and their applicable licensors if any The furnishings of this document to you does not give you any rights or licenses express or implied with respect to the product or technology to which it pertains and this document does not contain or represent any commitment of any kind on the part of Oracle or Fujitsu Limited or any affiliate of either of them This document and the product and technology described in this document may incorporate third party intellectual property copyrighted by and or licensed from
87. RC 3 1 1 or earlier For Oracle VM Server for SPARC 3 1 1 1 or later you do not need to stop the I O domain If the boot disk is not in a redundant configuration stop the I O domain 1 If the hotplug service is disabled for the I O and root domains execute the svcadm enable hotplug command to enable the hotplug service svcadm enable hotplug 2 Stop the I O domain This step is unnecessary if the dynamic reconfiguration function for PCIe end point devices is used Perform step 3 ldm stop domain lt I O domain name gt 3 Delete the physical I O device from the I O domain ldm remove io lt device name gt lt I O domain name gt Chapter5 Preparations Required for Maintenance 63 4 Transition the root domain to delayed reconfiguration mode This step is unnecessary if the dynamic reconfiguration function for PCIe end point devices is used Perform step 5 ldm start reconf lt root domain name gt 5 Reassign the physical I O device deleted in step 3 to the root domain ldm add io lt device name gt lt root domain name gt 6 Log in to the root domain and restart Oracle Solaris This step is unnecessary if the dynamic reconfiguration function for PCIe end point devices is used Perform step 7 telnet localhost lt port number gt shutdown i6 g0 y 7 Ifyou need to disable the hotplug service for the I O and root domains disable the hotplug service svcadm disable hotplug In t
88. Static Electricity 1 Hold the tabs of the link cables on the PCI expansion unit side and then remove the two cables Note Record the positions of the link cables before removing them to ensure that they are reinstalled correctly 196 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 11 2 Removing the link cable 2 Remove the two link cables from the SPARC M10 1 or SPARC M10 4 M10 4S using the same procedure Note Record the positions of the link cables before removing them to ensure that they are reinstalled correctly 11 4 Installing a Link Cable This section describes the procedure for installing the link cables 1 Hold the link cable connectors and connect the two link cables to the link board mounted in the PCI expansion unit Note Reinstall the link cables in their original positions by referring to the record you made before the start of maintenance Note Confirm that the link cables are connected and firmly secured 2 Connect the two link cables to the link card mounted in the SPARC M10 1 or SPARC M10 4 M10 4S by using the same procedure Note Reinstall the link cables in their original positions by referring to the record that you made before the start of maintenance Note Confirm that the link cables are connected and firmly secured Now the work for mounting the FRU is complete While referencing Chapter 7 Maintenance Flow continue the maintenance
89. Systems Service Manual March 2015 Chapter 10 Maintaining the Link Cards This chapter describes the procedure for maintaining a link card mounted in the PCI expansion unit Link cards can be replaced expanded and removed Before Maintaining a Link Card Configuration of a Link Card Removing the Link Card Installing the Link Card 10 1 Before Maintaining a Link Card This chapter describes only the configuration removal work and mounting work of a link card Before removal work of an FRU perform necessary work items while referencing Chapter 7 Maintenance Flow For maintenance types of the link card see Link card in Table 7 2 List of available maintenance types by FRU 10 2 Configuration of a Link Card This section describes the configuration and location of the link cards Mount a link card in the PCI Express PCIe slot of the SPARC M10 1 or SPARC M10 4 M10 4S and then for each link card connect one PCI expansion unit to add a PCle slot In the case of the SPARC M10 1 up to two link cards can be mounted up to six for the SPARC M10 4 and up to five for the SPARC M10 4S 189 Figure 10 1 Link card mounting position SPARC M10 1 190 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 10 2 Link card mounting position SPARC M10 4 M10 4S SPARC M10 4 SPARC M10 4S Chapter 10 Maintaining the Link Car
90. able hotplug 3 Execute the cfgadm 1 command to mount the link card to which the PCI expansion unit is connected on the logical domain For the Ap_ID enter the Ap_ID of the link card you noted down before the start of maintenance Note For link card maintenance in an environment on Oracle Solaris 11 2 or later the link card is automatically incorporated on the server simply by mounting it on the server Therefore you do not need to perform an operation to incorporate the link card using the Q EN ag rot on 5 Q 3 3 D 2 gt cfgadm c configure Ap ID 4 Ifyou have replaced a link card or I O board match the firmware version with the one before replacement a Check the firmware version of the PCI expansion unit For details see Check method when the system has been started in 6 6 1 How to check the serial number and firmware version of the PCI expansion unit If the firmware version is the same as the one before replacement go to step 5 b If the firmware version is different from the one before replacement match it with the one before replacement See 6 6 4 Updating the firmware for details to perform work up to step 3 c To reflect the firmware release the link card using PHP cfgadm c disconnect Ap ID d Execute the cfgadm 1M command to incorporate the link card into the logical domain cfgadm c configure Ap ID e Confirm that the firmware version is now the same as the one before r
91. after stopping the system Reference 5 2 Switching the Mode Switch to Service Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual or Fujitsu M10 1 SPARC M10 1 Service Manual 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 3 Checking the Firmware Version of the PCI Expansion Unit 5 4 1 Checking the operating condition of the physical partition or logical domain 5 5 3 Releasing the link card or PCIe card after stopping the system 5 7 Accessing an FRU Chapter 8 Maintaining the PCI Express Cards Chapter 9 Maintaining the Link Board Chapter 11 Maintaining the Link Cables Chapter 12 Maintaining the Management Cable Chapter 13 Maintaining the Power Supply Units Chapter 14 Maintaining the Fan Units Chapter 15 Maintaining the PCI Tray Chapter 16 Maintaining the I O Board Chapter 17 Maintaining the Fan Backplane 6 1 Restoring the Chassis of the PCI Expansion Unit 6 3 Diagnosing a Replacement FRU in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 4 Diagnosing a Replacement FR M10 1 SPARC M10 1 Service Manual in the Fujitsu 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S5 SPARC M10 4 SPARC M10 4S Service Manual 6 7 Checking the FRU Status after Maintenance in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 6 Updating the Firmware of the PCI
92. aintenance Replacing a faulty link card Restoring the System Performing a diagnosis test Confirming that there is no problem with the replacement FRU Checking the firmware version and matching it with the pre replacement one Returning the mode switch on the operation panel to Locked mode Powering On the Physical Partition Requiring Maintenance Closing the rack door Inactive replacement work procedure link card replacement with the physical partition stopped Reference 5 2 Switching the Mode Switch to Service Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 3 Checking the Firmware Version of the PCI Expansion Unit 5 4 1 Checking the operating condition of the physical partition or logical domain 5 5 Powering Off the Physical Partition Requiring Maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 9 2 Lowering the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Chapter 8 Maintaining the PCI Express Cards in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 1 2 Fixing the cable support in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 1 Diagnosing the system board in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Servi
93. an error message is displayed on the console When an error is displayed as a result of executing a command for checking the status When an error is displayed in the error log Determining the causes of individual failures This section explains the flow for determining the causes of failures 37 4 1 2 Figure 4 1 Troubleshooting flow Are the power OK YES and AC OK LEDs off NO Was e mail sent by YES the XSCF mail function Confirm that an error message is displayed on the OS and XSCF consoles Execute showlogs on XSCF to display failure information Check var adm messages on Oracle Solaris Write down the displayed failure information Contact our service engineer Identifying a failure Is the power cord connected correctly YES NO Connect the power cord correctly This section explains the method for identifying a failure Use the flow described in 4 1 1 Determining the causes of individual failures to determine the appropriate way of checking for a failure Checking the LED indications Identify the component requiring maintenance by checking the LEDs on both the front and rear panels of the PCI expansion unit and the LEDs on the link card of the server to which the PCI expansion unit is connected To maintain a component determine its status from the LEDs and then start the maintenance work 38 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual
94. and fixed Now the work for mounting the FRU is complete While referencing Chapter 7 Maintenance Flow continue the maintenance work Chapter 13 Maintaining the Power Supply Units 205 206 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 14 Maintaining the Fan Units This chapter describes the procedure for maintaining fan units mounted in a PCI expansion unit Before Maintaining Fan Units Configuration of the Fan Units Removing a Fan Unit Installing a Fan Unit 14 1 Before Maintaining Fan Units This chapter describes only the configuration removal work and mounting work of fan units Before removal work of an FRU perform necessary work items while referencing Chapter 7 Maintenance Flow For maintenance types of the fan unit see Fan unit in Table 7 2 List of available maintenance types by FRU 14 2 Configuration of the Fan Units This section describes the configuration and the locations of the fan units A PCI expansion unit has three fan units for cooling the components in the chassis If one fan unit fails while the system is running the 2 1 redundant configuration allows the system to continue operating 207 Figure 14 1 Fan unit locations Location number Component 1 Fan unit FAN 0 2 Fan unit FAN 1 3 Fan unit FAN 2 14 3 Removing a Fan Unit This section describes the procedure for removing a fan unit Enable the removal
95. art domain 7 Perform 1 Dynamic reconfiguration for root complexes is supported by XCP 2240 or later and Oracle VM Server for SPARC 3 2 or later In addition Oracle Solaris 11 2 SRU11 2 8 or later must be used for the control domain 2 If dynamic reconfiguration for root complexes is not used assign the physical I O device root complex and then start the root domain 1 Reassign the I O device root complex released from the root domain to the root domain ldm add io lt physical I O device name gt lt root domain name gt 2 Start the root domain This step is unnecessary if the dynamic reconfiguration function for root complexes is used ldm start domain lt root domain name gt 6 4 3 Work to return the physical I O device to the I O domain If you released the I O device assigned to an I O domain before maintenance according to 2 Deleting the assignment of the I O of 5 5 1 Releasing the link card or PCIe card using PCI hot plug PHP or 5 5 2 Releasing the link card or PCIe card using physical partition dynamic reconfiguration DR restore the physical I O device PCIe end point device to its original status The PCle slot is automatically assigned to the root domain So to reassign it to the T O domain first release it from the root domain and then assign it Chapter6 Understanding the Preparations for Restoring the System 89 Table 6 3 end point devices is used Item Task 1 Enabling the hotplug service
96. ate For details on the work and the commands used see 3 2 Operations and Commands Related to Logical Domain Configurations in the Fujitsu M10 SPARC M10 Systems Domain Configuration Guide and Oracle VM Server for SPARC Reference Manual for the version being used The following procedure returns a logical domain to the state in the Idm list io output results that were checked in step 4 in 5 4 2 Checking the assignment status of I O devices Returning the root complex to the control domain Return the root complex released from the control domain to the control domain Chapter6 Understanding the Preparations for Restoring the System 87 Table 6 1 Differences in the procedures according to whether the dynamic reconfiguration function for root complexes is used Unnecessary Item Task Command Dynamic Dynamic reconfiguration reconfiguration is used 1 is not used 2 1 Setting the control domain in delayed ldm start reconf Perform reconfiguration mode 2 Assigning the physical I O device to the ldm add io Perform Perform control domain 3 Restarting the control domain shutdown i6 g0 y z Perform 1 Dynamic reconfiguration for root complexes is supported by XCP 2240 or later and Oracle VM Server for SPARC 3 2 or later In addition Oracle Solaris 11 2 SRU11 2 8 or later must be used for the control domain 2 If dynamic reconfiguration for root complexes is not used set the control domain in delayed reconfiguration mode and then as
97. ce Manual 6 7 Checking the FRU Status after Maintenance in the Fujitsu M10 4 Fujitsu M10 45 SPARC M10 4 SPARC M10 4S Service Manual 6 6 Updating the Firmware of the PCI Expansion Unit 6 6 Returning the Mode Switch to Locked Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 7 Powering On the Physical Partition Requiring Maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Inactive addition link card installation with the physical partition stopped Stop the physical partition in which the link card is to be installed and then mount the link card Chapter 7 Maintenance Flow 155 Figure 7 21 Inactive addition flow link card installation with the physical partition stopped 1 Opening the rack door 2 Switching the mode switch on the operation panel of the destination server to Service mode 3 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 4 Checking the operating condition of the system and the I O device usage status 5 Powering Off the Physical Partition Requiring Maintenance 6 Enabling access to the FRU requiring maintenance 7 Installing the link card 8 Restoring the System 9 Performing a diagnosis test i 10 Confirming that there is no problem with the installed FRU 1
98. ce Manual March 2015 17 3 17 3 1 17 3 2 Removing the Fan Backplane This section describes the procedure for removing the fan backplane Enable the removal of the fan backplane before attempting to remove it For details see Chapter 7 Maintenance Flow Caution Before you handle any components wear a wrist strap to ground any static electricity If you perform this procedure without a wrist strap individual components or the overall system may be damaged For details see 1 4 Notes Regarding Static Electricity Accessing the fan backplane 1 Pull the following components out halfway Power supply unit For details see 13 3 Removing a Power Supply Unit PCI Express PCIe card cassette For details see 8 3 2 Removing a PCI Express card cassette 2 Remove the front cover For details see 5 7 3 Removing the front cover 3 Remove all the fan units For details see 14 3 2 Removing a fan unit 4 Remove the PCI tray For details see 15 3 2 Removing the PCI tray Removing the fan backplane 1 Remove the two cables connected to the fan backplane Note Record the positions of the cables before removing them to ensure that they are reinstalled correctly Chapter 17 Maintaining the Fan Backplane 229 Figure 17 2 Fan backplane cable 2 Remove the screw securing the fan backplane 230 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 17 3 Fan backp
99. cking the operating condition of the system and the I O device usage status Lowering the cable support Installing a PCle card Fixing the cable support Incorporating the PCIe card Confirming that there is no problem with the installed FRU Turning off the CHECK LED locator of the PCI expansion unit requiring maintenance Closing the rack door Cold addition Reference 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 4 1 Checking the operating condition of the physical partition or logical domain 5 4 2 Checking the assignment status of I O devices 5 7 1 Lowering the cable support Chapter 8 Maintaining the PCI Express Cards 6 1 3 Fixing the cable support 6 3 3 Mounting a PCIe card on the server using PCI hot plug PHP 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 7 Checking the FRU Status after Maintenance in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 5 Turning off the CHECK LED Locator of the PCI Expansion Unit This section describes the cold addition workflow References to detailed descriptions are written in the work procedure tables See any of them as required Cold addition work varies depending on the method of releasing the link card mounted on the destination server The following are the work patterns Cold addition releasing the link card afte
100. command re perform the virtual function creation and assignment to the domain For details see 6 4 4 Work to restore the SR IOV function c If a virtual device vnet vdisk of a guest domain has been kept unused before maintenance return it to its original status 86 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 For details see 6 4 5 Work to restore a virtual device 8 If you have added a new PCI Express card see the following manual to perform appropriate operations In the case that dynamic reconfiguration for PCIe end point devices is used 15 3 1 Adding a physical I O device to an I O domain in the Fujitsu M10 SPARC M10 Systems System Operation and Administration Guide Note Dynamic reconfiguration for PCle end point devices is supported by XCP 2230 or later and Oracle VM Server for SPARC 3 1 1 1 or later In the case that dynamic reconfiguration for PCIe end point devices is not used 3 2 15 Configuring an I O device and 3 2 16 Creating or destroying the SR IOV virtual function in the Fujitsu M10 SPARC M10 Systems Domain Configuration Guide 9 If you have enabled the hotplug service execute the svcadm command to disable the hotplug service svcadm disable hotplug 6 4 6 4 1 Restoring the Logical Domain to the Status It Had before Maintenance This section describes the flow of work in restoring a logical domain after FRU maintenance to the pre maintenance st
101. command to update the firmware In the following example the PCIBOX 2007 firmware is updated to the 1100 version XSCF gt ioxadm c update PCIBOX 2007 s 1100 Firmware update is started version 1100 Firmware update has been completed After Firmware update is started is displayed nothing is displayed for about 30 minutes before Firmware update has been completed is displayed 3 Execute the ioxadm command to check the firmware version Confirm that the firmware version of the updated firmware is correct XSCF gt ioxadm v list For details see 6 6 1 How to check the serial number and firmware version of the PCI expansion unit 4 Toreflect the firmware you need to release and reconnect the link card Methods a b and c below are available for releasing the link card a Releasing and reconnecting the link card after stopping the system Stop the system after restoring the original Auto boot setting of the target physical partition Also after stopping the system return the mode switch on the operation panel to Locked mode The firmware will be reflected at the next startup Chapter6 Understanding the Preparations for Restoring the System 99 0 ok setenv auto boot true XSCF gt poweroff p 0 b Releasing the link card using PHP For details see 5 5 1 Releasing the link card or PCIe card using PCI hot plug PHP and 6 3 1 Mounting a link card using PCI hot plug PHP c Releasing the link card usi
102. domain with an appropriate procedure and use the Idm unbind command to transfer it to the inactive state shutdown i0 g0 y 11 Check the OpenBoot PROM environment variable with the printenv command 0 ok printenv Variable Name Value Default Value ttya rts dtr off false false ttya ignore cd true true keyboard layout reboot command security mode none No default security password No default Chapter 1 Before Starting Maintenance Work 11 security badlogins 0 o default diag switch false false local mac address true true fcode debug false false scsi initiator id 7 7 oem logo o default oem logo false false oem banner o default oem banner false false ansi terminal true true screen columns 80 80 screen rows 34 34 ttya mode SEOL PE oe N a 9600 85 n L output device input device virtual console virtual console virtual console virtual console auto boot on error false false load base 16384 16384 auto boot false true network boot arguments boot command boot boot boot file boot device pci 8000 pci 4 pci 0 pe disk net multipath boot false false boot device index 0 0 use nvramrc false false nvramrc error reset recovery boot boot If there is any omitted section with review this section 0 ok printenv boot device pci 8000 pci 4 pci 0 pci 0 scsi 0 disk p0 0 boot device 12 Execute the poweroff command of the XSCF firmware to turn off the power to the phy
103. ds 191 10 3 A Removing the Link Card This section describes the preparations that must be completed prior to removing the link card Caution Before you handle any components wear a wrist strap to ground any static electricity If you perform this procedure without a wrist strap individual components or the overall system may be damaged For details see 1 4 Notes Regarding Static Electricity 1 Remove the link card requiring maintenance from the server The removal procedure for the link card is the same as that for a PCIe card of each server For details on the SPARC M10 1 see 8 3 Removing a PCI Express Card in the Fujitsu M10 1 SPARC M10 1 Service Manual For details on the SPARC M10 4 M10 4S see 8 4 Removing a PCI Express Card in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 10 4 Installing the Link Card This section describes the procedure for installing the link card Caution Before you handle any components wear a wrist strap to ground any static electricity If you perform this procedure without a wrist strap individual components or the overall system may be damaged For details see 1 4 Notes Regarding Static Electricity 1 If you use PCI hot plug PHP to maintain the link card pull all the PCle cassettes containing PCle cards out of the PCI expansion unit a Raise the lever of the PCIe card cassette while pushing the latch For details see Figure 8 3 b
104. e them into server Log in to the XSCF shell Execute the addboard command with the c configure option specified to incorporate the target chassis into the physical partition mow Enter y in response to the confirmation message The following example mounts PSB 00 0 on PPAR ID N XSCF gt addboard c configure p 0 00 0 x Execute the showresult command to check the end status of the addboard command that has just been executed The following example shows that 0 is returned as the end status and the execution of the addboard command has been completed successfully XSCF gt showresult ca Execute the showboards command to confirm that the chassis requiring maintenance has been incorporated into the physical partition The BB ID of the server requiring maintenance is indicated by the system board PSB number XSCF gt showboards a a Execute the showstatus command to confirm that the FRU is operating normally after the completion of maintenance Nothing is displayed if it is normal XSCF gt showstatus Ss Execute the showhardconf command to check the hardware configuration and the status of each FRU Confirm that no asterisk is displayed in front of any FRU XSCF gt showhardconf 7 If the state of the master chassis requiring maintenance has been switched to the standby state before maintenance execute the switchscf command to Chapter6 Understanding the Preparations for Restoring the Sy
105. e and the Destination Physical Partition 5 4 Checking the Operating Condition and Resource Usage Status 5 5 2 Releasing the link card or PCIe card using physical partition dynamic reconfiguration DR 5 7 1 Lowering the cable support 5 7 2 Removing the power cord Chapter 8 Maintaining the PCI Express Cards 6 1 2 Installing the power cords 6 1 3 Fixing the cable support 6 3 2 Mounting the link card on the server using physical partition dynamic reconfiguration DR 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual FRU Removal Workflow for the PCI Expansion Unit This section describes the removal workflow for the following FRU of the PCI expansion unit PCI Express card Hot removal This section describes the hot removal workflow References to detailed descriptions are written in the work procedure tables See any of them as required Chapter 7 Maintenance Flow 125 The following patterns are available for hot removal Hot removal for a PCI Express card 126 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Hot removal for a PCI Express card Hot removal is applicable to the PCI Express card Perform removal according to the following procedure Figure 7 10 Hot removal flow for a PCI Express card 1 Opening the rack door 2 Locating the PCI expansion unit requir
106. e of the update procedure for the firmware See the relevant sections Table 6 6 Flow of update process Task 1 Preparing the firmware 2 Importing the firmware into the system 3 Updating the firmware Updating the firmware This section describes the procedure for updating the firmware Preparing the firmware 1 Download the firmware file from the following website Chapter6 Understanding the Preparations for Restoring the System 97 98 Download the firmware file to an arbitrary folder on a PC or workstation that is connected to the system Note The information on these sites is for customers of Fujitsu Japanese site Customers who subscribe to SupportDesk can download the firmware file from the SupportDesk Web Global site For the latest firmware file contact your sales representative 2 Check the downloaded firmware version number For the firmware version refer to the four digit number of the firmware file tar gz Importing the firmware into the system This section describes how to import the firmware with the CLI using the USB memory When using the XSCF Web the procedure is the same as that for updating the XSCF Note Perform this task with a user account that has the platadm or fieldeng privilege 1 Log in to the XSCF shell in the master chassis For the SPARC M10 1 M10 4 The chassis is always a master chassis because this is a single chassis configuration For the SPARC M10 45S
107. ecautions for Maintenance 44 4 2 1 Precautions for replacement 44 4 2 2 Precautions for installation 48 4 2 3 Precautions for removal 49 Chapter5 Preparations Required for Maintenance 51 5 1 Preparing Tools Required for Maintenance 51 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 52 5 3 Checking the Firmware Version of the PCI Expansion Unit 53 5 4 Checking the Operating Condition and Resource Usage Status 54 5 4 1 Checking the operating condition of the physical partition or logical domain 54 5 4 2 Checking the assignment status of I O devices 55 5 4 3 Checking the usage status of the internal disk 55 5 5 Releasing the Link Card or PCIe Card from the System 56 5 5 1 Releasing the link card or PCIe card using PCI hot plug PHP 56 5 5 2 Releasing the link card or PCIe card using physical partition dynamic reconfiguration DR 62 5 5 3 Releasing the link card or PCIe card after stopping the system 68 iv PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 5 6 Stopping the FRU Requiring Maintenance 69 5 7 Accessing an FRU 69 5 7 1 5 7 2 5 7 3 Lowering the cable support 69 Removing the power cord 72 Removing the front cover 73 Chapter6 Understanding the Preparations for Restoring the System 75 6 1 Restoring the Chassis of the PCI Expansion Unit 75 6 1 1 6 1 2 6 1 3 Installing the front cover 75 Installing the power cords 76 Fixing the cable suppor
108. ed Stop the physical partition from which the link card is to be removed and then remove the link card 158 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 7 22 Inactive removal flow link card removal with the physical partition stopped 1 Opening the rack door 2 Switching the mode switch on the operation panel of the destination server to Service mode 3 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 4 Checking the operating condition of the system and the I O device usage status 5 Powering Off the Physical Partition Requiring Maintenance 6 Enabling access to the FRU requiring maintenance 7 Removing the link card 8 Restoring the System i 9 Performing a diagnosis test 10 Confirming that there is no problem with the replacement FRU 11 Returning the mode switch on the operation panel to Locked mode 12 Powering On the Physical Partition Requiring Maintenance 13 Closing the rack door End Chapter 7 Maintenance Flow 159 Table 7 24 Item 1 2 10 11 12 13 7 4 3 160 Task Opening the rack door Switching the mode switch on the operation panel of the destination server to Service mode Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the operating condition of the system and the I
109. ed or discarded before maintenance Based on the retained output result of the idm list io command re perform the virtual function creation and assignment to the domain For details see 6 4 4 Work to restore the SR IOV function 18 Ifa virtual device vnet vdisk of a guest domain has been kept unused before maintenance return it to its original status For details see 6 4 5 Work to restore a virtual device 6 3 3 Mounting a PCle card on the server using PCI hot plug PHP This section describes the procedure for mounting a PCIe card on the server using PCI hot plug PHP For details on the cfgadm 1M command used here see 2 Dynamically Configuring Devices in the Managing Devices in Oracle Solaris 11 2 In the case of Oracle Solaris 10 see SCSI Hot Plugging With the cfgadm Command Task Map in the Oracle Solaris Administration Devices and File Systems Note To perform PCIe card maintenance using the hot plug function confirm that the card is PHP enabled To do so check Appendix A Cards that Support PCI Hot Plug and Dynamic Reconfiguration in the Fujitsu M10 SPARC M10 Systems PCI Card Installation Guide Mount the PCI Express card in the target PCI slot Then incorporate it into a logical domain in the following procedure 1 Login as an Oracle Solaris super user Chapter6 Understanding the Preparations for Restoring the System 85 2 Execute the svcadm enable hotplug command to activate the hotplug service
110. ed to the PCI expansion unit configuration are as follows Table 1 2 Necessary processes related to the PCI expansion unit configuration PCI expansion unit Domain configuration Reconfiguration of Resetting of OpenBoot installation Oracle VM Server for PROM environment SPARC config variable Unsupported factory default Unnecessary Unnecessary Unsupported Contains a guest domain Unnecessary Unnecessary Supported factory default Unnecessary Unnecessary Supported Contains a guest domain Necessary XML Necessary Notes on using the direct I O function SPARC M10 4 M10 45 If the firmware version XCP2044 or later is used on the SPARC M10 4 or a firmware version XCP2050 or later is used on the SPARC M10 4S and one of the following operations is performed with the setpciboxdio command the logical domain configuration of the physical partition will return to the factory default at the next startup If the domain configuration contains a guest domain the OpenBoot PROM environment variable of the control domain will also be initialized Changing the enable disable setting for the direct I O function of the PCI Chapter1 Before Starting Maintenance Work 7 8 expansion unit Installing removing replacing the PCI expansion unit for the PCle slot of the SPARC M10 chassis for which the direct I O function of the PCI expansion unit is enabled Save the logical domain configuration information from Oracle Solaris to XML in advance To save it t
111. emoving them to ensure that they are reinstalled correctly Removing a PCI Express card cassette There are two types of PCle card cassette The shapes of the latch A in the figure and lever B in the figure vary depending on the type of the PCle card cassette but the same removal procedure applies For the types of PCIe card cassettes see Unless otherwise specified the figures explained in this section use Type 1 Figure 8 2 Types of PCIe card cassettes Type 1 Type 2 1 Raise the lever B in the figure while pressing the latch A in the figure of the PCle card cassette Chapter8 Maintaining the PCI Express Cards 173 Figure 8 3 Latch and lever of the PCIe card cassette 2 Hold the lever and carefully pull the PCle card cassette out of the PCle slot Note Place the removed PCle card cassette on the grounded ESD mat to ground any static electricity Figure 8 4 Removing the PCIe card cassette 174 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 8 3 0 Removing a PCI Express card 1 While pressing the cover stop remove the cover Figure 8 5 Cover stop 2 Remove the screw securing the PCle card cassette and remove the bracket of the PCle card Chapter 8 Maintaining the PCI Express Cards 175 Figure 8 6 Removal of bracket 3 Remove the PCle card from the PCle card cassette For expansion you do not have to perform this step Remove the filler unit for the PCIe
112. enance from the system Lowering the cable support of the chassis requiring maintenance and removing the power cord Removing the link card Installing the power cord and fixing the cable support Incorporating the chassis requiring maintenance into the system Diagnosing the chassis requiring maintenance Mounting the released link card on the system using physical partition dynamic reconfiguration DR Confirming that there is no problem with the replacement FRU Closing the rack door Active removal work procedure removing the link card using DR Reference 5 4 Checking the Operating Condition and Resource Usage Status 5 5 2 Releasing the link card or PCIe card using physical partition dynamic reconfiguration DR 5 8 1 Releasing of the SPARC M10 4S chassis possible only in a system with a building block configuration in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 9 Accessing an FRU in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Chapter 8 Maintaining the PCI Express Cards in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 1 Restoring the Chassis in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 2 1 Incorporation of the SPARC M10 4S chassis possible only in a system with a building block configuration in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual
113. enance 135 7 4 2 Inactive maintenance 151 7 4 3 Maintenance with the system stopped 160 Chapter 8 Maintaining the PCI Express Cards 171 8 1 Before Maintaining a PCI Express Card 171 8 2 Configuration of the PCIe Cards 171 8 3 Removing a PCI Express Card 172 8 3 1 Accessing a PCI Express card cassette 173 8 3 2 Removing a PCI Express card cassette 173 8 3 3 Removing a PCI Express card 175 8 4 Installing a PCI Express Card 177 8 4 1 Installing a PCI Express card 177 8 4 2 Installing a PCI Express card cassette 179 8 4 3 Restoring the chassis 180 Chapter9 Maintaining the Link Board 181 9 1 Before Maintaining a Link Board 181 9 2 Configuration of the Link Board 181 9 3 Removing the Link Board 182 9 3 1 Accessing the link board 182 9 3 2 Removing the link board 183 vi PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 9 4 Chapter 10 1 10 2 10 3 10 4 Chapter T11 11 2 11 3 11 4 Installing the Link Board 186 9 4 1 Installing the link board 186 9 4 2 Restoring the chassis 188 10 Maintaining the Link Cards 189 Before Maintaining a Link Card 189 Configuration of a Link Card 189 Removing the Link Card 192 Installing the Link Card 192 11 Maintaining the Link Cables 195 Before Maintaining a Link Cable 195 Configuration of the Link Cable Connection Ports 195 Removing a Link Cable 196 Installing a Link Cable 197 Chapter 12 Maintaining the Management Cable 199 12 1 12 2 12 3 12 4 Before Maintaini
114. encing Chapter 7 Maintenance Flow continue the maintenance work 210 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 15 Maintaining the PCI Tray This chapter describes the procedure for maintaining the PCI tray mounted in the PCI expansion unit Before Maintaining the PCI Tray Configuration of the PCI Tray Removing the PCI Tray Installing the PCI tray 15 1 Before Maintaining the PCI Tray This chapter describes only the configuration removal work and mounting work of a PCI tray Before removal work of an FRU perform necessary work items while referencing Chapter 7 Maintenance Flow For maintenance types of the PCI tray see PCI tray in Table 7 2 List of available maintenance types by FRU 15 2 Configuration of the PCI Tray This section describes the configuration and location of the PCI tray The PSU backplane I O board and fan backplane are mounted on the PCI tray When the PCI tray is to be replaced replace the PCI tray and PSU backplane at the same time The I O board and the fan backplane must be reinstalled on the new PCI tray 211 Figure 15 1 PCI tray location 1 Location number Component 1 PCI tray TRAY 15 3 A 15 3 1 Removing the PCI Tray This section describes the procedure for removing the PCI tray Enable the removal of the PCI tray before attempting to remove it For details see Chapter 7 Maintenance Flow
115. ep is unnecessary if the dynamic SR IOV function is used Perform step 3 ldm start reconf lt root domain name gt 3 Create the virtual function ldm create vf lt PF name gt 4 Assign the virtual function VF to the I O domain ldm add io lt VF name gt lt I O domain name gt a Log in to the root domain and restart Oracle Solaris This step is unnecessary if the dynamic SR IOV function is used Perform step 7 telnet localhost lt port name gt shutdown i6 g0 y 6 Start the I O domain ldm start lt I O domain name gt 92 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 a If you need to disable the hotplug service for the I O and root domains disable the hotplug service svcadm disable hotplug If virtual devices of a guest domain have been kept unused or the redundant configuration was released in 5 4 1 Releasing the assignment of I O devices resume the use of the virtual devices or set the redundant configuration again For details on the work see the manual for the application being used so Confirm that the status of the physical I O device is the same as before maintenance ldm list io 10 Confirm that the operating condition of the logical domain remains unchanged ldm list domain 6 4 5 Work to restore a virtual device If a virtual device vnet vdisk of a guest domain has been kept unused before maintenance return it to
116. eplacement For details see Check method when the system has been started in 6 6 1 How to check the serial number and firmware version of the PCI expansion unit 5 Install all the PCle card cassettes into the PCI expansion unit Lower the lever while pressing the latch of the PCle card cassette to secure it 6 Execute the cfgadm 1M command to supply power to the PCle card Enter the Ap_ID of the PCIe card that was checked in step 6 Chapter6 Understanding the Preparations for Restoring the System 81 cfgadm c connect Ap ID 7 Execute the cfgadm 1M command to incorporate the PCle card into the logical domain For the Ap_ID enter the Ap_ID of the link card you noted down before the start of maintenance cfgadm c configure Ap ID 8 Execute the cfgadm 1M command to confirm that the PCle card requiring maintenance has been incorporated into the logical domain In the following example the Receptacle is displayed as connected and the Occupant as configured indicating that the PCle card requiring maintenance is incorporated into the logical domain cfgadm a Ap_Id Type Receptacle Occupant Condition BB 0 PCLI 0 iobE7a77 pcie4 etherne hp connected configured 9 If you have released the I O device assigned to an I O domain before maintenance restore the physical I O device PCle end point device to its original status For details see 6 4 3 Work to return the physical I O device to the I O domain 1
117. equiring Maintenance and the Destination Physical Partition This section checks the PCI expansion unit requiring maintenance and the physical partition to which the PCI expansion unit belongs 1 Log in to the XSCF shell 2 Execute the ioxadm locator command to cause the CHECK LED of the PCI expansion unit requiring maintenance to blink so that you can locate the unit You do not have to perform this step when performing maintenance on the link card mounted on a server In the following example the CHECK LED of PCIBOX 2007 is specified XSCF gt ioxadm locator on PCIBOX 2007 3 For the building block configuration of the SPARC M10 4S execute the ioxadm list command to check the BB ID of the server that is connected to the PCI expansion unit requiring maintenance The following example shows that the BB ID is BB 01 XSCF gt ioxadm list PCIBOX 2007 PCIBOX Link PCIBOX 2007 BB 01 PCI 05 4 For the building block configuration of the SPARC M10 4S execute the showboards a command to check the PPAR ID of the physical partition using the link card The BB ID of the server to which the PCI expansion unit requiring maintenance is connected is displayed as a system board PSB number In the following example the PPAR ID of PSB 01 0 is 01 XSCF gt showboards a PSB PPAR ID LSB Assignment Pwr Conn Conf Test Assigned Passed Normal Assigned Passed Normal 52 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service
118. equiring maintenance to blink to locate it For details on the location of the CHECK LED and the procedure for checking it see 2 2 1 Front LEDs on the PCI expansion unit 1 Login to the XSCF shell 2 Execute the ioxadm command to cause the CHECK LED of the PCI expansion unit requiring maintenance to blink to locate it In the following example the CHECK LED of PCIBOX 2008 is specified XSCF gt ioxadm locator on PCIBOX 2008 Location Sensor Value Resolution Units PCIBOX 2008 LOCATE Blink LED PCIBOX 2008 FAN 0 SERVICE Blink LED 4 2 4 2 1 44 Precautions for Maintenance This section describes the precautions for maintenance Precautions for replacement This section describes the precautions for replacement PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Precautions for PCle card replacement For active maintenance by PHP a multipath setting is necessary depending on the use of the PCle cards For active maintenance by PHP confirm that the card is PHP supported To do so check Appendix A Cards that Support PCI Hot Plug and Dynamic Reconfiguration in the Fujitsu M10 SPARC M10 Systems PCI Card Installation Guide If you perform active replacement by combining PHP with dynamic SR IOV or the dynamic reconfiguration function for PCle end point devices confirm that the card supports the relevant function Check Appendix B Cards that Support SR IOV and Appendix D Cards
119. es into the PCI expansion unit Lower the lever while pressing the latch of the PCIe card cassette to secure it 12 Execute the svcadm enable hotplug command to activate the hotplug service svcadm enable hotplug 13 Execute the cfgadm 1M command to supply power to the PCle card Enter the Ap_ID of the PCIe card that was checked in step 6 cfgadm c connect Ap ID 14 Execute the cfgadm 1M command to incorporate the PCle card into the logical domain For the Ap_ID enter the Ap_ID of the link card you noted down before the start PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 of maintenance cfgadm c configure Ap ID 15 Execute the cfgadm 1M command to confirm that the PCle card requiring maintenance has been incorporated into the logical domain In the following example the Receptacle is displayed as connected and the Occupant as configured indicating that the PCle card requiring maintenance is incorporated into the logical domain cfgadm a Ap_Id Type Receptacle Occupant Condition BB 0 PCI 0 iobE7a77 pcie4 etherne hp connected configured 16 If you have released the I O device assigned to an I O domain before maintenance restore the physical I O device PCle end point device to its original status For details see 6 4 3 Work to return the physical I O device to the I O domain 17 Suppose that the virtual function VF using the SR IOV function has been delet
120. everse order to that in which it was created ldm destroy vf lt VF name gt Chapter5 Preparations Required for Maintenance 6 oa 6 Log in to the root domain and restart Oracle Solaris This step is unnecessary if the dynamic SR IOV function is used Perform step 7 telnet localhost lt port number gt shutdown i6 g0 y 7 Ifyou need to disable the hotplug service for the I O and root domains disable the hotplug service svcadm disable hotplug 3 Removing the root complex Remove the root complex from the root domain If the I O device of the chassis requiring maintenance is assigned to the root domain delete the physical I O device root complex from the root domain The work procedure depends on whether the dynamic reconfiguration function for root complexes is used For details see Table 5 6 Table 5 6 Differences in the procedures according to whether the dynamic reconfiguration function for root complexes is used Unnecessary Item Task Command Dynamic Dynamic reconfiguration reconfiguration is used 1 is not used 2 1 Stopping the root domain ldm stop domain Perform 2 Removing the physical I O device root Idm remove io Perform Perform complex from the root domain 1 Dynamic reconfiguration for root complexes is supported by XCP 2240 or later and Oracle VM Server for SPARC 3 2 or later In addition Oracle Solaris 11 2 SRU11 2 8 or later must be used for the control domain 2 If dy
121. f these units fails The system should not however be operated for an extended period with one failed unit When multiple power supply units are to be replaced perform the replacement one by one If redundancy of the power supply units cannot be secured cold maintenance must be executed Do not force a power supply unit into its slot Using excessive force may damage the component or the chassis Precautions for fan unit replacement When replacing multiple fan units replace no more than one unit at a time If the redundancy of the fan units cannot be secured system stopped maintenance must be executed The fan units have a redundant configuration therefore even if one fan unit fails the system can continue operating Do not however operate the system for an extended period with a failed fan unit Replace the failed fan unit as quickly as possible Precautions for I O board replacement Before replacing the I O board check the firmware version The firmware of the PCI expansion unit has been installed on the link card and the I O board When replacing the I O board however the firmware version cannot be automatically matched to the one before the replacement For this reason you must use a confirmed firmware version to match it to the firmware version before the Chapter 4 Preparation and Precautions for Maintenance 47 4 2 2 replacement If you cannot recognize the PCI expansion unit from the XSCF check the f
122. for maintenance with the system stopped or for active maintenance Check method when the system is stopped 1 Switch the mode switches for the master chassis and those chassis whose XSCFs are in the standby state to Service mode For the SPARC M10 1 M10 4 This is always the master chassis since it has a single unit configuration For a building block configuration no crossbar box Change the mode switches of BB ID 00 and 01 to Service mode For a building block configuration with crossbar boxes Change the mode switches of BB ID 80 and 81 to Service mode 2 Log into the XSCF shell of the master chassis 94 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 For the SPARC M10 1 M10 4 This is always the master chassis since it has a single unit configuration For the SPARC M10 4S no crossbar box The master chassis is the chassis where the BB ID 00 or 01 MASTER LED green at the rear of the chassis is on For the SPARC M10 4S with crossbar boxes The master chassis is the chassis where the BB ID 80 or 81 MASTER LED green at the rear of the chassis is on 2 Execute the setpparparam command and disable the Auto boot function of the physical partition In the following example PPAR ID 0 is set for Auto boot If Auto boot is disabled for the physical partition go to step 4 XSCF gt setpparparam p 0 s bootscript setenv auto boot false 4 Execute the poweron command to
123. for root complexes is supported by XCP 2240 or later and Oracle VM Server for SPARC 3 2 or later In addition Oracle Solaris 11 2 SRU11 2 8 or later must be used for the control domain 2 If dynamic reconfiguration for root complexes is not used set the control domain in delayed reconfiguration mode and then remove the physical I O device 1 Transition the control domain to delayed reconfiguration mode This step is unnecessary if root complexes are dynamically reconfigured Perform step 2 ldm start reconf lt control domain name gt 2 Delete the physical I O device from the control domain ldm remove io lt physical I O device name gt lt control domain name gt 3 Restart the control domain This step is unnecessary if the dynamic reconfiguration function for root complexes is used Chapter5 Preparations Required for Maintenance 67 shutdown i6 g0 y Note When set to delayed reconfiguration mode the control domain does not have control over the guest domains If you have set the control domain to delayed reconfiguration mode restart the control domain as soon as possible 4 Releasing the link card using dynamic reconfiguration DR For details on the procedure for dynamic reconfiguration DR see 5 4 3 Releasing a chassis requiring maintenance from the physical partition in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 5 5 2 Releasing the link card or PCle card after stopping t
124. g Caution Indications Labels Tags Safety Precautions Notes Regarding Static Electricity Other Precautions Emergency Power Off Important Information about the XCP Firmware Warning Caution Indications This manual uses the following conventions to indicate warning and alert messages which are intended to prevent injury to the user and others as well as damage to property Warning WARNING indicates a potential hazard that could result in death or serious personal injury if the user does not perform the procedure correctly Caution CAUTION indicates a potential hazard that could result in minor or moderate personal injury if the user does not perform the procedure correctly This also indicates that damage to the unit or other property may occur if the user does not perform the procedure correctly 1 2 Labels Tags This section describes the labels and tags attached to the chassis When performing maintenance always observe the precautions on the standard labels attached to the chassis A Caution Do not remove the labels or tags Note The contents of the labels and tags described here may differ from those actually attached to the chassis The system nameplate label A in the figure provides the product model number serial number and version number required for maintenance and management The standard label B in the figure contains notes and the following approved standards Security
125. g the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 3 Checking the operating condition of the system and the I O device usage status 4 Releasing the link card mounted on the server from the system using PCI hot plug PHP 5 Lowering the cable support 6 Removing the power cords of the PCI expansion unit 7 Installing the target FRU 8 Mounting the released link card on the system using PCI hot plug PHP 9 Fixing the cable support 10 Confirming that there is no problem with the replacement FRU 11 Closing the rack door Chapter 7 Maintenance Flow 121 Table 7 10 Item 1 2 10 11 122 Task Opening the rack door Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the operating condition of the system and the I O device usage status Releasing the link card mounted on the server from the system using PCI hot plug PHP Lowering the cable support Removing the power cords of the PCI expansion unit Installing the target FRU Mounting the released link card on the system using PCI hot plug PHP Fixing the cable support Confirming that there is no problem with the replacement FRU Closing the rack door Cold addition work procedure Releasing the link card using PHP Reference 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination P
126. gure Ap ID SS Execute the cfgadm c command to stop the power supply to the link card or PCle card requiring maintenance Enter the Ap_ID of the link card or PCIe card that you noted in advance cfgadm c disconnect Ap ID N Execute the cfgadm 1M command to cause the locator LED of the link card or PCle card to blink Enter the Ap_ID of the link card or PCle card that you noted in advance cfgadm x led attn mode blink Ap ID go If you need to disable the hotplug service disable the hotplug service svcadm disable hotplug Chapter5 Preparations Required for Maintenance 6 02 Releasing the link card or PCle card using physical partition dynamic reconfiguration DR This section describes the procedure for releasing the physical partition containing the link card from the system using dynamic reconfiguration DR The procedure can be performed only for a system with a building block configuration of the SPARC M10 4S Note To use the physical partition dynamic reconfiguration PPAR DR function you need to configure a logical domain beforehand according to 2 5 Dynamic Reconfiguration Conditions and Settings in the Fujitsu M10 SPARC M10 Systems Domain Configuration Guide If the PCIe end point of the PCIe card in the chassis requiring maintenance is assigned to an I O domain or if the SR IOV virtual function is assigned return the PCle end point to the root or control domain Then remove the root complex f
127. he case that the virtual function of SR IOV is used If the virtual function VF using the SR IOV function has been created for a PCIe card mounted on the PCI expansion unit and it has been assigned to the domain execute the Idm command from the control domain to delete and discard the virtual function The work procedure is different for the dynamic SR IOV function or for static SR IOV function For details see Table 5 3 For details on the procedure see the Oracle VM Server for SPARC Administration Guide released by Oracle Table 5 5 Differences in the releasing procedures between the dynamic SR IOV function and static SR IOV function Unnecessary Item Task Command Dynamic SR IOV Static SR IOV function function is used 1 is used 1 Enabling the hotplug service svcadm enable hotplug Perform Perform 2 2 Stopping the I O domain ldm stop domain Perform 3 Deleting the VF from the I O domain ldm remove io Perform Perform 4 Setting the root domain of the PF in ldm start reconf Perform delayed reconfiguration mode 5 Discarding the VF ldm destroy vf Perform Perform 64 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 5 5 Differences in the releasing procedures between the dynamic SR IOV function and static SR IOV function continued Unnecessary Item Task Command Dynamic SR IOV Static SR IOV function function is used 1 is used 6 Restarting the root domain shutdown i6 g0 y 7 Perform 7
128. he front cover For details see 6 1 1 Installing the front cover Install all the components listed below Power supply unit For details see 13 4 Installing a Power Supply Unit PCle card cassette For details see 8 4 2 Installing a PCI Express card cassette Now the work for mounting the FRU is complete While referencing Chapter 7 Maintenance Flow continue the maintenance work Chapter 17 Maintaining the Fan Backplane 235 236 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Appendix A Component List This appendix describes the components of the PCI expansion unit Figure A 1 shows the locations of the components mounted in the chassis Figure A 1 Locations of components 237 Location Component number I O board PCI tray Fan backplane Fan unit PSU backplane Power supply unit PCI Express card cassette Link board o AN BD oO FF WO N e PCI Express card 238 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Appendix B Component Specifications This appendix describes the specifications of the components B 1 PCI Express Card Link Board Power Supply Unit Fan Unit PCI Tray T O Board Fan Backplane PCI Express Card Up to eleven 11 PCIe cards can be mounted Table B 1 lists the specifications of the PCle card Table B 1 Specifications of the PCIe card Item Description Maximum number of PCle 11
129. he system Cold maintenance of the PCI expansion unit can be performed after stopping the system This section describes the procedure for stopping a system to which the PCI expansion unit is connected For details on the procedure see 5 6 Stopping the Entire System in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual or 5 5 Stopping the Entire System in the Fujitsu M10 1 SPARC M10 1 Service Manual Stopping the system with the XSCF command Confirm that the mode switch on the operation panel is in Service mode Log in to the XSCF shell Execute the poweroff command Check that the POWER LED on the operation panel is off Execute the showpparstatus command to confirm that the power to the physical partition is turned off VF ON ea 6 Execute the showdomainstatus command to confirm the operating condition of the logical domain Stopping the system from the operation panel Confirm that the mode switch on the operation panel is in Service mode Press the power switch on the operation panel for 4 seconds or more Check that the POWER LED on the operation panel is off Execute the showpparstatus command to confirm that the power to the Po NS 68 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 physical partition is turned off 5 Execute the showdomainstatus command to confirm the operating condition of the logical domain 5 6 Stopping the FRU Requiring Mainte
130. he text Each command has a section number in a command name to prompt users to refer to it e Notes on Safety Read the following documents thoroughly before using or handling any SPARC M10 Systems SPARC M10 Systems Important Legal and Safety Information SPARC M10 Systems Safety and Compliance Guide Preface xiii Syntax of the Command Line Interface CLI The command syntax is as follows A variable that requires the input of a value must be put in Italics An optional element must be enclosed in A group of options for an optional keyword must be enclosed in and delimited by Document Feedback If you have any comments or requests regarding this document please take a moment to share it with us by indicating the manual code manual title and page and stating your points specifically through the following websites Japanese site http jp fujitsu com platform server sparc manual m Global site http www fujitsu com global services computing server sparc downloads manual xiv PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 1 1 1 gt gt Before Starting Maintenance Work This chapter describes the safety precautions that must be observed before starting any maintenance work and provides important information that you should know Note the meanings of each of the following symbols and labels to ensure that the work is done correctly Warnin
131. hysical Partition 5 4 1 Checking the operating condition of the physical partition or logical domain 5 4 2 Checking the assignment status of I O devices 5 5 1 Releasing the link card or PCIe card using PCI hot plug PHP 5 7 1 Lowering the cable support 5 7 2 Removing the power cord Chapter 8 Maintaining the PCI Express Cards 6 3 1 Mounting a link card using PCI hot plug PHP 6 1 3 Fixing the cable support 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Cold addition releasing the link card using DR Cold addition of a PCI Express card to the PCI expansion unit can be performed after releasing the chassis in which the link card is mounted from the system using physical partition dynamic reconfiguration DR This can be done only when the connection destination of the PCI expansion unit has the building block configuration of the SPARC M10 4S Chapter 7 Maintenance Flow 123 124 Figure 7 9 Cold addition flow releasing the link card using DR 1 Opening the rack door 2 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 3 Checking the operating condition of the system and the I O device usage status 4 Releasing the link card mounted on the server from the system using physica
132. i6 g0 y 7 Ifyou need to disable the hotplug service for the I O and root domains disable the hotplug service svcadm disable hotplug In the case that the virtual function of SR IOV is used If the virtual function VF using the SR IOV function has been created for a PCle card mounted on the PCI expansion unit and it has been assigned to the domain execute the Idm command from the control domain to delete and discard the virtual function The work procedure is different for the dynamic SR IOV function or for static SR IOV function For details see Table 5 3 For details on the procedure see the Oracle VM Server for SPARC Administration Guide released by Oracle Table 5 3 Differences in the releasing procedures between the dynamic SR IOV function and static SR IOV function Unnecessary Item Task Command Dynamic SR IOV Static SR IOV function function is used 1 function is used 1 Enabling the hotplug service svcadm enable hotplug Perform Perform 2 2 Stopping the I O domain ldm stop domain Perform 3 Deleting the VF from the I O domain ldm remove io Perform Perform 4 Setting the root domain of the PF in ldm start reconf Perform delayed reconfiguration mode 5 Discarding the VF Idm destroy vf Perform Perform 6 Restarting the root domain shutdown i6 g0 y Perform 7 Disabling the hotplug service svcadm disable hotplug Perform Perform 2 1 Dynamic reconfiguration for the virtual function of SR IOV is supported by XC
133. ice of the boot disk If the boot disk is in a redundant configuration stop the I O domain for Oracle VM Server for SPARC 3 1 1 or earlier For Oracle VM Server for SPARC 3 1 1 1 or later you do not need to stop the I O domain If the boot disk is not in a redundant configuration stop the I O domain j If the hotplug service is disabled for the I O and root domains execute the svcadm enable hotplug command to enable the hotplug service svcadm enable hotplug 2 Stop the I O domain This step is unnecessary if the dynamic reconfiguration function for PCIe end point devices is used Perform step 3 ldm stop domain lt I O domain name gt 9 Delete the physical I O device from the I O domain ldm remove io lt device name gt lt I O domain name gt 4 Transition the root domain to delayed reconfiguration mode This step is unnecessary if the dynamic reconfiguration function for PCIe end point devices is used Perform step 5 ldm start reconf lt root domain name gt g Reassign the physical I O device deleted in step 3 to the root domain ldm add io lt device name gt lt root domain name gt 58 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 6 Login to the root domain and restart Oracle Solaris This step is unnecessary if the dynamic reconfiguration function for PCle end point devices is used Perform step 7 telnet localhost lt port number gt shutdown
134. in ldm start domain guest domain If the binding failed because an overlapping resource exists delete the relevant resource from the logical domain The following example shows that an overlapping resource has been deleted ldm bind root domain No free matching I O device for LDom root domain name PCIE1 ldm start reconf primary ldm rm io PCIE1 primary Notice The primary domain is in the process of a delayed reconfiguration Any changes made to the primary domain will only take effect after it reboots If you deleted a resource restart the control domain shutdown i6 g0 y If you deleted a resource restart the control domain and then bind and start other logical domains For logical domains with dependency start them in the correct order In the following example both root domain and guest domain are bound and Chapter 1 Before Starting Maintenance Work 15 started ldm bind root domain ldm start domain root domain ldm bind guest domain ldm start domain guest domain 24 Execute the Idm Is command to confirm that the logical domain is operating normally ldm 1s NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active n cv UART 8 8G 66 root domain active t 5000 8 4G 19 25 After restoring the configuration information execute the Idm add spconfig command to save the configuration information to the XSCF In the following example the configuration information of test1 ha
135. ine the error log information The log information is listed in the order of date with the oldest appearing first The following example shows that an Alarm status occurred in PSU 1 and PSU 2 at 12 45 31 on Oct 20 the Alarm status changed to a Warning status at 15 45 31 on the same day and then the Alarm status further expanded to PSU 1 PSU 2 and PSU 3 at 17 45 31 on the same day XSCF gt showlogs error Date Oct 20 12 45 31 JST 2012 Code 00112233 4455667788 99aabbcc 8 8 99aabbcceef 0011223344 Status Alarm Occurred Oct 20 12 45 31 000 JST 2012 FRU PSU 1 PSU 2 Msg ACFAIL occurred ACS 3 FEP type Al Chapter 4 Preparation and Precautions for Maintenance 43 Date Oct 20 15 45 31 JST 2012 Code 00112233 445566778899aabbcc 88 99aabbcceef 0011223344 Status Warning Occurred Oct 20 15 45 31 000 JST 2012 FRU PSU 1 PSU 2 Msg ACFAIL occurred ACS 3 FEP type Al Date Oct 20 17 45 31 JST 2012 Code 00112233 445566778899aabbcc 88 99aabbcceeff0011223344 Status Alarm Occurred Oct 20 17 45 31 000 JST 2012 FRU PSU 1 PSU 2 PSU 3 Msg ACFAIL occurred ACS 3 FEP type Al 4 1 3 Locating the PCI expansion unit requiring maintenance This section describes the procedure for locating a PCI expansion unit requiring maintenance by causing the CHECK LED locator on the front panel of the chassis to blink Execute the ioxadm command of the XSCF firmware to cause the CHECK LED for the PCI expansion unit r
136. ing maintenance by causing the CHECK LED locator of the unit to blink 3 Checking the operating condition of the system and the I O device usage status 4 Releasing the assignment of the PCle card for removal 5 Lowering the cable support 6 Removing the PCle card 7 Fixing the cable support 8 Confirming that there is no problem with the chassis from which the PCle card was removed 9 Turning off the CHECK LED locator of the PCI expansion unit requiring maintenance 10 Closing the rack door End Chapter 7 Maintenance Flow 127 Table 7 12 Item 1 2 o N FD A 10 7 3 2 128 Task Opening the rack door Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the operating condition of the system and the I O device usage status Releasing the assignment of the PCIe card for removal Lowering the cable support Removing the PCle card Fixing the cable support Confirming that there is no problem with the chassis from which the PCle card was removed Turning off the CHECK LED locator of the PCI expansion unit requiring maintenance Closing the rack door Cold removal Hot removal work procedure for a PCI Express card Reference 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 4 1 Checking the operating condition of the physical partition or logical domain 5 4
137. ing the cable support Performing a diagnosis test Confirming that there is no problem with the chassis from which the PCIe card was removed Returning the mode switch on the operation panel to Locked mode Starting the system Closing the rack door Cold removal work procedure releasing the link card after stopping the system Reference 5 2 Switching the Mode Switch to Service Mode in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual or Fujitsu M10 1 SPARC M10 1 Service Manual 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 4 1 Checking the operating condition of the physical partition or logical domain 5 5 3 Releasing the link card or PCIe card after stopping the system 5 7 1 Lowering the cable support 5 7 2 Removing the power cord Chapter 8 Maintaining the PCI Express Cards 6 1 2 Installing the power cords 6 1 3 Fixing the cable support 6 3 Diagnosing a Replacement FRU in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 4 Diagnosing a Replacement FRU in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 7 Checking the FRU Status after Maintenance in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 6 Returning the Mode Switch to Locked Mode in the Fujitsu M10
138. irmware version of a part that has not been replaced after the replacement to match it to the firmware version before the replacement When you replace the I O board and the link card at the same time check the firmware version before the replacement If you replace the I O board and the link card at the same time without checking the firmware version you may not be able to match the version Do not replace the I O board and the fan backplane at the same time When you replace the I O board and the fan backplane at the same time all the serial numbers of the PCI expansion unit are changed to 0 0000000000 If all these serial numbers are changed to 0 0000000000 use the ioxadm command of the XSCF firmware to restore the serial numbers If you replace both the I O board and the fan backplane replace either one and then start the physical partition connected to the PCI expansion unit to determine the error log information on the PCI expansion unit After confirming that no error log information is displayed replace the other Field Replaceable Unit FRU If multiple PCI expansion units are used and they are combined such that their firmware versions match there will be no problem with the operation even if the PCI expansion units are using different firmware versions Precautions for updating the firmware Always read the latest Product Notes It is not possible to update the firmware on more than one unit at a time Update the firm
139. its original status 1 If virtual devices of a guest domain have been kept unused or the redundant configuration was released resume the use of the virtual devices or set the redundant configuration again For details on the work see the manual for the application being used N Confirm that the status of the physical I O device is the same as before maintenance ldm list io w Confirm that the operating condition of the logical domain remains unchanged ldm list domain Chapter 6 Understanding the Preparations for Restoring the System 93 6 5 Turning off the CHECK LED Locator of the PCI Expansion Unit This section describes the procedure for turning off the CHECK LED locator of the PCI expansion unit requiring maintenance 1 Login to the XSCF shell 2 Execute the ioxadm locator command and confirm that the CHECK LED locator of the PCI expansion unit requiring maintenance is turned off In the following example the CHECK LED of PCIBOX 12B4 is specified XSCF gt ioxadm locator off PCIBOX 12B4 6 6 Updating the Firmware of the PCI Expansion Unit This section describes the precautions to be observed when updating and the method of checking the firmware version 6 6 1 How to check the serial number and firmware version of the PCI expansion unit The largest firmware version number indicates the latest firmware Check the current firmware version before starting update The procedures will be different
140. ive replacement replacing the link card using DR Release the link card mounted on the destination server from the system using physical partition dynamic reconfiguration DR This can be done only when the link card is mounted on the M10 4S and the system is in a building block configuration of the SPARC M10 4S Chapter 7 Maintenance Flow 139 Figure 7 15 Active replacement flow replacing the link card using DR 1 Opening the rack door 2 Checking the firmware version of the PCI expansion unit 3 Checking the operating condition of the system and the I O device usage status 4 Releasing the link card mounted on the server from the system using physical partition dynamic reconfiguration DR 5 Releasing the chassis requiring maintenance from the system 6 Lowering the cable support of the chassis requiring maintenance and removing the power cord 7 Replacing a faulty link card 8 Installing the power cord and fixing the cable support 9 Incorporating the chassis requiring maintenance into the system 10 Diagnosing the chassis requiring maintenance 11 Mounting the released link card on the system using physical partition dynamic reconfiguration DR 12 Confirming that there is no problem with the replacement FRU End 13 Closing the rack door 140 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 7 17 Item 1 2 10 11 12 13 Task
141. jitsu M10 SPARC M10 Systems XSCF Reference Manual Fujitsu M10 SPARC M10 Systems RCIL User Guide 3 Fujitsu M10 SPARC M10 Systems XSCF MIB and Trap Lists Fujitsu M10 SPARC M10 Systems Product Notes Fujitsu M10 SPARC M10 Systems Glossary 1 The listed manuals are subject to change without notice 2 The printed manual comes with the product 3 This document applies specifically to the FUJITSU M10 and FUJITSU ETERNUS storage system xii PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Text Conventions This manual uses the following fonts and symbols to express specific types of information Font Symbol Meaning Example AaBbCc123 What you type when contrasted with on screen XSCF gt adduser jsmith computer output This font represents the example of command input in the frame AaBbCcl123 The names of commands files and directories XSCF gt showuser P on screen computer output User Name jsmith This font represents the example of command output Privileges useradm auditadm Italic Indicates the name of a reference manual a variable See the Fujitsu M10 SPARC M10 or user replaceable text Systems Installation Guide uoa Indicates names of chapters sections items buttons See Chapter 2 Network Connection or menus Command syntax in the text While the XSCF commands have a section number of 8 or 1 it is omitted from the text The Oracle Solaris commands have a section number such as 1M in t
142. l partition dynamic reconfiguration DR 5 Lowering the cable support 6 Removing the power cords of the PCI expansion unit 7 Installing the target FRU 8 Installing the power cords of the PCI expansion unit 9 Fixing the cable support 10 Mounting the released link card on the system using physical partition dynamic reconfiguration DR 11 Confirming that there is no problem with the replacement FRU 12 Closing the rack door PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 7 11 Item il 2 10 11 12 Task Opening the rack door Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the operating condition of the system and the I O device usage status Releasing the link card mounted on the server from the system using physical partition dynamic reconfiguration DR Lowering the cable support Removing the power cords of the PCI expansion unit Installing the target FRU Installing the power cords of the PCI expansion unit Fixing the cable support Mounting the released link card on the system using physical partition dynamic reconfiguration DR Confirming that there is no problem with the replacement FRU Closing the rack door 7 3 7 3 1 Cold addition work procedure Releasing the link card using DR Reference 5 2 Checking the PCI Expansion Unit Requiring Maintenanc
143. lane screw 3 Slide the fan backplane in the direction of the arrow and align the groove of the fan backplane with the guides four places to remove it Note Do not remove the fan backplane by holding one of its connectors Note Place the removed fan backplane on the grounded ESD mat to ground any static electricity Chapter 17 Maintaining the Fan Backplane 231 Figure 17 4 Fan backplane removal 17 4 Installing the Fan Backplane This section describes the procedure for installing the fan backplane 17 4 1 Installing the fan backplane 1 Install the fan backplane by aligning it with the guides four places and then sliding it in the direction of the arrow 232 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Fan backplane installation Figure 17 5 Secure the fan backplane with the screw 2 233 Chapter 17 Maintaining the Fan Backplane Figure 17 6 Fan backplane screw 3 Connect the two cables to the fan backplane Note Connect the cables in their original positions by referring to the record that you made before the start of maintenance 234 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 17 4 2 Figure 17 7 Fan backplane cable Restoring the chassis Ii Install the PCI tray For details see 15 4 2 Installing the PCI tray Install all the fan units For details see 14 4 1 Installing a fan unit Install t
144. ldm remove io lt device name gt lt root domain name gt 4 Login to the root domain and restart Oracle Solaris on the root domain This step is unnecessary if the dynamic reconfiguration function for PCIe end point devices is used Perform step 5 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 telnet localhost lt port number gt shutdown i6 g0 y 5 Assign the physical I O device to the I O domain ldm add io lt device name gt lt I O domain name gt 6 Start the I O domain This step is unnecessary if the dynamic reconfiguration function for PCle end point devices is used Perform step 7 ldm start lt I O domain name gt 7 If you need to disable the hotplug service for the I O and root domains disable the hotplug service svcadm disable hotplug 6 4 4 Work to restore the SR IOV function If the SR IOV function was used to remove and discard a virtual function VF before maintenance according to 2 Deleting the assignment of the I O of 5 5 1 Releasing the link card or PCIe card using PCI hot plug PHP or 2 Deleting the assignment of the I O of 5 5 2 Releasing the link card or PCIe card using physical partition dynamic reconfiguration DR create the virtual function again and assign it to the domain based on the retained output data from the Idm list io command Table 6 4 Differences in the releasing procedures between the dynamic SR IOV function and static SR
145. llowing example shows that the configuration information at the next startup is not the factory default In this case execute the setdomainconfig command to set the configuration information at the next startup of the physical partition to the factory default XSCF gt showdomainconfig p 0 PPAR ID 0 Booting config Current itest Next stest2 XSCF gt setdomainconfig p 0 i 1 XSCF gt showdomainconfig p 0 PPAR ID 20 Booting config Current testl Next factory default config name factory default domains iL date created 15 Check auto boot of the OpenBoot PROM environment variable to stop in the OpenBoot PROM state If the value is true change it to false Chapter 1 Before Starting Maintenance Work 13 14 XSCF gt setpparparam p 0 s bootscript setenv auto boot false PPAR ID of PPARs that will be affected 0 OpenBoot PROM variable bootscript will be changed Continue yln ty If you changed the value check the OpenBoot PROM environment variable XSCF gt showpparparam p 0 use nvramrc security mode bootscript setenv auto boot false 16 Execute the poweron command to restart the physical partition XSCF gt poweron p 0 17 Execute the showdomainstatus command to check the status of the control domain Confirm that the status of the control domain is displayed as OpenBoot Running indicating that it is in the OpenBoot PROM state XSCF gt showdomainstatus p 0
146. lowing patterns are available for active maintenance of the link card Active replacement replacing the link card using PHP Active replacement replacing the link card using DR Active addition installing the link card using PHP Active addition installing the link card using DR Active removal removing the link card using PHP Active removal removing the link card using DR PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Active replacement replacing the link card using PHP Use the PCI hot plug PHP to release the link card mounted on a server from the system This can be done only when the link card is mounted on the SPARC M10 4 M10 4S and the setting for the direct I O function is disabled Figure 7 14 Active replacement flow replacing the link card using PHP 1 Opening the rack door 2 Checking the firmware version of the PCI expansion unit 3 Checking the operating condition of the system and the I O device usage status 4 Releasing the link card mounted on the server from the system using PCI hot plug PHP 5 Lowering the cable support 6 Replacing a faulty link card 7 Mounting the released link card on the system using PCI hot plug PHP 8 Fixing the cable support 9 Confirming that there is no problem with the replacement FRU 10 Closing the rack door End Chapter 7 Maintenance Flow 137 Table 7 16 Item 1 2 10 138 Task Opening the rack
147. ly for a redundant configuration 7 1 FRU Replacement Workflow for the PCI Expansion Unit This section describes the replacement workflow for the following FRUs of the PCI expansion unit PCI Express card Link board Link cable Management cable Power supply unit Fan unit PCI tray 1 O board Fan backplane 102 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 TA Hot replacement This section describes the hot replacement workflow References to detailed descriptions are written in the work procedure tables See any of them as required The following patterns are available for hot replacement Hot replacement for a PCI Express card Hot replacement for a power supply unit or fan unit Hot replacement for a PCI Express card Hot replacement is applicable to the PCI Express card Perform replacement according to the following procedure Chapter 7 Maintenance Flow 103 Figure 7 1 Hot replacement flow for a PCI Express card 1 Opening the rack door 2 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 3 Checking the operating condition of the system and the I O device usage status 4 Releasing the PCle card from the system using PCI hot plug PHP 5 Lowering the cable support 6 Replacing a faulty PCle card 7 Fixing the cable support 8 Mounting the replacement PCle card and restoring
148. main 3 Deleting the PCIe end point device from the 1 O domain 4 Setting the root domain in delayed reconfiguration mode 5 Assigning the PCIe end point device to the root domain 6 Restarting the root domain Command svcadm enable hotplug ldm stop domain ldm remove io ldm start reconf Idm add io shutdown i6 g0 y Dynamic reconfiguration is used 1 Perform Perform Perform Unnecessary Dynamic reconfiguration is not used 2 Perform 3 Perform Perform Perform Perform Perform Chapter 5 Preparations Required for Maintenance 57 Table 5 2 Differences in the procedures according to whether the dynamic reconfiguration function for PCle end point devices is used continued Unnecessary Item Task Command Dynamic Dynamic reconfiguration reconfiguration is used 1 is not used 2 7 Disabling the hotplug service svcadm disable hotplug Perform Perform 3 1 Dynamic reconfiguration for PCIe end point devices is supported by XCP 2230 or later and Oracle VM Server for SPARC 3 1 1 1 or later 2 If dynamic reconfiguration for PCIe end point devices is not used stop the I O domain delete the physical I O device and then restore the physical I O device to the root domain 3 If dynamic reconfiguration for PCIe end point devices is not used the hotplug service operates whether it is enabled or disabled Note Note the following when deleting a physical I O device PCIe end point dev
149. mary primary primary primary primary primary primary iodom00 lt SR IOV function used iodom01 lt SR IOV function used iodom02 lt SR IOV function used V He oO e sO E a OU BAU E7 BO CMUL NETO BO CMUL SASHBA BO PCI BO E BO BO BO PCI8 BO CMUL NET2 BO E BO E BO E BO E BO PCI BO PCI10 BO PCIO IOVNET PFO BO PCIO IOVNET PF1 BO CMUL NETO IOVNET PFO BO CMUL NETO IOVNET PFL BO CMUL NET2 IOVNET PF1 BO CMUL NET2 IOVNET PFO BO PCIO IOVNET PFO VEFO BO PCIO IOVNET PFO VF1 BO PCIO IOVNET PFO VF2 wee eee ee ae Ss Se A AeA a G ma O ra O ra E O B ra G ra D ma nr PrRrrRe e POOP RPATATNADADAUABWWNNRFTON DU BWDNHEF OO res Ct tt SY a a a tt a a a Ct tt a a a a PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 4 Preparation and Precautions for Maintenance This chapter describes preparations that must be completed prior to maintenance and the precautions for various work and maintenance Troubleshooting Precautions for Maintenance 4 1 4 1 1 Troubleshooting This section explains suspected failure conditions Use the flow to determine the cause of a failure and identify the failure location in the following cases For details on the flow for determining the cause of a failure see 4 1 1 Determining the causes of individual failures When the CHECK LED is on When
150. n a system with a building block configuration in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 1 Diagnosing the system board in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 3 2 Mounting the link card on the server using physical partition dynamic reconfiguration DR 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S5 SPARC M10 4 SPARC M10 4S Service Manual 1 For details on the work for installing a PCI expansion unit see the Fujitsu M10 SPARC M10 Systems Installation Guide 146 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Active removal removing the link card using PHP Use the PCI hot plug PHP to release the link card mounted on a server from the system This can be done only when the link card is mounted on the SPARC M10 4 M10 4S and the setting for the direct I O function is disabled Figure 7 18 Active removal flow removing the link card using PHP 1 Opening the rack door 2 Checking the operating condition of the system and the I O device usage status 3 Releasing the link card mounted on the server from the system using PCI hot plug PHP 4 Lowering the cable support 5 Removing the link card 6 Fixing the cable support 7 Confirming that there is no problem with the chassis after the removal 8 Closing the rack door End Chapter 7 Maintenance
151. n the PCI expansion unit the direct I O function is disabled Therefore it is not necessary to save restore each item of information PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 1 7 3 Table 1 4 Operations when installing removing replacing the PCI expansion unit for the PCIe slot of the SPARC M10 chassis for which the direct I O function is enabled Maintenance environment Domain configuration Reconfiguration of Oracle VM Server for SPARC config When stopping PPAR for factory default Unnecessary installation removal Contains a guest Necessary XML domain When stopping PPAR to factory default Unnecessary replace the failed PCI Contains a guest Necessary XML expansion unit 1 domain When stopping PPAR to factory default Unnecessary replace the normal PCI Contains a guest Unnecessary expansion unit 1 domain 1 This includes the replacement of the link card link cable management cable or link board Resetting of OpenBoot PROM environment variable Unnecessary Necessary Unnecessary Necessary Unnecessary Unnecessary How to save restore the logical domain configuration information and the OpenBoot PROM environment variable 1 Execute the Idm Is spconfig command on Oracle Solaris super user prompt to display the list of the configuration information and then confirm the configuration information to be saved The following example shows that test3 is the curre
152. n to the Status It Had before Maintenance after maintenance 5 4 3 Checking the usage status of the internal disk In the following execution example the configurations and states of devices in pools and errors generated from the devices are checked 1 Log in to Oracle Solaris on the logical domain 2 Execute the zpool status command to check the configurations and operating conditions of devices The following example shows that the system volume of the control domain has a mirror configuration zpool status rpool pool rpool state ONLINE scan resilvered 28 7M in OhOm with 0 errors on Tue Jan 21 10 10 01 2014 config Chapter 5 Preparations Required for Maintenance 55 errors NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror 0 ONLINE 0 0 0 c2t50000393E802CCE2d0s0 ONLINE 0 0 0 c3t50000393A803B13Ed0s0O ONLINE 0 0 0 No known data errors 5 5 5 5 1 Releasing the Link Card or PCle Card from the System This section describes the procedure for releasing the link card mounted on the server or a PCIe card mounted in the PCI expansion unit from the system These cards are released from the system using one of the following methods Using the PCI hot plug PHP using physical partition dynamic reconfiguration DR and stopping the system Note The link card is mounted in the PCI Express PCIe slot of the SPARC M10 1 or SPARC M10 4 M10 4S The procedure for removing the link card is the same as
153. n workflow for the following FRU of the PCI expansion unit PCI Express card 7 2 1 Hot addition This section describes the hot addition workflow References to detailed descriptions are written in the work procedure tables See any of them as required The following patterns are available for hot addition Hot addition for a PCI Express card 116 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Hot addition for a PCI Express card Hot addition is applicable to the PCI Express card Perform addition according to the following procedure Figure 7 6 Hot addition flow for a PCI Express card 1 Opening the rack door 2 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 3 Checking the operating condition of the system and the I O device usage status 4 Lowering the cable support 5 Installing a PCle card 6 Fixing the cable support 7 Incorporating the PCle card 8 Confirming that there is no problem with the installed FRU 9 Turning off the CHECK LED locator of the PCI expansion unit requiring maintenance 10 Closing the rack door End Chapter 7 Maintenance Flow 117 Table 7 8 Hot addition work procedure Item Task 1 Opening the rack door 2 Locating the PCI expansion unit requiring N OA a A 10 22 118 maintenance by causing the CHECK LED locator of the unit to blink Che
154. namic reconfiguration for root complexes is not used stop the root domain and then remove the physical I O device 1 Stop the root domain This step is unnecessary if root complexes are dynamically reconfigured Perform step 2 ldm stop domain lt root domain name gt 2 Remove the physical I O device root complex from the root domain 66 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 ldm remove io lt physical I O device name gt lt root domain name gt 3 Confirm that the I O device of the chassis to be released is unused and unassigned ldm list io Remove the root complex from the control domain If the I O device of the chassis requiring maintenance is assigned to the control domain delete the physical I O device root complex from the control domain The work procedure depends on whether the dynamic reconfiguration function for root complexes is used For details see Table 5 7 Table 5 7 Differences in the procedures according to whether the dynamic reconfiguration function for root complexes is used Unnecessary Item Task Command Dynamic Dynamic reconfiguration reconfiguration is used 1 is not used 2 1 Setting the control domain in delayed ldm start reconf 7 Perform reconfiguration mode 2 Deleting the physical I O device from the ldm remove io Perform Perform control domain 3 Restarting the control domain shutdown i6 g0 y Perform 1 Dynamic reconfiguration
155. nance This section describes the procedure for stopping the power supply to the FRU requiring maintenance using the ioxadm command 1 Log in to the XSCF shell 2 Execute the ioxadm command to stop the FRU requiring maintenance In the following example PSU 0 of PCIBOX 2007 is specified to stop the corresponding PSU XSCF gt ioxadm poweroff PCIBOX 2007 PSU 0 In the following example FAN 0 of PCIBOX 2007 is specified to stop the corresponding fan unit XSCF gt ioxadm poweroff PCIBOX 2007 FAN 0 3 Confirm that the CHECK LED of the FRU requiring maintenance blinks Ont 5 7 1 Accessing an FRU This section describes the operations to be performed before accessing the FRU requiring maintenance Lowering the cable support To perform maintenance on any of the following components on the rear of or inside the chassis lower the cable support before performing the work PCI Express card Link board Power supply unit PCI tray I O board Chapter 5 Preparations Required for Maintenance 69 70 Note There are two types of cable supports Distinguish them by the locations of the screws fixing the cable support For details on the locations of the screws see Figure 5 1 and Figure 5 2 Lowering the cable support 1 Loosen the right and left screws fixing the cable support and unlock it by pulling the upper and lower metal fittings towards you in the direction of the arrow Figure 5 1 Locations of
156. nance and system stopped maintenance Active maintenance Maintenance is performed when the physical partition to which an FRU requiring maintenance is assigned is operating Inactive maintenance Maintenance is performed when the physical partition to which an FRU requiring maintenance is assigned is stopped System stopped maintenance Maintenance is performed with all physical partitions stopped Definition according to the power supply status In addition for each of the three Definition according to the system operation status there are two types hot maintenance and cold maintenance Hot maintenance Maintenance is performed with the power cord of the server requiring maintenance connected Cold maintenance Maintenance is performed with the power cord of the server requiring maintenance removed 30 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 3 4 Mounting location SPARC M10 1 Configuration of 1 SPARC M10 4 M10 4S SPARC M10 4S building block configuration Active hot PHP Enabled 1 Enabled 2 Active hot DR Enabled 3 Types of maintenance for the link card Inactive hot Enabled Enabled Inactive cold System stopped hot Enabled Enabled Enabled Not maintainable System stopped cold Enabled Enabled Enabled 1 It is necessary to use the PCI Hot Plug PHP to release the link card from the system If the set
157. ndicates the normal state Or the breaker is open or the power supply is otherwise off LEDs on rear of PCI expansion unit An LED is mounted on each component If a component experiences an error check the LEDs to see which component requires maintenance Check the LEDs before starting maintenance work The following LEDs are mounted on the rear panel of the PCI expansion unit Power supply unit A in the figure T O board B in the figure Link board C in the figure PCle slot D in the figure Figure 2 6 LEDs on the rear of the chassis The LEDs mounted on each component and the statuses indicated by those LEDs are listed below PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 2 3 LEDs and status of power supply unit Name POWER FAIL oe A Color Green Amber Status On Blinking 1 On Blinking 1 Off 1 The blink interval is 1 second 1 Hz Description The input power is turned on and power is being supplied normally Standby condition Indicates that an error has occurred Warning state an error has occurred but the power unit is still operating The input power is turned off Table 2 4 LEDs and status of I O board Name CHECK Indicator AN Color Amber Status On Blinking 1 Off 1 The blink interval is 1 second 1 Hz Description Indicates that an error has occurred Indicates that the component re
158. ndition of the system and the I O device usage status 5 Releasing the link card mounted on the server from the system using PCI hot plug PHP 6 Enabling access to the FRU requiring maintenance 7 Replacing a faulty FRU 8 Installing the front cover if you have removed it 9 Mounting the released link card on the system using PCI hot plug PHP 10 Fixing the cable support 11 Confirming that there is no problem with the replacement FRU 12 Closing the rack door End 112 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 7 6 Item Task 1 Opening the rack door 2 Locating the PCI expansion unit requiring 10 11 12 maintenance by causing the CHECK LED locator of the unit to blink Checking the firmware version of the PCI expansion unit if you are replacing an I O board Checking the operating condition of the system and the I O device usage status Releasing the link card mounted on the server from the system using PCI hot plug PHP Enabling access to the FRU requiring maintenance Replacing a faulty FRU Installing the front cover if you have removed it Mounting the released link card on the system using PCI hot plug PHP Fixing the cable support Confirming that there is no problem with the replacement FRU Closing the rack door Cold replacement work procedure Releasing the link card using PHP Reference 5 2 Checking the PCI Expa
159. ng a Management Cable 199 Configuration of the Management Cable Connection Port 199 Removing the Management Cable 200 Installing the Management Cable 201 Chapter 13 Maintaining the Power Supply Units 203 13 1 13 2 13 3 13 4 Before Maintaining Power Supply Units 203 Configuration of the Power Supply Units 203 Removing a Power Supply Unit 204 Installing a Power Supply Unit 205 Chapter 14 Maintaining the Fan Units 207 14 1 14 2 14 3 14 4 Before Maintaining Fan Units 207 Configuration of the Fan Units 207 Removing a Fan Unit 208 14 3 1 Accessing a fan unit 208 14 3 2 Removing a fan unit 209 Installing a Fan Unit 210 14 4 1 Installing a fan unit 210 Contents vii 14 4 2 Restoring the chassis 210 Chapter 15 Maintaining the PCI Tray 211 15 1 Before Maintaining the PCI Tray 211 15 2 Configuration of the PCI Tray 211 15 3 Removing the PCI Tray 212 15 3 1 Accessing the PCI tray 212 15 3 2 Removing the PCI tray 213 15 3 3 Removing the components of the PCI tray 214 15 4 Installing the PCI Tray 214 15 4 1 Installing the components of the PCI tray 214 15 4 2 Installing the PCI tray 215 15 4 3 Restoring the chassis 215 Chapter16 Maintaining the I O Board 217 16 1 Before Maintaining the I O Board 217 16 2 Configuration of the I O Board 217 16 3 Removing the I O Board 218 16 3 1 Accessing the I O board 218 16 3 2 Removing the I O board 219 16 4 Installing the I O Board 222 16 4 1 Installing the I O board 222 16 4 2 Restoring
160. ng a Power Supply Unit This section describes the procedure for removing a power supply unit Enable the removal of the power supply unit before attempting to remove it For details see Chapter 7 Maintenance Flow Caution Before you handle any components wear a wrist strap to ground any static electricity If you perform this procedure without a wrist strap individual components or the overall system may be damaged For details see 1 4 Notes Regarding Static Electricity 1 Push the lever of the power supply unit requiring maintenance A in the figure in the direction of the arrow to release the lock and then pull out the unit by grasping the handle B in the figure 204 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 13 2 Removing a power supply unit 2 Using your other hand support the power supply unit from below and carefully remove it from the slot Note Place the removed power supply unit on a grounded antistatic ESD mat 13 4 Installing a Power Supply Unit This section describes the procedure for installing a power supply unit 1 With one hand support the power supply unit from below and then carefully insert it into its slot Note Do not forcibly push the power supply unit into its slot Using excessive force may damage the component or the chassis 2 Push the power supply unit fully home Note Make sure that the power supply unit is firmly inserted
161. ng physical partition dynamic reconfiguration DR For details see 5 5 2 Releasing the link card or PCIe card using physical partition dynamic reconfiguration DR and 6 3 2 Mounting the link card on the server using physical partition dynamic reconfiguration DR 100 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 7 Maintenance Flow This chapter describes the maintenance workflow Be sure to check the notes on FRUs requiring maintenance in the Fujitsu M10 SPARC M10 Systems Product Notes for the latest XCP version FRU Replacement Workflow for the PCI Expansion Unit FRU Installation Workflow for the PCI Expansion Unit FRU Removal Workflow for the PCI Expansion Unit Check the notes on maintenance in 4 2 Precautions for Maintenance before performing maintenance and then proceed with your work according to the following maintenance workflow Table 7 1 Maintenance workflow Item Procedure Reference 1 Identifying the FRU tobe replaced 4 1 2 Identifying a failure 2 Checking the available maintenance Table 7 2 List of available maintenance types types by FRU 3 Hot maintenance 1 3 a Hot replacement 7 1 1 Hot replacement 3 b Hot addition 7 2 1 Hot addition 3 c Hot removal 7 3 1 Hot removal 4 Cold maintenance 4 a Cold replacement 7 1 2 Cold replacement 4 b Cold addition 7 2 2 Cold addition 4 c Cold removal 7 3 2 Cold removal 1 To perform PCIe card maintenance using
162. ng the link card after stopping the system 1 Opening the rack door 2 Switching the mode switch on the operation panel of the destination server to Service mode 3 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 4 Checking the operating condition of the system and the I O device usage status 5 Stopping the system 6 Lowering the cable support 7 Removing the power cords of the PCI expansion unit 8 Removing the PCle card 9 Installing the power cords of the PCI expansion unit 10 Fixing the cable support 11 Performing a diagnosis test 12 Confirming that there is no problem with the chassis from which the PCle card was removed 13 Returning the mode switch on the operation panel to Locked mode 14 Starting the system 15 Closing the rack door Chapter 7 Maintenance Flow 129 Table 7 13 Item Task 1 Opening the rack door 2 Switching the mode switch on the 10 11 12 13 14 15 130 operation panel of the destination server to Service mode Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the operating condition of the system and the I O device usage status Stopping the system Lowering the cable support Removing the power cords of the PCI expansion unit Removing the PCle card Installing the power cords of the PCI expansion unit Fix
163. ng the rack door 2 Switching the mode switch on the operation panel of the destination server to Service mode 3 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 4 Checking the firmware version of the PCI expansion unit 5 Checking the operating condition of the system and the I O device usage status 6 Powering Off the Physical Partition Requiring Maintenance 7 Enabling access to the FRU requiring maintenance 8 Replacing a faulty link card 9 Restoring the System 10 Performing a diagnosis test 11 Confirming that there is no problem with the replacement FRU 12 Checking the firmware version and matching it with the pre replacement one 13 Returning the mode switch on the operation panel to Locked mode 14 Powering On the Physical Partition Requiring Maintenance 15 Closing the rack door Chapter 7 Maintenance Flow 153 Table 7 22 Item Task 1 Opening the rack door 2 Switching the mode switch on the 10 11 12 13 14 15 154 operation panel of the destination server to Service mode Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the firmware version of the PCI expansion unit Checking the operating condition of the system and the I O device usage status Powering Off the Physical Partition Requiring Maintenance Enabling access to the FRU requiring m
164. nsion Unit Requiring Maintenance and the Destination Physical Partition 5 3 Checking the Firmware Version of the PCI Expansion Unit 5 4 1 Checking the operating condition of the physical partition or logical domain 5 4 2 Checking the assignment status of I O devices 5 5 1 Releasing the link card or PCIe card using PCI hot plug PHP 5 7 Accessing an FRU Chapter 8 Maintaining the PCI Express Cards Chapter 9 Maintaining the Link Board Chapter 11 Maintaining the Link Cables Chapter 12 Maintaining the Management Cable Chapter 13 Maintaining the Power Supply Units Chapter 14 Maintaining the Fan Units Chapter 15 Maintaining the PCI Tray Chapter 16 Maintaining the I O Board Chapter 17 Maintaining the Fan Backplane 6 1 1 Installing the front cover 6 3 1 Mounting a link card using PCI hot plug PHP 6 1 3 Fixing the cable support 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Chapter 7 Maintenance Flow 113 Cold replacement releasing the link card using DR Cold maintenance can be performed for all the FRUs mounted on the PCI expansion unit after releasing the chassis in which the link card is mounted from the system using physical partition dynamic reconfiguration DR This can be done only when the connection destination of the PCI expansion unit has the building block configuration of the SPARC M10 4S Figure 7
165. nt configuration information ldm 1ls spconfig factory default testl test2 test3 current If the current configuration is shown as next poweron execute the Idm add spconfig command to save the current configuration information This is because the configuration information stored in the XSCF is different from that stored in the control domain In the following example the current configuration information is saved in test4 because test3 is next poweron ldm 1ls spconfig factory default testl test2 test3 next poweron ldm add spconfig test4 Chapter1 Before Starting Maintenance Work 9 10 ldm 1ls spconfig factory default testl test2 test3 test4 current Execute the Idm set spconfig command to specify the configuration information to be set The following example specifies test1 ldm set spconfig testl ldm 1ls spconfig factory default testl next poweron test2 test3 Execute the poweroff and poweron commands of the XSCF firmware to turn off on the power to the physical partition Before turning off the power to the system stop the logical domain with an appropriate procedure and use the Idm unbind command to transfer it to the inactive state The following example shows that the power to PPAR 0 is turned off on XSCF gt poweroff p 0 XSCF gt poweron p 0 4 Execute the Idm Is spconfig command to confirm that the configuration information is set as specified
166. nt des marques d pos es de SPARC International Inc aux Etats Unis et dans d autres pays Tout autre nom mentionn peut correspondre a des marques appartenant a leurs propri taires respectifs Si ce logiciel ou la documentation qui l accompagne est conc d sous licence au Gouvernement des Etats Unis ou toute entit qui d livre la licence de ce logiciel ou l utilise pour le compte du Gouvernement des Etats Unis la notice suivante s applique U S GOVERNMENT END USERS Oracle programs including any operating system integrated software any programs installed on the hardware and or documentation delivered to U S Government end users are commercial computer software pursuant to the applicable Federal Acquisition Regulation and agency specific supplemental regulations As such use duplication disclosure modification and adaptation of the programs including any operating system integrated software any programs installed on the hardware and or documentation shall be subject to license terms and license restrictions applicable to the programs No other rights are granted to the U S Government Avis de non responsabilit les seules garanties octroy es par Oracle et Fujitsu Limited et ou toute soci t affili e de l une ou l autre entit en rapport avec ce document ou tout produit ou toute technologie d crits dans les pr sentes correspondent aux garanties express ment stipul es dans le contrat de licence r gissant le produit
167. nting a link card using PHP When mounting the link card on the system using PHP do so with the PCIe card of the PCI expansion unit pulled out Precautions for link cable replacement Do not place any heavy objects on the link cables When performing replacement replace the two link cables as a set Do not bend the link cables unduly Excessive bending could damage the link cables The allowable bending radius for each link cable is shown below Link cable electrical Static dynamic 40 mm 1 6 in or greater Link cable optical Static 35 mm 1 4 in or greater dynamic 58 mm 2 3 in or greater Before use confirm that labels A in the figure are affixed to the connectors at both ends of the link cable For the location of the labels see Figure 4 3 and Figure 4 4 Figure 4 2 Label location on a link cable electrical Kh 46 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 4 3 Label location on a link cable optical If there is no label affix one of the labels supplied with the link cable Affix labels with the same number 0 or 1 to the connectors at both ends of the link cable Figure 4 4 State of labels when correctly attached Notes on power supply unit replacement Hot maintenance is supported only for a redundant configuration The power supply has a redundant configuration featuring two units Thus the system can continue to function even if one o
168. o XML execute the ldm list constraints x command and to restore it from XML execute the Idm init system i command Also save the setting information for the OpenBoot PROM environment variable of the control domain to the notepad and then reset it To display it execute the printenv command while the ok prompt is displayed For details on these procedures see 1 7 3 How to save restore the logical domain configuration information and the OpenBoot PROM environment variable When executing the setpciboxdio command to change the enable disable setting for the direct I O function of the PCI expansion unit the following operations must be performed to save restore each item of information Table 1 3 Operations to change the enable disable setting with the setpciboxdio command PCl expansion unit Domain configuration Reconfiguration of Resetting of OpenBoot configuration Oracle VM Server for PROM environment SPARC config variable Existence non factory default Unnecessary Necessary existence Existence non Contains a guest domain Necessary XML Necessary existence Suppose that the setpciboxdio command is executed to install remove or replace the PCI expansion unit for the PCIe slot of the SPARC M10 chassis for which the direct I O function of the PCI expansion unit is enabled The following operations must be performed to save restore each item of information Note If you use the PCI Hot Plug PHP function to perform maintenance o
169. oblem with the replacement FRU 11 Closing the rack door End Chapter 7 Maintenance Flow 131 Table 7 14 Item 1 2 10 11 132 Task Opening the rack door Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the operating condition of the system and the I O device usage status Releasing the link card mounted on the server from the system using PCI hot plug PHP Lowering the cable support Removing the power cords of the PCI expansion unit Removing the PCle card Mounting the released link card on the system using PCI hot plug PHP Fixing the cable support Confirming that there is no problem with the replacement FRU Closing the rack door Cold removal work procedure Releasing the link card using PHP Reference 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 4 1 Checking the operating condition of the physical partition or logical domain 5 4 2 Checking the assignment status of I O devices 5 5 1 Releasing the link card or PCIe card using PCI hot plug PHP 5 7 1 Lowering the cable support 5 7 2 Removing the power cord Chapter 8 Maintaining the PCI Express Cards 6 3 1 Mounting a link card using PCI hot plug PHP 6 1 3 Fixing the cable support 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10
170. ormed using PHP from the root domain See 7 in the execution example of the Idm list io 1 command Other than above when the assignment to the I O domain is made using direct I O Active hot maintenance can be performed using PHP after returning the PCle slot from the I O domain to the control domain or root by using the dynamic reconfiguration function for PCIe end point devices See 6 and 8 in the execution example of the Idm list io command However this is supported only by Oracle VM Server for SPARC 3 1 1 1 or later 3 3 3 Checking whether the SR IOV function is used This section checks whether the SR IOV function is used Chapter 3 Types of Maintenance 35 36 Note For details on the PCIe card supporting the SR IOV function see Appendix B Cards that Support SR IOV in the Fujitsu M10 SPARC M10 Systems PCI Card Installation Guide 1 Log in to the XSCF shell 2 Execute the showlogs command to identify the PCle card requiring maintenance XSCF gt showlogs error Connect to the control domain console of the physical partition Execute the Idm list io command to determine whether the SR IOV function is being used VF displayed in TYPE indicates that the SR IOV function is being used ldm list io DOMAIN STATUS primary primary primary primary primary primary primary primary primary primary primary primary primary primary primary primary primary primary primary primary primary pri
171. oving a PCI Express card cassette 218 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 2 Remove the front cover For details see 5 7 3 Removing the front cover 3 Remove all the fan units For details see 14 3 2 Removing a fan unit 4 Remove the PCI tray For details see 15 3 2 Removing the PCI tray 16 3 2 Removing the I O board 1 Remove the six cables connected to the I O board Note Record the positions of the cables before removing them to ensure that they are reinstalled correctly Chapter 16 Maintaining the I O Board 219 Figure 16 2 T O board cables 2 Loosen the two screws securing the I O board 220 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 16 3 I O board screws 3 Hold and slide the guide A in the figure of the I O board toward the rear of the chassis and then remove the board Note Do not remove the I O board by holding its connectors Note Place the removed I O board on the grounded ESD mat to eliminate any static electricity Chapter 16 Maintaining the I O Board 221 Figure 16 4 I O board removal 16 4 Installing the I O Board This section describes the procedure for installing the I O board 16 4 1 Installing the I O board 1 Install the I O board adjusting it to align with the six stoppers Hold and slide the guide to the front of the chassis 222 PCI Expansion Unit for Fujitsu M10 SP
172. ower cords 6 1 3 Fixing the cable support 6 3 Diagnosing a Replacement FRU in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 4 Diagnosing a Replacement FRU in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 7 Checking the FRU Status after Maintenance in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 6 Returning the Mode Switch to Locked Mode in the Fujitsu M10 4 Fujitsu M10 45 SPARC M10 4 SPARC M10 4S Service Manual 6 8 Returning the Mode Switch to Locked Mode in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 8 Starting the Entire System in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 9 Starting the System in the Fujitsu M10 1 SPARC M10 1 Service Manual PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Cold addition releasing the link card using PHP Cold addition of a PCI Express card to the PCI expansion unit can be performed after releasing the link card mounted on the destination server from the system using the PCI hot plug PHP This can be done only when the connection destination of the PCI expansion unit is a SPARC M10 4 M10 4S and the setting for the direct I O function is disabled Figure 7 8 Cold addition flow releasing the link card using PHP 1 Opening the rack door 2 Locatin
173. pci 8000 pci 4 pci 0 pci 9 network 0 network 0 1 BBO CMUL SASHBA PCIE PCIEO primary Occ pci 8000 pcit 4 pci 0 pci d scsi 0 iport f disk w50000393a82368b2 0 scsi 0 iport f smp w500000e0e06d03bf scsi 0 iport f enclosure w500000e0e06d03bd 0 scsi 0 iport v0 BBO PCIO PCIE PCIE1 primary OCC lt 5 pci 8100 pcit 4 pci 0 pci d BBO PCI3 PCIE PCIE2 primary EMP pci 8200 pcit 4 pci 0 pci d BBO PCI4 PCIE PCIE2 iodoml OCC lt 6 pci 8200 pcit 4 pci 0 pci s BBO PCI7 PCIE PCIE3 rootdom EMP pci 8300 pcit 4 pci 0 pci d BBO PCI8 PCIE PCIE3 rootdom EMP lt 7 pci 8300 pcit 4 pci 0 pci s BBO CMUL NET2 PCIE PCIE4 primary OCC pci 8400 pcit 4 pci 0 pci a network 0 network 0 1 BBO PCI1 PCIE PCIES rootdom EMP pci 8500 pcit 4 pci 0 pci s BBO PCI2 PCIE PCIES rootdom OCC pci 8500 pcit 4 pci 0 pci 9 BBO PCI5 PCIE PCIE6 iodom2 OCC lt 8 pci 8600 pci 4 pci 0 pci 9 BBO PCI6 PCIE PCIE6 rootdom OCC 34 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 pci 8600 pci 4 pci 0 pci l1l1 BBO PCI9 PCIE PCIE7 primary OCC pci 8700 pci 4 pci 0 pci 9 BBO PCI10 PCIE PCIE7 primary OCC pci 8700 pci 4 pci 0 pci ll1 Logical domain to which the PCle card is assigned The name indicated in DOMAIN of the slot line where NAME indicates BBX PCIx of the PCIe card requiring maintenance indicates the logical domain to which that PCle card is assigned 5 to
174. peration panel of the destination server to Service mode 3 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 4 Checking the operating condition of the system and the I O device usage status 5 Stopping the system 6 Enabling access to the FRU requiring maintenance 7 Removing the link card 8 Restoring the System 9 Performing a diagnosis test i 10 Confirming that there is no problem with the replacement FRU 11 Returning the mode switch on the operation panel to Locked mode 12 Starting the system 13 Closing the rack door End Table 7 27 Cold removal work procedure removing the link card after stopping the system Item Task Reference 1 Opening the rack door Chapter 7 Maintenance Flow 169 Table 7 27 Item 2 10 11 12 13 170 Task Switching the mode switch on the operation panel of the destination server to Service mode Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the operating condition of the system and the I O device usage status Stopping the system Enabling access to the FRU requiring maintenance Removing the link card Restoring the System Performing a diagnosis test Confirming that there is no problem with the replacement FRU Returning the mode switch on the operation panel to Locked mode Starting the system
175. quires maintenance This function is also referred to as the locator Indicates the normal state Or the breaker is open or the power supply is otherwise off Table 2 5 LEDs and status of link board Name LINK STATUS PCI Ex left side LINK STATUS Management right side Color Green Green Status On Blinking 1 Off On Blinking 1 Off 1 The blink interval is 1 second 1 Hz Chapter 2 Description The link is established with PCI Express Gen3 x8 The link is established with other than PCI Express Gen3 x8 degradation condition The link is down The management link is established The management link is disconnected Indicates that power is not being supplied Understanding the PCI Expansion Unit Components 23 Table 2 6 LEDs and status of PCIe slot Name Color Status Description FOWER Green On Indicates that power is being supplied D Off Indicates that power is not being supplied ATTENTION Amber On Indicates that an error has occurred A Blinking Indicates that the component requires maintenance 1 This function is also referred to as the locator Off Indicates the normal state 1 The blink interval is 1 second 1 Hz 2 2 3 LEDs on link card The link card is mounted in the PCIe slot of the server 24 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 2 7 LEDs of link card SPARC M10 1 SPARC M10 4 M10 4S 1
176. r stopping the system Cold addition releasing the link card using PHP Cold addition releasing the link card using DR Cold addition releasing the link card after stopping the system Cold addition of a PCI Express card to the PCI expansion unit can be performed after stopping the system to which the PCI expansion unit is connected This can be done when the connection destination of the PCI expansion unit is a SPARC M10 1 M10 4 M10 4S PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 7 7 Cold addition flow releasing the link card after stopping the system 1 Opening the rack door 2 Switching the mode switch on the operation panel of the destination server to Service mode 3 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 4 Checking the operating condition of the system and the I O device usage status 5 Stopping the system 6 Lowering the cable support 7 Removing the power cords of the PCI expansion unit 8 Installing the target FRU 9 Installing the power cords of the PCI expansion unit 10 Fixing the cable support 11 Diagnosing the installed FRU 12 Confirming that there is no problem with the installed FRU 13 Returning the mode switch on the operation panel to Locked mode 14 Starting the system 15 Closing the rack door End Chapter 7 Maintenance Flow 119 Table 7
177. re Maintaining a Link Board Configuration of the Link Board Removing the Link Board Installing the Link Board 9 1 Before Maintaining a Link Board This chapter describes only the configuration removal work and mounting work of a link board Before removal work of an FRU perform necessary work items while referencing Chapter 7 Maintenance Flow For maintenance types of the link board see Link board in Table 7 2 List of available maintenance types by FRU 9 2 Configuration of the Link Board This section describes the configuration and location of the link board The link board is mounted in a dedicated slot among the PCI Express PCIe slots Connect the link board to the link card mounted on the SPARC M10 1 M10 4 M10 4S with the link cables and the management cable 181 Figure 9 1 Location of the link board Location number Component 1 Link board LINKBD 9 3 A 9 3 1 Removing the Link Board This section describes the preparations that must be completed prior to removing the link board Enable removal of the link board before attempting to remove it For details see Chapter 7 Maintenance Flow Caution Before you handle any components wear a wrist strap to ground any static electricity If you perform this procedure without a wrist strap individual components or the overall system may be damaged For details see 1 4 Notes Regarding Static Electricity Accessing
178. ress PCIe slot of the SPARC M10 1 or SPARC M10 4 M10 4S The procedure for installing the link card is the same as that for installing a PCIe card to be mounted on the SPARC M10 1 or SPARC M10 4 M10 4S For details on the SPARC M10 1 see 8 3 Removing a PCI Express Card in the Fujitsu M10 1 SPARC M10 1 Service Manual For details on the SPARC M10 4 M10 4S see 8 4 Removing a PCI Express Card in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Mounting a link card using PCI hot plug PHP This section describes the procedure for mounting a link card on the server using PCI hot plug PHP For details on the cfgadm 1M command used here see 2 Dynamically Configuring Devices in the Managing Devices in Oracle Solaris 11 2 In the case of Oracle Solaris 10 see SCSI Hot Plugging With the cfgadm Command Task Map in the Oracle Solaris Administration Devices and File Systems Note If you use PCI hot plug PHP to mount the link card on the server pull all the PCIe cassettes containing PCIe cards out of the PCI expansion unit in advance Mount the link card on the server before installing the pulled out PCle cassettes in the PCI expansion unit Then use PHP to incorporate them into the server 1 Login as an Oracle Solaris super user 80 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 2 Execute the svcadm enable hotplug command to activate the hotplug service svcadm en
179. rform maintenance work on the system xi Related Documentation All documents for your server are available online at the following locations Sun Oracle software related documents Oracle Solaris etc http www oracle com documentation Fujitsu documents Japanese site http jp fujitsu com platform server sparc manual Global site http www fujitsu com global services computing server sparc downloads manual The following table lists documents related to SPARC M10 Systems SPARC M10 Systems related documentation 1 Fujitsu M10 SPARC M10 Systems Getting Started Guide 2 Fujitsu M10 SPARC M10 Systems Quick Guide Fujitsu M10 SPARC M10 Systems Important Legal and Safety Information 2 Software License Conditions for Fujitsu M10 SPARC M10 Systems Fujitsu M10 SPARC M10 Systems Safety and Compliance Guide Fujitsu M10 SPARC M10 Systems Security Guide Fujitsu M10 SPARC M10 Systems SPARC Enterprise PRIMEQUEST Common Installation Planning Manual Fujitsu M10 SPARC M10 Systems Installation Guide Fujitsu M10 1 SPARC M10 1 Service Manual Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Crossbar Box for Fujitsu M10 SPARC M10 Systems Service Manual PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual Fujitsu M10 SPARC M10 Systems PCI Card Installation Guide Fujitsu M10 SPARC M10 Systems System Operation and Administration Guide Fujitsu M10 SPARC M10 Systems Domain Configuration Guide Fu
180. ring the PCI expansion unit Mounting the released link card on the system using physical partition dynamic reconfiguration DR Confirming that there is no problem with the replacement FRU Closing the rack door Cold replacement work procedure Releasing the link card using DR Reference 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 3 Checking the Firmware Version of the PCI Expansion Unit 5 4 Checking the Operating Condition and Resource Usage Status 5 5 2 Releasing the link card or PCIe card using physical partition dynamic reconfiguration DR 5 7 Accessing an FRU Chapter 8 Maintaining the PCI Express Cards Chapter 9 Maintaining the Link Board Chapter 11 Maintaining the Link Cables Chapter 12 Maintaining the Management Cable Chapter 13 Maintaining the Power Supply Units Chapter 14 Maintaining the Fan Units Chapter 15 Maintaining the PCI Tray Chapter 16 Maintaining the I O Board Chapter 17 Maintaining the Fan Backplane 6 1 Restoring the Chassis of the PCI Expansion Unit 6 3 2 Mounting the link card on the server using physical partition dynamic reconfiguration DR 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Chapter 7 Maintenance Flow 115 TZ FRU Installation Workflow for the PCI Expansion Unit This section describes the installatio
181. rm that the connectors on both the chassis and PCIe card cassette do not have any bent pins and that the pins are aligned correctly If you attempt to install a PCle card cassette while any of the connector pins are bent the chassis or the PCIe card cassette may be damaged When performing installation perform the work carefully so as not to bend any pins 1 Hold the lever of the PCle card cassette and carefully insert the cassette into the PC slot Note Slide the PCle card cassette along the rail at the bottom of the PCle slot Note Do not forcibly push the PCle card cassette into the PCI slot Using excessive force may damage the component or the chassis Chapter8 Maintaining the PCI Express Cards 179 2 Lower the lever while pressing the latch of the PCle card cassette to secure it Note Confirm that the PCIe card cassette is securely and correctly inserted 8 4 3 Restoring the chassis 1 Connect all the cables to the PCle card cassette Note Connect the cables in their original positions by referring to the record that you made before the start of maintenance Now the work for mounting the FRU is complete While referencing Chapter 7 Maintenance Flow continue the maintenance work 180 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 9 Maintaining the Link Board This chapter describes the procedure for maintaining a link board mounted in the PCI expansion unit Befo
182. rom the root or control domain 1 Stopping the use of the I O device If there is any logical domain that is physically or virtually using the PCIe card mounted in the chassis requiring maintenance stop the use of the I O device to keep it unused Releasing the I O device in the case of a redundant configuration With the duplication function release the target I O device so that it will not be used For details on the work see the manual for the application being used Stop the use of the I O device in the case of a non redundant configuration Place the I O device in the unused state such as by deactivating the network or unmounting the disk For details on the work see the document of Oracle Solaris or the manual for the application being used 2 Deleting the assignment of the I O In the case that a virtual I O is used If the virtual I O service is used delete the virtual I O device For details see the Oracle VM Server for SPARC Administration Guide released by Oracle In the case that the PCle end point assignment is used If the PCle end point device of the chassis requiring maintenance is assigned to an T O domain delete the PCIe end point device from the I O domain and return it to the root domain including the control domain The work procedure depends on whether the dynamic reconfiguration function for PCIe end point devices is used For details see Table 5 2 62 PCI Expansion Unit for Fujitsu M10 SPARC M10 Sys
183. rsion of the link card CARD shown on Type and that of the I O board IOBOARD shown on Type The following example shows that 2007 is the last four digits of the serial number of the PCI expansion unit and 1080 is the firmware version of the link card and the I O board XSCF gt ioxadm v list Location Type FW Ver Serial Num Part Num State PCIBOX 2007 PCIBOX PZ21242007 On PCIBOX 2007 PSU 0 PSU FEJD1212000521 CA01022 0750 D 7060988 On PCIBOX 2007 PSU 1 PSU FEJD1201000738 CA01022 0750 D 7060988 On 96 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 PCIBOX 2007 I0OB IOBOARD 1080 PP12470297 CA20365 B66X 010AJ 7061033 On PCIBOX 2007 1 LINKBD BOARD PP1244027P CA20365 B60X 001AA 7061035 On PCIBOX 2007 BB 00 PCI 10 FANBP FANBP PP12470298 CA20365 B68X O005AD 7061025 On CARD 1080 PP124401L2Z CA20365 B59X 001AA 7061040 On 6 6 2 6 6 3 6 6 4 A firmware version is displayed as a four digit release number in xyyz format Each release number is indicated in xyyz format as follows x Major release number yy Minor release number z Micro release number Time required for updating The following shows the time required for updating Table 6 5 Time required for updating System status Updating time The system is operating with a PCI expansion unit recognized from the About 30 XSCF minutes Flow of update process The following table lists the sequenc
184. rt 6 3 3 Mounting a PCIe card on the server using PCI hot plug PHP 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual 6 7 Checking the FRU Status after Maintenance in the Fujitsu M10 1 SPARC M10 1 Service Manual 6 5 Turning off the CHECK LED Locator of the PCI Expansion Unit Chapter 7 Maintenance Flow 105 Hot replacement for a power supply unit or fan unit Hot replacement is applicable to the power supply unit and fan unit Perform replacement according to the following procedure Figure 7 2 Hot replacement flow for a power supply unit fan unit 1 Opening the rack door 2 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 3 Checking the operating condition of the system and the I O device usage status 4 Stopping the power supply to the FRU requiring maintenance 5 Enabling access to the FRU requiring maintenance 6 Replacing a faulty FRU 7 Restoring the chassis requiring maintenance 8 Confirming that there is no problem with the replacement FRU 9 Turning off the CHECK LED locator of the PCI expansion unit requiring maintenance 10 Closing the rack door End 106 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 7 4 Item Task 1 Opening the rack door 2 Locating the PCI expansion unit requiring
185. s This section describes the configuration and location of the PCIe cards Up to eleven 11 PCIe cards can be mounted After maintenance return the PCIe cards to their original mounting locations 171 Figure 8 1 Location of PCIe card O a lo iy Vm 9 11 Location number Component PCIe card PCIECS 1 PCle card PCIECS 2 PCle card PCIECS 3 PCIe card PCIECS 4 PCIe card PCIECS 5 PCle card PCIECS 6 PCle card PCIECS 7 PCIe card PCIECS 8 PCle card PCIECS 9 PCIe card PCIECS 10 PCle card PCIECS 11 oO AN DD oO FF QO NY e e pe O 8 3 Removing a PCI Express Card This section describes the procedure for removing a PCIe card Perform reduction using the same procedure 172 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 8 3 1 8 3 2 For expansion remove the PCle card filler unit rather than a PCle card itself Enable removal of the PCIe card before attempting to remove it For details see Chapter 7 Maintenance Flow Caution Before you handle any components wear a wrist strap to ground any static electricity If you perform this procedure without a wrist strap individual components or the overall system may be damaged For details see 1 4 Notes Regarding Static Electricity Accessing a PCI Express card cassette 1 Remove all the cables connected to the PCle cassette requiring maintenance Note Record the positions of the cables before r
186. s been saved to the XSCF ldm add spconfig test1 ldm 1ls spconfig factory default testl current 26 If there are multiple pieces of configuration information to be restored restore all of them Repeat from step 14 to step 25 to restore the configuration information 16 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 2 Understanding the PCI Expansion Unit Components This chapter describes the components of the PCI expansion unit Before starting any maintenance work check and understand the configuration of the components and the LED indications Identifying the Names and Locations of Components m Checking the Status with the LED For the specifications of each component see Appendix B Component Specifications 2 1 Identifying the Names and Locations of Components This section describes the names and locations of each component Components that can be accessed from the front Figure 2 1 Location of components that can be accessed from the front Location Component number dl Fan unit Components that can be accessed from the rear Figure 2 2 Location of components that can be accessed from the rear 1 2 3 18 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Location Component number 1 Power supply unit 2 Link board 1 3 PCI Express P
187. s of maintainable FRUs and maintenance types Table 3 1 When the connection destination of the PCI expansion unit is a SPARC M10 1 Not maintainable FRU Hot maintenance Cold maintenance Reference PCI Express card 1 Enabled 2 Enabled Chapter 8 Link board Enabled Chapter 9 Link cable Enabled Chapter 11 Management cable Enabled Chapter 12 Power supply unit Enabled 3 Enabled Chapter 13 Fan unit Enabled Enabled Chapter 14 PCI tray Enabled Chapter 15 T O board Enabled Chapter 16 Fan backplane Enabled Chapter 17 1 The maintenance types of the PCIe card vary depending on the status of the PCIe card requiring maintenance For details see Table 3 5 2 Maintenance work is performed using PCI hot plug PHP 3 This is supported only for a redundant configuration Table 3 2 When the connection destination of the PCI expansion unit is a SPARC M10 4 M10 4S in a single unit configuration Not maintainable FRU Hot maintenance Cold maintenance Reference PCI Express card 1 Enabled 2 Enabled Chapter 8 Link board Enabled 3 Enabled Chapter 9 Link cable Enabled 3 Enabled Chapter 11 Management cable Enabled 3 Enabled Chapter 12 Power supply unit Enabled 4 Enabled Chapter 13 Fan unit Enabled Enabled Chapter 14 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 3 2 When the connection destination of the PCI expansion unit is a SPARC M10 4 M10 4S in a single unit config
188. s the procedure for installing the PCI tray 15 4 1 Installing the components of the PCI tray 1 Install the I O board on the replaced PCI tray For details see 16 4 1 Installing the I O board 2 Install the fan backplane on the PCI tray For details see 17 4 1 Installing the fan backplane 214 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 15 4 2 15 4 3 Installing the PCI tray 1 Hold the PCI tray with both hands and carefully insert it into the chassis Caution Confirm that the PCI tray is inserted and securely fixed 2 Tighten the two screws holding the PCI tray Restoring the chassis 1 Install all the fan units For details see 14 4 1 Installing a fan unit 2 Install the front cover For details see 6 1 1 Installing the front cover 3 Install all the components listed below Power supply unit For details see 13 4 Installing a Power Supply Unit PCle card cassette For details see 8 4 2 Installing a PCI Express card cassette Now the work for mounting the FRU is complete While referencing Chapter 7 Maintenance Flow continue the maintenance work Chapter 15 Maintaining the PCI Tray 215 216 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 16 Maintaining the I O Board This chapter describes the procedure for maintaining the I O board mounted in the PCI expansion unit Before Maintaining the I O
189. screws holding the cable support 2 Lower the cable support PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Lowering the cable support if it is a new model 1 Loosen the right and left screws A in Figure 5 2 fixing the cable support Figure 5 2 Locations of screws holding the cable support 2 Lift the cable support to release the hooks B in Figure 5 3 and pull it out Figure 5 3 Releasing the cable support 3 Lower the cable support Chapter5 Preparations Required for Maintenance 71 5 7 2 Figure 5 4 Handling the cable support Removing the power cord Remove the cable clamp from each power cord and then remove the power cords 1 Release the latch A in the figure of the cable clamp Pulling the cable clamp toward the rear of the chassis facilitates the release of the latch Figure 5 5 Removing the cable clamp 2 Remove the power cord from the power supply unit 72 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 5 6 Removing the power cords 5 7 3 Removing the front cover To perform maintenance on any of the following components on the front panel or inside the chassis remove the front cover Fan unit PCI tray I O board 1 Release the slide locks on both the left and right sides of the front cover and
190. sical partition XSCF gt poweroff p 0 13 In accordance with the model in use restore the configuration information of the logical domain from the XML file For the SPARC M10 1 When updating from the firmware version XCP2043 or earlier to the firmware version XCP2044 or later in the system with the PCI expansion unit restore the configuration information of the logical domain from the XML file in step 14 and later For details on the firmware update see the Fujitsu M10 SPARC M10 Systems System Operation and Administration Guide For the SPARC M10 4 M10 4S When executing the setpciboxdio command to change the enable disable setting for the direct I O function of the PCI expansion unit restore the 12 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 configuration information of the logical domain from the XML file in step 14 and later For details on the setpciboxdio command see the Fujitsu M10 SPARC M10 Systems XSCF Reference Manual 14 Execute the showdomainconfig command of the XSCF firmware to confirm that the configuration information at the next startup of the physical partition is the factory default The following example shows that the configuration information at the next startup is the factory default XSCF gt showdomainconfig p 0 PPAR ID Ho Booting config Current factory default factory default config name factory default domains sal date _created The fo
191. sign the physical I O device 1 Log in to Oracle Solaris on the control domain of the physical partition 2 Transition the control domain to delayed reconfiguration mode This step is unnecessary if root complexes are dynamically reconfigured Perform step 3 ldm start reconf lt control domain name gt 3 Reassign the physical I O device to the control domain ldm add io lt physical I O device name gt lt control domain name gt 4 Restart Oracle Solaris on the control domain This step is unnecessary if the dynamic reconfiguration function for root complexes is used shutdown i6 g0 y Note When set to delayed reconfiguration mode the control domain does not have control over the guest domains If you have set the control domain to delayed reconfiguration mode restart the control domain as soon as possible 6 4 2 Work for returning the root complex to the root domain Return the physical I O device root complex released from the root domain to the root domain 88 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 6 2 Differences in the procedures according to whether the dynamic reconfiguration function for root complexes is used Unnecessary Item Task Command Dynamic Dynamic reconfiguration reconfiguration is used 1 is not used 2 1 Assigning the physical I O device root ldm add io Perform Perform complex to the root domain 2 Starting the root domain ldm st
192. stalling the front cover 6 1 2 Installing the power cords If the power cords were removed to perform maintenance reinstall the power cords in the PCI expansion unit and then secure them with the cable clamp 1 Install the power cords to the power supply unit of the PCI expansion unit Figure 6 2 Installing the power cords 76 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 2 Clip the power cords with the cable clamp and secure the cable clamp Lock the latch A in the figure and then push the cable clamp toward the front of the chassis to firmly secure the cable clamp Figure 6 3 Locking the cable clamp Fixing the cable support If the cable support was lowered to perform maintenance lift the cable support and fix it in place Note There are two types of cable supports Distinguish them by the locations of the screws fixing the cable support For details on the locations of the screws see Figure 5 1 and Figure 5 2 Fixing the cable support 1 Lift the cable support and lock it in place by pushing the upper and lower screw fittings toward the back direction of the arrow Chapter6 Understanding the Preparations for Restoring the System 77 Figure 6 4 Locking the cable support 2 Tighten the right and left screws to fix the cable support in place Note Check that the cable support is firmly attached and fixed Fixing the cable support if i
193. start the system For a building block configuration specify the physical partition connected to the PCI expansion unit and start the system In the following example PPAR ID 0 is specified XSCF gt poweron p 0 5 If multiple PCI expansion units are connected execute the ioxadm list command to check the combination of the link card and the I O board XSCF gt ioxadm list PCIBOX Link PCIBOX 2007 BB OO PCI 10 PCIBOX 2006 BB 00 PCI 09 PCIBOX 2005 BB 00 PCIL 08 PCIBOX 2004 BB 00 PCI 07 6 Execute the ioxadm v list command to check the serial number and the firmware version of the PCI expansion unit Location displays the last four digits of the serial number of the PCI expansion unit and FW Ver displays the firmware version of the link card CARD shown on Type and that of the I O board IOBOARD shown on Type The following example shows that 2007 is the last four digits of the serial number of the PCI expansion unit and 1080 is the firmware version of the link card and the I O board XSCF gt ioxadm v list Location Type FW Ver Serial Num Part Num State PCIBOX 2007 PCIBOX PZ21242007 On PCIBOX 2007 PSU 0 PSU FEJD1212000521 CA01022 0750 D 7060988 On PCIBOX 2007 PSU 1 PSU FEJD1201000738 CA01022 0750 D 7060988 On Chapter 6 Understanding the Preparations for Restoring the System 95 PCIBOX 2007 IOB IOBOARD 1080 PP12470297 CA20365 B66X 010AJ 7061033 On PCIBOX 2007 LINKBD BOARD PP1244027P CA20365
194. stem 83 84 switch the chassis to the master XSCF XSCF gt switchscf t Standby 8 Add the root complex that was deleted from the control domain before maintenance For details see 6 4 1 Returning the root complex to the control domain 9 Start the root domain to which the root complex was assigned before maintenance from the unbind state For details see 6 4 2 Work for returning the root complex to the root domain 10 If you have replaced a link card or I O board match the firmware version with the one before replacement a Check the firmware version of the PCI expansion unit For details see Check method when the system has been started in 6 6 1 How to check the serial number and firmware version of the PCI expansion unit If the firmware version is the same as the one before replacement go to step 11 b If the firmware version is different from the one before replacement match it with the one before replacement See 6 6 4 Updating the firmware for details to perform work up to step 3 c To reflect the firmware log in to the root domain and restart Oracle Solaris telnet localhost lt port number gt shutdown i6 g0 y d Confirm that the firmware version is now the same as the one before replacement For details see Check method when the system has been started in 6 6 1 How to check the serial number and firmware version of the PCI expansion unit 11 Install all the PCle card cassett
195. t special incidental or consequential damages even if advised of the possibility of such damages DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS REPRESENTATIONS AND WARRANTIES INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE OR NON INFRINGEMENT ARE DISCLAIMED EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID Copyright 2007 2014 Fujitsu Limited Tous droits r serv s Oracle et ou ses affili s ont fourni et v rifi des donn es techniques de certaines parties de ce composant Oracle et ou ses affili s et Fujitsu Limited d tiennent et contr lent chacun des droits de propri t intellectuelle relatifs aux produits et technologies d crits dans ce document De m me ces produits technologies et ce document sont prot g s par des lois sur le droit d auteur des brevets et d autres lois sur la propri t intellectuelle et des trait s internationaux Ce document le produit et les technologies aff rents sont exclusivement distribu s avec des licences qui en restreignent l utilisation la copie la distribution et la d compilation Aucune partie de ce produit de ces technologies ou de ce document ne peut tre reproduite sous quelque forme que ce soit par quelque moyen que ce soit sans l autorisation crite pr alable d Oracle et ou ses affili s et de Fujitsu Limited et de leurs ventuels conc dants de licence Ce document bien qu il vous ait
196. t 77 6 2 Supplying Power to the FRU Requiring Maintenance 79 6 3 Mounting a Link Card or PCIe Card on the System 80 6 3 1 6 3 2 6 3 3 Mounting a link card using PCI hot plug PHP 80 Mounting the link card on the server using physical partition dynamic reconfiguration DR 82 Mounting a PCIe card on the server using PCI hot plug PHP 85 6 4 Restoring the Logical Domain to the Status It Had before Maintenance 87 6 4 1 6 4 2 6 4 3 6 4 4 6 4 5 Returning the root complex to the control domain 87 Work for returning the root complex to the root domain 88 Work to return the physical I O device to the I O domain 89 Work to restore the SR IOV function 91 Work to restore a virtual device 93 6 5 Turning off the CHECK LED Locator of the PCI Expansion Unit 94 6 6 Updating the Firmware of the PCI Expansion Unit 94 6 6 1 6 6 2 6 6 3 6 6 4 How to check the serial number and firmware version of the PCI expansion unit 94 Time required for updating 97 Flow of update process 97 Updating the firmware 97 Contents v Chapter 7 Maintenance Flow 101 7 1 FRU Replacement Workflow for the PCI Expansion Unit 102 7 1 1 Hot replacement 103 7 1 2 Cold replacement 108 7 2 FRU Installation Workflow for the PCI Expansion Unit 116 7 2 1 Hot addition 116 7 2 2 Cold addition 118 7 3 FRU Removal Workflow for the PCI Expansion Unit 125 7 3 1 Hot removal 125 7 3 2 Cold removal 128 74 Link Card Maintenance Workflow 135 7 4 1 Active maint
197. t is a new model 1 Lift the cable support and fasten the hooks B in Figure 6 5 on the screws C in Figure 6 5 78 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 6 5 Handling the cable support aS pee f oneone N AV 2 Tighten the right and left screws A in Figure 6 6 to fix the cable support in place Figure 6 6 Locking the cable support Wp DOD COROA N Z PAY y eN 6 2 Supplying Power to the FRU Requiring Maintenance This section describes the procedure for supplying power to the FRU requiring maintenance using the ioxadm command 1 Log in to the XSCF shell 2 Execute the ioxadm command to supply power to the FRU requiring Chapter6 Understanding the Preparations for Restoring the System 79 maintenance In the following example PSU 0 of PCIBOX 12B4 is specified XSCF gt ioxadm poweron PCIBOX 12B4 PSU 0 In the following example FAN 0 of PCIBOX 12B4 is specified XSCF gt ioxadm poweron PCIBOX 12B4 FAN 0O 6 3 6 3 1 Mounting a Link Card or PCle Card on the System This section describes the procedure for mounting a link card or a PCIe card of the PCI expansion unit on the system The card is mounted on the system using either of the following methods Using PCI hot plug PHP and using physical partition dynamic reconfiguration DR Note The link card is mounted in the PCI Exp
198. tems Service Manual March 2015 Table 5 4 end point devices is used Differences in the procedures according to whether the dynamic reconfiguration function for PCle Unnecessary Item Task Command Dynamic Dynamic reconfiguration reconfiguration is used 1 is not used 2 1 Enabling the hotplug service svcadm enable hotplug Perform Perform 3 2 Stopping the I O domain ldm stop domain Perform 3 Deleting the PCIe end point device from the ldm remove io Perform Perform I O domain 4 Setting the root domain in delayed ldm start reconf Perform reconfiguration mode 5 Assigning the PCIe end point device tothe Idm add io Perform Perform root domain 6 Restarting the root domain shutdown i6 g0 y Perform 7 Disabling the hotplug service svcadm disable hotplug Perform Perform 3 1 Dynamic reconfiguration for PCIe end point devices is supported by XCP 2230 or later and Oracle VM Server for SPARC 3 1 1 1 or later 2 If dynamic reconfiguration for PCIe end point devices is not used stop the I O domain delete the physical I O device and then restore the physical I O device to the root domain 3 If dynamic reconfiguration for PCIe end point devices is not used the hotplug service operates whether it is enabled or disabled Note Note the following when deleting a physical I O device PCIe end point device of the boot disk If the boot disk is in a redundant configuration stop the I O domain for Oracle VM Server for SPA
199. ter 16 Fan backplane 7 Enabled Chapter 17 1 The maintenance types of the PCIe card vary depending on the status of the PCIe card requiring maintenance For details see Table 3 6 2 Maintenance work is performed using PCI hot plug PHP Combined with dynamic SR IOV or the dynamic reconfiguration function for PCIe end point devices it also enables active maintenance of a PCIe card assigned as the SR IOV virtual function or PCIe end point to an I O domain 3 It is necessary to use the PCI Hot Plug PHP to release the link card mounted on a destination server from the system If the setting for the direct I O function is enabled the link card cannot use PHP So release the link card from the system using physical partition dynamic reconfiguration DR 4 This is supported only for a redundant configuration Chapter3 Types of Maintenance 29 3 2 Types of Maintenance for the Link Card This section describes the types of maintenance for the link card The link card is mounted in the PCI Express PCIe slot of the SPARC M10 1 or SPARC M10 4 M10 4S Table 3 4 lists the types of maintenance for the link card The link card maintenance type refers to the status of the server side The terms are defined as follows Definition according to the system operation status The types of maintenance depend on the system operation status during maintenance The supported maintenance is divided into three types active maintenance inactive mainte
200. that for removing a PCIe card mounted on the SPARC M10 1 or SPARC M10 4 M10 4S For details on the SPARC M10 1 see 8 3 Removing a PCI Express Card in the Fujitsu M10 1 SPARC M10 1 Service Manual For details on the SPARC M10 4 M10 4S see 8 4 Removing a PCI Express Card in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Note If the setting of the direct I O function is enabled the link card cannot be released by using PHP Using physical partition dynamic reconfiguration DR release the chassis requiring maintenance from the system Note To add or delete an I O device in the control or root domain enter delayed reconfiguration mode add or delete the I O device and then restart the domain However a combination of XCP 2230 or later and Oracle VM Server for SPARC 3 1 1 1 or later supports the dynamic reconfiguration function for PCIe end point devices Therefore you do not need to place the domain in delayed reconfiguration mode or restart the domain Before adding or deleting the I O device enable the hotplug service Releasing the link card or PCle card using PCI hot plug PHP This section describes the procedure for releasing the link card or PCIe card from the system using PCI hot plug PHP 56 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 1 Stopping the use of the I O device If there is any logical domain that is physically or virtually using the PCle card
201. that support the dynamic reassignment function for the PCIe end point device PCIe card in the same manual Chapter 4 Preparation and Precautions for Maintenance 45 Precautions for link card replacement The link card replacement procedure is the same as the replacement procedure for PCIe cards on the server side See the service manual for each server Before replacing the link card check the firmware version The firmware of the PCI expansion unit has been installed on the link card and the I O board When replacing the link card however the firmware version cannot be automatically matched to the one before the replacement For this reason you must use a confirmed firmware version to match it to the firmware version before the replacement If you cannot recognize the PCI expansion unit from the XSCF check the firmware version of a part that has not been replaced after the replacement to match it to the firmware version before the replacement When you replace the I O board and the link card at the same time check the firmware version before the replacement If you replace the I O board and the link card at the same time without checking the firmware version you may not be able to match the version If multiple PCI expansion units are used and the firmware versions match within the combinations there will be no problem with the operation even if firmware versions differ among PCI expansion units Note the following when mou
202. the link board 1 Remove the two link cables and one management cable connected to the link board requiring maintenance 182 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 9 3 2 Note Record the positions of the cables before removing them to ensure that they are reinstalled correctly Note Remove the link cable electrical or link cable optical by holding the tab A in the figure of the cable and pulling it straight toward the cable itself At this time do not hold the cable part when pulling out the crossbar cable Pulling the cable part without the connector lock completely released may cause damage Figure 9 2 Removing the link cable A 2 Remove the PCle card cassette For details see 8 3 2 Removing a PCI Express card cassette Removing the link board 1 While holding down the cover stopper lift the cover away Chapter9 Maintaining the Link Board 183 Figure 9 3 Cover stop 2 Remove the screw securing the PCle card cassette and then remove the bracket of the link board 184 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 9 4 Removing the bracket 3 Remove the link board from the PCle card cassette Note Do not use excessive force to remove the link board from the PCle card cassette The use of excessive force could damage the link board Note Place the removed link board on the ESD mat to ground any static electricit
203. the logical domain 9 Confirming that there is no problem with the replacement FRU 10 Turning off the CHECK LED locator of the PCI expansion unit requiring maintenance 11 Closing the rack door End 104 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 7 3 Item Task 1 Opening the rack door 2 Locating the PCI expansion unit requiring oN OD OF 10 11 maintenance by causing the CHECK LED locator of the unit to blink Checking the operating condition of the system and the I O device usage status Releasing the PCIe card from the system using PCI hot plug PHP Lowering the cable support Replacing a faulty PCle card Fixing the cable support Mounting the replacement PCIe card and restoring the logical domain Confirming that there is no problem with the replacement FRU Turning off the CHECK LED locator of the PCI expansion unit requiring maintenance Closing the rack door Hot replacement work procedure for a PCI Express card Reference 5 2 Checking the PCI Expansion Unit Requiring Maintenance and the Destination Physical Partition 5 4 1 Checking the operating condition of the physical partition or logical domain 5 4 2 Checking the assignment status of I O devices 5 5 1 Releasing the link card or PCIe card using PCI hot plug PHP 5 7 1 Lowering the cable support Chapter 8 Maintaining the PCI Express Cards 6 1 3 Fixing the cable suppo
204. the operation panel of the destination server to Service mode 3 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 4 Checking the operating condition of the system and the I O device usage status 5 Stopping the system 6 Enabling access to the FRU requiring maintenance 7 Installing the link card 8 Restoring the System 9 Performing a diagnosis test i 10 Confirming that there is no problem with the installed link card 11 Checking the firmware version of the PCI expansion unit and updating the firmware as required 12 Returning the mode switch on the operation panel to Locked mode 13 Starting the system 14 Closing the rack door i End 166 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Table 7 26 Item Task 1 Opening the rack door 2 Switching the mode switch on the 10 11 12 13 operation panel of the destination server to Service mode Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the operating condition of the system and the I O device usage status Stopping the system Enabling access to the FRU requiring maintenance Installing the link card 1 Restoring the System Performing a diagnosis test Confirming that there is no problem with the installed link card Checking the firmware version of the
205. the suppliers to Oracle and or its affiliates and Fujitsu Limited including software and font technology Per the terms of the GPL or LGPL a copy of the source code governed by the GPL or LGPL as applicable is available upon request by the End User Please contact Oracle and or its affiliates or Fujitsu Limited This distribution may include materials developed by third parties Parts of the product may be derived from Berkeley BSD systems licensed from the University of California UNIX is a registered trademark of The Open Group Oracle and Java are registered trademarks of Oracle and or its affiliates Fujitsu and the Fujitsu logo are registered trademarks of Fujitsu Limited SPARC Enterprise SPARC64 SPARC64 logo and all SPARC trademarks are trademarks or registered trademarks of SPARC International Inc in the United States and other countries and used under license Other names may be trademarks of their respective owners If this is software or related documentation that is delivered to the U S Government or anyone licensing it on behalf of the U S Government the following notice is applicable U S GOVERNMENT END USERS Oracle programs including any operating system integrated software any programs installed on the hardware and or documentation delivered to U S Government end users are commercial computer software pursuant to the applicable Federal Acquisition Regulation and agency specific supplemental regulations As such use
206. then remove the front cover Figure 5 7 Removing the front cover Chapter5 Preparations Required for Maintenance 73 74 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Chapter 6 Understanding the Preparations for Restoring the System This chapter describes the procedures for restoring the system after component maintenance See this chapter as required when performing the maintenance work on each component described in Chapter 7 onwards Restoring the Chassis of the PCI Expansion Unit Supplying Power to the FRU Requiring Maintenance Mounting a Link Card or PCIe Card on the System Restoring the Logical Domain to the Status It Had before Maintenance Turning off the CHECK LED Locator of the PCI Expansion Unit Updating the Firmware of the PCI Expansion Unit 6 1 6 1 1 Restoring the Chassis of the PCI Expansion Unit This section describes how to restore the chassis of the PCI expansion unit The procedure for restoring a chassis depends on the components that have been maintained Installing the front cover If the front cover has been removed for maintenance mount the components on the front panel or inside the chassis and then reinstall the front cover 1 Insert the tabs of the front cover A in the figure into the notches located on the lower side of the front panel of the chassis B in the figure to install the front cover 75 Figure 6 1 In
207. ting for the direct I O function is enabled then maintenance with the system stopped must be performed because the link card cannot use PHP 2 It is necessary to use the PCI Hot Plug PHP to release the link card from the system If the setting for the direct I O function is enabled the link card cannot use PHP Using physical partition dynamic reconfiguration DR release the link card from the system 3 Using physical partition dynamic reconfiguration DR it is necessary to release the server containing the link card from the physical partition 3 3 Sad Table 3 5 Maintenance Types of the PCle Card This section describes the maintenance types of the PCIe card mounted in the PCI expansion unit Understanding the maintenance types of the PCle card The maintenance types of the PCIe card vary depending on the status of the PCIe card requiring maintenance For details see the following table single unit configuration Domain to which the PCI expansion unit belongs Control domain PCle card requiring maintenance Direct I O 1 Unsupported Unsupported Supported PHP support Supported Unsupported Unsupported SR IOV 2 Unsupported supported Unsupported supported Unsupported Maintenance types of the PCIe card of the PCI expansion unit when the destination server has a Not maintainable PCI card maintenance type Hot use of PHP Enabled 3 5 Cold Enabled
208. topping the system 7 Enabling access to the FRU requiring maintenance 8 Replacing a faulty FRU 9 Restoring the PCI expansion unit 10 Performing a diagnosis test 11 Confirming that there is no problem with the replacement FRU 12 Checking the firmware version and matching it with the pre replacement one if you have replaced an I O board 13 Returning the mode switch on the operation panel to Locked mode 14 Starting the system 15 Closing the rack door End Chapter 7 Maintenance Flow 109 Table 7 5 Item Task 1 Opening the rack door 2 Switching the mode switch on the 10 11 12 13 110 operation panel of the destination server to Service mode Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink Checking the firmware version of the PCI expansion unit if you are replacing an I O board Checking the operating condition of the system and the I O device usage status Stopping the system Enabling access to the FRU requiring maintenance Replacing a faulty FRU Restoring the PCI expansion unit Performing a diagnosis test Confirming that there is no problem with the replacement FRU Checking the firmware version and matching it with the pre replacement one if you have replaced an I O board Returning the mode switch on the operation panel to Locked mode Cold replacement work procedure Releasing the link card
209. ts 2 Redundancy 1 1 redundant configuration Location Rear of chassis Active hot maintenance Supported Active cold maintenance Supported with SPARC M10 4 M10 4S connected Inactive hot maintenance Supported Inactive cold maintenance Supported System stopped maintenance Supported 240 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 For the maintenance procedure see Chapter 13 Maintaining the Power Supply Units B 4 Fan Unit Three fan units are mounted in the chassis to create flow that forces air out of the chassis The redundant configuration of the fan units allows the system to continue running even if a fan unit fails Table B 4 lists the specifications of the fan unit Table B 4 Item Number of fan units Redundancy Location Active hot maintenance Active cold maintenance Inactive hot maintenance Inactive cold maintenance System stopped maintenance Specifications of fan units Description 3 2 1 redundant configuration Front of chassis Supported Supported with SPARC M10 4 M10 45S connected Supported Supported Supported For the maintenance procedure see Chapter 14 Maintaining the Fan Units B 5 PCI Tray The PSU backplane I O board and fan backplane are mounted on the PCI tray When the PCI tray is to be replaced replace the PCI tray and PSU backplane at the same time Remount the I O board and fan backplane on the replaced PCI tray Table B
210. uration continued Not maintainable FRU Hot maintenance Cold maintenance Reference PCI tray Enabled Chapter 15 1 O board 7 Enabled Chapter 16 Fan backplane d Enabled Chapter 17 1 The maintenance types of the PCIe card vary depending on the status of the PCIe card requiring maintenance For details see Table 3 5 2 Maintenance work is performed using PCI hot plug PHP Combined with dynamic SR IOV or the dynamic reconfiguration function for PCIe end point devices it also enables active maintenance of a PCIe card assigned as the SR IOV virtual function or PCIe end point to an I O domain 3 It is necessary to use the PCI Hot Plug PHP to release the link card mounted on a destination server from the system If the setting for the direct I O function is enabled then maintenance must be performed with the system powered off because the link card cannot use PHP 4 This is supported only for a redundant configuration Table 3 3 When the connection destination of the PCI expansion unit is a SPARC M10 45 in a building block configuration Not maintainable FRU Hot maintenance Cold maintenance Reference PCI Express card 1 Enabled 2 Enabled Chapter 8 Link board Enabled 3 Enabled Chapter 9 Link cable Enabled 3 Enabled Chapter 11 Management cable Enabled 3 Enabled Chapter 12 Power supply unit Enabled 4 Enabled Chapter 13 Fan unit Enabled Enabled Chapter 14 PCI tray Enabled Chapter 15 T O board Enabled Chap
211. w 107 Cold replacement This section describes the cold replacement workflow References to detailed descriptions are written in the work procedure tables See any of them as required Cold replacement work varies depending on the method of releasing the link card mounted on the destination server The following are the work patterns Cold replacement releasing the link card after stopping the system Cold replacement releasing the link card using PHP Cold replacement releasing the link card using DR Cold replacement releasing the link card after stopping the system Cold replacement can be performed for all the FRUs mounted on the PCI expansion unit after stopping the system to which the PCI expansion unit is connected This can be done when the connection destination of the PCI expansion unit is a SPARC M10 1 M10 4 M10 4S 108 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 7 3 Cold replacement flow releasing the link card after stopping the system 1 Opening the rack door 2 Switching the mode switch on the operation panel of the destination server to Service mode 3 Locating the PCI expansion unit requiring maintenance by causing the CHECK LED locator of the unit to blink 4 Checking the firmware version of the PCI expansion unit if you are replacing an I O board 5 Checking the operating condition of the system and the I O device usage status 6 S
212. ware one unit at at time The firmware can be updated if any of the following conditions are met when the system is operating and the PCI expansion unit can be recognized from the XSCF The PCI expansion unit is displayed when executing the ioxadm command The control domain to which the PCI expansion unit is connected is started and runs until the OpenBoot PROM state is set Do not operate the system power supply during firmware update To reflect the applied firmware restart the physical partition or the I O domain to which the PCI expansion unit belongs or turn on the power again Ask the system administrator to do this If multiple PCI expansion units are being used and are combined in such a way that their firmware versions match there will be no problem with their operation even if some of the PCI expansion units use different firmware versions Precautions for installation The following describes precautions for installation For PCIe card installation by PHP a multipath setting is necessary depending on the use of the PCIe cards For active addition by PHP confirm that the card is PHP supported To do so check Appendix A Cards that Support PCI Hot Plug and Dynamic Reconfiguration 48 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 in the Fujitsu M10 SPARC M10 Systems PCI Card Installation Guide If you perform active addition by combining PHP with dynamic SR IOV or the dynamic
213. y Chapter9 Maintaining the Link Board 185 Figure 9 5 Link board removal 9 4 Installing the Link Board This section describes the procedure for installing the link board 9 4 1 Installing the link board 1 Install the link board by inserting it into the connector of the PCle card cassette Note Confirm that the tab of the link board A in the figure is inserted into the notch B in the figure of the PCle card cassette 186 PCI Expansion Unit for Fujitsu M10 SPARC M10 Systems Service Manual March 2015 Figure 9 6 Link board installation 2 Secure the bracket of the link board with the screw Chapter9 Maintaining the Link Board 187 9 4 2 Figure 9 7 Installing the bracket 3 Mount the cover on the PCle card cassette Restoring the chassis 1 Install the PCle card cassette For details see 8 4 2 Installing a PCI Express card cassette 2 Connect the two link cables and one management cable Note Connect the cables in their original positions by referring to the notes that you made before the start of maintenance Note Insert the link cable electrical or link cable optical by holding the connector part of the cable and inserting it straight into the opening Do not hold the cable part when inserting the cable Now the work for mounting the FRU is complete While referencing Chapter 7 Maintenance Flow continue the maintenance work 188 PCI Expansion Unit for Fujitsu M10 SPARC M10
214. ysical partition dynamic reconfiguration DR 6 3 3 Checking the FRU status after maintenance in the Fujitsu M10 4 Fujitsu M10 4S SPARC M10 4 SPARC M10 4S Service Manual Link Card Maintenance Workflow This section describes the link card maintenance workflow Note When performing work for any replacement addition or removal of the link card check 1 7 Important Information about the XCP Firmware in advance Active maintenance This section describes the flows of the active replacement active addition and active removal for the link card References to detailed descriptions are written in the work procedure tables See any Chapter 7 Maintenance Flow 135 136 of them as required Precautions for active maintenance of the link card If the link card is mounted on the SPARC M10 1 active maintenance cannot be performed Perform maintenance with the system stopped If the SPARC M10 4 or SPARC M10 4S is in a single unit configuration and the setting for the direct I O function is enabled active maintenance of the link card cannot be performed Perform maintenance with the system stopped If the setting for the direct I O function is enabled in a building block configuration active maintenance using PHP cannot be performed Perform active maintenance using physical partition dynamic reconfiguration DR Active addition of the link card using PHP is supported by Oracle Solaris 11 2 SRU11 2 2 8 0 or later The fol

PCI Expansion Unit for Fujitsu M10/SPARC M10 Systems Service

Contents

Download Pdf Manuals

Related Search

Related Contents